d3rlpy.models.q_functions.FQFQFunctionFactory

class d3rlpy.models.q_functions.FQFQFunctionFactory(n_quantiles=32, embed_size=64, entropy_coeff=0.0)

Fully parameterized Quantile Function Q function factory.

References

• Yang et al., Fully parameterized quantile function for distributional reinforcement learning.

Parameters

• n_quantiles – the number of quantiles.

• embed_size – the embedding size.

• entropy_coeff – the coefficiency of entropy penalty term.

Methods

create_continuous(encoder)

Returns PyTorch’s Q function module.

Parameters

encoder (d3rlpy.models.torch.encoders.EncoderWithAction) – an encoder module that processes the observation and action to obtain feature representations.

Returns

continuous Q function object.

Return type

d3rlpy.models.torch.q_functions.ContinuousFQFQFunction

create_discrete(encoder, action_size)

Returns PyTorch’s Q function module.

Parameters

• encoder (d3rlpy.models.torch.encoders.Encoder) – an encoder module that processes the observation to obtain feature representations.

• action_size (int) – dimension of discrete action-space.

Returns

discrete Q function object.

Return type

d3rlpy.models.torch.q_functions.DiscreteFQFQFunction

get_params(deep=False)

Returns Q function parameters.

Returns

Q function parameters.

Parameters

deep (bool) –

Return type

Dict[str, Any]

get_type()

Returns Q function type.

Returns

Q function type.

Return type

str

Attributes

TYPE: ClassVar[str] = 'fqf'

embed_size

entropy_coeff

n_quantiles