d3rlpy.models.q_functions.FQFQFunctionFactory¶

class d3rlpy.models.q_functions.FQFQFunctionFactory(bootstrap=False, share_encoder=False, n_quantiles=32, embed_size=64, entropy_coeff=0.0)[source]¶

Fully parameterized Quantile Function Q function factory.

References

Yang et al., Fully parameterized quantile function for distributional reinforcement learning.

Parameters

bootstrap (bool) – flag to bootstrap Q functions.
share_encoder (bool) – flag to share encoder over multiple Q functions.
n_quantiles (int) – the number of quantiles.
embed_size (int) – the embedding size.
entropy_coeff (float) – the coefficiency of entropy penalty term.

Methods

create_continuous(encoder)[source]¶

Returns PyTorch’s Q function module.

Parameters: encoder (d3rlpy.models.torch.encoders.EncoderWithAction) – an encoder module that processes the observation and action to obtain feature representations.
Returns: continuous Q function object.
Return type: d3rlpy.models.torch.q_functions.fqf_q_function.ContinuousFQFQFunction

create_discrete(encoder, action_size)[source]¶

Returns PyTorch’s Q function module.

Parameters

encoder (d3rlpy.models.torch.encoders.Encoder) – an encoder module that processes the observation to obtain feature representations.
action_size (int) – dimension of discrete action-space.

Returns

discrete Q function object.

Return type

d3rlpy.models.torch.q_functions.fqf_q_function.DiscreteFQFQFunction

get_params(deep=False)[source]¶

Returns Q function parameters.

Returns: Q function parameters.
Parameters: deep (bool) –
Return type: Dict[str, Any]

get_type()¶

Returns Q function type.

Returns: Q function type.
Return type: str

Attributes

TYPE: ClassVar[str] = 'fqf'¶

bootstrap¶

embed_size¶

entropy_coeff¶

n_quantiles¶

share_encoder¶