Optimizers¶

d3rlpy provides OptimizerFactory that gives you flexible control over optimizers. OptimizerFactory takes PyTorch’s optimizer class and its arguments to initialize, which you can check more here.

import d3rlpy
from torch.optim import Adam

# modify weight decay
optim_factory = d3rlpy.optimizers.OptimizerFactory(Adam, weight_decay=1e-4)

# set OptimizerFactory
dqn = d3rlpy.algos.DQNConfig(optim_factory=optim_factory).create()

There are also convenient alises.

# alias for Adam optimizer
optim_factory = d3rlpy.optimizers.AdamFactory(weight_decay=1e-4)

dqn = d3rlpy.algos.DQNConfig(optim_factory=optim_factory).create()

`d3rlpy.optimizers.OptimizerFactory`	A factory class that creates an optimizer object in a lazy way.
`d3rlpy.optimizers.SGDFactory`	An alias for SGD optimizer.
`d3rlpy.optimizers.AdamFactory`	An alias for Adam optimizer.
`d3rlpy.optimizers.RMSpropFactory`	An alias for RMSprop optimizer.
`d3rlpy.optimizers.GPTAdamWFactory`	AdamW optimizer for Decision Transformer architectures.

Learning rate scheduler¶

d3rlpy provides LRSchedulerFactory that gives you configure learning rate schedulers with OptimizerFactory.

import d3rlpy

# set lr_scheduler_factory
optim_factory = d3rlpy.optimizers.AdamFactory(
    lr_scheduler_factory=d3rlpy.optimizers.WarmupSchedulerFactory(
        warmup_steps=10000
    )
)

`d3rlpy.optimizers.LRSchedulerFactory`	A factory class that creates a learning rate scheduler a lazy way.
`d3rlpy.optimizers.WarmupSchedulerFactory`	A warmup learning rate scheduler.
`d3rlpy.optimizers.CosineAnnealingLRFactory`	A cosine annealing learning rate scheduler.