Logging

d3rlpy provides a customizable interface for logging metrics, LoggerAdapter and LoggerAdapterFactory.

import d3rlpy

dataset, env = d3rlpy.datasets.get_cartpole()

dqn = d3rlpy.algos.DQNConfig().create()

dqn.fit(
   dataset=dataset,
   n_steps=100000,
   # set FileAdapterFactory to save metrics as CSV files
   logger_adapter=d3rlpy.logging.FileAdapterFactory(root_dir="d3rlpy_logs"),
)

LoggerAdapterFactory is a parent interface that instantiates LoggerAdapter at the beginning of training. You can also use CombineAdapter to combine multiple LoggerAdapter in the same training.

# combine FileAdapterFactory and TensorboardAdapterFactory
logger_adapter = d3rlpy.logging.CombineAdapterFactory([
   d3rlpy.logging.FileAdapterFactory(root_dir="d3rlpy_logs"),
   d3rlpy.logging.TensorboardAdapterFactory(root_dir="tensorboard_logs"),
])

dqn.fit(dataset=dataset, n_steps=100000, logger_adapter=logger_adapter)

LoggerAdapter

LoggerAdapter is an inner interface of LoggerAdapterFactory. You can implement your own LoggerAdapter for 3rd-party visualizers.

import d3rlpy

class CustomAdapter(d3rlpy.logging.LoggerAdapter):
    def write_params(self, params: Dict[str, Any]) -> None:
        # save dictionary as json file
        with open("params.json", "w") as f:
            f.write(json.dumps(params, default=default_json_encoder, indent=2))

    def before_write_metric(self, epoch: int, step: int) -> None:
        pass

    def write_metric(
        self, epoch: int, step: int, name: str, value: float
    ) -> None:
        with open(f"{name}.csv", "a") as f:
            print(f"{epoch},{step},{value}", file=f)

    def after_write_metric(self, epoch: int, step: int) -> None:
        pass

    def save_model(self, epoch: int, algo: Any) -> None:
        algo.save(f"model_{epoch}.d3")

    def close(self) -> None:
        pass

d3rlpy.logging.LoggerAdapter	Interface of LoggerAdapter.
d3rlpy.logging.FileAdapter	FileAdapter class.
d3rlpy.logging.TensorboardAdapter	TensorboardAdapter class.
d3rlpy.logging.WanDBAdapter	WandB Logger Adapter class.
d3rlpy.logging.NoopAdapter	NoopAdapter class.
d3rlpy.logging.CombineAdapter	CombineAdapter class.

LoggerAdapterFactory

LoggerAdapterFactory is an interface that instantiates LoggerAdapter at the beginning of training. You can implement your own LoggerAdapterFactory for 3rd-party visualizers.

import d3rlpy

class CustomAdapterFactory(d3rlpy.logging.LoggerAdapterFactory):
    def create(self, experiment_name: str) -> d3rlpy.logging.FileAdapter:
        return CustomAdapter()

d3rlpy.logging.LoggerAdapterFactory	Interface of LoggerAdapterFactory.
d3rlpy.logging.FileAdapterFactory	FileAdapterFactory class.
d3rlpy.logging.TensorboardAdapterFactory	TensorboardAdapterFactory class.
d3rlpy.logging.WanDBAdapterFactory	WandB Logger Adapter Factory class.
d3rlpy.logging.NoopAdapterFactory	NoopAdapterFactory class.
d3rlpy.logging.CombineAdapterFactory	CombineAdapterFactory class.