d3rlpy.dataset.ReplayBufferBase¶

class d3rlpy.dataset.ReplayBufferBase[source]¶

An interface of ReplayBuffer.

Methods

abstract append(observation, action, reward)[source]¶

Appends observation, action and reward to buffer.

Parameters:

observation (Union[ndarray[Any, dtype[Any]], Sequence[ndarray[Any, dtype[Any]]]]) – Observation.
action (Union[int, ndarray[Any, dtype[Any]]]) – Action.
reward (Union[float, ndarray[Any, dtype[Any]]]) – Reward.

Return type:

None

abstract append_episode(episode)[source]¶

Appends episode to buffer.

Parameters:: episode (EpisodeBase) – Episode.
Return type:: None

abstract clip_episode(terminated)[source]¶

Clips current episode.

Parameters:: terminated (bool) – Flag to represent environmental termination. This flag should be False if the episode is terminated by timeout.
Return type:: None

abstract dump(f)[source]¶

Dumps buffer data.

with open('dataset.h5', 'w+b') as f:
    replay_buffer.dump(f)

Parameters:: f (BinaryIO) – IO object to write to.
Return type:: None

abstract classmethod from_episode_generator(episode_generator, buffer, transition_picker=None, trajectory_slicer=None, writer_preprocessor=None)[source]¶

Builds ReplayBuffer from episode generator.

Parameters:

episode_generator (EpisodeGeneratorProtocol) – Episode generator implementation.
buffer (BufferProtocol) – Buffer implementation.
transition_picker (Optional[TransitionPickerProtocol]) – Transition picker implementation for Q-learning-based algorithms.
trajectory_slicer (Optional[TrajectorySlicerProtocol]) – Trajectory slicer implementation for Transformer-based algorithms.
writer_preprocessor (Optional[WriterPreprocessProtocol]) – Writer preprocessor implementation.

Returns:

Replay buffer.

Return type:

ReplayBuffer

abstract classmethod load(f, buffer, episode_cls=<class 'd3rlpy.dataset.components.Episode'>, transition_picker=None, trajectory_slicer=None, writer_preprocessor=None)[source]¶

Builds ReplayBuffer from dumped data.

This method reconstructs replay buffer dumped by dump method.

with open('dataset.h5', 'rb') as f:
    replay_buffer = ReplayBuffer.load(f, buffer)

Parameters:

f (BinaryIO) – IO object to read from.
buffer (BufferProtocol) – Buffer implementation.
episode_cls (Type[EpisodeBase]) – Eisode class used to reconstruct data.
transition_picker (Optional[TransitionPickerProtocol]) – Transition picker implementation for Q-learning-based algorithms.
trajectory_slicer (Optional[TrajectorySlicerProtocol]) – Trajectory slicer implementation for Transformer-based algorithms.
writer_preprocessor (Optional[WriterPreprocessProtocol]) – Writer preprocessor implementation.

Returns:

Replay buffer.

Return type:

ReplayBuffer

abstract sample_trajectory(length)[source]¶

Samples a partial trajectory.

Parameters:: length (int) – Length of partial trajectory.
Returns:: Partial trajectory.
Return type:: PartialTrajectory

abstract sample_trajectory_batch(batch_size, length)[source]¶

Samples a mini-batch of partial trajectories.

Parameters:

batch_size (int) – Mini-batch size.
length (int) – Length of partial trajectories.

Returns:

Mini-batch.

Return type:

TrajectoryMiniBatch

abstract sample_transition()[source]¶

Samples a transition.

Returns:: Transition.
Return type:: Transition

abstract sample_transition_batch(batch_size)[source]¶

Samples a mini-batch of transitions.

Parameters:: batch_size (int) – Mini-batch size.
Returns:: Mini-batch.
Return type:: TransitionMiniBatch

abstract size()[source]¶

Returns number of episodes.

Returns:: Number of episodes.
Return type:: int

Attributes

buffer¶

Returns buffer.

Returns:: Buffer.

dataset_info¶

Returns dataset information.

Returns:: Dataset information.

episodes¶

Returns sequence of episodes.

Returns:: Sequence of episodes.

trajectory_slicer¶

Returns trajectory slicer.

Returns:: Trajectory slicer.

transition_count¶

Returns number of transitions.

Returns:: Number of transitions.

transition_picker¶

Returns transition picker.

Returns:: Transition picker.