d3rlpy.dataset.ReplayBufferBase¶

class d3rlpy.dataset.ReplayBufferBase[source]¶

An interface of ReplayBuffer.

Methods

abstract append(observation, action, reward)[source]¶

Appends observation, action and reward to buffer.

Parameters

observation (Union[numpy.ndarray[Any, numpy.dtype[Any]], Sequence[numpy.ndarray[Any, numpy.dtype[Any]]]]) – Observation.
action (Union[int, numpy.ndarray[Any, numpy.dtype[Any]]]) – Action.
reward (Union[float, numpy.ndarray[Any, numpy.dtype[Any]]]) – Reward.

Return type

None

abstract append_episode(episode)[source]¶

Appends episode to buffer.

Parameters: episode (d3rlpy.dataset.components.EpisodeBase) – Episode.
Return type: None

abstract clip_episode(terminated)[source]¶

Clips current episode.

Parameters: terminated (bool) – Flag to represent environmental termination. This flag should be False if the episode is terminated by timeout.
Return type: None

abstract dump(f)[source]¶

Dumps buffer data.

with open('dataset.h5', 'w+b') as f:
    replay_buffer.dump(f)

Parameters: f (BinaryIO) – IO object to write to.
Return type: None

abstract classmethod from_episode_generator(episode_generator, buffer, transition_picker=None, trajectory_slicer=None, writer_preprocessor=None)[source]¶

Builds ReplayBuffer from episode generator.

Parameters

episode_generator (d3rlpy.dataset.episode_generator.EpisodeGeneratorProtocol) – Episode generator implementation.
buffer (d3rlpy.dataset.buffers.BufferProtocol) – Buffer implementation.
transition_picker (Optional[d3rlpy.dataset.transition_pickers.TransitionPickerProtocol]) – Transition picker implementation for Q-learning-based algorithms.
trajectory_slicer (Optional[d3rlpy.dataset.trajectory_slicers.TrajectorySlicerProtocol]) – Trajectory slicer implementation for Transformer-based algorithms.
writer_preprocessor (Optional[d3rlpy.dataset.writers.WriterPreprocessProtocol]) – Writer preprocessor implementation.

Returns

Replay buffer.

Return type

d3rlpy.dataset.replay_buffer.ReplayBuffer

abstract classmethod load(f, buffer, episode_cls=<class 'd3rlpy.dataset.components.Episode'>, transition_picker=None, trajectory_slicer=None, writer_preprocessor=None)[source]¶

Builds ReplayBuffer from dumped data.

This method reconstructs replay buffer dumped by dump method.

with open('dataset.h5', 'rb') as f:
    replay_buffer = ReplayBuffer.load(f, buffer)

Parameters

f (BinaryIO) – IO object to read from.
buffer (d3rlpy.dataset.buffers.BufferProtocol) – Buffer implementation.
episode_cls (Type[d3rlpy.dataset.components.EpisodeBase]) – Eisode class used to reconstruct data.
transition_picker (Optional[d3rlpy.dataset.transition_pickers.TransitionPickerProtocol]) – Transition picker implementation for Q-learning-based algorithms.
trajectory_slicer (Optional[d3rlpy.dataset.trajectory_slicers.TrajectorySlicerProtocol]) – Trajectory slicer implementation for Transformer-based algorithms.
writer_preprocessor (Optional[d3rlpy.dataset.writers.WriterPreprocessProtocol]) – Writer preprocessor implementation.

Returns

Replay buffer.

Return type

d3rlpy.dataset.replay_buffer.ReplayBuffer

abstract sample_trajectory(length)[source]¶

Samples a partial trajectory.

Parameters: length (int) – Length of partial trajectory.
Returns: Partial trajectory.
Return type: d3rlpy.dataset.components.PartialTrajectory

abstract sample_trajectory_batch(batch_size, length)[source]¶

Samples a mini-batch of partial trajectories.

Parameters

batch_size (int) – Mini-batch size.
length (int) – Length of partial trajectories.

Returns

Mini-batch.

Return type

d3rlpy.dataset.mini_batch.TrajectoryMiniBatch

abstract sample_transition()[source]¶

Samples a transition.

Returns: Transition.
Return type: d3rlpy.dataset.components.Transition

abstract sample_transition_batch(batch_size)[source]¶

Samples a mini-batch of transitions.

Parameters: batch_size (int) – Mini-batch size.
Returns: Mini-batch.
Return type: d3rlpy.dataset.mini_batch.TransitionMiniBatch

abstract size()[source]¶

Returns number of episodes.

Returns: Number of episodes.
Return type: int

Attributes

buffer¶

Returns buffer.

Returns: Buffer.

dataset_info¶

Returns dataset information.

Returns: Dataset information.

episodes¶

Returns sequence of episodes.

Returns: Sequence of episodes.

trajectory_slicer¶

Returns trajectory slicer.

Returns: Trajectory slicer.

transition_count¶

Returns number of transitions.

Returns: Number of transitions.

transition_picker¶

Returns transition picker.

Returns: Transition picker.