d3rlpy.preprocessing.MinMaxActionScaler¶

class d3rlpy.preprocessing.MinMaxActionScaler(dataset=None, maximum=None, minimum=None)[source]¶

Min-Max normalization action preprocessing.

Actions will be normalized in range [-1.0, 1.0].

\[a' = (a - \min{a}) / (\max{a} - \min{a}) * 2 - 1\]

from d3rlpy.dataset import MDPDataset
from d3rlpy.algos import CQL

dataset = MDPDataset(observations, actions, rewards, terminals)

# initialize algorithm with MinMaxActionScaler
cql = CQL(action_scaler='min_max')

# scaler is initialized from the given episodes
cql.fit(dataset.episodes)

You can also initialize with d3rlpy.dataset.MDPDataset object or manually.

from d3rlpy.preprocessing import MinMaxActionScaler

# initialize with dataset
scaler = MinMaxActionScaler(dataset)

# initialize manually
minimum = actions.min(axis=0)
maximum = actions.max(axis=0)
action_scaler = MinMaxActionScaler(minimum=minimum, maximum=maximum)

cql = CQL(action_scaler=action_scaler)

Parameters

dataset (d3rlpy.dataset.MDPDataset) – dataset object.
min (numpy.ndarray) – minimum values at each entry.
max (numpy.ndarray) – maximum values at each entry.
maximum (Optional[numpy.ndarray]) –
minimum (Optional[numpy.ndarray]) –

Methods

fit(episodes)[source]¶

Estimates scaling parameters from dataset.

Parameters: episodes (List[d3rlpy.dataset.Episode]) – a list of episode objects.
Return type: None

fit_with_env(env)[source]¶

Gets scaling parameters from environment.

Parameters: env (gym.core.Env) – gym environment.
Return type: None

get_params(deep=False)[source]¶

Returns action scaler params.

Parameters: deep (bool) – flag to deepcopy parameters.
Returns: action scaler parameters.
Return type: Dict[str, Any]

get_type()¶

Returns action scaler type.

Returns: action scaler type.
Return type: str

reverse_transform(action)[source]¶

Returns reversely transformed action.

Parameters: action (torch.Tensor) – action vector.
Returns: reversely transformed action.
Return type: torch.Tensor

reverse_transform_numpy(action)[source]¶

Returns reversely transformed action in numpy array.

Parameters: action (numpy.ndarray) – action vector.
Returns: reversely transformed action.
Return type: numpy.ndarray

transform(action)[source]¶

Returns processed action.

Parameters: action (torch.Tensor) – action vector.
Returns: processed action.
Return type: torch.Tensor

Attributes

TYPE: ClassVar[str] = 'min_max'¶