d3rlpy.metrics.comparer.compare_continuous_action_diff

d3rlpy.metrics.comparer.compare_continuous_action_diff(base_algo)

Returns scorer function of action difference between algorithms.

This metrics suggests how different the two algorithms are in continuous action-space. If the algorithm to compare with is near-optimal, the small action difference would be better.

\[\mathbb{E}_{s_t \sim D} [(\pi_{\phi_1}(s_t) - \pi_{\phi_2}(s_t))^2]\]

from d3rlpy.algos import CQL
from d3rlpy.metrics.comparer import compare_continuous_action_diff

cql1 = CQL()
cql2 = CQL()

scorer = compare_continuous_action_diff(cql1)

squared_action_diff = scorer(cql2, ...)

Parameters

base_algo (d3rlpy.metrics.scorer.AlgoProtocol) – algorithm to comapre with.

Returns

scorer function.

Return type

Callable[[d3rlpy.metrics.scorer.AlgoProtocol, List[d3rlpy.dataset.Episode]], float]