Categorical DQN (C51)

Categorical DQN (C51)

• https://arxiv.org/abs/1707.06887
• https://docs.cleanrl.dev/rl-algorithms/c51/

C51 introduces a distributional perspective for DQN: instead of learning a single value for an action, C51 learns to predict a distribution of values for the action. Empirically, C51 demonstrates impressive performance in ALE.