Our paper “Adaptive Rational Activations to Boost Deep Reinforcement Learning” was accepted at the International Conference on Learning Representations (ICLR) 2024. We are honored to have been selected for a spotlight presentation!
In the paper, we present how a plug&play alternative to current activation functions – the rational activation function – a parameter efficient way to boost any neural network’s plasticity. At the hand of reinforcement learning benchmarks, we demonstrate that this plasticity leads to the model reaching higher accuracy as a consequence of its increased adaptivity, while at the same time mitigating the overestimation problem observed in e.g. Deep (Double) Q-Learning.
We additionally confirm, on some popular image classification tasks, that we can reach higher, or similar accuracy at a much lower overall parameter count when using rational activations.
Read the abstract below and the full paper for more detailed information:
“Latest insights from biology show that intelligence not only emerges from the connections between neurons, but that individual neurons shoulder more computational responsibility than previously anticipated. Specifically, neural plasticity should be critical in the context of constantly changing reinforcement learning (RL) environments, yet current approaches still primarily employ static activation functions. In this work, we motivate the use of adaptable activation functions in RL and show that rational activation functions are particularly suitable for augmenting plasticity. Inspired by residual networks, we derive a condition under which rational units are closed under residual connections and formulate a naturally regularised version. The proposed joint-rational activation allows for desirable degrees of flexibility, yet regularises plasticity to an extent that avoids overfitting by leveraging a mutual set of activation function parameters across layers. We demonstrate that equipping popular algorithms with (joint) rational activations leads to consistent improvements on different games from the Atari Learning Environment benchmark, notably making DQN competitive to DDQN and Rainbow.”
Abstract of our paper “Adaptive Rational Activations to Boost Deep Reinforcement Learning“