Dynamically Scaled Activation Steering

Researchers Alex Ferrando de las Morenas, Xavier Suau Cuadros, Jordi Gonzàlez Sabaté and Pau Rodríguez Lopez introduced Dynamically Scaled Activation Steering (DSAS), described as a method-agnostic steering framework for guiding the behavior of generative models toward desired outcomes such as toxicity mitigation.

The reported problem is that most existing activation steering methods apply interventions uniformly across all inputs, which degrades model performance when steering is unnecessary. DSAS decouples when to steer from how to steer, adaptively modulating the strength of existing steering transformations across layers and inputs, and intervening strongly only when undesired behavior is detected.

At generation time, DSAS computes context-dependent scaling factors that selectively adjust the strength of any steering method, according to the work. The authors also state that DSAS can be jointly optimized end-to-end together with the steering function.

When combined with existing steering methods, DSAS consistently improves the Pareto front with respect to steering alone, achieving a better trade-off between toxicity mitigation and utility preservation, the authors report. They further demonstrate its generality by applying it to a text-to-image diffusion model, showing how adaptive steering allows the modulation of specific concepts.

The work states that DSAS introduces minimal computational overhead while improving interpretability, pinpointing which tokens require steering and by how much. The code will be available on GitHub. Two of the authors are affiliated with the Centre de Visió per Computador.