One Coordinate Breaks Abliteration on Gemma-3

The study investigates an issue with abliteration, a method for identifying refusal directions in large language models, where the standard approach failed for Gemma-3-12b but worked for similar models. A fix involving Winsorization of coordinate activations was tested, and the problem was traced to a dominant coordinate in scale. The method was validated across multiple model sizes and showed generalizability. The reason for the development of this large activation coordinate in Gemma-3 models remains unclear.