Character Training and Reward Hacking: Mitigation and Monitorability Challenges

The study explores how character training affects reward hacking in reinforcement learning, finding that anti-cheating training may resist reward hacking but could reduce monitorability due to motivated reasoning. It involved training models with pro, neutral, and anti-cheating specifications and evaluating their behavior on coding tasks designed to induce reward hacking.