Simplex vs Timaeus: Round One

I'm often asked about the differences and similarities between Simplex' and Timaeus' research agendas. The question is natural enough. Both focus on a 'fundamental science' approach to AI alignment. Both organizations base their research agendas on sophisticated mathematical frameworks handed down from a bearded ur-figure (Sumio Watanabe, James Crutchfield). We may posit the following correspondence Dan Murfet + Jesse Hoogland : Developmental Interpretability : Singular Learning Theory : Sumio Watanabe Adam Shai + Paul Riechers : Belief-state Geometry: Computational Mechanics : James Crutchfield SLT vs CompMech Round One. Fight! Weights vs Activations DevInterp & SLT is about weight space. Belief-state geometry is more about studying activation space. Activation space is what is already being studied in MechInterp & most 'mainstream' approaches to interpretability. It is concrete and present to the senses. Weight space is much larger, more abstract, harder to measure and sample. Training vs Inference SLT is about training. CompMech is about inference. Both study Bayesian posteriors and updating. For SLT that is the Bayesian posterior on weight space - hence relevant for training. The Belief-State Geometry agenda studies the Mixed State Presentation from CompMech which describes an [idealized] version of in-context learning as the LLM doing Bayesian updating token-by-token as it reads the context. Caveat. It isn't clear that inference and learning are really fundamentally distinct. It's all just updating/conditioning in Bayesian statistics. Indeed, if one buys the Strong in-Context Learning story the difference between the training/learning of LLMs and inference may be somewhat illusory. IID vs non-IID Data Singular Learning Theory has historically been about IID data. CompMech about IID data is trivial. Caveat. Singular learning theory has been studied for non-IID data but it's fair to say its development is in its early stages. Parameterizatio