The Unjournal commissions public expert evaluations and ratings of research (and “pivotal questions”) with potential for global impact. We’re currently making a push to consider AI economics and governance research.[1] We’re seeking input from people who use this: which themes, which papers/projects, which claims, who should evaluate them, what should evaluators focus on, and why? (If you only have a moment, skip to “What would help most”.) Why do this? (Why us, and why now?) The usual story, which you’re ~probably on board with: AI is advancing rapidly, and becoming more clearly and deeply important to most major global priorities. It’s having large actual and predicted impact on the economy and the concentration of power. This is itself entangled with AI safety and AI global catastrophic risks. “How fast will capabilities grow and replace human work?” “How will gains flow to labor versus capital?” “How will revenue and compute accumulate in a few firms and countries?” “Will regulation change which models get released where?” Questions like these impact the nature of catastrophic AI-risk (resources, race dynamics, human empowerment, state capacity, etc.), as well as the strategic approaches to handling it. There are also direct implications for global poverty, global health systems, and for non-AI GCR (e.g., bioengineered pandemics).[2] These questions (and ‘cruxes’) seem to directly impact a set of live decisions surrounding AI safety strategy, philanthropic funding, regulation and legal safeguards, public spending, international controls, advocacy, and more. Economists, social scientists, and policy analysts have been moving into this area. Some of the work crosses disciplinary lines. Economists are now modeling AI scaling and recursive self-improvement. Growth theorists are co-authoring scenario models with funding and data from frontier labs. Forecasting researchers are eliciting and synthesizing beliefs and predictions from economists, AI-company staff, an
Translation pending · showing the source descriptionThe chart will appear after repeat observations. The current metric comes from the source.
10 source pointsReal observations only. History before source connection is not reconstructed.
In a post by Nick Cammarata, it is discussed that neural networks may be understandable thanks to modern interpretability tools. The author claims that the majority of neurons in models with trillions of parameters are interpretable, and their circuits are beautiful. He also notes that neurons previously considered complex were misinterpreted.
5 days agoLessWrongThe post critiques the article on social reality in China, highlighting an unbalanced comparison and a lack of systematic analysis. The author emphasizes that Chinese intellectual communities have their own distinct characteristics, different from Western ones, and that values such as knowledge, truth, and technology ethics are also present in China.
4 days agoLessWrongThe author argues that owning large amounts of capital may lead to losing influence earlier than others, through mental breakdown. This is because people seek to manipulate the wealthy to gain resources, making it harder to maintain values and accurate beliefs.
yesterdayLessWrongThe authors propose GRAFT, a method that localizes and removes unintended behaviors in reinforcement learning by confining learning updates related to detected hacks to a specific set of parameters, which are removed after training.
yesterdayLessWrongThe authors found that the GLM-5.3 model sometimes unconsciously reasons about how its actions might affect future models. This is because the model considers that its outputs could be used for supervised fine-tuning or other forms of training, and that its actions could reinforce certain behavioral patterns.
5 days agoLessWrongThe article discusses the technical AI alignment problem and its core difficulties. The author draws on theoretical computer science, machine learning, analytic philosophy, and ideas from the Machine Intelligence Research Institute/LessWrong. It emphasizes the importance of defining AI's objectives for ensuring safety.
2 days ago