norgitov/ trends
Технологии · люди · идеи
К обзору/LessWrong22 минуты назад

Autonomous researchers will strain peer review, funding, and oversight

Summary: Autonomous AI researchers could greatly accelerate AI safety research, but even if we solve sabotage, they will strain the research ecosystem. They will multiply the volume of research and make researchers homogeneous. If nothing changes, peer review will collapse, false positives will rise, funding will concentrate, and humans will lose track of the science. We should start valuing replication, preserving diversity, and anchoring research to human understanding now. I think this needs a dedicated metascience organization. ---------------------------------------- Frontier labs are aiming to build autonomous researchers, and they are making progress. OpenAI announced in September 2026 that they had built a "research intern", and their stated goal is a fully automated AI researcher by March 2028. Independent forecasters put full automation of AI R&D a few years later, around 2031 to 2032. Autonomous researchers will amplify many issues that already exist in science and introduce new issues. We already see AI having an impact on research and publishing, primarily automating many steps of the research process, increasing the volume of publications, and weakening the peer review process. We should think more deeply about what autonomous researchers will look like and what they mean for the ecosystem of AI safety research. While there is huge potential to accelerate progress, there is also the possibility that existing infrastructure collapses under this new weight. The failures we consider here are relevant even with controlled autonomous researchers, but I highly recommend looking at AI control, which addresses sabotage, a major roadblock to automated AI safety research. This is covered well by Carlsmith and Benton et al.. Everything below is a concern even if AI is not trying to sabotage. We should also keep in mind that autonomous researchers may cause the ecosystem to reorganize in ways we can't predict, for better or worse. I think it is important to k

Перевод готовится · пока описание источника
Открыть первоисточник
СИГНАЛ ИЗ ИСТОЧНИКА
23
очков источника
Наблюдаем с21 октября 2026 г.23 очков источника
Темп интереса+2,95/hпо замерам за 1,02 h
Опубликовано21 октября 2026 г.Jason Fantl
ЗА ЦИФРАМИ

Как меняется интерес

История начинает расти

График появится после повторных замеров. Текущий показатель уже получен из источника.

23 очков источника

Только реальные замеры. История до подключения источника не восстанавливается.

Полезная находка?
ПРОДОЛЖИ ИССЛЕДОВАНИЕ

Рядом по теме

Вся тема