The article discusses the contradiction between theoretical arguments about the information inefficiency of RL and practical results where RL shows high effectiveness in training language models.
The chart will appear after repeat observations. The current metric comes from the source.
13 source pointsReal observations only. History before source connection is not reconstructed.
This may be useful for understanding the mechanisms of language model training and the effectiveness of different optimization methods.
The article discusses the Piltdown man hoax, a fraudulent artifact that misled evolutionary biology for decades. It draws a parallel between the Piltdown man and the Voynich manuscript, suggesting the latter might also be a meaningless fabrication. The author argues that while there is evidence supporting the idea that the Voynich manuscript is a hoax, it does not meet modern standards for proving fraud.
4 days agoLessWrongThe author investigated training GPT-2 with full-batch gradient descent on a subset of FineWeb data. Results showed that the full-batch approach achieved lower training loss but worse generalization on validation data compared to standard training.
yesterdayLessWrongThe article describes the efforts of the author and his friend Mike to automate the process of transferring data from a website to a spreadsheet using AI. Mike, who works in data processing, spends significant time manually entering information, which the author considers inefficient.
4 days agoLobstersThe source text raises a question about whether Ente Locker copied code without proper attribution.
2 days agoHacker NewsOpenCode 2.0 is a project related to open source code.
yesterdayLobstersLinus Torvalds is working on a significant Linux kernel improvement using code generated by Claude.
3 days ago