O Futuro da Informação
O Futuro da Informação
@natolambert
Open model research @ something new. Prev. co-led Olmo at Ai2. Contact via email. Writes @interconnectsai, @readsail Wrote The RLHF Book, 🏔️🏃♂️
It's been a great effort by the early and growing American open-model labs since last June to put the US much more back on the map. We were getting totally owned last June. Nvidia, Ai2, Arcee, Gemma, GPT-OSS and a few others will be seen as saving American open AI.
Generally it's the who's who of people who released useful models from June 2025 to June 2026. Nemotron Ultra 3 seems like a watershed moment in how the equilibrium is viewed long term.
I feel like this also goes for a lot of people without Mythos as they learn to use agents too tbf x.com/scaling01/stat…
Safety by narrow control has shown to fail many times. Need more transparency on the absolute frontier, and openness close behind. x.com/scaling01/stat…
We have another 65 page frontier model report from Nvidia to read @eliebakouch @stochasticchasm and gang
@eliebakouch @stochasticchasm research.nvidia.com/labs/nemotron/…
@eliebakouch @stochasticchasm As usual, lots of datasets to dig into, base model, and a large judge model.
Nvidia joined the multi-teacher, on-policy distillation (MODP) gang! Is industry standard post-training right now. The multi-teacher SFT to RL that Microsoft did in their first model was the standard established by DeepSeek R1. I expect MAI 2 to be MODP. x.com/kuchaev/status…
Great little video on modern on-policy distillation in post-training recipes. Wish I had this when writing the section on distillation for my book. And where I've been bearish on a lot of the academic work for self-distillation, it seems impactful at the frontier. x.com/dwarkesh_sp/st…