@srush_nlp
Researcher at Cursor https://t.co/cZl0wTfqGz
Forget math proofs, we’re getting to the real frontier now.
Fantastic slides from the ICML tutorial of @MarkSchmidtUBC (“is opt theory relevant in 2026”) cs.ubc.ca/~schmidtm/Docu…
My brain is pretty split between ML I know how to teach and ML I do in practice. Really appreciate the honesty about when these are bridgeable.
(Annotated) Structure and Interpretation of Classicical Mechanics - srush.github.io/annotated-sicm An experiment in formalized textbooks. Implements SICM Ch 1 in Typescript / Jax with code-math alignment and a simulator.
This is an interesting book because it is written in a code literate style. Cursor was able to convert it over relatively easily. Interesting questions are when formal typing is useful for text like this, versus the style of the book.
Frontend Code Arena 😂 x.com/DavidSacks/sta…
I circled all the Transformer parts of k3. (I'm a sore loser). x.com/srush_nlp/stat…
Time for everyone’s favorite game! Is it a Transformer?
anyway, very cool model.
No idea what Thinking Machines is working on, but this line goes hard.
This is an incredible blog post. Peak “centaur chess” phase of AI coding. x.com/jarredsumner/s…
I love the idea that the problem *can* be fully specified in one sentence, but each of the individual steps still require a brilliant programmer and a Rube Goldberg machine of interlocking checks to get there.
It’s crazy that BPE was only 1 year before Transformers. I can’t even remember the before times. x.com/jzhou_jz/statu…
Look at the absolute slop people were publishing in Feb 2016.
It’s really convenient that you don’t have to read past Chapter 2 of Sutton+Barto anymore.
The Short Lesson.
@distributionat But yes, I agree with you that personally the specifics of warmup + layernorm mattered a lot, and were not at all obvious to me coming from an RNN background.
This @jxmnop guy is worth following. x.com/icmlconf/statu…
An incredible group. 🍾 x.com/EngramLab/stat…
unfortunately I have learned almost nothing new about robust information extraction.
@hallerite @Harman26Singh @agarwl_ (Mostly just curious)
@Harman26Singh @hallerite @agarwl_ Other thoughts: * Someone should open source a good KL kernel and remove that confounder * I don’t get why people use other rollout trajectories as PI * would love to see more work on generating PI
