Another month, another batch of links I wanted to save. Here’s what caught my attention in February 2025.

DeepSeek, China, OpenAI, and AI Megaclusters - Lex Fridman Podcast #459

  • I first encountered Dylan Patel on the Dwarkesh podcast with Asianometry, so I was glad to see him on Lex Fridman’s. He specializes in analyzing AI hyperscalers, so his comments on infrastructure are useful. Recommended if you’re interested in that side of AI.

How to Scale Your Model

  • This is published by a group of researchers at Google DeepMind, particularly Sholto Douglas. It’s a step-by-step guide to scaling up compute using infrastructure, specifically TPUs. I’ve been working my way through it. It starts with the basics but gets complicated quickly. It is good to see closed AI labs leaving these “breadcrumbs” for the open-source community.

The Ultra-Scale Playbook: Training LLMs on GPU Clusters

  • Similar in nature to the DeepMind scaling book, but this one is from the Hugging Face team. It is also detailed, interactive, and quite dense.

Thinking like Transformer

  • The author created a Python library and a detailed blog post explaining how computation works inside a transformer network. He also developed an abstract conceptual framework for it. It’s complicated, and I still don’t fully grasp it, but it is fascinating.

Gwern - Anonymous Writer Who Predicted AI Trajectory on $12K/Year Salary

  • This is from the Dwarkesh Podcast. He introduces a blogger named Gwern, and you really need to read his work to understand. He has an extensive blog called gwern.net. His posts are deep and heavily researched, more like standalone articles. The depth stands out, even compared with reputable sources like the New York Times. It also made me think about how the current internet is dominated by big tech. In the earlier internet, there were more niche individual creators posting strange, unique, high-effort material. Gwern feels like a holdover from that era.

The Melancholy of Subculture Society - Gwern

  • Gwern’s writing is long, so it requires dedicated time. This piece is a meditation on how the internet, as technology evolves, is segregating society in strange ways. I think it is well thought out.

2024 Letter - Zhengdong Wang

  • This blog post, written by a researcher at Google DeepMind, makes a strong claim about current language model research. He argues that if the evaluation is clearly defined, models will eventually succeed at it. If there are evals, any evals, the model will get there. It’s an audacious statement, and the implications are significant. I should probably write a dedicated blog post about this.

Sesame Research - Crossing the Uncanny Valley of Voice

  • Sesame revealed an interesting voice-to-voice language model. They have a demo, and it works quite well. The key difference between Sesame and ChatGPT’s Advanced Voice Mode is that Sesame produces nuanced tones, like “ums” and “ahs,” very naturally. It feels realistic. For the first 10 minutes, I was impressed. But as I probed the model further, it became clear that it doesn’t have the same level of intelligence as stronger LLMs. ChatGPT’s Advanced Voice Mode has a similar issue, though OpenAI seems to have limited that model heavily for safety and policy reasons. In Sesame’s case, I suspect the limitations are more about model and compute constraints.


That’s it for February. I will keep saving these as I go.