A curated list of links I found insightful in January 2025. This is the first of what I hope will be a monthly series. Feel free to explore and click through to read more!

Is AI Progress Slowing Down? - AI Snake Oil

  • This post offers a comprehensive take on where AI stands as of December 2024. The author takes a rational and conservative approach, using first principles. For example, while not buying into the hype from Sam Altman and other leading AI labs, they also believe that model scaling is far from over and that there’s no real evidence to suggest otherwise. It’s interesting that the blog points out that Altman and Sutskever have an incentive to shape the narrative, and their statements might not be the whole truth. The post touches on inference scaling too. Overall, a well-balanced piece.

Computing Inside AI - Will Whitney

  • I think Human-Computer Interaction (HCI) is super important. A computer is fundamentally a tool, and it’s most useful when we streamline the interaction with humans, eliminating bottlenecks and inconveniences. Human-AI interaction isn’t really an established field yet, but I believe it will be. The author argues that instead of interacting with current LLMs as if they were people, we should treat them as tools. They suggest an LLM could dynamically generate an interface, similar to a computer. What would that even look like? It’s a really interesting idea because, as the author points out, conversing with AI can be slow and involves this isolated back-and-forth. Human-computer interaction isn’t like that. What if we fundamentally changed how we interact with AI? This is a very thoughtful post.

Building a Virtual Machine Inside ChatGPT - Jonas Degrave

  • I first encountered this blog post at the end of 2022, around the advent of ChatGPT, and I recently rediscovered it. The author attempted to simulate a Unix terminal environment using ChatGPT, and, surprisingly, they succeeded. This might not seem remarkable now, but it completely blew my mind when I first started using ChatGPT. The author is really clever, and they take it a step further by going “all meta,” which I won’t spoil. This clearly demonstrated that the LLM could simulate a system - in this case, a Unix system. It gave me this sense that this LLM was something truly special and that it would fundamentally change everything. I still believe that, and I owe that kind of inspiration and shock to this blog post.

Francois Chollet - Why The Biggest AI Models Can’t Solve Simple Puzzles

  • I initially knew François Chollet due to his activity on Twitter as an AI skeptic (or, at least, somewhat of an AI skeptic) and his work on ARC prizes. So, it was exciting to see him interviewed by Dwarkesh Patel. I really enjoyed it because Dwarkesh grilled François on his views on LLMs. I think Dwarkesh’s approach was a bit unfair because, while François made some valid points, Dwarkesh seemed intent on pushing him to say what he wanted, but he ultimately failed to do so. While listening, I felt torn between them. I agree with Dwarkesh’s overall viewpoint but align with François’s specifics regarding intelligence and the core of reasoning. It was a conflicting yet invigorating discussion, and I think I should re-watch it to savor it more.

François Chollet: Keras, Deep Learning, and the Progress of AI - Lex Fridman Podcast #38

  • After the Dwarkesh podcast, I turned to Lex Fridman’s podcast to hear more from François Chollet. And, in classic Lex Fridman fashion, he delivered. This first one is an earlier interview from 2019, when François was much younger. In current public settings, François appears more glum and calm, but in this interview, I could see a sparkle and a smile in his eyes and on his face, which was refreshing. This podcast delves into the advent of Keras and TensorFlow, which I really enjoyed.

François Chollet: Measures of Intelligence - Lex Fridman Podcast #120

  • In this podcast, both François and Lex are a bit older. I watched this because I was simultaneously reading François’s paper on the measure of intelligence. He’s soft-spoken but offers insightful perspectives on intelligence. While I don’t agree with everything he says - I believe that LLMs and other AI systems are capable of more generalization than François suggests - I think his views as an AI skeptic aren’t unfounded. His logic is sound, but I believe he underestimates the intuitive power of these machine learning systems.

Feature Visualization

  • This is an excellent method for visualizing and interpreting models. The paper interprets, reverse-engineers, and reconstructs what features each neuron or layer is attending to in an image. The basic idea is that, through iteration, we generate an image that a particular neuron fires most strongly for. This yields fascinating images of what the neuron has learned, and I find it very compelling.

DeepSeek FAQ - Stratechery

  • This week was all about the buzz surrounding DeepSeek’s release of a model called R1, and Stratechery did it again. Ben Thompson distilled what was important and the broader implications. Ultimately, the biggest winners are the customers and the general public. I think the overall point is that open-sourcing is both morally and practically right, since AI models are becoming commoditized, and costs are racing to the bottom.

DeepSeek CEO Interview - ChinaTalk

  • This is an interview with the DeepSeek CEO, and based on the interview, he is very ambitious and really focused on innovation, which I admire. Previously, China was primarily focused on keeping up with the US, but he feels they need a culture of innovation - not just following, but innovating themselves and becoming trendsetters. It will be a really interesting dynamic, and I’m glad that he is bullish on open-sourcing and that his company remains committed to it.

AI’s Uneven Arrival - Stratechery

  • This is a very interesting take, drawing an analogy between Facebook’s unconventional approach to advertising and the previous advertising agency model. Facebook directly connected consumers and advertisers via algorithms. Meanwhile, traditional advertising companies acted as middlemen between ad sellers and big brands. The author connects this to the OpenAI reason model and basically says that those agents will not replace individuals in traditional companies that use people. Instead, AI-driven organizations will likely start without them from the outset.

DeepSeek and Export Controls - Dario Amodei

  • It’s pretty interesting to see Amodei’s reaction to this, and I think he’s right. If you’re on the American side, the most logical thing to do is to clamp down on AI exports - specifically, chip exports. But the thing is, those chip exports… the Biden administration didn’t constrain the chips on the bandwidth side, so that’s why DeepSeek had a workaround utilizing maximum bandwidth. But if they clamp down on both bandwidth and compute, I’m not sure that would be the fundamental solution to this arms race.


That’s it for January! I’m excited to keep exploring and sharing more in the months ahead - hope you found something thought-provoking here.