Here’s a collection of links that I truly enjoyed reading in 2024. It’s not a complete list, as schoolwork often kept me from meticulously tracking everything, but these are the highlights. Moving forward, I’m aiming to share these finds monthly in 2025, with more detail and a broader scope.

Cruise

  • I’ve always admired George Hotz for his unconventional ideas, and this one is no exception. It’s a movie plot he conceived, and while it may not be as mind-blowing as some of his other work, I enjoyed it.

Valuable Humans in Transit

  • This story exemplifies great science fiction writing. It masterfully employs the “show, don’t tell” principle. In good science fiction, especially hard science fiction, the context isn’t immediately clear. The story unfolds gradually, and the world-building reveals itself through the narrative. This story does exactly that. Initially, the narrator’s words might seem confusing, but the situation becomes grippingly clear as you continue reading.

Can AI Unlock the Secrets of the Ancient World?

  • This article details a competition where scientists are attempting to decipher scrolls that were burned and buried during the eruption of Mount Vesuvius. Using advanced CT scans and AI for pattern recognition, they’re making progress in reading these seemingly destroyed texts. I recall hearing about this project a couple of years ago, and seeing its fruition is truly astonishing. It reinforces my optimism about technology.

All My Thoughts After 40 Hours in the Vision Pro - Tim Urban

  • Tim Urban’s review of the Vision Pro resonated with me. While I agree with his overall assessment, I believe the convenience factor needs more consideration. A large display strapped to your face can be cumbersome and inconvenient, which might deter widespread adoption.

Sam Altman’s Knack for Dodging Bullets - With a Little Help From Bigshot Friends - WSJ

  • I’ve always recognized Sam Altman’s intelligence from his talks and podcasts, but I hadn’t realized the extent of his manipulative tendencies. This article provides a more grounded perspective on the OpenAI drama of late 2023, revealing it as a power struggle where Ilya Sutskever ultimately lost.

The Inside Story of Microsoft’s Partnership with OpenAI - The New Yorker

  • This article offers Microsoft’s perspective on the OpenAI upheaval. It vividly portrays Microsoft’s frustration and how they leveraged their position to their advantage.

What Are Dreams For? - The New Yorker

  • Explores the curious phenomenon of twitching during dreams. It turns out our brains aren’t causing the twitches; it’s the other way around. Our bodies twitch, and our brains respond. The hypothesis is that our brains are recalibrating by listening to our bodies during sleep. Fascinating!

How Much of the World Is It Possible to Model? - The New Yorker

  • As someone intrigued by the simulation hypothesis, I expected this article to delve into the philosophical implications of simulating the world. Instead, it provided a concise overview of the concept and history of simulation. While it briefly touched on large language models as simulations of thought, I found it somewhat lacking in depth.

Fine-tuning GPT3.5-turbo based on 140k slack messages

  • This short blog post details the author’s experience fine-tuning GPT-3.5 with a massive dataset of Slack messages to mimic their writing style. The results, as shown in the image below, demonstrate that fine-tuning can be remarkably effective.

The Itch - The New Yorker

  • This article includes a fascinating quote: “If visual sensations were primarily received rather than constructed by the brain, you’d expect that most of the fibres going to the brain’s primary visual cortex would come from the retina. Instead, scientists have found that only twenty per cent do; eighty per cent come downward from regions of the brain governing functions like memory. Richard Gregory, a prominent British neuropsychologist, estimates that visual perception is more than ninety per cent memory and less than ten per cent sensory nerve signals.” This suggests our brains heavily compress visual information.

New Year’s Resolutions for an Anteater - The New Yorker

  • A truly humorous piece. It makes me want to be an anteater!

Prompt Library - Anthropic

  • A useful library of prompts designed for use with Anthropic’s Claude model.

Making Deep Learning Go Brrrr From First Principles

  • This post effectively captures the essence of the “bitter lesson” and its implications for the development of large language models. The memes are also quite entertaining.

My Last Five Years of Work - Palladium Magazine

  • Written by an Anthropic employee, this article reflects on the changing nature of work in the age of potential superintelligence. It offers a glimpse into the thoughts of AI lab insiders, although I don’t fully agree with the author’s predictions. Nevertheless, it’s a thought-provoking read.

AI’s $600B Question

  • This article presents an interesting perspective: as technology advances, compute power will become cheaper, and GPUs will depreciate faster than anticipated. I believe there’s truth to this.

FineWeb: decanting the web for the finest text data at scale - a Hugging Face Space by HuggingFaceFW

  • This provides a detailed explanation of how to preprocess vast amounts of web data for LLM pretraining. It emphasizes the importance of careful deduplication and filtering for educational content using LLM labeling.

The Friendship That Made Google Huge

  • Another excellent New Yorker article, this one about Jeff Dean and his colleagues. It shows Dean’s brilliance and how Google’s ability to scale was its true strength. I wonder if that still holds true today. The close intellectual synchronization between Jeff Dean and Sanjay Ghemawat, developed through years of collaboration, is particularly intriguing.

How I Use AI - Nicholas Carlini

  • An insightful piece by a DeepMind researcher on how he utilizes AI. His approach aligns with my own and what I advocate for.