Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)

In Lenny’s Podcast, host Lenny Rachitsky speaks with Emmanuel Ameisen, an interpretability researcher at Anthropic, about what examining the internal workings of large language models reveals beyond their visible outputs. Ameisen previously worked in applied machine learning at Stripe and Insight and wrote Building Machine Learning Powered Applications. The episode’s central thesis is that describing models simply as next-token predictors understates the internal representations and planning that can support their responses.

The show notes identify three research findings behind that argument: models form general representations of concepts, plan many tokens ahead without explicitly stating those plans, and may exhibit a structural distinction resembling conscious and unconscious processing in humans. The discussion frames models as “grown” rather than conventionally built and examines why next-token prediction is harder than it initially sounds. It also explores how smooth scaling curves can produce apparently sudden capability breakthroughs, with the Golden Gate Claude experiment serving as a named interpretability example.

These questions matter as understanding AI capabilities increasingly requires distinguishing observable performance from the mechanisms producing it. The episode connects interpretability to alignment and safety, explicitly addressing why watching model behavior is not enough to solve alignment. It also considers when anthropomorphizing AI can be useful: human analogies may help describe some patterns, but the notes’ qualified language about similarities to human processing does not establish that models possess human-like consciousness.

Several issues remain open in the discussion, including whether AI could be conscious, what the “J space” reveals about model self-awareness, and how interpretability can contribute to alignment. The episode includes reasons for optimism about alignment and a discussion of entering interpretability research. However, the notes do not provide the detailed experimental methods, results, or limitations needed to independently assess the findings, so the briefing’s conclusions remain at the level of the research claims and questions described.