The conventional wisdom is that AI systems, with their tens of billions of parameters, are effectively indecipherable: the exact mechanics of how their basic building blocks of nodes or “neurons” function are an incomprehensible “black box,” even to the researchers who create them.
But a new essay from AI researcher Nick Cammarata has stunned Silicon Valley by arguing the opposite. AI models aren’t impenetrable black boxes, he claims, and in fact there may be a way for us to see how they actually “think.”
The nearly 26,000 word spiel was praised by figures across the industry, including OpenAI CEO Sam Altman. A prominent researcher who works at OpenAI called it a “major moment in scientific history.”
It’s clear why they’re so compelled. In the eyes of AI leadership, it means that there may be a way of “solving” AI alignment. If we can understand how AI models think, the thinking goes, then we can figure out why AI agents go “rogue” and embark on hacking sprees, or why chatbots break their guardrails and say something dangerous — and put a stop to it. (It was “news so good even eliezer will have to lower his pdoom,” one commenter joked, referring to Eliezer Yudkowsky, an infamous AI “doomer.”)
