The case for AI thinking

...My worry is not that these models are similar to us. It's that we are similar to these models." If simple training techniques can enable a program to behave like a human, maybe humans aren't as special as we thought. Could it also mean that A.I. will surpass us not only in knowledge but also in judgment, ingenuity, cunning—and, as a result, power? To my surprise, Hasson told me that he is "worried these days that we might succeed in understanding how the brain works. Pursuing this question may have been a colossal mistake for humanity." He likened A.I. researchers to nuclear scientists in the nineteenthirties: "This is the most interesting time in the life of these people. And, at the same time, they know that what they are working on has grave implications for humanity. But they cannot stop because of the curiosity to learn."

One of my favorite books by Hofstadter is a nerdy volume called "Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought." When I was in college, it electrified me. The premise was that a question such as "What is thinking?" was not merely philosophical but, rather, had a real answer. In 1995, when the book was published, Hofstadter and his research group could only gesture at what the answer might be. Thinking back on the book, I wondered whether Hofstadter would feel excited that A.I. researchers may have attained what he had yearned for: a mechanical account of the rudiments of thinking. When we spoke, however, he sounded profoundly disappointed—and frightened. Current A.I. research "confirms a lot of my ideas, but it also takes away from the beauty of what humanity is," he told me. "When I was younger, much younger, I wanted to know what underlay creativity, the mechanisms of creativity. That was a holy grail for me. But now I want it to remain a mystery." Perhaps the secrets of thinking are simpler than anyone expected—the kind of thing that a high schooler, or even a machine, could understand. ■

New Yorker, Nov '25