学进去

The artificial-intelligence chatbot ChatGPT has shaken educators since its November release. New York City public schools have banned it from their networks and school devices. There is, perhaps surprisingly, one subject area that doesn’t seem threatened, It turns out ChatGPT is quite bad at maths.

“I’m not hearing maths instructors express concern about ChatGPT,” said Paul von Hippel, a professor at the University of Texas who studies data science and statistics. “I’m not sure it’s useful for maths at all, which feels strange because maths was the first-use case for the artificial-intelligence devices.”

ChatGPT’s struggle with maths is inherent in this type of AI, known as a large language model. It scans a large amount of text from across the web and develops a model that might be extremely effective for writing grammatically correct responses to essay requirement, but not for solving a maths problem.

In an email, I asked Debarghya Das, a search-engine engineer, why ChatGPT gets some simple questions right but others completely wrong. “Maybe the right analogy (类比) is if you ask a room of people, who have no idea what maths is but have read many hieroglyphics (象形文字), ‘What comes after 2+2,’ they might say, ‘Usually, we see a 4,’ That’s what ChatGPT is doing.” But, he adds, “Maths isn’t just a series of hieroglyphics. It’s the process of calculating.”

It isn’t great for pretending you know it through a maths class because you only recognize the mistakes if you know the maths. Another reason that maths instructors are less anxious about this innovation is that they have been here before. The field was upended for the first time decades ago with the general availability of computers and calculators.

“Maths has had the biggest revolution based on machinery of any mainstream subject,” said Conrad Wolfram, the strategic director of Wolfram Research. “In the real world, since computers came along, have maths, science and engineering gotten conceptually simpler? No, completely the opposite. We’re asking harder and harder questions, going up a level.”

Eventually, AI will probably get to the point where its maths answers are not only confident but correct. A pure large language model might not be up for the job, but the technology will improve. In general, however, AI, like computers, will likely ultimately be most useful for those who already know a field well. They know the questions to ask, how to identify the shortcomings and what to do with the answer. A tool, in other words, is for those who know the most maths, not the least.