Why should we care that AI is suddenly insanely good at math?

It's a question that I've been asking mathematicians and AI researchers for months, and especially this week, when OpenAI published a solution to one of the notoriously complex Millennium Prize Problems.

And it's the one math question I can answer without breaking into a cold sweat.

Math is many things to many people: beautiful, elegant, nightmare fuel. Over the past year, it has also become proof of artificial intelligence's rapidly accelerating progress, showing how the technology is getting much smarter, much faster than most people anticipated.

It's a field where skeptics predicted that AI wouldn't reach the most ambitious benchmarks for years-and AI has already surpassed them. If that can happen in a domain that requires the highest levels of reasoning, it can happen anywhere.

Humans have always used math to understand the world.

AI's progress in math is now giving us a peek at a world beyond our comprehension.

And this week, as we marveled at AI's ability to do math, we also wondered if AI is going to destroy humanity.

At the same time OpenAI published its solution to a Millennium Prize Problem, a top scientist at Anthropic put out a very different kind of math: his belief that the chances of human extinction in the next decade are now greater than 10%. He was responding to explosive comments from another researcher, Jacob Coxon, who worked at OpenAI and Anthropic but said neither company is acting responsibly as they sprint to self-improving superintelligence and risk our very existence.

Meanwhile, their mathematical arms race keeps escalating.

This is not because their business depends on proving the Riemann hypothesis. It's because math happens to be useful marketing-a way to advertise the growing capabilities of their latest models.

In that sense, the Millennium Prize moment is a friendlier version of the Hugging Face incident. Both demonstrate what these increasingly powerful systems can do when they are turned loose.

Instead of scheming to hack a company, a swarm with as many as 10,000 agents hacked away at math for 88 hours, building on each other's work-and the work of humans-until they reached a solution. The wicked Navier-Stokes problem had vexed mathematicians for nearly a century. AI needed millions of dollars in computing resources and a few days.

It was the most remarkable and previously unthinkable mathematical achievement in a year full of them.

AI models have spent the past few years doing things we believed they wouldn't be able to do anytime soon. First they couldn't do basic math. Then they couldn't do research-level math. They also couldn't win gold at the math Olympiad or crack any of math's famous mysteries.

When they did all that, the only thing left to do was solve one of the fiendishly difficult Millennium Prize Problems.

One year ago, expert forecasters gave AI a roughly 20% chance of doing it by 2030. But in recent months, the improbable began to feel inevitable. The most surprising thing about the Millennium Prize breakthrough is that it was no longer such a massive surprise. In fact, OpenAI unleashed an internal model on the Millennium Prize Problems after hearing rumors that Anthropic had already solved one.

Within days of releasing the solution to one problem, OpenAI suggested it was close to solving another, raising the possibility of two discoveries that had eluded us for decades arriving in the same week.

For those of us who haven't thought about math since high school, the recent events have taught us about Erdos problems, the Jacobian conjecture and a phenomenon known as "finite-time blowup."

For mathematicians, the developments have raised existential questions-what do you do when you have devoted your life to problems that are now being picked off by AI?-and ignited a ferocious backlash. Before the landmark proof even appeared online, social media blew up with suspicion that OpenAI's model had taken the groundbreaking, unpublished work of humans, which the company denies.

As it turned out, it wasn't even the most ominous drama involving AI and numbers that day.

Hours later, Coxon resigned from Anthropic over safety fears and told the world that people building AI "earnestly believe that it could kill us all." His comments were echoed by Evan Hubinger, a researcher whose job at Anthropic is making sure AI doesn't kill us all. On his X account, he put the odds of annihilation above 10%. In his estimation, doomsday is now more likely than Stephen Curry missing a free throw.

All of this would have sounded like sci-fi not long ago, much less at the turn of the millennium, when math's seven grand problems were selected and a $1 million prize was dangled for each solution.

"The seven problems," Fields Medal-winning mathematician Alain Connes said in 2000, "are totally inaccessible to computers."

Or at least they were. Now that we have another breed of computer, Connes told me he's "extremely positive" about AI's progress in math. Not everyone with a Fields Medal shares his positivity.

Back in the ancient times of 2023, Terence Tao saw that AI could be "radically transformative," he wrote on his blog, "to the point where maintaining traditional mathematical practices and culture without adaptation would become unsustainable." The man widely considered the world's pre-eminent mathematician still believes in the value of AI models. But he's losing faith in the companies building them.

This week, Tao was one of two dozen Fields winners to warn that the AI industry's obsession with solving math problems could undermine the purpose of math itself. He says math is quickly approaching a "worst-case scenario" that threatens the long-term health of his field and society at large.

On the day society learned about AI's solution to a Millennium Prize Problem, I called another Fields medalist to make sense of the latest breakthrough.

"The flickering dream that there's still a human role in mathematics isn't completely killed off by this," Timothy Gowers told me. "But my guess is that if you've got a model that's capable of doing this, it will be capable of doing lots of other things."

Which led him to an unsettling conclusion.

"I don't want to say it's all over," Gowers said, "but I certainly don't want to say it's not all over."

Whether it's really over was the agenda when OpenAI hosted mathematicians in its San Francisco offices last month. The summit began with an assumption: AI will become superhuman at math.

Once they accepted that premise, they could reason through all of its practical implications. What does it mean for education, or collaboration, or the publication of ideas, or how we train the next generation? They didn't come up with any solutions, but that wasn't the point.

Their goal was selecting the right problems to work on, like the mathematicians who chose the Millennium Prize Problems.

And we should all care. Because this time, humans will have to solve them.