In 2023, hundreds of top AI researchers signed a one-sentence public statement putting the risk of extinction from AI on the same footing as pandemics and nuclear war.
Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.
Paradoxically, the statement was signed by some of the most aggressive pursuers of the technology, including the CEOs of OpenAI and Anthropic, arguably two of the world’s leading AI companies.
That warning became a rallying cry for people working in AI who envision the development of superintelligent AI (SAI) both as a Holy Grail to be pursued and as a cool doomsday scenario to scare all of us with. It was recently brought to the center of national attention in the United States by Jacob Coxon, a researcher at Anthropic and a former researcher at OpenAI, who accused his former employers of racing toward self-improving superintelligence and thereby risking human extinction. Coxon’s own thread on X received more than 170 million views within days; other researchers at Anthropic backed his warning1. The company's own CEO called for slowing the pace of frontier-model development; and panicked politicians began talking about aggressive federal oversight measures. It does not take a prophet to see how the story could spark a movement of its own, intensely vocal and moralizing, modeled on climate activism, demanding not just regulation but an outright halt to AI research.
Despite internal warnings from the company’s own researchers and CEO, OpenAI kept racing toward superintelligent AI (SAI). On September 8, 2026, the same day that Coxon made his warnings public, OpenAI announced an AI-generated solution to the Navier-Stokes problem, one of the seven Millennium Prize Problems. The feat, which OpenAI says involved roughly 10,000 concurrent AI “agents” and 130 billion output tokens, caps a dizzying run of ever more impressive achievements by artificial intelligence in mathematics - the discipline that represents the highest expression of human rationality. On October 6, OpenAI followed this announcement with a mass release of hundreds of AI-generated manuscripts claiming advances across a wide range of mathematical fields.2 OpenAI has already signaled further releases, deepening fears among mathematicians that their profession faces an existential threat.3
If AI can solve mathematical problems that have resisted generations of mathematicians, how can we dismiss the possibility that it will eventually surpass us across all cognitive tasks? This raises fundamental questions about human abilities and the prospect of automating them, with enormous implications for research, education and communication - indeed, almost every sphere of human activity.
Extinction problem
The true origin of that extinction warning goes back much earlier, to a 1983 essay by Vernor Vinge, in which he claimed that humans will soon create “intelligences greater than our own” - that is, an SAI exceeding human ability across all cognitive tasks. When this is achieved, Vinge wrote, “human history will have reached a kind of singularity, an intellectual transition as impenetrable as the knotted space-time at the center of a black hole, and the world will pass far beyond our understanding.”4 The proposed mechanism is recursive self-improvement: humans design an AI capable of improving its own design. The improved system then produces further improved systems at an accelerating rate, leading to a runaway intelligence explosion - the singularity. Some highly accomplished people invoke precisely this metaphor. On February 23, 2025, Elon Musk declared, “We are on the event horizon of the singularity.”5 In June 2025, Sam Altman went further: “We are past the event horizon; the takeoff has started.”6 The event horizon marks a point of no return - a metaphor for the achievement of a system capable of autonomous self-improvement. But has AI actually crossed that threshold and begun the irreversible descent to the singularity?7
Recursive self-improvement
Self-improvement covers several different achievements. AlphaZero, for example, learned to play chess and Go through self-play: it improved its performance within a fixed, human-designed learning framework. The Darwin Gödel Machine goes a step further by repeatedly modifying its coding-agent software and testing the changes on programming benchmarks. Those modifications can also improve its ability to make subsequent modifications. AlphaEvolve proposes and evaluates changes to algorithms, including algorithms used in training its underlying language model.8 These systems can improve not only their performance but also parts of the machinery through which they learn and solve problems. They do not come close to demonstrating a self-sustaining acceleration of general intelligence of the kind envisioned in the singularity scenario.
Biology has been trying for the last 70 years to solve a more modest problem: the synthesis of self-replicating life from ordinary chemistry. The goal is to establish a stable copying mechanism that permits heritable genetic variation, allowing natural selection to act on that variation and generate genuine biological novelty.9 The results have not merely been disappointing. It appears that the more scientists investigate the problem, the more complexity they discover; indeed, the synthesis problem appears even further from resolution today than ever before.
As with AI self-improvement, there has been substantial progress in producing specific components.10 The problem is how to make their activities sustain one another. The chemist James Tour poses the challenge in vivid terms: grant researchers the requisite molecules, their desired genetic sequences, modern laboratories and the accumulated scientific literature. The remaining task is to assemble a functioning cell, and in that respect, he claims, we are as far as we have ever been from a solution.11 Biochemist Anthony Futerman's critique also highlights a difficulty directly relevant to laboratory synthesis: reactions that succeed in isolation may fail when other compounds are introduced. In the lipid-synthesis experiment he discusses, adding another compound sharply reduced the yield. Constructing life therefore requires more than producing its components separately: the processes that produce and maintain them must work together without disabling one another.12
Even an optimist like John Sutherland, a leading researcher on the chemical origins of life, acknowledges the difficulty. Writing in 2017 about demonstrating the transition from simple chemicals to a living system, he stated: “We are not yet close to achieving this end.”13
Thus, even in biology, where self-replication is an observable fact, constructing a living system from nonliving chemical components remains a formidable challenge. Yet Vinge’s prediction is far more ambitious: to build an entirely novel system, with no known counterpart in the physical world, that continually enhances its own capacity for self-improvement, eventually surpassing the human mind in every respect.
The analogy concerns autonomy: a living system reproduces and sustains an evolutionary process without scientists continually redesigning it, drawing on materials and energy from its environment. The corresponding ambition for AI is a system that sustains its own improvement without continued human intervention, drawing only on computing resources, power and access to data. Improving particular components under human direction14 is a radically different achievement from establishing such a self-sustaining process.
The prospect of machines taking over invention is central to the singularity scenario. In 1965, I. J. Good argued that the first ultraintelligent machine would be the last invention humans need ever make.15 Vinge adopted this argument, predicting that superhuman intelligence would create still more intelligent successors on ever shorter timescales.16 Present AI systems improve performance relative to objectives supplied by us: predicting the next token, minimizing an error, searching for the best fit, writing faster code or learning through rewards and penalties. But the great discontinuities in science - paradigm shifts - do not arise merely from moving faster within an existing framework. What is the loss function for discovering algebra, calculus, evolution, genetics, Maxwell’s equations, atomic theory or quantum mechanics?17 What is the gradient from Newtonian mechanics to general relativity? An optimization process can produce unexpected abilities within a prescribed space of possibilities. It is up to those predicting an intelligence explosion to explain how such a process could generate new conceptual frameworks and sustain their transformation at an ever-accelerating pace.
The strongest empirical response to this challenge is the success of scaling. Its proponents argue that continued human-directed development could produce superintelligent AI, capable of inventing new conceptual frameworks, even without autonomous self-improvement. Empirical scaling laws show that a model’s text-prediction error decreases approximately as a power of its size, the amount of training data and the computing resources used.18 This regular improvement has been accompanied by the recent extraordinary gains in language, programming and mathematical problem solving.
These regularities concern goals fixed in advance. They show that a system keeps getting better at what it is trained to do, according to fixed rules designed by us. Yet big discoveries that lead to paradigm changes do not at appear to be reducible to fixed rules of procedure. These scaling laws also indicate diminishing returns: further reductions in text-prediction error require increasingly large amounts of computation. Moreover, the supply of good human-written text is limited, so more and more of the training material would have to be generated by the machines themselves, with the risk that they recycle their own errors. Training by rewards works where a reliable way of checking answers exists, but there is no such check for deciding which problems matter or for inventing a new definition. To say that new concepts will emerge at sufficient scale is to extend a trend into the future rather than to identify a mechanism, and it is close to the appeal to “emergence” that I questioned above. The burden remains on those who predict an intelligence explosion to explain how scale alone will create new conceptual frameworks, and not merely better performance within existing ones.
All this leaves me skeptical of predictions that AGI is around the corner. For the foreseeable future, I expect major improvements in AI requiring new ideas or fundamental changes in design to remain the work of humans rather than AI agents. Yet the machines we build will increasingly outperform us at tasks traditionally regarded as measures of intelligence. This confronts us with a paradox: are the machines we create more intelligent than their human creators? Is AlphaZero more intelligent than the engineers who created it simply because it vastly surpasses them at chess or Go? If a powerful AI solves mathematical problems that have resisted generations of mathematicians, does that render mathematicians obsolete?
What is intelligence?
Before declaring the human mind obsolete, we had better ask what our current measures of intelligence leave out. Standard IQ tests do not adequately capture some of our most important capacities: choosing worthwhile goals, sustaining focused effort in pursuing them and exercising judgment about when to persist, change direction or abandon them. Above all, the human mind has the uncanny ability to refuse to get trapped in any finite space; whenever it becomes aware of its confinement, it seeks and often finds a way out of it. It can question the assumptions of a given problem, reformulate it and invent concepts that open possibilities previously beyond its horizon. We are finite creatures endowed with free will and self-awareness as well as an irrepressible intuition of infinity19. Though free will has often been contested,20 it is hard to dispute that whenever we become aware of a specific intellectual limitation, we can take steps to overcome it or even act against it. In that sense the Socratic injunction “know thyself” is both an act of liberation and an engine of human creativity.
Humans are slow at performing calculations, but we compensate by thinking in terms of principles and structures that embed the crucial steps of a calculation in the architecture of a theory, making those steps appear natural and easy to verify. Mathematical insight can also precede its expression in words: we may recognize a pattern, picture a geometric relation or sense a unifying principle before we can formulate it precisely. As Einstein explained to Hadamard, “The words or the language, as they are written or spoken, do not seem to play any role in my mechanism of thought.”21 The written argument gives logical form to ideas whose discovery followed a very different path - one that cannot be reconstructed. This distinction between an idea and its expression is familiar across human creative activities. A composer may hear music inwardly before committing it to notation, just as a mathematician may grasp a structure before finding the words and symbols to describe it. Mozart, for example, distinguished the two stages in a letter to his father about Idomeneo: “the composing is finished, but not the writing out.”22
LLMs, by contrast, learn from vast collections of written material, including the mathematical literature available on the internet.23 These texts record ideas after they have found expression in words and symbols, leaving out the thought processes through which a new concept, geometric picture or unifying scientific principle first emerged - or the work of imagination that gave rise to a great work of literature or art. The finished work is available to the machine; the act of imagination that brought it into existence is not.
AI is at its best when human judgment directs it toward clearly defined tasks within restricted domains of knowledge, grounded in verified results or informed expert opinion. Without such direction, it relies on statistical patterns learned from the internet, too often recycling the clichés, muddled opinions and outright nonsense that abound there. Worse still, it can “hallucinate,” inventing facts, quotations or references and presenting them with all the confidence of an expert.
Human thought draws on more than language and abstract reasoning: it is shaped by the experience of living in a body. We encounter the world through perception, movement, physical effort, emotion and interaction with other people. Our bodies and minds have a complexity, at both microscopic and macroscopic levels, that remains far beyond our understanding. Giving a machine cameras, sensors and mechanical limbs does not give it the bodily experience that shapes human thought: hunger, pain, fear and desire - “the heartache and the thousand natural shocks that flesh is heir to.” Even the robots envisioned by AI enthusiasts, let alone those available today, fall far short of reproducing the richness of bodily experience that informs our understanding and creative process.24
Human creativity also draws on taste, wisdom, aesthetic sense, faith, intuition, agency, moral sense, empathy, altruism, ego, competitiveness and other attributes whose workings do not appear to be reducible to algorithms and calculations. These are inseparable from human intelligence: they shape what we care about, which questions we pursue and what we judge worth creating.
AI enthusiasts argue that future AI could possess both free will and consciousness, together with other attributes of the human mind. Some suggest that consciousness could emerge in artificial systems with sufficient complexity and the right organization.25 Yet merely invoking “emergence,” without specifying the process by which consciousness would arise, is not so different from invoking the hand of God. It simply gives the mystery a fancy, scientific-sounding name rather than a serious explanation.
Whether we regard this prospect as plausible appears to depend on our view of a deeper question: can the mind, at least in principle, be fully understood in terms of the physical components and physiological processes of the brain? Materialists26 answer yes, pointing to the impressive progress of neuroscience in relating subjective experiences to specific patterns of brain activity. Yet establishing such connections is not the same as explaining the experiences themselves. Computers provide a useful analogy for this mind-body problem: software requires physical hardware to run, but a description of the hardware alone does not explain the program’s logic, structure or meaning. Likewise, a radio can reproduce music from a distant source. The sound can be stopped or otherwise disrupted by the radio’s circuitry, but no one would claim that the radio itself produces the music.27
I hold that objective reality extends beyond what we call the physical world. Mathematics offers the clearest example: its objects are accessible to the mind rather than the senses, yet their properties resist our wishes and constrain our reasoning as firmly as physical objects constrain our actions. As I argued in my essay on Wigner, even our understanding of physical reality depends on mathematical structures whose meaning and coherence extend far beyond their physical applications.28 The mind does more than process information about the material world: it discovers and explores this larger reality.
AI and mathematics
Even though AI may not acquire consciousness or surpass us at every task anytime soon, it is already having a major impact on society. For a microcosm of that impact, consider mathematics, my own field of research and teaching. Paradoxically, although AI is itself largely a product of mathematics29, its impact on mathematics has already been extraordinary.. In the past month alone, AI-generated work has claimed breakthroughs on a large number of famous mathematical problems, including work on four of the Clay Millennium Prize Problems: the Navier–Stokes existence and smoothness problem, the Riemann hypothesis, the Hodge conjecture and the Birch and Swinnerton-Dyer conjecture. These claimed results, part of the October 6 mass release described above, have shaken the mathematical community to its core. Even before these announcements, Jacob Tsimerman, one of the 2026 Fields Medalists, concluded his lecture at the International Congress of Mathematicians in Philadelphia that July by predicting that AI would become superhuman at mathematics within a few years, producing proofs and proposing new conjectures and insightful definitions.30 These predictions are having a profound effect on mathematicians, especially younger researchers, many of Hundreds wonder whether it still makes sense to pursue a career in mathematics.
Though the challenges posed by AI's extraordinary recent advances are serious, Tsimerman’s predictions, delivered with full confidence and the authority of a newly awarded Fields Medal, are, in my view, as irresponsible as predictions that AGI will soon kill us all - particularly given their demoralizing effect on young mathematicians. We need to distinguish serious concerns about how AI is presently used from sweeping predictions presented with unwarranted certainty.
The significance of these announced AI breakthroughs depends first on whether the AI-generated proofs released by OpenAI are correct.31 But even if the claimed results are correct - and many are - they do not by themselves justify Tsimerman’s predictions. A proof may resolve a major conjecture through a new conceptual idea, a powerful extension of existing methods or computations previously beyond reach. These are all achievements, but they provide different grounds for assessing predictions about the future of mathematics.
This dim view of the future of mathematics, in which humans are replaced by machines, rests on the mistaken belief that the main purpose of mathematics is to prove theorems. But a mathematician does more than turn coffee into theorems, as Rényi's famous quip, often repeated by Erdős, would have it. Like any other science, mathematics seeks to discover significant truths and understand their connections. In the natural sciences, claims about the physical world are tested through observation and experiment and unified by mathematical theories that yield further testable predictions. In mathematics, proof is the means by which discoveries are validated. If AI can help us find proofs of statements that have resisted our best efforts, so much the better. The natural sciences already rely extensively on computer-assisted calculations. Automating parts of mathematical proof would make mathematics more like the natural sciences in this respect: machines perform more of the technical work, while scientists seek the ideas that give that work meaning and direction.
Mathematicians have long considered the possibility of automating proofs. Hilbert’s tenth problem, posed in 1900, asked for an algorithm to determine whether a polynomial equation with integer coefficients has an integer solution. The later Entscheidungsproblem, or decision problem, asked for a procedure that would always terminate and determine whether any given statement in first-order logic32 is provable.
Church and Turing showed in 1936 that no such decision procedure exists. This does not rule out automated proof search: within a fixed, effectively specified formal system, an algorithm can systematically examine all possible proofs and eventually find a proof whenever one exists. If none exists, however, the search may continue forever. Searching through all possible proofs is usually hopelessly slow. AI can make the search more effective by suggesting promising approaches and useful intermediate steps, then changing course when an attempt fails. This can bring previously inaccessible proofs within reach, but it does not overcome the logical limitations established by Church and Turing: there is still no procedure guaranteed to settle every problem. More importantly, its growing ability to find proofs must also be judged against the deeper goal of mathematical understanding. If a proof succeeds only because a machine can examine a myriad of cases faster than humans, without revealing any unifying principle, it may establish a fact while offering little insight into why it holds. Good mathematical proofs do more than validate statements: they reveal connections and, at their best, introduce conceptual frameworks that lead to further discoveries. A striking example is the contrast between Abel's proof that the general polynomial equation of degree five or higher cannot be solved by radicals and Galois' subsequent theory. Abel settled the impossibility question; Galois revealed the underlying structure, relating solvability to the group of symmetries of the roots. Abel's original proof has been essentially forgotten, while Galois' conceptual framework transformed algebra and became fundamental throughout mathematics and science.
The uniformization theorem provides another striking example. Its PDE proof, based on finding a conformally related metric of constant curvature, is more revealing than the classical complex-analytic proof, going back to Riemann. It exposes the relation between conformal geometry, curvature and nonlinear differential equations, and points toward higher-dimensional generalizations, notably the Yamabe problem of finding metrics of constant scalar curvature within a given conformal class. The celebrated Hamilton-Perelman proof of the Poincaré conjecture carries this geometric PDE approach much further. Hamilton introduced Ricci flow, an evolution of the metric governed by a nonlinear PDE, with the aim of obtaining canonical geometries; Perelman's later breakthroughs made it possible to establish the broader geometrization conjecture.
In my own work on the stability of black holes, we now have a complete proof spread across many highly technical papers totaling well over 2,000 pages.33 I can think of no more important goal than finding a shorter, more conceptual proof - one that reveals the essential mechanisms and opens the way to further developments.
Mathematics abounds in examples showing that the ideas revealed by a proof are more important than the result it establishes. Nothing illustrates this more dramatically than new methods that transform mathematics even when, at first glance, they do nothing more than prove old theorems. Cartesian geometry provided a systematic way to translate problems in Euclidean geometry into algebraic equations. Its significance extended far beyond new proofs of familiar results: it transformed the relation between geometry and algebra, helped pave the way for calculus and provided a mathematical language for the emerging science of physics. Grothendieck, one of the twentieth century's great builders of mathematical theories, compared two approaches to proving a theorem with two ways of opening a nut: striking it with a hammer and chisel or immersing it in water until the shell softens enough to open by hand. The second captures his approach to mathematics: patiently developing concepts and structures until a previously resistant problem becomes a simple natural consequence of a broader theory.34 He also described his approach through the image of a rising sea: “Little by little it becomes a peninsula, then an island, then an islet, which in turn is finally submerged.”35
Physicists and engineers use mathematical theories - their concepts, equations and methods. They rarely concern themselves with the precise statements of theorems, much less their proofs. The extraordinary usefulness of mathematics rests primarily on its power to provide theoretical frameworks for understanding and describing the world. When Einstein wrote in “On the Method of Theoretical Physics” that “the truly creative principle resides in mathematics,” he was pointing to this power of mathematics to guide physical discovery, rather than to any particular mathematical result.36 He was keenly aware that the search for his beloved unified theory might require mathematics that had yet to be discovered.
An obsessive fixation on solving famous problems can sometimes be counterproductive. The announced AI solution to the forced Navier–Stokes Clay problem37, for example, while technically impressive, teaches us little about the behavior of real fluids. The exceedingly complicated blow-up solutions it constructs are highly unphysical and most likely unstable, though this remains to be established. Before computational assistance became available, I would have regarded encouraging someone to construct such a solution by hand as professional malpractice. By contrast, showing that solutions remain regular for all time outside a suitably defined exceptional set of initial conditions is a foundational question - a first step in a program aimed at understanding the behavior of fluids, including turbulence. That problem remains far beyond the current abilities of both humans and AI.
Another misconception is that mathematics consists of fixed fields - such as number theory, algebraic geometry, topology, differential geometry and probability - organized around lists of open problems. Tsimerman refers to mathematics as a closed system. But it appears closed only if we reduce its purpose to settling famous problems within these established fields. Such a setting is particularly favorable to AI: the questions are already formulated, the relevant concepts developed and a substantial body of methods and results available to draw upon. AI can search this literature, combine techniques and test possible routes to a proof. What this picture leaves out is the creation of new concepts, new questions and new fields of mathematics.
In fact, mathematics continually enlarges and transforms its frameworks. Contact with other sciences, especially physics, has been indispensable to this renewal. Such exchanges were central to the golden age stretching from Descartes, Galileo and Newton to Gauss, Riemann, Maxwell and Einstein. As Poincaré emphasized, nature presents questions that mathematical imagination alone would never have conceived, preventing mathematics from endlessly turning in circles.38 AI's growing ability to resolve conjectures within established fields may encourage a renewal of this interaction. By freeing mathematicians to concentrate on conceptual development, it may stimulate both new unifying theories within existing fields and the creation of new fields in response to questions arising in other sciences.
Even if theorem proving were eventually delegated entirely to AI - a prospect still far from today’s reality - mathematics would retain its central creative tasks: deciding which questions matter and developing the concepts and theories that make their answers illuminating. This would mark not the end of mathematics, as many fear, but the beginning of an exciting new stage in its development.
There's never been a better time to go into math
Grant Sanderson, creator of 3Blue1Brown, chose this provocative title for a recent video arguing that AI's advances open new opportunities for mathematicians.39 I share his optimism: mathematics extends far beyond proving theorems, and automating parts of that work may indeed create room for new ideas and directions. But realizing these opportunities will require confronting serious challenges - in how we assign credit, train young mathematicians and preserve the open exchange of ideas.40
My own work with AI illustrates a new mode of mathematical research: a constant dialogue guided by my goals, ideas and judgment. AI helps explain papers, check calculations, develop approaches I suggest and test examples and counterexamples. It also assists with LaTeX and exposition. Through repeated questioning and correction, I examine its arguments and assumptions: a plausible answer may contain an error or leave the central difficulty untouched. Making each step explicit helps clarify the obstruction and decide how to proceed.
If OpenAI releases the internal model behind its recent collection of mathematical results, I may also be able to ask AI to carry an approach through to a complete proof. If the proof strikes me as interesting, it could open a further phase of research: digesting the proof, identifying its essential ideas and developing them into concepts and methods that reach beyond the original problem. The proof would then become a starting point for new questions and a broader theory. I would welcome this: it would allow me to concentrate on the part of mathematics I enjoy most - finding the concepts that unify what I know and developing a vision for new directions of research.
All this is very promising, but there are also serious risks. AI could greatly expand what mathematicians can accomplish while undermining the community that makes those accomplishments possible. We must distinguish between challenges that are serious but manageable and those that threaten the continued vitality of mathematics as a human enterprise.
OpenAI's hostile mass release exposes a darker side of AI. Hundreds of proposed results arrive at once, with little explanation of the central new ideas behind the arguments or how those ideas build on earlier work in the field. The burden of checking the proofs, identifying those ideas and understanding their implications falls heavily on the mathematical community. AI can generate papers much faster than mathematicians can check and understand them. The resulting imbalance threatens to overwhelm the attention, rigor and conceptual understanding on which mathematical progress depends.
Hugo Duminil-Copin, whose reaction was cited earlier, described OpenAI's release as having “annihilated” his field of research. In his October 14 Numberphile interview, “The Strip Mining of Mathematical Proofs,” he explains what such a disruption means for the mathematicians who have devoted their lives to developing the field.41 He has also warned that young researchers will be especially affected and that the fear of companies using publicly shared ideas to claim results threatens the culture of open exchange.42 The mining metaphor captures the danger of treating mathematical ideas as deposits from which finished results can be extracted, without sustaining the community that develops those ideas. That community does more than supply problems: it cultivates understanding, trains successors and creates new directions. An abundance of extracted results would be a poor exchange for damage to those activities.
Tristan Buckmaster's account is particularly telling. He explicitly credits Diego Córdoba and Luis Martínez-Zoroa with the ideas behind the program that he and Levent Alpöge developed with AI assistance. He also describes pressure to release their results before they had time to turn the generated proofs into readable mathematical arguments. In recounting his exchanges with OpenAI's Sébastien Bubeck, Buckmaster also describes an implicit threat to his career: when he said he would make the circumstances public if OpenAI released its result in the proposed manner, he reports being asked, “Why would you ruin your career?” When he challenged that remark, the response, according to his statement, was: “If you don't want me to be nice, then I don't have to be nice.”43
Whether OpenAI intended these releases to generate publicity, justify its enormous valuations or bolster its claim to be approaching superintelligence, we do not know. But the announcements have been deeply demoralizing, especially for young researchers, some of whom now question whether they have a future in mathematics.
Flooding mathematics with claims faster than they can be assessed, pressuring researchers to release work before it is ready and threatening those who object are abuses of power that the mathematical community must confront together. The mathematical community, and more broadly the entire scientific community, must establish enforceable standards of verification, attribution and open exchange, and make compliance a condition of collaboration with AI companies.
The only acceptable model is one in which AI works under the direction and supervision of the researchers responsible for the work. They set the goals, judge the significance of the results and take responsibility for the work.
But even under this model, serious difficulties remain. First, if providers such as OpenAI or Anthropic can use a researcher's exchanges with AI to train their models, unpublished ideas may find their way into responses given to others. Researchers need firm guarantees that their work will remain confidential and will not be used for training without their explicit consent.
Second, even if confidentiality is secured, the speed with which AI can extend and adapt results threatens to disrupt both mathematical communication and the allocation of academic credit. An idea shared in a lecture or preliminary manuscript could be developed and published by others before its originator has time to carry it through. If credit goes chiefly to whoever first produces a complete proof, mathematicians may become reluctant to share the ideas on which collective progress depends.
Third, collaboration with AI is producing a rapidly growing volume of papers that threatens to overwhelm journals and the peer-review system.44 Generating manuscripts can take far less time than checking their correctness, identifying their original contributions and assessing their significance. The resulting flood risks burying important ideas beneath work that few mathematicians have time to read carefully.
Fourth, AI challenges mathematical training based on written homework, though greater reliance on oral examinations can help address this. The more serious difficulty concerns doctoral students, whose initial research training typically consists of understanding and extending work done by others. Students can now ask AI to carry out much of this work, potentially obtaining results without developing the understanding and independence that the doctoral training is meant to cultivate.
Protecting unpublished work entrusted to AI requires resources and institutional guarantees beyond the reach of the mathematical community alone.
I therefore propose an academic AI consortium bringing together the NSF, American universities, the Simons Foundation and other private institutions.45 Its mission would be to provide powerful AI tools dedicated to scientific research, with enforceable guarantees that unpublished ideas will neither be used for training nor shared with others without the researchers’ explicit consent. The consortium would negotiate protected access to commercial models, operate models itself or support their development. Scientists should not have to choose between access to the best research tools and control over their own ideas. Creating this consortium should be a top priority for the entire scientific community.
The remaining challenges are ours, as mathematicians, to meet: preserving the open exchange of ideas, developing better measures of success that recognize broad intellectual contributions, establishing high ethical standards and effective means of enforcing them, training mathematicians capable of independent thought and finding new ways to teach our beloved subject. None of these tasks will be easy, but I see no reason to doubt that we can meet them. That is why I share Sanderson's optimism about the opportunities awaiting the next generation of mathematicians.
A few comments on the broader impact of AI on society
Public discussion of AI spans a broad spectrum, from Gregory Conti's deep pessimism to Tyler Cowen's optimism tempered by concern and Shyam Sankar's forceful rejection of AI alarmism.46 These disagreements extend beyond the widely exaggerated fears of human extinction to what AI's growing capabilities will mean for human creativity, work and the possibility of a meaningful life.
Cowen and Balwit welcome AI's promise while worrying about our place in a world where machines surpass us. But that worry already concedes the premise I have challenged throughout this essay.
My colleague Gregory Conti fears that AI will destroy the opportunities for meaningful achievement through which people develop their abilities and find a vocation. His concern goes beyond the loss of employment: it concerns the loss of reasons to strive. But his conclusion assumes that machines will exhaust the possibilities for worthwhile human endeavor. The discussion of mathematics above suggests otherwise. A problem solved can reveal connections we had not suspected and bring into view questions we could not previously formulate. AI may further blunt the intellects of the lazy, as Conti rightly fears, but it may also inspire the ambitious and the daring to reach new heights of scientific achievement or artistic expression and deepen our understanding of the human condition.
It is not surprising that academics, in particular, feel threatened. Earlier technological revolutions were mostly experienced by educated professionals as disruptions to other people's jobs, chiefly those requiring less formal education. AI brings that disruption directly into their own world. Activities long regarded as protected by advanced education - writing, programming, research and professional judgment - are increasingly open to automation. Some of the alarm surrounding AI may reflect this reversal: those accustomed to explaining why others must adapt now face that demand themselves.
In many ways, I think that the opposite of what Conti fears may come to pass. If capable robots become plentiful and inexpensive, their combination with AI could give ordinary people access to assistance once reserved for aristocrats. Much of the labor required to maintain everyday life could be delegated to machines. Why assume that the resulting freedom must impoverish our lives? Less compulsory work could mean more time for family, friendship, learning, artistic creation and public service. Leisure does not guarantee a meaningful life, but neither does economic necessity. The opportunity would be extraordinary, provided that access to these benefits is widely shared.
The dangers that deserve our immediate attention concern how people design, deploy and use these systems: to concentrate power, manipulate others, evade responsibility or exercise force. Errors and failures of control also matter, even without malicious intent. None of these dangers requires a machine with ambitions of its own. They require us to take responsibility for the powers we are creating.
Sankar identifies effective altruism as the ideology behind much of the alarm.47 His central objection is that speculative claims about humanity's distant future are being used to override the interests and freedoms of people living today. As he puts it, “Putting disputed moral judgments and uncertain prophecies into an equation does not turn them into scientific facts.” He compares those claiming authority on this basis to Lenin's vanguard: a small group convinced that its privileged understanding of humanity's future entitles it to direct everyone else. Against this ambition, he points to concrete uses of AI in hospitals and shipbuilding that expand what people can accomplish.
One of the most frequently voiced fears concerns the so-called alignment problem: how to ensure that AI acts in accordance with human intentions and values. But human beings themselves disagree profoundly about values, purposes and acceptable conduct. With whose values should AI be aligned, and who gets to decide? We should demand reliable behavior, clear limits and accountability in particular applications. Expecting a machine to resolve humanity's moral disagreements is an absurd ambition.
Ensuring that AI performs its assigned tasks reliably is a practical engineering challenge. The fear that a superintelligent system might escape human control is a different concern, arising from the ambition to build such a system in the first place.
Paradoxically, the main AI companies warn that we may lose control of AI while racing to build machines intended to surpass us in every intellectual capacity. We should instead build powerful AI tools to advance science, transform medicine and strengthen industry. Our task is to enlarge human possibilities, not to chase the fantasy of making ourselves obsolete.