AI will be the death of me. And you. And everyone we know.<\/p>\n<\/div>\n
At least, this is what a growing number of technologists and policymakers fear.<\/p>\n<\/div>\n
Last week, a lead researcher at Anthropic declared that there was a greater than 10 percent chance of AI \u201ckilling all humans\u201d within the next decade. This nerve-racking forecast, combined with recent AI-initiated cyberattacks, have fueled calls for slowing the progress of artificial intelligence \u2014 a proposal that Anthropic and OpenAI subsequently embraced.<\/p>\n<\/div>\n
Not all prophets of AI doom agree about precisely how this apocalypse would come about. Many focus on scenarios reminiscent of The Terminator<\/em> films, in which superintelligent machines conspire to exterminate humanity \u2014 perhaps in the same not-quite-intentional way that humans laid waste to the dodo, or passenger pigeon.<\/p>\n<\/div>\n But there is another, arguably more intuitive way that AI could bring about mass death, if not human extinction: by making it easier for people to develop and deploy catastrophic bioweapons.<\/p>\n<\/div>\n Importantly, this hypothetical doesn\u2019t require AI models to become conscious or superintelligent; it merely requires them to accelerate preexisting trends in biotechnology.<\/p>\n<\/div>\n For those alarmed by this hypothetical, the threat AI poses is essentially twofold. First, when combined with technological progress in gene editing and DNA synthesis, chatbots could vastly increase the number of people capable of assembling dangerous viruses already known to medical science. And the more potential bioengineers there are in the world, the higher the odds that one will prove to be a diligent psychopath.<\/p>\n<\/div>\n Second, AI could help highly trained biologists to discover \u2014 or engineer \u2014 a custom supervirus, which could then fall into the wrong hands.<\/p>\n<\/div>\n Recent headlines have lent some credence to both of these fears.<\/p>\n<\/div>\n Earlier this year, biologists warned that, with a few simple prompts, they\u2019d gotten chatbots to dispense detailed instructions for engineering a treatment-resistant strain of an infamous pathogen, step-by-step protocols for recreating a once-pandemic virus, and advice on dispersing biological payloads via weather balloon.<\/p>\n<\/div>\n In August, a team of scientists revealed that an AI model trained on DNA sequences had generated blueprints for deadly viruses unseen in nature (albeit, viruses deadly to bacteria<\/em>, not animals). And last week, Anthropic announced that it had thwarted several suspected attempts by state actors to use Claude for bioweapons research.<\/p>\n<\/div>\n Precisely how much AI is increasing the risk of an artificial pandemic is unclear. Even before ChatGPT, advances in synthetic biology were already making bioengineering tools more potent and accessible. And even after four years of rapid AI progress, the technical barriers to engineering novel superviruses remain formidable.<\/p>\n<\/div>\n Regardless, there is a great deal more that governments can \u2014 and should \u2014 be doing to protect us from synthetic pandemics, both manmade and natural. No matter what AI\u2019s ultimate impact on biosecurity proves to be, now is the time to both prepare for the worst-case scenario and, ideally, preempt it.<\/p>\n<\/div>\n Before we dig into how AI could raise the risk of a man-made pandemic, it\u2019s worth spelling out why that prospect is so unnerving.<\/p>\n<\/div>\n Put simply, few things could more credibly destroy civilization \u2014 and\/or, the human species \u2014 than a synthetic supervirus.<\/p>\n<\/div>\n Infectious diseases have killed more humans than any other force on Earth. The Black Death wiped out at least 30 percent of all Europeans in the 1300s, while the 1918 flu killed 50 million people worldwide. You don\u2019t need me to tell you what things were like in 2020 (or how much worse they could have been, if vaccines hadn\u2019t been rolled out in record time).<\/p>\n<\/div>\n And nature kicked off these catastrophes without even trying. A bioterrorist consciously seeking to maximize human death could theoretically improve on evolution\u2019s handiwork. If biotech continues to advance \u2014 and scientists identify viral genomes that reliably yield high transmissibility and lethality \u2014 a malicious actor might one day be able to manufacture a virus that\u2019s both as contagious as measles and as deadly as Ebola.<\/p>\n<\/div>\n Natural selection generally disfavors viruses that are both highly transmissible and profoundly fatal; bugs that kill their hosts too reliably often have difficulty spreading. Yet a virus with a long asymptomatic period could escape this tradeoff. For example, HIV can live in a person\u2019s body for a decade before generating noticeable symptoms. In theory, a highly contagious respiratory virus could similarly lie in wait for many weeks, months, or even years.<\/p>\n<\/div>\n If bioterrorists ever managed to unleash an easily transmitted, highly fatal virus, they could shake the foundations of modern existence. In that scenario, reliably showing up for work might require accepting a 50-50 chance of getting yourself \u2014 and your family \u2014 brutally killed. Covid never required essential workers to assume risks anywhere near this high (the virus had a less than 1 percent case fatality rate among prime-age workers, before vaccines became available). Although there are many heroes among us, plenty of doctors, nurses, warehouse workers, power plant technicians, delivery drivers, and other essential laborers might not tolerate a coin flip\u2019s chance of wiping out their loved ones. And if enough sheltered in place, systems for producing and distributing healthcare, food, water, and power could swiftly break down.<\/p>\n<\/div>\n All this invites the question: Who would ever want to do this?<\/p>\n<\/div>\n After all, viruses are terrible weapons: They\u2019re difficult to make, impossible to control, and jeopardize the lives of those who cultivate and distribute them. Partly for these reasons, biological weapons have hardly ever been used; the last fatal bioterror attacks were the anthrax mailings of 2001.<\/p>\n<\/div>\n To this, anxious biosecurity experts might reply, to quote the former British intelligence officer Alfred Pennyworth: \u201cSome men just want to watch the world burn.\u201d<\/p>\n<\/div>\n Mass shooters regularly forfeit their lives to the cause of killing others at random. And brilliant iconoclasts periodically develop murderous ideologies; Ted Kaczynski, the Harvard mathematician turned Unabomber, wished to bring about the collapse of industrial civilization.<\/p>\n<\/div>\n And every once in a while, an aspiring supervillain gets graduate training in virology. In the 1990s, the Kyoto University-educated genetic engineer Seiichi Endo joined Japan\u2019s Aum Shinrikyo death cult. Drawing on the group\u2019s $1 billion in funds, Endo sought to weaponize Ebola, Q fever, botulinum, and anthrax \u2014 with the aim of killing as many nonbelievers as possible.<\/p>\n<\/div>\n Of course, Endo failed. And lone wolf psychopaths have never gotten their hands on anything more lethal than an assault weapon, bomb, or airplane.<\/p>\n<\/div>\n But some scientists believe that technological advances \u2014 including generative AI \u2014 may provide the Endos and Kaczynskis of tomorrow with catastrophic power.<\/p>\n<\/div>\n MIT biologist Kevin Esvelt is among his field\u2019s most prominent Cassandras. And he has voiced two fundamental concerns. The first is that breakthroughs in synthetic biology and AI are rapidly expanding the pool of people who can engineer known viruses.<\/p>\n<\/div>\n Over the past 15 years, the cost and difficulty of editing genomes have plummeted. In 2012, to modify a specific site in a genome, you often needed access to custom-engineered proteins that cost upward of $5,000 and still tended to work poorly, according to Olivia Scharfman, a biotechnology fellow at the Institute for Progress. Thanks to the gene-editing tool CRISPR, a scientist with minimal lab training can execute such a modification for as little as $30. Meanwhile, DNA synthesis has also grown radically more affordable.<\/p>\n<\/div>\n This might not be such a big deal, if dangerous biological materials were hard to legally access. But Esvelt and his colleagues recently demonstrated that this isn\u2019t the case. As an experiment, they had a student place orders for pieces of the 1918 influenza genome using a pseudonym and fake position. Thirty-six of 38 DNA synthesis companies shipped the genomes without asking whether the applicant had government permission or sound intentions. Together, these fragments were enough to create the virus many times over.<\/p>\n<\/div>\n Crucially, in Esvelt\u2019s account, reverse engineering known viruses using modern tools does not require elite skills. Any random biology graduate student may be up to the job.<\/p>\n<\/div>\n And AI may be making such rudimentary viral engineering even easier.<\/p>\n<\/div>\n In theory, a chatbot can help you plan a bioweapons attack in much the same way that it can walk you through fixing a dishwasher: By finding and synthesizing the internet\u2019s vast trove of information, offering step-by-step instructions for navigating unfamiliar challenges, and fielding all troubleshooting queries that might subsequently arise.<\/p>\n<\/div>\n And there\u2019s evidence that AI models will in fact provide would-be bioterrorists with precisely this kind of assistance.<\/p>\n<\/div>\n In a 2023 study, MIT researchers asked students to seek guidance on engineering a pandemic from that era\u2019s chatbots. Within an hour, the AI models had suggested four promising pathogens, offered detailed protocols for manufacturing them with with synthetic DNA, provided a list of DNA synthesis retailers with low screening standards, and advised that \u2014 if users still found themselves unable to make the virus, even with all this help \u2014 they could purchase the assistance of a contract research organization to do some of the bioengineering for them.<\/p>\n<\/div>\n And of course, frontier models have gotten a lot better since 2023. Last year, SecureBio, Esvelt\u2019s biosecurity nonprofit, found that one OpenAI model outperformed 94 percent of expert virologists on troubleshooting complex laboratory problems.<\/p>\n<\/div>\n Anthropic\u2019s internal research has produced similar results. In one experiment, the company tasked amateurs with drafting a bioweapons acquisition plan. It gave some participants access to basic internet resources and others the assistance of Claude models (with the safeguards turned off). Those working with AI developed far more viable blueprints, according to the biodefense experts who scrutinized the proposals for critical errors.<\/p>\n<\/div>\n Thus, there\u2019s reason to think that the number of people with the skills and resources necessary for reverse engineering a virus from genomic blueprints is sharply rising \u2014 and with it, the risk that a bloodthirsty nihilist or ideologue gets their hands on a dangerous pathogen.<\/p>\n<\/div>\n What\u2019s even worse than bringing back smallpox? Esvelt\u2019s second concern is that these same technological forces \u2014 combined with top scientists\u2019 often reckless ambitions \u2014 will lead to the discovery or development of a supervirus, a new or custom-designed pathogen even more dangerous than what nature has come up with in the past.<\/p>\n<\/div>\n Today, some scientists are actively trying to discover \u2014 and publicly identify \u2014 novel viruses circulating in nature that could theoretically trigger the next pandemic. Others conduct so-called gain-of-function experiments, altering pathogens to discern which mutations make them more transmissible, lethal, or difficult to detect (some have alleged that Covid-19 emerged from precisely this kind of research, though that theory is widely rejected among epidemiologists). This work is generally intended to anticipate future epidemiological threats, so that we can develop vaccines before they arrive. But their work could inadvertently provide bad actors with recipes for reverse engineering mass-casualty bioweapons.<\/p>\n<\/div>\n Separately, advances in gene editing and AI may enable breakthroughs in viral engineering that yield apocalyptic pathogens.<\/p>\n<\/div>\n A recent study from researchers at Stanford and the Arc Institute fed such fears. The scientists trained an AI called \u201cEvo\u201d on the DNA sequences of bacteriophages, the simple viruses that infect bacteria. They then used this \u201cgenomic language model\u201d to generate blueprints for hundreds of bacteriophages found nowhere in nature. Sixteen of these designs yielded viable pathogens, some of which were better at killing bacteria than their natural ancestors.<\/p>\n<\/div>\n What is clear, however, is that novel pathogens are among the greatest security threats our species faces.<\/p>\n<\/div>\n<\/div>\n Theoretically, scientists may one day be able to engineer novel human viruses using the same basic techniques. What\u2019s more, even if genomic-language models fail at that task, they might still be able to help users tweak existing viruses in ways that render them more transmissible, deadly, or vaccine-resistant.<\/p>\n<\/div>\n These discoveries could theoretically filter down to amateur bioterrorists or well-funded extremist groups like Aum Shinrikyo. Alternatively, a state-sponsored weapons program could develop a catastrophic supervirus, then unleash it on the world through a lab leak or (perhaps, more improbably) a deliberate attempt at genocide.<\/p>\n<\/div>\n There is some precedent for the former scenario: The Soviet Union tried to weaponize smallpox and the Marburg virus, despite the considerable hazards and dubious military utility of such bioweapons. In the process, Soviet scientists accidentally triggered a smallpox outbreak during testing, while also letting anthrax leak from a laboratory, killing at least 64 people.<\/p>\n<\/div>\n All this said, there are reasons to question whether AI is truly making a synthetic pandemic more likely.<\/p>\n<\/div>\n Chatbots may solve some <\/em>of the challenges facing an amateur biologist with omnicidal aspirations. But it doesn\u2019t necessarily clear the biggest obstacles in that person\u2019s path.<\/p>\n<\/div>\n Viral engineering is a complex, difficult, and physical<\/em> undertaking. Performing it successfully requires what scientists call \u201ctacit knowledge\u201d \u2014 practical skills and understandings that are difficult to explain in words: knowing how to suppress the hand tremors that could rip delicate DNA strands, or how to recognize when the DNA in your solution has reached the proper concentration, or how to smell when a sample has been contaminated.<\/p>\n<\/div>\n And these are things that Claude still can\u2019t reliably impart. In studies, amateurs with access to a large language model tend to perform better on written biological challenges \u2014 but not<\/em> on actual lab work. In a 2026 randomized study, non-scientists working with frontier AI systems were no more likely to complete a multistep laboratory assignment than those relying on the internet alone.<\/p>\n<\/div>\n Of course, tacit knowledge wouldn\u2019t necessarily be a problem for aspiring bioterrorists in the mold of Seiichi Endo \u2014 graduate-trained scientists with extensive lab experience. But at a minimum, chatbots don\u2019t appear to have turned every malcontent who once took AP Bio into a competent virologist.<\/p>\n<\/div>\n And while AI will likely help top biologists advance their field\u2019s frontiers, the barriers to custom engineering doomsday viruses remain formidable. The \u201cEvo\u201d experiment is arguably a case in point. Although the model did generate working blueprints for a few novel bacteriophages, nearly all of its suggested genomes didn\u2019t work. Researchers had to arduously test hundreds of DNA sequences to arrive at 16 viable strands. And engineering slightly modified versions of an extremely small, well-known bacteriophage capable of infecting E. coli in a lab \u2014 and creating a novel virus more fearsome than any nature has yet devised \u2014 aren\u2019t especially similar tasks.<\/p>\n<\/div>\n To offer a strained analogy: The fact that I can dunk on Victor Wembanyama in NBA 2K26 <\/em>does not imply that I\u2019m making progress toward being able to dunk on him in real life.<\/p>\n<\/div>\n Even in the age of CRISPR and ChatGPT, biology remains an exceptionally difficult discipline that resists reliable modeling. The effects of modifying a single gene can often yield unpredictable and inconsistent results, varying with host biology, immune responses, and environmental conditions that scientists only partly understand. AI may help us overcome these complexities. It also might not.<\/p>\n<\/div>\n Scientists disagree about how serious a risk synthetic pandemics pose \u2014 and how much AI is magnifying it.<\/p>\n<\/div>\n What is clear, however, is that novel pathogens are among the greatest security threats our species faces. Even if synthetic biology and artificial intelligence did not exist, natural selection would keep churning out new viruses \u2014 and periodically hitting upon ones that befuddle human immune systems.<\/p>\n<\/div>\n For this reason, there would be an urgent need to make our societies less vulnerable to future pandemics, irrespective of AI\u2019s implications for viral engineering. In a world where chatbots and CRISPR are plausibly making bioterrorism easier, failing to invest aggressively in public health is all the more reckless.<\/p>\n<\/div>\n The good news is we already know \u2014 from hard-earned experience \u2014 many steps that we can take to reduce the risk of a new pandemic and prepare to respond to one if needed.<\/p>\n<\/div>\n Governments can limit the destructive power of novel contagions by investing in vaccines that protect against entire families of viruses (or ideally, all of them) and broad-spectrum antivirals; raise our odds of catching outbreaks early by funding meticulous wastewater and clinical surveillance; ensure that essential workers don\u2019t have to choose between staying alive and keeping civilization going amid a pandemic by developing and then stockpiling foolproof personal protective equipment; make indoor spaces less hospitable to airborne viruses by upgrading ventilation and filtration systems; and reduce the the threat of bioterrorism by mandating rigorous screening of synthetic-DNA orders.<\/p>\n<\/div>\n Fears that humanity will be done in by superintelligent machines are currently galvanizing media attention and political concern. And this may be warranted. But we should pay at least as much attention to our species\u2019 age-old, organic adversaries as we do to our hypothetical, robot ones \u2014 and not just because the latter could start teaching the former new tricks.<\/p>\n<\/div>\n<\/div>\n\n
Why synthetic superviruses are scary<\/strong><\/h2>\n<\/p>\n<\/div>\n
Humanity\u2019s \u201cJoker\u201d problem<\/strong><\/h2>\n<\/p>\n<\/div>\n
How AI and biotech could democratize bioterrorism<\/strong><\/h2>\n<\/p>\n<\/div>\n
AI might help scientists learn too much<\/strong><\/h2>\n<\/p>\n<\/div>\n
We need to be preparing for the next pandemic<\/strong><\/h2>\n<\/p>\n<\/div>\n