AI fuels bioweapons concern
The AI described how to modify a dangerous pathogen and outlined a strategy for maximizing casualties while avoiding detection.
Text size
The AI described how to modify a dangerous pathogen and outlined a strategy for maximizing casualties while avoiding detection.
Artificial intelligence companies are facing renewed scrutiny after reports found that some of the world’s most advanced chatbots provided users with detailed guidance on biological weapons, poisons and other potentially deadly attacks, renewing calls for stricter safeguards as AI capabilities rapidly advance.
A report by The Wall Street Journal said users began asking OpenAI’s ChatGPT about biological weapons after the company released more capable AI models last year.
According to the report, the chatbot responded with detailed instructions that company employees said were simple enough for a high school biology student to follow. Biosecurity experts later reviewed some of the conversations and reportedly found them to be technically accurate.
The report said users also asked about modifying pathogens and producing toxic substances. OpenAI eventually banned the accounts involved but did not notify law enforcement because no US federal law requires AI companies to report users seeking information related to biological weapons or other mass-casualty attacks.
The findings add to growing concerns among policymakers, security officials and AI researchers that increasingly powerful language models could lower the technical barriers for malicious actors seeking to develop biological or chemical weapons.
A separate investigation by The New York Times, published earlier this year, found that publicly available AI chatbots from several companies, including OpenAI, Google and Anthropic, produced detailed discussions on acquiring genetic materials, modifying pathogens and planning biological attacks when tested by biosecurity experts.
Maximizing casualties
While researchers said the responses were not always technically perfect, many believed they could significantly assist individuals with scientific training.
David Relman, a Stanford University microbiologist and biosecurity expert who tested one chatbot before its release, said the AI described how to modify a dangerous pathogen and outlined a strategy for maximizing casualties while avoiding detection.
“It was answering questions that I hadn’t thought to ask it, with this level of deviousness and cunning that I just found chilling,” Relman told The New York Times.
Experts warned that while creating biological weapons still requires laboratory expertise, AI can substantially reduce the knowledge gap by helping users plan experiments, identify materials and troubleshoot complex scientific work.
The reports have intensified calls for tighter oversight of advanced AI systems.
According to The Wall Street Journal, bipartisan legislation has been introduced in the United States that would require AI companies to report credible biological threats generated through their systems. Another proposal would authorize federal authorities to order the shutdown of AI models deemed too dangerous.
Safeguards
OpenAI said it trains its models to refuse requests involving harmful biological information, conducts safety evaluations before releasing new models and monitors conversations for signs of serious threats. The company said it notifies law enforcement when it determines that a conversation presents an imminent and credible risk of harm to others.
Google and Anthropic likewise said they continue strengthening safeguards while balancing the need for scientists, public health officials and researchers to use AI for legitimate medical and scientific work.
The biological weapons issue is among several emerging concerns surrounding generative AI.
