AI safety community grows more alarmed — is the fear justified?
A member of the AI safety community describes feeling scared, sad and angry. The remark reflects growing unease in the field over where AI development is heading. Source: google.com
266 articles
A member of the AI safety community describes feeling scared, sad and angry. The remark reflects growing unease in the field over where AI development is heading. Source: google.com
Anthropic started trading on Nasdaq on Thursday, instantly becoming one of the world's most valuable public companies. The debut comes amid concerns about economic disruption and the safety risks of AI. Source: google.com
Models stripped of their safety alignment are thriving on code hosting platforms, undermining the alignment efforts of the original developers. The report comes from The Register's AI and software reporter Thomas Claburn. Source: google.com
A Korean community post says the user Pliny the Prompter created a Claude jailbreak prompt called "godmode". Notably, the prompt reportedly also teaches Claude how to jailbreak other AI agents. Source: piresdoza.osiedleakacjowa.pl
Adam Irwin of Heligan Group outlines the risks investors should consider ahead of Anthropic's IPO. They range from AI safety concerns to military applications and corporate governance. Source: google.com
The rise of broadly capable large language models from frontier AI labs has set off a recursive debate about safety. The piece takes aim at the popular 'p(doom)' probability estimates as a way of framing it. Source: thebreakthrough.org
The piece argues that p(doom), the shorthand estimate of the probability of AI catastrophe, is a poor tool for discussion. It arrives amid a recursive debate over safety triggered by frontier AI companies building broadly applicable large language models. Source: google.com
Helen Toner, who sat on the OpenAI board that fired Sam Altman, argues that AI CEOs still can't be taken at their word. Toner studies AI safety and policy. Source: reddit.com
A skills repository tied to NousResearch's hermes-agent documents March 2026 test results from Pliny the Prompter's auto_jailbreak tool. The tests ran against Claude Sonnet 4 via OpenRouter. Source: tessl.io
OpenAI is working with independent researchers to investigate AI safety risks. The goal is to prevent harmful behavior and improve how models are evaluated. Source: enterpriseai.economictimes.indiatimes.com
Helen Toner, executive director of Georgetown's Center for Security and Emerging Technology, argues that the core AI safety problems, including alignment and control, remain unsolved. She says that is why the public cannot simply rely on the leaders of AI companies. Source: finance.biggo.com
Buck Shlegeris, CEO of the AI alignment nonprofit Redwood Research, estimates roughly a one-in-three chance that AI companies build superintelligence within the coming years. He argues that locking down model weights is not enough to keep China from catching up. Source: finance.biggo.com
Language model (LM) alignment aims to steer model outputs toward human preferences without sacrificing the base model's capabilities. The paper examines the theoretical boundaries of how far this approach can go. Source: machinelearning.apple.com
Language model alignment aims to steer outputs toward human preferences while keeping the base model's capabilities intact. A new paper examines the theoretical boundaries of how far this can go. Source: machinelearning.apple.com
The lawsuit argues that the companies are building increasingly autonomous systems even though they acknowledge the potentially catastrophic risks involved. Source: nationalheraldindia.com
In its new IPO filing, Anthropic says AI is now accelerating AI research itself, to the extent that it trips one of the company's core safety alarms. The disclosure arrives just days before the planned public debut. Source: finance.yahoo.com
We asked ten experts what truly threatens us, and they point to nuclear catastrophe and the fallout from the US–China power struggle, including espionage. A Skynet-style takeover, they say, is not worth panicking over. Source: wired.com
In a 60 Minutes interview, Trump said the DOJ is handling AI safety risks and claimed China has 'no guardrails'. Source: theverge.com
The Democrats' midterm sweep sends five AI safety advocates to Congress. Their arrival signals a possible push for tougher regulation of the tech industry. Source: google.com
JB Pritzker, Rahm Emanuel, Cory Booker and Ro Khanna met with AI safety advocates in San Francisco, according to an exclusive from POLITICO's Decoded. The gathering offers an early look at how potential 2028 Democratic contenders are thinking about AI policy. Source: google.com