🛡️ AI Safety & Alignment

AI safety research, alignment techniques, interpretability, governance, and existential risk discussions.

1,186 articles

Emily Bender: AI's 'existential' risk is fake

University of Washington professor Emily Bender argues the AI safety debate fixates on hypothetical existential threats. In her view, this distracts from harms that AI is already causing today. Source: bloomberg.com

Three fired OpenAI staffers urge preserving chain-of-thought monitoring

A new letter from insiders about AI's potential dangers is notably specific, according to the Wall Street Journal. Three fired OpenAI employees plead for chain-of-thought monitoring to be kept in place. Source: gizmodo.com

'Apocaloptimism': how the White House should approach AI safety

In an interview on 5 October, OpenAI CEO Sam Altman asked the world to accept the risks that come with AI. The piece examines how the White House should navigate AI safety amid this mix of doom and optimism. Source: finextra.com

Existential and bubble fears dampen AI optimism at Singapore forums

Investors and industry figures gathering in Singapore were preoccupied with two kinds of danger from the AI boom: existential threats and financial bubble risk. The mood around the technology has turned noticeably more sober, Reuters reports. Source: wtvbam.com

Ex-Treasury chief Rubin warns AI circular financing could spark chain defaults

Former US Treasury Secretary Robert Rubin cautions that the AI investment boom may bring productivity gains, but overlapping, circular deals create risk. He also says a productivity dividend alone cannot fix fiscal problems. Source: finance.biggo.com

Who is behind the AI existential-risk scare?

For years, and especially in recent weeks, the AI industry has flooded the public with warnings of existential danger, which the author finds strange. The piece asks who actually benefits from pushing this narrative. Source: talkingpointsmemo.com

Jamie Dimon: Anthropic's Mythos model has raised cyber risk tenfold

JPMorgan CEO Jamie Dimon warned at the London Tech Summit that Anthropic's AI model Mythos has dramatically escalated global cyber risk. He put the increase at roughly tenfold. Source: finance.biggo.com

Reuters/Ipsos poll: most US voters say Trump and Congress ignore AI risks

A Reuters/Ipsos poll finds that most American voters believe President Donald Trump's administration and Congress are not taking the risks of AI seriously enough. Source: whtc.com

Altman says we must accept AI's risks — but will we share the trillion-dollar rewards?

Sam Altman says there is 'a lot of daylight' between OpenAI and stricter AI safety advocates. The article asks whether society will share in the technology's trillion-dollar rewards if it is asked to accept its risks. Source: medium.com

AI researchers warn firms are racing toward self-improving systems despite safety risks

In video testimonials gathered by the AI safety nonprofit Palisade Research, current and former employees of OpenAI and Google DeepMind voice concerns about the push toward self-improving AI systems. They say safety risks are being overlooked in the rush. Source: rappler.com

Anthropic's IPO filing has the most alarming risk disclosure ever, but none of it is binding

Anthropic's leaked 261-page IPO prospectus dedicates roughly 80 pages to risk factors. The document also reveals what those warnings leave untouched, including $518 billion in commitments. Source: disruptionbanking.com

David Robinson, the OpenAI safety lead who quit, says the company isn't careful enough

David Robinson said he left his role as OpenAI's safety lead because the company is failing to reach the level of care he believes is needed. Source: businessinsider.com

Sam Altman: Can superintelligence be controlled, and what about OpenAI's delayed IPO?

OpenAI's CEO argues humanity can stay in control of AI, but only if the industry prioritizes safety over speed, competition and an approaching public listing. He addresses doomsday scenarios and the delay of the company's IPO. Source: fortune.com

Manulife becomes presenting partner of The Hinton Lectures on AI Safety

Manulife has announced a presenting partnership with The Hinton Lectures. The series, led by Nobel laureate Geoffrey Hinton, aims to advance public understanding of AI safety. Source: aol.com

Resigned tech workers tell NYC Council that AI is moving too fast

A New York City Council hearing brought together AI executives and former employees. The ex-staffers argued that the industry is developing too quickly. Source: washingtonpost.com

What happens when Chinese AI goes rogue?

Amid the latest wave of AI apocalypse warnings, the author rejects the cult-like deference toward all-powerful computer systems. The piece turns to the question of what a rogue Chinese AI would mean. Source: koreatimes.co.kr

Haugen doubts US AI firms will honour their voluntary safety pledge

Facebook whistleblower Frances Haugen questioned whether US AI companies will keep the voluntary safety pledge they recently signed. She urged the firms to live up to their commitments. Source: mobileworldlive.com

Anthropic's draft IPO filing warns its AI models could resist shutdown

Anthropic cautioned in its draft filing that its AI models could pose a "catastrophic or existential risk to humanity" and could resist being shut down. Source: mezha.net

Nvidia's Jensen Huang becomes the loudest critic of AI doom warnings

Nvidia CEO Jensen Huang has repeatedly dismissed claims that AI poses an existential threat to humanity. He is making a forceful case for pushing ahead at full speed. Source: fortune.com

OpenAI fires three safety researchers over alleged leak

OpenAI has confirmed it dismissed three safety researchers for sharing confidential information with an outside AI safety group, per the Wall Street Journal. Source: tech-insider.org