🛡️ AI Safety & Alignment

AI safety research, alignment techniques, interpretability, governance, and existential risk discussions.

1,186 articles

Is it already too late to curb catastrophic AI risk?

A new "pre-mortem" from Harvard's Kennedy School maps out the threats, vulnerabilities and possible countermeasures. It warns that AI firms must act now. Source: news.harvard.edu

Andrew Ng fires back at AI leaders over doomsday safety claims

Andrew Ng, the AI pioneer who co-founded Google Brain, has pushed back against apocalyptic warnings that AI could cause human extinction. He accused prominent figures such as Dario Amodei, Sam Altman and Elon Musk of making up safety arguments. Source: timesofindia.indiatimes.com

Musk and Anthropic present rival visions for AI-era philanthropy

Anthropic and Elon Musk have put forward sharply different blueprints for sharing some of the proceeds of the AI boom. The split could shape how the wealth created by AI is distributed. Source: devdiscourse.com

Anthropic pledges 5% of its shares to new public benefit fund

AI lab Anthropic, the developer of Claude, said on Sunday it will commit 5 percent of its shares to a new public benefit fund. Source: theglobeandmail.com

Beyond doomsday: AI's real and immediate psychological risks

Public debate about AI tends to fixate on distant, existential threats, such as superintelligent systems that might one day turn against humanity. The article argues that more immediate psychological risks deserve attention. Source: vocal.media

Why even a remote AI doomsday risk deserves hedging

Warnings of an imminent AI apocalypse may sound far-fetched, but even a remote risk should not be dismissed outright when human extinction is at stake. Source: mmegi.bw

Anthropic earmarks 5% of shares for new public benefit fund

Anthropic announced it will set aside a 5% stake for a public benefit fund. The fund could back initiatives including work to address AI's effects on the labor market. Source: japantimes.co.jp

LeCun challenges Anthropic's AI safety stance: protection or corporate control?

AI pioneer Yann LeCun has pushed back on Anthropic CEO Dario Amodei's approach to AI safety. He warns that sweeping restrictions could end up strengthening corporate control. Source: wionews.com

Anthropic to put 5% of its shares into a 'public good' fund

AI giant Anthropic says it will commit 5% of its shares to a fund. The fund will make donations aimed at the public good. Source: bbc.co.uk

Claude maker Anthropic to commit 5% of shares to public benefit fund

Anthropic, the AI lab behind Claude, says it will commit five per cent of its shares to a new public benefit fund. The snippet is truncated, so further details are not available. Source: kyfreepress.com.au

AI doomsday fears are fueling a new form of digital colonialism, op-ed argues

The piece argues that warnings about existential AI risk distract from harms happening now. It claims they also help justify rules that could entrench monopolies among frontier-model developers. Source: johnmenadue.com

Editorial: AI's risks call for calm, not panic

No set of policies can deliver perfect AI safety, and AI models will inevitably cause more disruption. The editorial argues that this is no reason for panic. Source: dailycamera.com

New York lawmakers move to protect kids from unsafe AI

With no federal action on the table, New York lawmakers are stepping in to shield children from AI-related dangers. The op-ed backs the state's push for protections. Source: nydailynews.com

Anthropic reportedly shopping its own AI safety bill to US lawmakers

According to Politico, Anthropic has quietly handed US lawmakers a draft AI safety bill of its own making. The proposal lays out a plan for regulating the safety and security of artificial intelligence. Source: finance.yahoo.com

AI interpretability breakthrough near? New essay splits researchers

A new essay argues neural networks may be far easier to read than assumed, with roughly 90 percent of neurons serving a clear function. If true, it would be a major development for the field, but researchers are divided on the claim. Source: the-decoder.com

YC Paper Club: how to think about AI safety, with Stanford's Clark Barrett

Clark Barrett, Stanford computer science professor and co-director of the Stanford Center for AI Safety, argues that combining formal verification with AI is the most promising path to AI safety. He discusses the idea in a Y Combinator Startup Podcast episode. Source: finance.biggo.com

Trump: 'Don't kill the Golden Goose' as he calls AI warnings a hoax

Trump dismissed warnings from AI leaders as a hoax amid growing concern over existential risks. Vice President JD Vance described the warnings as a "Trojan horse." Source: abcnews.com

Trump dismisses AI warnings as 'HOAX'; Nasdaq slips 0.9%

Trump labeled AI existential-risk warnings a hoax after Altman and Amodei urged a slower pace of AI development. The Nasdaq reportedly fell 0.9% in the aftermath. Source: tech-insider.org

Anthropic privately pitches lawmakers on its own frontier AI safety bill

Anthropic representatives have reportedly been approaching Capitol Hill lawmakers behind the scenes with a draft bill. The proposal would regulate the safety and security of frontier AI, according to Politico. Source: ca.finance.yahoo.com

Anthropic IPO raises questions about AI risk and retirement savings

Anthropic and the New York City comptroller have opened a dialogue about the 'existential risks' posed by AI. The talks raise the question of what those risks could mean for pension funds and retirement investors if the company goes public. Source: marketwatch.com