🛡️ AI Safety & Alignment

AI safety research, alignment techniques, interpretability, governance, and existential risk discussions.

1,186 articles

Anthropic to open its doors to outsiders as rogue-AI fears grow

Anthropic will allow outside parties to look inside the company amid mounting concern about rogue AI systems. Elon Musk says SpaceX will 'probably do the same' as calls for greater transparency increase. Source: uk.finance.yahoo.com

NYC comptroller urges Anthropic to pause its planned IPO

In a letter to CEO Dario Amodei, New York City Comptroller Mark Levine calls on Anthropic to pause its planned public offering. The letter is part of a growing push to scrutinize the company's AI risk disclosures. Source: valueedgeadvisors.com

Pro-AI super PAC falters as safety-focused groups gain traction

Leading the Future, a pro-AI super PAC, is struggling as it faces voters who want AI regulated. Meanwhile, PACs focused on AI safety are gaining ground. Source: transformernews.ai

Essay: humanity's demise is a sales pitch, not a forecast

The author argues that doomsday predictions about humanity's end are being pushed by people with something to sell. What is really on the way, the piece claims, is the most magical age in human history. Source: fortune.com

NYC Comptroller asks Anthropic to hold off on IPO over safety worries

Comptroller Mark Levine has urged Anthropic to pause its plans to go public, pointing to what he calls a myriad of safety concerns around AI. Source: crainsnewyork.com

Levine presses Anthropic to delay IPO and prioritize AI safety

New York City Comptroller Mark Levine is calling on Anthropic to postpone its public offering and devote its energy to AI safety issues instead. Source: pymnts.com

Nvidia's Huang calls for U.S.-China talks on 'potentially dangerous' AI

Jensen Huang told CNBC that Nvidia "absolutely must" work with Chinese AI companies to monitor risks, describing AI as potentially dangerous. Source: cnbc.com

Palantir emerges as a Pentagon AI alternative after CTO rejects safety alarmism

Palantir Technologies drew attention after its CTO publicly criticized extreme AI safety narratives and Silicon Valley's regulatory mindset. Source: simplywall.st

Huang welcomes Pope Leo XIV's 'right questions' on AI dangers

Nvidia CEO Jensen Huang responded to Pope Leo XIV's criticism on AI safety and governance, calling the pontiff's questions necessary. Source: timesofindia.indiatimes.com

Why safety pledges from frontier AI labs won't rescue cybersecurity

This commentary argues that the cybersecurity industry, like the wider world, has become absorbed in the AI safety debate. It questions whether pledges from frontier model developers can actually protect security. Source: scworld.com

Op-ed: AI public offerings should be postponed

AI companies are warning investors that frontier systems could pose catastrophic or even existential risks. The piece argues that governments should impose licensing requirements. Source: johnmenadue.com

Hinton, Bengio and 400 scientists call for global body to contain cyber-capable AI

The scientists want AI models with cyber capabilities kept air-gapped from networks. They also propose an international body that would record every containment failure. Source: technology.org

Kamala Harris urges international AI standards

Harris calls for global AI safety standards and warns that the window to act is closing. She says the US and China should cooperate despite their strategic rivalry. Source: kfiam640.iheart.com

Nvidia puts $250 million of compute behind UK AI safety research

Nvidia has committed $250 million in computing resources over five years to advance AI safety research in the UK. Source: m.economictimes.com

Both camps in the AI safety debate pour hundreds of millions into US midterms

Tech figures on both sides of the AI safety argument are spending heavily ahead of next month's midterm elections. At least $48 million is reportedly headed to politicians seen as likely winners. Source: nypost.com

Nvidia commits $250 million in compute to UK AI safety work

Nvidia will provide $250 million worth of computing resources to Britain over five years to fund AI security research. The pledge strengthens the UK's push to become a hub for AI safety. Source: insurancejournal.com

AI safety is not a corporate plot, argues commentary

Companies like Anthropic and OpenAI have long acknowledged that their products could pose existential risks. The piece argues that these warnings should not be dismissed as mere corporate strategy. Source: astralcodexten.com

El-Sayed releases plan to rein in AI; Rogers hasn't gone that far

Abdul El-Sayed has published a plan to sharply curtail AI development and add safeguards against the technology's worst consequences. His opponent Mike Rogers has not gone nearly as far. Source: mlive.com

El-Sayed unveils AI platform, calls it an 'existential crisis'

Michigan's Dr. Abdul El-Sayed is the first Democrat in a battleground Senate race to publicly call for a pause on artificial intelligence development. Source: nytimes.com

Anthropic tells pre-IPO investors: revenue surging, 'thinking' caps, no human extinction

At a Wednesday meeting with prospective investors, Anthropic presented its third-quarter revenue figures ahead of a possible IPO. The Reuters report also covers limits on model 'thinking' and the company's stance on extinction risk. Source: wkzo.com