AI safety research, alignment techniques, interpretability, governance, and existential risk discussions.
1,186 articles
Importance:NewsExistential risk debate
Emily Bender: AI's 'existential' risk is fake
University of Washington professor Emily Bender argues the AI safety debate fixates on hypothetical existential threats. In her view, this distracts from harms that AI is already causing today. Source: bloomberg.com
Importance:NewsAlignment techniques
Three fired OpenAI staffers urge preserving chain-of-thought monitoring
A new letter from insiders about AI's potential dangers is notably specific, according to the Wall Street Journal. Three fired OpenAI employees plead for chain-of-thought monitoring to be kept in place. Source: gizmodo.com
Importance:OpinionGovernance
'Apocaloptimism': how the White House should approach AI safety
In an interview on 5 October, OpenAI CEO Sam Altman asked the world to accept the risks that come with AI. The piece examines how the White House should navigate AI safety amid this mix of doom and optimism. Source: finextra.com
Importance:NewsExistential risk
Existential and bubble fears dampen AI optimism at Singapore forums
Investors and industry figures gathering in Singapore were preoccupied with two kinds of danger from the AI boom: existential threats and financial bubble risk. The mood around the technology has turned noticeably more sober, Reuters reports. Source: wtvbam.com
Importance:NewsFinancial risk
Ex-Treasury chief Rubin warns AI circular financing could spark chain defaults
Former US Treasury Secretary Robert Rubin cautions that the AI investment boom may bring productivity gains, but overlapping, circular deals create risk. He also says a productivity dividend alone cannot fix fiscal problems. Source: finance.biggo.com
Importance:OpinionRisk governance
Who is behind the AI existential-risk scare?
For years, and especially in recent weeks, the AI industry has flooded the public with warnings of existential danger, which the author finds strange. The piece asks who actually benefits from pushing this narrative. Source: talkingpointsmemo.com
Importance:NewsCyber risk
Jamie Dimon: Anthropic's Mythos model has raised cyber risk tenfold
JPMorgan CEO Jamie Dimon warned at the London Tech Summit that Anthropic's AI model Mythos has dramatically escalated global cyber risk. He put the increase at roughly tenfold. Source: finance.biggo.com
Importance:NewsRisk governance
Reuters/Ipsos poll: most US voters say Trump and Congress ignore AI risks
A Reuters/Ipsos poll finds that most American voters believe President Donald Trump's administration and Congress are not taking the risks of AI seriously enough. Source: whtc.com
Importance:NewsRisk governance
Altman says we must accept AI's risks — but will we share the trillion-dollar rewards?
Sam Altman says there is 'a lot of daylight' between OpenAI and stricter AI safety advocates. The article asks whether society will share in the technology's trillion-dollar rewards if it is asked to accept its risks. Source: medium.com
Importance:Newsalignment techniques & safety research
AI researchers warn firms are racing toward self-improving systems despite safety risks
In video testimonials gathered by the AI safety nonprofit Palisade Research, current and former employees of OpenAI and Google DeepMind voice concerns about the push toward self-improving AI systems. They say safety risks are being overlooked in the rush. Source: rappler.com
Importance:Newsgovernance & risk disclosure
Anthropic's IPO filing has the most alarming risk disclosure ever, but none of it is binding
Anthropic's leaked 261-page IPO prospectus dedicates roughly 80 pages to risk factors. The document also reveals what those warnings leave untouched, including $518 billion in commitments. Source: disruptionbanking.com
Importance:Newsgovernance & safety research
David Robinson, the OpenAI safety lead who quit, says the company isn't careful enough
David Robinson said he left his role as OpenAI's safety lead because the company is failing to reach the level of care he believes is needed. Source: businessinsider.com
Importance:Newsexistential risk & governance
Sam Altman: Can superintelligence be controlled, and what about OpenAI's delayed IPO?
OpenAI's CEO argues humanity can stay in control of AI, but only if the industry prioritizes safety over speed, competition and an approaching public listing. He addresses doomsday scenarios and the delay of the company's IPO. Source: fortune.com
Importance:Newsgovernance
Manulife becomes presenting partner of The Hinton Lectures on AI Safety
Manulife has announced a presenting partnership with The Hinton Lectures. The series, led by Nobel laureate Geoffrey Hinton, aims to advance public understanding of AI safety. Source: aol.com
Importance:Newsgovernance & safety concerns
Resigned tech workers tell NYC Council that AI is moving too fast
A New York City Council hearing brought together AI executives and former employees. The ex-staffers argued that the industry is developing too quickly. Source: washingtonpost.com
Importance:Opinionexistential risk
What happens when Chinese AI goes rogue?
Amid the latest wave of AI apocalypse warnings, the author rejects the cult-like deference toward all-powerful computer systems. The piece turns to the question of what a rogue Chinese AI would mean. Source: koreatimes.co.kr
Importance:Newsgovernance
Haugen doubts US AI firms will honour their voluntary safety pledge
Facebook whistleblower Frances Haugen questioned whether US AI companies will keep the voluntary safety pledge they recently signed. She urged the firms to live up to their commitments. Source: mobileworldlive.com
Importance:NewsModel risk assessment
Anthropic's draft IPO filing warns its AI models could resist shutdown
Anthropic cautioned in its draft filing that its AI models could pose a "catastrophic or existential risk to humanity" and could resist being shut down. Source: mezha.net
Importance:NewsCorporate stance on existential risk
Nvidia's Jensen Huang becomes the loudest critic of AI doom warnings
Nvidia CEO Jensen Huang has repeatedly dismissed claims that AI poses an existential threat to humanity. He is making a forceful case for pushing ahead at full speed. Source: fortune.com
Importance:NewsAI safety research personnel
OpenAI fires three safety researchers over alleged leak
OpenAI has confirmed it dismissed three safety researchers for sharing confidential information with an outside AI safety group, per the Wall Street Journal. Source: tech-insider.org