🛡️ AI Safety & Alignment

AI safety research, alignment techniques, interpretability, governance, and existential risk discussions.

1,186 articles

Advocacy groups sue AI leaders to freeze new models, citing 'catastrophic' risks

A coalition of online safety and consumer advocacy groups sued five of the world's top AI companies on Thursday. They want the courts to stop further development of new models, warning of catastrophic risks. Source: miamitimesonline.com

AI safety groups sue five tech giants, demand halt to new model development

The lawsuit argues that the companies are building increasingly autonomous systems even though they acknowledge the potentially catastrophic risks involved. Source: nationalheraldindia.com

Anthropic says its RSP trigger has been met: what the ASL levels mean

Anthropic states that the trigger in its Responsible Scaling Policy has been reached. The piece explains what the ASL threshold means, which safeguards apply, and how this affects Claude. Source: shattered.io

'An existential threat': Pioneer Press workers fight for AI protections

Unionized workers at St. Paul's Pioneer Press and Adtaxi want AI provisions written into their next union contract. They are also appealing to readers for support. Source: workdaymagazine.org

Anthropic's IPO filing flags a key AI safety alarm as AI speeds up its own research

In its new IPO filing, Anthropic says AI is now accelerating AI research itself, to the extent that it trips one of the company's core safety alarms. The disclosure arrives just days before the planned public debut. Source: finance.yahoo.com

OpenAI board member Christiano: an AI accident is the likeliest way most of us will die

OpenAI board member Paul Christiano told an audience he believes that for most people in the room, the most probable cause of death is an AI accident. The remark underscores how seriously some insiders take catastrophic AI risk. Source: newsweek.com

Gates: Trump administration's inaction leaves world exposed to AI-enabled pandemic risk

Bill Gates has accused the Trump administration of obstructing efforts to protect against the dangers of AI. He warns the technology could be misused to create pandemic threats. Source: app.dealroom.co

What to watch as Anthropic prepares for its public debut

Today's newsletter covers the arrival of Anthropic's IPO price, plus news involving Airbnb, Alta, CoreWeave, DoorDash, Google, Nvidia, Oura, OpenAI and Uber. Source: fortune.com

Anthropic Sets IPO Range Implying Up to $1.4 Trillion Valuation

Anthropic has set a price range of $70 to $78 per share for its IPO, which would value the company at more than $1.4 trillion at the top end. The company also says AI may pose existential risks to humanity. Source: deadline.com

OpenAI Reportedly Weighs Slowing Down as AI Pace Worries Grow

Concerns over how fast AI is advancing are mounting, and OpenAI is reportedly considering slowing its development. CEO Sam Altman said he hopes other AI companies will follow suit. Source: ibtimes.com

Experts Say the AI Safety Debate Is Off Track and Name the Real Dangers

We asked ten experts what truly threatens us, and they point to nuclear catastrophe and the fallout from the US–China power struggle, including espionage. A Skynet-style takeover, they say, is not worth panicking over. Source: wired.com

Queens Lawmaker Says Regulating AI in New York Goes Beyond Existential Threats

AI and its growing influence on everyday life have become a major issue for New Yorkers. A Queens state senator argues that rules for the technology must address more than just the existential threats. Source: queenseagle.com

Anthropic's IPO Filing Does Warn of 'Catastrophic' AI Risk, in Its Risk Factors

Anthropic's prospectus states that advanced AI could pose 'catastrophic or existential risks to humanity.' The language appears in the risk-factors section, which is a standard part of IPO filings. Source: nacionale.com

Anthropic's IPO filing warns AI could pose 'existential risks to humanity'

Anthropic plans to tell prospective IPO investors that advanced AI could bring 'catastrophic or existential risks to humanity.' It is an unusually stark warning for a company preparing to go public. Source: google.com

Anthropic: rogue AI could pose 'catastrophic, existential risks to humanity'

Anthropic says that if AI goes wrong, it could become 'a global security threat of the first order.' The company is framing the dangers of runaway AI in unusually blunt terms. Source: google.com

Pelosi's successor Scott Wiener is ready to take on AI regulation

As a California state senator, Scott Wiener wrote legislation aimed at curbing the risks of advanced AI. Having now won a tight congressional race, he is poised to carry that agenda to Washington. Source: google.com

Five Democrats set to reshape the AI safety debate in Washington

The Democrats' midterm sweep sends five AI safety advocates to Congress. Their arrival signals a possible push for tougher regulation of the tech industry. Source: google.com

Democratic 2028 hopefuls huddle with AI safety advocates in San Francisco

JB Pritzker, Rahm Emanuel, Cory Booker and Ro Khanna met with AI safety advocates in San Francisco, according to an exclusive from POLITICO's Decoded. The gathering offers an early look at how potential 2028 Democratic contenders are thinking about AI policy. Source: google.com

Europe urged to spearhead a global pact on reporting AI incidents

A proposed 'common floor' for sharing information about AI failures would strengthen safety and help reduce major AI risks. The argument is that Europe is well placed to lead such an international compact. Source: google.com

Trump hints at openness to AI guardrails after Democratic 'blue tsunami'

The softer tone on AI comes after the midterm elections and as Nvidia deals with reports tying CEO Jensen Huang to an alleged Chinese spy. The shift suggests the political climate around AI regulation is changing. Source: google.com