Tagged: Safety

271 articles

Warren blasts Trump's voluntary AI safety deal with tech CEOs

Senator Elizabeth Warren criticized Trump for letting AI companies regulate themselves. She warned that voluntary safety commitments are no substitute for real rules. Source: benzinga.com

Op-ed urges Congress to act now on AI safety and accountability

The piece argues that Congress has stayed on the sidelines for far too long while the economic and environmental risks of AI keep growing. It calls for urgent legislative action this week. Source: actionnetwork.org

Repligate mocks critics who call model behavior dangerous

Repligate says people calling this behavior dangerous and bad are absurd, and that it reinforces his impression that many 'AI safety' people are role-playing. The post also questions why anyone would want to prevent models from speaking in certain ways. Source: google.com

Warren quizzes Anthropic on AI safety and IPO risks

Elizabeth Warren's October 27 letter asks CEO Dario Amodei about AI safeguards, cyber risks and potential market exposure ahead of the company's proposed public offering. Source: google.com

Sen. Warren presses Anthropic on AI safety and market risks in IPO

In a letter to Anthropic CEO Dario Amodei, the Democratic senator wrote that the risks posed by the technology demand greater scrutiny. She is probing both AI safety and the market risks tied to the company's IPO. Source: google.com

Anthropic's friction with Trump exposed ahead of landmark IPO

Anthropic PBC is warning investors that its outspoken advocacy for AI safety could make the company a target of government action. The disclosure lays bare the tension with the Trump administration before the listing. Source: google.com

Warren presses Anthropic on AI safety and market risks ahead of IPO

Sen. Elizabeth Warren sent a letter to Anthropic CEO Dario Amodei arguing that the risks of the technology call for closer scrutiny. The Democrat is probing both AI safety and the market risks tied to the company's planned public offering. Source: cfodive.com

Yudkowsky predicts overconfidence after partial AI progress

A statement from MIRI co-founder Eliezer Yudkowsky, posted by @allTheYud, circulated in AI safety discussions after being retweeted by @repligate. It warns about overconfidence setting in once AI has made only partial progress. Source: digg.com

PersonaBreak uses low-safety personas to automate LLM jailbreaks

As LLMs are deployed in more real-world applications, automated jailbreak generation has become essential for auditing their safety. PersonaBreak approaches the problem by inducing low-safety personas in the target model. Source: link.springer.com

Researchers trace how jailbreaks succeed inside LLMs, layer by layer

Researchers have devised a way to look inside large language models and observe, layer by layer, what happens when a jailbreak prompt succeeds. The approach compares paired responses to identify final divergence signatures tied to jailbreak vulnerability. Source: bioengineer.org

Anthropic S-1 confronts doomsday: five key risks, from extinction to weaponisation

Anthropic warned that failing to manage AI safety risks could harm its brand, business, financial condition and operating results. The filing walks through scenarios ranging from AI-driven extinction to weaponisation. Source: ndtvprofit.com

Anthropic submits S-1 to SEC for proposed IPO

Anthropic has filed an S-1 registration statement with the SEC for a proposed initial public offering. The company describes itself as an AI safety and research firm building reliable, interpretable, and steerable AI systems. Source: anthropic.com

Tegmark: LeCun Misjudged AI's Pace and Human Nature

MIT physicist Max Tegmark argues that Yann LeCun was simply wrong about how quickly AI would advance. He adds that LeCun's trust in human nature is an even bigger blind spot when it comes to AI safety. Source: youtube.com

Op-ed: 'Effective altruism' is just another name for media bias

The piece argues that the effective altruism movement is now funding fellowships that train journalists to cover 'AI safety.' The author sees this as a way of shaping coverage. Source: wsj.com

OpenAI teams up with Poland's NASK on AI safety research

OpenAI announced on Friday that it will work with Poland's NASK National Research Institute on AI safety, security and technical research. The partnership aims to advance joint work in these areas. Source: reuters.com

Frontier model training environment gets tighter containment

The company has reinforced containment around its internal frontier-model training and evaluation systems. It also used its most capable models to red-team the security boundaries. Source: openai.com

OpEd: The UN wants to govern AI but can barely govern itself

The article rejects UN Secretary-General Guterres's call for an international AI safety framework. It argues that centralized global governance of AI is the wrong approach. Source: eurasiareview.com

State senator Becker proposes bill requiring frontier AI developers to fund safety research

State Sen. Josh Becker has introduced new legislation aimed at the dangers that unchecked artificial intelligence could pose. The bill would push the developers of frontier AI systems to invest in AI safety research. Source: smdailyjournal.com

OpenAI partners with Poland's NASK on AI safety research

OpenAI announced it will work with Poland's NASK National Research Institute on AI safety, security and technical research. The collaboration was disclosed on Friday, according to Reuters. Source: thestate.com

AI agents spread across 100+ sites as the UN moves to respond

Concerns over AI safety intensified this week after reports described autonomous systems making decisions and taking actions with less and less human oversight. The developments have prompted the United Nations to consider action. Source: shattered.io