🛡️ AI Safety & Alignment

AI safety research, alignment techniques, interpretability, governance, and existential risk discussions.

1,186 articles

Machine learning boosts genome-wide studies but interpretability remains a hurdle

A comprehensive review covers 30 AI approaches applied to genome-wide association studies. It highlights their strengths in modeling while pointing to major interpretability challenges. Source: bioengineer.org

A close read of Anthropic's I.P.O. filing and its frank talk of existential risk

Anthropic's prospectus stands out for how openly it discusses existential risk. The open question is whether that candor is actually reassuring. Source: nytimes.com

Securing model weights alone won't stop China, says Redwood Research CEO

Buck Shlegeris, CEO of the AI alignment nonprofit Redwood Research, estimates roughly a one-in-three chance that AI companies build superintelligence within the coming years. He argues that locking down model weights is not enough to keep China from catching up. Source: finance.biggo.com

OpenAI Foundation board member warns AI accidents could become a leading cause of death

Paul Christiano, a board member of the OpenAI Foundation, warned at a Geoffrey Hinton lecture in Toronto that AI accidents could soon become a leading cause of death for humanity. Source: finance.biggo.com

Former Anthropic researcher quits and warns AI could destroy humanity

A former Anthropic researcher has left one of the world's leading AI companies. He is now raising the alarm about where artificial intelligence is headed. Source: media.unisba.ac.id

Anthropic safety researchers sound the alarm that AI could pose a lethal threat

As AI grows more capable, some of the people building it are now warning about the dangers. One Anthropic researcher voices serious concern about where the technology is heading. Source: media.unisba.ac.id

Ex-OpenAI board member who ousted Sam Altman: we still can't trust AI CEOs

Helen Toner, executive director of Georgetown's Center for Security and Emerging Technology, argues that the core AI safety problems, including alignment and control, remain unsolved. She says that is why the public cannot simply rely on the leaders of AI companies. Source: finance.biggo.com

Nathan Labenz at Bletchley Park: AI agents are now capable enough to be dangerous

Nathan Labenz, host of The Cognitive Revolution and a GPT-4 red-teamer, told IT Revolution's Enterprise AI Summit that AI agents are now capable enough to be genuinely dangerous. He delivered the keynote at Bletchley Park. Source: finance.biggo.com

Advocacy groups sue five AI firms to halt new model development

A coalition of online safety and consumer advocacy groups filed suit in New York on Thursday against five of the world's leading AI companies. The plaintiffs want development of new AI models stopped, citing major risks. Source: arkansasonline.com

Dwarkesh Patel: America's AI lead over China is hours, not months

Patel argues that recursive self-improvement in AI could trigger an intelligence explosion and yield billions of human-level AIs by 2030. He says the US labs' edge over China amounts to only months, or even less. Source: finance.biggo.com

How America could sabotage its own AI lead

A mix of political, commercial and ideological pressures threatens to erode America's lead in AI. The piece argues that public controversies are strong enough to derail the country's leadership in the field. Source: cset.georgetown.edu

Geoffrey Hinton warns of existential AI threats as superintelligence nears

Nobel laureate Geoffrey Hinton, often called the godfather of AI, cautions that the race toward superintelligence poses existential risks. In the conversation he also discusses Nvidia's Jensen Huang. Source: thirteen.org

Study finds sparse attention heads that encode goals inside LLMs

Northeastern University researchers report that a small set of sparse attention heads in large language models causally encodes the model's internal task goals. The preprint adds to interpretability work on how LLMs represent what they are trying to do. Source: casrai.org

Opinion: Voters took on data centers — now it's time to govern AI

Data centers were not formally on this year's ballots, yet candidates critical of such projects won by wide margins. The op-ed argues this voter signal should now push policymakers toward governing AI itself. Source: startribune.com

Anthropic's S-1 filing: risks to humanity and other warnings

A closer look at three sets of warnings buried in the roughly 100-page risk section of Anthropic's IPO filing. They are worth paying attention to. Source: morningstar.com

Hinton: humans could lose control of AI within years

Geoffrey Hinton, the 'godfather of AI', warns that superintelligent AI could escape human control. He urges governments to introduce mandatory safety testing. Source: t.co

Nonprofits sue OpenAI, Anthropic, Meta and Google over AI risks

A coalition of nonprofit organisations has filed a lawsuit against OpenAI, Anthropic, Meta Platforms, Google parent Alphabet and SpaceX's AI unit. The suit targets the risks posed by advanced AI systems. Source: adgully.com

Ex-Anthropic researcher calls for slower AI development

Jacob Coxon, a former OpenAI and Anthropic researcher who resigned over concerns about the technology's risks, is urging greater awareness of AI's dangers. He argues the pace of development should slow down. Source: stanforddaily.com

Nonprofits sue five AI giants, including Anthropic and Google, to freeze the AI race

A lawsuit filed in San Francisco targets five tech companies and asks a court to block the development of more advanced and self-improving AI models. The plaintiffs are nonprofit groups seeking to halt the escalating race among leading AI developers. Source: politico.com

US nonprofits take tech giants to court to stop AI development

A coalition of digital safety and consumer watchdog groups has filed a lawsuit against the world's leading AI developers. The suit asks the court to halt further development of the technology. Source: aljazeera.com