📢 New paper: Prioritization of Risks from Artificial Intelligence: A Delphi Study of 272 International Experts AI creates many risks, from discrimination, privacy loss, and fraud to more emerging concerns such as overreliance, dangerous capabilities being misused in weapons or cyberattacks, and AI systems pursuing unintended goals. But which risks are most severe? Who is most vulnerable? And who is most responsible for addressing them? To answer these questions, we conducted a three-round expert consultation with 272 AI experts. 💡 Four insights from our findings: 1️⃣ If things continue as they are over the next 5 years, experts assigned ≥10% probability of catastrophic outcomes (e.g., >1 million deaths or >$100 billion in losses) to 18 of 24 risks. Top concerns: cyberattacks and weapons, dangerous AI capabilities, competitive dynamics, power centralization, and disinformation and influence at scale. 2️⃣ Even assuming pragmatic mitigations, 5 risks remained above the 10% catastrophic threshold: dangerous AI capabilities, cyberattacks and weapons, environmental harm, inequality, and power centralization. 3️⃣ Vulnerability is broadly distributed, but responsibility is concentrated. Experts assigned the highest vulnerability to AI users and the general public, while assigning primary responsibility for mitigation to frontier AI developers, governments, regulators, and standards bodies. 4️⃣ Information, finance, and national security were rated the sectors most vulnerable to AI risks. 🔗How can you engage? See our (fancy) new webpage for our interactive summaries of the findings and preprint, and please share with anyone working on AI risk, governance, or policy. airisk.mit.edu/priorities This research is part of the MIT AI Risk Initiative (@MITAIRisk), which aims to help society understand, prioritize, and manage risks from AI. The initiative includes the MIT AI Risk Repository, a living database of more than 1,700 AI risks, the AI Incident Tracker, a collaboration with the Responsible AI Collaborative, which connects risks to over 1,400 incidents, and the MIT AI Governance Map, which analyzes risk coverage across more than 1,000 laws, standards, policies, and other governance documents curated by the Center for Security and Emerging Technology (CSET). #AI #AIrisk #AISafety #AIGovernance #ResponsibleAI #RiskManagement

Great to see the MIT AI incident tracker, led by Simon Mylius, featured in @TIME: time.com/7346091/ai-harm-ris… The AI Incident Tracker maps incidents in the AI Incident Database according to the MIT AI Risk Repository’s causal and domain taxonomies, and assigns each incident a harm-severity score. Using an LLM, it processes raw incident reports, providing a scalable methodology that can be applied cost-effectively across much larger datasets as numbers of reported incidents grow. In the dashboard you can explore trends such as: - distribution of incident classifications by year - distribution of incident sub-domains by year - incidents with high direct harm severity scores by year - incidents causing severe harm in more than one harm category - distribution of harm severity scores by year Our last update added new evaluation fields for each incident, including: - 5 categories of NatSec impact: Physical Security & Critical Infrastructure / Information Warfare & Intelligence Security / Sovereignty & Government Functions / Economic & Technological Security / Societal Stability & Human Rights - A Fishbone/Ishikawa diagram that presents a number of potential causes for each incident - The primary goal of the AI system involved Visit our website to explore the data. Congratulations also to Daniel Atherton and the AI Incident Database (@IncidentsDB) for the coverage. We are lucky to be able to build on their critical work. Thanks to Harry Booth for the write-up (@HarryBooth59643)

🧵 📢 What are the risks from Artificial Intelligence? We present the first-ever AI Risk Repository: a comprehensive living database of 700+ risks extracted, with quotes and page numbers, from 43(!) existing frameworks. Read and explore here: airisk.mit.edu/ To categorize the identified risks, we adapt two existing frameworks into taxonomies. Our Causal Taxonomy categorizes risks based on three factors: the Entity involved, the Intent behind the risk, and the Timing of its occurrence. Our Domain Taxonomy categorizes AI risks into 7 broad domains, and 23 more specific subdomains. For example, 'Misinformation' is one of the domains, while 'False or misleading information' is one of its subdomains. 💡 Four insights from our analysis: 1️⃣ 51% of the risks extracted were attributed to AI systems, while 34% were attributed to humans. Slightly more risks were presented as being unintentional (37%) than intentional (35%). Six times more risks were presented as occurring after (65%) than before deployment (10%). 2️⃣ Existing risk frameworks vary widely in their scope. On average, each framework addresses only 34% of the risk subdomains we identified. The most comprehensive framework covers 70% of these subdomains. However, nearly a quarter of the frameworks cover less than 20% of the subdomains. 3️⃣ Several subdomains, such as *Unfair discrimination and misrepresentation* (mentioned in 63% of documents); *Compromise of privacy* (61%); and *Cyberattacks, weapon development or use, and mass harm* (54%) are frequently discussed. 4️⃣ Others such as *AI welfare and rights* (2%), *Competitive dynamics* (12%), and *Pollution of information ecosystem and loss of consensus reality* (12%) were rarely discussed. 🔗 How can you engage? Website: tinyurl.com/554k67ej Preprint: tinyurl.com/t87ze8nk Database: tinyurl.com/yhtjhb6m Feedback: tinyurl.com/mue7bmyj 🙏 Please help us spread the word by sharing with anyone relevant! Thanks to everyone involved: @aksaeri, @StephenLCasper @mnoetel @ProfNeilT @RistoUuk @soroushjp @jmsdao