Articles
AI4Democracy: How AI Can Be Used to Inform Policymaking?
LLMs offer new capacities of particular relevance to soliciting public input when used to process large volumes of qualitative inputs and produce aggregate descriptions in natural language. In this paper, we discuss the use of an LLM-based collective decision-making tool, Talk to the City, to solicit, analyze, and organize public opinion, drawing on three current applications of the tool at varying scales: union decision making, coordination within DAOs, and nation- state consultations. We highlight the ways in which current-generation LLM tools can help leaders understand the needs of their constituents, review what measures are necessary to mitigate the flaws in these existing tools, and explore what future progress in foundation models would be most beneficial for the progress of tools like Talk to the City. We conclude that rapidly advancing AI capabilities offer substantial potential for informing and refining the process of governance, but demand strong and careful governance to mitigate their risks and take full advantage of their benefits. If applied carefully and with an understanding of the social context of its use, AI-driven technology for democratic decision-making has the potential to support collective agency in ways that systematically feed back into AI governance and AI safety institutions, creating a virtuous circle of improving AI’s impact on society.
Machina Economica, Part II: The Commodification of Risk
This is the second entry in a series on AI integration in the economy, exploring its financial, sociopolitical and historical implications. This entry focuses on economic risk and its financialisation.
Morally Guided Action Reasoning in Humans and Large Language Models: Alignment Beyond Reward
Although large language models (LLMs) like GPT-4 made their big splash as chat bots, they might soon be doing a lot more: taking real-world actions. Research teams are using LLMs to play video games, make transactions on the internet, and even help drive autonomous cars. But language models weren’t initially designed to act. We believe that empirically understanding their emergent action reasoning abilities is crucial for these approaches to be safe, corrigible, and beneficial. Here, we describe and motivate our approach to this research problem, as taken in our Moral Learning project – in which we use theories of human cognition to identify potential ways in which LLMs might solve action decision problems. We focus in particular on how LLMs, vs humans, might combine multiple morally relevant properties of naturalistic scenarios to derive their action choices.
Machina Economica, Part I: Autonomous Economic Agents in Capital Markets
The first entry in a series on AI integration in the global economy, exploring its ethical, sociopolitical, and financial consequences. This entry focuses on financial markets and firm behaviour.
Amplifying Voices: Talk to the City in Taiwan
In the heart of Taiwan's technological and democratic evolution, Talk to the City is making significant strides. Deger Turan, the project lead, visited Taiwan in October 2023, engaging with key figures such as the Minister of Digital Affairs, Audrey Tang, and influential members of the Taiwanese open-source community. Their discussions focused on how Talk to the City could be integrated into Taiwanese deliberations, aiming to enhance public participation through advanced data analysis of deliberation. This integration builds on the existing vTaiwan project, which has used Polis to elicit and aggregate the opinions of large numbers of Taiwanese citizens, by analyzing free-form text responses of respondents' opinions.
Announcing Colleen McKenzie as the New Executive Director of the AI Objectives Institute
It is my great privilege to welcome Colleen McKenzie as the new Executive Director of the AI Objectives Institute! Colleen has led as AOI’s Director of Strategy, demonstrating an incredible capacity for both rigorous academic research and product management. She has a natural talent for building strong, effective teams and partnerships, and I am confident she is the right leader for this next phase of AOI. I am also excited to announce I am starting the next step of my journey and am joining the forecasting platform Metaculus as CEO.
Using AI to Give People a Voice, a Case Study in Michigan
At the AI Objectives Institute, we have been exploring how modern AI technologies could assist under-resourced communities. In this post, we are sharing our learning from a recent case study, where we used our AI-powered analysis platform, Talk to the City, to help formerly incarcerated individuals raise awareness of the challenges that they face when reintegrating into society.
How AI Agents Will Improve the Consultation Process
This post explores how maybe using AI bots built with OpenAI's Custom GPT to make consultations more interactive. These AI agents could help people better understand their views and communicate them effectively during consultations, enabling more accurate input cost-effectively, though not as a replacement human facilitators.
The Problem With the Word ‘Alignment’
The purpose of our work at the AI Objectives Institute (AOI) is to direct the impact of AI towards human autonomy and human flourishing. In the course of articulating our mission and positioning ourselves -- a young organization -- in the landscape of AI risk orgs, we’ve come to notice what we think are serious conceptual problems with the prevalent vocabulary of ‘AI alignment.’ This essay will discuss some of the major ways in which we think the concept of ‘alignment’ creates bias and confusion, as well as our own search for clarifying concepts.
Modeling incentives at scale using LLMs
Humans routinely model what the incentives of other humans may be. There is however a natural limit to how much modeling and analysis a single person or a small team can do and this is where we see an opportunity to leverage LLMs.
How can LLMs help with value-guided decision making?
AOI is excited to be presenting at a NeurIPS workshop focused on morality in human psychology and AI! Our recent work draws on human moral cognition and the latest research on moral representation in large language models (LLMs). For this project, we asked, how could we use LLMs to represent value for real-world action decision making?
Talk to the City: an open-source AI tool for scaling deliberation
Talk to the City is an open-source LLM interface for improving collective deliberation and decision-making by analyzing detailed, qualitative data. It aggregates responses and arranges similar arguments into clusters.
Roadmap for a collaborative prototype of an Open Agency Architecture
Open Agency Architecture (OAA) is a bold proposal for AI alignment. It requires a wide ranging formal-modeling enterprise that integrates into a comprehensive world model. We plan to begin developing OAA by iterating on smaller, domain-specific applications that can find near-term use as institutional decision-making tools and provide feedback to continue improving the OAA framework . This document decomposes that goal into sub-agendas designed to engage different technical communities in an international collaboration.
AI and the Transformation of Capitalism (Talk)
Reward hacking in markets, why capitalism is a form of artificial intelligence, and how conversational AI could help us know a million desires at once