Articles

AI4Democracy: How AI Can Be Used to Inform Policymaking?
Colleen McKenzie Colleen McKenzie

AI4Democracy: How AI Can Be Used to Inform Policymaking?

LLMs offer new capacities of particular relevance to soliciting public input when used to process large volumes of qualitative inputs and produce aggregate descriptions in natural language. In this paper, we discuss the use of an LLM-based collective decision-making tool, Talk to the City, to solicit, analyze, and organize public opinion, drawing on three current applications of the tool at varying scales: union decision making, coordination within DAOs, and nation- state consultations. We highlight the ways in which current-generation LLM tools can help leaders understand the needs of their constituents, review what measures are necessary to mitigate the flaws in these existing tools, and explore what future progress in foundation models would be most beneficial for the progress of tools like Talk to the City. We conclude that rapidly advancing AI capabilities offer substantial potential for informing and refining the process of governance, but demand strong and careful governance to mitigate their risks and take full advantage of their benefits. If applied carefully and with an understanding of the social context of its use, AI-driven technology for democratic decision-making has the potential to support collective agency in ways that systematically feed back into AI governance and AI safety institutions, creating a virtuous circle of improving AI’s impact on society.

Read More
Morally Guided Action Reasoning in Humans and Large Language Models: Alignment Beyond Reward
Brittney Gallagher Brittney Gallagher

Morally Guided Action Reasoning in Humans and Large Language Models: Alignment Beyond Reward

Although large language models (LLMs) like GPT-4 made their big splash as chat bots, they might soon be doing a lot more: taking real-world actions. Research teams are using LLMs to play video games, make transactions on the internet, and even help drive autonomous cars. But language models weren’t initially designed to act. We believe that empirically understanding their emergent action reasoning abilities is crucial for these approaches to be safe, corrigible, and beneficial. Here, we describe and motivate our approach to this research problem, as taken in our Moral Learning project – in which we use theories of human cognition to identify potential ways in which LLMs might solve action decision problems. We focus in particular on how LLMs, vs humans, might combine multiple morally relevant properties of naturalistic scenarios to derive their action choices.

Read More
Amplifying Voices: Talk to the City in Taiwan
Brittney Gallagher Brittney Gallagher

Amplifying Voices: Talk to the City in Taiwan

In the heart of Taiwan's technological and democratic evolution, Talk to the City is making significant strides. Deger Turan, the project lead, visited Taiwan in October 2023, engaging with key figures such as the Minister of Digital Affairs, Audrey Tang, and influential members of the Taiwanese open-source community. Their discussions focused on how Talk to the City could be integrated into Taiwanese deliberations, aiming to enhance public participation through advanced data analysis of deliberation. This integration builds on the existing vTaiwan project, which has used Polis to elicit and aggregate the opinions of large numbers of Taiwanese citizens, by analyzing free-form text responses of respondents' opinions.

Read More
Announcing Colleen McKenzie as the New Executive Director of the AI Objectives Institute
Deger Turan Deger Turan

Announcing Colleen McKenzie as the New Executive Director of the AI Objectives Institute

It is my great privilege to welcome Colleen McKenzie as the new Executive Director of the AI Objectives Institute! Colleen has led as AOI’s Director of Strategy, demonstrating an incredible capacity for both rigorous academic research and product management. She has a natural talent for building strong, effective teams and partnerships, and I am confident she is the right leader for this next phase of AOI. I am also excited to announce I am starting the next step of my journey and am joining the forecasting platform Metaculus as CEO.

Read More
Using AI to Give People a Voice, a Case Study in Michigan
Brittney Gallagher Brittney Gallagher

Using AI to Give People a Voice, a Case Study in Michigan

At the AI Objectives Institute, we have been exploring how modern AI technologies could assist under-resourced communities. In this post, we are sharing our learning from a recent case study, where we used our AI-powered analysis platform, Talk to the City, to help formerly incarcerated individuals raise awareness of the challenges that they face when reintegrating into society.

Read More
How AI Agents Will Improve the Consultation Process
Brittney Gallagher Brittney Gallagher

How AI Agents Will Improve the Consultation Process

This post explores how maybe using AI bots built with OpenAI's Custom GPT to make consultations more interactive. These AI agents could help people better understand their views and communicate them effectively during consultations, enabling more accurate input cost-effectively, though not as a replacement human facilitators.

Read More
The Problem With the Word ‘Alignment’
Brittney Gallagher Brittney Gallagher

The Problem With the Word ‘Alignment’

The purpose of our work at the AI Objectives Institute (AOI) is to direct the impact of AI towards human autonomy and human flourishing. In the course of articulating our mission and positioning ourselves -- a young organization -- in the landscape of AI risk orgs, we’ve come to notice what we think are serious conceptual problems with the prevalent vocabulary of ‘AI alignment.’ This essay will discuss some of the major ways in which we think the concept of ‘alignment’ creates bias and confusion, as well as our own search for clarifying concepts.

Read More
Modeling incentives at scale using LLMs
Brittney Gallagher Brittney Gallagher

Modeling incentives at scale using LLMs

Humans routinely model what the incentives of other humans may be. There is however a natural limit to how much modeling and analysis a single person or a small team can do and this is where we see an opportunity to leverage LLMs.

Read More
How can LLMs help with value-guided decision making?
Colleen McKenzie Colleen McKenzie

How can LLMs help with value-guided decision making?

AOI is excited to be presenting at a NeurIPS workshop focused on morality in human psychology and AI! Our recent work draws on human moral cognition and the latest research on moral representation in large language models (LLMs). For this project, we asked, how could we use LLMs to represent value for real-world action decision making?

Read More
Roadmap for a collaborative prototype of an Open Agency Architecture
Deger Turan Deger Turan

Roadmap for a collaborative prototype of an Open Agency Architecture

Open Agency Architecture (OAA) is a bold proposal for AI alignment. It requires a wide ranging formal-modeling enterprise that integrates into a comprehensive world model. We plan to begin developing OAA by iterating on smaller, domain-specific applications that can find near-term use as institutional decision-making tools and provide feedback to continue improving the OAA framework . This document decomposes that goal into sub-agendas designed to engage different technical communities in an international collaboration.

Read More