Introducing Talk to the City: Collective Deliberation at Scale

Written By Guest User

  • Explanation: An interactive LLM tool to improve collective decision-making – by finding the key viewpoints and cruxes in any discourse. 

  • Deliverables:  Open-source toolkit, research report

  • Status: In Development – Testing a Prototype

  • Team Lead: Bruno Marnette

  • Early Preview: Click here

  • Questions/Work Together: colleen@objective.is

 

TOOL SUMMARY

Talk to the City is a LLM deliberation tool to radically improve collective decision-making, by collecting and representing the nuanced views of 50-5+ million people.

This unprecedented detail at scale can vastly improve the quality at which people's opinions can be heard at scale, over simplified categorical surveys. It generates digestible and interactive reports (e.g., summaries and chatbots with specific views), creating a new information paradigm. Respondents and stakeholders can easily understand all the key views in the discourse around them, highlighting the agreements, disagreements, and cruxes between viewpoints.

The newly approachable presentation of this information can improve collective understanding, allow respondents to update their views in response, and create forecasts for how and why people will respond to new proposed policies – ultimately improving collective decision-making. This technology will give better insight to policy-makers than ever before about the views of the populations they work with.

WHY ARE WE BUILDING THIS

  • Collective deliberation is not just a democratic ideal. It is a crucial tool to create consensus and improve decision-making among groups with diverse preferences.

    However, it is hard; deliberation does not scale well. Beyond a small in-person discussion, information is poorly shared; different views get simplified into broad catch-all summaries or ignored.  Even simple yes/no surveys across 100,000 individuals can take months and millions of dollars. This changes with language models - the cost and time to build be dramatically reduced while allowing a new level of detail. 

    Deliberative tools such as Polis have already been successfully deployed in real-world scenarios, and recent work from Deepmind shows promising potential in using language models to bridge diverse preferences. TttC aims to use the latest generative AI tools to build and test language-model-based deliberation at scale.

  • Poor collective coordination can be accelerated by future advanced AI systems - similar, but far worse to how social media accelerated political polarization. AOI believes that misaligned societal structures also pose an existential risk to society - a breakdown of collective organization could cause extreme fragmentation and unrepresentative and misguided policies. Our long-term goal is to help build a tool to overcome this and make an optimistic future.

  • Improved collaboration and collective decision-making between AGI labs are vital for the safe development of advanced AI. Competitive race dynamics can encourage the fast deployment of foundation models before they are adequately vetted. TttC has already been used in a leading survey across AGI lab researchers to highlight their key proposals to improve safety and coordination - results to be published soon.

TECHNICAL DETAILS

  • We use LLMs to extract positions on an issue from unstructured text, and then structure those positions into arguments. The system is based on OpenAI and Anthropic’s LLMs – donated for this research project.

  • Views are aggregated into clusters, based on similarity of reasoning and conclusions

  • A visual, interactive interface displays these clusters of related views, to help policymakers and other users understand them in precise detail

  • A LLM chatbot trained to represent each cluster presents a conversational interface to the most relevant thinking from each cluster. Leaders can query the chat representatives to find cruxes, misunderstandings, and common ground – helping them iterate on policy more quickly. Chat representatives from each cluster can interact with each other to simulate deliberation between different positions

RISKS AND SHORTCOMINGS

We recognize that there are risks from misunderstanding or over-relying on the capabilities of current-generation LLMs. But we also believe that if used with a clear understanding of their limitations, language models may be a valuable tool for analyzing and simulating discourse at scale. Our intention is to explore this possibility first through small, bounded experiments – which will shed light on the capabilities and the limits of these models, and may reveal the ways in which over-reliance could be problematic.

TttC is an experimental approach to utilizing LLMs for deliberation, with the aim of validating how well an LLM-powered deliberation works. We're aware of a number of potential shortcomings and downside risks, and will be paying close attention to the following criteria when testing our prototype:

  • Accurately reflecting users' views

  • Preventing the marginalization of minority voices

  • Avoiding domination by small, vocal minorities.

  • Ensuring simulated deliberation is reliable and representative

To mitigate these risks, we plan to test TttC incrementally, beginning with small-scale, low-stakes environments. This approach will allow us to carefully monitor the system's effectiveness and make improvements as needed.



PROGRESS

TttC is currently in its early stages of development. An early preview is available at talktothe.city.

We've explored potential partnerships for future applications of TttC to national political discourse and peace mediation situations, once the tool has been tested and proven viable. In the initial stages, we are testing our prototype with Mina Protocols and – to allow collective deliberation on new blockchain protocols – publically available Polis survey data.


CONTACT

If you would like to explore working with us or contribute a dataset for experimentation, please get in touch with us at colleen@objective.is. 

AOI Related Reading

External Related Reading

Guest User
Previous

Mapping the Discourse on AI Safety & Ethics
Next

Aligning recommender systems with Jonathan Stray (Talk)