Amplifying Voices: Talk to the City in Taiwan

Written By Brittney Gallagher
“Ten years ago, the vTaiwan scope was limited, because stakeholder groups needed to adapt to the technology at the time. Today, this is being bridged with the advent of language models that can adapt to their needs. Back in 2014, it was impossible with the capacity of g0v contributors to interview a mini public of people and aggregate their ideas while preserving the full nuance.

But now, with Talk to the City’s help, that cost has been reduced to essentially zero. It’s broad-listening and it can change the nature of this recursive public.”
— Audrey Tang, Taiwan’s 1st Digital Minister and co-author of Plurality.net

The user base is growing: landing in Taiwan

In the heart of Taiwan's technological and democratic evolution, Talk to the City is making significant strides. Deger Turan, visited Taiwan in October 2023, engaging with key figures such as the Minister of Digital Affairs, Audrey Tang, and influential members of the Taiwanese open-source community. Their discussions focused on how Talk to the City could be integrated into Taiwanese deliberations, aiming to enhance public participation through advanced data analysis of deliberation. This integration builds on the existing vTaiwan project, which has used Polis to elicit and aggregate the opinions of large numbers of Taiwanese citizens, by analyzing free-form text responses of respondents' opinions.

The first deployment of Talk to the City in Taiwan occurred at the Alignment Assemblies of AI, organized by the Ministry of Digital Affairs (moda). In this project, moda first gathered statements on AI from Taiwanese citizens through the Polis tool online, and then used Talk to the City to organize these statements into visual interactive diagrams. For more details, please refer to the moda official website. The reports below were produced during the development phase of Talk to the City, in collaboration with the Taiwanese civic tech community. The application scenarios include policy participation platforms, AI deliberation workshops, and the 2024 presidential election. 

The objective of Talk to the City is to scale democratic deliberation through the application of frontier Large Language Models (LLMs), and demonstrate the potential of this application in collaboration with Taiwan. Talk to the City uses LLMs to extract key arguments, cluster similar claims, and generate summary analyses of the content of those claims. Consequently, Talk to the City aims to generate a report without adding spin or bias from any respondents' perspective. 

Here are some examples applications Talk to the City:

Talk to the City report translations to English were done using GPT-4

Dig Deeper Into AI Assembly’s Case Study

AI Assembly 2023 workshops
Source : Taiwan AI Academy - TW AI Assembly https://ai-assembly.tw/

Summary

  • TW AI Assembly held 4 deliberation workshops in Taiwan and covered topics from LLM development, innovation in medical technology, education, to intersection with web3. 

  • 2,000+ opinions from 400+ participants were collected during the process. 

  • Talk to the City offered topic clustering, post-workshop analysis, and online interaction with the dataset.

About TW AI Assembly

The AI Assembly is a project by the Taiwan AI Academy, inspired by AI Impact Workshops, a 2-day workshop held by TW AI Academy discussing the impacts AI has brought to local industries. The initiative sprung from a shared mission and civic spirit among speakers and participants: to spread knowledge about the logic, use, limits, and impacts of technology. Its goal is to enlighten Taiwanese society about the changes and challenges ahead, encouraging informed responses and actions. Feedback from 400+ attendees at spring and summer workshops, and the Generative AI Symposium, has been compiled. These discussions reflect Taiwan's early adopters' perspectives on AI's impacts.

Recognizing the need for a democratic and diverse approach, the AI Assembly aims to make workshops a space for stakeholder dialogue. It focuses on assessing available resources, choosing the right paths and objectives, and promoting inclusive values through AI – advocating for its use in serving society, and avoiding exploitation. The team plans to streamline their efforts via issue analysis, deliberative meetings, policy suggestions, and international sharing, aiming for a more systematic engagement.

Background

Traditionally, when communicating policies to the public in Taiwan, the government resorts to holding public hearings or listening sessions to collect and understand citizens' opinions. Public hearings have a lower barrier for public participation, but they tend to have minimal impact on the policy itself. Only invited citizens and stakeholders would be invited to public hearings, despite the records being public, it’s still not accessible for the general public.  These barriers create difficulties for citizens in democratic nations to engage in policy-making.

In response, AI Assembly pioneered a cycle in 2023 that included four AI Deliberative Workshops. After anonymizing the speakers' contributions, over 2,000 opinions were collected, with more than 400 participants involved.

Deliberation workshop methods

 
 

Phase One: Expert Interviews

The process began with deep-dive interviews with experts on specific discussion topics. These sessions helped to define the crucial questions that workshop participants would later explore, ensuring that debates were grounded in informed perspectives.

Phase Two: Pre-Workshop Preparation

To align participants' understandings before diving into discussions, the AI Assembly team provided materials on the chosen topics for pre-reading. This step aimed to enhance common knowledge among participants, fostering more productive and informed deliberations.

Phase Three: Conducting the Workshops

Unlike traditional deliberative workshops, AI Assembly 2023 experimented with different workshop formats, because different groups of participants require different discussion formats. For example, for the workshop on the development of LLMs in Taiwan, AI Assembly chose small group discussions. Participants with different professional backgrounds were evenly divided into each group, so that each group could have a more diverse perspective.

For the workshop on web3 x AI, AI Assembly used a "fishbowl" discussion format. In this format, participants can decide whether to enter an inner circle to express their opinions, or remain in the audience observing that inner circle. AI Assembly conducted a pre-workshop study of the participants' backgrounds and found that this group of participants had a relatively high level of autonomy. The Fishbowl format was chosen because it was thought to be more likely to stimulate discussion.

Phase Four: Data Digitization and Analysis

Using the latest in AI technology, every spoken word was transcribed and summarized using OpenAI's gpt-4-1106-preview model, which significantly outperformed previous models in both efficiency and accuracy. This process involved an initial categorization based on  expert interviews. The AI Assembly team categorized the opinions into the 5 questions below:

  1. Should Taiwan invest in building its own Large Language Model (LLM)?

  2. What constitutes high-quality data for training a Taiwanese LLM?
    Where can Taiwan source the data needed for its LLM?

  3. Should a Taiwanese LLM prioritize reflecting Taiwanese values and ethics?

  4. How can Taiwan effectively integrate LLMs into various applications?

  5. What potential social and economic impacts should Taiwan prepare for with the introduction of LLMs?

This categorization process was followed by a second-tier classification using ChatGPT (gpt-4-1106-preview) that distilled the 2,000 opinions for further analysis.

Phase Five: Online Opinion Collection

After the workshop's conclusion, the AI Assembly team summarized these 2,000+ opinions using various techniques, including both manual review and online tools like Polis & Talk to the City for interactive online summaries.

How Talk to the City helped in the scene?

After generating the report, collaborators from moda and AI Objectives conducted interviews with several users who had experience with other online deliberation tools (e.g. Polis). Among them, a civic tech community participant with a background in philosophy mentioned that Talk to the City can help participants understand the overall issue. In comparison, Polis is more like a traditional polling tool, reflecting the preferences of user groups.

This improvement ensured that the opinions expressed during the deliberations were captured with relatively less bias than an opinionated person might introduce, and the reports themselves became more accessible. Even individuals who didn't participate in the deliberations or those without a technical background could easily grasp a deeper understanding of the discussions concerning the development of LLM in Taiwan, as uncovered during the four AI Assembly deliberative workshops last year.

Looking Forward: Exponential Growth in Democratic Deliberation Tools

The methodologies discussed represent a significant leap forward in expanding discussions around AI policy from a select few to a broader audience. As we look further into 2024, tools aimed at democratizing deliberation in the AI field are set to grow exponentially. Projects like Talk to the City will collaborate with more initiatives, promising to further bridge the gap between public opinion and policy formulation in the ever-evolving landscape of technology and governance.

使用者族群成長：落地台灣

“十年前，多方利害關係人在 vTaiwan 的討論範疇，往往受限於當時的技術。如今隨著語言模型的發展，我們可以跨越這些障礙、適應各方的需求。在 2014 年，g0v 社群未必有足夠的能量歸納全方位的訪談，但現在，有了「我城對談」這類數位審議工具的幫忙，使得歸納訪談的成本趨近於零。如此「廣聽」（而非「廣播」）的技術，可望大幅改寫公民參與模式，促成不斷迭代的遞迴式公民社會。”
— 唐鳳, 台灣第一位數位部長與 Plurality.net 共同作者

Talk to the City (TttC) 正在為台灣的科技民主引進一個新的維度。2023 年底， TttC 專案負責人 Deger Turan 來台拜訪台灣數位部長唐鳳及該部會相關成員， Deger 也與台灣開源社群的其他深度參與者聚會討論。該次對談焦點著重於該如何使用 Talk to the City 整合進台灣現有的審議流程，並且透過最先進的資料分析工具加強民眾的公共參與。這項整合建基於現有的vTaiwan計畫，透過分析受訪者的自由形式文字回覆意見，運用Polis來徵集並彙總大量台灣公民的意見。

Talk to the City 在台灣第一次的部署是由數位部的 AI 對齊大會負責 ，目前已成功部署至該機關內供人員使用。在此次合作中，台灣數位部也將原本在 Polis 上台灣公民針對 AI 的討論透過 Talk to the City 進行進階分析，透過可互動的視覺畫圖表，讓公民可以針對公共議題有更多參與。更多詳細內容可前往台灣數位部網站查看。

另外，Talk to the City 也於這半年與一些台灣的公民社群合作產出分析報告，分析資料來源多樣，比如政策參與平台、AI 審議工作坊，甚至 2024 台灣總統大選...等， Talk to the City 的目標是使用最先進的大型語言模型來擴展民主化審議。

Talk to the City 的目標是藉由運用先進的大型語言模型(LLM)，擴大民主審議的規模，並展現在與台灣的合作中應用的潛力。Talk to the City 使用LLM來提取關鍵論點、歸類類似的主張，並產生對這些主張內容的摘要分析。Talk to the City 的目標是，生成一份不受任何發表意見者觀點影響的報告。

以下為 Talk to the City 於台灣的使用案例：

Talk to the City 報告的英文翻譯是使用 GPT-4 完成的

台灣 AI Assembly 使用案例深度探討

AI Assembly 2023 workshops AI 民主化審議工作坊
資料來源 : 台灣人工智慧學校 - TW AI Assembly 

TL;DR

  • 台灣 AI Assembly 舉辦了四場民主化 AI 審議工作坊，主題包含台灣 LLM 之發展、智慧醫療、教育，以及 web3 領域。

  • 共計由四百多位參與者中蒐集兩千多個意見。

  • Talk to the City 協助將參與者意見分群，並且應用於工作坊後盤點分析與線上資料互動。

關於 TW AI Assembly

2023年3月，台灣人工智慧學校舉辦了台灣第一場「生成式 AI 衝擊工作坊」，在這場為期兩天的工作坊當中，與會者共同討論了人工智慧對台灣產業帶來的影響。而後同年11月，承襲著對該議題的關注，台灣人工智慧學校又接續在同年11月發起了 TW AI Assembly 這項計畫。。一群使命感十足、熱心關注社會議題的公民，攜手啟動了這項重要計劃：傳播關於科技邏輯、使用、限制和影響的知識。其目標是啟發台灣社會對未來變革和挑戰的認知，鼓勵參與者做出明智的回應和行動。該計畫彙整了400多名參加春季和夏季工作坊以及生成式人工智慧研討會的與會者的回饋意見，這些討論反映了台灣 AI 早期參與者對人工智慧衝擊產業的觀點。

為了秉持民主並達到多元涵容，TW AI Assembly 致力將工作坊打造成可以對話的空間，讓各個利益相關者能夠彼此交流。。該計畫重視評估可用資源、選擇正確的路徑和目標，藉由AI工具，鼓勵和支持能促進社會包容多元的價值觀，探討AI如何為社會所用，並且避免造成剝削。為了讓整個計畫更有系統，該計畫團隊全面分析問題，舉辦審議會議、提出政策建議，並和國際人士進行交流。

背景

以往在台灣與民眾溝通政策時，政府常常使用舉辦公聽會或聽證會的形式搜集及聆聽民眾意見，公聽會行民眾參與門檻較低，但是對政策影響力微乎其微。而聽證會是以接近司法程序般嚴格的方式進行的，通常只有相關的市民與利害關係人會被邀請，所以參與門檻較高，在民主國家的民眾要參與政策制定其實也不是一件簡單的事。

而在發展 LLM 訓練模型時，其實跟民主國家制定政策時遇到類似的問題，比如如何有效地蒐集民眾的意見，而非只是舉辦形式上的公開研討會，或是在如何避免搜集民眾意見後卻沒下文。 AI Assembly 於 2023 年舉辦了四場 AI 審議工作坊，邀集400 多位參與者參與其中，經過發言去識別化後共搜集了 2000 多個意見。

審議工作坊流程

 
 


階段一：專家訪談

針對特定討論議題，召集專家進行深度訪談，定義審議工作坊中值得參與者一起討論的問題。

階段二：事前告知

為了確保審議工作坊的討論品質， 台灣 AI Assembly 團隊會在審議工作坊開始之前針對特定議題準備相關素材要求參與者先行閱讀，以提升參與者的認知對齊程度。

階段三：辦理實體審議工作坊

不同於一般審議工作坊，AI Assembly 於 2023 年實驗了不同形式的工作坊辦理形式，原因在於針對不同族群的參與者，需要於籌備期間與引導師討論適合該群體的討論形式，像是針對 LLM 於台灣的發展，台灣 AI Assembly 挑選了小組討論，也於事前精心地將不同職能的參與者平均分散在各組，讓每組的討論可以有更多元的視角。

而另一場 web3 x AI 的審議工作坊則是使用 Fish-bowl 形式的討論，讓參與者自行決定是否進入內圈發表意見，這麼做是因為 AI Assembly 於事前針對使用者背景研究發現該群受眾自主性相對較高，Fish-bowl 形式討論反而可以激起大家的討論慾。

階段四：資料數位化及分析

逐字稿摘要：使用 ChatGPT 將各段逐字稿轉成摘要。在整個過程中，台灣 AI Assembly 採用了 OpenAI 的 gpt-4-1106-preview 模型，其表現在多方面顯著優於先前的 gpt-3 模型，顯著提升了意見整理的效率和準確度。在我們的最近一次測試中，我們使用了 tier-2 等級的 gpt-4-1106-preview 模型，這個等級每天可以處理高達 1500000個tokens。特別是不用再擔心每日限制，而是會在每分鐘回覆約 1000 tokens，這樣的設計提供了更大的靈活性和連續性。

首先從第一階段專家訪談逐字稿，整理出五大議題並進行第一次分類，五大議題如下：

  1. 台灣政府需要發展自己的 LLM 嗎？如何發展？

  2. 資料很重要，什麼是好資料？從哪裡來？

  3. LLM 要有價值觀或道德嗎？

  4. LLM 應該怎麼使用？

  5. 對 LLM 衝擊擔憂與因應？

接著就這五大議題的摘要進行第二層分類，然後將這些素材匯入至 Talk to the City 整理出 2000個意見。

階段五：線上意見徵集

在經過前述四個階段，台灣 AI Assembly 團隊針對這 2000 多筆意見以不同方法做了摘要，其中包含先人工審閱再摘要、使用 polis & Talk to the City 等工具進行線上摘要及互動。

Talk to the City 如何協助台灣的民主場景?

數位發展部和 AI Objectives 的協作者於這幾個月的專案合作後，對曾使用過其他線上審議工具（如 Polis）的使用者進行了訪談。一位具有哲學背景的公民社群參與者提到， Talk to the City 可以幫助參與者進行議題整體理解，從報告中也可以直接看出個主題於 LLM 中的距離與偏好，比起來 Polis 反而更像傳統民調工具，只是反應使用者偏好分群。

這套新興工具讓我們在討論過程中能夠更公正地記錄各種意見，同時也讓報告更容易理解。有些人即便沒有參與審議，或者不具備相關技術知識，都可以輕鬆瞭解 TW AI Assembly 去年舉辦的四次工作坊內容，瞭解與會者針對發展大型語言模型等議題進行的深入討論。

展望未來：民主審議工具突飛猛進

本次研討會所探討的方法論，讓人看見時代的一大進步，AI工具可以讓公共討論不再侷限於少數專學者，使公眾有更多機會參與其中。我們期待2024年有更多更進步的工具，讓AI領域的審議能夠變得民主。我們期待像 Talk to the City 這樣的專案與更多倡議行動合作。科技不斷演進，它可以架起更多溝通的橋樑，縮小公眾意見與政策制定之間的差距，幫助更多人參與治理。

 

A addition was made on May 30, 2024: Added the quote from Audrey Tang in English/Traditional Chinese.


Brittney Gallagher
Previous

Machina Economica, Part I: Autonomous Economic Agents in Capital Markets
Next

Announcing Colleen McKenzie as the New Executive Director of the AI Objectives Institute