Arman Irani

Arman Irani

Head of AI Technology, AllSides · PhD, Computer Science, UC Riverside

I study arguments: how to find them in text, what they're about, and which side they take. I did my PhD in Computer Science at UC Riverside, co-advised in Political Science, and applied that work to Congressional hearings, Reddit, and Telegram propaganda networks.

I now lead AI at AllSides, which has been out front on intentional, responsible AI use. We take a constitutional approach to our own use of AI, with the principles worked out through deliberation across the whole staff. I also build tools for civic deliberation at scale, so a public consultation can hear from thousands of people on one question and still trace every finding back to who said it.

Google Scholar GitHub LinkedIn

Research

Evaluating Language Models

A lot of what we're told about a model's behavior comes from what its developer can show on paper: a system prompt, a disclosure, a stated principle. I'm interested in whether those things change what the model actually does. In a pre-registered study, two frontier models rated the political bias of 300 news articles that experts had already rated, with and without instructions to be neutral. The ratings didn't move, and the political lean in the models' written explanations held up under every prompt I tried. My takeaway is that claims about how a model behaves need to be checked against how it behaves. I also work on when an LLM's judgments are reliable enough to stand in for human ones.

Pre-registered 300 expert-rated articles, two frontier models

Information Integrity

I trace how narratives move through networks and where they start. With Mark Perry, I mapped how Russian state-affiliated Telegram channels launder war reporting: proxy authorities seed a story, war correspondents and milbloggers copy it, and state media amplify it. The seeders write in the most neutral, least argumentative language of anyone in the chain, so counter-messaging aimed at state media misses where the narratives are actually built. A second study, of 166 Telegram channels tied to accelerationist groups, found that about a third of the channels carry 90% of the influence, and that narratives are a steadier thing to monitor than any individual channel.

Journal of Information Warfare Hague Conference on International Cyber Security, 2023 and 2024

Democratic Deliberation

I compare how arguments work in formal politics and in public online discussion, scoring Congressional hearings and Reddit threads on the same deliberation measures. In abortion hearings, Republican members' arguments tracked the witnesses' more closely when their party held the majority. Reddit swung up to six times more in the diversity of its arguments than the hearings did.

Hearings from 2005–2023 10,000+ hearing statements, 120,000+ Reddit arguments

Computational Argumentation

Underneath all of this is a set of methods for finding arguments in text. WIBA fine-tunes open models on about a thousand examples to decide whether a passage makes an argument, what it's about, and which side it takes, beating BERT baselines by 20 to 45 points. Along the way I hand-relabeled 1,500 examples from two widely used benchmarks to measure how noisy their labels were. ArguSense applies these methods to online forums, and ArguBias looks at where embedding models misjudge whether two arguments are similar.

120,000+ arguments analyzed Code: WIBA

Publications

Selected papers. Full list on Google Scholar.

Transnational Hate: How Racially Motivated Violent Extremists Mobilize Across Decentralized Networks Manuscript

Builds a dataset of 166 Telegram channels tied to 45 accelerationist groups and their forwarding networks, then fits a multivariate Hawkes process to attribute each channel's activity to the channels that influenced it. European channels tend to originate narratives that U.S. and Oceania channels then adopt. Presented at the 2024 Hague Conference on International Cyber Security.

ArguSense: Argument-Centric Analysis of Online Discourse ICWSM 2024

A Irani, M Faloutsos, K Esterling

Applies argument-centric analysis to 205,000+ Reddit posts, measuring argument topics, structure, and diversity. Finds echo chambers reinforced two ways: through who replies to whom, and through what gets upvoted.

Paper
ArguBias: Quantifying the Impact of Semantic-Positional Misalignment on Argument Similarity Manuscript

A Irani et al.

Introduces a framework and 8,000-pair corpus for identifying cognitive bias structures that cause embedding models to misjudge argument similarity. Benchmarks 10 models and shows fine-tuning reduces bias vulnerability by up to 11.6pp while improving scores on BWS and AFS benchmarks.

Earlier work

IKEA: Unsupervised Domain-Specific Keyword Expansion ASONAM 2022

J Gharibshah, J Tachaiya, A Irani, E Papalexakis, M Faloutsos

An unsupervised method for expanding domain-specific keyword sets using dual embedding spaces (word-word and post-post). Evaluated on security forums spanning five years, achieving 0.82+ MAP.

Paper
Wheats the Deal? Understanding the GMO Debate in Online Forums ASONAM 2022

A Irani, K Esterling, M Faloutsos, D Pagliaccia

Introduces Forumlyze, a framework for analyzing user beliefs in online discourse, applied to GMO discussions on Reddit farming communities. Finds Climate Change, Monsanto, and Soil Science as dominant concepts.

Paper

Platforms

Research software I've built.

WIBA

Argument mining framework that detects whether text contains an argument, identifies the topic, and classifies the stance. Built on fine-tuned LLMs. Processes data from Reddit, congressional hearings, and social media. Used by 5+ research teams.

Source Code

DALiSM

Web platform for analyzing and visualizing discourse dynamics. Upload CSV data or retrieve via API. Covers Congressional hearings, Reddit threads, and other text sources. Used at 5+ universities.

Contact

Email [email protected]
Position Head of AI Technology, AllSides
Education PhD, Computer Science, UC Riverside
Scholar Google Scholar
GitHub @Armaniii
LinkedIn arman-irani