DigitalMANIPULATIVE
LLM Comment Flooding
What it is
Filling reply threads, review sections, legislators' inboxes, or regulatory comment dockets with machine-written messages that each look individually composed, so that volume is mistaken for public sentiment.
How it works
Real-world examples
- •In October 2019 researcher Max Weiss submitted 1,001 bot-generated comments to the federal docket on Idaho's Medicaid waiver, then withdrew them and disclosed the test; survey participants asked to tell them from human comments were right about 49 percent of the time.
- •The New York Attorney General's 2021 report on the FCC's 2017 net neutrality docket found that nearly 18 million of the more than 22 million comments were fake, including millions funded by broadband industry groups using real people's names without consent and millions generated by a single college student supporting net neutrality. That predates language models and shows that both sides of a policy fight have flooded a docket.
- •Meta reported in May 2024 that an Israel-based network posted likely AI-generated comments under the Facebook Pages of media organizations and U.S. lawmakers, many unrelated to the posts they replied to.
- •Kreps and Kriner sent 32,398 emails, half written by GPT-3, to 7,132 U.S. state legislators; response rates to the machine-written letters were only about two percentage points lower.
Ethical guidelines
- ●Helping real people say what they actually think, with their knowledge and under their own names, is advocacy. Generating the people, or the opinions, is fraud on the process.
- ●Using a real person's name on a comment they never saw is identity misuse regardless of which side of the issue it supports.
- ●Agencies and offices that treat comment volume as a vote invite this attack; the ethical burden is shared by those who design the listening process.
How to defend against it
- ►If you run a consultation or an inbox, weigh substance over count. U.S. notice-and-comment rulemaking is not a plebiscite; one comment with new evidence outweighs ten thousand that restate a position.
- ►Verify a sample. Legislators in the Kreps and Kriner study suggested calling back or replying with a question; a real constituent answers and has a local address.
- ►As a reader, treat a lopsided reply section as weather, not as a poll. Look for a measure that samples people rather than counting posts.
- ►Look for the behavioral tells that paraphrase cannot hide: bursts of submissions in a short window, replies that do not engage with the post they sit under, and accounts with no other history.
- ►Do not rely on AI-text detectors to sort the pile. They are unreliable and penalize non-native writers (see detector-overtrust).
References
- Weiss, M. (2019). Deepfake Bot Submissions to Federal Public Comment Websites Cannot Be Distinguished from Human Submissions. Technology Science, 2019121801 · linkThe 1,001 generated Medicaid-waiver comments and the chance-level human detection rate.
- Kreps, S., & Kriner, D. L. (2023). The potential impact of emerging technologies on democratic representation: Evidence from a field experiment. New Media & Society · linkLegislators responded to GPT-3-written constituent emails nearly as often as to human-written ones, and the verification practices legislators proposed.
- New York State Office of the Attorney General (2021). Fake Comments: How U.S. Companies and Partisans Hack Democracy to Undermine Your Voice. Report, May 2021 · linkFindings on fabricated comments in the FCC 2017 net neutrality proceeding from both industry-funded and pro-net-neutrality sources.
- Meta (2024). Adversarial Threat Report, First Quarter 2024. Meta Transparency Center, May 2024 · linkLikely AI-generated comments posted under the Pages of media organizations and lawmakers by an Israel-based network.
Last reviewed
Suggest a correctionRelated Articles
Dark Patterns in UX: How Apps Manipulate Your Behavior
Subscription traps, misleading interfaces, and engineered addiction. Understanding the persuasion techniques built into the apps you use every day.
7 min read
OSINT for Beginners: Open Source Intelligence Explained
Open Source Intelligence (OSINT) uses publicly available data to gather actionable insights. Here is a beginner-friendly guide to what OSINT is and how it is used.
10 min read