DigitalMANIPULATIVE

LLM Comment Flooding

What it is

Filling reply threads, review sections, legislators' inboxes, or regulatory comment dockets with machine-written messages that each look individually composed, so that volume is mistaken for public sentiment.

How it works

Institutions that listen to the public use crude proxies for sentiment: how many comments arrived, how many letters a legislator received, how lopsided the replies are. Older flooding campaigns were caught by duplicate detection, because thousands of identical form letters are easy to cluster. Language models defeat that specific check by producing endless paraphrase. In 2019 a Harvard student, Max Weiss, showed the problem before modern chatbots existed: 1,001 machine-generated comments submitted to a federal Medicaid waiver docket made up more than half of all comments, and trained human raters sorted them from real ones at chance. A 2023 field experiment by Kreps and Kriner found state legislators responded to AI-written constituent emails almost as often as to human-written ones. The flood does not need to persuade anyone; it needs only to corrupt the count, or to bury real comments so that no one reads them.

Real-world examples

  • In October 2019 researcher Max Weiss submitted 1,001 bot-generated comments to the federal docket on Idaho's Medicaid waiver, then withdrew them and disclosed the test; survey participants asked to tell them from human comments were right about 49 percent of the time.
  • The New York Attorney General's 2021 report on the FCC's 2017 net neutrality docket found that nearly 18 million of the more than 22 million comments were fake, including millions funded by broadband industry groups using real people's names without consent and millions generated by a single college student supporting net neutrality. That predates language models and shows that both sides of a policy fight have flooded a docket.
  • Meta reported in May 2024 that an Israel-based network posted likely AI-generated comments under the Facebook Pages of media organizations and U.S. lawmakers, many unrelated to the posts they replied to.
  • Kreps and Kriner sent 32,398 emails, half written by GPT-3, to 7,132 U.S. state legislators; response rates to the machine-written letters were only about two percentage points lower.

Ethical guidelines

  • Helping real people say what they actually think, with their knowledge and under their own names, is advocacy. Generating the people, or the opinions, is fraud on the process.
  • Using a real person's name on a comment they never saw is identity misuse regardless of which side of the issue it supports.
  • Agencies and offices that treat comment volume as a vote invite this attack; the ethical burden is shared by those who design the listening process.

How to defend against it

  • If you run a consultation or an inbox, weigh substance over count. U.S. notice-and-comment rulemaking is not a plebiscite; one comment with new evidence outweighs ten thousand that restate a position.
  • Verify a sample. Legislators in the Kreps and Kriner study suggested calling back or replying with a question; a real constituent answers and has a local address.
  • As a reader, treat a lopsided reply section as weather, not as a poll. Look for a measure that samples people rather than counting posts.
  • Look for the behavioral tells that paraphrase cannot hide: bursts of submissions in a short window, replies that do not engage with the post they sit under, and accounts with no other history.
  • Do not rely on AI-text detectors to sort the pile. They are unreliable and penalize non-native writers (see detector-overtrust).

References

  1. Weiss, M. (2019). Deepfake Bot Submissions to Federal Public Comment Websites Cannot Be Distinguished from Human Submissions. Technology Science, 2019121801 · link
    The 1,001 generated Medicaid-waiver comments and the chance-level human detection rate.
  2. Kreps, S., & Kriner, D. L. (2023). The potential impact of emerging technologies on democratic representation: Evidence from a field experiment. New Media & Society · link
    Legislators responded to GPT-3-written constituent emails nearly as often as to human-written ones, and the verification practices legislators proposed.
  3. New York State Office of the Attorney General (2021). Fake Comments: How U.S. Companies and Partisans Hack Democracy to Undermine Your Voice. Report, May 2021 · link
    Findings on fabricated comments in the FCC 2017 net neutrality proceeding from both industry-funded and pro-net-neutrality sources.
  4. Meta (2024). Adversarial Threat Report, First Quarter 2024. Meta Transparency Center, May 2024 · link
    Likely AI-generated comments posted under the Pages of media organizations and lawmakers by an Israel-based network.
Last reviewed
Suggest a correction

Detect LLM Comment Flooding in any text

Paste any message, email, or article into our free Manipulation Detector to see if LLM Comment Flooding or other techniques are being used on you.

Related Articles