DigitalDUAL-USE

The Bot-or-Not Fallacy

What it is

The analyst error of treating high activity, an odd handle, a default avatar, imperfect English, or simple disagreement as proof that an account is automated or fake.

How it works

The error is a base-rate problem wearing a technical costume. Popular bot tells — a handle ending in eight digits, hundreds of posts a day, no profile photo — are each common among real people: Twitter auto-generated numeric handles for new users, retirees and the chronically ill post at volumes that look inhuman, and privacy-conscious users avoid photos. When genuine automation is rare among the accounts a person actually meets, even a fairly accurate cue produces mostly false positives. Classifiers have the same weakness. Rauchfleisch and Kaiser (2020) tested Botometer against known human and bot accounts and found that scores shifted over time and that any usable threshold misclassified many humans, particularly non-English speakers. Gallwitz and Kreil (2022) manually examined hundreds of accounts labelled as social bots in peer-reviewed studies and reported finding ordinary humans. The developers of Botometer dispute the strongest version of that critique and say the tool was never meant for labelling individuals. The binary itself misleads: much coordinated activity is run by paid humans, which no automation test detects.

Real-world examples

  • •Rauchfleisch and Kaiser (2020) ran Botometer on verified lists of German and American politicians and known bots, and found that conservative thresholds still flagged real parliamentarians as bots while missing many actual automated accounts.
  • •Accounts with names such as a first name followed by eight digits are routinely called bots in replies; that pattern is what Twitter itself assigned to new sign-ups who did not choose a handle, so it mostly marks people who did not bother to customize a handle.
  • •In April 2018 one of the accounts reported in British media as a Russian bot, on grounds that included posting volume, belonged to a retired British man who subsequently appeared on live television.
  • •Pew Research Center's 2018 study of links shared on Twitter estimated that a large share came from automated accounts, but many of those were openly automated and harmless, such as news-outlet feeds — automation is not the same thing as deception.

Ethical guidelines

Where the line is

Screening accounts for signs of automation is legitimate research and moderation practice when it is validated, reported with error rates, and used to describe populations. It becomes an error, and then a smear, when an unvalidated cue or score is presented as proof that a particular named person is fake.

  • ●A classifier score is a probability about a population, not a finding about a person. Do not publish it as a verdict on a named account.
  • ●Report error rates and validation alongside any bot estimate, and say what human checking was done.
  • ●Separate three questions that are usually blurred: is it automated, is it inauthentic, and is it coordinated. Only the last two are about deception.
  • ●If you publicly called a real person a bot, correct it where you said it.

How to defend against it

  • ►Ask for the base rate: of accounts that show this cue, what fraction are actually automated? If nobody knows, the cue proves nothing.
  • ►Look for evidence that a single account cannot fake on its own: identical text across many accounts, synchronized posting to the minute, batch registration dates, shared links to the same obscure domain.
  • ►Read the account's history before judging. Replies that respond to context, typos, personal photos over years, and arguments with friends are hard to automate at scale.
  • ►Treat bot percentages in headlines with suspicion unless the method was validated against hand-labelled accounts, and check whether the tool's authors endorse that use.
  • ►When unsure, say unsure. Mute or ignore instead of accusing; the cost of being wrong falls on a stranger.

References

  1. Rauchfleisch, A., & Kaiser, J. (2020). The False Positive Problem of Automatic Bot Detection in Social Science Research. PLOS ONE, 15(10), e0241045 · link
    Botometer thresholds produce substantial false positives and false negatives and scores are unstable over time and across languages.
  2. Gallwitz, F., & Kreil, M. (2022). Investigating the Validity of Botometer-Based Social Bot Studies. Disinformation in Open Online Media (MISDOOM 2022), Lecture Notes in Computer Science, Springer · link
    Manual review of accounts labelled as social bots in published studies found ordinary human users.
  3. Wojcik, S., Messing, S., Smith, A., Rainie, L., & Hitlin, P. (Pew Research Center) (2018). Bots in the Twittersphere. Pew Research Center, April 9, 2018
    Much link-sharing automation on Twitter is overt and benign, illustrating that automation and deception are different properties.
Last reviewed
Suggest a correction

Detect The Bot-or-Not Fallacy in any text

Paste any message, email, or article into our free Manipulation Detector to see if The Bot-or-Not Fallacy or other techniques are being used on you.

Try the Detector FreeCreate Free Account

Related Articles

Dark Patterns in UX: How Apps Manipulate Your Behavior

Subscription traps, misleading interfaces, and engineered addiction. Understanding the persuasion techniques built into the apps you use every day.

7 min read

OSINT for Beginners: Open Source Intelligence Explained

Open Source Intelligence (OSINT) uses publicly available data to gather actionable insights. Here is a beginner-friendly guide to what OSINT is and how it is used.

10 min read