A Taxonomy of AI-Generated Disinformation Campaigns

Two days before a national election

On 28 September 2023, two days before Slovakia’s parliamentary election, an audio recording appeared on Facebook. In it, Michal Šimečka, leader of the liberal Progressive Slovakia party, appeared to discuss rigging the vote with Monika Tódová, a journalist at the newspaper Denník N. Fact-checkers established that the audio was synthetic. C2.01 Established

The forgery itself was not especially good. What made it effective was arithmetic. Slovakia observes a 48-hour pre-election moratorium during which media outlets and politicians are expected to stay silent. The recording was published inside that window. The people best placed to rebut it, namely the two individuals impersonated, their party and the newspaper, were the people most constrained from doing so. A further gap compounded it: the clip exploited a limitation in Meta’s manipulated-media policy at the time, which addressed faked video of people saying things they had not said and did not cleanly cover audio. C2.02 Established

This is the observation that organises the chapter. Analysts habitually classify synthetic disinformation by modality, meaning audio, image, video or text, or by how convincing it is. Neither predicts much. The Slovak clip would not have survived a week of scrutiny. It did not need to. It needed to survive 48 hours.

Learning objectives

1. Classify AI-generated influence operations by the condition each requires to succeed, rather than by modality.
2. Define the rebuttal window and explain why it governs impact more reliably than artefact quality.
3. Distinguish attribution, evidence, saturation and process attacks, and identify which defences apply to each.
4. Explain the liar’s dividend and why it inverts the usual relationship between detection and trust.
5. Evaluate the enforcement record of existing legal remedies against documented election cases.
6. Apply the taxonomy to an unfamiliar incident and derive the appropriate institutional response.

A taxonomy organised by requirement

A useful classification tells a defender what to do. Organising by modality does not, because the same defence rarely follows from the same medium. Organising by what the attack requires in order to work does, because each requirement can be denied.

1. Attribution attacks

The attacker places words or actions with a real, identifiable person. Success requires the audience to accept the attribution, and typically requires that the person cannot promptly and credibly deny it. Slovakia is the clean example. So is the New Hampshire robocall discussed below.

What denies it: pre-established, rapid channels of authoritative denial, and provenance on genuine material so that the absence of provenance becomes informative.

2. Evidence attacks

Here the target is not a person but the status of a record. Two directions exist, and the second is the more corrosive. The attacker may fabricate evidence of an event that did not occur, or, more cheaply, may rely on the mere existence of synthetic media to dismiss genuine evidence as fabricated. That second move is known as the liar’s dividend.

3. Saturation attacks

Volume rather than quality. Generative tooling lowers the marginal cost of a plausible artefact to near zero, which allows an attacker to exhaust finite verification capacity. The target is not belief in any single item but the throughput of fact-checkers, moderators and journalists. A defender who measures success by the proportion of items debunked will report progress while losing, because the denominator is set by the attacker.

4. Process attacks

The least discussed and the most directly measurable. The content is instrumental: it exists to change a logistical behaviour, such as whether, when or where someone votes. Success does not require sustained belief, only a decision taken before correction arrives.

Table 2.1 — Taxonomy by requirement. The final column is the chapter’s central claim: for two of the five types, improving detection does not help, and for one of them it actively hurts. Source: author’s taxonomy.

Case study: the New Hampshire primary robocall

Why it is the most instructive election case available. Unlike most incidents, the actor is identified, the method is documented, the actor admitted it, and the legal process ran to a conclusion. It is as close to a controlled experiment in remedy as this field offers.

Classification. It is simultaneously an attribution attack and a process attack. The attribution supplied the authority; the instruction, stay home, was the actual payload. Note that the process element required no sustained belief. A recipient who later doubted the call had already, in some cases, taken the only decision that mattered.

Where the controls failed, and where they did not. Rebuttal was fast: the voice was identified as synthetic quickly and reported widely. Detection was not the gap. The gap was that the telephone network delivered a spoofed-origin political message at scale to a target population, and that nothing in the chain required the caller to be who they claimed.

What the outcome establishes about remedy. This is the finding worth carrying into Chapter 11. The penalties were substantial and the prosecution serious, yet reporting indicates that neither the $6m fine nor a subsequent $22,500 award to voters had been paid. C2.03 Deterrence that arrives nine months after the vote, against a defendant who does not pay, is a weak instrument. Institutions should plan on the assumption that legal remedy will not arrive inside the window that matters.

Evidence limitations. The number of recipients who changed their behaviour is not established, and probably cannot be. Attributing any electoral effect to the calls would exceed the evidence; the case is instructive about mechanism and remedy, not about outcome.

Practical scenario

Executive implications

• Pre-authorise the denial. The binding constraint in a live incident is usually approval, not evidence. Decide now who may issue an authoritative denial without further sign-off, and through which channel, because that decision cannot be taken well at the time.
• Publish provenance on your own output. You cannot stop a forgery being made, but you can make the absence of provenance informative for material attributed to your organisation. This is the only durable answer to the liar’s dividend.
• Do not budget on legal deterrence. The best-documented election case produced a $6m fine that reporting indicates went unpaid, nine months after the vote.
• Triage by consequence. Against saturation, counting debunked items measures the attacker’s output, not your defence.

Common misconceptions

Limitations and open questions

• Effect sizes are unestablished. No documented election case supports a quantified claim about votes changed. Analyses that assert electoral impact generally infer it from reach.
• The taxonomy is analytical, not empirical. It is offered as a tool for deciding what to do, and has not been validated against a labelled corpus of incidents. That validation is a legitimate research project and would be the right way to test it.
• Moratorium interaction is under-studied. Slovakia suggests that pre-election silence rules may be exploitable by design. Whether this generalises across jurisdictions with similar rules is unknown and would be worth comparative work.

Teaching materials

Discussion questions

1. Pre-election moratoria exist to protect deliberation. Does Slovakia show that they now create an exploitable window, and if so what should replace them?
2. Why can detection not address the liar’s dividend, and what can?
3. The New Hampshire fine appears unpaid. What follows for an institution planning its own response?
4. Is labelling synthetic content more useful against fabrication or against denial? Defend your answer.
5. Which of the five attack types is your own organisation least prepared for, and why that one?

Assessment questions

1. Define the rebuttal window and compute it for both case studies, stating your assumptions. (10 marks)
2. Classify three current incidents using Table 2.1 and justify each placement. (15 marks)
3. Construct the argument that improved detection can increase harm, and identify its limits. (15 marks)

Practical exercise

Measure your organisation’s actual rebuttal window. From a hypothetical 02:00 publication of a fabricated statement attributed to your chief executive, how many hours pass until an authoritative denial is live, counting approvals and out-of-hours delay? Most organisations have never measured this and are surprised by the answer.

Research assignment

Select one national election between 2023 and 2026 with a documented synthetic-media incident. Establish the rebuttal window, the platform policy gap if any, and the enforcement outcome, distinguishing evidence from inference throughout.

Seminar topic

“Pre-election moratoria should be abolished because they are now an attack surface.” Argue either side.

Further reading

1. Chesney, R., & Citron, D. (2019). Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Review, 107. Origin of the liar’s dividend argument.
2. International Press Institute. (2023). Slovakia: Deepfake audio of Denník N journalist offers worrying example of AI abuse. ipi.media
3. Federal Communications Commission. (2024). Forfeiture order in the matter of Steve Kramer. See contemporaneous reporting at CyberScoop.
4. AI Incident Database. Incident records for the Slovak audio and the New Hampshire robocall. incidentdatabase.ai
5. National Institute of Standards and Technology. (2024). NIST AI 100-4: Reducing risks posed by synthetic content. nvlpubs.nist.gov

Chapter synthesis

Classifying synthetic disinformation by medium produces tidy categories and poor decisions. Both documented cases in this chapter were audio, both were identified as synthetic quickly, and they failed and succeeded for entirely different reasons. The Slovak clip worked because the response was constrained by electoral rule. The New Hampshire calls were answered within hours, and the remedy still arrived nine months late and, on the reporting, uncollected.

The taxonomy offered here is organised around what each attack needs, because that is what a defender can deny. Its most consequential column is the last one in Table 2.1. For attribution and fabrication attacks, detection contributes something if it is consulted in time. For saturation it is a triage aid at best. For process attacks it is close to irrelevant. And for the liar’s dividend it is counterproductive, because a public trained to expect forgery is a public before whom authentic evidence can be waved away.

Measured against Figure 0.1, this chapter does something uncomfortable to the framework. Chapter 1 argued that Layer 2, detection, cannot carry the weight placed on it. Chapter 2 argues that for a substantial class of attacks Layer 2 is not merely insufficient but the wrong place to look: the operative constraints sit at Layer 4, institutional process, in the form of who may speak and how quickly, and at Layer 5, governance, in the form of whether remedy exists at all. The framework survives this chapter, but only in the version where the layers are not a sequence of filters. They are independent failure points, and the attacker chooses.