Chapter 7 of 1212 min read

Adversarial Robustness and Evasion of Deepfake Detectors

Flipping the lowest bit of every pixel turns a 0.95 AUC detector into one that is reliably wrong. And if that happened to you, nothing would tell you.

Below chance

Carlini and Farid, at the CVPR workshops in 2020, took a classifier that reached an AUC of 0.95 across almost all existing image generators while trained on only one, and attacked it five ways. With full access they flipped the least significant bit of each pixel and the AUC fell to 0.0005. Perturbing one per cent of the image area gave 0.08. A single added noise pattern gave 0.17. A black-box attack, without access to the classifier, gave 0.22. C7.01 Established

Note what those numbers mean. An AUC of 0.5 is chance. Four of the five attacks drove the detector below chance, which is a stronger result than breaking it: a detector at 0.0005 is not confused, it is inverted. Its output has become a reliable indicator of the opposite of the truth, and an institution acting on it would do better to flip every answer.

Two of those figures matter more than the headline. The single noise pattern at 0.17 is one perturbation, computed once, reusable without per-image optimisation. The black-box figure of 0.22 required no access to the model at all. Hold that beside the finding of Chapter 6, that detectors do not transfer across generator families, and the asymmetry of this field becomes visible. The attacks generalise. The defences do not.

Learning objectives

State the measured effect of white-box and black-box evasion on a strong detector, and explain what an AUC below 0.5 implies operationally.
Explain why the transferability of attacks and the non-transferability of detectors is a structural asymmetry rather than a coincidence.
Describe why adversarial perturbations survive the compression chain, and cite the evidence.
Identify the absent feedback channel in content authenticity, and contrast it with fraud.
Explain why “we have observed no evasion” is uninformative in this domain.
Decide what an institution should do given an unmeasurable evasion rate.

Figure 7.1 — Four of five attacks drive a strong detector below chance. Bars are drawn to scale from the figures reported in the paper. The black-box result is the operationally important one, because it assumes no access to the model. Source: C7.01.

0.0005AUC after LSB flipFrom an unattacked 0.95. The detector is inverted, not merely degraded.

0.22Black-box AUCAchieved without access to the classifier.

4 of 5Attacks below chanceWorse than a coin, by a wide margin.

NoneFeedback signalsA successful evasion produces no observable event. See Figure 7.2.

The objection that does not hold

The standard reply to adversarial results is that they are laboratory artefacts: a perturbation at the level of the least significant bit cannot possibly survive a platform upload, a transcode, a resize and a recompression. It is a reasonable objection and it has been tested.

Hussain, Neekhara and colleagues, at WACV 2021, built white-box and black-box pipelines that fool neural deepfake detectors into classifying fake videos as real, and state in their abstract: “We further demonstrate that our adversarial perturbations are robust to image and video compression codecs, making them a real-world threat.” C7.02 Established

An attacker who wants the perturbation to survive the pipeline can simply include the pipeline in the optimisation. There is nothing exotic about this. It is the same technique that makes adversarial road signs work in sunlight.

The asymmetry, stated plainly

	The attacker	The defender
Needs	One success, on one item	To hold on every item
Chooses	Which item, which generator, which moment	Nothing
Knows the opponent	Yes. Detector architectures and signals are published	No. The generator may be proprietary or unreleased
Transfers across instances	Yes. A universal pattern dropped AUC to 0.17; black-box to 0.22	No. See C6.02: failure even across diffusion models
Gets a feedback signal	Yes. The content either circulates or it does not	No. A successful evasion is indistinguishable from authentic media
Cost of iteration	Near zero. Recompute and resubmit	Retraining, revalidation, redeployment

Table 7.1 — Every row favours the attacker, but the fifth is the one without a precedent in ordinary security practice. Sources: C7.01, C7.02, C6.02; final row is analysis, C7.03.

The missing feedback channel

Security controls improve because failures are observable. A fraud control fails and money leaves an account, so somebody reconciles a statement and the organisation learns. A spam filter fails and a user sees the message. An intrusion detection system fails and, eventually, an incident is discovered. The loss signal is what converts an attack into an improvement.

Content authenticity has no such signal. When a detector is successfully evaded, the item is classified as authentic and nothing further happens. It is published, or believed, or filed, exactly as a genuine item would be. There is no reconciliation, no complaint, no breach notification, because from the institution’s side the two outcomes are identical. C7.03 Analysis

Figure 7.2 — Fraud has a loss signal. Authenticity does not. This is why an institution’s confidence in its detector tends to grow with time in service, which is precisely the wrong direction. Source: author’s analysis, C7.03.

The consequence is uncomfortable and worth stating without hedging. The real-world evasion rate of any deployed content detector is unmeasurable in principle by the organisation deploying it. Not merely unmeasured. There is no instrument, because the quantity has no observable consequence. An organisation that has run a detector for three years without incident has exactly the same evidence as an organisation that has been evaded weekly for three years.

Case study: five attacks on one classifier

Case study 7.1 · documented, primary source

Target: A classifier reaching 0.95 AUC on almost all existing image generators while trained on only one, which is a strong generalisation result by the standards of Chapter 6
White-box results: Least-significant-bit flip across all pixels: AUC 0.0005. One per cent of image area perturbed: 0.08. Single additive noise pattern: 0.17
Black-box result: AUC 0.22, without access to the classifier
Source: Carlini & Farid (2020), CVPR Workshops. Established

Why the target choice matters. This was not a weak or outdated model picked to make a point. It was a detector with the cross-generator property the field had been pursuing, which is to say the better the detector, the more there was to break.

What the universal pattern implies for cost. A single reusable noise pattern dropped the AUC to 0.17. That is not an attack requiring expertise per item; it is an attack requiring expertise once. The marginal cost of evading every subsequent item is the cost of adding a pattern to an image.

What it does not establish. One classifier, in 2020, under the authors’ own experimental conditions. Detectors have changed and some have been hardened. The result establishes the mechanism and its cheapness, not a current figure for any particular system.

The attack runs in both directions

The published work perturbs synthetic media so that detectors call it authentic. Nothing in the mechanism restricts it to that direction. The same perturbation technique can make authentic media classify as synthetic, and that version is arguably the more useful attack: it converts the defender’s own detector into a machine for discrediting real evidence, with an institutional score attached.

This is the liar’s dividend from Chapter 2, no longer a rhetorical move but an engineered one. An organisation that publishes detector verdicts has created an attack surface on authenticity itself. Analysis

What adversarial training does and does not buy

Hardening a detector against a specified perturbation class works, and it is worth doing. Two limits should be understood before relying on it. Robustness is relative to the threat model it was trained against, so an attacker who changes the perturbation family is outside the guarantee. And hardening generally costs clean accuracy, which interacts badly with Chapter 4: a small loss of specificity moves you a decade down the precision curve in Figure 4.2.

The result is a control that is more robust against the attack you anticipated and less useful against the base rate you actually have. Analysis

Practical scenario

Illustrative scenario · not a documented incident

Your board asks the obvious question: “Has our detector ever been beaten?” Your logs show 41 flags in two years, 38 confirmed synthetic on review, three false positives, and no reported incident of a synthetic item reaching publication.

Write the honest answer in three sentences. It should state what the 38 confirmations do establish, explain why the absence of known failures establishes nothing at all, and propose the one thing that would generate actual evidence. If your answer sounds reassuring, it is wrong.

Executive implications

Never report “no known evasions” as assurance. It is guaranteed output regardless of the truth. Reporting it as comfort is the single most common error in this area and it misleads boards in a specific, correctable way.
Generate your own evidence, since the environment will not. Red-team the detector deliberately: commission adversarial submissions and measure what gets through. This is the only way to obtain the loss signal Figure 7.2 shows the system does not produce.
Do not publish detector verdicts externally. A published score is an attack surface in both directions, and the inbound direction discredits authentic material.
Assume evasion is cheap for a motivated actor. A universal pattern at 0.17 AUC and a black-box attack at 0.22 are not state-actor capabilities.
Keep the control at Layer 4. Every finding in Part II points the same way: out-of-band verification and authorisation design do not have an evasion rate, because there is nothing to perturb.

Common misconceptions

Misconception 1 · “Evasion needs white-box access”

The black-box attack in the same study reached 0.22 AUC with no access to the classifier. Keeping a model private raises the attacker’s cost somewhat and does not approach a defence, and it also removes any possibility of independent scrutiny of the model you are relying on.

Misconception 2 · “Compression will wash the perturbation out”

Tested and reported otherwise: the WACV 2021 authors state that their adversarial perturbations are robust to image and video compression codecs, making them a real-world threat. An attacker who cares about surviving the pipeline includes the pipeline in the optimisation.

Misconception 3 · “We have seen no evasion, so it must be rare”

You would see no evasion either way. A successful evasion is classified as authentic and generates no event, no complaint and no reconciliation difference. The observation is uninformative, and treating a guaranteed observation as evidence is the error, not the conclusion.

Limitations and open questions

The figures are from 2020 and from one classifier. They establish mechanism and cost, not current performance of any system. A fair reading is that detectors have improved and so have attacks, and the asymmetry in Table 7.1 is unchanged by either.
No claim is made about how often evasion occurs in practice. The chapter argues such a claim is unavailable in principle from the defender’s side. Anyone asserting a field evasion rate should be asked how they measured it.
The both-directions attack is reasoned, not demonstrated here. The cited work perturbs synthetic media toward “authentic”. The reverse direction follows from the mechanism, and is marked as analysis rather than established.
Certified defences exist for narrow threat models. This chapter does not survey them. The relevant limitation is that certification is relative to a perturbation bound the attacker is under no obligation to respect.
Nothing here implies detection is worthless. Chapter 4 specified an envelope in which it earns its place. This chapter narrows that envelope where an adversary is deliberately present, which is not every deployment.

Teaching materials

Discussion questions

An AUC of 0.0005 is worse than useless. Is an inverted detector more dangerous than no detector, and to whom?
Why do attacks transfer across detectors when detectors do not transfer across generators? Is there a reason to expect this asymmetry to persist?
Design the cheapest honest experiment that would give your organisation a real evasion signal.
Should detector verdicts ever be made public? Consider both attack directions.
Adversarial training trades clean accuracy for robustness. Using Figure 4.2, what does that trade cost you at your prevalence?

Assessment questions

Explain what an AUC below 0.5 means and why it is qualitatively different from poor performance. (10 marks)
Construct the argument that a deployed detector’s evasion rate is unmeasurable, and identify the strongest objection to it. (20 marks)
Compare the feedback structure of a fraud control and a content detector, and derive two consequences for governance. (15 marks)

Practical exercise

Write the three-sentence board answer from the scenario above for a control your organisation actually operates. Then identify who currently believes the reassuring version.

Research assignment

Find a published claim about how often deepfake detectors are evaded in the real world. Trace it to its method. Report what you find, including if you find no method.

Seminar topic

“Controls whose failures are unobservable should not be relied upon for consequential decisions, however accurate they appear.” Argue either side, with examples from outside this field.

Chapter synthesis

Part II is now complete, and it has produced five independent reasons that a detector’s published performance does not constitute an institutional control. The test distribution is drawn from the training generators, and the ranking can invert. Precision collapses at institutional prevalence, as arithmetic. The operational channel removes the evidence, correctly and by design. The signal expires because generator authors improve their products, and the benchmark cannot see it. And an adversary who is actually aiming at the detector can drive it below chance for the cost of one reusable noise pattern, with perturbations that survive the codecs.

The fifth reason is the one that changes governance rather than procurement, and it is the absent feedback channel rather than the attack itself. Every other control an institution runs reports its own failures eventually, because failure has a consequence someone notices. This one does not. An evaded detector says “authentic” and the organisation proceeds, and the comfort that accumulates with years of quiet service is manufactured by the structure of the problem rather than earned by the control.

Against Figure 0.1, Layer 2 can now be closed out. Detection is an instrument with a narrow envelope, an unpublished expiry date, a cheap and transferable evasion path, and no means of telling you when it has failed. Part III turns to the threats that are arriving rather than established, and Part IV to the layers that do not share these properties: provenance, which attaches evidence to authentic material rather than hunting for traces in hostile material, and institutional process, which has held in every documented case in this book so far.