Chapter 3 of 1213 min read

Deepfakes as a Cybersecurity Threat Vector

Fraud, identity theft and social engineering. The documented record contains no case in which detecting the fake is what saved the money.

The question that broke a deepfake

In July 2024 an executive at Ferrari received WhatsApp messages apparently from the chief executive, Benedetto Vigna, about an urgent confidential acquisition, followed by a call in a convincing imitation of his voice. The executive did not analyse the audio. He said, in substance, that he needed to identify the caller, and asked which book Vigna had recommended to him a few days earlier. The call ended abruptly. C3.01 Established

That question is the most instructive artefact in this chapter, because of what it did not require. It required no detector, no forensic tool, no judgement about whether the voice sounded right, and no training in synthetic media. It required one fact that both parties held and the attacker could not obtain, asked by the party under pressure rather than offered by the party applying it.

Read the whole documented record of corporate deepfake fraud and the same shape appears. The synthetic artefact never performs the fraud. It supplies authority: it makes an instruction appear to come from someone entitled to give it. The money moves, or does not move, further down the chain, at the point where an organisation decides whether an instruction is authorised. That is the attack surface, and it is a process, not a perception.

Learning objectives

Locate the point in an authorisation path at which synthetic media actually operates, and distinguish it from the point at which loss occurs.
Explain why no documented corporate case was resolved by detecting the forgery, and what resolved each one instead.
Design an out-of-band verification challenge, and identify the three ways such challenges are commonly built wrong.
Map where synthetic media enters an identity-verification pipeline, using the FinCEN red-flag set as a reference.
Distinguish impersonation fraud from synthetic-identity fraud, which has no victim to call.
Assess an existing payment or access-approval control for single-channel dependency.

The authorisation path

An instruction that moves money or grants access passes through five stages. It is worth setting them out plainly, because the habit of discussing deepfakes as a media problem obscures the fact that only one stage is about media at all.

Identity claim. Someone asserts who they are. This is the only stage synthetic media touches.
Channel. The claim arrives somewhere: a phone line, a messaging app, a video call, an email thread.
Request. An instruction is made, usually with urgency and confidentiality attached.
Authorisation. Someone or something decides the instruction is legitimate.
Execution. The payment leaves, or the credential is handed over.

A forged identity claim is only consequential if stages 2 to 4 permit the same channel that carried the claim to also satisfy the authorisation. Where they do, a good enough voice is sufficient. Where they do not, the quality of the voice is irrelevant. This is why the cases that failed did not fail at stage 1.

Figure 3.1 — Synthetic media attacks stage 1; losses are decided at stage 4. The three cut points are the only interventions that appear in the record as having worked, and none of them requires an opinion about the artefact. Source: author’s analysis of the cases in Table 3.1.

The documented record

The table below is restricted to incidents with a named organisation and a primary or first-party source. It excludes the large volume of vendor-reported figures on deepfake fraud growth, which are discussed under limitations.

Case	Channel	Outcome	What decided it	Detection involved?
UK energy firm March 2019	Telephone	€220,000 lost	One caller, one channel, both initiating and authorising	No
Retool August 2023	SMS then voice call	27 customer accounts compromised	Cloud-synced one-time codes collapsed two factors into one	No
Arup (Hong Kong) early 2024	Multi-party video call	HK$200m, about US$25.6m, in 15 transfers	A video call accepted as sufficient authorisation	No
LastPass April 2024	WhatsApp voice messages	No loss	Channel was outside policy; employee reported it	No
WPP May 2024	WhatsApp then Teams, voice clone plus video footage	No loss	The request itself was anomalous	No
Ferrari July 2024	WhatsApp then call	No loss	Out-of-band shared-secret question	No

Table 3.1 — Documented corporate cases with a named organisation. The final column is uniform, and that uniformity is the chapter’s finding rather than an artefact of selection: no case in this set turned on identifying the media as synthetic. Sources: see claims C3.01 to C3.05 and C1.01.

0 of 6Decided by detectionNo documented case was resolved by anyone identifying the artefact as synthetic.

HK$200mArup, 2024About US$25.6m across fifteen transfers after a synthetic video call.

27Retool customersCompromised after one employee surrendered a second one-time code.

9FinCEN red flagsIndicators published for deepfake-enabled identity fraud, November 2024.

Case study: Retool, and why the deepfake was the least important part

Case study 3.1 · documented, first-party account

Event: Attacker phished Retool staff by SMS posing as internal IT, then called one employee using a synthetic voice resembling a known colleague
Date: August 2023, disclosed by Retool in September 2023
Mechanism: The employee, already suspicious, supplied an additional one-time code. Because the authenticator application synchronised codes to a cloud account, control of that account yielded the codes
Impact: 27 cloud customer accounts compromised, concentrated in cryptocurrency firms
Evidence status: Established first-party disclosure plus contemporaneous reporting

Why it belongs in a cybersecurity chapter rather than a media one. The synthetic voice bought the attacker one additional code from an employee who was already uneasy. On its own that should have been survivable. It was not survivable because a design decision elsewhere had quietly removed a factor: one-time codes synchronised to a cloud account are no longer a second factor once that account is compromised, and the organisation had not registered that change.

The general lesson. Synthetic media does not create new failures so much as collect on failures already present. An authorisation path with a single effective factor was vulnerable to an ordinary pretext call; the voice clone merely raised the success rate. Treating this incident as a deepfake story leads to detector procurement. Treating it as an authorisation story leads to the control that would actually have held.

Designing an out-of-band challenge

The Ferrari question worked for structural reasons that are worth stating, because challenges are routinely built without them.

The fact was shared and unpublished. A private exchange days earlier, not a biographical detail available online. Mother’s maiden name and date of birth fail this test; so does anything in a public profile.
The suspicious party chose the question. The person under pressure set the test. A challenge offered by the caller, or drawn from a list the caller could anticipate, is theatre.
It was cheap and immediate. One sentence, no tooling, no delay to justify to a superior.

For higher-value requests, the stronger form is a callback on a number held in your own directory, not one supplied in the message, followed by a second approver contacted separately.

Three ways out-of-band verification is built wrong

The callback number comes from the attacker. A number in the signature, the message, or the meeting invitation is in-band. Only a number you already hold is out of band.

The second approver is reachable only through the same thread. If the attacker controls the channel, they can supply both parties to the conversation.

The challenge is answerable from public sources. Most organisations publish enough about their people to answer the questions they use to identify them.

Identity theft and synthetic identity

Impersonation fraud has a real person behind the claim, which means there is someone to call. Synthetic-identity fraud does not. Here generative tooling is used to manufacture a customer who never existed, through forged identity documents, a face that passes a selfie comparison, and in some cases a live response that passes a liveness check. There is no victim to telephone, and the fraud is only visible as a pattern across accounts.

The United States Financial Crimes Enforcement Network published an alert on this on 13 November 2024, after an increase in suspicious activity reports describing deepfake media in fraud against financial institutions and their customers. It sets out nine red-flag indicators and a reporting key term. C3.06 Established It is a useful reference precisely because it is a supervisory document rather than a vendor paper: the indicators describe inconsistencies between an identity document and the device or geography presenting it, rather than claims about detector accuracy.

Figure 3.2 — Four entry points, and what each one looks like from the institution’s side. Note that the published indicators are mostly inconsistencies between artefacts, not verdicts about an artefact. Source: author’s arrangement of the FinCEN red-flag categories, C3.06.

Practical scenario

Illustrative scenario · not a documented incident

Your treasury team receives a video call from a number matching the group finance director, who appears on camera with two colleagues from a subsidiary. The call concerns a competitor acquisition under embargo. The director instructs a payment to an escrow account, says that legal has approved it, and asks that nothing be put in writing until the announcement. All three faces and voices are familiar. Your payments policy requires a second approver for amounts above a threshold; the director is the second approver.

Consider: at which stage of Figure 3.1 this attack is operating; what the policy’s second-approver requirement is actually worth here; and what single change to the policy would have made the call irrelevant without anyone having to judge the video.

Executive implications

Audit for single-channel authorisation. For every payment and access-approval path, ask whether the channel that carries the request can also satisfy the approval. Wherever it can, the control is one convincing voice away from failing.
Make confidentiality a red flag, not an override. Every case in Table 3.1 attached secrecy to urgency. A policy that allows an embargo to suspend verification has inverted its own purpose.
Check what your second factor still is. Retool is the warning: a factor can be removed by a product change elsewhere without anyone recording the loss.
Do not train staff to spot deepfakes. Train them to verify out of band regardless. The record shows the second skill works and gives no evidence that the first one does.
Treat synthetic identity separately. It has no victim to contact, so it is detected across accounts rather than within a call, which makes it a monitoring problem rather than a verification one.

Common misconceptions

Misconception 1 · “Better detection would have prevented these losses”

Not on this evidence. Three cases failed for the attacker, and in none of them did anyone identify the artefact as synthetic. They were stopped by a shared-secret question, by a channel being off-policy, and by the request being anomalous. A detector at stage 1 would have had to be consulted, trusted and fast, and in each case something cheaper worked first.

Misconception 2 · “A video call is strong verification”

The largest documented loss came through the most convincing-looking channel. Video presence feels like identity evidence because it historically was; the Arup case establishes that it is no longer, and the comfort it provides now works for the attacker.

Misconception 3 · “This is an IT security problem”

The losses occurred in finance processes and in helpdesk procedure. Information security owns the channel, but it does not own the payment approval matrix, the delegation thresholds, or the rule that permits an embargo to bypass them. A programme that sits entirely inside IT cannot reach the control point.

Limitations and open questions

The sample is small and biased toward disclosure. Six named cases are what the public record supports. Organisations that lost money and did not say so are absent, and it is reasonable to assume they outnumber those that spoke. The uniform final column of Table 3.1 should be read as a strong absence of evidence for detection-led saves, not as proof that none has ever occurred.
Prevalence figures are unreliable. Widely circulated growth rates for deepfake fraud originate largely with vendors selling detection or verification, are rarely accompanied by a published method, and are excluded here. No percentage increase is asserted in this chapter.
The 2019 case is thinly sourced. It is the most-cited early example and rests on an insurer’s account of a claim, with the company unnamed and the use of AI described as an inference by analysts rather than an established finding. It is retained because it is historically important and is marked accordingly.
Liveness-attack evidence is weaker than the vendor literature implies. Figure 3.2 describes where attacks enter; it does not quantify success rates against specific commercial systems, and the published independent evidence for such rates is thin.

Teaching materials

Discussion questions

If no documented case was decided by detection, what should an organisation buy first, and why is that not the common answer?
Ferrari’s executive asked about a book. Construct a challenge that would work in your own organisation, then explain who could defeat it.
Retool lost a factor without noticing. What else in your stack could be silently downgraded by someone else’s product decision?
Why does synthetic-identity fraud need a different control model from impersonation fraud?
Should confidentiality requests be allowed to suspend verification under any circumstances? If yes, specify them precisely.

Assessment questions

Using Figure 3.1, explain why artefact quality is a weak predictor of loss. (10 marks)
Design an out-of-band verification standard for payments above a threshold, and identify its three most likely failure modes. (20 marks)
Compare the control implications of impersonation fraud and synthetic-identity fraud. (15 marks)

Practical exercise

Take one real payment-approval path in your organisation and mark on it the five stages of Figure 3.1. Identify every point at which the same channel could carry both the request and the approval. Count them. In most organisations the number is not zero, and nobody has previously counted it.

Research assignment

Locate a corporate deepfake fraud case with a first-party account published by the affected organisation. Establish what stopped it or failed to, and whether any party made a judgement about the authenticity of the media. Report the evidence quality, not only the narrative.

Seminar topic

“Deepfake awareness training is worse than useless, because it teaches staff to trust their own judgement about artefacts.” Argue either side.

Chapter synthesis

The uniform column in Table 3.1 is the chapter in one line. Across six documented corporate cases, not one outcome turned on anybody deciding that a voice or a face was synthetic. The three that failed were stopped by a question about a book, by a messaging app being outside policy, and by a request that did not fit. The three that succeeded succeeded because a single channel was permitted to both make and authorise an instruction.

This reframes the problem in a way that should be uncomfortable for the market that has grown around it. Synthetic media is an attack on stage 1 of a five-stage path. Losses are decided at stage 4. Investment aimed at stage 1 competes with a generator that improves continuously; investment at stage 4 competes with nothing, because the authorisation rule either holds or it does not, and it does not degrade when the models improve.

Against Figure 0.1, the argument from Chapters 1 and 2 now hardens. Chapter 1 showed that Layer 2, detection, cannot carry the weight placed on it. Chapter 2 showed that for timing attacks the binding constraints sit at Layers 4 and 5. Chapter 3 shows something narrower and more actionable: in the fraud case, Layer 4 is not merely where the constraint sits but where the entire outcome is determined, and the controls that work there are old, cheap and already understood. The organisation that cannot answer “can the channel that made this request also approve it?” has a synthetic media problem no detector will solve. That question is the subject of Chapter 12.