Chapter 9 of 1214 min read

Emerging Generative Threats

Real-time cloning, text-to-video, agentic impersonation and synthetic identities. A stress test of this book’s own recommendation against the capabilities arriving next.

An advisory that arrives at the same control

On 15 May 2025 the FBI’s Internet Crime Complaint Center issued public service announcement I-051525-PSA, describing a campaign active since April that sent smishing texts and AI-generated vishing voice messages impersonating senior US federal and state officials. The targets were current and former officials and the people in their address books, and the objective was to move recipients onto other platforms and links in order to harvest credentials and take over accounts. C9.01 Established

The advisory’s own recommended protection is to verify any unsolicited message claiming to come from an official by calling a known number.

That is the control this book derived in Chapter 3 from the fraud record, and again in Chapter 5 from the physics of the telephone channel. A federal law-enforcement body facing an active campaign arrived at it independently. This chapter is where that convergence gets tested, because the capabilities now arriving are aimed at precisely the human judgements the control does not depend on, and it is worth being exact about which is which.

Learning objectives

Distinguish controls that rest on content judgement from controls that rest on channel control.
Explain why real-time synthesis defeats the first category and not the second.
List the human verification heuristics already consumed by synthesis capability, and the ones still intact.
Describe how agentic tooling changes the economics of social engineering rather than its mechanism.
Identify the single case in which this book’s recommended control has no counterparty, and state what replaces it.
Assess an emerging capability for which of your controls it actually consumes.

Content judgement versus channel control

Every verification practice in use falls into one of two families, and the distinction decides what survives.

Content judgement asks somebody to assess the artefact. Does this sound like her? Does the face move correctly? Is the lighting consistent? Did the caller answer my question fluently? Every member of this family has a generative capability pointed at it, and each capability that lands removes a heuristic permanently.

Channel control asks nothing about the artefact. I ended the call and dialled a number from my own directory. The approval requires a second person I contacted separately. The challenge requires a fact that is not published. These do not improve or degrade with synthesis quality, because the artefact is not the evidence. The evidence is the path.

Read the FBI advice again with that distinction in hand. “Calling a known number” is not advice about listening carefully. It is advice about who initiates and to what address, and it would hold if the impersonation were flawless.

Figure 9.1 — The heuristics being consumed are all of one kind. Each consumed row is tied to an established claim elsewhere in this book. The intact rows are intact because synthesis quality is irrelevant to them. Source: author’s analysis over the cited claims, C9.03.

I-051525FBI IC3 advisory15 May 2025. Campaign active since April, targeting officials and their contacts.

Address booksThe real targetImpersonation of officials aimed at the people who trust them.

4 of 4Consumed heuristicsEvery verification practice lost so far was a judgement about the artefact.

1Genuine gapSynthetic identity, where there is no real person to call back.

What each arriving capability actually consumes

Capability	Breaks content judgement?	Breaks channel control?	Evidence status
Real-time face and voice interactive impersonation	Yes, completely. Removes the last perceptual test, interactive response	No. It still has to reach you on a channel you did not choose	Established operationally by C1.01
Text-to-video with audio synchronised dialogue, likeness features	Yes for broadcast material; production quality is no longer a signal of provenance	Not applicable. This is a publication threat, not an authorisation threat	Established capability, C9.02
Agentic impersonation autonomous multi-turn social engineering	Yes, and adds persistence and patience that a human operator cannot afford	No, but it makes attempting every channel cheap, so coverage gaps get found	Emerging — anticipated from capability, not documented at scale
Synthetic identities people who never existed	Yes	Yes. There is no genuine party to call back, so the control has no counterparty	Established as a supervisory concern, C3.06

Table 9.1 — Three of four arriving capabilities consume perception and leave channel control standing. The fourth is the exception, and it is the exception because it removes the person rather than imitating them. Source: author’s analysis, C9.03.

Does real-time cloning kill the callback?

This is the question the chapter exists to answer, and the honest answer requires separating two things the callback was doing.

It was never doing what people assume. Its strength is not that a live conversation is hard to fake, and a book that recommended it on those grounds would now have to withdraw the recommendation. Real-time synthesis is operationally proven: the Arup transfers followed a multi-party video call in which several apparent colleagues were synthetic. Interactivity as a perceptual test is finished. C1.01

What the callback actually does is shift the attacker’s task from producing convincing content to controlling a channel you selected. To defeat a callback to a number held in your own directory, an attacker must compromise the number, the directory, or the network, which is a materially different and more expensive operation than cloning a voice from a conference talk. The quality of the clone they can produce once you reach them is irrelevant, because you are no longer using the clone as evidence.

What “out of band” has to mean, stated precisely

Three properties, all required. Weakening any one of them returns you to content judgement.

You initiate. Not a transfer, not a hold, not a number they read out. You end the contact and start a new one.
To an address you already held. From your own directory or records, obtained before this interaction began.
On a different channel from the request. A request by message is verified by voice; a request by voice is verified in writing to a known address.

Note that none of the three mentions the voice, the face, the wording or the plausibility. That is the point, and it is why this control is the one still standing in Figure 9.1.

Agentic impersonation: an economic change, not a new mechanism

Autonomous agents that hold a multi-turn conversation, maintain a pretext across days, consult what they have been told and act on a system do not introduce a new attack. They change the price of the attack documented in Chapter 3.

Social engineering has been expensive because it needed a skilled human per target, improvising in real time, with limited patience. That cost is why the documented cases are a handful of high-value attempts against named executives rather than a flood. Remove the per-target human and the attribution attack of Chapter 2 acquires the volume property of the saturation attack, and the authorisation paths of Chapter 3 get probed exhaustively rather than once.

Marked as anticipated, not documented

No case known to this book establishes an autonomous agent conducting end-to-end impersonation fraud at scale. The reasoning above is an inference from available capability and from the economics of the documented cases, and it is logged as Emerging for that reason.

Two consequences follow if it is right, and both are testable rather than rhetorical. Attack volume against mid-value targets should rise faster than against executives, because the saving is in the operator rather than the research. And coverage gaps should start to matter more than control strength, because an attacker who can try every path cheaply will find the one path you forgot. An organisation can watch for the first and audit for the second now, without waiting for the literature.

Synthetic identities: the one real gap

Here the recommended control fails, and it is worth saying so plainly rather than stretching it. A callback verifies that the person who contacted you is the person you believe them to be. When the identity is manufactured, there is no genuine person behind it and nothing for a callback to reach. The documents are generated, the face passes comparison, and the whole apparatus of out-of-band verification has no counterparty.

What replaces it is not verification but monitoring, and the shift is from the individual interaction to the population. FinCEN’s indicators are instructive on exactly this point: they are predominantly inconsistencies between artefacts and behaviours, such as device or geographic data that does not match the identity document presented, or a newly opened account with rapid volume to high-risk payees, rather than verdicts about whether a document is real. C3.06

The institutional implication is uncomfortable for the structure of most organisations. Impersonation is caught at the point of interaction, by whoever is being spoken to. Synthetic identity is caught across accounts, over time, by a function that sees the whole population. Those are different teams, usually different systems, and often different budgets.

A capability worth noticing for a different reason

Text-to-video with synchronised dialogue arrived in mainstream products in 2025: Sora 2, released on 30 September, generates video with synchronised dialogue, sound effects and lip-synced speech, and includes a likeness feature in which a person uploads a short reference video so that their face and voice can be used, under a consent framework giving them visibility of videos featuring them and the ability to revoke access. C9.02 Established

The generative capability is the expected news. The consent framework is the more interesting development, because it is the first mainstream instance of a likeness being handled as a permissioned asset with an audit trail and revocation, implemented inside a generation product rather than imposed on it. That is a governance control at the point of synthesis, which is where Chapters 10 and 11 argue the durable answers live. Whether such frameworks are robust, and what happens to likeness rights when they are not, is Chapter 11’s subject.

Practical scenario

Illustrative scenario · not a documented incident

A member of your finance team receives a message from the group CFO’s known mobile number asking them to join a video call. On the call the CFO appears, responds naturally to questions, and asks for a payment. Your policy requires out-of-band verification, and the staff member reasons that the number was already in their contacts and the person answered questions in real time, so both tests are satisfied.

Identify the two errors against the three properties in the tip box above. Then decide whether your policy wording would have prevented them, or whether it merely says “verify out of band” and leaves the staff member to interpret it.

Executive implications

Rewrite “verify out of band” into the three explicit properties. Most policies state the principle and leave the staff member to infer the mechanics, which is where the scenario above goes wrong. You initiate, to an address you already held, on a different channel.
Stop investing in perceptual training. Four for four, the heuristics consumed so far were content judgements. Training staff to spot synthetic media is training in a skill with a known expiry.
Protect the address book, not only the principal. The FBI campaign targeted officials’ contacts. Your executives’ counterparties are your attack surface, and they are outside your controls.
Audit for coverage gaps, not control strength. If agentic tooling makes attempting every path cheap, the forgotten path matters more than the hardened one. Enumerate paths before strengthening any.
Assign synthetic identity to a population-level function. It cannot be caught in the interaction, so giving it to the people who answer the phone guarantees it is missed.

Common misconceptions

Misconception 1 · “Real-time deepfakes make verification impossible”

They make perceptual verification impossible, which was already failing. They have no effect on who initiates a call or which directory supplied the number. The conflation matters because it leads organisations to conclude that nothing works, at the moment when the thing that works has just been endorsed by a federal advisory.

Misconception 2 · “A call from a known number is verified”

Caller identification is assertable by the caller, and spoofing featured in the New Hampshire case in Chapter 2. An inbound call displaying a known number satisfies none of the three properties: you did not initiate, you did not select the address, and the channel is the one the attacker chose.

Misconception 3 · “Agentic attacks are science fiction”

The mechanism is the documented one from Chapter 3, with the human operator removed. That is an economic change, and economic changes in attack cost have historically produced volume changes rather than novel incidents. It is logged here as anticipated rather than observed, with two specific predictions attached so that it can be checked rather than believed.

Limitations and open questions

This is the most speculative chapter in the book, and it is marked as such throughout. Two claims are established from primary sources. The agentic argument is explicitly Emerging, and the heuristic ledger is analysis over claims established elsewhere.
No prevalence or growth figures appear. Emerging-threat discussion is where vendor statistics are least reliable and most abundant. None are used, and the absence is deliberate.
Real-time synthesis quality is not characterised. Latency, failure modes under interruption and robustness to unexpected questions are the properties that would decide how well interactive impersonation holds up, and they are not consistently published.
The consent-framework observation is about design, not efficacy. Whether likeness permissioning is robust in practice, or circumvented by other tooling, is untested here.
Agentic predictions are falsifiable and unfalsified. The two stated consequences have not been measured. If attack volume against mid-value targets does not rise, the argument is wrong and should be withdrawn.

Teaching materials

Discussion questions

Why were all four consumed heuristics content judgements? Is there a reason to expect the next one to be too?
The FBI reached this book’s recommendation independently. Why is that stronger evidence than either arriving at it alone?
Does a likeness consent framework inside a generation product represent governance, or the appearance of it?
Synthetic identity breaks the recommended control. What would it take for your organisation to detect it at population level?
If agentic tooling makes attempting every path cheap, is your weakest path or your strongest control the better thing to measure?

Assessment questions

Define content judgement and channel control, and classify six verification practices from your own organisation. (15 marks)
Explain why real-time synthesis does not defeat a callback, and identify the conditions under which it would. (15 marks)
State the agentic-impersonation argument, its evidence status, and the observation that would refute it. (10 marks)

Practical exercise

Find your organisation’s written guidance on verifying unusual requests. Check it against the three properties. Most such guidance says “verify independently” and specifies none of them, which means the control exists on paper and is improvised in practice.

Research assignment

Locate a documented case of autonomous agentic impersonation conducting an end-to-end fraud. Report the evidence quality honestly, including if the strongest available source is a vendor blog post.

Seminar topic

“Deepfake awareness training should be discontinued and the budget moved to authorisation design.” Argue either side, using Figure 9.1.

Chapter synthesis

Part III was supposed to be the chapter where the book’s recommendation came under pressure, and in one respect it did. Interactivity is gone as a perceptual test, and gone with operational proof rather than in principle: the Arup transfers followed a synthetic multi-party video call. Anyone whose verification rests on a live exchange feeling right has lost the control they thought they had.

But the pattern across the four arriving capabilities is consistent, and it is not an accident. Synthesis competes with perception. It has no purchase on who initiated a call, which directory supplied the number, whether the approver was contacted separately, or whether a challenge requires an unpublished fact. Every heuristic consumed so far belonged to the first category and every heuristic still standing belongs to the second, which is why a federal advisory confronting an active campaign recommends calling a known number rather than listening harder.

Two honest qualifications keep this from being a comfortable conclusion. Agentic impersonation is logged as anticipated rather than documented, with two falsifiable predictions attached, and if attack volume against mid-value targets does not rise the argument should be withdrawn. And synthetic identity genuinely breaks the recommendation, because manufacturing a person who never existed leaves nobody to call back. That case moves from verification to population-level monitoring, which in most organisations is a different team with a different budget, and it is the clearest instance in this book of a threat whose control does not sit where the threat is noticed.

Against Figure 0.1, Part III closes with Layers 1 and 2 settled and the weight now entirely on Layers 3 to 5. Part IV takes them in turn: provenance, which attaches evidence to authentic material instead of hunting traces in hostile material; the legal and regulatory position, including whether a remedy ordered is ever a remedy collected; and the organisational framework that the last nine chapters have, one case at a time, been specifying.