Synthetic Media, Deepfakes, and Digital Trust

An evidence-based account of what fails, what holds, and how each claim can be checked

For the sources behind specific claims, use the evidence ledger instead, which records evidence status and stated limits for each of the 47 claims.

1. Foundations of Synthetic Media Generation

  1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial networks. arXiv. arXiv:1406.2661
  2. Ho, J., Jain, A., & Abbeel, P. (2020). Denoising diffusion probabilistic models. arXiv. arXiv:2006.11239
  3. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. arXiv. arXiv:2112.10752
  4. Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. arXiv. arXiv:1812.04948
  5. National Institute of Standards and Technology. (2024). Reducing risks posed by synthetic content: An overview of technical approaches to digital content transparency (NIST AI 100-4). nvlpubs.nist.gov
  6. Coalition for Content Provenance and Authenticity. (2026). C2PA technical specification, version 2.4. spec.c2pa.org

Read chapter 1.

2. A Taxonomy of AI-Generated Disinformation Campaigns

  1. Chesney, R., & Citron, D. (2019). Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Review, 107. Origin of the liar’s dividend argument.
  2. International Press Institute. (2023). Slovakia: Deepfake audio of Denník N journalist offers worrying example of AI abuse. ipi.media
  3. Federal Communications Commission. (2024). Forfeiture order in the matter of Steve Kramer. See contemporaneous reporting at CyberScoop.
  4. AI Incident Database. Incident records for the Slovak audio and the New Hampshire robocall. incidentdatabase.ai
  5. National Institute of Standards and Technology. (2024). NIST AI 100-4: Reducing risks posed by synthetic content. nvlpubs.nist.gov

Read chapter 2.

3. Deepfakes as a Cybersecurity Threat Vector

  1. Financial Crimes Enforcement Network. (2024). Alert FIN-2024-Alert004: Fraud schemes using deepfake media, 13 November. fincen.gov
  2. Retool. (2023). Account of the August 2023 social-engineering incident. See also BleepingComputer for the authenticator-sync analysis.
  3. MIT Sloan Management Review. (2024). How Ferrari hit the brakes on a deepfake CEO. sloanreview.mit.edu
  4. Sophos News. (2019). Account of the UK energy firm voice-fraud case, including its insurer provenance. news.sophos.com
  5. AI Incident Database. Records 567 (Retool), 983 (WPP) and 1100 (LastPass). incidentdatabase.ai

Read chapter 3.

4. Deep Learning and Multimodal Deepfake Detection

  1. Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2019). FaceForensics++: Learning to detect manipulated facial images. ICCV. arXiv:1901.08971
  2. Dolhansky, B., et al. (2020). The DeepFake Detection Challenge (DFDC) dataset. arXiv:2006.07397
  3. Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020). Celeb-DF: A large-scale challenging dataset for DeepFake forensics. CVPR. openaccess.thecvf.com
  4. Haliassos, A., Vougioukas, K., Petridis, S., & Pantic, M. (2021). Lips don’t lie: A generalisable and robust approach to face forgery detection. CVPR. arXiv:2012.07657
  5. Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. CVPR. The backbone used as the FaceForensics++ detection baseline.

Read chapter 4.

5. Voice Cloning and Audio Deepfake Detection

  1. Wang, C., et al. (2023). Neural codec language models are zero-shot text to speech synthesizers (VALL-E). arXiv:2301.02111
  2. Müller, N., et al. (2022). Does audio deepfake detection generalize? Interspeech. isca-archive.org
  3. ASVspoof consortium. (2024). ASVspoof 5: Crowdsourced speech data, deepfakes and adversarial attacks at scale. arXiv:2408.08739
  4. Yamagishi, J., et al. (2021). ASVspoof 2021: Accelerating progress in spoofed and deepfake speech detection. arXiv:2109.00537
  5. ITU-T Recommendation G.711. Pulse code modulation of voice frequencies. The narrowband baseline referred to throughout this chapter.

Read chapter 5.

6. Benchmarking, Datasets and Cross-Domain Generalisation

  1. Durall, R., Keuper, M., & Keuper, J. (2020). Watch your up-convolution: CNN based generative deep neural networks are failing to reproduce spectral distributions. CVPR. arXiv:2003.01826
  2. Corvi, R., et al. (2023). On the detection of synthetic images generated by diffusion models. ICASSP.
  3. Yan, Z., Zhang, Y., Yuan, X., Lyu, S., & Wu, B. (2023). DeepfakeBench: A comprehensive benchmark of deepfake detection. NeurIPS Datasets and Benchmarks. github.com/SCLBD/DeepfakeBench
  4. Marra, F., et al. (2019). Do GANs leave artificial fingerprints? Early evidence for generator-specific traces.
  5. Dolhansky, B., et al. (2020). The DFDC dataset. arXiv:2006.07397 — for the rank-inversion result in Table 6.1.

Read chapter 6.

7. Adversarial Robustness and Evasion

  1. Carlini, N., & Farid, H. (2020). Evading deepfake-image detectors with white- and black-box attacks. CVPR Workshops. openaccess.thecvf.com
  2. Hussain, S., Neekhara, P., Jere, M., Koushanfar, F., & McAuley, J. (2021). Adversarial deepfakes: Evaluating vulnerability of deepfake detectors to adversarial examples. WACV. cseweb.ucsd.edu
  3. Neekhara, P., et al. (2021). Adversarial threats to deepfake detection: A practical perspective. On transferability across detectors.
  4. ASVspoof consortium. (2024). ASVspoof 5. arXiv:2408.08739 — for degradation under adversarial attack in the audio case, C5.03.
  5. Durall, R., Keuper, M., & Keuper, J. (2020). arXiv:2003.01826 — for the contrast between obsolescence and evasion, C6.01.

Read chapter 7.

8. LLM-Generated and Multimodal Disinformation

  1. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7). cell.com/patterns
  2. Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023). Can AI-generated text be reliably detected? arXiv:2303.11156
  3. OpenAI. (2023). New AI classifier for indicating AI-written text, and its subsequent withdrawal. openai.com
  4. Kirchenbauer, J., et al. (2023). A watermark for large language models. ICML. The scheme whose paraphrase vulnerability is reported in C8.03.
  5. Carlini, N., & Farid, H. (2020). CVPR Workshops — for the parallel evasion result in the image case, C7.01.

Read chapter 8.

9. Emerging Generative Threats

  1. Federal Bureau of Investigation, IC3. (2025). Public service announcement I-051525-PSA, on smishing and AI-generated vishing impersonating senior US officials.
  2. OpenAI. (2025). Sora 2 is here, 30 September. openai.com/index/sora-2
  3. Financial Crimes Enforcement Network. (2024). Alert FIN-2024-Alert004. fincen.gov — for the population-level indicator approach, C3.06.
  4. AI Incident Database. Record 1077, on the FBI-reported campaign. incidentdatabase.ai
  5. Wang, C., et al. (2023). arXiv:2301.02111 — for the three-second cloning cost underlying row two of Figure 9.1, C5.01.

Read chapter 9.

10. Content Provenance and Watermarking

  1. Coalition for Content Provenance and Authenticity. C2PA technical specification, version 2.x, and the Content Credentials explainer. spec.c2pa.org
  2. ISO/DIS 22144, Authenticity of information — Content credentials. The standardisation route for the C2PA v2 architecture.
  3. Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., & Feizi, S. (2024). Robustness of AI-image detectors: Fundamental limits and practical attacks. ICLR. arXiv:2310.00076
  4. Google DeepMind. SynthID, and the Nature publication of SynthID Text, October 2024. deepmind.google
  5. Sadasivan, V. S., et al. (2023). arXiv:2303.11156 — for the text-watermark paraphrase result, C8.03.

Read chapter 10.

11. The Legal and Regulatory Landscape

  1. Regulation (EU) 2024/1689 (AI Act), Article 50, and Regulation (EU) 2026/1744 (Digital Omnibus on AI). See the Commission’s FAQ on Article 50 transparency obligations. digital-strategy.ec.europa.eu
  2. TAKE IT DOWN Act, Public Law, 19 May 2025. Congressional Research Service product LSB11314. congress.gov
  3. Federal Trade Commission. (2026). Take It Down Act enforcement starts now. ftc.gov
  4. Advisory Committee on Evidence Rules. Proposed FRE 707 and Rule 901(c), including the May 2026 withdrawal and continuing study.
  5. Chesney, R., & Citron, D. (2019). California Law Review 107 — for the liar’s dividend that Rule 901(c) would formalise, see C2.03 and Chapter 2.

Read chapter 11.

12. Building an Organisational Defence Framework

  1. Federal Bureau of Investigation, IC3. (2025). Public service announcement I-051525-PSA. The clearest official statement of the top-ranked control, C9.01.
  2. Financial Crimes Enforcement Network. (2024). Alert FIN-2024-Alert004. fincen.gov — the population-level indicator model behind control 8.
  3. Coalition for Content Provenance and Authenticity. C2PA technical specification. spec.c2pa.org — for controls 3 and 5.
  4. Liang, W., et al. (2023). Patterns 4(7). cell.com/patterns — the basis for the withdrawal at rank 6.
  5. Carlini, N., & Farid, H. (2020). CVPR Workshops. openaccess.thecvf.com — why control 7 is necessary wherever control 9 is used.

Read chapter 12.