Generative AI 101

Overview

Generative AI 101 explains the two architectures behind almost every generative product. Autoregressive models such as GPT, Claude, Gemini, and Llama produce text one token at a time. A token is a small unit of text, and every additional token requires another pass through the model, which is why latency and cost grow with response length. Diffusion models such as Midjourney, Stable Diffusion, and Sora work differently: they start from random noise and refine it step by step into a coherent image or video. Multimodal systems connect the two by mapping text and images into a shared vector space.

The module then widens the frame. The model is only one component of a working system. Training data determines capability, so cleaning, deduplication, filtering of harmful content, and removal of personal information all happen before training starts. Parameters are the billions of internal weights that store learned patterns. The context window caps how much text a model can process in one request. Time to first token and tokens per second are the two inference metrics that drive most product latency decisions.

After reading, you will be able to ask which model family fits a task before anyone discusses prompts. You will know the three license categories, proprietary, open-weight, and open-source, and why most production systems route traffic across more than one model. You will also understand why Mixture of Experts designs deliver more capability per unit of compute at the cost of higher memory.

The failure modes are predictable. Teams pick one frontier model for every workload and spend their budget on trivial queries. They ignore context window limits until users see truncated answers. They treat scores on benchmarks such as MMLU and HumanEval as proof of production readiness, even though benchmark contamination means final selection must rest on your own evaluation data. Treat the model as replaceable; the data layer and the evaluation harness should outlast any single vendor.

In one paragraph

Generative AI products are built on one of two foundational architectures. Autoregressive models such as GPT, Claude, Gemini, and Llama generate text one token at a time, which is why latency and cost scale with response length. Diffusion models such as Midjourney, Stable Diffusion, and Sora start from random noise and iteratively denoise it into a coherent image or video. Modern systems combine these with multimodal capabilities by mapping text and images into a shared vector space. The model is only one piece; data pipelines, training methods, evaluation, safety, and serving infrastructure complete the system.

Key ideas

• 01

  A model's capability is determined by its training data. Cleaning, deduplication, filtering of harmful content, and removal of personally identifiable information all happen in the data pipeline before training begins.

• 02

  Parameters are the billions of internal weights that encode learned patterns. Larger parameter counts increase capability and memory footprint.

• 03

  Context window is the maximum amount of text a model can process in one request, measured in tokens. Everything must fit inside it.

• 04

  Inference performance is usually measured by time to first token and tokens per second. These two metrics drive product latency decisions.

• 05

  Foundation models come in three license categories: proprietary, open-weight, and open-source. Most production systems route traffic across more than one.

• 06

  Mixture of Experts architectures activate only a subset of parameters per request, giving more capability per unit of compute at the cost of higher memory.

• 07

  Benchmarks such as MMLU, HumanEval, and Chatbot Arena narrow the field, but benchmark contamination means final selection must use your own evaluation data.

How a senior leader uses this

• Ask which model family fits the task before discussing prompts.

• Require a routing plan that sends simple traffic to smaller, cheaper models.

• Treat the model as replaceable. The data layer and evaluation harness should outlast any single vendor.

Common failure modes

• Picking one frontier model for every workload and burning budget on trivial queries.

• Ignoring context window limits until users start seeing truncated answers.

• Treating benchmark scores as production readiness.