Articles

AI, machine learning, deep learning and LLMs: what nests inside what

How AI, machine learning, deep learning and LLMs nest — and what each term does not imply.

Reading: 5 minAutomation & AI

Article cover: AI, machine learning, deep learning and LLMs: what nests inside what

Four words are used as if they were interchangeable. They are not: they name progressively narrower things, and each one adds a condition the term before it does not require. Artificial intelligence (AI) is the widest label. NIST’s AI Risk Management Framework describes an AI system as “an engineered or machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, or decisions influencing real or virtual environments”, designed to operate with varying levels of autonomy (a definition the framework adapts from the OECD Recommendation on AI:2019 and ISO/IEC 22989:2022). Machine learning (ML) is one way of building such a system: the rules are learned from data instead of being written by hand. Deep learning (DL) is a kind of machine learning whose learned representations are layered. A large language model (LLM) is one kind of deep-learning model: a neural network trained on a large text corpus to predict the next token. Each step inward adds a requirement; none of the four words promises intelligence, correctness or quality.

The mental model: a nesting, not a ladder

AI              engineered or machine-based systems that can generate
                outputs (predictions, recommendations, decisions) toward
                objectives, with varying levels of autonomy.
                A knowledge base or a rule engine can be AI without
                learning anything.
  └─ ML         building such a system by learning from data instead of
                specifying the rules by hand.
     └─ Representation learning
     │          learning the features themselves, not only the mapping
     │          from features to answer.
     └─ DL      machine learning whose representations are composed in
                layers: each layer is built from the one below it.
          └─ LLM a deep-learning model specialised on text: a neural
                network trained to predict the next token of a corpus.

Read downwards, the labels get more specific; read upwards, each partial answer is still true. No arrow claims a more capable system. The order is a definition of scope, not a ranking.

Terminology the reader needs

  • AI system — the definition above. It is a system, and it does not require learning: a capability can be “rule-based or leverage additional machine learning”, as NIST’s Generative AI Profile puts it when it discusses content filters.
  • Machine learning — a technique that lets computer systems improve with experience and data; what it replaces is a human operator formally specifying all the knowledge the machine needs. The learned object (parameters, features, a mapping) is what separates ML from classical rule-based AI.
  • Representation — how data is described internally. In the classic presentation of the field, ML sits inside AI, and representation learning sits between ML and deep learning, because a plain ML algorithm may still depend on features a human chose.
  • Deep learning — representation learning with depth: representations expressed in terms of simpler representations, with successive layers extracting increasingly abstract features.
  • Large language model — GPT-3 was presented as “an autoregressive language model with 175 billion parameters”; the Generative AI Profile describes the family as models that “generate outputs that approximate the statistical distribution of their training data; for example, LLMs predict the next token or word in a sentence or phrase”.

Reading the nesting in the correct direction

The four terms are not four answers to one question; they are four consecutive questions, and each becomes meaningful only after the previous one is answered.

  1. Does the system produce outputs such as predictions, recommendations or decisions toward some objective? → AI.
  2. Were the rules that produce those outputs learned from data rather than written by a person? → ML.
  3. Are the learned representations layered, each built on simpler ones? → DL.
  4. Is it a neural network trained on a large text corpus with the objective of predicting the next token? → LLM.

That last objective is stated plainly in the standard formulation of language modelling: the model is trained to supply, for every prefix of a text, the token it judges most likely to come next. The objective itself and the generate-and-feed-back loop are the subject of the sibling sheet on tokens, tokenizers and autoregressive generation; what matters here is only that everything the model produces is downstream of that single learned objective.

What each term denotes, and what it does not

Term What it denotes The extra condition it adds What it does not imply
AI a class of systems that generate outputs toward objectives, with some autonomy none — it is the widest label that anything is learned, or that the system resembles a mind
ML building such a system by learning from data the rules are learned, not hand-written that it is neural, that it is “AI” in a strong sense, or that it is an LLM
DL machine learning with layered representations representations are composed in layers that depth means quality, or that every deep model is a language model
LLM a large, deep language model trained to predict the next token over a text corpus the training objective and the data domain general intelligence, a database, agency, or correctness

Limits, and the common conceptual error

The common conceptual error is to read the nesting as a ladder of quality. It is a ladder of conditions: “deep” says how the representations are built, not that they are better than shallower ones; “machine learning” says the rules were learned, not that the system is more capable than a rule-based one at a given job; “AI” says only that outputs are produced toward objectives, which a knowledge base and a rule engine already do. Asking “is this AI?” therefore settles almost nothing. The useful question is the narrower one, and for the systems this batch is about it is the LLM-level question: what objective was the model trained on, and over what data.

Symmetrically, none of the four words describes the system built around the model. An LLM is a model: weights and an architecture that predict tokens. The objectives, the autonomy and the decisions in the AI-system definition belong to the application that wraps it. That boundary — the model versus the application that decides what to do next — is another sheet’s subject and is deliberately not developed here.

Level and prerequisites

L1 — the vocabulary and the nesting only: what each term denotes and what it does not imply. Tokenization, decoding, the runtime stack and who chooses the next step are other sheets. Prerequisites: none beyond general IT literacy.

Where to go next

References

  • National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (January 2023) — the definition of an AI system and its adaptation from the OECD Recommendation on AI:2019 and ISO/IEC 22989:2022.
  • National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 (July 2024) — next-token prediction and the statistical distribution of training data; the rule-based versus machine-learning distinction.
  • Tom B. Brown et al., Language Models are Few-Shot Learners (GPT-3 paper, 2020) — GPT-3 as an autoregressive language model; in-context learning.
  • Nicholas Carlini et al., Extracting Training Data from Large Language Models (2021) — the training objective of a language model and the iterative sampling of generated text.