Chatbots, workflows and agents: what actually differs
Not model power but control: who picks the next step — the code, the model, or no one.

The three terms are usually read as a ladder of increasing intelligence. They are better read as one architectural question: who chooses what happens next. Anthropic draws the line itself — it will “draw an important architectural distinction between workflows and agents”. “Workflows are systems where LLMs and tools are orchestrated through predefined code paths”, while agents are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks”. A chatbot is the third case: a model call whose output is handed back as text, with no process to direct at all.
What separates the three is not the model: the same model can sit behind any of them. What changes is the arrangement of code around it, and this sheet fixes the one distinction the rest of this layer builds on.
One building block, three arrangements
All three shapes stand on the same substrate, which Anthropic calls the augmented LLM: “The basic building block of agentic systems is an LLM enhanced with augmentations such as retrieval, tools, and memory.” Adding tools does not by itself make something an agent; it makes the model able to act. The tool layer is a separate concern, reachable over a protocol — one way is to “integrate with a growing ecosystem of third-party tools with a simple client implementation” — and all three shapes can call the same tools.
Where control sits: predefined paths versus dynamic direction
A workflow fixes its sequence in code. OpenAI’s plain version: “A workflow is a sequence of steps that must be executed to meet the user’s goal.” The developer writes the steps; the model fills each one. Because the path is known in advance, a workflow is predictable and easy to evaluate — and it can only travel the paths someone wrote.
An agent hands the sequence to the model. OpenAI: “Agents are systems that independently accomplish tasks on your behalf.” Anthropic describes the mechanics: an agent will “plan and operate independently”, taking “ground truth” from the environment after each action, and it keeps control of the order until a stopping condition ends the run. The next step is chosen at run time, not at design time.
That is the whole axis: a workflow is model-assisted code, an agent is code-assisted model. When the next step is not written down, the system is an agent.
A chatbot is not a small agent
OpenAI removes the ambiguity: “Applications that integrate LLMs but don’t use them to control workflow execution–think simple chatbots, single-turn LLMs, or sentiment classifiers–are not agents.” A chatbot generates text; it does not drive a process. Retrieval and a knowledge base change what the chatbot reads, not whether it acts. Once a system starts acting on the conversation — issuing a refund, updating a record — the control question returns, and the familiar chat surface becomes the front end of a workflow or an agent. That is why the two coexist rather than compete: Anthropic observes that “customer support combines familiar chatbot interfaces with enhanced capabilities through tool integration”.
The workflow family, and its one dynamic member
“Workflow” is a family, not a single pattern: prompt chaining, routing, parallelisation, orchestrator-workers and evaluator-optimizer all keep the developer in charge of the shape. One member looks agent-like: in orchestrator-workers “a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results”. A model picks the decomposition — yet the frame around it (delegate, collect, synthesise) is still fixed in code. A model can make local choices inside a predetermined structure without the system becoming an agent. The boundary is a judgement about where the unpredictable decision sits, not a binary switch.
What each shape costs
Complexity is paid for on different lines. Anthropic: “Agentic systems often trade latency and cost for better task performance”, and it warns that “the autonomous nature of agents means higher costs, and the potential for compounding errors”. A workflow pays less at run time because its path is known; a chatbot pays least of all because it takes no action. The vendor advice points the same way — pick the simplest shape that works and move up only when a simpler one provably fails: “Start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short.” The label is not cosmetic; it predicts what has to be tested. A chatbot is judged on the quality of its answers, a workflow on the correctness of its fixed paths, an agent on whether its run-time decisions stay bounded and recoverable.
Level and prerequisites. L4 — the first sheet of the workflow-and-agents layer, and the one that fixes the vocabulary the rest of it uses; it stays conceptual and builds nothing. Prerequisites: the L1 sheet on who decides the next step in a model-backed application, and the L3 sheets on the difference between generated text and executed actions and on tool calling (schema, call and result).
Where to go next
- Automation & AI — the macro-area this sheet belongs to.
- Applications — the node this sheet sits in.
References
- Anthropic — Building effective agents — the workflow/agent distinction, the augmented LLM, the workflow-pattern family, and the cost and stopping-condition warnings.
- OpenAI — A practical guide to building agents — the definitions of workflow and agent, and the statement that simple chatbots are not agents.
- Model Context Protocol — Architecture — the tool layer as a protocol between hosts, clients and servers, separate from the control layer.