LLM vs RAG vs Agent vs Agentic
You have now met all four: a model that writes, a model that looks things up first, an agent that plans and acts, and a team of agents that divide the work. This lesson puts them side by side — what each actually is, what it costs, what it breaks on — and answers the question that matters when you are the one building: which of these does my problem actually need? The answer is almost always simpler than it feels.
Define each of the four architectures and draw its request flow; explain how each one builds on the previous; compare them on tools, memory, planning, autonomy, cost, and build time; pick the right architecture for a given problem and justify it; and articulate why starting at the highest rung is the most common and most expensive mistake in AI engineering.
Think about getting a question answered. (1) You ask a clever friend who answers from memory — that’s an LLM. (2) Your friend checks the actual book first, then answers — that’s RAG. (3) Your friend goes off, looks things up, tries stuff, and comes back when the whole job is done — that’s an agent. (4) Your friend brings a whole team, each person doing the bit they’re best at — that’s agentic AI. Each one can do more… and each one is slower, pricier, and easier to get wrong.
One ladder, four rungs
☺ Like you’re 10: These aren’t four different animals — they’re one animal growing up. Each step keeps everything from the step before and adds one new ability.
The most useful thing to understand first: these are not four competing technologies. Each one contains the previous. RAG is an LLM with retrieval bolted on. An agent is an LLM with tools and a loop. Agentic AI is several agents with a coordinator. The LLM is the engine in all four — what changes is how much scaffolding you build around it.
What each one is
☺ Like you’re 10: Same four friends as before — here’s exactly what each one has in its backpack.
| LLM | RAG | AI Agent | Agentic AI | |
|---|---|---|---|---|
| In one line | A text generator that answers from patterns it learned in training. | Searches your documents first, then answers grounded in what it found. | Plans steps, picks tools, and completes a whole task autonomously. | A coordinated team of specialist agents collaborating on a large job. |
| Core parts | One pretrained model. | LLM + vector database + embedding model + ingestion pipeline. | LLM + tool registry + working memory + a feedback loop. | Several specialized agents + shared memory + an orchestrator + inter-agent messaging. |
| Reaches the world? | No. Works alone with no access to your apps or files. | Reads from a document store, but cannot take actions. | Yes — search, code, APIs, email, files (this is what MCP standardizes). | Yes, and each agent carries the tools suited to its role. |
| Memory | Forgets everything when the session ends. | Retrieves fresh each time; no memory of past conversations. | Working memory across the task; can write notes to long-term storage. | A shared pool, so one agent’s findings are visible to all. |
| Planning | None — one forward pass, no self-correction. | Limited to what the retrieved text contains. | Plans, self-evaluates, and adjusts when something fails. | Agents critique each other and converge on a decision. |
Notice how the last three rows are really one story: each rung adds a feedback loop the previous one lacked. RAG adds a loop out to your documents. An agent adds a loop over its own actions. Agentic AI adds a loop between minds.
What each one costs you
☺ Like you’re 10: Every new ability has a price tag — in money, in waiting, and in how many ways it can go wrong.
This is the table people skip, and it is the one that should drive your decision:
| LLM | RAG | AI Agent | Agentic AI | |
|---|---|---|---|---|
| Cost | Lowest — tokens only. | Medium — tokens + vector DB + embedding pipeline. | Higher — many model calls per task, tool API costs, longer runs. | Highest — several agents each making many calls, plus orchestration overhead. |
| Time to build | Hours — call an API. | Days — ingest, embed, tune retrieval. | About a week — define tools, the loop, and error handling. | Weeks — design roles, coordination, and hardening. |
| Human oversight | High — a human reads and acts on every output. | Medium-high — grounded, but sources still need judging. | Medium — set the goal, review the result, unstick it when it loops. | Medium — the orchestrator coordinates, but critical decisions need a human. |
| Fails by | Hallucinating; stale knowledge; no memory; can’t act. | Only as good as your documents; weak when the answer requires reasoning beyond the text. | Getting stuck in loops; acting wrongly without good guardrails; cost growing with task length. | Cascading errors between agents; hard to debug; hard to audit who decided what. |
| Best for | Drafting, summarizing, explaining, brainstorming, quick code help. | Questions over internal knowledge — product docs, policies, contracts, research. | End-to-end task automation, research pipelines, coding assistance. | Large parallel projects: full development pipelines, broad research, enterprise workflows. |
The rule: climb only when forced
☺ Like you’re 10: Don’t bring a whole team to change a lightbulb. Start with the simplest thing that could work, and only get fancier when the simple thing actually fails.
The most expensive mistake in applied AI is not picking the wrong architecture — it is picking a more powerful one than the problem requires. Multi-agent systems are exciting to build and miserable to debug. So invert the instinct: start at the lowest rung that could plausibly work, and let a specific, observed failure push you up.
Two questions decide almost every case. Does it need facts only you have? If yes, you need retrieval. Does it need to do things rather than just say things? If yes, you need tools and a loop — an agent. Most production systems stop there. The jump to a multi-agent team should follow a measured limitation of a single agent — a context window you genuinely cannot fit, or truly parallel subtasks — not the appeal of the architecture diagram.
Benny the Beaver: I’m going to build it with a team of six agents! A researcher, a writer, a reviewer, a—
Cami the Chameleon: Hold on, Benny. What’s the actual job?
Benny the Beaver: Answering questions about our product manual.
Cami: Then you need one model that reads the manual first. That’s it. That’s RAG — a couple of days’ work, and you can explain every answer it gives.
Benny: But six agents would be so much more capable…
Professor Owl: Capable of more failures too, Benny. Six agents is six times the cost, and when the answer comes back wrong you must work out which one went wrong — and whether it misled the other five.
Timmy the Turtle: Build the simple one. Measure it. If it truly falls short, then climb — and you’ll know exactly what you’re fixing.
Where you’ve already seen these
Each rung has its own lesson in this course — this page is the map, not the manual:
- LLM — the engine itself: How AI Models Work, and getting the most from it in Prompting & Context.
- RAG — Retrieval & RAG for the concept; later, in the Applied tier, Retrieval Engineering covers making it work on real corpora.
- Agent — Agentic AI Concepts for the loop, MCP for the tools, Agent Memory for what it remembers, and Building Your Own Agents to ship one.
- Agentic AI — Multi-Agent Systems for orchestration patterns and the failure modes that come with them.
- Building any of them well — Patterns & Anti-Patterns, the named shapes and the recurring mistakes that come with each.
Climbing this ladder is also a security decision. Rung one can only produce text. By rung three your system reads untrusted content and holds real tools — the exact combination covered in AI Security. And whatever rung you land on, you cannot tell whether it works without measurement: build the evals from Evaluation & Testing before you climb, or you will be upgrading on a hunch.
Practise the choice on real problems. (a) Classify six briefs. For each — “summarize this meeting transcript,” “answer staff questions about our HR policies,” “every morning, check our error tracker and open tickets for new crashes,” “write marketing copy variants,” “research five competitors and produce a comparison report with citations,” “translate our docs into four languages” — name the lowest rung that solves it, and one sentence saying what would force you up a rung. (b) Cost it out. Take the one you picked as an agent and estimate its model calls per run versus the LLM-only version. (c) Find the over-build. Look at any AI product you use and guess which rung it is on, then find one feature that suggests it is a rung lower than the marketing implies.
(1) In what sense are these four not really four separate technologies? (2) Which two questions decide most architecture choices? (3) Give the single biggest weakness of each rung. (4) Why is “fails by” the most important row in the cost table? (5) What should justify a move from one agent to a team of agents?
Check your answers
- Not four technologies: Each one contains the previous — the LLM is the engine in all four. RAG is an LLM plus retrieval; an agent is an LLM plus tools and a loop; agentic AI is several agents plus an orchestrator and shared memory. What changes between rungs is the amount of scaffolding around the same engine, and each rung adds a feedback loop the previous one lacked.
- The two questions: “Does it need facts only I have?” (if yes → retrieval/RAG) and “Does it need to do things rather than just say things?” (if yes → tools and a loop, i.e. an agent). Most production systems stop after those two.
- Biggest weakness of each: LLM — hallucination plus a knowledge cutoff, with no memory and no ability to act. RAG — it is only as good as the documents you give it, and it struggles when the answer needs reasoning beyond the retrieved text. Agent — it can get stuck in loops and act wrongly without careful tools and guardrails, with cost rising as tasks lengthen. Agentic AI — cascading errors between agents, plus real difficulty debugging and auditing which agent decided what.
- Why “fails by” matters most: Because moving up a rung does not remove the failure modes below it — it inherits them and adds new ones. An agentic system can still hallucinate; it now does so holding real tools and under less supervision, so the consequences are larger.
- Justifying a team: A measured limitation of a single agent — a context window you genuinely cannot fit, or subtasks that are truly parallel — not the appeal of the architecture. Build the simple version, measure it, and let an observed failure push you up.