Evaluation & Testing

You built something with AI. Now the hard question: how do you know it’s any good — and how do you know your next tweak made it better, not quietly worse? Normal software has tests that pass or fail; AI outputs vary and “correct” is fuzzy, so you need a different instrument. That instrument is an eval, and it’s the single habit that separates a demo from something you can trust in front of real users.

“It worked when I tried it” is not evidence

☺ Like you’re 10: Rolling a six once doesn’t mean the dice always land on six. Trying your AI once and liking the answer tells you almost nothing about the hundred people who’ll use it next.

When you test ordinary code, you assert add(2, 2) === 4 — one right answer, checked exactly. Two things break that for AI:

• It’s non-deterministic. The same input can produce different wordings each time, so you can’t compare against one fixed string.
• “Correct” is fuzzy. There are many good ways to “summarize this ticket” — there’s no single true answer sitting in a column to diff against.

So the usual move — try it once, eyeball the answer, ship it — is really just vibes. It feels like testing, but it measures nothing: you can’t spot a rare failure in one try, and when you change the prompt or swap the model tomorrow you’ll have no idea whether you helped or broke it. That blind spot is the eval gap, and closing it is what this lesson is about.

What an eval actually is

☺ Like you’re 10: It’s a spelling test with an answer key. You write a list of words to test (the inputs) and what counts as right (the marking scheme), then you can score anyone — or any change — the same way every time.

Strip away the jargon and an eval is just two things:

1. A dataset — a fixed set of representative inputs. Real questions your system will face: easy ones, hard ones, weird edge cases, and the “there’s no good answer” ones. Ideally drawn from real usage, not made up at your desk.
2. A scorer — a way to turn each output into a judgment: right/wrong, a number, or pass/fail against a rule. This is the part that replaces the missing “answer column.”

Run every input through your system, score each output, and you get one honest number: “82% good on the cases I care about.” Now change your prompt and run it again — 84% means you helped, 78% means you broke something. That number, computed the same way every time, is the whole point: it turns “I think it’s better” into “it is better, and here’s by how much.”

Three ways to score — from strict to flexible

☺ Like you’re 10: Some questions have one right answer you can check instantly (spelling). Some need a checklist (“did the essay have a beginning, middle, and end?”). And some are so open you ask a smart friend to judge. Evals use all three.

The “scorer” isn’t one thing — pick the strictest one that fits the question:

Reach for the cheapest, strictest scorer that works: an exact rule is free and never wrong about itself, while an LLM-as-judge is powerful but is itself an AI — so you calibrate it against a few human-graded examples before you trust its numbers. (The Evaluating AI Systems lesson in the AI Engineering track goes deep on judge bias, faithfulness scoring, and building these at scale.)

Turn every failure into a permanent test

☺ Like you’re 10: When you get a spelling word wrong, you add it to your practice list so you never miss it again. Same here: every bug your AI makes becomes a question it must answer forever after.

This is the highest-leverage habit in the whole lesson: a regression test. The moment your system gets something wrong in the real world, you don’t just fix it and move on — you capture that exact case as a permanent entry in your eval dataset:

eval case "refund-window"
  input:    "how long do I have to return something?"
  expected: mentions "14 days"
  scorer:   contains("14 day")

Now that failure is a tripwire. Any future prompt tweak or model swap that would reintroduce the “30 days” bug fails the eval before it reaches a user. Bugs stop being things you fix over and over; each one gets fixed once and then guarded forever. Do this consistently and your eval set quietly grows into a map of every way your system has ever been wrong.

Gate your releases on the number

☺ Like you’re 10: A ride at the fair has a “you must be this tall” bar. Your release gets one too: it only ships if it scores above the line — no exceptions, no “but it felt fine.”

An eval is only worth the effort if it can stop a bad change. So wire it into your release process: run the whole dataset automatically on every change, and block the deploy if the score drops below a threshold you set (say, 95% pass, and zero regression failures). This is running evals “in CI,” and it turns quality from a hope into a gate.

There are two moments to measure, and you want both:

• Offline — score your fixed dataset before release, as the gate. Fast, repeatable, catches known failure modes.
• Online — watch real traffic after release (thumbs up/down, A/B tests, spot-checks). Reality always has cases your dataset didn’t — and each new failure you catch here becomes tomorrow’s offline test case, closing the loop.

Evaluation is the thread running through the whole course: it’s the “Timmy check” behind RAG (did retrieval find the right chunk?), AI security (did the guardrail hold?), and responsible AI (is it fair across groups?). You first met it as a stage in AI Pipelines; here it earned its own page. When you’re ready to build evals for real — golden datasets from production, judge calibration, faithfulness metrics, and CI gating in depth — continue to Evaluating AI Systems in the AI Engineering track.