Production Deployment

A demo only has to work once, for you, on a good network day. Production means the same Claude calls have to survive real traffic: picking where Claude runs, surviving rate limits gracefully, keeping cost under control at scale, and treating prompts like the code they are.

Where Claude runs: direct API, Bedrock, and Vertex AI

☺ Like you're 10: Same phone, different carriers — Claude is the phone, and the direct API, Bedrock, and Vertex AI are carriers you pick based on which one your family already has a plan with.

Every example so far in this course has hit api.anthropic.com directly. That's the fastest path to new models and the full feature set, but it isn't the only option. Claude is also available through Amazon Bedrock and Google Cloud's Vertex AI, plus a separate Anthropic-operated "Claude Platform on AWS" offering and Microsoft Foundry. Teams pick a cloud platform over the direct API for reasons that have nothing to do with the model itself: existing cloud spend commitments, a procurement process that already approved AWS or GCP as a vendor, IAM-based auth that plugs into infrastructure your security team already trusts, or data-residency requirements met by a specific cloud region.

The Messages API shape is nearly identical across platforms, with two concrete differences on Vertex: the model goes in the endpoint URL rather than the request body, and the body needs anthropic_version set to the literal string vertex-2023-10-16 instead of the header-based versioning the direct API uses.

// Vertex AI request body — model is NOT a body field here,
// it's part of the endpoint path instead
{
  "anthropic_version": "vertex-2023-10-16",
  "max_tokens": 1000,
  "messages": [
    {"role": "user", "content": "Summarize this quarter's incident report."}
  ]
}

Note the naming difference on AWS specifically: Bedrock exposes Claude under anthropic.-prefixed model IDs, while the separate Claude Platform on AWS and Microsoft Foundry offerings are Anthropic-operated and use the same first-party model IDs (like claude-opus-5) you'd use against the direct API. If you migrate between platforms, that's the first thing to check.

Handling rate limits

☺ Like you're 10: Think of a bouncer with three separate counters — how many people per minute, how many words coming in, how many words going out. Cross any one line and you're waiting outside, even if the other two are fine.

Rate limits are enforced per organization using a token-bucket algorithm across three dimensions per model: requests per minute (RPM), input tokens per minute (ITPM), and output tokens per minute (OTPM), scaled by your usage tier (Start, Build, Scale, or Custom). Only uncached input tokens count toward ITPM for most models — one more reason prompt caching, covered below, effectively raises your real throughput without needing a rate-limit increase at all.

When you exceed a limit, the API returns HTTP 429 with a rate_limit_error body. The response headers tell you exactly what to do next: a retry-after header gives you seconds to wait, and a family of anthropic-ratelimit-* headers reports the limit, remaining count, and reset time for requests, input tokens, and output tokens separately. Official guidance is to retry both 429 responses and 5xx (api_error, overloaded_error) responses with exponential backoff.

import anthropic

# Let the SDK handle retries — usually all you need
client = anthropic.Anthropic(max_retries=5)

For cases where you're calling the raw HTTP endpoint, running on a platform without SDK support, or want custom jitter behavior, here's a manual backoff loop that reads retry-after when it's present and falls back to exponential backoff with jitter when it isn't:

import random
import time
import requests

def call_with_backoff(payload, headers, max_retries=6):
    for attempt in range(max_retries):
        response = requests.post(
            "https://api.anthropic.com/v1/messages",
            headers=headers,
            json=payload,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("retry-after")
            if retry_after is not None:
                delay = float(retry_after)
            else:
                delay = min(60, 2 ** attempt) + random.uniform(0, 1)
            time.sleep(delay)
            continue
        response.raise_for_status()
        return response.json()
    raise RuntimeError("Exceeded max retries after repeated 429 responses")

async function callWithBackoff(payload, headers, maxRetries = 6) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const response = await fetch("https://api.anthropic.com/v1/messages", {
      method: "POST",
      headers,
      body: JSON.stringify(payload),
    });
    if (response.status === 429) {
      const retryAfter = response.headers.get("retry-after");
      const delay = retryAfter
        ? Number(retryAfter) * 1000
        : Math.min(60000, 2 ** attempt * 1000) + Math.random() * 1000;
      await new Promise((resolve) => setTimeout(resolve, delay));
      continue;
    }
    if (!response.ok) throw new Error(`API error: ${response.status}`);
    return response.json();
  }
  throw new Error("Exceeded max retries after repeated 429 responses");
}

Cost optimization

☺ Like you're 10: Three separate dials for your bill: use a cheaper model when the job is easy, remember old context instead of re-reading it every time, and batch up anything that isn't urgent.

Model routing

Not every request needs your most capable model. Official guidance frames model selection as a capability/speed/cost tradeoff, and recommends either starting cheap (e.g. Haiku) and upgrading only where quality falls short, or starting capable and optimizing down later once you know where quality margin exists. Third-party analyses report 40-80% cost reduction from routing by task complexity — a cheap classifier call decides whether the real request goes to a fast, inexpensive model or a slower, more capable one.

import anthropic

client = anthropic.Anthropic()

def classify_complexity(query: str) -> str:
    triage = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=10,
        messages=[{
            "role": "user",
            "content": f"Classify this support query as 'simple' or 'complex'. "
                        f"Reply with one word.\n\nQuery: {query}",
        }],
    )
    label = next(b.text for b in triage.content if b.type == "text")
    return "complex" if "complex" in label.strip().lower() else "simple"

def answer(query: str):
    model = "claude-opus-5" if classify_complexity(query) == "complex" else "claude-haiku-4-5"
    return client.messages.create(
        model=model,
        max_tokens=1000,
        messages=[{"role": "user", "content": query}],
    )

Prompt caching

When repeated calls share a long, stable prefix — a system prompt, a large reference document or codebase, a set of tool definitions, few-shot examples — mark a breakpoint with cache_control and let the API skip reprocessing that prefix on subsequent calls. A cache read costs roughly 10% of the base input price (a 90% saving on that portion of the prompt), while writing to the cache costs more up front: 1.25x base input price for the default 5-minute TTL, or 2x for a 1-hour TTL. The minimum cacheable prefix is 512-4096 tokens depending on model, and you get up to 4 cache breakpoints per request, checked up to 20 content blocks back.

{
  "model": "claude-sonnet-5",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Full product manual follows.\n\n<manual>...50,000 tokens of docs...</manual>",
      "cache_control": {"type": "ephemeral"}
    }
  ],
  "messages": [
    {"role": "user", "content": "How do I reset a device that won't power on?"}
  ]
}

Batch processing

For workloads that don't need a synchronous response — nightly report generation, bulk classification, evaluation runs — the Message Batches API processes requests asynchronously at a flat 50% discount on both input and output tokens, across all models. Most batches complete within an hour, with a hard cutoff at 24 hours; a single batch can hold up to 100,000 requests or 256MB, and results stay retrievable for 29 days. Not every parameter is supported in batch mode — no streaming, no fast mode, no Threads, no cache-hint — so design batch requests as self-contained calls.

[
  {
    "custom_id": "ticket-4471",
    "params": {
      "model": "claude-haiku-4-5",
      "max_tokens": 500,
      "messages": [{"role": "user", "content": "Summarize support ticket #4471."}]
    }
  },
  {
    "custom_id": "ticket-4472",
    "params": {
      "model": "claude-haiku-4-5",
      "max_tokens": 500,
      "messages": [{"role": "user", "content": "Summarize support ticket #4472."}]
    }
  }
]

Logging, monitoring, and prompt versioning

The Usage and Cost Admin API exposes /v1/organizations/usage_report/messages and /v1/organizations/cost_report endpoints for token and cost tracking broken down by model, workspace, API key, and time bucket (1-minute, 1-hour, or 1-day granularity). It requires a separate sk-ant-admin... Admin key, data typically appears within about 5 minutes, and Anthropic recommends polling roughly once per minute for near-real-time dashboards. For deeper observability, official partner integrations include CloudZero, Datadog, Grafana Cloud, Honeycomb, and Vantage. If you're on Vertex AI, turn on Google's request-response logging service and retain logs on at least a 30-day rolling basis — that's Anthropic's own recommendation for monitoring activity and investigating misuse on that platform.

Prompt versioning is less standardized than you might expect. For the plain Messages API, there's no dedicated versioning product beyond Console-side prompt templates and variables (using the same {{PLACEHOLDER}} convention covered in the prompt-engineering module) plus a prompt generator/improver tool that tracks changes to prompt structure over time. The more formal mechanism lives in Claude's Managed Agents system: every agents.update call produces a new immutable version, and callers pin a specific version ID — so rolling a prompt forward or back is a config change, not a code deploy. Anthropic documents this directly in the "Managed Agents tutorial: prompt versioning and rollback" cookbook.

Pattern vs. anti-pattern