Building on the OpenAI API

This is where you stop using someone else’s app and build your own product on OpenAI’s models. It’s the OpenAI-track counterpart to Building with Claude and the GitHub Copilot Codex track — the Responses API, the agentic loop, tools and function calling, structured output, and the levers that make it reliable in production. If you want the app itself, that’s ChatGPT; this page is the wiring underneath.

The API basics — endpoint, messages & roles

☺ Like you’re 10: Talking to the model is like passing notes through one mail slot. You push in the whole conversation so far — plus a sticky note about who said each line — and the model slides one note back. Everything fancier in this lesson is just a better use of that same slot.

Almost everything you do with OpenAI goes through one call. You send a list of messages (each tagged with a role) and get a reply back. Historically this was the Chat Completions endpoint (/v1/chat/completions); OpenAI’s newer, recommended surface is the Responses API (/v1/responses), which folds in tools, reasoning, and multi-step state. Both are current at time of writing — Chat Completions isn’t going away — but new apps should start on Responses.

from openai import OpenAI
client = OpenAI()                       # reads OPENAI_API_KEY from the environment

resp = client.responses.create(
    model="gpt-5.1",                    # pick a model; versions move fast
    instructions="You are a concise coding assistant.",   # the system role, one field
    input="Explain MCP in one sentence.",
)
print(resp.output_text)                 # convenience: the assembled text reply

The three roles you’ll use constantly: system (called instructions on Responses — the standing rules and persona), user (what the human said), and assistant (what the model said last turn). You resend the whole conversation every call — the API is stateless by default, so the messages list is the memory. (Responses can also persist state server-side via previous_response_id, but the mental model stays the same.)

This is the same underlying idea as Anthropic’s Messages API in Building with Claude — the endpoint names and field shapes differ, the concept is identical. For the model side of “what is a token, what is a context window,” see How models work; for writing the system brief well, see Prompting.

Tool / function calling

☺ Like you’re 10: A tool is a labelled button you hand the model. The label explains what the button does and what to type before pressing it. The model can’t press it itself — it can only ask you to — so your code presses it and tells the model what happened.

A tool (OpenAI calls it a function) is a name, a description, and a JSON schema for its inputs. The model reads the description to decide when to call it, so write it like documentation. Crucially, the model never runs your code — it only emits a request saying “call get_weather with these arguments.” Your code runs the real function and returns the result.

tools = [{
    "type": "function",
    "name": "get_weather",
    "description": "Get the current weather for a city. Use for any 'weather in X' question.",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string", "description": "City name, e.g. 'Pune'"}},
        "required": ["city"],
    },
}]

resp = client.responses.create(model="gpt-5.1", input=messages, tools=tools)

# When the model wants a tool, the output contains a function_call item:
#   1. read its .name and .arguments (JSON), and its .call_id
#   2. run your real function with those arguments
#   3. append a function_call_output item carrying call_id + the result
#   4. call responses.create again — the model continues with the result in hand

Use tool_choice to steer: "auto" lets the model decide, "required" forces it to call some tool, or name a specific function to force that one. You can also set parallel_tool_calls so the model requests several tools at once. Same shape as Claude’s tool use and the same principle throughout Agentic AI: a tool is a contract, and the description is the part the model actually reads.

You don’t ask the model to “be an agent.” You run a loop and let the output steer it: while the model keeps emitting function_call items, run them and feed the results back; when it returns a plain message with no tool call (and a status of completed / a final finish_reason of stop), you’re done. That stop condition is the whole engine of an agent — the exact rule from Agentic AI and the mirror of Claude’s stop_reason loop.

Structured output (JSON Schema)

☺ Like you’re 10: Instead of letting the model write a messy paragraph, you hand it a form with labelled blanks and say “fill in exactly these.” With Structured Outputs, the form is enforced — the model literally can’t colour outside the lines.

When you need clean data, not prose, don’t beg the model in the prompt and hope. OpenAI supports Structured Outputs: attach a JSON Schema and set strict: true, and the model is constrained to produce JSON that validates against your schema — no missing fields, no stray commentary, no markdown fences.

resp = client.responses.create(
    model="gpt-5.1",
    input="Extract the person from: 'Ada Lovelace, born 1815, London.'",
    text={"format": {
        "type": "json_schema",
        "name": "person",
        "strict": True,
        "schema": {
            "type": "object",
            "properties": {
                "name": {"type": "string"},
                "birth_year": {"type": "integer"},
                "city": {"type": "string"},
            },
            "required": ["name", "birth_year", "city"],
            "additionalProperties": False,
        },
    }},
)
data = json.loads(resp.output_text)     # guaranteed to match the schema

Two things to know. First, strict mode has schema rules — every property must be listed in required, and objects need additionalProperties: false. Second, this differs from the older json_object mode, which only guarantees valid JSON, not your shape — Structured Outputs guarantees the shape. Enforcing structure in code beats hoping the prompt is obeyed; it’s the same “programmatic over prompt-based” lesson that runs through AI Pipelines. (Claude reaches the same end by forcing a tool whose schema is the output — different mechanism, same goal.)

Streaming

☺ Like you’re 10: Without streaming, you wait for the whole letter to be written, sealed, and mailed before you see a word. With streaming, you watch it being written one word at a time — it feels instant even though the total time is the same.

By default a call returns only when the model has finished. For any chat UI that’s too slow-feeling, so you stream: set stream=True and the API sends a series of events over the connection (server-sent events) as the reply is generated — text deltas, tool-call deltas, and lifecycle markers — which you print as they arrive.

stream = client.responses.create(
    model="gpt-5.1",
    input="Write a haiku about beavers.",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)   # print each chunk as it lands
    elif event.type == "response.completed":
        pass                                       # final usage/totals arrive here

Streaming changes how you receive the reply, not what it says — and it doesn’t make the model finish faster, it just shows the work in progress. You still handle tool calls the same way (the tool arguments arrive as deltas, then you run them once complete). Nearly every responsive assistant UI, including ChatGPT and Claude, is built on this.

Reasoning, multimodal & embeddings

☺ Like you’re 10: Some models can stop and think hard before answering — you can turn that dial up for tricky puzzles or down for quick chit-chat. Other models can look at pictures or listen to sound, and a special one turns text into numbers so you can search by meaning.

OpenAI’s reasoning models (the GPT-5 family and the earlier o-series at time of writing) can spend hidden “thinking” tokens before they answer. You control the trade-off between quality, latency, and cost with a couple of knobs — set them per call to match the task:

Reasoning maps to the concepts in Reasoning — and note the hidden thinking still costs tokens (see the next section). On multimodal: the same call accepts more than text. You can pass images in the input for vision (“what’s in this photo?”), and OpenAI offers audio models for speech-to-text (transcription) and text-to-speech, plus image generation — the general ideas are in Multimodal.

# Vision: mix text and an image in one message
resp = client.responses.create(model="gpt-5.1", input=[{
    "role": "user",
    "content": [
        {"type": "input_text",  "text": "What's in this picture?"},
        {"type": "input_image", "image_url": "https://example.com/dam.jpg"},
    ],
}])

# Embeddings: a different endpoint — text → a vector of meaning
emb = client.embeddings.create(model="text-embedding-3-small",
                               input="how do I get my money back?")
vector = emb.data[0].embedding          # feed this into a vector store for search

The embeddings endpoint is separate from chat — it doesn’t generate text, it turns text into a vector so you can search by meaning. That’s the engine under Retrieval & RAG: embed your documents, embed the question, retrieve the nearest chunks, and hand them to a chat model as grounding.

The Agents SDK — don’t rebuild the loop

☺ Like you’re 10: You could build a robot friend from loose LEGO bricks, or grab the ready-made kit that already snaps together. The Agents SDK is that kit — it runs the ask-a-tool-then-loop dance for you so you don’t hand-wire the plumbing every time.

You can hand-roll the loop from the last few sections, and for full control you sometimes should. But for a production agent, OpenAI ships the Agents SDK — a small framework that runs the tool loop, manages the conversation, and adds the pieces you’d otherwise build yourself: handoffs (route to another agent), guardrails (validate inputs/outputs), sessions (memory across turns), and tracing (see every step). It also speaks MCP, so a Model Context Protocol server’s tools plug straight in.

from agents import Agent, Runner, function_tool

@function_tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"Sunny in {city}."

agent = Agent(name="Helper", instructions="Be concise.", tools=[get_weather])
result = Runner.run_sync(agent, "What's the weather in Pune?")
print(result.final_output)              # the SDK ran the loop and stop condition for you

Under the hood it’s exactly the loop from the schematic above — the SDK just owns the “run the tool, append the result, call again, stop when done” cycle. This is OpenAI’s counterpart to the Claude Agent SDK; the vendor-neutral patterns (single-agent loops, tools, handoffs, multi-agent designs) live in Building Agents and Multi-Agent Systems. Reach for the raw Responses API when you want full control; reach for the SDK when you want a reliable agent fast.

Tokens, cost, and API vs app

☺ Like you’re 10: The model charges by the word-piece, both for what you send and what it says back — and its private “thinking” counts too. Building on the API is like renting a kitchen to cook exactly your dish; using the app is buying the finished meal. Only rent the kitchen if you’re actually opening a restaurant.

You pay per token — roughly a word-piece — and both directions cost, usually at different rates. Three costs surprise people: input tokens (everything you send, including the whole resent conversation and any big system prompt or retrieved docs), output tokens, and — for reasoning models — the hidden reasoning tokens the model spends thinking, which you’re billed for even though you never see them. Watch resp.usage to track all of it.

So when should you build on the API at all? Use the ChatGPT app when a human is doing the work interactively — it’s finished, maintained, and free of code. Build on the API when you need the model inside your own product: your UI, your data and tools, automation with no human in the loop, or behaviour you must control and version. If you only need coding help in your editor, the Copilot/Codex track fits better than raw API calls.