Models & Local LLMs

Modern coding assistants are multi-model by design — GitHub Copilot, Anthropic’s Claude Code, Cursor, and others all let you swap the model behind the tool. Here’s how to pick a better-fitting model, and how to run a model entirely on your own machine, with a full walkthrough for a Windows 11 VirtualBox VM.

There are two distinct reasons to change the model behind your assistant. First, to pick a different hosted model for a task — a faster one for routine edits, a stronger reasoner for a gnarly refactor (say, swapping from a small model to a frontier reasoner like Anthropic’s Claude Opus, OpenAI’s top GPT, or Google’s Gemini Pro for a hard problem). Second, to bring your own model — including one running locally — for privacy, cost control, or fully offline work. We cover both across the four assistant tracks (GitHub Copilot, Claude, ChatGPT, and Gemini) and via the API, then go deep on the local case. The concrete VirtualBox walkthrough later uses GitHub Copilot as its worked example, but the same ideas apply to Claude Code, Cursor, and other agentic assistants.

Switching the hosted model

☺ Like you’re 10: Imagine your helper is a team of robots — a speedy one for easy chores and a really careful one for tricky puzzles. Here you learn how to tap the one you want, like choosing which friend to ask for help.

Two things matter here, and people constantly conflate them: the chat model and the completions model are set separately. Most assistants make this same split — the picker in your tool (Copilot’s dropdown, Claude’s model selector, the ChatGPT model menu, Gemini’s picker) chooses the chat/agent model, while inline completions are configured elsewhere.

The chat model

Every assistant surfaces a current-model dropdown — at the bottom of GitHub Copilot Chat, in Claude’s model selector, in the ChatGPT model menu, in Gemini’s picker — where you click and pick a model. Changing it mid-conversation switches which model answers your next prompt. Most tools now ship an Auto (or “smart routing”) option that evaluates your task (reasoning depth, code-gen complexity, bug-diagnosis difficulty, tool orchestration) against real-time model health and availability, then routes to the best fit — which both improves quality and reduces rate-limiting. Copilot’s Auto is a good example: it can hover over any response to show which model produced it, and lets you switch away from Auto to a specific model whenever you want. When routing is billed by whatever model it picks, it’s often capped to a cheaper tier (Copilot, for instance, currently caps Auto to models in the 0×–1× multiplier range).

The completions model

Inline “ghost text” completions use their own model, set independently from chat. In GitHub Copilot, open the Command Palette (Ctrl+Shift+P / ⌘⇧P) and run “GitHub Copilot: Change Completions Model.” The list is smaller and managed more tightly than chat — and changing your chat model has no effect on completions, which trips people up constantly. Whichever assistant you use, treat the completions model as a separate setting from the chat/agent model.

Bring Your Own Model (BYOK)

☺ Like you’re 10: It’s like bringing your own LEGO set to a friend’s house instead of only playing with theirs — you plug in a “brain” you already have a key for, and the app uses it just like its built-in ones.

“Bring Your Own Key” (BYOK) lets you connect a model your assistant doesn’t host — using your own API key or endpoint — and then use it from the same model picker. This is a general idea across tools: GitHub Copilot, Cursor, and other agentic editors all support it, and CLIs like Claude Code point at models via their own API keys. Typical BYOK providers you can wire in include Anthropic (Claude), OpenAI (GPT), Google (Gemini), Azure OpenAI, OpenRouter, and Hugging Face; you can run one locally with Ollama or Foundry Local; or point at a Custom Endpoint that speaks a standard API — Chat Completions, the OpenAI Responses API, or Anthropic’s Messages API (self-hosted or enterprise).

Under the hood this is just the model parameter: whichever assistant you use, BYOK sets the model (and base URL/key) the tool sends to a provider. In GitHub Copilot the entry point is one command — open the Command Palette and run “Chat: Manage Language Models” (or click the gear / Manage Models in the picker) to open the Language Models editor, add a provider, enter the key/endpoint, and choose which models appear. Other tools expose the same idea through a settings pane or a config file with base_url / api_key / model fields.

Deep dive: a local LLM in Copilot on Windows 11 / VirtualBox

☺ Like you’re 10: A “virtual machine” is a pretend computer living inside your real one, like a dollhouse inside your house. We’ll put a little AI brain inside that dollhouse and plug it into your coding helper.

We’ll run Ollama (the simplest local-model runtime) inside a Windows 11 guest on Oracle VirtualBox, and wire it into your assistant’s chat. We use Copilot Chat here as the concrete example, but any BYOK-capable tool (Claude Code, Cursor, and others) connects to a local Ollama endpoint the same way. First, a reality check that will save you hours.

Step 0 — Prepare the VM

In the VirtualBox settings for your Windows 11 guest (shut down first):

• Memory: give it at least 8 GB, ideally 12–16 GB. The model must fit in RAM with overhead (an 8B model at Q4 is ~5 GB).
• Processors: assign 4+ vCPUs (more cores help, with diminishing returns past ~8–12).
• Enable virtualization: System → Processor → enable Nested VT-x/AMD-V isn’t needed for Ollama, but ensure host hardware virtualization is on; System → Acceleration → enable Nested Paging. (VT-x/AMD-V must also be enabled in your host BIOS/UEFI.)
• Disk: models are large — keep 10–30 GB free in the guest.
• Guest Additions: install them for a smoother experience.

Step 1 — Install Ollama (in the VM)

Inside the Windows 11 guest, download the Windows installer from ollama.com and run it. Ollama installs a background service that listens on http://localhost:11434. Verify it’s live by opening that URL in the guest’s browser — you should see the plain-text response “Ollama is running.”

Step 2 — Pull a model

Open a terminal (PowerShell) in the guest and pull a small, capable model. For agent mode you need tool calling, so pick accordingly:

# A solid general/tool-calling pick (~4.7 GB):
ollama pull llama3.1:8b

# Smaller / code-focused options if RAM is tight:
ollama pull qwen2.5-coder:7b
ollama pull phi4

# Sanity-check it runs and responds:
ollama run llama3.1:8b
# (type a question; Ctrl+D to exit)

Step 3 — Install VS Code + the Copilot extension (in the VM)

Install VS Code and the GitHub Copilot extension inside the guest. Signing into GitHub is optional for BYOK chat, but sign in if you also want inline completions and semantic search (which BYOK does not provide).

Step 4 — Connect Ollama to Copilot

1. Open the Command Palette and run “Chat: Manage Language Models” (or open the Chat model picker and click the Manage Models gear).
2. Choose the Ollama provider.
3. When prompted for the endpoint URL, enter the local Ollama address and confirm:

http://localhost:11434

4. VS Code connects and auto-discovers the models you’ve pulled. Select the one(s) you want, and confirm.
5. Open Copilot Chat, click the model picker, and select your Ollama model. Ask it something — the response is now generated entirely on your machine.

Optional — the Copilot CLI with a local model

The terminal CLI supports the same BYOK idea via environment variables, and can run fully air-gapped:

$env:COPILOT_PROVIDER_BASE_URL = "http://localhost:11434"
$env:COPILOT_MODEL = "llama3.1:8b"
$env:COPILOT_OFFLINE = "true"   # never contact GitHub's servers
copilot                          # start the CLI using your local model

Models used this way must support tool calling and streaming, or the CLI returns an error rather than silently falling back to a hosted model.

Performance path: run Ollama on the host (with the GPU)

☺ Like you’re 10: The dollhouse computer can’t reach the fast graphics chip, so instead we keep the AI brain out in the big house where the fast chip lives, and let the dollhouse phone it up when it needs an answer.

Because the VM is CPU-bound, the fastest setup is to run Ollama on the Windows host (where it can use your GPU) and connect to it from the VM. Two changes make this work:

1. Make the host’s Ollama listen beyond localhost. By default Ollama binds to 127.0.0.1, which a VM can’t reach. On the host, set the environment variable OLLAMA_HOST=0.0.0.0 (System Properties → Environment Variables) and restart Ollama, so it listens on all interfaces.
2. Give the VM a network path to the host. In VirtualBox, use a Host-only adapter (the host is typically reachable at something like 192.168.56.1) or a Bridged adapter (the VM joins your LAN). Find the host’s IP on that network with ipconfig on the host.

Then in the VM’s VS Code, add the Ollama provider exactly as in Step 4, but enter the host’s IP instead of localhost:

http://192.168.56.1:11434      # use your host's actual IP

Another local runtime: Microsoft Foundry Local

☺ Like you’re 10: Ollama is one brand of oven for “cooking” AI answers on your own computer — Foundry Local is a different brand that plugs into the same kitchen, and it can use extra speed chips your computer might have.

Ollama isn’t the only way to run models locally. Foundry Local (Microsoft’s on-device runtime, formerly Azure AI Foundry Local, now generally available) is a strong alternative — and it plugs into Copilot the same way. It runs models entirely on your device with no Azure account, no per-token cost, and offline support, and it exposes an OpenAI-compatible local endpoint so any tool that speaks the OpenAI API can use it.

Quickstart in the guest: install it, list and run a model (the runtime downloads and loads it, then serves it on the local OpenAI-compatible endpoint):

winget install Microsoft.FoundryLocal   # one-line install
foundry model list                      # see the curated catalog
foundry model run phi-4                 # download + run + interactive chat
# or a code/tool-calling model:
foundry model run qwen2.5-coder
foundry service status                  # shows the local endpoint URL

To use it in Copilot, add it through Chat: Manage Language Models — pick the Foundry Local provider, or, because the endpoint is OpenAI-compatible, use the Custom Endpoint provider pointed at the URL from foundry service status. From there it behaves like any BYOK model (chat, tools, MCP, agent mode if the model supports tool calling).

Going fully offline (air-gapped)

☺ Like you’re 10: This is teaching your helper to work with the internet unplugged, like playing a board game that doesn’t need Wi-Fi — everything happens right there on your own machine.

Pair a local runtime with your editor’s offline support and you get a coding assistant that never touches the network. Since VS Code 1.122, BYOK works without signing into GitHub — once you’ve configured at least one local model, the Chat view becomes available and sign-in prompts are suppressed, so chat, tools, and MCP servers all work in an air-gapped environment.

An air-gapped checklist: (1) install Ollama or Foundry Local and pull a tool-calling model; (2) in VS Code, add it via Chat: Manage Language Models; (3) point the two chat.utility* settings at a local model; (4) accept that completions and semantic search won’t be available offline. For the terminal, the Copilot CLI goes fully air-gapped with COPILOT_OFFLINE=true plus a local provider — it disables telemetry and only talks to your local model.

Local vs. hosted — choosing well

☺ Like you’re 10: A brain on your laptop keeps secrets at home and works with no internet, but a giant brain in the cloud is usually smarter and faster — so you pick the right one for the job, like choosing a bike for a short trip and a car for a long one.

Local models are excellent for privacy (code never leaves your machine), cost (no premium requests), offline/air-gapped work, and experimentation with open-weight models. The trade-offs are real, though: open models are generally less capable than frontier hosted models like Anthropic’s Claude, OpenAI’s GPT, or Google’s Gemini, CPU inference in a VM is slow, and you lose completions and semantic search. A common pattern is to keep your assistant on a strong hosted model (Copilot’s Auto, Claude in Claude Code, a top GPT in ChatGPT, Gemini Pro — or the right model value over the API) for daily speed and quality, and switch to a local model for sensitive code, offline sessions, or when you’ve hit your request budget.