Quick goal: check whether a local Ollama model is using a larger context window than your task needs—and whether it has spilled from GPU memory onto the CPU.
A slow coding assistant does not automatically mean you need a bigger GPU. Context allocation, other loaded models, and parallel requests also affect memory. Start with one model, one short request, and an explicit context setting.
This is a documentation-verified diagnostic workflow, not a hardware benchmark. The commands below are for a macOS or Linux terminal with Ollama already installed and running. No performance improvement is guaranteed.
1. Find out what is actually loaded
List installed models, then inspect the models currently in memory:
ollama ls
ollama ps
An installed model is not necessarily loaded. If ollama ps has no model rows, send a request before inspecting it again.
According to the Ollama FAQ, 100% GPU means the model is entirely in GPU memory; 100% CPU means system memory; a mixed percentage means partial offloading. Newer output also exposes a CONTEXT column. Column availability can differ by version.
| Observation | What it tells you | Next check |
|---|---|---|
| No loaded models | There is nothing resident to inspect yet. | Run the request in step 2. |
| 100% GPU | The model is fully GPU-resident. | Inspect context and competing requests; residency alone does not prove fast generation. |
| Mixed CPU/GPU | Some model data is offloaded. | Try less context or a smaller model, then inspect again. |
| 100% CPU | The model is using system memory. | Check GPU support and available memory; CPU-only use may be intentional. |
2. Make context explicit for one small request
The FAQ uses llama3.2 in its context-setting example. If it is not installed, download it first:
ollama pull llama3.2
This downloads model files and needs disk space and network access. Review the model’s license and suitability on its library page. It is an example model here, not a recommendation for every coding task.
Send a short, non-sensitive prompt with an explicit 4,096-token context:
curl http://localhost:11434/api/generate \
-H 'Content-Type: application/json' \
-d '{
"model": "llama3.2",
"prompt": "Explain what a Git branch is in two sentences.",
"stream": false,
"options": { "num_ctx": 4096 },
"keep_alive": "5m"
}'
The FAQ documents options.num_ctx; the generate API reference documents non-streaming responses and keep_alive. The latter keeps the model available briefly so you can inspect it. This request-level setting avoids changing the whole server.
3. Inspect before increasing anything
ollama ps
Check the allocated context and processor split while the model is still loaded. Record your model, context, processor split, and whether another application was generating at the same time. Do not infer memory savings from the requested context value alone.
Ollama’s context guide says larger windows require more memory. It also recommends at least 64,000 tokens for large-context work such as agents and coding tools. That recommendation does not mean every laptop can run every model at that size.
There is a documentation wrinkle: the FAQ describes a 4,096-token default, while the context guide lists VRAM-dependent defaults. Explicit settings plus runtime inspection are safer than assuming either default applies to your installation.
4. Change one variable, not five
If this small request works but a coding agent struggles, repeat your real task with less input or fewer retrieved files. If the task genuinely needs more context, change only num_ctx—for example, from 4096 to 8192—and inspect again. Stay within the model’s supported context and your machine’s capacity.
If other models are loaded, stop only one you know is idle. For the example model, the documented command is:
ollama stop llama3.2
This unloads the model; it does not delete its downloaded files. Do not unload a model serving another active application. The FAQ also explains that parallel processing increases context-memory requirements, so compare one request at a time before changing concurrency settings.
5. Separate capacity problems from other failures
A connection-refused error points to the local service or endpoint, not proof of insufficient VRAM. CPU-only operation on supported GPU hardware deserves a check against the GPU compatibility guide. Persistent crashes need the troubleshooting logs, with secrets removed before sharing.
Keep this diagnostic on localhost. Do not solve a local connection issue by exposing port 11434 to the internet. Local inference still costs storage, electricity, and memory; it also does not make every surrounding editor extension or MCP server private.
Done means: you know which model is loaded, its actual context allocation, and its CPU/GPU split. Then choose the smallest workable input and context for your task—not the largest number the model advertises.
For a distinct next task, see extracting validated local JSON with Ollama, JavaScript and Zod.
Sources checked November 22, 2026. The linked documentation does not consistently provide publication dates. No personal hardware tests or measured speedups are claimed.
Discover more from Juzhax Technology
Subscribe to get the latest posts sent to your email.
