What to Do With a Private AI Lab Once It Actually Works

The setup post gets Ollama, Open WebUI, and a private RAG pipeline onto your own hardware. That post covers the install. The question I actually cared about a month later: now that the stack answers LOCAL MODEL WORKS and can retrieve a launch code from a Markdown file, what is it for?

I did not build a ChatGPT clone so I could type into localhost:3000 for fun. I built it so scripts — and the rest of a publishing pipeline — can talk to a model that never leaves the Mac, and so that model can be forced to answer from my posts rather than from whatever it half-remembers about Git remotes.

Two doors. Same lab. Ollama is a local model server that exposes an HTTP API on port 11434 for running LLMs such as llama3.2:latest. Open WebUI is a self-hosted chat interface that adds document chunking, embeddings, retrieval, and its own REST API on port 3000. Retrieval-augmented generation (RAG) is a pattern that fetches passages from your own documents and injects them into the model’s prompt, so answers come from your archive rather than from the model’s memory.

If you only remember one thing: Planner-style jobs go to Ollama. “Ground this in the blog” goes to Open WebUI. Mixing them up is how you either skip retrieval or pay for a Docker container you are not using.

Check Open WebUI health before running Python scripts

docker compose up -d printing Started means the container exists, not that the UI is serving. Open the browser too early and you get a blunt 500: Internal Error — usually Open WebUI still migrating, or Docker just OOM-killed it and it is coming back. I have watched an 8 GB Docker VM murder a perfectly innocent 600 MB UI because the VM was already full of other stacks. An OOM-killed container is not an Ollama problem.

curl http://127.0.0.1:3000/health

You want {"status":true}. Then open the UI, or run a script. Not before.

Call Ollama from Python with the /api/chat endpoint (door one)

Use this when the job does not need your archive. Classify a subject line. Draft three titles. Ask for a one-sentence summary of a paragraph you are about to paste in. The process on :11434 is the same engine the UI uses; you are just skipping the orchestration layer.

The model id is llama3.2:latest. I will keep saying that. Ollama itself resolves a bare llama3.2 to the :latest tag, so the short name works here. I have not checked how Open WebUI treats it, so I use the full tagged id in both places and never have to remember which door is forgiving.

pip install requests python-dotenv

The Python snippets in this post are syntax-checked but not executed end-to-end here: they need the running lab (and, for Open WebUI, your own API key and knowledge-base id) to produce output.

import requests

r = requests.post(
    "http://127.0.0.1:11434/api/chat",
    json={
        "model": "llama3.2:latest",
        "messages": [{"role": "user", "content": "One sentence: what is this lab?"}],
        "stream": False,
        "options": {"temperature": 0, "num_ctx": 8192},
    },
    timeout=180,
)
r.raise_for_status()  # a typo in the model id fails here, not as a KeyError below
print(r.json()["message"]["content"])

The first call after a quiet spell is slower than the rest. Ollama has to load the model into memory, and it unloads it again after a few minutes idle (keep_alive, five minutes by default). On my Mac a first request took about 2.8 seconds, 2.6 of them spent loading the model. A slow first answer is not a crashed container.

Embeddings for a homemade index — if you ever outgrow Open WebUI’s store — are the same host:

POST http://127.0.0.1:11434/api/embed
model: nomic-embed-text:latest

There is an ollama Python package if you prefer ollama.chat(...). I have been using requests because it is already in the lab tests and it makes the HTTP obvious.

Call Open WebUI from Python for RAG over a knowledge base (door two)

🔒 Subscribe to keep reading.

Private AI lab use cases for a Python blogger

🔒 Subscribe to keep reading.

Stage blog posts for Open WebUI RAG without drowning the index

🔒 Subscribe to keep reading.

Limits of a local Llama 3.2 3B lab

🔒 Subscribe to keep reading.

Quick-start checklist for the private AI lab

🔒 Subscribe to keep reading.

References

🔒 Subscribe to keep reading.