What to Do With a Private AI Lab Once It Actually Works

The setup post gets Ollama, Open WebUI, and a private RAG pipeline onto your own hardware. That is the install. This is the month-later question I actually cared about: now that the stack answers LOCAL MODEL WORKS and can retrieve a launch code from a Markdown file, what is it for?

I did not build a ChatGPT clone so I could type into localhost:3000 for fun. I built it so scripts — and the rest of a publishing pipeline — can talk to a model that never leaves the Mac, and so that model can be forced to answer from my posts rather than from whatever it half-remembers about Git remotes.

Two doors. Same lab.

If you only remember one thing: Planner-style jobs go to Ollama. “Ground this in the blog” goes to Open WebUI. Mixing them up is how you either skip retrieval or pay for a Docker container you are not using.

Wait until it is actually up

docker compose up -d printing Started means the container exists. It does not mean the UI is serving. Open the browser too early and you get a blunt 500: Internal Error — usually Open WebUI still migrating, or Docker just OOM-killed it and it is coming back. I have watched an 8 GB Docker VM murder a perfectly innocent 600 MB UI because the VM was already full of other stacks. That is not an Ollama problem.

curl http://127.0.0.1:3000/health

You want {"status":true}. Then open the UI, or run a script. Not before.

Door one: Ollama from Python

Use this when the job does not need your archive. Classify a subject line. Draft three titles. Ask for a one-sentence summary of a paragraph you are about to paste in. The process on :11434 is the same engine the UI uses; you are just skipping the orchestration layer.

The model id is llama3.2:latest. I will keep saying that, because llama3.2 looks right and returns 400 Model not found.

import requests

r = requests.post(
    "http://127.0.0.1:11434/api/chat",
    json={
        "model": "llama3.2:latest",
        "messages": [{"role": "user", "content": "One sentence: what is this lab?"}],
        "stream": False,
        "options": {"temperature": 0, "num_ctx": 8192},
    },
    timeout=180,
)
print(r.json()["message"]["content"])

Embeddings for a homemade index — if you ever outgrow Open WebUI’s store — are the same host:

POST http://127.0.0.1:11434/api/embed
model: nomic-embed-text:latest

There is an ollama Python package if you prefer ollama.chat(...). I have been using requests because it is already in the lab tests and it makes the HTTP obvious.

Door two: Open WebUI when the documents have to win

🔒 Subscribe to keep reading.

What I actually use it for

🔒 Subscribe to keep reading.

Putting the archive on disk without drowning the index

🔒 Subscribe to keep reading.

What I am not doing with it yet

🔒 Subscribe to keep reading.

Quick-start checklist

🔒 Subscribe to keep reading.

References

🔒 Subscribe to keep reading.