Why I wanted a local AI lab
It has been a stranger few months than usual for anyone who builds on top of rented AI models, and by September the pattern was hard to ignore.
On 12 June, a US export-control directive forced Anthropic to pull Claude Fable 5 and Mythos 5 offline worldwide. The directive itself only targeted foreign nationals, but Anthropic disabled both models for every customer, everywhere, to stay compliant — a frontier model that plenty of developers had already wired into their workflows simply evaporated within days of launch. (The directive was lifted later that month, but the outage still made the point: access to a rented model can vanish on a government’s schedule, not yours.)
On 1 June, GitHub Copilot moved to usage-based billing. The flat monthly fee gained a meter: Copilot plans now bill against GitHub AI Credits, with extra usage charged once your monthly allowance runs out.
And while all that was happening, OpenCode — an open-source, model-agnostic coding agent that is perfectly happy talking to a local Ollama server — reached 7% developer adoption in JetBrains’ 2026 developer survey, with 42% of developers aware of it despite no big company name behind it.
The common thread is hard to miss. The cloud model you build on can be revoked, repriced, or rate-limited on someone else’s timetable; an open-weight model sitting on your own disk cannot. That has always been the quiet argument for a local AI lab, and this summer made it loud.
The good news is that the local AI ecosystem has hit a practical tipping point at exactly the right moment. Models that would have required a data centre two years ago now run on a mid-range laptop, and the tooling to serve, interact with, and augment those models has become genuinely approachable.
This post walks through a complete local AI lab: a model engine, a web interface that feels like ChatGPT, and a private RAG pipeline that lets your AI answer questions grounded in your own documents.
I run this stack on an Apple M1 MacBook, so where the steps differ from other platforms I have called that out explicitly.
A companion post covers the security picture — specifically, what changes when you add autonomous agents to this setup. Once the lab is actually answering, a follow-up on calling it from Python covers the two APIs and the jobs that only make sense after both doors are wired.
What we are actually building, and the five phases from Ollama to a private RAG, follow below.
What we’re actually building
🔒 Subscribe to keep reading.
Phase 1: The model engine — Ollama
🔒 Subscribe to keep reading.
Phase 2: The web interface — Open WebUI
🔒 Subscribe to keep reading.
Phase 3: Private RAG — your AI on your documents
🔒 Subscribe to keep reading.
Phase 4: Prove the lab
🔒 Subscribe to keep reading.
Phase 5: Operate the lab
🔒 Subscribe to keep reading.
How much hardware do you actually need?
🔒 Subscribe to keep reading.
What “private” actually means
🔒 Subscribe to keep reading.
Where agents change the threat model
🔒 Subscribe to keep reading.
Quick-start checklist for a local AI lab
🔒 Subscribe to keep reading.
References
🔒 Subscribe to keep reading.
You've hit a Deep Dive tutorial.
I spend dozens of hours researching, coding, and breaking things to write these guides. This content is free, but reserved for my subscriber community. Drop your email below to unlock this guide (and all past/future deep dives):
Full content temporarily unavailable — refresh in a moment
Already a subscriber? Use the magic link from your last newsletter, or reset your password.
Log in to unlock
New subscribers get an inbox mail: Set a password to unlock articles. The form does not log you in — use the same email afterwards.
