The problem, stated precisely
Almost everyone who sets out to build an assistant builds a chat window and stops. A chat window is a place you go. An assistant is something that is already there, already knows, and has already done the small thing you would have asked for.
The difference is not model quality. It is four unglamorous properties: it runs without being launched, it remembers across sessions, it reaches the systems you actually use, and it knows which decisions are not its to make. None of those are in the model. All of them are yours to build, and all of them are ordinary software.
This is the shape running in this laboratory. Four agents share one memory, one of them is always up, and the whole thing costs a few dollars a month to operate because the expensive model is only consulted when judgement is genuinely needed.
What you will have built One assistant with a single job it does daily, persistent memory, one channel done properly, a background process that heals itself, tools it may use without asking, and one narrow seam where a human approves before anything reaches the outside world.
Who this is for You can write a script and read a log. You need The Memory Layer first, or at least a store this can call; steps three onward assume it.
1. Give it one job, and refuse the rest
The failure is always the same and it happens on day one. You describe a general assistant, so the system prompt becomes a list of everything, and the thing is mediocre at all of it and trusted with none of it.
Write down one job it does every day without being asked. Draft the replies that arrived overnight. Turn yesterday's notes into tomorrow's list. Watch a feed and tell you when something matters. One sentence, in the present tense, naming a thing that happens whether or not you show up.
Everything below is in service of that sentence. When you are later tempted to add a capability, the test is whether the sentence needed it.
2. Pick the brain, then pick the cheaper one
You need two models, and the interesting one is not the big one.
A local model for the work that happens constantly: classifying, extracting, summarising, deciding whether something is worth your attention. It is free, it is private, and it is fast enough. In this house that runs through Ollama, kept resident so it never cold-loads.
A cloud model for the work where being wrong is expensive: judgement, writing you will put your name on, anything irreversible.
Route between them explicitly. A router is a dozen lines and it decides your running cost far more than your choice of provider.
Keep the local model resident A model that unloads between calls pays its load cost on every request, and on a contended machine that can be minutes rather than seconds. Pin it with a periodic no-op request that asks it to stay. This one line is the difference between an assistant that answers in a second and one you stop using.
3. Give it memory — and do not build it twice
Everything from here assumes the assistant can recall what it was told last week. Build that first, from The Memory Layer, or point this at a store you already have.
Two rules from that method matter here more than anywhere:
- Recall before answering, automatically. An assistant that must be told to remember is a chat window with extra steps.
- Extract at the end of a turn; never store the transcript. Assistants are the worst offenders for this, because they see everything.
Two failures that will look like something else
It goes quiet after a while. Almost never the model. Check first that the background service is actually running, then that a credential has not expired. A token with a lifetime measured in weeks will fail silently on a schedule long after you have forgotten it exists — log the reason a call failed, not just that it failed, or you will debug the wrong layer for a day.
It works from your terminal and fails as a service. Step seven. It is the environment, every time.