The Hermes Saga Continues: Quest for a Functional Brain

Having built a personal AI agent that delivers a morning briefing, the obvious next ambition was to give her a second brain: capture, recall, connections, foresight. What I actually spent the day on was Unix file permissions, a container that rebuilds its own group table, and a database column that lived in the wrong table. The intelligence was never the problem. The title, for what it is worth, is only half about the machine.

---

Previously, on the Hermes Saga

The last episode ended in a state of qualified success. Eliza, my Hermes-based agent running on rogaldorn (*.*.*.95), was delivering a daily morning briefing again: agenda, weather, life context, written in her own voice and posted to Discord at 06:00. Along the way I had learned that Nous Research returns HTTP 404 for credit exhaustion rather than the 402 the fallback classifier expects, that a small local model will interpret "reply with [SILENT] if there is nothing to say" as licence to say nothing on a slow morning, and that a model asked about a Belgian village will cheerfully attribute the weather forecast to a cartoon marsupial.

The briefing worked. So naturally I decided it was not enough. A briefing is a monologue. What I wanted was a second brain: something that captures the things I tell it, remembers where it put them, surfaces the connections I would otherwise miss, and looks a week ahead rather than a day. This is the story of building that, which is to say, this is the story of file permissions.

The Ambition

Eliza already had the beginnings of a memory. Behind her sits a small dashboard service, running on the host as a plain systemd process, that indexes several of my self-hosted stores into a single snapshot: bookmarks from Karakeep, documents from Paperless, home inventory from Homebox, notes from SiYuan, and her own persistent memory files. She reaches it through a command-line tool called eliza-recall, which she runs herself inside her terminal. Ask her about a receipt, a plane ticket, or a half-remembered article, and she queries the snapshot and answers from what she finds. This part predated today and worked.

The plan was to fill in the three obvious gaps:

• Capture: a way to tell her, in the flow of conversation, to keep something. A standing preference should shape how she behaves. A concept worth keeping should become a note. A link should be filed as a bookmark.
• Foresight: a week-ahead view, so she can tell me on a quiet Monday that Thursday is a wall of meetings.
• Connections: a weekly pass that reviews what I saved recently and points out where it touches what I am already working on.

None of this is conceptually hard. An agent that can already fetch calendars, curl RSS feeds, and query a document index can obviously also write a note or read one more source. I estimated an afternoon. I was correct about the intelligence and wrong about everything underneath it.

The Bridge That Bypasses the Guard

The first realisation is that Eliza cannot be trusted with the host. She runs in a Podman container as an unprivileged user, UID 965, which is entirely the point: an autonomous agent that can execute shell commands should not be able to read every file on the machine. So when she needs to write to one of my stores, she cannot simply open the file. She has to ask something that can.

That something is the dashboard. It runs as root on the host, so it can reach the loopback-only services and the local databases. The container already talks to it for reads, so extending it for writes was natural: a POST /api/capture endpoint that routes a note into SiYuan or a link into Karakeep on her behalf. The same bridge, one more direction.

There was, however, a wrinkle that is worth stating plainly because it is the kind of thing that is easy to wave past. The dashboard binds 0.0.0.0, not loopback. It is reachable on the LAN, and it is proxied through Caddy behind Authelia for the web interface. The trap is assuming Authelia protects it. Authelia protects the Caddy virtual host. It does not protect the raw port, which is open on the local network and answers to anyone who asks. For a read-only snapshot this was already generous. For a write endpoint that can append to my knowledge base and queue crawls, it is a liability. So the capture route authenticates independently, with a bearer token that lives in sops, is handed to the container through its environment, and is checked on the dashboard with a constant-time comparison. "It only accepts structured JSON" is not a security control. A token is.

The Column That Was Never There

With capture and foresight built, I turned to the part I actually cared about most: training. I track my cycling and my sleep through a Polar watch, funnelled into a self-hosted workout tracker with a SQLite database. The recovery and sleep data in there is genuinely useful for deciding how hard to train on a given day, and I wanted Eliza to factor it into her advice. A dedicated eliza-training script reads the Polar tables for the last three days and prints a compact JSON summary: sleep quality, recovery status, recent load, a suggested intensity.

I had wired an earlier version of this into the morning briefing directly. That was a mistake, and an instructive one. Folding the Polar query into the briefing prompt made an already busy prompt busier, and on the mornings the primary model was unavailable and the job fell to the local fallback, the output degraded in ways that were almost comedic: five distinct meetings compressed into one vague line, the office-versus-home verdict inverted, and, on one memorable occasion, the editorial commentary delivered in fluent Mandarin. A 14 billion parameter model on CPU, handed a prompt with too many jobs, does the jobs badly and sometimes changes language while doing so. The fix was to stop overloading the prompt and give training its own briefing, at 06:01, a minute after the main one.

Which is when the training briefing stopped arriving entirely.

Not with an error. The cron scheduler reported the job ran, and reported it as ok, every single morning. It simply delivered nothing. Regular readers of the previous episode will recognise the shape of this: a status field that records that a job fired, not that it succeeded. I have now been bitten by the same distinction twice, which suggests the lesson is mine to learn rather than the software's to fix.

The prompt is written to reply with [SILENT], suppressing delivery, if the training script reports no data. So a silent briefing meant the script was returning "no data". Running the script by hand returned a clean, populated JSON payload. So the script worked, except when Eliza ran it, in which case it did not. I pulled the actual tool output from her session store and found this:

{"available": false, "error": "db error: no such column: nightly_recharge_status"}

The sleep query selected a column called nightly_recharge_status from the polar_sleep table. That column does not exist in polar_sleep. It exists in polar_recovery, one table over. I had transcribed it into the wrong query when I first wrote the script, and because the script is engineered to never crash the briefing, it caught the resulting error, returned "no data available", and let the prompt do the rest. Every morning, faithfully, it failed in exactly the way designed to look like nothing was wrong. Moving the column to its correct table fixed the query. This was the shallow bug. Underneath it was a deeper one that the column error had been politely hiding.

Root Will Lie to You

With the column fixed, I ran the script by hand again to confirm. Populated JSON, available: true, real sleep and training figures. I deployed, triggered the briefing, and got [SILENT] again. The tool output this time:

{"available": false, "error": "db error: unable to open database file"}

The script could not open the database. In her terminal. But it opened fine when I ran it. The two facts sat next to each other refusing to reconcile until I noticed the one variable I had been holding constant without realising it: who was running the script.

When I tested, I used podman exec, which defaults to root. Root does not care about file permissions. When Eliza runs the script in a cron turn, she runs it as the unprivileged UID 965. The workout tracker's data directory is mode 0700, owned by the workout-tracker service user. Nobody else may enter it. The file inside is world-readable, but the door to the room it sits in is locked, and UID 965 does not have a key. Root walked straight past the lock every time I tested, which is precisely why my tests kept passing. My verification method had a superpower the actual workload did not, and that superpower was hiding the bug I was trying to verify was gone.

This is the part I want to underline, because it is the most generally useful thing in this entire post. If you are debugging a permission problem and your reproduction runs as root, you are not reproducing the problem. You are reproducing a world in which the problem cannot occur. Test as the user that actually does the work, or the test will lie to you with a straight face.

The Group That Could Not Hold

The obvious fix is a group. Add the hermes user to the workout-tracker group, make the directory group-traversable, done. I had, in fact, already done the first half of this in a previous session, and could confirm on the host that the group membership was in place. It changed nothing, for two independent reasons, either of which would have been sufficient on its own.

The first: the directory is 0700. That grants nothing to the group. Group membership against a 0700 directory is a permission to do nothing, granted to a group that can already do nothing.

The second is more interesting, and is the reason I did not simply relax the directory to 0750 and move on. The container's hermes user does not actually have the host group. The host and the container maintain separate group databases, and the supplementary group I granted on the host does not cross the boundary. Podman can be told to pass a group in with --group-add, but the container's entrypoint drops privileges to hermes using a helper that calls initgroups(), which rebuilds the supplementary group list from the container's own /etc/group. Any group the kernel granted at launch, but which has no matching entry in that file, is silently discarded between process start and the dropped shell. This is not my discovery. It is documented, at length, in a comment in the Hermes upstream's own entrypoint hook, which had to solve exactly this problem for the Docker socket and left behind a small monument to the pain of doing so.

So the group approach was a dead end that looked like a road. To make it work I would have to relax the directory permissions on another service's private data, inject a matching group entry into the container's password database, and fight the privilege-drop mechanism the container is specifically built around. Three fragile changes, one of which contradicts the container's design, to grant a sandboxed agent a key to a room it was deliberately locked out of.

So the Brain Reads Through a Proxy

The correct answer was sitting in plain sight, because I had already built it for everything else. Every other database Eliza can see, she sees through the dashboard, which reads it as root on the host and hands her the result. The workout tracker was the sole exception: the one store I had bolted directly into her container as a bind mount, which is exactly why it was the one store that hit a wall the others never did.

So I moved it to match. The dashboard grew a GET /api/training endpoint that runs the existing, unchanged training script as root, reads the database it is allowed to read, and returns the JSON. Eliza fetches it with a small client, the same shape as eliza-recall, authenticated with the same bearer token as capture. The bind mount is gone. The group grant is gone. The unprivileged container touches no host database at all. My personal fitness agent reads my personal fitness data through a root proxy, because it cannot be trusted with a file handle, and that is, correctly, the whole point.

It worked on the first real cron run: populated data, an honest available: true, and a delivered briefing. After days of a job that reported success and produced nothing, watching it produce something felt disproportionately satisfying.

The Race, Again

Removing the bind mount changed the container's identity, which forced Podman to rebuild it, which is where the deploy failed. The activation aborted on the provisioner that registers Eliza's cron jobs, with:

Error: crun: executable file `python3` not found in $PATH:
OCI runtime attempted to invoke a command that was not found

This is the second time this exact race has broken a deploy, and having been asked directly what could be done about it rather than merely worked around, I finally sat with it. The provisioner waits for the container to be ready before it runs, but it was waiting for the wrong signal. It checked that the Hermes binary answered, which it does early, using an absolute path that needs no PATH lookup. Then it ran a small python3 snippet, by way of podman exec, to patch model and prompt settings into the jobs file. On a freshly rebuilt ubuntu:24.04 container, python3 is not yet on the PATH when the Hermes binary already answers, because the container's own setup has not finished installing it. The readiness check passed a gate that the actual work then walked into a wall behind.

The tempting fix is to make the readiness check also wait for python3. The better fix is to stop needing the container's python3 at all. The jobs file is bind-mounted: the copy inside the container is the same file as one on the host. The provisioner runs on the host, as root. So it now does the patch on the host, with the host's python3, editing the file directly with an atomic write, and never execs into the container for it. The only container operations left are the ones that use the Hermes binary the readiness check already verifies. The race is not mitigated. It is removed, because the operation that could lose it no longer happens.

There is a line in the previous episode I have been made to eat: "the correct response to adding a workaround is to immediately also add the system that makes the workaround unnecessary." I wrote that about a permission watchdog I had put off for weeks. I then spent this session adding a workaround for a race, twice, before removing the thing that caused it. The lesson does not appear to have taken the first time. It has taken now.

Feasible, Not Just Advisable

With the data finally flowing, a smaller point surfaced from actually using it. The training briefing would recommend an intensity based purely on my recovery, in a vacuum, with no notion of whether I had any time to train at all. "Go hard today" is not useful advice on a day that is meetings from nine to six. Recovery is only half the equation. The other half is the calendar.

The fix was small precisely because the pieces already existed. The same script that fetches my agenda for the morning briefing can be run by the training briefing too. The prompt now asks Eliza to read the day's schedule, judge how busy it looks, and match the session to the window that actually exists: a short spin in the evening on a packed day, a proper effort on a clear one. With a deliberate constraint, because the day view carries start times rather than durations: she reasons in coarse windows, early, midday, evening, and is told not to invent precise free slots she cannot actually derive. A second brain that suggests a two-hour ride during a meeting you have told it about is not a brain, it is a calendar you have to argue with.

An Inbox She Was Never Meant to Have

The last correction of the day was the most human, in that it was entirely my fault and had nothing to do with a computer misbehaving. When I built the note-capture lane into SiYuan, I built it as an inbox: a single document that Eliza appended timestamped lines to. Capture a thought, get a dated bullet in a running log.

This was the exact inverse of the intent. SiYuan is my knowledge base, not a scratchpad for the agent. The point was never for her to keep an inbox of my remarks. The point was for her to author real notes in my knowledge base when the conversation produced something worth keeping and I asked her to keep it: a proper document, with a clear title, capturing the substance of what we discussed in her own words, not a transcript of what I typed. I had built a machine that took dictation. What was wanted was a machine that takes notes.

The distinction matters, and the failure to see it is a familiar one. I had a working mental model of "capture to SiYuan", implemented it faithfully, and never checked that my model matched the one in the other person's head. The rebuild was straightforward once the misunderstanding was named: the capture now creates a titled document with body content she composes, and the inbox is gone. But the lesson is not really about SiYuan. It is that the most confident I feel about a requirement is exactly when I have stopped checking whether I have understood it.

What a Functional Brain Actually Is

Standing back from the day, the shape of the work is clear, and it is not the shape I expected. I set out to build intelligence: capture, recall, connection, foresight. What I actually built was plumbing. A bearer token because a port bypasses a guard. A root proxy because a directory is 0700. A host-side file edit because a container's python3 arrives late. A column moved from one table to the correct one. A note that is a note rather than a log line. Not one of these is about thinking. Every one of them stood between the thinking and its ability to do anything at all.

The intelligence, throughout, was the easy part. The model, when reached and given a clean prompt and real data, produces genuinely good output: a sleep summary, an honest recovery read, a training suggestion that respects my calendar. It never once struggled to think. It struggled to read a file, to resolve a column, to survive a rebuild. The brain was fine. The nervous system was the project.

This tracks with everything I have ever learned running services, dressed in slightly newer clothes. A second brain is not, in the end, a cleverer model. It is the same unglamorous discipline as any other piece of infrastructure: least privilege, honest error handling, a proxy where a sandbox meets the host, a test that runs as the right user, and the humility to check that the thing you built is the thing that was asked for. The quest for a functional brain turned out to be a quest for functional plumbing, and the plumbing, as ever, was where all the interesting failures lived.

There is a small admission in the title, and it is not about the code. A second brain is, by definition, insurance against the first, and I did not commission one because my own memory is flawless and I fancied a spare for show. No unaided human brain holds a homelab's worth of state, a week of meetings, everything worth reading, and a training load in working memory at once, and mine is emphatically no exception. That is the honest incentive: not that the machine is cleverer than I am, but that it is better at not forgetting, which on a Tuesday afternoon amounts to much the same thing. The quest for a functional brain is, in the end, mostly a quest to stop losing the threads I already have.

Tomorrow the briefing arrives at 06:00 and the training briefing a minute later, and if the credits hold it will tell me, correctly, that I slept poorly, that my recovery is nothing special, and that the only real window to ride today is after eighteen hundred. Which is more than my old paper training log ever managed, and it only cost me an afternoon of file permissions to get there. I am, once again, choosing to regard that as a feature.