Governed Corpus Evolution
A write-up of a system anybody can use to maximize domain-knowledge injection into your agents when working on a large project. Written fairly detail-free, to best illustrate why I came up with it.
Like many others, I have been working on a large, long-running project using an agentic-development-first approach and encountered again and again the issue of the agent making the same mistakes, and docs bloating and regularly drifting from reality. I started adding rules, skills, architecture design records, and curating the whole protocol around something like Karpathy's Wiki. However, the issues were stark: context bloat, rule misses, ADR staleness, and a lot of proactive involvement from me, the developer.
The needed system is one that can reliably inject domain-specific, subject-matter-expert knowledge into a session. Over a few months of refinement, I came to the realization that I can leverage the concepts of LLM attention and transfer them to the docs themselves. This has borne the Governed-Corpus Evolution philosophy: an evolving, owner-led doctrine that curates and acts as the associative memory for the agent. Your artifacts are self-contained neural-network-style self-training loops, and you are in charge of the loss function.
In a snapshot
To quickly illustrate the philosophy, here's how a session could look in a long-running project. An agent is handed a task. It then scopes out the task (so far so good). Then, the agent walks a series of steps — let's say 2 for the illustration. One step is ADR consultation, and the other is rule adoption. In step 1 the agent reads each decision's summary, and decisions that are relevant to the task "fire", which causes the agent to read the decision itself. The rule-guidance step is similar, but not quite the same: rules are project-wide rules that are too heavy to carry per-surface. Thus, for each rule, the agent must ask a series of predefined questions (latches) against each rule's summary. Latches that bind to a rule "fire" and the agent then reads the entire rule. Thus, the agent programmatically derives all the rules and decisions necessary for the implementation of the task.
You're asking yourself: "isn't that just skills? All that's happening is progressive disclosure after all." Well, I mean, yes but not entirely. The basic implementation of each artifact type is essentially a skill, but the form by which it is defined allows it to be programmatically evolving. That is, if each artifact carries the attention learning loop within itself, the project can essentially continually refine the context, the latching correctness, and the lifecycle of the artifacts.
The crux
Each artifact carries the following shape: Activation (when does this fire?) · Payload (the judgment itself, stated generally enough to transfer) · Warrant (what makes it correct, i.e. evidence independent of the artifact) · Enforcement (what a mechanical check covers, and what it doesn't) · Lifecycle (who consumes it, and how it retires).
The most important thing: no durable artifact is born outside of the human's judgement. A system that tries to continually update itself without an external oracle is self-referential. It only acts on what it sees, and its blindspots compound. This 5-tuple, i.e. the pentad (real word, look it up), allows us to mimic the learning process by modeling each session as a forward pass, owner steer/agent misses as the loss function, and human judgement of artifact promotion into a one/few-shot backwards pass.
The lifecycle
Towards the end of a session, an agent walks a series of latches (notice the callback) to review what has been accomplished: what surprises occurred during a session (where it had to correct a previous assumption it made) or what owner steers accomplished that otherwise would have been missed. The session then logs each miss into the observation ledger, alongside a "re-check when" trigger. In a future session, an agent encounters an observation that has been previously recorded. That makes the observation into a promotion candidate for an ADR. This is the forward pass, if we use the neural net analogy.
An episodic pass occurs (usually at some session count cadence but up to you): in that pass another agent walks the observation ledger, and finds observations that occurred multiple times. It then surfaces to the user "promotion candidates" for an ADR. The user then decides on if it warrants an ADR, or more instances need to occur before it is generalizable (or it shouldn't merit an ADR at all). This is the crucial backwards pass — you need to decide on if and how a promotion occurs. The system is only as good as the developer behind it.
A similar thing happens for ADRs: if there are a couple of ADRs that have a similar shape across project surfaces, then a promotion for a Rule is proposed. The user then again is consulted. It is important to be vigilant — needless promotions are a standing tax on an agent, and incorrect promotion can lead to degraded performance.
Finally, artifacts that don't fire often or that are proven incorrect-as-written in the future can be demoted. Artifacts that carry the same duty across more than one of the artifact kind can be leafed.
Where this works
Programming concepts that are standard practice are already encoded in the models themselves, and as we know models are very very smart. This system is for user-side injection of domain expertise: the things that matter for your project, as reasoned by you. It is only as good as the engineer that governs it. You can insert field-specific knowledge as required for your work, or general practices of convenience for you.
It is also a standing tax for short-lived projects. If the entire project or at least large parts of it can live in a single session's memory, then you don't gain much from constructing an institutional knowledge distilled as associative recall. The benefit comes from auto-injection of domain knowledge that is accrued over many sessions or through a lot of experience on the side of the developer/engineer.
The full constitution and system philosophy is much more detailed and broad than just ADRs. It can be used for refinement of testing/planning/implementing agents, and continuous refinement and evolution of any record kind.
Note: the website/paper is largely AI-written from my internal notes. It's A LOT of content, so I haven't had the time to manually write down ~30 pages of implementation notes. You can skip to section 3 of the paper if you only care about the details and not the formality.
Hope this helps somebody!