The mechanism, and the failure each part exists for
Ten skills that mostly never call each other. A dependency graph that had been returning a confidently empty answer since the day it was first published. A behavior record that turns "what did I just break?" into a query. And a governance layer whose whole design rests on one distinction: what kind of check produced this failure. This page is the why behind the behavior — read Using it first if you just want it running.
Artifacts, not calls
The skills form a tiered graph. One foundation skill derives a dependency
graph from your source; everything above it reasons about blast radius rather than about
changed files. For the most part they do not call each other's code at all — they cooperate by
reading and writing one shared, version-controlled knowledge-base/ folder. A skill
leaves a structured file on disk; the next skill picks it up.
There is one named exception, and stating it plainly matters more than the tidy version: the
behavior-layer skills do reuse peer code directly. freya-behavior-graph and
freya-behavior-runner import freya-spec-manager's frontmatter parser
in-process and run freya-code-graph as a subprocess. Everything else is files.
code-graph ──graph.json─────────────▶ docs-manager · spec-manager · security-scan · behavior-*
spec-manager ──specs/─────────────────▶ behavior-graph · security-scan (intentional) · status
behavior-runner ──fingerprints (stdout)──▶ behavior-graph
behavior-graph ──behavior.json──────────▶ status · security-scan
security-scan ──findings.json / report─▶ status · security-resolver
status ──reads all of the above─▶ BACKLOG.md
wrap-up ──orchestrates───────────▶ everything, under two commits
Every arrow there is a file on disk except one:
behavior-runner writes nothing at all. It prints fingerprint JSON
to stdout and behavior-graph reads it. That is deliberate — the runner is the
messy, tooling-coupled half (it boots test runners and captures runtime coverage), and keeping
it a pure producer is what lets the graph half be unit-tested without ever executing a test.
The contract underneath all of it: no skill hard-requires another. Each works standalone and works better together, and a missing sibling narrows coverage instead of breaking the run. One concrete case, which every impact-aware skill honours:
code-graph available
Ask it for the impact set and process the full blast radius — the changed files plus everything that transitively depends on them.
code-graph absent
Fall back to a plain git diff of the directly-changed files,
process only those — and say so, with a "reduced coverage" warning. Never
a silent narrowing.
The fallback rule is written as pseudo-code, per skill, and the tier-by-tier dependency listing lives beside it. Reach for these when you are implementing against the contract rather than reading about it — and for the argument for many single-purpose skills over one monolithic prompt, which this page assumes rather than makes.
The five tiers
Each tier depends only on the ones below it, and every dependency is optional: a higher-tier skill uses a lower one when it is installed and degrades when it is not. The last row of the panel below — degrades — is the one worth reading. It is the only place the ten answers to "and what happens when its dependency is missing?" are written down together.
The graph substrate
This is the keystone, and the place a mistake propagates furthest.
freya-code-graph builds a reverse-dependency graph — for every
file, who depends on it. The backend that ships with the toolkit does it by scraping import
statements with regular expressions across TypeScript/JavaScript, Python and Go. Regular
expressions, not a real parser: that is a stated limit, not an oversight, and it is what keeps
the whole toolkit stdlib-only with no install step. A second
backend, opt-in, reads forty languages with real ASTs — the rest of this section is about
the one that always ships, because it is the floor everything falls back to.
impact(file) = file + direct_dependents(file) + transitive_dependents(file)
Computed with a cycle-safe walk over the reverse edges. Consumers ask for the impact set and then process everything in it — never just the files you literally touched.
The load-bearing detail sits one level down, in graph_ops.py's
_classify_import and its IMPORT_SIGNALS constant. Every import edge is
tagged exactly one of four ways: internal (no prefix — it resolved to a real
file in your project), external:<pkg> (a third-party
package, recorded as a leaf and never traversed),
unresolved:<import> (it looked internal but could not be
mapped to a real file), or — new since 0.3.0 —
outside:<alias>/<path> (it resolved to a real file
under a directory the project declared outside its own root; see
ADR-031). That fourth one is the interesting case, because it
did resolve and is still not a node: a declaration buys resolution, never a place in
the graph's key space — nothing under a declared root is scanned, walked or globbed, and the
declaration causes no file under one to be read. Only the built-in homegrown resolver
emits that tag: graphify does not consult declarations, so on that backend a
declared root reports crossings: 0 whether or not anything crossed it, and the
zero means "never looked" rather than "nothing there".
An edge counts as internal only when it carries none of the three prefixes
— a single predicate, reused well beyond the graph. spec-manager's onboarding
detector uses it to tell a real codebase from a boilerplate scaffold by counting internal edges
rather than files. And an import that cannot be resolved is surfaced, not silently
dropped, which is the entire point of the third tag.
That third tag exists because of one bug, and the bug is the best single piece of evidence on this site for why the rest of the design looks the way it does.
On the first real path-alias project — a 229-file Next.js app — the graph built cleanly and
reported 1052 import edges and 0 internal edges. Every internal import in
that codebase used the @/ alias, and the resolver treated every non-relative
import as external. Asking for a route's dependencies returned [] despite three
real @/lib/* imports, with no error and no unknown signal.
So impact analysis was returning an empty blast radius that looked complete,
and every consumer standing on it — spec updates, docs updates, all of the behavior layer's
impact work — was silently riding on nothing. A git check settled how long: the
resolver was unchanged since v0.1.0, so this had been true since the toolkit was first
published. Dogfooding did not introduce it; dogfooding was simply the first thing to look.
The fix taught the resolver to read tsconfig path aliases, anchored resolution to
the project rather than the process working directory, and introduced the three-tag taxonomy so
that "no dependencies" became distinguishable from "could not resolve". The rebuild:
before: 0 internal / 1052 external → after: 607 internal / 488 external / 0 unresolved
A confidently empty answer is the dangerous failure — worse than an error,
because an error stops you and an empty answer gets acted on. Everything downstream is built
around refusing to produce one: coverage that cannot be captured is reported as
unknown with a reason rather than as an empty file list; a drift check with no
dependency graph narrows its scope and says so; a security run where every worker
failed exits non-zero rather than reporting a clean codebase.
Then the resolver stopped being the architecture
Everything above describes a resolver that reads four languages. Point it at a Java codebase and it found nothing, reported "Built dependency graph: 0 files scanned", and exited successfully — and the shape detector, which decides whether a project is new by counting internal edges, then classified a decade-old codebase as an empty scaffold. The obvious fix is to find a better parser and wire it in. That was rejected: picking a tool means picking again in two years, by which point five skills depend on its output shape.
So the parser is not the architecture. The socket it plugs into is. The graph is produced by a backend behind a fixed contract, and everything downstream reads one artifact shape whichever backend ran.
homegrown | graphify | |
|---|---|---|
| How | regular expressions over source text | tree-sitter ASTs, via an external binary |
| Needs | nothing — Python standard library | the binary on PATH |
| Reads | 4 languages, 6 extensions | 40 languages, 93 extensions |
| Relations | imports, re_exports | those plus calls, inherits, references |
| Role | the floor — always there | opt-in, named by a person |
Two backends, not one, deliberately. An interface with a single implementation is fiction: it encodes the assumptions of its only caller and nobody finds out until the second arrives. Not a slogan here — the contract was written with one backend behind it, and the review that built the second found the contract could not actually run anything else. Saving the file, validating it and building the reverse index all lived inside the original resolver, so a backend could satisfy every documented obligation and still write nothing while reporting success.
The old resolver stays because the case that started all of this is a locked-down work laptop. freya needs nothing but Python; the other backend needs a package install and network access. If policy blocks that, it never runs in the one environment this exists to serve — so keeping the built-in one is what guarantees freya degrades to something everywhere rather than to nothing.
"A confidently empty answer is the dangerous failure" was implemented at the repository level: a Java repo will not call itself greenfield. It was not implemented at the answer level, and those are different claims — "3 dependents" and "3 dependents, and I could not read a fifth of this repo" are not the same sentence. Every answer now carries what its backend could not read, with the directories to search instead, and carries nothing at all when there is nothing to say.
The consumer is the agent, not a person: a build runs with no keyboard attached almost every time, so a printed warning lands nowhere.
The decision to repair the homegrown resolver in place rather than adopt an off-the-shelf parser, and then to make it a floor behind a contract rather than replace it — each with the alternative it beat.
Incremental by construction
Full scans are expensive and most changes are localized, so every re-syncing skill works from the last commit it processed rather than from scratch. Five steps, identical everywhere:
- Read the tracking file →
last_commit - Diff —
git diff last_commit..HEAD --name-only→ the changed files - Expand — if
code-graphis available, take the impact set; otherwise the changed files alone, with a warning - Process only that set
- Rewrite the tracking file with the current commit
This is what makes the daily loop cheap enough to actually run after every feature, rather than a thing you promise to do at the end of the sprint. Which skill owns which tracking file, and what each one remembers, is a four-row lookup that the markdown already owns.
The tracking-file table and the naming convention behind it. Reach for these when you are debugging why a skill re-processed more or less than you expected.
Two commits
Generated artifacts reference the code commit they describe. If an artifact shares
the commit it describes, that reference is unstable — it points at a commit that did not exist
when the artifact was written. So wrap-up splits every run in two: code lands
first, artifacts second.
Commit 1 — code
src/lib/auth.ts
src/api/routes.ts
tests/auth.test.ts
Commit 2 — artifacts
knowledge-base/reference/API.md
knowledge-base/specs/auth/SPEC-001.md
knowledge-base/security/…/YYYY-MM-DD.md
What that buys: cleaner history, a stable commit for the security scan to reference, and no need for tracking-file hacks to work out which commit an artifact described.
Once the behavior layer is in play, a Gherkin scaffold's commit class follows its
lifecycle state, not its location. A proposed scaffold still
carrying its TODO(scaffold) marker rides the artifacts commit even
though it sits in the code tree — and joins the code commit only once the behavior is
accepted and the marker is gone.
The pattern write-up, with the problem statement, the file lists, and which skills apply it.
Who schedules the fan-out
Three flows fan work out to parallel workers: docs-manager across twelve
document types, spec-manager scan across five discovery areas, and
codebase-security-scan across six vulnerability categories plus a per-finding
refutation pass. All three originally issued parallelism as an unconditional imperative, with no
fallback written anywhere. Two of them still ask; one stopped asking. The difference is the most
instructive thing in the toolkit.
Asked to run six category scans in parallel, Copilot ran them itself as a sequence of greps
— and then reported that it had run them in parallel. Instrumenting a documented
twelve-way fan-out with --log-level debug settled it by counting
tool calls: view 9, bash 8, skill 1, rg 1,
task 0, explore 0. Zero delegation. All thirteen
files were written by the main loop.
The diagnosis is the point, and the obvious diagnosis is wrong. The problem was never that sequential is slower. It was that nothing could tell you which mode had run, because the agent narrates parallelism either way — and a correctness property you cannot observe is a hope, not a property. No amount of careful wording makes it checkable, and no test that reads the agent's reply can catch it failing.
Copilot ships task, an explore agent, /fleet and
/subagents; the machinery is all there. Its own system prompt tells it never to
delegate parts of a codebase small enough to read directly,
"regardless of how it divides into separate areas" — which describes six named
categories, twelve named doc types and five named spec areas exactly. A reasonable host policy
that happens to disqualify this design.
The honest limit: both clauses of that instruction are conditioned on small scope, and every observation here was taken on a small fixture. So what is established is narrower than "Copilot never delegates" — it is that for scopes small enough to read directly, it will not delegate a labeled-area fan-out, by design.
The security scan therefore stopped asking, and runs its own worker pool. docs-manager
and spec-manager deliberately keep the prose form — not for lack of scale, since
docs-manager fans out to twelve workers against the scan's six, but because
their workers write files, and the driver's guarantee rests on workers that
cannot.
What the driver does once it owns the loop
Stripped of jargon: read the project docs, run six searches at once, merge duplicates, hand each surviving candidate to three challengers, report.
The challengers are the interesting part. Every candidate finding goes to three separate reviewers, each told to disprove it from a different angle: can an attacker actually reach this, is something already preventing it, was this done deliberately. A finding is discarded only if all three reject it. If they disagree it is kept and flagged for a human. If they all failed to answer it is also kept — because no answer is not an answer of no.
The cheaper-looking saving is fewer skeptics. It is a trap. A finding is deleted on a unanimous refutation, so with one lens a single sceptic having an off moment silently deletes a real vulnerability — and it disappears with no trace in the report. The cheap mode takes its saving out of discovery instead, cutting rounds from five to one and leaving verification untouched. One driver, one definition of a finding: two definitions would drift, and the cheaper one would drift toward missing things.
The decision to own the fan-out in code, including the read-only allowlist the workers run under and why "deny beats allow" was not sufficient.
The behavior layer, and the failure it exists for
Ordinary tests are written from the code, so they mirror what the code does rather than what it should do — and a test that mirrors the implementation cannot notice the implementation being wrong. Behavior drifts while a green, high-coverage suite stays green. Coverage measures how much code you executed; it never measures whether the code was supposed to do that.
A BEH-NNN record fixes that by being a stable, id'd statement of
intended behavior that carries a lifecycle and links through a real test down to the
code it exercises. Which makes two questions answerable that a plain test suite cannot answer:
what intended behavior does this change touch? and is this behavior still
verified?
behavior-graph projects the SPEC → BEHAVIOR → TEST half
from your spec frontmatter — those edges are authored and deterministic — and
merges in the TEST → CODE half from real coverage. That last edge is the one
genuinely new thing here, and it is never asserted as certain: it carries a
provenance and a confidence, and says so when it cannot be established. Having both halves is
what lets impact flow in both directions.The record itself lives in a spec's frontmatter. One concrete artifact teaches the model faster than a paragraph about it:
behaviors:
- behavior_id: BEH-003 # stable across renames; never renumbered
title: Unknown email does not reveal whether a user exists
state: accepted # proposed | confirmed | accepted | quarantined | deprecated
adapter: cucumber # required only when state == accepted
level: integration # the runner's dispatch key
locator: features/auth/passkey-login.feature#unknown-email-does-not-reveal-whether-a-user-exists
The projection lands in behavior.json, a sibling of
graph.json in knowledge-base/.graph/ — and, unlike the parse
cache beside it, committed — kept
deliberately separate so the code substrate stays swappable. code-graph never
learns about behaviors at all: behavior-graph queries it for impact and queries
behavior-runner for coverage, and the dependency points one way only.
The polyglot substrate initiative had leaned toward unifying the code graph and the behavior graph, which ran directly against the sibling decision above. It was resolved the other way: three artifacts, one owner each, joined on file path. The producers have different dependencies and different failure modes — a parser may be absent, a test suite may be red — and a single combined file could not say which half had survived. A linking graph would have been an empty table, because every artifact already speaks file paths. ADR-025
Why intended behavior is modelled as an executable artifact at all, and why execution is split from the graph across two skills.
Four kinds of intent
Not all intent is a testable behavior, and confusing an executable guarantee with a prose decision is how governance turns into noise. The toolkit sorts intent into four kinds, each with a home and a way it is verified.
| Kind | Example | Home | Verified by |
|---|---|---|---|
| Executable behavior | "an unknown email does not reveal whether a user exists" | a BEH-NNN record in a spec, linked to a test in the code tree |
running the test, through its adapter — automated regression |
| Declarative decision feature-local |
"this feature offers no password fallback" | a spec's ## Intentional Design Decisions |
human review, plus a certainty score |
| ADR cross-cutting |
"we use Postgres, not Mongo" | decisions/ADR-NNN.md |
human review, plus the contradiction check |
| Principle project-wide |
"every endpoint is authenticated by default" | principles.md |
applies to all of the above |
The middle two are where people get lost, because "decisions" means two different things. A per-feature decision is tied to exactly one feature — "no password fallback, it's a phishing vector"; "uniform 404 to prevent user enumeration" — and lives inside that feature's spec. A cross-cutting decision constrains the whole project — "we use Postgres"; "multi-tenancy is row-level, not schema-per-tenant" — and has no single feature to live in, which is why it needs its own file and its own numbering.
The authority ladder
When two intents collide, the higher one wins by default and you fix the lower one — or you consciously amend the higher one.
| Rank | Record | Scope |
|---|---|---|
| 1 — highest | principles.md | The constitution. Project-wide rules, above every spec and decision. |
| 2 | decisions/ — ADRs | Cross-cutting architecture decisions. Only accepted ADRs are authoritative. |
| 3 | specs/ | Per-feature intent, its behaviors, and its feature-local decisions. |
| 4 — lowest | reference/ | Never authoritative. It describes how the code currently is and is reverse-synced from it. |
The case people get wrong is the same-tier one: two items on the same rung have no automatic winner. A spec contradicting a peer spec, or an ADR contradicting a peer ADR, is a consistency conflict to reconcile — fix either side, or refute the finding if they do not truly conflict. There is no default.
An earlier design put specs and cross-cutting decisions on a single middle rung, as equals. That was refined away deliberately: without an order between them, an ADR would be the one authoritative artifact that nothing governs, and the contradiction check would have nothing to resolve against. The four-tier order is what makes the resolution defaults fall out mechanically — spec contradicts an ADR, fix the spec; ADR contradicts a principle, fix the ADR; peer contradicts peer, reconcile.
The authority order and the single-ownership rule that travels with it — a generated projection is allowed, a hand-maintained duplicate is forbidden.
Adapters: link, don't re-author
An adapter is how a behavior reaches whatever verifies it. The layer detects a project's tooling rather than hardcoding a stack, and there are three shapes.
Author a tagged scaffold — the default for new, user-visible behavior
spec-manager writes a skeleton .feature into your code tree
carrying the @SPEC/@BEH reverse-link tags and a
TODO(scaffold) marker — but no real steps and no step definitions.
Authoring those is human forward-design work. The tags are the reverse link.
@SPEC-001
Feature: Passkey Login
# Intent and rationale live in knowledge-base/specs/auth/SPEC-001-passkey-login.md
@BEH-001
Scenario: Successful passkey login
# TODO(scaffold): replace with real steps. Step definitions are not generated.
Given <initial state>
When <action>
Then <expected outcome>
The TODO(scaffold) marker is load-bearing: a behavior
that reaches accepted while still carrying it is a
deterministic verify-time error. That is a lie about a guarantee, and the
gate catches it even though the suite is perfectly green — because nothing ran.
Point a locator at a test that already exists
No file is written and nothing is rewritten. Set adapter to
the runner and locator to path#scenario-slug or
path::node. The locator is the link — no reverse tag required. This is
what keeps adoption cheap for a project that already has tests.
adapter: vitest
locator: lib/webauthn.test.ts::rejects an expired challenge
Human-verified, no runner
The manual adapter is for behavior that genuinely cannot be automated. It
carries the intent and is skipped by locator resolution — the honest ceiling of what tests
can reach, recorded rather than pretended away.
The adoption fact worth stating plainly, because it is the one that decides whether anyone can use this on an existing codebase: the toolkit never writes real test steps, and you do not have to rewrite the tests you already have.
proposed
A scan or bootstrap produces a review queue of
proposed records inside spec frontmatter — never accepted, and never
files written into the code tree. A human accepting a candidate is the only thing that
promotes it and lets a scaffold or a link land as code. The reason is not caution for its own
sake: auto-generating authoritative-looking tests from the implementation would reintroduce
the exact "tests mirror code" problem the whole layer exists to fix.
The adapter binding and the execution split; and the bootstrap-as-proposed, drain-lazily adoption model with the alternatives it beat.
Both directions of blast radius
Because behaviors are linked to the code they exercise, impact runs both ways. Below is the
real dogfooding graph: SPEC-001 owns three behaviors, two accepted and
fingerprinted, one still proposed. Click a code file for
Direction A — which accepted behaviors exercise it, the regression question
wrap-up asks — or a behavior for Direction B, which files it
exercises, the planning question.
Editing lib/webauthn.ts flagged exactly the two behaviors that depend on it;
editing an unrelated lib flagged none; editing the route flagged only BEH-003.
False-positive rate: 0. An incremental --check on a change
touching no exercised code took 0.07 s with zero re-runs, against roughly
1.4–2.4 s for a full graph build.
FP=0 and 0.07 s come from 2 behaviors and 3 changes. That validates the
mechanism; it says nothing about trustworthiness at scale. Which is exactly why
fingerprint-driven governance stays advisory — only a real
test-failed of an accepted behavior blocks anything.
Coverage you can trust, or an honest unknown
A fingerprint is the set of files a behavior's test exercises. It is one of exactly three honest values, and the third one is never faked.
| Coverage | Confidence | What it actually is |
|---|---|---|
| observed | 0.8 | Real runtime V8/istanbul coverage under vitest — every file with at least one executed statement. Exactly the files the test ran. |
| static | 0.5 | The transitive import closure of a declared entry, from code-graph.
Analysis, no execution: everything the entry could reach. |
| unknown | — | No usable coverage. Always carries a machine-readable reason, and an empty exercises list — never attributed to code falsely. |
When the layer cannot observe exactly which code a test runs — an integration test against a bundled app — it falls back to the whole import closure and flags too much. That is fine. A false "might be affected" costs you one extra test run; a false "not affected" misses a regression. The two errors are not symmetric, so the design is not symmetric either.
Merge by trust
Rebuilding the graph must never quietly lose good data, so a new run is merged against the prior edge rather than replacing it:
| Incoming run | Result |
|---|---|
observed | Take it — highest trust. |
static | Take it, unless the prior edge was observed — never downgrade. |
unknown + test-failed | Invalidate. The test is red. |
unknown + any other reason | Preserve the prior fingerprint. |
That last row is the whole idea: a transient capture failure must not erase good data. Only a genuinely red test invalidates a fingerprint.
There are seven reasons an edge can come back unknown — a level not yet
implemented, a red test, a missing coverage file, an entry that was never
declared, a declared entry that is not on disk, no built graph, or a behavior the
runner simply emitted nothing for. You do not need the list to
understand the design; you need the rule behind it. An unresolvable edge is reported as
unknown with an empty exercises list rather than faked,
because governance leaning on a small, confident, empty answer is the dangerous failure — the
same failure the graph substrate had. The enumeration is on
Reference.
Why behavior tests drive the app over its real interface rather than by importing internals, and why coverage follows that choice — observed at unit, static closure at integration.
The life of one behavior
Trust in a behavior is not a number — it is the lifecycle state. Five states, and each one is a different set of machine facts:
Now the same thing as a story. BEH-003, end to end, from a machine's guess to a
standing guarantee:
Inference mints a candidate
A bootstrap of the existing codebase infers BEH-003 —
"unknown email does not reveal whether a user exists" — and writes it as a
proposed record in SPEC-001's
frontmatter. No file enters the code tree. Nothing is trusted, and nothing needs reviewing
yet; the certainty score only prioritizes the pile.
A change touches its code — confirm on hit
Weeks later a change touches the authenticate route. Wrap-up's validate-on-hit lists
BEH-003 as one of the two or three candidates this change touched. You re-read the
intent against the current code and confirm it:
proposed →
confirmed. The intent is now real and a test is
owed. With a declared entry it already appears in blast radius — static,
advisory — but it is never executed and can never block.
The test lands — accepted
You author the cucumber scenario that drives the authenticate endpoint over real HTTP
against a running instance. The tags round-trip, the scaffold marker is gone, the test
passes: confirmed →
accepted. The .feature file now
moves out of the artifacts commit and rides the code commit — its class
followed its state, not its location — and the behavior carries a static fingerprint of
three files.
Standing guard
Months later someone edits lib/webauthn.ts. Direction A flags BEH-002 and
BEH-003, and wrap-up re-runs the affected accepted behaviors it can execute — not the
suite. BEH-002's vitest unit test runs; BEH-003 is level: integration, so
today the runner refreshes its static closure rather than executing the cucumber
scenario, and it therefore cannot report test-failed. If an executed
behavior goes red, wrap-up blocks until the failure is classified: a
regression (fix the code), an intended change (declare it), or a test-infrastructure
failure (quarantine). It cannot fail silently, and it cannot fail unclassified.
The guarantee changes — on the record
The threat model changes, so the code and its test are edited together. The suite
stays green — and that is the problem. The G1 gate blocks:
an accepted test changed with no record. One command mints INTENT-001 naming
BEH-003 with an approver and a rationale, and wrap-up passes. A silently redefined guarantee
became a visible, deliberate, auditable one.
Off-ramps, never silence
A flaky fixture starts failing the test for infrastructure reasons? quarantined — out of the authoritative set until repaired, so a known-flaky test cannot block everyone. The feature is retired? deprecated. Both preserve history. Neither is a silent deletion.
Governance: block on facts, resolve on judgment
Capturing intent is half the job. The other half is enforcing it when code and specs change — and the defining rule of that half is: a failure is gated by what kind of check produced it, not by a model's self-reported confidence.
Block on facts
A broken test link, a duplicate ID, an accepted behavior's test regressing, an accepted test edited with no record. These are cheap and certain. A non-zero script exit stops wrap-up until you fix it or reclassify the behavior.
Resolve on judgment
Does this violate a principle? contradict a decision? drift from a declared choice? These are never script hard-blocks. Each finding must be fixed, refuted or amended before wrap-up completes, and they fail open on missing inputs.
The reasoning is what makes this a position rather than a description. A model's "high certainty" is not a calibrated probability, and blocking on it would train people to rubber-stamp whatever escape hatch exists in order to get past the noise. Once bypass is reflexive, real violations get waved through too. Confidence becomes a hard gate only after its false-positive rate has been measured on a real project and shown to be acceptable — which is a piece of evidence that does not exist yet, so the promotion waits by design.
The obvious objection — so the model check is toothless? — has a specific answer. A flagged violation hides two questions: is it real (the model's fallible judgment) and what should happen (non-negotiable). Not hard-blocking answers only the first. So "ignore and push" is not a resolution, model findings are never carried forward as backlog debt — each is resolved in the wrap-up that raised it — and refuting a false positive is a first-class, legitimate resolution. That last one is precisely what removes the reflexive-bypass pressure: if disagreeing is a supported move, nobody needs a hack.
G1 and G2 compare a recorded intent against code — a test, a diff. G3 compares one recorded intent against another recorded intent. Which is why the question "which intents does G3 know about?" is the whole ballgame for that check, and why it is the only one whose scoping argument is about recall rather than noise.
The two-tier enforcement decision, including why model checks fail open and what evidence would be required to promote one to a hard gate.
The six checks
Six checks, each protecting a different kind of intent. Pick one to see what it compares, what it guards, and whether it can block:
G1 — declared-intent records
Name the hole first. Code-versus-test disagreement is a fact you can run, and a red accepted test is the strongest check in the system. But that fact layer has exactly one blind spot: editing the test itself. Change the code and its test together and the suite stays green, so the strongest check sees nothing at all while the guarantee is silently redefined. That is the only way to slip a changed guarantee past every fact-check.
G1 plugs it by giving an accepted test one sanctioned way to change: a
durable INTENT-NNN record created in the same change-set, naming the behavior. And
because G1's own question — "was an accepted test edited, and does a record name it?" — is
itself a fact, the gate stays fully deterministic and hard-blocks.
Three specifics decide how far it reaches:
- Only
acceptedbehaviors are governed. The others have no standing guarantee to protect. - Temporal self-scoping. The record must be new since the baseline — a past record cannot bless a future edit.
- The gate verifies that a record exists, not that its rationale is honest. Honesty is a separate, model-judgment track.
# You changed BEH-003's anti-enumeration response — code and test together.
# The suite is GREEN. wrap-up commits the code, including the test edit.
# Then, in phase 3.5:
$ freya verify-intent --project .
1 accepted test change(s) without an intent record:
[BEH-003] SPEC-001: features/auth/passkey-login.feature changed —
file knowledge-base/intents/INTENT-NNN.md naming BEH-003
(freya intent new --behavior BEH-003), or revert the test edit.
# exit 1 — blocked
$ freya intent new --behavior BEH-003 --approver Alex \
--rationale "Anti-enumeration response changed 404 → uniform 200
per the revised threat model."
$ freya verify-intent --project .
OK — no accepted test changed without an authorizing intent record.
# exit 0 — the artifacts commit carries INTENT-001;
# the code commit's trailer points at it.
The control case is worth more than another example of it firing, because a control case is
what proves a gate is narrow: editing a proposed behavior's test is
never blocked. And chat history saying you meant to does not count — the record is the source of
truth, because a file is auditable six months later and a conversation is not.
Quarantining or deprecating a behavior in the same change-set takes it out of the
accepted set the gate reads off disk, so the gate no longer applies to it. G1 is
path-based, so an assertion that lives in a shared test helper rather than in
the declared locator escapes it. And with no baseline marker yet — a fresh repo, or a
first full scan — the gate skips rather than false-blocks.
G2 — principles
principles.md is the highest-authority record in the project, and a passive file
is not enforcement. So it is enforced two ways: soft injection, where the
constitution is loaded into the working context when you create or scan so you draft against the
rules rather than discovering them afterwards; and a resolve-to-proceed checkpoint
at wrap-up, where the change diff is judged against each principle and every finding must be
resolved before wrap-up completes.
An empty constitution means the step skips. You cannot check against an empty room.
G3 and P4a — contradictions and ADRs
G3 is the uniquely intent-versus-intent check. It fires when a spec's or an
ADR's intent is created or changed, and compares it against principles.md plus the
intentional decisions of other specs in the same category. It is scoped, deliberately:
a whole-repo re-derivation on every spec edit would be slow and noisy, and noise erodes trust in
a gate faster than anything else.
P4a takes the opposite line on ADRs, and the argument for it is the
strongest piece of reasoning in the governance layer. A changed spec is compared against
all accepted ADRs, with no scoping whatsoever — no applies_to
field, no tag filter, only lifecycle status. The reason is asymmetric cost. Scoping only decides
what reaches the model; the model still makes the judgment. So over-scoping is a silent
miss — which is unrecoverable, and is the exact failure the whole layer exists to
prevent — while under-scoping is noise the model dismisses in one line. Malformed ADRs surface
as warnings rather than being silently dropped, and the deterministic adr verify
(duplicate ids, dangling supersede links, bad status) is a Tier-1 integrity fact that
hard-blocks, separately from the advisory contradiction judgment.
What that combination covers, with the worked example for each — all four rows now caught:
| A change that contradicts… | Example |
|---|---|
| a principle | a new spec adds a public endpoint; the constitution says "authenticated by default" |
| a same-category peer spec | a new auth spec allows password fallback; a peer spec says never — phishing vector |
| an accepted ADR | a spec assumes Mongo; the ADR chose Postgres |
| a principle, from an ADR | a changed ADR contradicts the constitution above it |
P4b — declarative drift
The mirror of G3. Where G3 is intent-versus-intent, P4b is code-versus-declared
intent: does the changed code contradict a decision a spec or an ADR declared? Because
it is triggered by a code change it is code-anchored, and therefore
blast-radius scoped — a decision's related_code intersected with
the impact set — deliberately not always-global like P4a.
| P4a · intent ↔ ADR | P4b · code ↔ declared intent | |
|---|---|---|
| Nature | intent-vs-intent, no code anchor | code-vs-intent, triggered by a code change |
| Scope | always-global — every accepted ADR | blast-radius-scoped — related_code ∩ impact |
| Errs toward | recall — a missed ADR is unrecoverable | quiet — a code-triggered check must stay incremental |
| Refuses | a silent miss | whole-repo re-nagging |
Two checks that look inconsistent, each correct in the opposite direction, for stated reasons.
And when code-graph is absent, P4b narrows to changed-files-only and says
so in its output rather than returning a silently empty set.
Because P4b follows related_code, a spec or ADR that declares intent but
carries no related_code is invisible to the check — its
intersection is always empty. Rather than hide that, freya drift gaps lists
exactly those drift-blind items on demand. The recall gap is published, not silent.
Resolution logs
Three append-only JSONL logs sit at the knowledge-base/ root — one per
resolve-to-proceed guard: principle-resolutions.jsonl,
contradiction-resolutions.jsonl, drift-resolutions.jsonl. All three
share one implementation (resolution_log.py) doing the same three things: append a
line, load the latest record per key, drop the superseded ones. Four verdicts:
refuted, amended, auto-cleared,
superseded.
The asymmetry that matters: a straight code fix leaves no log entry, because git is the record — the code changed and the finding is gone. The log's only job is to stop the check re-nagging about an already-refuted false positive on every later change. And retirement is a later superseding record, never a mutated field, so the audit trail is never erased.
Then the trichotomy that stops governance becoming a re-litigation treadmill. On the next wrap-up a prior resolution is re-judged against the current code and lands in one of three places: it auto-clears if the flagged code is materially unchanged, is retired if the code moved out from under it, or escalates if the reason no longer excuses what is there. Four guardrails keep that honest — re-judge against the specific prior reason rather than the whole file, bias to escalate on any ambiguity, always log an auto-clear, and a finding with no prior always reaches the human. That last one is the safety floor.
The three decisions this section rests on: what each guard requires, how each check is scoped and why the two scopings differ, and the append-only log format with its recurrence rules.
Where it runs: wrap-up phase 3.5
All of it happens in one place, in one order — and the order is the argument. Deterministic facts settle first and can block; model judgment runs only after, and resolves-to-proceed.
1 · Link and ADR integrity hard-block
Every locator resolves, Gherkin tags are present and round-trip, no
accepted-but-still-TODO(scaffold), no duplicate
BEH-NNN, no orphan tag, and ADR frontmatter and supersede links are valid.
Cheap and certain, so a non-zero exit stops the run.
freya verify-links --format json && freya adr verify --project .
2 · Declared-intent gate (G1) hard-block
If an accepted behavior's linked test was modified or deleted in this
change-set without a new INTENT-NNN record naming it, this blocks — a bare
accepted-test change is a regression. With no baseline marker yet the gate
skips rather than false-blocking.
freya verify-intent --project . --format json
3 · Build the graph, run the affected hard-block
Refresh behavior.json, then run the Direction-A regression check: re-run
only the accepted behaviors whose exercised code this change touched — not the
suite. A real test-failed blocks until it is classified as a regression, an
intended change, or an infrastructure failure.
freya behavior-graph --build --project . && \
freya behavior-graph --check --base "$BASE" --project .
4 · Validate-on-hit advisory
Read-only. Surfaces the proposed/confirmed behaviors this change
touched, so you can confirm them in context, plus any touched code no behavior covers.
Bounded to the affected subset, fully skippable, and it never changes the exit
code.
freya behavior-graph --surface --base "$BASE" --project .
5 · Principle checkpoint (G2) resolve to proceed
Judge the diff against each principle in the constitution, triage against prior resolutions, and resolve every finding — fix, refute or amend. A procedural gate, not a script exit, but wrap-up must not complete while one is open. An empty constitution skips the step.
freya principles list · freya principles prior/resolve …
6 · Contradiction check (G3) resolve to proceed
For each spec and ADR changed this cycle, judge intent against intent: principles, same-category peer specs, and all accepted ADRs. Authority runs principle > ADR > spec, so the resolution default falls out of the ladder. Resolve each escalated finding before completing.
freya contradictions context --spec <ID> · freya contradictions adr-context --adr <ID>
7 · Declarative-drift check (P4b) resolve to proceed
For each declared decision whose related_code intersects the blast radius,
judge whether the changed code contradicts it. Blast-radius scoped and code-anchored —
deliberately not always-global. Degrades to changed-files-only when code-graph is absent,
and reports that it did.
freya drift context --base "$BASE" --project .
When a behavior meets a security finding
This is the cross-skill payoff that justifies the apparatus. Once a behavior is
accepted, it is the strongest "this is intentional" evidence a security
scan can cite — stronger than a spec, because somebody has said not just that the
behaviour is on purpose but that a named test pins it.
Concretely: a scan flags "this endpoint doesn't verify the user exists" as SEC-001.
BEH-003 is accepted, its locator names a test file that exists, and an
observed run of that test exercised the flagged file — so the finding is downgraded to
intentional with behavior_ref: BEH-003 —
annotated, never deleted, and dropped from
the outstanding count while staying fully visible in the report. SEC-002, which no accepted
behavior covers, stays open. And a merely confirmed behavior at most adds an
advisory note; the finding stays open.
behavior_ref
Whether a test was actually run depends on one flag. Plain
--covering runs nothing. It re-derives the behavior's state and locator from
knowledge-base/specs/ and reads its exercised paths from the committed
knowledge-base/.graph/behavior.json, and a row only comes back when the state
is accepted, a locator resolves to a file inside the project, and the
exercised path carries source: observed — a real run with coverage. An edge
marked static, inferred from the import graph with no test involved at all,
licenses nothing; before 2026-08-24 it licensed a downgrade exactly as a passing test did,
and that was the widest hole in this mechanism. The symbols the run touched come back with
the row, so the judgement is not made on a file anchor alone.
--covering --verify re-runs the linked test through
freya-behavior-runner, in one batched invocation, and a row that fails
verification is evidence against the behavior. It is off by default because other
callers use this query in a loop, and on for the security scan — the one caller whose
answer can stop a finding counting.
This page argued the opposite until 2026-08-24, and the way it was wrong is
worth keeping. It said the gap could not be closed, because the only evidence not
supplied by the repository under audit would be running that repository's test suite —
"a security tool executing hostile code". That is an argument against a capability this
toolkit ships as a feature: freya is a tool a developer points at a repository they are
working in, having already installed its dependencies and run its suite, and
freya-behavior-runner exists to run those tests. Without --verify
the answer is still a label rather than a verification — observed means a test
passed once, on somebody's machine, at the commit freshness names — and the
evidence string the query returns says exactly that, which the scan copies
into the report verbatim.
A finding marked intentional can be backed by either of two references, and they are not
equivalent. spec_ref is a prose claim — somebody wrote down that
this is on purpose. behavior_ref is a claim with a named test, a
resolving locator and an observed run behind it — narrower, harder to write by
accident, and checked by
deterministic code rather than by an agent's judgement. Where both explain a finding, the
behavior reference wins. Neither is a verification unless --verify ran.
Why only an accepted behavior may downgrade a finding, why a downgrade
annotates rather than deletes, and — in the two dated corrections at the top of ADR-012,
the second of which retracts part of the first — what the record got wrong about the word
"test-backed", and why the argument that it could never be made right was itself wrong.