Security & Response

Autonomous Response

The AI never writes a command. It picks a number off a list, so the worst a hostile input can do is make it pick a different approved action.

Designed, built, and operatePythonBashsystemdCloudflare APITOTPMulti-model quorum

No field in the protocol can carry a host, a path or a command. A target is not something the validator catches, it is something the schema cannot express.
Ten gates stand between a chosen option and a state change, and any of them can stop it.
Destructive actions are human only. Tested by trying, not assumed.

The problem

Everyone builds this the same way. Let the model decide what to do about a security event, let it write the command, then inspect the command before you run it. Congratulations, you now own a filter. Filters are things people work on. Somebody will spend a weekend on yours, and they only have to be right once.

I did not want a better filter. I wanted nothing to filter. So the response protocol has seven keys, and not one of them can name a target.

The shape of the protocol

The privileged side builds the list of options for each case and publishes it. The model gets the case and the list, and hands back three things: which option, how confident, and why. No target field. No command field. No address field. There is nowhere to put one.

If a response comes back carrying a target anyway, that is not a malformed message. That is a compromised worker, or an injection that worked, and it pages as tampering, because there is no innocent way for that field to exist.

The privileged side then looks up what that option ID means in a file it wrote itself, minutes earlier, and never showed anyone. An allowlist would still mean receiving an attacker-influenced value and checking it. Here there is nothing to check, because there is nothing to tamper with.

ALLOWED_CHOICE_KEYS = {"schema", "case_id", "option_id", "option_nonce",
                       "confidence", "rationale", "worker"}

OPTION_ID_RE = re.compile(r"^O(?:[1-9]|1[0-9])$")

# a complete, valid response:
{
  "schema":       "soar-choice/v1",
  "case_id":      "auto-…-1782930902",
  "option_id":    "O3",
  "option_nonce": "… 32 hex …",
  "confidence":   0.82,
  "rationale":    "repeat auth failures from one source, no successful login",
  "worker":       "w-07"
}

The complete set of keys a response may carry, and one valid response. There is nowhere to put a hostname, an address, or a path. A target is not something the validator has to catch; it is something the schema cannot express. The worker field is filled in by the queue, not by the model. An option_id outside the range this ballot offered is not a bold choice, it is an id that does not exist, and it pages as tampering.

The gauntlet

Ten gates stand between a returned option and a state change. Every one of them can stop it:

Editing the action catalog invalidates the reversal proof for every case in flight, by design. The self-test receipt is pinned to the catalog's hash, so an edit makes the proof stale and the invariant fails rather than silently re-scoping work already in motion.

The ten gates, in order
Gate 1	Message is the shape the protocol expects
Gate 2	Authorization exists, unused, unexpired
Gate 3	One-time token matches, compared in constant time
Gate 4	The option was genuinely offered for this case
Gate 5	Action catalog unchanged since the options were built
Gate 6	Host sensitivity tier looked up live
Gate 7	Every precondition re-checked against the machine now
Gate 8	Evidence bundle still hashes to what was preserved
Gate 9	A maintenance window is open
Gate 10	End state asserted after the action, or it reverts

Ten gates between a returned option and a state change, each able to stop it. The last one runs after the action: the intended end state has to assert, or the change reverts itself.

The message is the shape the protocol expects.
The authorization exists, has not been used, and has not expired.
The one-time token matches. Compared in constant time, so a wrong guess and a near-miss take equally long.
The chosen option was genuinely one of the options offered for this case.
The action catalog is unchanged since those options were built.
The host's sensitivity tier is looked up live, not trusted from the case record.
Every precondition is re-checked against the machine as it is now.
The evidence bundle still hashes to what was preserved.
A maintenance window is open, so the monitoring expects the blip.
After the action, the intended end state is asserted, and failure reverts it automatically.

The state machine

New → preserving → preserved → offered → decided → applying or awaiting approval → verifying → active → reverted → closed, plus terminal branches for expiry, denial, refused tier, refused evidence, and failed-and-reversed.

Three rules shape all of it.

Preserve before decide, always
Evidence is perishable; approval is not. A deleted-but-running binary exists only as an open file handle. Preservation is read-only, so it is the one stage that runs on every case, everywhere, with no gate at all.
A timeout pages harder and does nothing
There is no edge from awaiting-approval to applying that is not a verified human approval. Expiry consumes the authorization, so a click arriving two hours later cannot fire a stale decision. The incident has to be re-cased against current state.
Verify the property, not the exit code
Backup agents, AV and control panels all race on the same directories. A zero from a move command proves nothing about who won. Failure to assert the intended end state auto-reverts.

Why the quorum cannot escalate

The worker pool is sorted by detection coverage, which is something I can check, instead of by trust, which is a feeling. Most of those voters are machines with no agent, no baseline, no patch coverage. If one of them gets owned, I will not find out. Fine for a job queue. Disqualifying for anything allowed to touch production.

So the arithmetic does the work, not the policy:

The quorum is an order statistic (the median-lowest rank) so a vote is a veto with an escalation cap. A single compromised voter can always pull a response down, never up.
Unmonitored workers are not denied the invasive options. They never see them. The ballot served to that class contains only options at or below the lowest rank.
Destructive actions are human-only. Three unanimous fully-monitored voters choosing the most invasive option on a disposable host still return “page a human.” Tested, not assumed.
Model diversity is enforced by giving each voting slot a disjoint slice of the model pool, selected by a stable hash. That rule exists because I went and counted instead of assuming. Most of the decisions that had cleared a two-provider bar were one model asked twice through two different keys, and some of the harshest verdicts in that set were a single voter agreeing with itself. Hashing the model rather than the provider name fixed the family axis, and the consumer now verifies the independence instead of trusting the pool to supply it.

What the first real corpus found

A month of synthetic testing passed cleanly. The first real ballots did not: a share of them carried a raw absolute path or hostname inside a field the detector fills in itself, because file-integrity rules interpolate the offending path into their own description text and the sanitiser copied that field through verbatim while pseudonymising everything around it.

Nothing had left the box (publication was off) but the more useful finding was the second one. The invariant that was supposed to catch this shared a defective pattern with the control it audited, so it had certified the corpus clean for its entire life. Both were rewritten to assert the property directly: whatever the sanitiser pseudonymised must not appear anywhere in the published document, read back from the authoritative source. It needs no hostname list, it cannot over-match, and it correctly failed on the day it was written.

The general lesson is the one I keep re-learning: a synthetic test that passes proves the test is synthetic.

Rollout is staged, and nothing arms because a stage went fine

Six stages, each gated by evidence you can paste rather than by confidence. The first gate: fourteen days, at least twenty real cases, zero invariant failures, zero tamper pages, every option set reviewed by hand, and zero identifiers in any published case throughout.

Some stages are marked never scheduled, with the reason written down, including every state-changing action on the host that holds third-party accounts in custody. That is a legal threshold, not a confidence one, and no amount of evidence moves it.

Engineering decisions

The calls I made, and what each one cost.

Host authority resolves from properties, not hostnames.

Move the key-custody service to another machine and its tier follows it. Onboard a new regulated tenant and it is top-tier on day one. Neither needs a code edit, and an unknown host resolves to the most sensitive tier, failing closed.

Some actions are forbidden permanently, and the reason is written down.

Letting the model write a patch is forbidden forever: it is the only action requiring the model to emit content rather than choose an index, and the only one whose target space is unbounded. Making a live file immutable is forbidden because it defends against an attacker who does not exist, and its failure mode is invisible, delayed, and survives reboots.

Automated quarantine on a user-writable path is gated permanently, by construction.

Not “until we are confident.” A tenant who can plant a signature-matching file and prove the system deletes files on demand has turned the SOC into their weapon.

The approval notification carries no acting URL.

A URL that acts is a capability held by everyone who can see it, including whatever read the notification off a lock screen. Approval takes four independent factors and the decisive one is checked on the privileged host, using a local time-based code, so it still works while the secrets service is sealed. That is exactly the incident class that seals it.

Click-to-deny stays a plain URL. Click-to-act never is.

An attacker who can spam denials achieves what the system already does when nobody answers: nothing.

Ship semantics, not identifiers.

Stripping identifiers costs triage quality, because the identifier is the signal. So the trusted side computes the meaning instead (file class, directory role, entropy of the name, host tier and role) and sends that. The model gets more decision-relevant signal than the raw path gave it, and no map of the fleet.

More systems

Reliability & Automation

I'm looking for Incident Commander and SRE roles.

If your team is drowning in toil, alert noise, or incidents that never quite close. That is the work I do.

Get in touch