Platforms & Products

Hacked-Site Recovery

A pipeline that tells small businesses their WordPress site is serving SEO spam. Models decide whether a site is hacked. Deterministic gates decide whether we're allowed to say so.

Designed, built, and operatePythonNext.jsPostgreSQLPlaywrightOpen-weight vision modelsClaude Code agents

Every host named on a generated notice must come from the scan evidence or a short allowlist. Anything else fails the build and nothing is published.
The process that publishes holds the deploy access. The models that judge the evidence run on sandboxed workers with no secrets.
A column computed from the captured evidence blocks every contact path, and the live API refuses a row without it.
A stop or do-not-contact reply is flagged the same round it's read, and every outreach route refuses that row from then on.

The problem

A notice in this program is a public page that tells a real business, by name, that its website has been hacked. If that's wrong, I've libeled a stranger under my own domain. If it's right but sloppy, it reads like a shakedown.

Language models are useful here. They know a partner list that mentions a casino isn't a casino link, and they write readable pages. They also invent things. So the design rule is simple: the AI decides, and deterministic gates enforce. A model can say no to a notice. It can't put a fact on a page the scanner never saw, and it never holds the credential that publishes.

From scan to notice

The scanner reads a site's homepage the way a browser does, with a delay between requests and no admin paths or vulnerability probing. It's a deterministic matcher: explainable, but with no judgment. An early version flagged a clean roofing company over its partner list.

So a scanner hit goes to a panel of three models, and all three must agree the evidence proves a compromise. The panel only sees what the scanner produced, so it can downgrade a hit but can't invent one. A split, or a panel short a member, goes to a separate verifier instead of a vote.

That verifier reproduces the evidence live. The homepage must still end on the business's own domain and still name the business, because expired domains get parked on casino pages. Injected links must still be there. Spam pages are screenshotted and shown to a vision model, and only a definite yes counts. If it can't check, it retries, then gives up.

The build gate

No person reads a generated notice before it goes up. That's deliberate, and it's why every check in the build gate fails closed.

The main check is the anti-invention one. Every hostname anywhere on the page has to trace back to the scan evidence or to a short allowlist of our own pages and the independent checkers. A model that hallucinates a casino domain fails the build instead of publishing it. A second check catches copy-paste drift, where the page for one business names or links to a different one.

Two more gates guard the screenshots and the outside links. At publish, a vision model looks at each spam-page shot and each hidden-content reveal, and any image it says shows no spam is removed before the page can claim it. A notice links to Sucuri SiteCheck or Google's Safe Browsing status only when that service itself flags the site, because a link to a page saying "no issues found" would undercut everything else on the notice. The check renders those pages on worker machines, rotating across boxes and browser engines, and one render never decides.

def decide(verdicts):
    """Two flagged and no clean = flagged. Any clean and no flagged = clean. Else unknown."""
    f, c = verdicts.count("flagged"), verdicts.count("clean")
    if f >= 2 and c == 0:
        return "flagged"
    if c >= 1 and f == 0:
        return "clean"
    return "unknown"

The rule for an outside checker, as it runs. Two renders must say flagged and none may say clean. Anything less links nothing.

Who holds the publish key

The publish runner turns an approved prospect into a live page: build, verify, deploy. It holds the deploy access. The workers that run scans, browsers and vision models are sandboxed with no secrets on purpose. Models decide whether a site gets a notice. The runner only does the how, and only for rows a decision already approved.

It also fails closed. A prospect is marked live only after the build exits clean, which means every gate passed, and after the deploy confirms the page is up and marked noindex. Any failure leaves the row where it was, with the reason.

Evidence before contact

Publishing a notice isn't the same as telling anyone about it. The page is noindex and nobody is pointed at it until a message goes out, and nothing goes out without my OK. Agents draft emails, call sheets and replies. A person sends each one from the signer's own account.

In October I tightened the rule: no email, form, call or follow-up goes out unless the notice itself shows a captured screenshot of the hack. A row with spam pages on its domain needs a shot of one of those pages. A link-injection row needs the reveal shot of its own page with the hidden links boxed. A list of URLs with no image doesn't count.

Putting the evidence rule into the database

A rule written in a plan gets skipped on a busy afternoon. This is how it became a column every contact path has to read.

SchemaAdded columns for each captured shot and why one is missing. The evidence flag is a generated column computed from those and the hack's shape, so no code path can set it by hand.
BackfillRead each live notice page, not the capture files on disk, because the rule is what a visitor sees. Every notice that failed went on hold.
Admin appEvery send, follow-up and call-list action refuses a needs-evidence row, and the API returns 409. The gate has its own tests, one of which runs the migration on a local Postgres, and the full suite passes.
Live checkMoved a needs-evidence row to contacted against the production API and got a 409. A row with evidence got the normal transition check. Neither row changed.
Contact formsThe form filler skips the row at dry run and at send, and its worker refuses a task without the evidence flag set.
QueuesThe batch, follow-up and call-sheet queries require the evidence flag. Run read-only against production, the first-touch and follow-up rules picked zero needs-evidence rows.

Nothing was sent while this was built. A row without evidence now shows "needs evidence" where the send button was.

Stops, takedowns and replies

Replies are read every five minutes. Each real reply becomes a task for an inbox agent, which sorts it and drafts an answer for me. A stop, an unsubscribe or an angry "leave me alone" is handled first, in the same round: the agent sets a do-not-contact flag, reads the row back to confirm it, and from then on every outreach route returns 409 for that business. The agent's job stops at a draft: it never sends, and it never clears a flag.

A weekly re-scan checks every published notice with both scan engines. When both say clean, the row moves to fixed and the page comes down within fifteen minutes. If a business asks for the notice to come down, it comes down that day, hacked or not. The page exists for them.

What it doesn't do yet

The publish-time vision check fails open. If no model answers, the screenshot stays, on the grounds that the in-browser reveal already gated it and an outage shouldn't strand the queue. Only a confident "no" removes an image. Reveals that box links skip vision and rely on the scanner-confirmed spam domains, since small link labels don't survive a vision model's downscale.

"Clean the day it's clean" really means the Monday after, unless someone tells us it's fixed and an on-demand re-scan runs. And a notice is not a cleanup. The agent built to do remediation stays on hold until I approve its design, so any cleanup today is done by hand.

The evidence gate also held back notices that have a good screenshot on disk the page doesn't display. Getting them back means republishing live pages, which needs my OK, so they're still on hold.

Engineering decisions

The calls I made, and what each one cost.

Unanimous panel, not majority.

With two of three, one model's confident hallucination plus another's coin flip publishes an accusation. Unanimity costs leads, and I'll pay that.

The model judges. The runner publishes. They never share a credential.

A prompt-injected or confused model on a worker can at most vote no or vote yes on evidence the scanner already produced. It can't write to the public site.

A gate that can't fire, or fires on everything, counts as a bug.

When notices moved to a shared layout, the drift check compared against an address that now matched every notice, so it passed everything and still printed "verified". Later a business name that was a common word blocked a run of good publishes. Both fixes carry the reason in the code.

Evidence in a generated column, enforced at every exit.

A plan rule depends on someone remembering. A generated column read at every exit doesn't. The cost was a large share of live notices going on hold the day it shipped.

More systems

Reliability & Automation

I'm looking for Incident Commander, SRE and AIOps roles.

If your team is drowning in toil, alert noise, or incidents that never quite close. That is the work I do.

Get in touch