Two changes to how definitions are distributed, sharing one mechanism.
Incremental updates. The old nightly rebuilt the full per-ecosystem pack
every day, so its hash changed and every client re-downloaded 44 MB of npm
daily. Now the server publishes an immutable baseline plus small daily delta
packs (crates/hound-defs/examples/build-delta.rs); the client — which
already fetches only packs whose sha256 it lacks — pulls the baseline once
and then kilobytes a day. When deltas pile up the builder folds them into a
fresh baseline and drops the old files from the index; the client prunes
whatever the index stops listing, so both the server dir and every client's
defs dir stay bounded. Verified end to end: day-2 fetched only the delta
(baseline untouched), day-3 rebaseline pruned the superseded packs.
Free community tier. Free now gets a recent subset of the public OSV feed
(build-community.rs, ~5,000 newest indicators) so a Free install detects
current threats out of the box — not just the heuristics. Pro is the full
235k corpus, daily/near-real-time freshness, and the curated threat pack.
The client always fetches the community channel and gates the full feed +
threat pack on the licence; a lapse prunes both back to exactly what a fresh
Free install has — community pack + built-in rules — while leaving any custom
.yar the user placed themselves untouched. reload_rules() lets a new threat
pack go live without a daemon restart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes Phase 3's delivery half. `hound update` now asks the definitions
host what exists, downloads what this machine lacks, verifies it, and
installs it. Verified end to end against the live host over TLS:
starting from an empty directory:
defs: no verified packs were found
hound update:
definitions: 1 pack(s) installed, 0 already current
crates-io-2026.08.21.pack — 19 indicators, version 2026.08.21
loaded 19 indicators from 1 pack(s) [2026.08.21]
and immediately afterwards a lockfile naming rustdecimal is flagged,
while tokio beside it is not.
running it again:
definitions: definitions are up to date (1 pack(s))
a pack with one byte flipped in its signed payload, advertised in the
index with a correct hash and a version of 9999.1.1:
definitions: 0 pack(s) installed, 1 up to date, 1 REJECTED
tampered-test.pack: the definitions pack is not signed by Hound and
was discarded
Three refusals are the design:
* THE INDEX IS A HINT, NEVER AN AUTHORITY. It says which packs exist and
what they hash to, and both are unverified — anyone who can serve the
index can lie about either. Only the Ed25519 signature decides whether
a pack is real. A tampered index can waste bandwidth and nothing else,
which is exactly what the test above demonstrates: correct hash,
higher version, still refused.
* NOTHING UNVERIFIED REACHES THE DEFINITIONS DIRECTORY. Downloaded to a
temp file, verified there, then moved with a rename inside the same
directory so it is atomic. The daemon cannot observe a half-written
pack, and a crash mid-download leaves a stray temp file rather than a
loadable one.
* A FILENAME FROM A REMOTE INDEX IS UNTRUSTED INPUT. The updater runs as
root, so an entry of ../../../etc/cron.d/evil.pack would be remote code
execution. Only a plain basename ending in .pack, with no separators,
no dot-dot and no leading dot, is accepted. Tested against eight
hostile shapes.
Sizes and timeouts are bounded — a pack is a list of package names, so a
server offering a hundred gigabytes is broken or hostile and the
difference does not matter to a full disk. The read is bounded
independently of Content-Length, which is also just something the server
said.
Definitions are fetched BEFORE rules are reloaded, and a failure to fetch
does not stop the reload. No network, a mirror down, a pack that will not
verify — none of those are a reason to skip the half that still works.
HOUNDD_DEFS_URL points the client at a mirror, which matters for the
air-gapped deployments that are a real part of the Fleet story.
build-pack now writes index.json beside the pack, so publishing is one
command.
358 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stage 3 passed on throughput — 27,339 events, zero rescues, Caddy
unmoved — and then the soak found what the load test could not.
Two more false positives, both the same shape as the ones before:
* EICAR matched an 8.5 MB rustc incremental-compilation cache, because
the test source being compiled contains the literal. Hound moved it to
quarantine mid-build and rustc panicked. The standard defines the
EICAR file as exactly that 68-byte string, optionally padded to 128,
so the rule now says filesize <= 128.
* The webshell rule matched a 4.3 MB AI session transcript, because the
conversation had been discussing webshells and therefore contained
"<?php", the eval pattern and "$_POST". The transcript was moved to
the vault and its history lost. A webshell is a PHP file: small, and
opening with a PHP tag. Now filesize < 1MB and $php in (0..4096).
The interesting part is why the second one happened at all. After the
first, I added a test asserting that a large file containing rule
strings is not a threat — and hand-listed the strings. I listed the
miner's and the rootkit's and forgot "<?php". The test passed and the
transcript was quarantined anyway.
So the test now extracts every string literal from the rule pack itself
and builds the haystack from those. A rule added tomorrow is covered
without anybody remembering to cover it. It also asserts the extractor
actually found the strings, because a parser that silently returns
nothing would make the whole thing vacuous.
Both fixes have a paired test that the detection still works: a real
68-byte EICAR file is caught, padded to 128 it is caught, and a real
webshell is caught.
Worth recording, because it is not a bug: six houndd tests failed while
the gate was armed. Hound quarantined the EICAR fixtures the test suite
had just written — correct behaviour, colliding with a suite that
creates real malware samples. Running the antivirus's own tests on a
gated machine needs thought; the tests are not wrong and neither is the
gate.
The definitions chain now works end to end: pack built from OSV, signed
with the release key, published to /srv/houndav/defs, installed, and
verified on load against the public half compiled into the agent —
"defs: 19 indicators from 1 pack(s) [2026.08.21]".
The public key is in the source on purpose. The agent is open source and
anybody should be able to check that the definitions they received are
the ones we published.
300 tests pass. Gate is off pending these fixes being soaked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lockfile parsing for npm (all three lockfile versions), yarn, cargo,
poetry, requirements.txt, go.sum, Gemfile.lock and composer.lock, wired
through the indicator index so a sweep checks real dependencies against
real definitions. Signed packs load in the daemon; the sweep gets the
index; `hound supply-chain` cites the OSV record it matched.
A lockfile is the right thing to read: it names every transitive
dependency at an exact version in one small file, and it lists what WILL
be installed rather than what already is — which matters when the
payload runs during installation.
Every parser is hand-written rather than pulling in a TOML and a YAML
crate. Two fields from each format, and a scanner parsing hostile input
should have as little parsing surface as it can.
The important part of this commit is a false positive it fixes.
Building a pack from the whole crates.io OSV export and sweeping a
project produced TWO criticals: rustdecimal, correctly, and **tokio
1.38.0**, which is not malware and never has been. The export is 1,524
GHSA and 1,206 RUSTSEC vulnerability advisories against 19 malicious-
package records, and the parser treated all of them as malware.
GHSA-2grh-hm3w-w7hv describes a tokio race condition fixed in 1.8.1;
Hound reported a version released years later as malicious.
Two independent bugs, either of which alone is fatal:
* Vulnerability advisories were ingested at all. A malicious package
should not exist; a vulnerable one is a legitimate library with a bug
and most of its versions are fine. Records must now PROVE they are
malicious-package reports — a MAL- id, the malicious-packages-origins
marker, or GHSA's "Malicious code in" wording — and anything
unrecognised is dropped.
* Unrecognised version ranges fell back to "all versions", which is the
opposite of safe. That is what turned a range of 1.8.0-to-1.8.1 into
a verdict on every tokio ever published.
Rebuilt against the same input, the pack now holds 19 indicators rather
than 3,614, rustdecimal is still caught and cites MAL-2022-1 rather than
a GHSA advisory, and tokio and serde are clean. The real tokio advisory
is now a regression fixture, because anything that flags tokio is a
product nobody trusts twice.
Also: definitions loading fails CLOSED on authenticity and OPEN on
everything else. No trusted key means no definitions and a message
saying so, because an operator who believes they are protected and is
not is worse off than one who knows. A pack that fails verification is
skipped and the rest still load. No packs at all is a working daemon —
install scripts, prompt injection, pickles and MCP audits need no feed.
There is deliberately no placeholder signing key compiled in. A fake key
that looks real is how a development shortcut becomes a shipped
vulnerability; an empty trust store is noisy in the way that gets fixed
before release. HOUNDD_DEFS_KEY supplies one for development.
294 tests pass across the workspace.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 3's foundation. Three jobs that share a data model:
osv parse the ossf/malicious-packages feed (Apache-2.0, ~226k
records, daily) into indicators
index answer "is this package known bad?" fast enough to ask it
thousands of times per project sweep
pack sign a definitions pack, and verify one before loading it
Validated against the real feed rather than fixtures: the whole
crates.io OSV export, 2,749 records, parsed with zero failures — 3,885
indicators across ten ecosystems, 19 of them MAL-. `rustdecimal` (the
real typosquat of rust_decimal) resolves in crates.io and stays clean
in npm and PyPI, which is the ecosystem isolation working.
Notes on the three:
* A malicious-package record is not a vulnerability record. It almost
always carries introduced:"0" with no fix, meaning EVERY version is
malicious — the package exists only to be malware, so there is no safe
version to upgrade to. Conflating that with a version-bounded
vulnerability either misses real hits or condemns safe versions of
legitimate packages, so the two are modelled separately.
* The index is a cuckoo filter in front of a map. Cuckoo rather than
bloom specifically because a definitions feed needs DELETION: OSV
withdraws records — it once withdrew 157 malware reports after a
false-positive incident — and a filter you cannot remove from means a
withdrawn record costs a probe forever or forces a rebuild. 226k
indicators fit in under 4 MB; the crates.io set is 8 KB.
The property that must never break is no false negatives, and it has
its own test. A false positive costs a hash lookup; a false negative
is malware reported as clean. That is also why a fingerprint hashing
to zero is nudged to one — zero marks an empty slot, so without the
nudge one key in 65,536 would silently vanish.
* Signing is not about entitlement; the subscription gates the server.
It answers "is this pack really from us?", because someone who can
substitute a definitions file can add an entry for /usr/bin/sudo and
have Hound quarantine it on every machine that updates — a supply
chain attack delivered through the security product. Verification
happens on the raw bytes BEFORE anything parses them, so a hostile
pack never reaches the parser at all.
256 tests pass across the workspace.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dev