Phase 3's foundation. Three jobs that share a data model:
osv parse the ossf/malicious-packages feed (Apache-2.0, ~226k
records, daily) into indicators
index answer "is this package known bad?" fast enough to ask it
thousands of times per project sweep
pack sign a definitions pack, and verify one before loading it
Validated against the real feed rather than fixtures: the whole
crates.io OSV export, 2,749 records, parsed with zero failures — 3,885
indicators across ten ecosystems, 19 of them MAL-. `rustdecimal` (the
real typosquat of rust_decimal) resolves in crates.io and stays clean
in npm and PyPI, which is the ecosystem isolation working.
Notes on the three:
* A malicious-package record is not a vulnerability record. It almost
always carries introduced:"0" with no fix, meaning EVERY version is
malicious — the package exists only to be malware, so there is no safe
version to upgrade to. Conflating that with a version-bounded
vulnerability either misses real hits or condemns safe versions of
legitimate packages, so the two are modelled separately.
* The index is a cuckoo filter in front of a map. Cuckoo rather than
bloom specifically because a definitions feed needs DELETION: OSV
withdraws records — it once withdrew 157 malware reports after a
false-positive incident — and a filter you cannot remove from means a
withdrawn record costs a probe forever or forces a rebuild. 226k
indicators fit in under 4 MB; the crates.io set is 8 KB.
The property that must never break is no false negatives, and it has
its own test. A false positive costs a hash lookup; a false negative
is malware reported as clean. That is also why a fingerprint hashing
to zero is nudged to one — zero marks an empty slot, so without the
nudge one key in 65,536 would silently vanish.
* Signing is not about entitlement; the subscription gates the server.
It answers "is this pack really from us?", because someone who can
substitute a definitions file can add an entry for /usr/bin/sudo and
have Hound quarantine it on every machine that updates — a supply
chain attack delivered through the security product. Verification
happens on the raw bytes BEFORE anything parses them, so a hostile
pack never reaches the parser at all.
256 tests pass across the workspace.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
