43e3d37b55
The machinery has been real for a while and the content was 19 crates.io
records. Now it is the whole malicious-packages feed:
npm 220,324
PyPI 11,702
RubyGems 3,512
Go 19
crates.io 19
Packagist 1
────────────────────
235,577 indicators, 45 MB across six signed packs
The striking number is npm: 220,323 of its 227,149 OSV records are
malicious packages rather than vulnerability advisories. NINETY-SEVEN
PERCENT. That is the whole thesis in one statistic — the dominant
security fact about the npm ecosystem is not that libraries have bugs,
it is that the registry is full of things that exist only to be malware.
It is also why the malicious-versus-vulnerability classifier had to come
first: without it this pack would have been 227,149 indicators, and
7,000 of them would have been ordinary libraries.
Verified end to end from a genuinely empty definitions directory:
before: defs: no verified packs were found
update: 6 pack(s) installed, 0 already current
loaded 235577 indicators from 6 pack(s) [2026.08.21]
elapsed: 1.4 seconds for 45 MB, verified and loaded
RSS: 214 MB (unit cap is 1G)
And it detects. A lockfile with 500 ordinary packages and three real
malicious ones — ineldua, dian-kue20-riris, restart-rocket-jasmine-jwt,
picked at random from the feed rather than chosen to work — produced
three criticals and no false positives, in 11 milliseconds. The cuckoo
filter is doing exactly what it was built for: 500 clean lookups never
touch the map.
tools/ingest-osv.py does the coarse filter so build-pack does not walk a
quarter of a million files it will discard. It mirrors the Rust
classifier deliberately, and the Rust side still applies its own — a
record slipping through the Python changes nothing.
`hound status` now reports definitions, and reports them when there are
none, with the reason. Silence would let somebody believe they were
protected by a feed they never received.
Known and not yet solved: every agent downloads 43 MB of npm pack on
first update, and the whole pack again whenever it changes. Incremental
updates and a CDN in front of defs.houndav.com are both wanted. The
subdomain split exists precisely so the second one is a DNS change.
362 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
6.7 MiB
6.7 MiB