Stage 3 passed on throughput — 27,339 events, zero rescues, Caddy unmoved — and then the soak found what the load test could not. Two more false positives, both the same shape as the ones before: * EICAR matched an 8.5 MB rustc incremental-compilation cache, because the test source being compiled contains the literal. Hound moved it to quarantine mid-build and rustc panicked. The standard defines the EICAR file as exactly that 68-byte string, optionally padded to 128, so the rule now says filesize <= 128. * The webshell rule matched a 4.3 MB AI session transcript, because the conversation had been discussing webshells and therefore contained "<?php", the eval pattern and "$_POST". The transcript was moved to the vault and its history lost. A webshell is a PHP file: small, and opening with a PHP tag. Now filesize < 1MB and $php in (0..4096). The interesting part is why the second one happened at all. After the first, I added a test asserting that a large file containing rule strings is not a threat — and hand-listed the strings. I listed the miner's and the rootkit's and forgot "<?php". The test passed and the transcript was quarantined anyway. So the test now extracts every string literal from the rule pack itself and builds the haystack from those. A rule added tomorrow is covered without anybody remembering to cover it. It also asserts the extractor actually found the strings, because a parser that silently returns nothing would make the whole thing vacuous. Both fixes have a paired test that the detection still works: a real 68-byte EICAR file is caught, padded to 128 it is caught, and a real webshell is caught. Worth recording, because it is not a bug: six houndd tests failed while the gate was armed. Hound quarantined the EICAR fixtures the test suite had just written — correct behaviour, colliding with a suite that creates real malware samples. Running the antivirus's own tests on a gated machine needs thought; the tests are not wrong and neither is the gate. The definitions chain now works end to end: pack built from OSV, signed with the release key, published to /srv/houndav/defs, installed, and verified on load against the public half compiled into the agent — "defs: 19 indicators from 1 pack(s) [2026.08.21]". The public key is in the source on purpose. The agent is open source and anybody should be able to check that the definitions they received are the ones we published. 300 tests pass. Gate is off pending these fixes being soaked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
106 lines
3.9 KiB
Rust
106 lines
3.9 KiB
Rust
//! Build and sign a definitions pack from a directory of OSV records.
|
|
//!
|
|
//! build-pack <osv-dir> <out.pack> <key-file> [version]
|
|
//!
|
|
//! If the key file does not exist a new signing key is generated into it
|
|
//! at mode 0600 and its public half is printed. That public key is what
|
|
//! goes into the agent's trust store; the private half never leaves the
|
|
//! build machine and is never committed.
|
|
//!
|
|
//! Key material comes straight from /dev/urandom rather than through a
|
|
//! random-number crate: it is thirty-two bytes from the kernel CSPRNG,
|
|
//! and fewer moving parts between the entropy source and the key file is
|
|
//! the right trade for something this consequential.
|
|
|
|
use ed25519_dalek::SigningKey;
|
|
use hound_defs::{osv, pack, Pack};
|
|
use std::io::Read;
|
|
|
|
fn load_or_create_key(path: &str) -> SigningKey {
|
|
if let Ok(bytes) = std::fs::read(path) {
|
|
if bytes.len() == 32 {
|
|
let mut seed = [0u8; 32];
|
|
seed.copy_from_slice(&bytes);
|
|
return SigningKey::from_bytes(&seed);
|
|
}
|
|
eprintln!("{path} is not a 32-byte key; refusing to overwrite it");
|
|
std::process::exit(2);
|
|
}
|
|
|
|
let mut seed = [0u8; 32];
|
|
std::fs::File::open("/dev/urandom")
|
|
.expect("opening /dev/urandom")
|
|
.read_exact(&mut seed)
|
|
.expect("reading key material");
|
|
|
|
std::fs::write(path, seed).expect("writing the key");
|
|
#[cfg(unix)]
|
|
{
|
|
use std::os::unix::fs::PermissionsExt;
|
|
let _ = std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600));
|
|
}
|
|
let key = SigningKey::from_bytes(&seed);
|
|
let hex: String = key
|
|
.verifying_key()
|
|
.to_bytes()
|
|
.iter()
|
|
.map(|b| format!("{b:02x}"))
|
|
.collect();
|
|
eprintln!("generated a new signing key in {path} (mode 0600)");
|
|
eprintln!("public key: {hex}");
|
|
eprintln!(" put that in the agent's trust store, or export it for development:");
|
|
eprintln!(" export HOUNDD_DEFS_KEY={hex}");
|
|
key
|
|
}
|
|
|
|
fn main() {
|
|
let args: Vec<String> = std::env::args().skip(1).collect();
|
|
if args.len() < 3 {
|
|
eprintln!("usage: build-pack <osv-dir> <out.pack> <key-file> [version]");
|
|
std::process::exit(2);
|
|
}
|
|
let (dir, out, key_path) = (&args[0], &args[1], &args[2]);
|
|
let version = args
|
|
.get(3)
|
|
.cloned()
|
|
.unwrap_or_else(|| "0.0.0-dev".to_string());
|
|
|
|
let key = load_or_create_key(key_path);
|
|
|
|
let mut indicators = Vec::new();
|
|
let mut files = 0usize;
|
|
for entry in std::fs::read_dir(dir).expect("readable OSV directory").flatten() {
|
|
let path = entry.path();
|
|
if path.extension().is_none_or(|e| e != "json") {
|
|
continue;
|
|
}
|
|
files += 1;
|
|
if let Ok(text) = std::fs::read_to_string(&path) {
|
|
indicators.extend(osv::parse_record(&text));
|
|
}
|
|
}
|
|
|
|
// Sorted so the same input always produces the same bytes. A pack's
|
|
// hash is its identity; directory order is not stable across machines
|
|
// and would make two identical builds disagree.
|
|
indicators.sort_by(|a, b| (&a.ecosystem, &a.name, &a.id).cmp(&(&b.ecosystem, &b.name, &b.id)));
|
|
indicators.dedup();
|
|
|
|
let p = Pack {
|
|
version: version.clone(),
|
|
// Passed in rather than read from the clock, for the same reason.
|
|
created: args.get(4).cloned().unwrap_or_else(|| "1970-01-01T00:00:00Z".into()),
|
|
sources: vec!["ossf/malicious-packages (Apache-2.0)".into(), "osv.dev".into()],
|
|
indicators,
|
|
};
|
|
|
|
let key_id = std::env::var("HOUND_KEY_ID").unwrap_or_else(|_| "hound-2026".into());
|
|
let signed = pack::sign(&p, &key, &key_id).expect("signing the pack");
|
|
std::fs::write(out, serde_json::to_string(&signed).expect("encoding")).expect("writing");
|
|
|
|
println!("read {files} OSV records");
|
|
println!("packed {} indicators", p.indicators.len());
|
|
println!("version {version}");
|
|
println!("sha256 {}", p.sha256().expect("hashing"));
|
|
println!("wrote {out}");
|
|
}
|