Antivirus/crates/houndd/src/defs.rs
Hound 2cf9a740c3 rules: every rule needs an anchor, and the test now proves it for all of them
Stage 3 passed on throughput — 27,339 events, zero rescues, Caddy
unmoved — and then the soak found what the load test could not.

Two more false positives, both the same shape as the ones before:

* EICAR matched an 8.5 MB rustc incremental-compilation cache, because
  the test source being compiled contains the literal. Hound moved it to
  quarantine mid-build and rustc panicked. The standard defines the
  EICAR file as exactly that 68-byte string, optionally padded to 128,
  so the rule now says filesize <= 128.

* The webshell rule matched a 4.3 MB AI session transcript, because the
  conversation had been discussing webshells and therefore contained
  "<?php", the eval pattern and "$_POST". The transcript was moved to
  the vault and its history lost. A webshell is a PHP file: small, and
  opening with a PHP tag. Now filesize < 1MB and $php in (0..4096).

The interesting part is why the second one happened at all. After the
first, I added a test asserting that a large file containing rule
strings is not a threat — and hand-listed the strings. I listed the
miner's and the rootkit's and forgot "<?php". The test passed and the
transcript was quarantined anyway.

So the test now extracts every string literal from the rule pack itself
and builds the haystack from those. A rule added tomorrow is covered
without anybody remembering to cover it. It also asserts the extractor
actually found the strings, because a parser that silently returns
nothing would make the whole thing vacuous.

Both fixes have a paired test that the detection still works: a real
68-byte EICAR file is caught, padded to 128 it is caught, and a real
webshell is caught.

Worth recording, because it is not a bug: six houndd tests failed while
the gate was armed. Hound quarantined the EICAR fixtures the test suite
had just written — correct behaviour, colliding with a suite that
creates real malware samples. Running the antivirus's own tests on a
gated machine needs thought; the tests are not wrong and neither is the
gate.

The definitions chain now works end to end: pack built from OSV, signed
with the release key, published to /srv/houndav/defs, installed, and
verified on load against the public half compiled into the agent —
"defs: 19 indicators from 1 pack(s) [2026.08.21]".

The public key is in the source on purpose. The agent is open source and
anybody should be able to check that the definitions they received are
the ones we published.

300 tests pass. Gate is off pending these fixes being soaked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 08:37:49 -05:00

418 lines
14 KiB
Rust

//! Loading definition packs.
//!
//! A pack is data the scanner will act on, so the only question that
//! matters before loading one is whether it is really ours. Everything
//! here fails **closed** on that question and **open** on everything else:
//!
//! * No trusted key configured → load nothing, say so plainly. Running
//! with unverified definitions would be worse than running with none,
//! because the operator would believe they were protected.
//! * A pack that fails verification → skipped, logged, and the others
//! still load. One bad file must not cost you the whole feed.
//! * No packs at all → the daemon runs fine. Install scripts, prompt
//! injection, pickles and MCP audits need no feed, and refusing to start
//! would leave the machine with nothing.
//!
//! The trust store holds the PUBLIC half of the release signing key, in
//! the source, where everyone can read it — that is the point. There was
//! never a placeholder here: a fake key that looks real is how a
//! development shortcut becomes a shipped vulnerability, so until the
//! real one existed the store was empty and said so loudly.
use anyhow::{Context, Result};
use ed25519_dalek::VerifyingKey;
use hound_defs::{pack, Index, SignedPack};
use std::path::PathBuf;
use std::sync::{Arc, RwLock};
use std::time::SystemTime;
/// Keys whose packs this build will load.
///
/// The PUBLIC half of Hound's definitions signing key. It belongs in the
/// source: the agent is open source, everyone can read it, and that is
/// the point — anybody can verify that the packs they receive are the
/// ones we published. The private half never leaves the build machine
/// and is not in this repository.
///
/// Rotation: add the new key beside the old one, ship that build, then
/// start signing with the new key and remove the old one a release later.
/// Never swap in one step, or every agent that has not updated yet stops
/// accepting definitions.
const TRUSTED_KEYS: &[(&str, [u8; 32])] = &[(
"hound-2026",
[
0x12, 0xba, 0x51, 0x9f, 0x13, 0xe6, 0xe8, 0x37, 0x00, 0xef, 0x3e, 0xfb, 0x07, 0xe9,
0x32, 0x85, 0xc4, 0x88, 0x79, 0x30, 0x26, 0x04, 0xa3, 0x20, 0xa0, 0x2d, 0xc3, 0x64,
0x29, 0x90, 0xb4, 0x51,
],
)];
/// Where signed packs live.
pub fn defs_dir() -> Option<PathBuf> {
if let Some(dir) = std::env::var_os("HOUNDD_DEFS_DIR") {
let p = PathBuf::from(dir);
return p.is_dir().then_some(p);
}
let system = PathBuf::from("/var/lib/hound/defs");
if system.is_dir() {
return Some(system);
}
let home = std::env::var_os("HOME")?;
let user = PathBuf::from(home).join(".local/share/hound/defs");
user.is_dir().then_some(user)
}
/// Assemble the trust store: compiled-in keys, plus a development key
/// from the environment when one is set.
pub fn trusted_keys() -> Vec<(String, VerifyingKey)> {
let mut keys: Vec<(String, VerifyingKey)> = TRUSTED_KEYS
.iter()
.filter_map(|(id, bytes)| {
VerifyingKey::from_bytes(bytes)
.ok()
.map(|k| ((*id).to_string(), k))
})
.collect();
if let Ok(hex) = std::env::var("HOUNDD_DEFS_KEY") {
match parse_hex_key(&hex) {
Some(k) => {
let id = std::env::var("HOUNDD_DEFS_KEY_ID").unwrap_or_else(|_| "dev".into());
eprintln!("defs: trusting development key {id} from the environment");
keys.push((id, k));
}
None => eprintln!("defs: HOUNDD_DEFS_KEY is not a 32-byte hex public key — ignored"),
}
}
keys
}
fn parse_hex_key(hex: &str) -> Option<VerifyingKey> {
let hex = hex.trim();
if hex.len() != 64 {
return None;
}
let mut bytes = [0u8; 32];
for (i, b) in bytes.iter_mut().enumerate() {
*b = u8::from_str_radix(hex.get(i * 2..i * 2 + 2)?, 16).ok()?;
}
VerifyingKey::from_bytes(&bytes).ok()
}
/// A loaded, verified set of definitions.
pub struct Loaded {
pub index: Index,
/// Highest pack version loaded, for `hound status`.
pub version: String,
pub indicators: usize,
pub packs: Vec<String>,
pub loaded_at: SystemTime,
/// Why nothing loaded, when nothing did.
pub detail: String,
}
impl Loaded {
fn empty(detail: impl Into<String>) -> Self {
Self {
index: Index::new(),
version: String::new(),
indicators: 0,
packs: Vec::new(),
loaded_at: SystemTime::now(),
detail: detail.into(),
}
}
}
/// Hot-swappable definitions, mirroring how rules are held.
#[derive(Clone)]
pub struct DefsStore {
inner: Arc<RwLock<Arc<Loaded>>>,
}
impl DefsStore {
/// Load whatever is on disk. Never fails: an empty store is a working
/// daemon with fewer detections, and that beats no daemon.
pub fn load() -> Self {
Self {
inner: Arc::new(RwLock::new(Arc::new(load_all()))),
}
}
pub fn current(&self) -> Arc<Loaded> {
Arc::clone(&self.inner.read().expect("defs store poisoned"))
}
pub fn reload(&self) -> Arc<Loaded> {
let fresh = Arc::new(load_all());
*self.inner.write().expect("defs store poisoned") = Arc::clone(&fresh);
fresh
}
}
fn load_all() -> Loaded {
let keys = trusted_keys();
if keys.is_empty() {
return Loaded::empty(
"no signing key is trusted by this build, so no definitions were loaded",
);
}
let Some(dir) = defs_dir() else {
return Loaded::empty("no definitions directory");
};
let trusted: Vec<(&str, VerifyingKey)> =
keys.iter().map(|(id, k)| (id.as_str(), *k)).collect();
let mut indicators = Vec::new();
let mut packs = Vec::new();
let mut version = String::new();
let mut files: Vec<PathBuf> = std::fs::read_dir(&dir)
.into_iter()
.flatten()
.flatten()
.map(|e| e.path())
.filter(|p| p.extension().is_some_and(|e| e == "pack"))
.collect();
files.sort();
for path in files {
let name = path
.file_name()
.map(|n| n.to_string_lossy().into_owned())
.unwrap_or_default();
match load_one(&path, &trusted) {
Ok(p) => {
if p.version > version {
version = p.version.clone();
}
packs.push(format!("{name} ({} indicators)", p.indicators.len()));
indicators.extend(p.indicators);
}
// A bad pack is skipped rather than fatal: losing one file
// must not cost the whole feed.
Err(e) => eprintln!("defs: skipping {name}: {e}"),
}
}
if indicators.is_empty() {
return Loaded::empty("no verified packs were found");
}
let count = indicators.len();
Loaded {
index: Index::build(indicators),
version,
indicators: count,
packs,
loaded_at: SystemTime::now(),
detail: String::new(),
}
}
fn load_one(path: &std::path::Path, trusted: &[(&str, VerifyingKey)]) -> Result<hound_defs::Pack> {
let text = std::fs::read_to_string(path).context("reading the pack")?;
let signed: SignedPack = serde_json::from_str(&text).context("the pack is not valid JSON")?;
// Verification happens inside, on the raw payload bytes, before any
// of the content is parsed.
pack::verify(&signed, trusted).map_err(|e| anyhow::anyhow!("{e}"))
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
use hound_defs::{Indicator, Pack, Versions};
fn a_pack(name: &str) -> Pack {
Pack {
version: "2026.08.21".into(),
created: "2026-08-21T12:00:00Z".into(),
sources: vec!["test".into()],
indicators: vec![Indicator {
ecosystem: "cratesio".into(),
name: name.into(),
versions: Versions::All,
id: "MAL-2022-1".into(),
summary: "malicious".into(),
}],
}
}
fn write_pack(dir: &std::path::Path, file: &str, p: &Pack, key: &SigningKey, id: &str) {
let signed = pack::sign(p, key, id).unwrap();
std::fs::write(dir.join(file), serde_json::to_string(&signed).unwrap()).unwrap();
}
fn tmp(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("hound-defs-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
fn hex_of(k: &SigningKey) -> String {
k.verifying_key()
.to_bytes()
.iter()
.map(|b| format!("{b:02x}"))
.collect()
}
#[test]
fn the_trust_store_holds_exactly_the_release_key() {
// A second key appearing here without a rotation plan is how a
// development shortcut becomes a shipped vulnerability.
assert_eq!(TRUSTED_KEYS.len(), 1);
assert_eq!(TRUSTED_KEYS[0].0, "hound-2026");
assert!(
VerifyingKey::from_bytes(&TRUSTED_KEYS[0].1).is_ok(),
"the compiled-in key must be a valid ed25519 public key"
);
}
#[test]
fn the_release_key_verifies_a_pack_signed_by_it() {
// Catches a fat-fingered byte in the constant, which would
// silently stop every agent from accepting definitions.
let published = std::path::Path::new("/srv/houndav/defs");
if !published.is_dir() {
return; // only meaningful on the build host
}
let trusted: Vec<(&str, VerifyingKey)> = TRUSTED_KEYS
.iter()
.filter_map(|(id, b)| VerifyingKey::from_bytes(b).ok().map(|k| (*id, k)))
.collect();
let mut checked = 0;
for e in std::fs::read_dir(published).into_iter().flatten().flatten() {
let p = e.path();
if p.extension().is_none_or(|x| x != "pack") {
continue;
}
checked += 1;
let text = std::fs::read_to_string(&p).unwrap();
let signed: SignedPack = serde_json::from_str(&text).unwrap();
assert!(
pack::verify(&signed, &trusted).is_ok(),
"the compiled key does not verify {p:?}"
);
}
let _ = checked;
}
#[test]
fn an_empty_definitions_directory_says_so_rather_than_staying_silent() {
// Silence would let an operator believe they were protected when
// nothing had loaded.
let dir = tmp("empty");
let _guard = crate::test_util::locked();
std::env::remove_var("HOUNDD_DEFS_KEY");
std::env::set_var("HOUNDD_DEFS_DIR", &dir);
let loaded = load_all();
std::env::remove_var("HOUNDD_DEFS_DIR");
assert_eq!(loaded.indicators, 0);
assert!(
!loaded.detail.is_empty(),
"an empty load must explain itself"
);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_signed_pack_loads_and_becomes_searchable() {
let dir = tmp("load");
let key = SigningKey::from_bytes(&[3u8; 32]);
write_pack(&dir, "linux.pack", &a_pack("rustdecimal"), &key, "dev");
let _guard = crate::test_util::locked();
std::env::set_var("HOUNDD_DEFS_KEY", hex_of(&key));
std::env::set_var("HOUNDD_DEFS_DIR", &dir);
let loaded = load_all();
std::env::remove_var("HOUNDD_DEFS_KEY");
std::env::remove_var("HOUNDD_DEFS_DIR");
assert_eq!(loaded.indicators, 1);
assert_eq!(loaded.version, "2026.08.21");
assert!(loaded.index.lookup("cratesio", "rustdecimal", "1.0.0").is_some());
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_pack_signed_by_the_wrong_key_is_refused() {
let dir = tmp("wrongkey");
let real = SigningKey::from_bytes(&[3u8; 32]);
let attacker = SigningKey::from_bytes(&[9u8; 32]);
write_pack(&dir, "evil.pack", &a_pack("sudo"), &attacker, "dev");
let _guard = crate::test_util::locked();
std::env::set_var("HOUNDD_DEFS_KEY", hex_of(&real));
std::env::set_var("HOUNDD_DEFS_DIR", &dir);
let loaded = load_all();
std::env::remove_var("HOUNDD_DEFS_KEY");
std::env::remove_var("HOUNDD_DEFS_DIR");
assert_eq!(
loaded.indicators, 0,
"a pack that could make Hound quarantine sudo must never load"
);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn one_bad_pack_does_not_cost_the_others() {
let dir = tmp("mixed");
let key = SigningKey::from_bytes(&[3u8; 32]);
let attacker = SigningKey::from_bytes(&[9u8; 32]);
write_pack(&dir, "a-good.pack", &a_pack("rustdecimal"), &key, "dev");
write_pack(&dir, "b-bad.pack", &a_pack("sudo"), &attacker, "dev");
std::fs::write(dir.join("c-garbage.pack"), b"not json at all").unwrap();
let _guard = crate::test_util::locked();
std::env::set_var("HOUNDD_DEFS_KEY", hex_of(&key));
std::env::set_var("HOUNDD_DEFS_DIR", &dir);
let loaded = load_all();
std::env::remove_var("HOUNDD_DEFS_KEY");
std::env::remove_var("HOUNDD_DEFS_DIR");
assert_eq!(loaded.indicators, 1);
assert!(loaded.index.lookup("cratesio", "rustdecimal", "1.0.0").is_some());
assert!(
loaded.index.lookup("cratesio", "sudo", "1.0.0").is_none(),
"the unsigned pack's content must not have leaked in"
);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_malformed_key_in_the_environment_is_ignored_not_trusted() {
let _guard = crate::test_util::locked();
std::env::remove_var("HOUNDD_DEFS_KEY");
let baseline = trusted_keys().len();
std::env::set_var("HOUNDD_DEFS_KEY", "obviously-not-hex");
let keys = trusted_keys();
std::env::remove_var("HOUNDD_DEFS_KEY");
assert_eq!(
keys.len(),
baseline,
"garbage in the environment must not enter the trust store"
);
}
#[test]
fn hex_keys_of_the_wrong_length_are_rejected() {
assert!(parse_hex_key("aabb").is_none());
assert!(parse_hex_key(&"a".repeat(63)).is_none());
assert!(parse_hex_key(&"zz".repeat(32)).is_none());
}
#[test]
fn a_valid_hex_key_parses() {
let key = SigningKey::from_bytes(&[5u8; 32]);
assert_eq!(
parse_hex_key(&hex_of(&key)).map(|k| k.to_bytes()),
Some(key.verifying_key().to_bytes())
);
}
}