rules: every rule needs an anchor, and the test now proves it for all of them
Stage 3 passed on throughput — 27,339 events, zero rescues, Caddy unmoved — and then the soak found what the load test could not. Two more false positives, both the same shape as the ones before: * EICAR matched an 8.5 MB rustc incremental-compilation cache, because the test source being compiled contains the literal. Hound moved it to quarantine mid-build and rustc panicked. The standard defines the EICAR file as exactly that 68-byte string, optionally padded to 128, so the rule now says filesize <= 128. * The webshell rule matched a 4.3 MB AI session transcript, because the conversation had been discussing webshells and therefore contained "<?php", the eval pattern and "$_POST". The transcript was moved to the vault and its history lost. A webshell is a PHP file: small, and opening with a PHP tag. Now filesize < 1MB and $php in (0..4096). The interesting part is why the second one happened at all. After the first, I added a test asserting that a large file containing rule strings is not a threat — and hand-listed the strings. I listed the miner's and the rootkit's and forgot "<?php". The test passed and the transcript was quarantined anyway. So the test now extracts every string literal from the rule pack itself and builds the haystack from those. A rule added tomorrow is covered without anybody remembering to cover it. It also asserts the extractor actually found the strings, because a parser that silently returns nothing would make the whole thing vacuous. Both fixes have a paired test that the detection still works: a real 68-byte EICAR file is caught, padded to 128 it is caught, and a real webshell is caught. Worth recording, because it is not a bug: six houndd tests failed while the gate was armed. Hound quarantined the EICAR fixtures the test suite had just written — correct behaviour, colliding with a suite that creates real malware samples. Running the antivirus's own tests on a gated machine needs thought; the tests are not wrong and neither is the gate. The definitions chain now works end to end: pack built from OSV, signed with the release key, published to /srv/houndav/defs, installed, and verified on load against the public half compiled into the agent — "defs: 19 indicators from 1 pack(s) [2026.08.21]". The public key is in the source on purpose. The agent is open source and anybody should be able to check that the definitions they received are the ones we published. 300 tests pass. Gate is off pending these fixes being soaked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
be5396821d
commit
2cf9a740c3
5 changed files with 229 additions and 21 deletions
|
|
@ -94,7 +94,8 @@ fn main() {
|
|||
indicators,
|
||||
};
|
||||
|
||||
let signed = pack::sign(&p, &key, "dev").expect("signing the pack");
|
||||
let key_id = std::env::var("HOUND_KEY_ID").unwrap_or_else(|_| "hound-2026".into());
|
||||
let signed = pack::sign(&p, &key, &key_id).expect("signing the pack");
|
||||
std::fs::write(out, serde_json::to_string(&signed).expect("encoding")).expect("writing");
|
||||
|
||||
println!("read {files} OSV records");
|
||||
|
|
|
|||
|
|
@ -20,7 +20,17 @@ rule EICAR_Test_File
|
|||
strings:
|
||||
$eicar = "X5O!P%@AP[4\\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*"
|
||||
condition:
|
||||
$eicar
|
||||
// The standard defines the EICAR file as exactly this 68-byte
|
||||
// string, optionally padded with whitespace to at most 128 bytes.
|
||||
// Without the size bound this rule matches any file that merely
|
||||
// CONTAINS the string — and on a live server it quarantined an
|
||||
// 8.5 MB rustc incremental-compilation cache, because the test
|
||||
// source being compiled contained the literal. That killed the
|
||||
// build with a compiler panic.
|
||||
//
|
||||
// Anyone whose source, logs or documentation mention EICAR has
|
||||
// the same problem, which is most security work.
|
||||
filesize <= 128 and $eicar
|
||||
}
|
||||
|
||||
rule Linux_Coinminer_XMRig
|
||||
|
|
@ -62,7 +72,16 @@ rule Linux_Webshell_PHP_Eval
|
|||
$src3 = "$_REQUEST"
|
||||
$src4 = "$_COOKIE"
|
||||
condition:
|
||||
$php and $eval1 and 1 of ($src*)
|
||||
// A webshell is a PHP file: small, and opening with a PHP tag.
|
||||
// Without those bounds this matched a 4.3 MB AI session
|
||||
// transcript on a live server — the conversation happened to
|
||||
// discuss webshells, so it contained "<?php", the eval pattern
|
||||
// and "$_POST". The transcript was moved to quarantine and its
|
||||
// history lost.
|
||||
filesize < 1MB
|
||||
and $php in (0..4096)
|
||||
and $eval1
|
||||
and 1 of ($src*)
|
||||
}
|
||||
|
||||
rule Linux_Rootkit_Preload
|
||||
|
|
|
|||
|
|
@ -13,10 +13,11 @@
|
|||
//! injection, pickles and MCP audits need no feed, and refusing to start
|
||||
//! would leave the machine with nothing.
|
||||
//!
|
||||
//! The signing key is deliberately **not** compiled in as a placeholder.
|
||||
//! A fake key that looks real is how a development shortcut becomes a
|
||||
//! shipped vulnerability; an empty trust store that refuses to load is
|
||||
//! noisy in exactly the way that gets fixed before release.
|
||||
//! The trust store holds the PUBLIC half of the release signing key, in
|
||||
//! the source, where everyone can read it — that is the point. There was
|
||||
//! never a placeholder here: a fake key that looks real is how a
|
||||
//! development shortcut becomes a shipped vulnerability, so until the
|
||||
//! real one existed the store was empty and said so loudly.
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use ed25519_dalek::VerifyingKey;
|
||||
|
|
@ -27,10 +28,24 @@ use std::time::SystemTime;
|
|||
|
||||
/// Keys whose packs this build will load.
|
||||
///
|
||||
/// EMPTY ON PURPOSE. The release key is injected at build time; until it
|
||||
/// is, `HOUNDD_DEFS_KEY` supplies one for development. See the module
|
||||
/// note above for why there is no placeholder here.
|
||||
const TRUSTED_KEYS: &[(&str, [u8; 32])] = &[];
|
||||
/// The PUBLIC half of Hound's definitions signing key. It belongs in the
|
||||
/// source: the agent is open source, everyone can read it, and that is
|
||||
/// the point — anybody can verify that the packs they receive are the
|
||||
/// ones we published. The private half never leaves the build machine
|
||||
/// and is not in this repository.
|
||||
///
|
||||
/// Rotation: add the new key beside the old one, ship that build, then
|
||||
/// start signing with the new key and remove the old one a release later.
|
||||
/// Never swap in one step, or every agent that has not updated yet stops
|
||||
/// accepting definitions.
|
||||
const TRUSTED_KEYS: &[(&str, [u8; 32])] = &[(
|
||||
"hound-2026",
|
||||
[
|
||||
0x12, 0xba, 0x51, 0x9f, 0x13, 0xe6, 0xe8, 0x37, 0x00, 0xef, 0x3e, 0xfb, 0x07, 0xe9,
|
||||
0x32, 0x85, 0xc4, 0x88, 0x79, 0x30, 0x26, 0x04, 0xa3, 0x20, 0xa0, 0x2d, 0xc3, 0x64,
|
||||
0x29, 0x90, 0xb4, 0x51,
|
||||
],
|
||||
)];
|
||||
|
||||
/// Where signed packs live.
|
||||
pub fn defs_dir() -> Option<PathBuf> {
|
||||
|
|
@ -246,26 +261,63 @@ mod tests {
|
|||
}
|
||||
|
||||
#[test]
|
||||
fn the_build_ships_no_placeholder_key() {
|
||||
// A fake key that looks real is how a development shortcut becomes
|
||||
// a shipped vulnerability.
|
||||
fn the_trust_store_holds_exactly_the_release_key() {
|
||||
// A second key appearing here without a rotation plan is how a
|
||||
// development shortcut becomes a shipped vulnerability.
|
||||
assert_eq!(TRUSTED_KEYS.len(), 1);
|
||||
assert_eq!(TRUSTED_KEYS[0].0, "hound-2026");
|
||||
assert!(
|
||||
TRUSTED_KEYS.is_empty(),
|
||||
"a placeholder signing key must never be compiled in"
|
||||
VerifyingKey::from_bytes(&TRUSTED_KEYS[0].1).is_ok(),
|
||||
"the compiled-in key must be a valid ed25519 public key"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn without_a_trusted_key_nothing_loads_and_it_says_why() {
|
||||
fn the_release_key_verifies_a_pack_signed_by_it() {
|
||||
// Catches a fat-fingered byte in the constant, which would
|
||||
// silently stop every agent from accepting definitions.
|
||||
let published = std::path::Path::new("/srv/houndav/defs");
|
||||
if !published.is_dir() {
|
||||
return; // only meaningful on the build host
|
||||
}
|
||||
let trusted: Vec<(&str, VerifyingKey)> = TRUSTED_KEYS
|
||||
.iter()
|
||||
.filter_map(|(id, b)| VerifyingKey::from_bytes(b).ok().map(|k| (*id, k)))
|
||||
.collect();
|
||||
let mut checked = 0;
|
||||
for e in std::fs::read_dir(published).into_iter().flatten().flatten() {
|
||||
let p = e.path();
|
||||
if p.extension().is_none_or(|x| x != "pack") {
|
||||
continue;
|
||||
}
|
||||
checked += 1;
|
||||
let text = std::fs::read_to_string(&p).unwrap();
|
||||
let signed: SignedPack = serde_json::from_str(&text).unwrap();
|
||||
assert!(
|
||||
pack::verify(&signed, &trusted).is_ok(),
|
||||
"the compiled key does not verify {p:?}"
|
||||
);
|
||||
}
|
||||
let _ = checked;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_definitions_directory_says_so_rather_than_staying_silent() {
|
||||
// Silence would let an operator believe they were protected when
|
||||
// nothing had loaded.
|
||||
let dir = tmp("empty");
|
||||
let _guard = crate::test_util::locked();
|
||||
std::env::remove_var("HOUNDD_DEFS_KEY");
|
||||
std::env::set_var("HOUNDD_DEFS_DIR", &dir);
|
||||
let loaded = load_all();
|
||||
std::env::remove_var("HOUNDD_DEFS_DIR");
|
||||
|
||||
assert_eq!(loaded.indicators, 0);
|
||||
assert!(
|
||||
loaded.detail.contains("no signing key"),
|
||||
"silence here would let an operator believe they were protected: {}",
|
||||
loaded.detail
|
||||
!loaded.detail.is_empty(),
|
||||
"an empty load must explain itself"
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -336,10 +388,16 @@ mod tests {
|
|||
#[test]
|
||||
fn a_malformed_key_in_the_environment_is_ignored_not_trusted() {
|
||||
let _guard = crate::test_util::locked();
|
||||
std::env::remove_var("HOUNDD_DEFS_KEY");
|
||||
let baseline = trusted_keys().len();
|
||||
std::env::set_var("HOUNDD_DEFS_KEY", "obviously-not-hex");
|
||||
let keys = trusted_keys();
|
||||
std::env::remove_var("HOUNDD_DEFS_KEY");
|
||||
assert!(keys.is_empty());
|
||||
assert_eq!(
|
||||
keys.len(),
|
||||
baseline,
|
||||
"garbage in the environment must not enter the trust store"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -303,6 +303,136 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
/// Rules must not match haystacks — and this test builds the haystack
|
||||
/// FROM the rule pack itself, so a new rule cannot be forgotten.
|
||||
///
|
||||
/// Every content rule needs an anchor: a file-type check, a size
|
||||
/// bound, or a position constraint. Without one it fires on anything
|
||||
/// that happens to contain its strings. That has now cost four
|
||||
/// separate incidents on a live server — the daemon's own binary, an
|
||||
/// agent's session transcript twice, and an 8.5 MB rustc incremental
|
||||
/// cache that took the build down with it.
|
||||
///
|
||||
/// The first version of this test hand-listed the strings to include,
|
||||
/// and duly missed the webshell rule's "<?php", which is exactly how
|
||||
/// the transcript got quarantined a second time. Extracting them from
|
||||
/// the source makes the property automatic.
|
||||
#[test]
|
||||
fn a_large_file_containing_every_rule_string_is_not_a_threat() {
|
||||
let src = builtin();
|
||||
|
||||
// Every double-quoted string literal in the pack.
|
||||
let mut literals: Vec<String> = Vec::new();
|
||||
for line in src.lines() {
|
||||
let t = line.trim();
|
||||
if !t.starts_with('$') || !t.contains('=') {
|
||||
continue;
|
||||
}
|
||||
let mut chars = t.chars().peekable();
|
||||
let mut current = String::new();
|
||||
let mut inside = false;
|
||||
while let Some(c) = chars.next() {
|
||||
match c {
|
||||
'\\' if inside => {
|
||||
// Keep the escape's target, drop the backslash, so
|
||||
// "\\PZX" contributes the bytes a file would hold.
|
||||
if let Some(n) = chars.next() {
|
||||
current.push(n);
|
||||
}
|
||||
}
|
||||
'"' => {
|
||||
if inside {
|
||||
if !current.is_empty() {
|
||||
literals.push(std::mem::take(&mut current));
|
||||
}
|
||||
inside = false;
|
||||
} else {
|
||||
inside = true;
|
||||
}
|
||||
}
|
||||
_ if inside => current.push(c),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
assert!(
|
||||
literals.len() >= 15,
|
||||
"expected to extract the pack's strings, got {}: {literals:?}",
|
||||
literals.len()
|
||||
);
|
||||
for expected in ["<?php", "stratum+tcp://", "ld.so.preload", "$_POST"] {
|
||||
assert!(
|
||||
literals.iter().any(|l| l == expected),
|
||||
"{expected:?} was not extracted — the parser missed a rule"
|
||||
);
|
||||
}
|
||||
|
||||
// One document containing all of them, the size of a real log.
|
||||
let mut haystack: Vec<u8> = b"// build artefact / log / session transcript\n".to_vec();
|
||||
for l in &literals {
|
||||
haystack.extend_from_slice(l.as_bytes());
|
||||
haystack.push(b'\n');
|
||||
}
|
||||
haystack.resize(4 * 1024 * 1024, b'\n');
|
||||
|
||||
let set = RuleSet::compile().unwrap();
|
||||
let mut scanner = yara_x::Scanner::new(&set.rules);
|
||||
let hits: Vec<String> = scanner
|
||||
.scan(&haystack)
|
||||
.unwrap()
|
||||
.matching_rules()
|
||||
.map(|r| RuleSet::detection_name(&r))
|
||||
.collect();
|
||||
assert!(
|
||||
hits.is_empty(),
|
||||
"a 4 MB document mentioning every rule string is not malware: {hits:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_real_webshell_is_still_caught() {
|
||||
// The bounds must not cost the detection they exist for.
|
||||
let set = RuleSet::compile().unwrap();
|
||||
let mut scanner = yara_x::Scanner::new(&set.rules);
|
||||
let shell = br#"<?php @eval(base64_decode($_POST['x'])); ?>"#;
|
||||
let hits: Vec<String> = scanner
|
||||
.scan(shell)
|
||||
.unwrap()
|
||||
.matching_rules()
|
||||
.map(|r| RuleSet::detection_name(&r))
|
||||
.collect();
|
||||
assert!(
|
||||
hits.iter().any(|h| h == "Linux.Webshell.PHP-Eval"),
|
||||
"got {hits:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_real_eicar_file_is_still_caught() {
|
||||
// The size bound must not cost the detection it exists for.
|
||||
let set = RuleSet::compile().unwrap();
|
||||
let mut scanner = yara_x::Scanner::new(&set.rules);
|
||||
let eicar = br"X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*";
|
||||
let hits: Vec<String> = scanner
|
||||
.scan(eicar)
|
||||
.unwrap()
|
||||
.matching_rules()
|
||||
.map(|r| RuleSet::detection_name(&r))
|
||||
.collect();
|
||||
assert!(hits.iter().any(|h| h == "EICAR-Test-Signature"), "got {hits:?}");
|
||||
|
||||
// And padded to the 128 bytes the standard allows.
|
||||
let mut padded = eicar.to_vec();
|
||||
padded.resize(128, b' ');
|
||||
let hits2: Vec<String> = scanner
|
||||
.scan(&padded)
|
||||
.unwrap()
|
||||
.matching_rules()
|
||||
.map(|r| RuleSet::detection_name(&r))
|
||||
.collect();
|
||||
assert!(hits2.iter().any(|h| h == "EICAR-Test-Signature"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_real_elf_miner_is_still_caught() {
|
||||
// Requiring ELF magic must not cost the detection it exists for.
|
||||
|
|
|
|||
BIN
dist/hound_0.1.0_amd64.deb
vendored
BIN
dist/hound_0.1.0_amd64.deb
vendored
Binary file not shown.
Loading…
Reference in a new issue