Armed the execution gate on the live server for the first time. It
reported itself armed on a dedicated tmpfs, and then let EICAR execute.
Counters: 0 allowed, 0 blocked. Not one event was ever delivered.
Cause: systemd gives the service a PRIVATE MOUNT NAMESPACE. Several
perfectly ordinary hardening options force one — ProtectProc,
ProtectKernelTunables, ProtectControlGroups — and none of them mention
it. FAN_MARK_MOUNT marks a vfsmount, and a private namespace holds its
own vfsmount for the same filesystem. So the daemon marked its copy,
every other process on the machine used the host's copy, and the gate
protected nothing while claiming to be armed.
That is the worst way for a security feature to fail: silently, with a
reassuring status line. Nothing in the unit tests could have caught it —
they run in the host namespace, where the mount mark works.
Fixed by always using FAN_MARK_FILESYSTEM, which marks the SUPERBLOCK.
A superblock is shared across namespaces, so events arrive from
everywhere, and scoping still works because a superblock is exactly one
filesystem: marking a dedicated mount covers that mount and nothing
else. mark_mount is kept for the smoke-test example, which runs outside
systemd, with a doc comment about when it lies to you.
Two more that only appeared once the gate was actually armed:
* SystemCallFilter=@system-service kills the daemon with SIGSYS the
moment the gate is switched on. fanotify_init and fanotify_mark live
in @privileged, which @system-service deliberately excludes. Granted
individually rather than by adding @privileged, which would also admit
setuid, chroot, bpf and kexec_load. Invisible until armed — the
service starts fine with the gate off.
* The capability reduction reported "60 capabilities could not be
dropped" while the end state was perfectly correct. systemd's
CapabilityBoundingSet had already done the work, and the service does
not hold CAP_SETPCAP afterwards, so every redundant drop failed EPERM.
It now checks what is actually present, attempts only that, and judges
by the end state rather than by return codes.
Also removed AmbientCapabilities from the unit. Ambient capabilities are
inherited by children, the daemon shells out to freshclam/rpm/pacman on
some paths, and a root process already receives the bounding set as
permitted — so it bought nothing except a way for CAP_SYS_ADMIN to leak
into a subprocess.
Performance, measured on the live server rather than guessed at:
+2.70 ms/exec as first written
+1.47 ms/exec after the reader blocked on poll() instead of sleeping
a millisecond between empty reads — that sleep sat on
the critical path of every execve
+1.38 ms/exec after answering cache hits in the reader thread, with
no channel handoff or worker wakeup
2,680 execs/sec sustained through the gate, 16-way parallel, with
ZERO watchdog rescues — the queue never fell behind. Ungated is 7,455.
Caddy stayed at sub-millisecond throughout and load did not rise.
Joe and Henry are right that the exec-heavy paths on this box — Docker
overlays, agent workspaces, PM2 — are the performance bar rather than an
exclusion list. Protecting agent workspaces from injected payloads is
the product. 2,680/sec with no backlog is roughly ten times what this
machine generates, so the bar looks clearable; stage 2 will say for sure.
295 tests pass, and the three-phase gate smoke test still passes
including the fail-open case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
184 lines
6.3 KiB
Rust
184 lines
6.3 KiB
Rust
//! End-to-end proof that the execution gate blocks, allows, and — above
|
|
//! all — fails open.
|
|
//!
|
|
//! Run as root. It creates its OWN tmpfs and marks only that mount, so a
|
|
//! bug here can freeze processes touching that scratch directory and
|
|
//! nothing else. Never point this at `/` while developing.
|
|
//!
|
|
//! cargo build --example gate-smoke
|
|
//! sudo ./target/debug/examples/gate-smoke
|
|
//!
|
|
//! Three phases:
|
|
//!
|
|
//! 1. A benign binary runs, and quickly.
|
|
//! 2. A malicious one is denied and never executes.
|
|
//! 3. With a scanner deliberately stalled past the deadline, the
|
|
//! watchdog lets the process through anyway. This is the phase that
|
|
//! matters: everything else is a feature, this is the promise that
|
|
//! a bug in Hound cannot wedge someone's machine.
|
|
|
|
use std::path::{Path, PathBuf};
|
|
use std::process::Command;
|
|
use std::sync::atomic::{AtomicBool, Ordering};
|
|
use std::sync::Arc;
|
|
use std::time::{Duration, Instant};
|
|
|
|
#[path = "../src/fanotify.rs"]
|
|
mod fanotify;
|
|
|
|
use fanotify::{Gate, GateConfig};
|
|
|
|
const EICAR: &str = r"X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*";
|
|
const MOUNT: &str = "/tmp/hound-gate-smoke";
|
|
|
|
fn sh(cmd: &str) -> bool {
|
|
Command::new("sh")
|
|
.arg("-c")
|
|
.arg(cmd)
|
|
.status()
|
|
.map(|s| s.success())
|
|
.unwrap_or(false)
|
|
}
|
|
|
|
fn main() {
|
|
let mount = PathBuf::from(MOUNT);
|
|
let _ = std::fs::create_dir_all(&mount);
|
|
|
|
if !sh(&format!("mount -t tmpfs -o size=16m tmpfs {MOUNT}")) {
|
|
eprintln!("could not mount the scratch tmpfs — are you root?");
|
|
std::process::exit(2);
|
|
}
|
|
|
|
let passed = std::panic::catch_unwind(|| run(&mount)).unwrap_or(false);
|
|
|
|
// Teardown runs whatever happened above.
|
|
let _ = sh(&format!("umount -l {MOUNT}"));
|
|
let _ = std::fs::remove_dir(&mount);
|
|
|
|
if passed {
|
|
println!("\nGATE SMOKE: PASS");
|
|
} else {
|
|
println!("\nGATE SMOKE: FAIL");
|
|
std::process::exit(1);
|
|
}
|
|
}
|
|
|
|
fn run(mount: &Path) -> bool {
|
|
let bad = mount.join("miner.sh");
|
|
let good = mount.join("hello.sh");
|
|
let slow = mount.join("slow.sh");
|
|
std::fs::write(&bad, format!("#!/bin/sh\n# {EICAR}\necho SHOULD_NEVER_PRINT\n")).unwrap();
|
|
std::fs::write(&good, "#!/bin/sh\nexit 0\n").unwrap();
|
|
std::fs::write(&slow, "#!/bin/sh\nexit 0\n").unwrap();
|
|
for f in [&bad, &good, &slow] {
|
|
sh(&format!("chmod +x {}", f.display()));
|
|
}
|
|
|
|
let gate = match Gate::init() {
|
|
Ok(g) => Arc::new(g),
|
|
Err(e) => {
|
|
eprintln!("fanotify_init failed: {e}");
|
|
return false;
|
|
}
|
|
};
|
|
if let Err(e) = gate.mark_mount(mount) {
|
|
eprintln!("fanotify_mark failed: {e}");
|
|
return false;
|
|
}
|
|
let watchdog = gate.start_watchdog();
|
|
|
|
// Flipped on for phase 3 to stall the scanner past the deadline.
|
|
let stall = Arc::new(AtomicBool::new(false));
|
|
let stall_rx = Arc::clone(&stall);
|
|
|
|
// Stands in for the engine: the gate is what is under test here.
|
|
// Note it receives bytes and never opens the gated file — reopening a
|
|
// path on a watched mount deadlocks the daemon against itself.
|
|
let scan = Arc::new(move |_p: &Path, bytes: &[u8]| -> Option<String> {
|
|
if stall_rx.load(Ordering::SeqCst) {
|
|
std::thread::sleep(Duration::from_secs(5));
|
|
}
|
|
String::from_utf8_lossy(bytes)
|
|
.contains("EICAR-STANDARD-ANTIVIRUS-TEST-FILE")
|
|
.then(|| "EICAR-Test-Signature".to_string())
|
|
});
|
|
|
|
let handles = gate.serve(
|
|
GateConfig {
|
|
workers: 4,
|
|
max_size: 100 * 1024 * 1024,
|
|
excludes: Vec::new(),
|
|
},
|
|
scan,
|
|
Arc::new(|p: &Path, name: &str, v: fanotify::Verdict| {
|
|
println!(" detect: {} ({name}) {v:?}", p.display());
|
|
}),
|
|
// No fast path in the smoke test: every event must reach a worker,
|
|
// so phase 3 can genuinely stall one.
|
|
Arc::new(|_e: &fanotify::Event| None),
|
|
);
|
|
println!("gate armed on {MOUNT} — 1 reader, 4 workers, watchdog live");
|
|
std::thread::sleep(Duration::from_millis(100));
|
|
|
|
let mut pass = true;
|
|
|
|
// ── 1. benign ──
|
|
println!("\n[1] benign binary expect: runs");
|
|
let t = Instant::now();
|
|
let benign_ran = sh(&format!("{} >/dev/null 2>&1", good.display()));
|
|
let benign_took = t.elapsed();
|
|
println!(" ran={benign_ran} in {benign_took:?}");
|
|
if !benign_ran {
|
|
eprintln!(" FAIL: the gate blocked a clean binary");
|
|
pass = false;
|
|
}
|
|
if benign_took > Duration::from_millis(200) {
|
|
eprintln!(" FAIL: {benign_took:?} of overhead on a clean exec");
|
|
pass = false;
|
|
}
|
|
|
|
// ── 2. malicious ──
|
|
println!("\n[2] EICAR binary expect: blocked");
|
|
let t = Instant::now();
|
|
let malicious_ran = sh(&format!("{} >/dev/null 2>&1", bad.display()));
|
|
println!(" ran={malicious_ran} in {:?}", t.elapsed());
|
|
if malicious_ran {
|
|
eprintln!(" FAIL: the EICAR binary executed");
|
|
pass = false;
|
|
}
|
|
|
|
// ── 3. the promise ──
|
|
println!("\n[3] scanner stalled 5s expect: watchdog lets it run anyway");
|
|
let (_, _, timed_out_before) = gate.responder().counters();
|
|
stall.store(true, Ordering::SeqCst);
|
|
let t = Instant::now();
|
|
let stalled_ran = sh(&format!("{} >/dev/null 2>&1", slow.display()));
|
|
let stalled_took = t.elapsed();
|
|
stall.store(false, Ordering::SeqCst);
|
|
let (_, _, timed_out_after) = gate.responder().counters();
|
|
let rescued = timed_out_after - timed_out_before;
|
|
println!(" ran={stalled_ran} in {stalled_took:?}, watchdog rescued {rescued} event(s)");
|
|
|
|
if !stalled_ran {
|
|
eprintln!(" FAIL: a stalled scanner blocked a process — this is the wedge");
|
|
pass = false;
|
|
}
|
|
if stalled_took > Duration::from_secs(3) {
|
|
eprintln!(" FAIL: held for {stalled_took:?}; the deadline did not apply");
|
|
pass = false;
|
|
}
|
|
if rescued == 0 {
|
|
eprintln!(" FAIL: nothing timed out, so the fail-open path never ran");
|
|
pass = false;
|
|
}
|
|
|
|
gate.stop();
|
|
for h in handles {
|
|
let _ = h.join();
|
|
}
|
|
let _ = watchdog.join();
|
|
|
|
let (allowed, denied, timed_out) = gate.responder().counters();
|
|
println!("\ncounters: allowed={allowed} denied={denied} timed_out={timed_out}");
|
|
pass
|
|
}
|