e92e865ff5
Two changes to how definitions are distributed, sharing one mechanism. Incremental updates. The old nightly rebuilt the full per-ecosystem pack every day, so its hash changed and every client re-downloaded 44 MB of npm daily. Now the server publishes an immutable baseline plus small daily delta packs (crates/hound-defs/examples/build-delta.rs); the client — which already fetches only packs whose sha256 it lacks — pulls the baseline once and then kilobytes a day. When deltas pile up the builder folds them into a fresh baseline and drops the old files from the index; the client prunes whatever the index stops listing, so both the server dir and every client's defs dir stay bounded. Verified end to end: day-2 fetched only the delta (baseline untouched), day-3 rebaseline pruned the superseded packs. Free community tier. Free now gets a recent subset of the public OSV feed (build-community.rs, ~5,000 newest indicators) so a Free install detects current threats out of the box — not just the heuristics. Pro is the full 235k corpus, daily/near-real-time freshness, and the curated threat pack. The client always fetches the community channel and gates the full feed + threat pack on the licence; a lapse prunes both back to exactly what a fresh Free install has — community pack + built-in rules — while leaving any custom .yar the user placed themselves untouched. reload_rules() lets a new threat pack go live without a daemon restart. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
82 lines
3.2 KiB
Bash
Executable file
82 lines
3.2 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
#
|
|
# Build, gate, sign and publish the Hound Linux threat pack.
|
|
#
|
|
# Unlike the definition feed (rebuilt nightly from OSV), the threat pack is
|
|
# curated YARA and changes only when a human edits the rules, so this is run
|
|
# by hand — or by CI on a change to crates/hound-defs/rules/hound-linux.yar.
|
|
#
|
|
# The builder refuses to sign a pack that does not compile or that matches a
|
|
# system binary (the goodware gate), so a bad edit fails here, not in the
|
|
# field. Publishing is atomic per file and the index is rewritten last.
|
|
set -euo pipefail
|
|
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
KEY="${HOUND_DEFS_KEY:-$HOME/agents/hound/.secrets/defs-signing.key}"
|
|
DEST="${HOUND_DEFS_DIR:-/srv/houndav/defs}"
|
|
SRC="${HOUND_RULES_SRC:-$ROOT/crates/hound-defs/rules/hound-linux.yar}"
|
|
NAME="hound-linux"
|
|
VERSION="${1:-$(date -u +%Y.%m.%d)}"
|
|
CREATED="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
|
|
log() { printf '%s %s\n' "$(date -u +%H:%M:%S)" "$*"; }
|
|
|
|
[ -f "$KEY" ] || { echo "no signing key at $KEY" >&2; exit 1; }
|
|
[ -f "$SRC" ] || { echo "no rules source at $SRC" >&2; exit 1; }
|
|
[ -d "$DEST" ] || { echo "no destination directory $DEST" >&2; exit 1; }
|
|
|
|
BUILDER="$ROOT/target/release/examples/build-rules-pack"
|
|
if [ ! -x "$BUILDER" ]; then
|
|
log "building the rules-pack builder"
|
|
( cd "$ROOT" && cargo build --release -p houndd --example build-rules-pack )
|
|
fi
|
|
|
|
STAGE="$(mktemp -d "${TMPDIR:-/var/tmp}/hound-rules-stage.XXXXXX")"
|
|
trap 'rm -rf "$STAGE"' EXIT
|
|
PACK="$STAGE/${NAME}-${VERSION}.rpack"
|
|
|
|
# This compiles, runs the goodware gate against this host's binaries, and
|
|
# signs — or exits non-zero without writing anything.
|
|
"$BUILDER" "$SRC" "$NAME" "$PACK" "$KEY" "$VERSION" "$CREATED"
|
|
|
|
base="$(basename "$PACK")"
|
|
cp "$PACK" "$DEST/.$base.tmp"
|
|
chmod 644 "$DEST/.$base.tmp"
|
|
mv -f "$DEST/.$base.tmp" "$DEST/$base"
|
|
log "published $base"
|
|
|
|
# Rebuild the index over everything on disk — every definition pack
|
|
# (baselines and deltas) and every rules pack. Identical logic to
|
|
# refresh-definitions.sh so running either keeps the delta feed intact;
|
|
# version comes from inside the signed payload, not the filename.
|
|
python3 - "$DEST" <<'PY'
|
|
import base64, hashlib, json, os, sys
|
|
dest = sys.argv[1]
|
|
def entry(f):
|
|
raw = open(os.path.join(dest, f), "rb").read()
|
|
try:
|
|
version = json.loads(base64.b64decode(json.loads(raw)["payload"])).get("version", "")
|
|
except Exception:
|
|
version = ""
|
|
return {"file": f, "sha256": hashlib.sha256(raw).hexdigest(),
|
|
"size": len(raw), "version": version}
|
|
packs, community, rules = [], [], []
|
|
for f in sorted(os.listdir(dest)):
|
|
if not os.path.isfile(os.path.join(dest, f)):
|
|
continue
|
|
if f.endswith(".rpack"):
|
|
rules.append(entry(f))
|
|
elif f.startswith("community-") and f.endswith(".pack"):
|
|
community.append(entry(f))
|
|
elif f.endswith(".pack"):
|
|
packs.append(entry(f))
|
|
tmp = os.path.join(dest, ".index.json.tmp")
|
|
with open(tmp, "w") as fh:
|
|
json.dump({"packs": packs, "community": community, "rules": rules}, fh, indent=2)
|
|
fh.write("\n")
|
|
os.chmod(tmp, 0o644)
|
|
os.replace(tmp, os.path.join(dest, "index.json"))
|
|
print(f"index.json: {len(packs)} feed, {len(community)} community, {len(rules)} rules pack(s)")
|
|
PY
|
|
|
|
log "done"
|