Signature feeds

Generations, not updates

A rule set is not a stream of edits; it is a generation: an immutable, signed directory. Vendor generation IDs are exactly YYYYMMDDTHHMMSSZ-NNNN, for example 20260903T120000Z-0001. Local overlays use local-<64 hex characters>.

<gen>/
├── manifest.json
├── manifest.sig
├── yara/       *.yar, rules.yarc, rules.yarc.meta
├── hashes/     *.hsb + *.hsb.meta.json
├── allow/      *.sha256
└── clam/       *.ndb, *.hdb, *.fp, *.sfp, *.crtdb

manifest.json is canonical JSON containing gen, created_at, expires_at, engine requirements (yara_version_min, clamav_min), the file list, counts and a key_id. Every file carries a SHA-256 digest, a size and full provenance: source URI, SPDX licence, author, upstream commit and whether it was modified. manifest.sig is a canonical JSON sidecar with an Ed25519 signature; the key ID is the first eight bytes of the SHA-256 of the public key.

Schema limits are hard: at most 4096 manifest files, at most ten million per count, an expiry no more than ten years after creation, and one to 4096 unique generations in an index.

Download and verification

The client demands HTTPS, normal certificate validation and same-origin for the base URL, redirects and artefacts. Defaults: 30 s HTTP timeout, 16 MiB per response, 64 MiB per file, 512 MiB per generation and 4096 files.

The order matters: the client checks the manifest digest from the signed index before it checks the manifest signature, then every file by size and digest, then verifies the whole generation again from the temporary directory before publishing it atomically into staging.

A normal fetch accepts only a generation strictly newer than the active vendor base. Older generations are reachable only through an explicitly requested rollback — which is itself fully verified and must not be expired.

Revocation

The signed revoked array of an index is absorbed into an append-only tombstone file before any new generation is selected. Entries are never removed, even if a later index stops listing them. An active or stored revoked generation is deactivated immediately, and the manager then loads the newest non-revoked, fully verified generation. If there is no safe replacement, the state stays visibly faulty rather than silently reloading a revoked feed. A rollback onto a denylisted generation is permanently blocked.

The corpus gate

No generation is activated without passing a gate:

• A known-good corpus and a known-bad corpus must both exist and contain at least one safe, regular file. Symlinks, special files, unsafe owners or modes and incomplete scan results are refused.
• The compiler must return without errors, and for vendor generations the rule set generation, rule count and hash count must match the manifest exactly.
• A licence violation is always a hard rejection. A plain compiler without licence awareness is not accepted, even when it reports success.
• Engine requirements are enforced against the versions the worker protocol actually reports. Missing, malformed, over-long or pre-release-suffixed versions are refused fail-closed, as is a version below what the manifest requires.
• Every malicious hit in the good corpus is a hard gate failure. The detection ratio on the bad corpus is recorded as a metric; below 90 % it produces a warning rather than an automatic rejection, because a low ratio is a quality problem and a false positive is an outage.

Activation

Activation and rollback are root-only. Signature, manifest canonicality and every file digest are re-verified immediately before the switch — including after the move out of staging, because a long gate run is a long window.

The generation is moved into the feed root without overwriting anything, and the current symlink is swapped atomically through a temporary symlink and rename. Only then does the worker reload, the cache invalidate the previous generation, and the audit and store records get written. Failures trigger a best-effort compensation that restores the previous symlink, worker state and Clam files. Where a compensation cannot restore a consistent state, the actually loaded state is reported as it is, with the error, rather than falsely claiming a rejection.

Retention always protects current, explicitly preserved generations, and any vendor base a local overlay refers to; beyond that the newest keep_generations (default 3) survive.

The circuit breaker

A multi-tenant circuit breaker compares the last hour against the 24-hour mean. By default it requires at least 50 quarantines and more than ten times the mean, and additionally at least three distinct accounts and at least 20 % of all active accounts. Both account thresholds are configurable.

The point of those two extra conditions is blunt: a single account can never trigger a server-wide rollback. One customer uploading a hundred infected files is a customer problem, not a rule problem.

Heuristic, missing or contradictory signals never trigger a rollback. After a successful rollback, alarm and audit hooks are retried if needed — without a second rollback.

Licences and provenance

The build only admits DRL-1.1, BSD-3-Clause, Apache-2.0 and MIT (plus GPL-2.0-only for ClamAV databases). Rejected files and rules appear deterministically on stderr; missing provenance is never a silent skip.

For every rule that can appear in a runtime signal, the metadata license, author and one of reference, ref or source must be present and match the provenance CSV exactly. A CSV row on its own is not enough. Mismatched or missing attribution is discarded fail-closed.

Where an upstream licence requires author, rule URI and licence notice to survive into detection reports, they do: the attribution travels into the panel display and the notification mail.

Source decisions, stated plainly:

Local overlays

An operator can build a local overlay on top of a verified vendor base. The only permitted construction verifies the complete signed vendor generation with pinned keys first. Local YARA and hash sources must come from root-owned directories that are not group- or world-writable; symlinks and hard links are refused. The local manifest is unsigned but references the vendor generation and its manifest digest, and inherits the vendor expiry.

Configuration

[feeds]
url = "https://feeds.example.invalid/shelltrap"
public_keys = ["/etc/shelltrap/keys/vendor.pub"]
interval = "1h"
keep_generations = 3
corpus_dir = "/var/lib/shelltrap/corpus"
rules_dir = "/etc/shelltrap/rules.d"
hashes_dir = "/etc/shelltrap/hashes.d"

url is a base URL; the client appends index.json itself. The timer uses interval with ±10 % jitter. With an empty url but configured pins, the timer still re-evaluates local rule and hash overlays. Health reports an unconfigured remote feed as feeds: not_configured, and that sub-status alone does not make the overall status degraded — a missing active generation does.

shelltrap feeds list
shelltrap feeds update
shelltrap feeds rollback 20260903T120000Z-0001

Next

• Licensing and activation — why the feed asks for a token
• Configuration and policies