fix(agent_hub): BUG redact outbound secrets by content, not file path - #969
Open
mushenL wants to merge 4 commits into
Open
fix(agent_hub): BUG redact outbound secrets by content, not file path#969mushenL wants to merge 4 commits into
mushenL wants to merge 4 commits into
Conversation
The comments added with the content-driven redaction layer restated the code, repeated the same rationale twice in one file, and carried a bug narrative that belongs in the commit history. What remains is only the non-obvious why: the case-sensitive Bearer match, the sk- left boundary, why scrub_url_secrets is not reused for free text, why the base64 pass is not behind the prefilter, and the never-raise / original-bytes / idempotent contracts. Also fixes the vendor labels, which were shifted one line off their patterns. Test methods are consolidated, not thinned: the vendor cases now cover all eleven prefix branches instead of two, and the homogeneous false-positive guards became subTest tables. Four redundant cases left the repo suite -- a multiline rerun of an already per-line corpus, a trivial report-formatting assertion, a per-framework benign check now folded into the leak test, and a dry-run report check already covered by two existing tests. No behavior change: the leak repro still reports 0 and the false-positive benchmark still reports 0 hits over 690 clean files. Suite goes 369 -> 361 passed with subtests 93 -> 102.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Summary
sanitize_outbound_filedecided whether to clean a file by its PATH: eachframework whitelisted its own root config (ms-agent settings.json/mcp.json,
qwenpaw agent.json, hermes config.yaml, openhuman config.toml) and returned
everything else verbatim -- openclaw, nanobot and qoder defined no hook at all.
The collect patterns take skills/* recursively plus the persona and memory
documents, so a key an AI assistant wrote into a skill script, a skill-local
mcp.json, SOUL.md or MEMORY.md was uploaded into the remote repo and its git
history.
Add a content-driven layer (_secrets.py) that runs after the per-framework hook
in _sync.sanitize_outbound, the only production caller of that hook, so all
seven frameworks are covered without editing a single spec:
mcpServersshapetrigger. A shape-triggered file is cleaned inside that subtree only -- the
shared vocabulary blanks any key named
tokens/keys, which would destroydata in a skill's JSON fixture or a memory dump.
with a
[REDACTED:<kind>]marker (vendor prefixes, JWT, Bearer, a narrowedkey-name vocabulary, URL credentials, base64 payloads that decode to a key).
The bag rules never apply here, and the value gate refuses variable
references, placeholders, identifier shapes and low-entropy strings so
documentation survives.
The layer never raises (the watch daemon swallows exceptions and would silently
stop syncing), returns the original bytes object when nothing was redacted
(
drop_unchanged_defaultsand the sha256 push-skip both compare bytes), and isidempotent (the watcher baselines the sanitized sha). Findings bubble up to a
Secrets redactedtable in upload -- printed before the --dry-run return -- andto a warning in the watch log; they carry the kind and key name, never the
secret.
Also promotes openhuman's private TOML scrubber to a shared
scrub_toml_secretsso .toml at any path uses one vocabulary.Redaction is outbound only:
sanitize_inbound_filealso serves local convertwrites, where it would strip the user's own keys from the converted agent.
Already-leaked secrets stay in the remote git history and must be rotated.
Verified: 18 sentinels across 7 frameworks leak 0 (was 31 hits on ms-agent +
qwenpaw); 0 modifications across 690 clean files (default templates, docs, SDK
source, real converted agent packages); tests/agent_hub 369 passed.
Checklist
pre-commit installandpre-commit run --all-filesbefore git commit, and passed lint check.