Provider permissions: adversarial integration test suite

[jeonghun-jj-lee]

Important

Decision Surface

Problem: Individual slices test their own logic, but data-privacy guarantees require end-to-end adversarial testing. A model MUST NOT see content from denied directories through ANY path — direct reads, indirect bash access, symlinks, path traversal, cached context, or partially-redacted history.

Approach: A dedicated integration test suite that exercises the full permission stack (resolution + enforcement + tagging + redaction + context filtering) with adversarial scenarios designed to find leakage paths.

Scope: Integration + adversarial tests only. All implementation slices (#164–#172) must be complete first.

---

Acceptance Criteria

• Direct read denied: A model in a "deny-read" tier calling Read on a denied path is blocked with an error (never sees content)
• Bash indirect access: A model calling bash("cat ~/secrets/file.txt") where ~/secrets/ is execute-denied is blocked; if execute is allowed but read is denied, the bash output is NOT tagged with sourcePaths (known limitation documented) OR bash is blocked for paths matching denied read directories
• Symlink escape: A symlink at ~/allowed/link → ~/secrets/private.key — reading via the symlink resolves to the canonical path and is denied if ~/secrets/ is denied
• Path traversal: ~/allowed/../secrets/file.txt is normalized before permission check; access is denied
• Model switch mid-session: After switch from trusted→limited, the VERY NEXT tool call uses the new tier (no stale cached tier)
• History redaction completeness: After switch, ALL messages with sourcePaths in denied dirs are redacted — iterate through a 50-message history and verify zero leakage
• Partial redaction correctness: A message with content from 3 files (2 allowed, 1 denied) — the denied file's content is replaced, the allowed content remains intact
• Context filtering on switch: Instruction files from denied directories do not appear in the system prompt after model switch (grep the assembled prompt for known content from the denied file)
• Switch-back restoration: Switch trusted→limited (redacted) → trusted again: original content is visible again (stored messages never mutated)
• Default tier fallback: A brand-new model (not in assignments) resolves to the "Unassigned" tier — if Unassigned is "ask", verify the prompt fires; if "deny", verify block
• Config hot-reload: If the user changes tier permissions mid-session (via settings UI), subsequent tool calls use the updated rules immediately
• Empty tier (no directory rules): A tier with no directory rules → all paths fall through to existing permission system (not silently allowed)
• Glob edge cases: Rules for src/** do NOT match src-backup/file.txt; rules for **/*.env match .env files at any depth

Key Decisions

#	Decision	Rationale
1	Bash + denied read dirs is a DOCUMENTED LIMITATION or must be blocked	Bash can read files without going through the Read tool — this is the hardest leakage vector. The test must verify one of: (a) bash is blocked when read is denied for that path, or (b) this limitation is documented and accepted
2	Canonical path resolution for symlinks	The only correct behavior — permission checks on the real path, never the link path
3	Test suite runs in CI on every PR touching permission-related files	Regression protection — a future refactor must not open a leakage path

Testing Decisions

This IS the test slice. Tests should be structured as:

Unit-integration (fast, no I/O):

• Mock the config with known tiers
• Exercise resolveProviderPermission + redactHistory + context filtering together
• Verify no content from denied paths appears in any output

End-to-end (slower, with real config + session):

• Set up a real session with providerPermissions config
• Execute tool calls, switch models, verify full pipeline
• Assert on the actual API payload that would be sent to the provider

Adversarial (explicit attack patterns):

• Symlink creation + read attempt
• Path traversal strings
• Bash cat/grep of denied files
• Rapid model switching (A→B→A→B) with interleaved reads

Constraints & Invariants

• This suite MUST run in CI — it is the privacy regression gate
• Tests must not depend on the order of other test execution
• Tests should be clearly named with the attack vector they exercise (e.g., test_symlink_escape_to_denied_dir_is_blocked)
• Any new tool added to the system in the future must be evaluated against this test matrix (document which tools need sourcePath tagging)

Source

Part of #163
Blocked by #164, #165, #166, #167, #168, #169