Uh oh!
There was an error while loading. Please reload this page.
feat(guardrails): capture-group scoping + configurable replacement for pii custom patterns - #1007
Conversation
…r pii custom patterns (AISIX-Cloud#1334)
- A custom pattern regex with at least one capture group now rewrites
only group 1 of each match, keeping the rest of the match verbatim,
so a rule can replace a value while preserving its key/label
("version": "12.1" -> "version": "***" stays parseable JSON).
Patterns without capture groups keep the whole-match semantics.
- Checksum validators (Luhn / ISO 7064) now run on the replaced span
(group 1 when present), so a prefixed pattern cannot silently
disable its validator.
- PiiCustomPattern gains an optional replacement field overriding the
default [<NAME>_REDACTED] token; empty string deletes the span; the
text is literal (no group expansion).
- replacement on a pattern whose effective action is block rejects the
row at build time (a knob is enforced as written or rejected, never
accepted-but-unread).
- e2e: capture-group rules drive a real DP end to end - request and
response rewritten in place (zh + en), hard negatives byte-identical,
embedded JSON still parses; regenerated guardrail.schema.json.Warning Review limit reachedYour included review limit has been reached. You’re in a promotional period — use the checkbox below to run this review for free:
On-demand reviews are free for the next 31 days. After that, they cost $0.25 per reviewed file. How can I continue?Run this review now using the option above, or comment You can also wait for the limit to reset (next review available in 1 minute), then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…groups group_scoped is auto-detected from the pattern, so an accidental capturing group in a future builtin would silently narrow its replacement to group 1. Assert captures_len() == 1 for every builtin (audit rider on #1007).
Uh oh!
There was an error while loading. Please reload this page.
Closes half of api7/AISIX-Cloud#1334 (DP side). Paired CP PR: api7/AISIX-Cloud#1347.
Problem
The first-level regex guardrail could detect but not usefully rewrite the POC's two fixed shapes.
mask_allreplacedcaps[0](the whole match) and the mask token was hard-derived from the detector name, so:"version": "12.1"→ the key, quotes and colon were swallowed, breaking the JSON structure;\d+\.\d+) false-positives on every duration/size/IP/file:line in a log;regexcrate has no lookaround, so there is no syntax-level escape — the semantics had to come from capture groups.Changes
custom_patterns[].replacement(optional, ≤256 chars): literal override for the default[<NAME>_REDACTED]token; empty string deletes the span; no$nexpansion. A replacement on a pattern whose effective action isblockrejects the row at build time — a knob is enforced as written or rejected, never accepted-but-unread.schemas/resources/guardrail.schema.json.Compatibility
replacementload and behave identically (serde default +skip_serializing_if).replacementrow-kills on every released DP: the guardrail config structs deny unknown fields behind the flattened kind enum (pinned byunknown_field_rejected_by_inner_kind_struct), so this is the row_rejected class, not field_ignored. The paired CP PR registers thedpCompatGateentry (MinDP unreleased) and carries the save-time warning plumbing + wire e2e.Reference check (CLAUDE.md §7)
The managed DLP services of the three major clouds and the leading open-source PII anonymizer all pair strong-format regexes with a configurable replacement value and context-anchored matching rather than lookaround; scoping the rewrite to an explicit capture group is the established way to express "replace the value, keep the label" when the engine (like Rust's
regex, see https://docs.rs/regex/latest/regex/#untrusted-input and its documented absence of lookaround) trades lookaround for linear-time matching. The branded survey lives in api7/AISIX-Cloud#1331 / #1334.Verification
cargo test -p aisix-guardrails -p aisix-core: 259 + 603 pass (9 new unit tests: group scoping, JSON parseability, hard negatives byte-diff, validator-on-group for mask and block, non-participating group, default/empty/literal replacement, no-group replacement).guardrail-pii-capture-group-e2e.test.tsagainst a real DP + etcd: request-side rewrite in place (zh + en labels), hard negatives (Elapsed: 12.345s,4.2 GB,0.13um,top.v:12:1,10.2.255.1) byte-identical via exact-equality assertions, no-hit prompt byte-identical, response-side JSON form rewritten and re-parsed (version == "***", sibling fields intact), whole flow 200 — mask never blocks. 3/3 pass.cargo fmt+clippy: clean.