Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Commit e3fda69

Browse files
atlowChemiaduh95
authored andcommitted
test_runner: extend tag filter with boolean expression DSL
Upgrades the experimental tag filter introduced by stage 1 to accept a boolean expression instead of a literal tag name. Grammar: `and`/`&&`, `or`/`||`, `not`/`!`, parentheses for grouping, and `*` wildcards inside identifiers. Standard precedence (`not > and > or`); binary operators are left-associative. Word forms require whitespace separation; punctuation forms do not. Untagged tests evaluate `false` for any include expression and `true` for `not X`, so excluding tags does not accidentally remove untagged tests. The flag and `testTagFilters` option are still repeatable; multiple expressions still AND together. Malformed expressions fail fast at the parent process at startup. Tag value validation tightens to reject whitespace, operator characters (`& | ! ( ) *`), and the reserved words `and`/`or`/`not` in any casing - a breaking change relative to the stage 1 ship, acceptable at Stability 1.0 (Early development). Signed-off-by: atlowChemi <chemi@atlow.co.il> PR-URL: #63054 Reviewed-By: Moshe Atlow <moshe@atlow.co.il> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
1 parent 575260d commit e3fda69

11 files changed

Lines changed: 920 additions & 97 deletions

β€Ždoc/api/cli.mdβ€Ž

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1480,22 +1480,28 @@ Enable module mocking in the test runner.
14801480

14811481
This feature requires `--allow-worker` if used with the [Permission Model][].
14821482

1483-
### `--experimental-test-tag-filter=<tag>`
1483+
### `--experimental-test-tag-filter='<expr>'`
14841484

14851485
<!-- YAML
14861486
added: v26.2.0
14871487
-->
14881488

14891489
> Stability: 1.0 - Early development
14901490
1491-
Run only tests whose tag set contains `<tag>`. Tests declare tags via the
1492-
`tags` option on `test()`, `it()`, `suite()`, or `describe()`; tags
1493-
inherit from suites to nested tests by union. Filtering is
1494-
case-insensitive.
1491+
Run only tests that match the provided boolean tag-filter expression. Tests
1492+
declare tags via the `tags` option on `test()`, `it()`, `suite()`, or
1493+
`describe()`. Tags inherit from suites to nested tests by union.
14951494

1496-
The flag may be specified more than once; tests must contain **every**
1497-
filter value to run. See [Test tags][] for details on declaring and
1498-
inheriting tags.
1495+
The expression supports boolean operators (`and`/`&&`, `or`/`||`,
1496+
`not`/`!`), parentheses for grouping, and `*` wildcards inside identifiers.
1497+
Standard precedence applies: `not` binds tighter than `and`, which binds
1498+
tighter than `or`. See [Test tags][] for the full grammar and behavior.
1499+
1500+
The flag may be specified more than once; multiple expressions are combined
1501+
with AND, so a test must satisfy every expression to run.
1502+
1503+
A malformed expression causes the test runner to exit with a non-zero status
1504+
before running any tests.
14991505

15001506
### `--experimental-vfs`
15011507

β€Ždoc/api/test.mdβ€Ž

Lines changed: 76 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -489,8 +489,8 @@ added: v26.2.0
489489
490490
Tags annotate tests and suites with arbitrary string labels. The
491491
[`--experimental-test-tag-filter`][] CLI flag (or the `testTagFilters`
492-
option on [`run()`][]) selects tests whose tag set contains every
493-
provided filter value.
492+
option on [`run()`][]) selects tests by a boolean expression over those
493+
labels.
494494

495495
Tags are an alternative to encoding metadata into test names. They are
496496
useful for cross-cutting axes such as subsystem, speed bucket, flakiness,
@@ -523,37 +523,89 @@ describe('database', { tags: ['db'] }, () => {
523523
});
524524
```
525525

526-
Tag values must be non-empty strings. Tags are matched case-insensitively;
527-
the canonical form is lowercase. Duplicates within a single `tags` array
528-
are collapsed on the lowercased form, preserving the first-seen
529-
declaration order.
526+
Tag values must be non-empty strings that contain no whitespace, no
527+
operator characters (`& | ! ( ) *`), and are not the reserved words
528+
`'and'`, `'or'`, or `'not'` in any casing. Tags are matched
529+
case-insensitively; the canonical form is lowercase. Duplicates within a
530+
single `tags` array are collapsed on the lowercased form, preserving the
531+
first-seen declaration order.
530532

531533
Hooks (`before`, `after`, `beforeEach`, `afterEach`) do not declare their
532534
own tags. They run as part of their owning suite, which carries the
533535
suite's tags.
534536

535-
### Filtering by tag
537+
### Filtering syntax
536538

537-
Each [`--experimental-test-tag-filter`][] value is a literal tag name. A
538-
test runs only when its tag set contains that name. The flag may be
539-
specified more than once; tests must match **every** filter to run. The
540-
same applies to the `testTagFilters` array on [`run()`][]. Filters are
541-
case-insensitive and AND'd with [`--test-name-pattern`][],
542-
[`--test-skip-pattern`][], and `.only` filtering.
539+
The filter expression supports:
543540

544-
Untagged tests are excluded under any non-empty filter, since the filter
545-
requires the tag to be present.
541+
* Identifiersβ€”any non-whitespace, non-operator characters. A literal
542+
identifier matches a tag of the same value (case-insensitive).
543+
*`*` wildcards inside an identifier match any sequence of characters.
544+
A bare `*` matches any tagged test.
545+
* Boolean operators with two equivalent forms:
546+
*`and` / `&&`
547+
*`or` / `||`
548+
*`not` / `!`
549+
* Parentheses for grouping.
546550

547-
### Reading tags from inside a test
551+
The word forms (`and`, `or`, `not`) require whitespace separation; the
552+
punctuation forms do not.
553+
554+
#### Operator precedence
555+
556+
The expression is evaluated with the standard precedence
557+
`not > and > or`. Binary operators are left-associative.
558+
559+
| Expression | Equivalent grouping |
560+
| -------------- | ------------------- |
561+
|`a or b and c`|`a or (b and c)`|
562+
|`not a and b`|`(not a) and b`|
563+
564+
Use parentheses to override:
565+
566+
| Expression | Selects |
567+
| ------------------------------ | ------------------------------------------ |
568+
|`(unit or smoke) and not slow`| unit-or-smoke tests that are not also slow |
569+
|`db && !flaky`| db tests that are not flaky |
570+
|`*`| every tagged test |
571+
572+
#### Untagged tests
573+
574+
Untagged tests behave as if they have an empty tag set. As a result:
575+
576+
| Filter expression | Untagged test | Why |
577+
| ------------------------ | ------------- | ------------------------------------------------ |
578+
|`db`| excluded | Positive match against an empty tag set is false |
579+
|`*`| excluded | The bare wildcard requires at least one tag |
580+
|`db or unit`| excluded | Both branches are false against an empty tag set |
581+
|`not flaky`| included | Negation against an empty tag set is true |
582+
|`not flaky and not slow`| included | Both negations are true against an empty tag set |
583+
|`db or not flaky`| included | The negated branch is true |
584+
585+
For example, `--experimental-test-tag-filter='not flaky'` runs every test
586+
that is not tagged `flaky`, including all untagged tests.
587+
588+
#### Composing multiple filters
589+
590+
[`--experimental-test-tag-filter`][] may be specified more than once on the
591+
command line. Multiple expressions compose by ANDβ€”a test must satisfy
592+
every expression to run. The same applies to passing an array to
593+
`testTagFilters` on [`run()`][]. The tag filter is also AND'd with
594+
[`--test-name-pattern`][], [`--test-skip-pattern`][], and `.only`
595+
filtering.
596+
597+
#### Reading tags from inside a test
548598

549599
The [`TestContext`][] object exposes the test's tags as a frozen array
550600
through [`context.tags`][], so tests can branch on their own metadata.
551601

552-
### Errors
602+
####Errors
553603

554604
A tag value that violates the validation rules above throws
555605
`ERR_INVALID_ARG_VALUE` at the registration site, before any test runs.
556-
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`.
606+
A non-array `tags` value throws `ERR_INVALID_ARG_TYPE`. A malformed
607+
filter expression on the CLI causes the test runner to exit with a
608+
non-zero status before running any test files.
557609

558610
## Extraneous asynchronous activity
559611

@@ -826,7 +878,7 @@ test runner functionality:
826878

827879
*`--test` - Prevented to avoid recursive test execution
828880
*`--experimental-test-coverage` - Managed by the test runner
829-
*`--experimental-test-tag-filter` - Filter values are validated by the parent
881+
*`--experimental-test-tag-filter` - Filter expressions are validated by the parent
830882
process and re-emitted to child processes
831883
*`--watch` - Watch mode is handled at the parent level
832884
*`--experimental-default-config-file` - Config file loading is handled by the parent
@@ -1740,10 +1792,11 @@ changes:
17401792
For each test that is executed, any corresponding test hooks, such as
17411793
`beforeEach()`, are also run.
17421794
**Default:**`undefined`.
1743-
*`testTagFilters` {string|string\[]} A tag name, or an array of tag names,
1744-
used to filter tests by their declared tags. Tests must contain every
1745-
listed tag to run. Equivalent to passing [`--experimental-test-tag-filter`][]
1746-
on the command line. See [Test tags][]. **Default:**`undefined`.
1795+
*`testTagFilters` {string|string\[]} A boolean expression, or an array of
1796+
boolean expressions, used to filter tests by their declared tags.
1797+
Multiple expressions compose by AND. Equivalent to passing
1798+
[`--experimental-test-tag-filter`][] on the command line. See
1799+
[Test tags][]. **Default:**`undefined`.
17471800
*`timeout` {number} A number of milliseconds the test execution will
17481801
fail after.
17491802
If unspecified, subtests inherit this value from their parent.

β€Ždoc/node.1β€Ž

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -810,14 +810,18 @@ collecting code coverage from tests for more details.
810810
Enable module mocking in the test runner.
811811
This feature requires \fB--allow-worker\fR if used with the Permission Model.
812812
.
813-
.ItFl-experimental-test-tag-filterNs=NsAr<tag>
814-
Run only tests whose tag set contains \fB<tag>\fR. Tests declare tags via the
815-
\fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or \fBdescribe()\fR; tags
816-
inherit from suites to nested tests by union. Filtering is
817-
case-insensitive.
818-
The flag may be specified more than once; tests must contain \fBevery\fR
819-
filter value to run. See Test tags for details on declaring and
820-
inheriting tags.
813+
.ItFl-experimental-test-tag-filterNs=NsAr'<expr>'
814+
Run only tests that match the provided boolean tag-filter expression. Tests
815+
declare tags via the \fBtags\fR option on \fBtest()\fR, \fBit()\fR, \fBsuite()\fR, or
816+
\fBdescribe()\fR. Tags inherit from suites to nested tests by union.
817+
The expression supports boolean operators (\fBand\fR/\fB&&\fR, \fBor\fR/\fB||\fR,
818+
\fBnot\fR/\fB!\fR), parentheses for grouping, and \fB*\fR wildcards inside identifiers.
819+
Standard precedence applies: \fBnot\fR binds tighter than \fBand\fR, which binds
820+
tighter than \fBor\fR. See Test tags for the full grammar and behavior.
821+
The flag may be specified more than once; multiple expressions are combined
822+
with AND, so a test must satisfy every expression to run.
823+
A malformed expression causes the test runner to exit with a non-zero status
824+
before running any tests.
821825
.
822826
.ItFl-experimental-vfs
823827
Enable the experimental \fBnode:vfs\fR module.

β€Žlib/internal/test_runner/runner.jsβ€Ž

Lines changed: 27 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const {
1212
ArrayPrototypePush,
1313
ArrayPrototypePushApply,
1414
ArrayPrototypeShift,
15+
ArrayPrototypeSlice,
1516
ArrayPrototypeSome,
1617
ArrayPrototypeSort,
1718
MathMax,
@@ -93,7 +94,7 @@ const {
9394
parseCommandLine,
9495
}=require('internal/test_runner/utils');
9596
const{
96-
validateAndCanonicalizeTagFilter,
97+
parseTagFilterExpression,
9798
}=require('internal/test_runner/tag_filter');
9899
const{ Glob }=require('internal/fs/glob');
99100
const{ once }=require('events');
@@ -182,7 +183,7 @@ function getRunArgs(path, { forceExit,
182183
inspectPort,
183184
testNamePatterns,
184185
testSkipPatterns,
185-
testTagFilters,
186+
testTagFilterExpressions,
186187
only,
187188
hasFiles,
188189
testFiles,
@@ -224,8 +225,8 @@ function getRunArgs(path, { forceExit,
224225
if(testSkipPatterns!=null){
225226
ArrayPrototypeForEach(testSkipPatterns,(pattern)=>ArrayPrototypePush(runArgs,`--test-skip-pattern=${pattern}`));
226227
}
227-
if(testTagFilters!=null){
228-
ArrayPrototypeForEach(testTagFilters,(value)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${value}`));
228+
if(testTagFilterExpressions!=null){
229+
ArrayPrototypeForEach(testTagFilterExpressions,(expr)=>ArrayPrototypePush(runArgs,`--experimental-test-tag-filter=${expr}`));
229230
}
230231
if(only===true){
231232
ArrayPrototypePush(runArgs,'--test-only');
@@ -872,19 +873,37 @@ function run(options = kEmptyObject) {
872873
});
873874
}
874875

876+
// The public contract of testTagFilters is `string | string[]`. The
877+
// parseCommandLine bootstrap path piggybacks the already-parsed AST array
878+
// on the same field, identifiable by the sibling testTagFilterExpressions
879+
// field which only that path sets. When that marker is present and the
880+
// first element isn't a string, treat the array as ASTs and skip the
881+
// public validation loop. Otherwise validate every element as a string,
882+
// so any non-string input throws ERR_INVALID_ARG_TYPE with the offending
883+
// index regardless of position.
884+
lettestTagFilterExpressions=null;
875885
if(testTagFilters!=null){
876886
if(!ArrayIsArray(testTagFilters)){
877887
testTagFilters=[testTagFilters];
878888
}
879889
if(testTagFilters.length===0){
880890
testTagFilters=null;
891+
}elseif(options.testTagFilterExpressions!=null&&
892+
typeoftestTagFilters[0]!=='string'){
893+
// Internal bootstrap: trust the AST array as already-parsed.
881894
}else{
882895
emitExperimentalWarning('Test tags');
883-
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>(
884-
validateAndCanonicalizeTagFilter(value,`options.testTagFilters[${i}]`)
885-
));
896+
testTagFilterExpressions=ArrayPrototypeSlice(testTagFilters);
897+
testTagFilters=ArrayPrototypeMap(testTagFilters,(value,i)=>{
898+
constname=`options.testTagFilters[${i}]`;
899+
if(typeofvalue!=='string'){
900+
thrownewERR_INVALID_ARG_TYPE(name,'string',value);
901+
}
902+
returnparseTagFilterExpression(value,name);
903+
});
886904
}
887905
}
906+
testTagFilterExpressions??=options.testTagFilterExpressions;
888907

889908
validateOneOf(isolation,'options.isolation',['process','none']);
890909
validateBoolean(coverage,'options.coverage');
@@ -986,7 +1005,7 @@ function run(options = kEmptyObject) {
9861005
inspectPort,
9871006
testNamePatterns,
9881007
testSkipPatterns,
989-
testTagFilters,
1008+
testTagFilterExpressions,
9901009
hasFiles: files!=null,
9911010
globPatterns,
9921011
only,

0 commit comments

Comments
Β (0)