fix(driver-memory): render the pipeline dump's RegExp operand instead of dropping it to {} (#7853)

.changeset/analytics-pipeline-dump-regexp.md

@@ -0,0 +1,44 @@
+---
+"@objectstack/driver-memory": patch
+---
+
+fix(driver-memory): the analytics pipeline dump shows its RegExp pattern instead of `{}` (#7853)
+
+`MemoryAnalyticsService.query()` returns `AnalyticsResult.sql` — a stage-by-stage
+dump of the mingo pipeline it actually executed, and the only thing an author
+debugging an in-memory chart is given. It dumped each stage with a bare
+`JSON.stringify`, and a `RegExp` has **no own enumerable properties**, so every
+pattern operand rendered as `{}`:
+
+```
+-- MongoDB Aggregation Pipeline on table: deal
+/* Stage 1: $match */ {"name":{"$regex":{}}}
+```
+
+The `$match` stage was reported as constraining `name` by an empty object. The
+one field the reader came for is the one the dump dropped. Measured across the
+twelve operators this face declares, exactly three carry a pattern and all three
+were affected: `$contains`, `$icontains`, and `$notContains` (nested inside
+`$not`). The same three now render:
+
+```
+/* Stage 1: $match */ {"name":{"$regex":"/et/"}}
+/* Stage 1: $match */ {"name":{"$regex":"/[Bb][Ee][Tt]/"}}
+/* Stage 1: $match */ {"name":{"$not":{"$regex":"/et/"}}}
+```
+
+**No executed behaviour changes.** This dump is explicitly not SQL — its own
+header says `-- MongoDB Aggregation Pipeline on table: …` — so it is a
+transparency surface, not a runnable one, and the rows `query()` returns and the
+SQL `generateSql()` emits are byte-identical before and after. The other nine
+operators' dumps are unchanged.
+
+**Why the pattern's own literal syntax** (`/source/flags`) and not the
+mongo-shaped `{"$regex":"…","$options":"…"}` the rest of the dump speaks: the
+`RegExp` sits AT the `$regex` key, so a value replacer producing the mongo pair
+renders the doubled `{"$regex":{"$regex":"et","$options":""}}` — a shape no mongo
+query has. Flattening it to the real spelling would mean rewriting the parent
+object, making the dump disagree with the pipeline it claims to dump, since what
+mingo executes at that key is a JS `RegExp`. The literal form is also the only
+one-token rendering that keeps the FLAGS, which matter here: `$icontains`' fold
+lives in the pattern source (#6520) while `$contains` is case-exact (#7723).

packages/drivers/driver-memory/src/memory-analytics.ts

@@ -480,6 +480,62 @@ function sizeDistinctSet(values: readonly unknown[]): number {
  return new Set(values.filter((v) => v !== null && v !== undefined)).size;
 }
 
+/**
+ * [#7853] A `JSON.stringify` replacer that renders a `RegExp` operand instead of
+ * dropping it — the one value type the pipeline dump carries that
+ * `JSON.stringify` erases.
+ *
+ * ## What was lost
+ *
+ * A `RegExp` has no own enumerable properties, so `JSON.stringify` renders it as
+ * `{}`. Three operators put one into the `$match` stage — `contains` and
+ * `icontains` as `{$regex: …}`, `notContains` as `{$not: {$regex: …}}` (measured:
+ * those three and no others, out of the twelve this face declares) — so
+ * `{name: {$contains: 'Industries'}}` dumped as
+ *
+ * ```
+ * /* Stage 1: $match *\/ {"name":{"$regex":{}}}
+ * ```
+ *
+ * The one field an author debugging a chart is looking for is the one the dump
+ * dropped. This is lost information on a transparency surface, not the #5333
+ * class: the dump is explicitly NOT SQL (its header says so) and `{}` reads as
+ * "something is missing here" rather than as a working predicate, which is why
+ * it is graded below #7117 rather than beside it.
+ *
+ * ## Why the pattern's own literal syntax and not `{"$regex":"…","$options":"…"}`
+ *
+ * The mongo-shaped form is what the rest of the dump speaks, and it was the
+ * first candidate. It cannot be reached from a value replacer, and the reason is
+ * structural rather than cosmetic: the `RegExp` sits AT the `$regex` key, so
+ * replacing it with `{$regex, $options}` renders the doubled
+ * `{"name":{"$regex":{"$regex":"Industries","$options":""}}}` — a shape no mongo
+ * query has. Flattening it into the real mongo spelling means rewriting the
+ * PARENT object, which would make the dump disagree with the pipeline it claims
+ * to be dumping: what mingo executes is a JS `RegExp` object at that key, not a
+ * source/options pair. Trading a degenerate rendering for a plausible-but-wrong
+ * one is the #5333 direction, and this card is explicitly not that.
+ *
+ * So the value is rendered as the JS literal it is, `/source/flags`, which is
+ * also the only one-token form that keeps the FLAGS. Flags are not decoration
+ * here: `$icontains`' fold lives in the pattern SOURCE (#6520) while `$contains`
+ * is case-EXACT (#7723, #4706 Q2 = A), so a rendering that dropped `i` would
+ * recreate a smaller copy of this same information loss on the one axis those
+ * two operators differ.
+ *
+ * ## What it deliberately does not touch
+ *
+ * Every other value on this path already renders faithfully, measured rather
+ * than assumed: a `Date` comparand is canonicalized to an ISO string by
+ * {@link MemoryAnalyticsService.comparandsFor} before it reaches here, and
+ * `toJSON` runs BEFORE a replacer in any case, so dates are unchanged. A
+ * `BigInt` comparand does throw — but out of mingo's own `Query.compile` during
+ * EXECUTION, before this dump is ever built, so no replacer here reaches it.
+ */
+function pipelineDumpReplacer(_key: string, value: unknown): unknown {
+ return value instanceof RegExp ? `/${value.source}/${value.flags}` : value;
+}
+
 /**
  * Configuration for MemoryAnalyticsService
  */
@@ -1280,9 +1336,13 @@ export class MemoryAnalyticsService implements IAnalyticsService {
  private generateSqlFromPipeline(table: string, pipeline: Record<string, any>[]): string {
  // Simplified SQL generation for debugging
  // This is a basic representation of the aggregation pipeline
+ //
+ // [#7853] The replacer is what keeps a `RegExp` operand from rendering as
+ // `{}` — see {@link pipelineDumpReplacer} for why the pattern's own literal
+ // syntax and not the mongo-shaped `{$regex, $options}`.
  const stages = pipeline.map((stage, idx) => {
  const op = Object.keys(stage)[0];
- return `/* Stage ${idx + 1}: ${op} */ ${JSON.stringify(stage[op])}`;
+ return `/* Stage ${idx + 1}: ${op} */ ${JSON.stringify(stage[op], pipelineDumpReplacer)}`;
  }).join('\n');
 
  return `-- MongoDB Aggregation Pipeline on table: ${table}\n${stages}`;

packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts

@@ -696,17 +696,63 @@ describe('[#5374] operator semantics — the analytics face against the live que
  * absent and the rows happen to line up", and the emitted `$match` is the
  * artifact the issue actually diagnosed.
  */
-it('the emitted $match wraps the negation around a pattern instead of a bare scalar', async () => {
+const matchStageOf = async (where: FilterCondition): Promise<string> => {
  const { sql } = await service.query({
  cube: COMPARAND_TABLE,
  measures: [`${COMPARAND_TABLE}.count`],
- where: { name: { $notContains: 'et' } } as FilterCondition,
+ where,
  });
- const matchStage = /\/\* Stage 1: \$match \*\/ (.*)/.exec(sql ?? '')?.[1] ?? '';
- // `JSON.stringify` renders a RegExp as `{}`, so assert on the STRUCTURE the
- // pipeline carries rather than on that rendering.
+ return /\/\* Stage 1: \$match \*\/ (.*)/.exec(sql ?? '')?.[1] ?? '';
+ };
+
+ it('the emitted $match wraps the negation around a pattern instead of a bare scalar', async () => {
+ const matchStage = await matchStageOf({ name: { $notContains: 'et' } } as FilterCondition);
  expect(matchStage).not.toBe('{"name":{"$not":"et"}}');
  expect(matchStage).toContain('"$not"');
  expect(matchStage).toContain('"$regex"');
  });
+
+ /**
+ * [#7853] The dump's CONTENT for the pattern operators, not merely its shape.
+ *
+ * The assertion above pins the structure `$not` wraps, and it passed for as
+ * long as the pattern itself was missing: `JSON.stringify` renders a `RegExp`
+ * as `{}` (no own enumerable properties), so `{name: {$contains: 'et'}}`
+ * dumped as `{"name":{"$regex":{}}}` — non-empty, structurally correct, and
+ * silent about the one field an author reading this dump came for. That is
+ * why the cases below assert the pattern TEXT: a `toBeDefined()` or a
+ * `toContain('$regex')` cannot tell the two states apart.
+ *
+ * The rendering is the pattern's own literal syntax, `/source/flags`. No
+ * operator this face declares carries a RegExp FLAG today — measured:
+ * `$icontains`' fold is compiled into the pattern source (#6520) and
+ * `$contains` is case-exact (#7723) — so the flags segment renders empty
+ * here; it is in the form because a fold that ever moved into a flag would
+ * otherwise vanish exactly the way the source did.
+ */
+ const PATTERN_DUMP_CASES: Array<[string, FilterCondition, string]> = [
+ // The pattern, plainly, for the operator the issue measured.
+ ['$contains', { name: { $contains: 'et' } } as FilterCondition, '{"name":{"$regex":"/et/"}}'],
+ // The ASCII fold is IN the source, so the dump shows the folded character
+ // classes rather than an `i` — this is what #6520 compiled, made visible.
+ ['$icontains', { name: { $icontains: 'BET' } } as FilterCondition, '{"name":{"$regex":"/[Bb][Ee][Tt]/"}}'],
+ // The negation still wraps a pattern, and now the pattern is legible.
+ ['$notContains', { name: { $notContains: 'et' } } as FilterCondition, '{"name":{"$not":{"$regex":"/et/"}}}'],
+ // A comparand carrying a regex metacharacter shows its ESCAPE. `a.p` is a
+ // literal here, not "any character between a and p" (#5567's direction), and
+ // the dump is the only place an author can see which of the two ran.
+ ['$contains with a metacharacter', { name: { $contains: 'a.p' } } as FilterCondition, '{"name":{"$regex":"/a\\\\.p/"}}'],
+ ];
+
+ for (const [label, where, expected] of PATTERN_DUMP_CASES) {
+ it(`the pipeline dump renders ${label}'s pattern instead of dropping it to {}`, async () => {
+ const matchStage = await matchStageOf(where);
+ expect(
+ matchStage,
+ `${label}: the dump lost its RegExp operand — this is the \`{"$regex":{}}\` state, ` +
+ 'which is non-empty and passes every assertion that only checks for presence',
+ ).not.toContain('{}');
+ expect(matchStage).toBe(expected);
+ });
+ }
 });