Skip to content

fix: align linear token sequence semantics - #113

Open
jbachorik wants to merge 1 commit into
agent/logs-reggie-l1bfrom
agent/logs-reggie-l1c
Open

fix: align linear token sequence semantics#113
jbachorik wants to merge 1 commit into
agent/logs-reggie-l1bfrom
agent/logs-reggie-l1c

Conversation

@jbachorik

Copy link
Copy Markdown
Collaborator

Summary

  • aligns Linear Token Sequence \s and \S handling with default JDK whitespace semantics
  • makes the bracket-tail shortcut require the literal-space delimiters it recognizes
  • restricts wildcard LTS admission to globally DOTALL patterns, preserving default-dot semantics
  • adds JDK differential and adversarial regression coverage

Validation

  • `./gradlew jacocoVerify build`
  • focused LTS and categorizer tests after final cleanup
  • final implementation-plan validation and code review run independently

Stacked on #112 (which is stacked on #111).

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens and aligns the Linear Token Sequence (LTS) fast-path semantics with the intended Java/JDK regex behavior by (a) matching JDK-style whitespace for \s/\S, (b) making the bracket-tail shortcut enforce the literal-space delimiters it recognizes, and (c) preventing LTS admission of wildcard-dot patterns unless the pattern is globally DOTALL.

Changes:

  • Align LTS whitespace/non-whitespace classification with JDK-style whitespace (and add JDK-differential regression coverage).
  • Require literal spaces around the bracket-tail shortcut and add adversarial coverage.
  • Refuse LTS routing for default-dot wildcard patterns (only allow wildcard LTS when globally DOTALL).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
FileDescription
reggie-runtime/src/test/java/com/datadoghq/reggie/runtime/LinearTokenSequenceMatcherTest.javaAdds JDK-differential tests for whitespace semantics, bracket-tail delimiter strictness, and DOTALL/default-dot routing.
reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/RuntimeCompiler.javaTightens LTS admission rules around wildcard-dot usage and DOTALL requirements.
reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/LinearTokenSequenceMatcher.javaUpdates LTS whitespace handling and enforces literal-space delimiters for the bracket-tail shortcut; adds SKIP_ANY_EXCEPT_NEWLINE execution path.
reggie-codegen/src/test/java/com/datadoghq/reggie/codegen/analysis/PatternCategorizerTest.javaUpdates categorizer fixture to explicitly be DOTALL for bracket-tail shortcut eligibility.
reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/PatternCategorizer.javaSplits dotall vs default-dot wildcard categorization; requires DOTALL .* for bracket-tail shortcut detection.
reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/PatternAtom.javaIntroduces ANY_STAR_EXCEPT_NEWLINE atom kind.
reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/LinearTokenSequencePlan.javaAdds SKIP_ANY_EXCEPT_NEWLINE op kind and mapping from the new atom kind.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jbachorik
jbachorik marked this pull request as ready for review August 14, 2026 12:53

@chatgpt-codex-connectorchatgpt-codex-connectorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit:a87ffa492c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@jbachorik
jbachorikforce-pushed the agent/logs-reggie-l1c branch from a87ffa4 to 47e531bCompareAugust 14, 2026 13:05
@codecov-commenter

codecov-commenter commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.88889% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.4%. Comparing base (44e3604) to head (81f6dc5).

Files with missing linesPatch %Lines
...hq/reggie/codegen/analysis/PatternCategorizer.java50.0%0 Missing and 5 partials ⚠️
.../com/datadoghq/reggie/runtime/RuntimeCompiler.java50.0%1 Missing and 4 partials ⚠️
...ghq/reggie/runtime/LinearTokenSequenceMatcher.java81.8%0 Missing and 4 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## agent/logs-reggie-l1b #113 +/- ##
=========================================================
- Coverage 84.4% 84.4% -0.1% 
Complexity 1 1 =========================================================
Files 147 147 Lines 42991 42993 +2 Branches 5859 5860 +1 =========================================================
Hits 36296 36296 - Misses 4906 4911 +5 + Partials 1789 1786 -3 
Files with missing linesCoverage Δ
...ggie/codegen/analysis/LinearTokenSequencePlan.java82.0% <100.0%> (-0.6%)⬇️
...datadoghq/reggie/codegen/analysis/PatternAtom.java87.5% <100.0%> (+0.4%)⬆️
...ghq/reggie/runtime/LinearTokenSequenceMatcher.java80.6% <81.8%> (+0.6%)⬆️
...hq/reggie/codegen/analysis/PatternCategorizer.java59.3% <50.0%> (-0.1%)⬇️
.../com/datadoghq/reggie/runtime/RuntimeCompiler.java87.9% <50.0%> (-0.4%)⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44e3604...81f6dc5. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jbachorik
jbachorikforce-pushed the agent/logs-reggie-l1c branch from 47e531b to 81f6dc5CompareAugust 14, 2026 13:24
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jbachorik@codecov-commenter