feat(p4): packrat memoization ON by default — parity corpus fallback 3→0 - #51
Merged
Conversation
Now that packrat parse memoization (unlaxer-parser #40) is proven parse-equivalent and the mapping phase no longer re-maps subtrees (#49), the only remaining reason the curated/extracted parity corpus fell back to the legacy parser was the 10s parse deadline being hit (memoize was opt-in/off). Three formulas fell back: if((1>0|10<20)&(true)){10}else{0} -> token-ast if(not(toUpperCase('cnjpuszn')[4:6].in(...))){1}else{0} -> deadline -> legacy if(not(not(... same ...))){1}else{0} -> deadline -> legacy With memoize ON all three parse fast and evaluate on the p4-typed path with correct results (10.0 / 0.0 / 1.0). Flip the default to ON (opt out with -Dtinyexpression.p4.memoize=false). Effect: - ThreeExecutionBackendExtractedCorpusParityTest: 0 legacy fallbacks (was 3), wall time 100s -> 19s. - P4AstEvaluatorCalculatorTest (giant nested-if fraud formulas, #38/#20): ~1263s -> ~167s. - Full suite 633 tests green; this also fixes the last baseline failure (TernaryExpressionTest#testTernaryInSinWithDoubleParensStillWorks), so test-baseline.txt is now empty. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景（fallback=0）
#40 packrat parse メモ化が parse-equivalent と実証され、#49 で mapping 段の再マッピングも解消した今、parity corpus が legacy にフォールバックする唯一の残因は 10s parse deadline 超過（memoize が opt-in/OFF だったため）でした。実測でフォールバックしていた3式:
memoize ON で3式とも高速 parse → p4-typed 経路で正答（10.0 / 0.0 / 1.0）。
変更
P4PreferredAstMapperの memoize を既定 ON化（-Dtinyexpression.p4.memoize=falseで opt-out）。効果
ThreeExecutionBackendExtractedCorpusParityTest: legacy フォールバック 3→0、所要 100s→19s。P4AstEvaluatorCalculatorTest（perf: 生成パーサの指数バックトラック — 二重カッコ三項/ソース再パース経路で秒〜分単位の遅延 #38/test: BackendSpeedComparisonTest が legacy パーサー経路 (JavaCodeCalculatorV3 ctor) でハング — #19 の残り #20 巨大ネスト if）:1263s→167s。TernaryExpressionTest#testTernaryInSinWithDoubleParensStillWorksも解消 →test-baseline.txtを空に。🤖 Generated with Claude Code