Skip to content

Fix #180 #181: unjudgeable checkers for problems 11, 22 and 210 - #184

Merged
joyemang33 merged 2 commits into
FrontierCS:mainfrom
xiawubing:fix/checkers-11-210
Sep 12, 2026
Merged

Fix #180 #181: unjudgeable checkers for problems 11, 22 and 210#184
joyemang33 merged 2 commits into
FrontierCS:mainfrom
xiawubing:fix/checkers-11-210

Conversation

@xiawubing

@xiawubing xiawubing commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Three algorithmic checkers score every submission 0 (or cap it) for reasons unrelated to the
submission. Fixes #180 (problem 11, and the identical bug in problem 22 found in review) and
#181 (problem 210); details and reproduction are in the issues. Problem 145 (#182) is tracked
separately because its fix is a different kind of change.

  • 11/chk.cc: enlarge the 30-byte message buffer (the formatted verdict is 41 characters, so
    every correct answer aborted in the fortified sprintf).
  • 22/checker.cpp: the same buffer and the same sprintf, same one-line fix. A search of
    every checker and interactor under algorithmic/problems finds no other instance.
  • 210/chk.cc: loosen the per-base g/c/d/v read bounds; the shipped 4.in-9.in
    carry v up to 250 and the checker FAILed on its own input for 6/10 cases. The observed
    ranges are added to the statement's "Constraints (from released datasets)".

Type of Change

  • Bug fix

Testing

Compiled with the judge's flags (g++ -std=gnu++17 -O2 -pipe -I algorithmic/judge/include).

Regression corpus: the exact per-case outputs the judge recorded for 32 archived agent runs
(16 per problem, two harness arms × 8), re-judged with the original and the patched checker.
Feeding the recorded outputs to the original checker reproduces the official per-case ratio
on every checker-judged case (11: 150/150, 210: 160/160), so the comparison is apples to apples.

  • 11: 52 cases that aborted with buffer overflow detected now score (all 1.000 — no archived
    path exceeded the length bound); the 45 -1 cases and the 53 wrong-answer cases are
    unchanged. Run rewards move from a uniform 0.300 to 0.3–1.0 (10 of 15 runs change). The ten
    shipped .ans files also score 1.0 / ok, a non-palindrome scores 0, a path 30× over the
    bound scores 0.
  • 22: the shipped .ans files are empty (the checker does not read them), so the corpus is
    the 40 judge-recorded outputs of 4 archived agent runs from another campaign, all officially
    *** buffer overflow detected ***: the original checker aborts on 40/40, the fixed one
    returns a verdict on 40/40 (all 1.0; their K is N or ≈ 3N), clean under ASan/UBSan. Unrelated
    to the crash, the score formula on line 120 of the same file has an operator-precedence issue
    (/ 2 * N) that removes the partial-credit gradient for 3N < K < 5N; it does not affect these
    outputs and, since it changes scoring, is left out of this PR.
  • 210: the 96 cases that FAILed on input (cases 4–9 × 16 runs) are now judged; the 64 other
    cases are unchanged. Cases 7 and 9 stay 0 for every run because those programs printed
    nothing there. Run rewards move from 0.000–0.187 to 0.000–0.351.

All three patched checkers run clean under AddressSanitizer + UBSan over the whole corpus plus the
shipped .ans files. For 11 the scoring formula was exercised on a synthetic 1×3 grid: length
below / at / between 1× and 2× / above 2× the bound gives 1.0 / 1.0 / 0.870 / 0.0, -1 on a
solvable instance gives 0, and an empty output on a 1×1 grid gives 1.0 as the statement
requires.

No testdata or .ans changes. The validate-algorithmic CI job will report "Missing reference
solution" as it did for #174; these problems ship no reference.cpp.

Checklist

  • Code follows the project structure and conventions
  • Self-review completed
  • Documentation updated (210 statement constraints)

🤖 Generated with Claude Code

https://claude.ai/code/session_01AUL12r3HsmHBUTDGZxXFQr

xiawubing and others added 2 commits September 9, 2026 22:50
- 11/chk.cc: the verdict was sprintf'd into a char[30] buffer although the
  formatted message is 41 characters, so every correct answer aborted with
  "buffer overflow detected" and only the three "-1" cases could score.
  Enlarge the buffer. Fixes FrontierCS#180.
- 210/chk.cc: the per-base g/c/d/v read bounds were tighter than the
  shipped testdata (v up to 250 against readLong(0, 200)), so cases 4-9
  FAILed on the input file before any contestant output was read. Loosen
  the bounds and document the observed ranges in the statement. Fixes FrontierCS#181.

Re-judging the judge-recorded outputs of 32 archived agent runs (16 per
problem) with the original checkers reproduces the official per-case
scores exactly; with the fixed checkers, 52 previously crashing cases on
11 and 96 previously FAILing cases on 210 are judged, and no case that
was judged before changes its score.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUL12r3HsmHBUTDGZxXFQr
algorithmic/problems/22/checker.cpp formats its verdict into char mes[30]
with the same 41-character sprintf as problem 11, so every valid answer
aborted with "buffer overflow detected". Enlarge the buffer.

Reported by a reviewer on FrontierCS#184. A repository-wide search of every checker
and interactor finds no other instance of this pattern.

Re-judging the 40 judge-recorded outputs of 4 archived agent runs: the
original checker aborts on all 40, the fixed one returns a verdict on all
40, and the build is clean under ASan/UBSan.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUL12r3HsmHBUTDGZxXFQr
@xiawubing xiawubing changed the title Fix #180 #181: unjudgeable checkers for problems 11 and 210 Fix #180 #181: unjudgeable checkers for problems 11, 22 and 210 Sep 10, 2026
@weiwch weiwch mentioned this pull request Sep 10, 2026
9 tasks
@whuang369

Copy link
Copy Markdown
Contributor

LGTM! Thanks

@joyemang33
joyemang33 merged commit ae69c52 into FrontierCS:main Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem 11 checker: sprintf into a 30-byte buffer aborts on every correct answer

3 participants