Skip to content

feat(matrix): state the vocabulary, and let two games have two clocks - #44

Merged
drewstone merged 1 commit into
mainfrom
fix/state-the-vocabulary
Aug 29, 2026
Merged

feat(matrix): state the vocabulary, and let two games have two clocks#44
drewstone merged 1 commit into
mainfrom
fix/state-the-vocabulary

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Three faults, all found by running the first two-game study this repo has completed.

A policy was never told what words the game takes

The streaming transport states them in brief.json, exactly so an agent does not have to guess. The persistent and per-decision transports sent frame, history and context and never said.

Measured: a control that cycles 2048's four words scored 1948 there and 0 on ALE Breakout across three replicates, with distinctInputs=1 and no lives lost — Breakout takes NOOP FIRE RIGHT LEFT and none of the four it emitted was legal. Every word was substituted, so it read as a player that did nothing rather than one that was never told the rules.

Both transports now carry commands in the request, and the bundled control reads them instead of assuming. With the words stated it launches the ball and loses five lives in 126 decisions.

assertJoinable refused the study it exists to serve

It rejected "2 protocols" across the whole pool. Two games legitimately have two clocks: a packaged reference pins the clock it was recorded at, so Breakout is derivable only at frameskip 4 while native-2048 runs at 1.

A transfer statistic ranks profiles inside each game and correlates the orders, so a raw score never crosses a game boundary. The check is now scoped per game. Two clocks inside one game is still refused — those rows get averaged and the dominant term disappears into the mean.

The bundled controls were unreachable

They lived under runs/, which is gitignored, so any definition naming them was broken for everyone but its author. They are in examples/policies/ now, alongside the two halves of the study that found all of this.

Result

12 cells, 0 blocked, and the first cross-game statistic this repository has produced:

transfer: tau=0.000 over 2 games (1 pairs), 2 of 2 arms distinct
— one game pair: this tau is a single observation, not evidence that an order transfers

The machinery works end to end and the study is uninformative, which are different things. Both controls score 0 on Breakout, so there is no order to correlate; the runner says so rather than presenting the zero as a finding. Separating profiles there needs agents, not cheap controls — neither control can hit a ball.

Local: 20 of 20 test files, boundary check, 0 typecheck errors.

Three faults, all found by running the first two-game study this repo has
completed.
A POLICY WAS NEVER TOLD WHAT WORDS THE GAME TAKES. The streaming transport
states them in `brief.json`, exactly so an agent does not have to guess. The
persistent and per-decision transports sent frame, history and context and
never said. Measured: a control that cycles 2048's four words scored 1948
there and 0 on ALE Breakout across three replicates, with `distinctInputs=1`
and no lives lost, because Breakout takes NOOP, FIRE, RIGHT and LEFT and none
of the four it emitted was legal. It read as a player that did nothing rather
than one that was never told the rules. Both transports now carry `commands`
in the request, and the bundled control reads them instead of assuming. With
the words stated it launches the ball and loses five lives in 126 decisions.
`assertJoinable` REFUSED THE STUDY IT EXISTS TO SERVE. It rejected two
protocols across the whole pool, and two games legitimately have two clocks: a
packaged reference pins the clock it was recorded at, so Breakout is derivable
only at frameskip 4 while native-2048 runs at 1. A transfer statistic ranks
profiles inside each game and correlates the orders, so a raw score never
crosses a game boundary. The check is now scoped per game. Two clocks inside
ONE game is still refused, because those rows are averaged and the dominant
term disappears into the mean.
The bundled controls lived under `runs/`, which is gitignored, so a definition
naming them was broken for everyone but its author. They are in `examples/`
now, with the two halves of the study that found all of this.
Measured end to end: 12 cells, 0 blocked, and the first cross-game statistic
this repository has produced. It reports `tau=0.000` and says why that is not
a finding: both controls score 0 on Breakout, so there is no order to
correlate, and one game pair is a single observation.
@drewstone
drewstone merged commit 5f97687 into mainAug 29, 2026
6 checks passed
@drewstone
drewstone deleted the fix/state-the-vocabulary branch August 29, 2026 08:45
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@drewstone