Uh oh!
There was an error while loading. Please reload this page.
Synchronize latest Public updates - #18
Merged
Merged
Conversation
`ruff check argus_skill tests` is the first step of the test workflow, so main is red until this file's import block is ordered. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Figure 1 advertised seven benchmark scores while the report's central claim had become that campaigns run without a person in the loop, and the two concepts the reframe introduced had no illustration at all. - replace the two nanochat cards in Figure 1 with the metrics the report now leads on: 40.7 hours per human request against roughly one hour for a turn-by-turn agent, and duty cycle against its infrastructure-degraded case - split the bottom card so upstream review by outside maintainers appears on the teaser, since it is the only result decided outside this project - add a figure pairing per-campaign duty cycle with the classification of all 38 interruptions, annotating the two lower bars with why they sit lower - add a figure for the containment relation between auto research and RSI and the loop in which each is a precondition for the other, colour-coded by what is measured here and what remains Figures are hand-authored SVG matching the existing palette and rendered to PDF, consistent with the other diagrams in the report. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The teaser advertised benchmark scores and autonomy metrics, but the work that took the most effort and carries the most external validation appeared only deep in the results. A reader who saw one page saw none of it. Add a delivery band of three domain cards below the runtime diagram, each naming what was built and who judged it: - infrastructure: four fla-org kernels accepted by outside maintainers, the FlashAttention-4 diffusion-LLM adaptation within 5-14% of native for under Y80, the SGLang integration, and a 62 GiB video model on a 24 GB laptop - science: MOF chemical control, structure judgment from 0.594 to 0.833 AUC, a simpler sampler admitted after a confound was removed, and the Erdos campaign keeping a falsified route with its evidence - silicon: ACE-2 functional closure, mapped SKY130 synthesis, and the human cost of that campaign at eight days and six interruptions, two of which were the server failing rather than the runtime asking Existing cards and their text are unchanged; the canvas grows to fit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`argus --web` was undiscoverable. The flag, its host/port options and a doctor check all existed, but the product-facing help never named it, so the only way to learn the web UI exists was to read the source or set a debug environment variable. `argus --web` could not report a broken install. The guard meant to print an actionable message caught only the `webapi.server` import, while that module imports uvicorn lazily inside `serve()` — so the pairing banner offered a URL and a bare ImportError escaped as a traceback. The message it could not print was wrong anyway: it advised installing an `argus-skill[web]` extra, and no such extra exists because fastapi and uvicorn are required dependencies. The web stack is now probed before anything promises a URL. `argus --config-help` hid three supported backends. The set was written out by hand in four places — the `--backend` choices, the `--advisor` choices, the readiness check and the operator knob help — and the knob help had drifted to five of the eight. A user reading the documented control surface could not learn that grok, qoder and dsh exist. All four now render from the `RunnerBackend` type, so the drift is unrepresentable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The runtime diagram still drew the four roles as a pipeline, which was the old framing. It showed neither which role answers which question, nor the channel by which a Reviewer can question the plan that produced the round it is reviewing. - label Planner with Q2 and Reviewer with Q1, and give each role its actual scope: the Reviewer runs read-only and returns one of four verdicts, and the Engineer may self-review low-risk work - replace the generic CONTROL box with the Q3 authority boundary, naming what stops for the operator and stating that only those stop - draw the plan-challenge path from Reviewer to Manager, with the Manager's keep, revise, or replace decision in its I/O row - rewrite the caption to read the diagram as a distribution of the three questions rather than a dataflow Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three items on the teaser assumed the reader had already read the report. - drop the bare 1.41x from the SWE-Bench card; a token ratio with no units or baseline stated next to it is unreadable on its own, and the figure is given properly in the results table - retitle the interruption card "How often it needs you", label the two bars Argus and a chat-style agent, and say which direction is better - retitle the duty cycle card "Share of time working", qualify it with "while it has a task in hand", and label the low bar with its cause Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The second bar on the duty cycle card showed Argus's own worst campaign, which answers a question nobody asked at that point in the figure. The comparison that belongs there is the one the report is about: a harness whose Driver is a person. That ceiling is arithmetic, not a measured system — a harness cannot outlast its operator's working day, so eight hours a day bounds it near a third of wall-clock time regardless of model capability. Both the card and the results text say so explicitly rather than presenting it as an observation. The degraded-infrastructure campaign remains in Figure 4 and Table 6, where it answers its own question. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ink puts stdin in raw mode, so `argus` from a pipe, a redirect, cron or CI announced that it was starting the backend and then died inside the bundled JavaScript: Error: Raw mode is not supported on the current process.stdin, which Ink uses as input stream by default. Read about how to prevent this error on https://github.com/vadimdemedes/ink at handleSetRawMode (file:///.../argus.mjs:119:2015) Reproduce with `argus < /dev/null`. The launcher already refuses a missing bundle, a missing Node and an old Node with one actionable line each; the most common way to have no cockpit was the one it did not check. Nothing starts now, and the reply names the surfaces that work without a terminal — `--web`, `--watch`, `--status`, `--daemon` — because that is the question the operator actually has. `ARGUS_SKILL_ALLOW_HEADLESS_TUI=1` keeps the old behaviour for a host that embeds the cockpit in its own pty. Both launcher test modules drive `main()` as a terminal invocation, so each grows an autouse fixture that says so rather than every test restating it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The formalism was written for the previous framing and had drifted from what the report now argues. Several symbols were decorative, one proposition was inflated, and the claims the report is proudest of had no formal support. Contradictions with the narrative: - the campaign tuple posited an evaluation procedure f, introduced once and never used again, while the report argues no such evaluator exists for the tasks it targets; replace it with an evidence structure that degrades to a scalar where one happens to exist - ManagerAdmit was an ordinary goal-update rule; state the two conditions that make it evidence-driven, namely that authority may refuse a revision but not erase what prompted it, and that standing intent is the only fixed point while the objective is a revisable hypothesis - the role state machine omitted replan_requested, so the relation contained no edge by which a better-informed role could question the decision it was working under Overreach: - demote Proposition 1 to a remark; it is the data-processing inequality, true by construction, and the strict case was asserted without argument. Say plainly that it locates the burden rather than discharging it - drop the monotonicity conditions on capability width and depth, which the same paragraph then contradicted and the longitudinal data refutes - separate the counterfactual reuse value from the confounded startup-mature difference actually measured, and rename the latter - state which term of the review-precision identity the evidence cannot constrain, rather than presenting Bayes' rule and moving on Additions: - formalize iterating without a score: the progress set that admits informative failure, the invariant forbidding a non-idea failure from refuting a hypothesis, and evidence profiles ordered by strictness - derive the two retention surfaces from the two terms of the compression objective instead of asserting the equation explains them Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The abstract had grown to 856 words as claims were added to it, and the conclusion to 1,000, most of which restated the abstract. Several paragraphs elsewhere had accumulated past the point where a reader tracks them. - abstract 856 to 505 words, in three movements: the Driver and why verification is what makes the seat delegable, the measurement, and the work those hours produced - conclusion 1,000 to 555 words, ending on what the report does not establish rather than on a restatement of its numbers - convert the terminology paragraph in the method section, which was a glossary written as prose, into a definition list - shorten the MLE-Bench and delivery passages, which explained the same comparison twice No claims or figures were added, removed, or weakened. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`argus --web --web-host 0.0.0.0` mints a token and tells the operator "a
token was generated for this run"; the flag help promises that a non-loopback
bind always requires a bearer token. Every POST honoured that. The reads did
not, so any host on the network could ask for and receive:
GET /api/projects 200
GET /api/projects/{sid}/transcript 200 <- the whole agent transcript
GET /api/projects/{sid}/journal 200
GET /api/projects/{sid}/events 200
GET /api/projects/{sid}/snapshot 200
GET /api/projects/{sid}/status 200
GET /api/projects/{sid}/doctor 200
Reproduce by binding to 0.0.0.0 and curling any of those from the LAN address
with no Authorization header.
This reads as an omission rather than a decision: `/api/system/doctor` carried
the dependency while `/api/projects/{sid}/doctor` beside it did not, and the
suite asserted "reads stay open (no auth on GET)" two lines above a note that
artifact reads are protected "because they expose project files" — which the
transcript, journal and snapshot do more of.
`require_auth` already returns immediately when no token is configured, so the
default `127.0.0.1` bind is untouched: it has no token and stays open. The web
UI sends `authHeaders()` on reads as well as writes, so a paired browser is
unaffected. Verified end to end against a real 0.0.0.0 bind: 401 without the
token, 200 with it, and unchanged on the default localhost bind.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>Future work already claimed the runtime produces the process data a training stage would consume, and that such data is scarce because nobody is paid to record reasoning that did not work. It did not say what distinguishes these traces from the agent corpora already being assembled, which is that they are days long. - report the measured spans in the results section: 4 of 27 campaigns exceed a week, the longest runs 8.1 days, and one trace records 61,797 events - argue in future work that span rather than volume is the property that matters, since supervision for long-horizon judgment is carried by dependencies a short episode structurally cannot contain -- a hypothesis proposed on day one, contradicted on day three, revised on day five, with every intermediate verdict retained - state the concrete form the second axis would take, using the verdicts the trajectories already carry as labels, and say plainly that we have not done it The claim stays in future work; nothing is presented as demonstrated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The report measures multi-day unattended operation and then, in future work, argues those traces are the supervision a next-generation model would need. The abstract stopped at the measurement. Add a closing movement: multi-day autonomy is a capability the harness reached before the models did, the runtime records reaching it at length, and scraped corpora contain the artifacts of such work and almost none of the deliberation. Whether long-horizon judgment can be trained into a model rather than supplied to it by scaffolding is stated as the direction this points at and explicitly not as a result. Offset by tightening two earlier passages; net growth is 87 words. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
An AST scan for byte-identical function bodies found 23 duplicated groups. These three were the ones where the copies could drift apart into a real defect rather than merely costing lines. `_srv` — four modules split out of `server.py` each carried the same three-line late-binding resolver, whose whole reason to exist (a test that monkeypatches `server.<dep>` must still be seen) is easy to lose when someone "simplifies" one copy into a module-scope import. `_sha256_file` — six copies across the image API, the run contract and four verticals. Provenance and evidence checks compare these digests across subsystems, so they have to be computed the same way in all of them. `_load_json` — five literary verticals, each raising the `ManifestError` defined in the shared contract module they all already import from. The helper now lives beside the error it raises. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`argus --gc --gc-days -5` — a plausible typo for `--gc-days 5` — moved 50 projects to `projects_trash/` on a real global root. Forty-two of them were ones the same run's `--gc-dry-run` had not listed a minute earlier, because a negative window puts the cutoff in the *future*: every project then reads as untouched for longer than it, including one modified a second ago. Found by running the command; the projects were restored from the trash. `retention_days_default()` already clamped the environment spelling with `max(0, ...)`, so a negative window was never meant to be legal — the clamp was simply missing from the explicit argument, which is the destructive path. The collector now refuses one from any caller, and the CLI names the mistake and its consequence before the sweep can start. Zero stays legal: "prune anything not live" is a real operator choice. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
"Who Drives the Harness?" named the role but not the interval, which is where the result actually lives: a person can drive a harness, just not for eight days without stopping. - title becomes "Who Drives the Harness for Days?" - subtitle trades "Long-Horizon" for "Multi-Day", which is concrete rather than jargon and matches what the campaigns measure - the introduction's answering line now carries the same implication: naming the driver as a person also answers how long the driving can last Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The report gave no indication that Argus began in May 2026 and that every campaign in it ran within the following four months. That omission cuts both ways: it hides a real limitation, and it lets the results be read as the output of a mature system rather than a new one. - open the limitations with the window, since the rest should be read against it: no evidence on long-term stability, and a runtime that changed underneath its own evaluation, so the campaigns are not instances of one frozen system - open future work by naming the preceding list as a route rather than a backlog, and this runtime as the first version of a program Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`argus --web --web-port 99999` printed the pairing banner's URL on that port and then raised `OverflowError: bind(): port must be 0-65535` out of uvicorn, so the operator was offered an address that could never exist and then shown a traceback. `--web-port abc` and a negative port did the same. The parser now carries the range, which is where argparse can say it without any consumer having to. Zero stays legal: it asks the kernel for a free port. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The results section carried two SWE-Bench Pro figures whose content overlapped: panel (a) of the evolution figure already reported adaptive routing and review outcomes, which was the whole of the separate reviewer mechanism figure. Remove the standalone figure and keep its one non-redundant point -- that verifier pass and strict rescue are nested inside the revision-requested group rather than being separate outcome classes -- as prose, with the counts carried by the existing macros. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The ledger was pinned with [H], so when it did not fit in the remaining space it moved wholesale to the next page and left roughly a quarter of page 24 empty. It is a summary table with no sentence depending on its exact position, so [!t] is the correct placement and the surrounding text closes the gap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both figures in the paper-production case study were pinned with [H]. The manuscript grid is tall enough that it could not fit in the space left on page 26, so it jumped whole and left a third of that page empty and nearly half of page 27. Neither figure is referenced positionally, so [!tb] is correct. The two now sit together at the top of page 27 and the surrounding prose closes both gaps. No page in the document now ends with more than a few blank lines. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A read-through from the position of someone who has never seen this project found the same handful of stumbles repeatedly. - define the four nested scopes once, at the top of the method section: vertical selects the rules, campaign is what persists, Stage is where it currently is, mission is what it is doing now. These were used throughout and never laid out together - say explicitly that RQ0-RQ4 are the questions we measure and Q1-Q3 are the questions the runtime decides; the two numbering schemes were confusable and a reader scanning for "Q1" landed in the wrong place - make the Figure 1 caption explain the plan-challenge path rather than name it, since the term is not introduced until thirty pages later - rename the ACE-2 section from "A Chip the Runtime Designed" to "RTL to Mapped Synthesis"; the old title invited a reading the section itself spends a paragraph retracting - replace "2x#1" for SOL-ExecBench, which reads as twice the top score, with what it means: beat the top-ranked entrant on two kernels Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The formal section defined density, research value, process-data dominance, typed compression, and reuse value -- none of which this report measures -- while the quantities it does measure appeared either as one equation buried in the results or as prose. That is backwards. - define duty cycle and interruption rate in the formal section, with the three contestable choices in the duty-cycle denominator stated plainly: call intervals rather than event gaps, external jobs neither credited nor penalised, and what subtracting unavailable time does to the question being answered - formalize the interruption partition, since only the research-judgment share measures what the Driver framing is about; the authority share is bounded below by design and the infrastructure share is a property of the deployment - formalize the review funnel as nested inclusions, which is what stops a reader adding recovery counts to the outcome partition - open the section by naming which equations carry data and which do not, and mark the untested ones individually where they appear Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three of the equations marked untested could be evaluated against the campaign ledgers directly, and two of them turned out to carry the strongest numbers in the section. - the progress set: across the campaigns the runtime recorded 604 frontier transitions, 598 in the progress set, of which only 167 are artifact_improved. The remaining 72.1% is reduced uncertainty, information gained from failure, retired risk, and bounded repair -- advances a scalar objective registers as zero or as loss. This is the empirical case for the typed vocabulary, and it was previously argued only in prose - process-data strictness: of 1,852 bounded missions, 346 did not reach a done verdict, including 103 explicitly declared blocked. Nearly a fifth of what the runtime did leaves no trace in what it produced - the refutation invariant: the campaigns ran under verticals that do not emit the four-state record, so we report it as an enforced property of the implementation and say we have no field rate, rather than reporting zero violations from an empty set Also narrow the section preamble to match what is now measured. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Qualifications had accumulated to the point of restating each other. The frozen-state replay appeared in the methodology, the results, the discussion, the conclusion, and the appendix; the delivery asymmetries were set out in full in both the results and the limitations; and the conclusion had become an index of section 8. - state the frozen-replay requirement once where it does argumentative work and refer to it elsewhere - compress the delivery caveat in the results to one sentence, keeping the distinction that matters -- the third comparison was decided by people with no stake in it -- and leaving the asymmetries to the limitations - end the conclusion on what the evidence supports rather than re-listing what it does not Hedging phrases drop from 28 to 20 across the report. No qualification is removed, only the repetitions of it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Lead the duty-cycle section with the result instead of the caveats, and state measured outcomes as findings rather than as things the data is merely 'consistent with'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rgus-wechat-group.jpg
An evidence gap could be closed by "a planned supplement, ablation, or claim downgrade", and a modest credible effect could justify "targeted improvement or a narrower claim". Listed as peers, the option that costs nothing wins. Each gap now names the supplement or ablation that would close it, and a downgrade is what is left when none is affordable. A modest credible effect is a reason to push the method further. Research-vertical only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The resume delta exists so a continuing role session is not charged for the immutable contract twice, and its header already promises that the state below supersedes stale session facts. It carried the journal, the runtime digest and the stage, and dropped the one block that moves most between cycles: the vertical's search altitude. Found by driving four live ICLR campaigns on the new build. The altitude rendered 534 characters through the prompt catalog and appeared zero times in the verbatim prompt log, because every long campaign resumes its Planner session and the resumed path never included it. This is not research-specific. A metric campaign loses its promoted floor, frozen count and recombination history the same way; each vertical still renders only its own block, and a vertical with no altitude still renders nothing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Across the four live ICLR campaigns, 55 of 125 completed missions were certification, scope, checklist, schema and package work against 41 that ran an experiment — about ten hours spent on the runtime agreeing with itself. Sample titles: "Resolve submission certification hold", "Prescribe final-submission Planner scope", "Making final submission certification host-visible", "Obtaining project-final independent certification" twice in a row. The same mechanism shows up in what the campaigns called their own analyses. Eight of run-01's eleven analysis directories are named `*_repair`, `*_limitation` or `*_strict_gate`; not one is named after a hypothesis. When the next mission is chosen as the next unsatisfied requirement, the work becomes a sequence of repairs and the paper never accumulates an argument. Both roles now say the same thing from their own side. The Planner picks the work that advances the argument — the experiment that decides a claim, the comparison that earns it, the rewrite that makes one insight carry the paper — names missions after the question they answer rather than the defect they repair, and treats certification, scope prescription, package assembly and schema conformance as finishing steps inside the mission whose work they certify. The Reviewer stops sending work back for them: a missing certificate or schema field belongs in next_action. Three checklist items go with it, all of them restatements rather than requirements: `benchmark.backbone` repeated the lock that `plan.backbone` sets and `run.backbone` proves, `run.environment_preflight` repeated `benchmark.environment_preflight`, and `review.placeholders` repeated a check the deterministic structural validator already performs. Every restatement is one more box an agent can open a mission to go tick. `submission.upstream` was removed and put back: alongside the bookkeeping it carries the recorded-Manager-decision skip that lets a literature review pass without an experiment section. Research-vertical only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The router sent agents to argus_builtin_skills/engineer/..., a path that exists nowhere; the renderer actually ships beside the skill. Five of the nine skill cross-references pointed at the same missing tree. With that path dead and PPT Master absent, the only route that ran was typing <rect> and <line> into a file, which is exactly what Figure 1 became. The renderer's own contract pushed the same way: --output *.svg extracts an <svg> the page already has, so a figure laid out in CSS fails with "figure root contains no SVG" and hand-writing one looks like the fix. It is not; --output *.pdf is the vector route for CSS, and LaTeX includes PDF directly. The error now says so. "Deterministic HTML/SVG" bundled a browser-rendered layout together with hand-typed markup and offered both as one option. Splitting them leaves the route that produces a publishable figure. Each figure now gets its own subagent, so one drawing gets one undivided attempt. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
FigureSpec was the third route pointing into argus_skill/builtin_skills/, a tree that does not exist, so between it, the browser renderer's dead path and an absent PPT Master, every route a figure could take failed. Typing SVG by hand was the only thing left that worked. Its path is now resolvable, but reaching it is not enough. Rendering the skill's own three-box example produces four rects and three texts with no gradient and no shadow: the flat diagram a reviewer reads as a whiteboard sketch. The router offered that as the route for "simple exact topology", and a method figure with three boxes and two arrows looks simple, so that is where Figure 1 kept landing. Figure 1 is the figure a reviewer looks at before reading a word. Three boxes still need hierarchy, depth and a type scale, so the simple row is now scoped to supporting figures and the opening figure takes a polished route regardless of how few nodes it has. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Persist deterministic stage-transition failures as unresolved Manager feedback so the next planning cycle repairs the missing evidence instead of repeating the same rejected advance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Distinguish project-domain verticals from optional research domains in the routing prompt, and retry a bounded standalone decision when the model puts a project slug in the wrong field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The daemon's stop is cooperative and lands between missions. A round that yields to external work waits inside its mission, in `while True`, for as long as that work keeps reporting a healthy heartbeat — so it never reaches a boundary, and the request never arrives. In practice the daemon logged "received signal 15, requesting stop", then sat in `time.sleep` for fifteen minutes until it was killed, with a running experiment as the only thing keeping it alive. Signals cannot be read from inside that loop, so the handler now records the request in one process-wide flag and the loop reads it. The cadence wait checks the same flag after each poll-sized sleep, which bounds the delay to one poll interval instead of a whole cadence. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use the configured branch remote and merge ref explicitly so stale fetch refspecs for deleted branches cannot block an otherwise safe fast-forward update. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep source updates on the canonical public release line instead of inheriting a checkout's private branch upstream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(manager): preserve autonomous route continuity Allow fresh operator handoffs to revise persisted workflow and research contracts while keeping active bounded campaign supplements fail-closed. Surface persisted route state and legal enums to Manager, preserve staged exploratory workflows, and add regression coverage for the observed direct-to-staged failure. * fix(core): keep final completion gates domain-blind Resolve final-stage validation through the active vertical contract instead of importing research modules from Manager, then refresh the release identity and shipped frontend artifacts so deployment/runtime checks match the source tree.
All four ICLR campaigns produced the same paper: a boundary study. run-03 is titled "A Boundary Study"; run-04's headline endpoint is AUROC 0.503 against random 0.501. That was not four unlucky ideas. The policy offered exactly two endings — a positive result, or "the remaining boundary, mechanism, scaling, or decision finding becomes the paper" — so a method that lost to its own baseline had nowhere to go but into a manuscript. The campaign now commits to the problem and treats a named mechanism as a disposable bet on it. Selection fixes the end-task claim, the strongest resource-matched baseline, the win that would matter, the cheapest run that discriminates, and a rescue budget that ends, and that run is bought early while the method is still cheap to abandon. Evidence leaving the method short at that final run is a bet settling rather than a defect to repair, so the Reviewer returns `replan_requested` at selection and the accumulated evidence funds the next candidate. The rollback path already existed; nothing described it as anything but a defect. Cheap probes still cannot veto an idea — only claim-bearing evidence at the scale named at selection settles one. A refutation still earns a paper when the belief it overturns is one the field already builds on. This campaign's own method failing is memory. The Planner's schema example also shipped `"title":"title"` with objective "work and decisive check", and every campaign ran at least one mission copied straight out of it. The placeholders now read as slots. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous pass gave a settled bet somewhere to go, but it asked the Reviewer to decide whether the shortfall was explained by the mechanism itself. That judgement is not reliably available — a method usually falls short because something in the implementation, the optimization, the data, the scale or the evaluator is not yet right, and telling those apart from a wrong mechanism is exactly what the campaign cannot yet do. Asking for it buys premature retirement, which trades one failure mode for another. Falling short is now a gap with a size, and closing it is the work: each round names what the shortfall is made of, buys the fix that addresses it, and measures again. Many rounds are the normal shape of the thing; an early miss is the starting position, not a result about the idea. That is how a leaderboard result gets earned, and it is what the campaign spends itself on once the baseline number exists — so selection now buys that measurement early, to have a number to improve for the rest of the run. What stays closed is the exit that produced four boundary studies: a loss never becomes the paper. Moving on means another attempt. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Closing the gap is the campaign's normal state, so nothing that happens inside a mission should be able to end an idea. The Reviewer now says as much, and the research vertical gives the Manager — which had no role policy at all — the one seat that can make that call. The call is written to be reluctant. It wants sustained optimization already spent across implementation, data, scale and evaluator, the gap unmoved by any of it, and a reason the next round would fail that is not simply that the last one did. Anything less is impatience wearing the costume of judgement: a shortfall stays an engineering shortfall until the engineering has actually been done. A retired idea rolls back to selection carrying its evidence. What it still never becomes is the paper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Argus grinds well already. It will read a failing test, hypothesise, fix, re-run and re-fix until it is green, and nobody has to tell it the third attempt is allowed. The science gets a stranger treatment: implement once, measure once, and start writing about what it found. The same system that spends twenty rounds on a harness spends two on the result. The gap was never capability, so this is not a new mechanism. It is the missing description of the work: the first number is a first draft and the six things it is usually made of; measure, diagnose one cause, buy that fix, measure again; reproduce the baseline in your own harness before believing anything measured against it. Flat stretches get their own section, because the response to one is where campaigns go wrong. A plateau is what the middle of a hard problem feels like from inside, not a signal — it is only informative once the diagnosis has stopped producing new causes. Lowering the target, softening the claim and starting to write all feel like progress there. And the part that is easiest to miss: after twenty rounds of local repairs the method is not the one that was proposed, and that is the research happening rather than drift to correct. So when the number lands, look at what is actually in your hands, find the part that carried it, and write about that. A real discovery buried in an implementation detail, under a paper about the original proposal, is the common way a good result becomes a forgettable paper. The last section is about judgement over procedure: chase the surprising ablation over the next three planned runs, spend attention where the uncertainty is, and change the metric only for a reason you can state without mentioning your own result. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ideation was one model asked once. Six candidates then share one model's taste and, more expensively, one model's blind spots — and the campaign that follows spends weeks on whichever of them that single taste liked. The panel asks each installed backend separately, so the proposals are genuinely independent, and then hands each seat the others' candidates and asks for the strongest reason each would fail plus the one it would actually bet a month of GPUs on. An objection a GPT-family model cannot see is often obvious to a Gemini- or Claude-family one, and a candidate that survives a stranger's cross-examination is a better bet than one nobody argued with. Availability decides the panel, because most machines do not have every CLI. A name Argus does not support and a CLI that is not installed are the same answer: no seat. Fewer than two seats is not a panel, so a single-backend box takes exactly the single-model path it took before — same call, same label, no debate section — and one panellist returning nothing degrades the same way rather than into a one-sided argument. Operators who want specific labs can name them, as `backend` or `backend:model`, and missing seats are dropped from the list rather than failing it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
An installed CLI is not a subscription. Someone who pays for one vendor still has the other launchers on PATH — this machine has codex on PATH and cannot log into it — so resolving a binary was the wrong question to ask. The panel would have seated codex, spent an ideation round finding out it has no credentials, and only then fallen back. Seats now go through the readiness check Argus already owns, once per process, and anything it cannot confirm is not a seat. On this box that takes the auto panel from two names to one, which is the honest answer: one usable backend is not a panel, so ideation runs exactly the call it ran before. Operators with a single subscription are not shut out. One backend often fronts several labs' weights, so naming them explicitly still seats a panel — the knob says so, with an example. What the framework will not do is guess model ids on their behalf, because a guessed id is just another failed call. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two defects that only a real debate could show, both hidden by the fail-open wrapper that was supposed to protect the campaign. A bare runner is not the campaign's wrapped one: it wants the thread stated, and it reads the agent_cli options type rather than the core one, so every call died in the command builder before reaching a model. The panel dutifully logged nothing and fell back, which is exactly the shape of bug that survives a green test suite. The second is worse, because it produced output. Seats were identified by their launcher, so a panel of two models reached through one CLI filtered both of them out of "the others" and handed each reviewer an empty page. Gemini said so in its own review — "since no candidate proposals were explicitly attached to the prompt, I have autonomously inferred three archetypal proposals" — and then reviewed candidates it had invented. A seat is now the model it speaks for, which is the case an operator with a single subscription depends on. Run after the fix: gpt-5.5 and gemini-3.1-pro-preview proposed four different candidates, then argued. GPT told Gemini its manifold was the SAE's consistency set rather than the model's, so the do-operator was probably not an intervention at all; Gemini answered that both of GPT's candidates were mathematical band-aids on the same terminal disease and bet zero GPUs on either. That disagreement is the thing worth having. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The panel appended its work to research/IDEA_CANDIDATES.md, which belongs to the Idea Creator step: that step reads the file, ranks what it finds, and writes the file back whole. In the first real campaign it did exactly that seven minutes after the panel finished, and 21KB of proposals and cross-examination — the entire reason the panel exists — went from 37KB to 20KB with no trace and no error. The debate now has its own file. The candidate file still receives the candidates, so ranking sees everything it saw before, plus a line telling it where the objections and the bets are. A rewrite of the candidate file can no longer take the argument with it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Argus parallelises missions — the daemon runs its supervisors in a thread pool — but ideation is one blocking call inside one mission, so none of that reached it and the panel asked its seats one after another. Seats do not read each other while proposing, so the round now costs the slowest model rather than the sum of them, using the same pool-with-single-item- fast-path shape the daemon already uses. A seat that raises loses its own seat and nothing else. The panel also had no ending. It produced proposals and objections and left the choosing to whatever read the file, which is a strange way to spend two models arguing. Each seat now reads the whole record — every candidate and every objection against it — and names the one it would run, including one it did not propose or one it attacked, with the risk it is knowingly accepting and the measurement that would show within a week that it picked wrong. Agreement is a strong signal, disagreement is information, and neither is decided in host code. One more shape of the availability problem: a backend named twice with no model is one model arguing with itself, which is worse than no panel because it looks like one. Those collapse to a single seat, so the campaign falls through to single-model ideation. Different models on one backend are still two labs and still seat a panel. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep status read-only in fresh directories, show the backend selected by setup, and reserve paper-specific workbench modules and labels for research routes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Scaled the question up: four fresh directions, two arms each, four candidates per arm, thirty-two candidates scored blind and shuffled by a judge that sat on neither arm. The panel lost. Control 16.12, panel 14.88, and the permutation test on that gap is p = 0.92 — the wrong side of the null, not a near miss. Two earlier experiments had said the opposite, and both were the same single direction. What survives all three is spread, not level. The panel produced the best candidate in the batch by a clear margin — the only one the judge advanced — and also nine of the thirteen weakest. Per direction, best against best, it is a draw: 18.50 to 18.25, two directions each. That is a real trade and it may well be the right one when selection is good. It is not a trade a campaign should inherit from which CLIs happen to be sitting on PATH, so auto-seating is gone. Naming seats turns it on; unset, ideation is exactly what it was. The knob says what the scoring found so the choice is made with the evidence rather than the pitch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Direct synchronization of the latest Public commits without additional modifications. Conflict files were resolved to the Public versions.