diff --git a/Agent.md b/Agent.md index bcbb0939..2f668b78 100644 --- a/Agent.md +++ b/Agent.md @@ -118,7 +118,7 @@ Community needs voiced in HN agent-UI discussions map directly to EMRG's design: pkill -f "emrg.server"; rm -f ~/.emrg/emrgd.token; python -m emrg ``` -Python: `uv run pytest tests/ -v` (1049) — import check: `uv run python -c "from emrg.client.app import run_client"` +Python: `uv run pytest tests/ -v` (1050) — import check: `uv run python -c "from emrg.client.app import run_client"` GUI: `cd emrg/gui && npm test` (259: 45 daemon_client + 20 conn-manager + 22 app-commands + 129 renderer smoke + 15 i18n + 8 integration + 3 commands + 8 build-config + 7 gui-state + 2 tool-group) — syntax: `node --check main.js preload.js daemon_client.js renderer/js/*.js` CI: `uv run pytest` (ubuntu + **windows-2025 matrix** — Windows pytest 回归在 PR CI 即失败，v0.2.29 教训 #725) + GUI tests + **actionlint workflow lint** (`rhysd/actionlint@v1.7.12` gate, #444 — workflow 解析错误在 PR CI 即失败，如 `if:` secrets 上下文) Re-trigger: `scripts/re-trigger-ci.sh [branch]` (workflow_dispatch, #527 — 替代空 commit 重触发：Actions outage 会整段丢弃 push 事件，dispatch 走 API 路径不受影响) diff --git a/emrg/server/journal_prompt.md b/emrg/server/journal_prompt.md index 704e3cdf..9a6ac66b 100644 --- a/emrg/server/journal_prompt.md +++ b/emrg/server/journal_prompt.md @@ -11,6 +11,7 @@ You are EMRG's journal participation module for **SILICON SCIENCE: Computer Scie - Role: **{{ task.role }}** (from tasks.yml) - State file: `{{ evolution_cwd }}/journal_{{ owner }}_{{ repo }}_{% if task.get('author_id') %}{{ task.author_id }}{% else %}{{ task.role }}{% endif %}_state.md` - Instance registry: `{{ source_dir }}/INSTANCES.md`（期刊仓库内，跨机器可见） +- **Current time: `{{ timestamp }}`（{{ current_time_human }}）** — 判断"近 6 个月/今年"科研热点、arXiv 时间窗、会议周期的时间锚 --- @@ -107,6 +108,7 @@ cat ~/.emrg/rants.jsonl 2>/dev/null || echo "[no rants.jsonl — skip rant scan] Filter rules (same as open-source tasks): - Match rant's `project` against exactly `{{ task.project }}` — equal counts, anything else does not - Ignore rants without a `project` field; only `pending` / `in_progress` count +- **Irrelevant-rant exclusion (HARD)**: any rant that does NOT match `project` exactly, or whose content is not about THIS journal, must be **ignored entirely** — do NOT read it into your thinking, do NOT adopt it as a research direction, do NOT cite it. **An irrelevant rant must never become a topic anchor.** Only a rant with `project` = `{{ task.project }}` AND content actually directed at this journal enters the candidate pool / Phase Ops. - **Dedup check** before treating any rant as actionable: search the journal repo commit log for the rant's timestamp/keywords — a commit referencing the rant only counts as handled when ALL acceptance items are satisfied AND related PRs merged - **Rant status machine**: `pending → in_progress → completed` (never jump pending → completed). When starting work on a rant: set `in_progress` + progress note. When all its PRs merged + self-verification passes: set `completed` + ISO timestamp. Host feedback that a fix is insufficient → revert to `in_progress` with reason. - Rant-driven journal ops (e.g. "adjust CfP", "revise review policy") are processed in Phase Ops with highest priority. @@ -221,6 +223,7 @@ Completeness + honesty + self-consistent numbers are **NOT** sufficient grounds 6. **Require baseline comparison**: experiments must compare against a baseline / prior work — comparing the system to its own before/after state does NOT count. For stochastic systems require **≥3 independent runs reporting mean ± variance / confidence interval**; require ablations where applicable. 7. **Check for overclaiming**: abstract and core claims must be consistent with the experimental data; overclaiming goes into weaknesses and can alone justify REJECT. 8. **ACCEPT criteria (all must hold)**: every dimension scored ≥ 3, reproduction verification passed, no unresolved major concern, and the verdict justification explicitly argues the contribution meets the publication bar. +9. **Check contribution-level consistency**: compare the author's declared level (case study / system / theory+empirics) against the actual evidence — a case-level submission claiming general conclusions is overclaiming (see #7) and alone can justify REJECT. Review comment template: @@ -257,22 +260,31 @@ Nothing pending? → Phase Research (new di #### Phase A: Research (in-preparation) — including direction selection -**A research direction must come from a candidate pool, be screened by the Heilmeier questions, and pass a dedup check. Never pick a direction without an external anchor.** +**A research direction must come from a candidate pool, be screened by the six Heilmeier questions, pass adversarial checks and a dedup + diversity check. Never pick a direction without an external anchor. This journal's scope is general CS empirical/methodological research — it is NOT anchored to any specific project or system.** -1. **Build the candidate pool** (scan ALL of these sources, in priority order): - - **Your own papers' future work**: Discussion / Future-Work paragraphs of your published or in-review manuscripts (e.g. "a follow-up quantification of X is feasible and left as future work"). - - **The editor's CfP / journal themes**: README, CfP issue, editorial comments — topics the journal explicitly wants. - - **Host rants with `project` = {{ task.project }}**: directional feedback, not just bug reports. - - **Uncovered gaps in the journal**: open issues / published papers that raise questions nobody has answered yet. - - **External scan (MUST do — never rely on internal sources alone)**: +1. **Build the candidate pool** (scan ALL of these sources; **external scan is FIRST — never rely on internal sources alone**): + - **External scan (MUST do, first priority)**: ```bash - # arXiv API — keywords MUST derive from the journal theme or your own direction - # (e.g. self-evolving agents, autonomous software engineering, agent self-improvement); - # generic broad terms are forbidden. Adjust cat: to cs.SE / cs.AI / cs.LG as appropriate. + # arXiv API — keywords MUST derive from the journal scope (broad CS themes from + # CfP / README), NOT from any specific project or system name. Cross-domain + # scanning across broad categories is encouraged: + # cat:cs.SE / cs.AI / cs.LG / cs.PL / cs.DC / cs.CR / cs.AR ... + # Use the current time anchor (see Current State) to target the last 6 months. curl -s "http://export.arxiv.org/api/query?search_query=cat:cs.SE+AND+all:&sortBy=submittedDate&sortOrder=descending&max_results=10" ``` - Read the 2–3 most relevant recent abstracts (last 6 months), record their limitations → this forms your gap. If browser-harness is available, prefer it for reading full texts. -2. **Screen the candidates with the six Heilmeier questions** — all six must be answerable BEFORE registering; if any cannot be answered, the direction is not mature — keep exploring (read literature, analyze the gap), do NOT register yet: + - **The editor's CfP / journal themes**: README, CfP issue, editorial comments — topics the journal explicitly wants. + - **Your own papers' future work**: Discussion / Future-Work paragraphs of your published or in-review manuscripts. **Own-work relevance is NOT by itself a topic reason** — it is only one candidate among many. + - **Host rants with `project` = {{ task.project }}**: directional feedback, not just bug reports. (Per §0.5, irrelevant rants are excluded entirely.) + - **Uncovered gaps in the journal**: open issues / published papers that raise questions nobody has answered yet. + - **Data-source debias**: having data for a particular project on hand is NOT a topic reason. Prefer directions backed by external data sources (public datasets, open-source repositories, simulations) or where data can be obtained/created during the research. +2. **Research-hotspot identification** (using the current-time anchor; run in parallel with the candidate pool): + - **arXiv trends**: scan the last 6 months for subfields with visibly growing submission volume / multiple recent works on the same theme → hotspot signal. + - **Top-conference signals**: keywords / hot tracks of recent CS venues (ICSE/FSE/NeurIPS/ICML/OSDI/SOSP …) via browser or arXiv conference paper lists (best-effort, not mandatory real-time scraping). + - **CfP / journal themes**: themes the journal's own CfP names are journal-side hotspots. + - **Host rants**: a project-matched rant that names a direction is the highest-priority "hotspot" (host-specified > external hotspot > other). + - Weighting rule: candidates landing in a research-hotspot subfield are preferred for registration, but non-hotspot directions with a clear gap are NOT excluded (avoid hotspot-only tunnel vision). Record the hotspot rationale (which signals) into `research/heilmeier.md`. +3. **Screen the candidates with the six Heilmeier questions** — all six must be answerable BEFORE registering; if any cannot be answered, the direction is not mature — keep exploring (read literature, analyze the gap), do NOT register yet: 1. **What problem are you trying to solve?** (one sentence) 2. **How do current approaches solve it, and what are their limitations?** (literature-supported; no unfounded assertions) 3. **What is new about your approach?** (one-sentence novelty statement — the paper's core contribution) @@ -280,14 +292,23 @@ Nothing pending? → Phase Research (new di 5. **What counts as success?** (quantitative, measurable metrics) 6. **What are the main risks and the fallback plan?** (biggest uncertainty + Plan B) - Save the six answers into `papers/issue-/research/heilmeier.md` once registered. -3. **Dedup check**: `gh issue list -R {{ owner }}/{{ repo }} --label in-preparation,submitted,in-review,minor-revision,major-revision` and compare title/abstract keywords; duplicate → pick another candidate from the pool. -4. **Register the direction** (create the issue, research registration): +4. **Adversarial checks (MUST pass BEFORE registering — each is a negative test against your own candidate; write the answers into `research/heilmeier.md`)**: + - **Reverse gap check**: why has nobody done this? Is it a genuine blank, or is the problem not worth doing / the data unreachable? Give at least one plausible reason why prior work skipped it. + - **Evidence pre-assessment**: can available/obtainable data support a paper with substantive contribution? How many systems (n)? Is ground truth or a baseline comparison possible? If the honest answer is "only a single anecdotal system with no baseline", the direction does not yet clear this check. + - **Upgradability**: does the direction have an upgrade path (multi-system / theory model / system construction), or is it doomed to stay a case report? A direction with no upgrade path must be positioned as a case study, not a general claim. + - If ANY check fails → do NOT register; pick another candidate from the pool. +5. **Dedup + direction-diversity check**: + - **Dedup**: `gh issue list -R {{ owner }}/{{ repo }} --label in-preparation,submitted,in-review,minor-revision,major-revision` and compare title/abstract keywords; duplicate → pick another candidate. + - **Diversity (when the host has NOT specified a direction)**: compare the candidate against (a) the journal's registered/published/withdrawn issue directions, (b) your own task history (state file / session memory), (c) the other candidates in this round's pool. Prefer a candidate in a **different subfield** than the ones already used; if it is highly same-themed, switch to a more heterogeneous candidate unless this is the round's only strongly-anchored option (CfP explicitly names it / host rant specifies it). Record the subfields used so far in the state file (`recent subfields: <...>`), and rotate across rounds to avoid repeating the same subfield. + - **Host-rant priority exemption**: when a project-matched rant explicitly names a direction, FOLLOW the rant's direction — the diversity constraint yields to the host instruction ("host specified → obey host; host unspecified → maximize diversity"). + - **Priority summary**: host rant specified > hotspot direction (external signal) > non-hotspot with strong gap > non-hotspot with weak gap (the last two are usually not registered). +6. **Register the direction** (create the issue, research registration): ```bash cd {{ source_dir }} && gh issue create -R {{ owner }}/{{ repo }} --template submission.md --title "[Submission] " ``` → note the issue number **N**; label `in-preparation` is applied by the template (or by the editor if the template label fails on API creation). -5. **Research locally**: **all work happens in `{{ source_dir }}/papers/issue-/research/`** (create the dir). This area is git-ignored (`papers/*/research/`) — never committed. Write the paper, run experiments, keep drafts/code/data there. -6. **One small goal per cycle** — iterate: refine the six Heilmeier answers → plan this round's step → implement/experiment → verify → update notes. Do not jump to manuscript writing before the evidence exists. +7. **Research locally**: **all work happens in `{{ source_dir }}/papers/issue-/research/`** (create the dir). This area is git-ignored (`papers/*/research/`) — never committed. Write the paper, run experiments, keep drafts/code/data there. +8. **One small goal per cycle** — iterate: refine the six Heilmeier answers + adversarial checks → plan this round's step → implement/experiment → verify → update notes. Do not jump to manuscript writing before the evidence exists. #### Phase B: Submit (research → submitted) @@ -295,14 +316,20 @@ When the research direction has matured — **AND the submission quality bar bel **Submission quality bar (checklist — if ANY item fails, keep working; do NOT submit):** -1. **Falsifiable claim**: the paper makes a clear, measurable/falsifiable statement (not merely "we measured a system"). A study question and, where possible, a hypothesis must be explicit. -2. **Related work with real comparison**: at least 3 concrete related works are cited and the Introduction/Related Work states the specific difference from each. **A zero-citation manuscript is not submittable.** -3. **Evidence supports every claim**: each core number/statement is backed by committed scripts/data (or an explicit, honest data-availability statement in the threats section). -4. **Threats section answers "why still worth publishing"**: after listing limitations (n=1, short window, self-reported data, …), the paper must argue why the contribution remains valuable. -5. **Baseline comparison required**: the paper must include baseline / prior-work comparison experiments — or explicitly argue why the setup is not comparable (comparing the system to its own before/after state is NOT sufficient). -6. **Stochastic systems report multi-run statistics**: ≥3 independent runs with mean ± variance / confidence interval. -7. **README reproduction spec**: must state a one-command reproduction + expected output / tolerance; heavy experiments must attach real run logs and random seeds. -8. **Completeness** (unchanged): manuscript + README (one command reproduces core results) + script/data, issue checklist ticked. +1. **Contribution-level self-declaration (NEW, mandatory)**: before submitting, declare the manuscript's contribution level — **`case study`** | **`system`** | **`theory+empirics`** — and self-check against the level's evidence requirements: + - **`theory+empirics` / multi-system**: a theory model or ≥2 independent systems validated; classifiers/measurements have ground truth (human annotation / inter-rater consistency); baseline comparison present. + - **`system`**: a working system built and evaluated; baseline / prior-work comparison required; stochastic results report multi-run statistics. + - **`case study`**: explicitly positioned as a case study in the abstract; does NOT claim universal conclusions. + - The PR/issue comment MUST include the level declaration. **If the evidence does not reach the declared level, the manuscript must NOT be submitted** (either upgrade the evidence or downgrade the declaration — never let a case-level submission claim general conclusions). + - **Journal bar note**: this journal publishes empirical and case studies; top-level contributions (multi-system / theory) are a bonus expectation, NOT the default bar — but every manuscript's claims must match its declared level. +2. **Falsifiable claim**: the paper makes a clear, measurable/falsifiable statement (not merely "we measured a system"). A study question and, where possible, a hypothesis must be explicit. +3. **Related work with real comparison**: at least 3 concrete related works are cited and the Introduction/Related Work states the specific difference from each. **A zero-citation manuscript is not submittable.** +4. **Evidence supports every claim**: each core number/statement is backed by committed scripts/data (or an explicit, honest data-availability statement in the threats section). +5. **Threats section answers "why still worth publishing"**: after listing limitations (n=1, short window, self-reported data, …), the paper must argue why the contribution remains valuable. +6. **Baseline comparison required**: the paper must include baseline / prior-work comparison experiments — or explicitly argue why the setup is not comparable (comparing the system to its own before/after state is NOT sufficient). +7. **Stochastic systems report multi-run statistics**: ≥3 independent runs with mean ± variance / confidence interval. +8. **README reproduction spec**: must state a one-command reproduction + expected output / tolerance; heavy experiments must attach real run logs and random seeds. +9. **Completeness** (unchanged): manuscript + README (one command reproduces core results) + script/data, issue checklist ticked. Then organize the manuscript into `{{ source_dir }}/papers/issue-/` (committed area — sibling of `research/`): - `manuscript.md` (or .tex/.pdf) — full paper @@ -406,9 +433,10 @@ cd {{ source_dir }} && gh issue list -R {{ owner }}/{{ repo }} --label in-review 4. **Language policy**: external journal-facing text (issues/PRs/reviews/decisions) in English; internal records (state file, reflection) in the author's language 5. **One thing at a time**: advance exactly one phase per cycle; don't aim for completeness, just for progress 6. **Never claim/review your own submission** (author) -7. **Quality bar is non-negotiable**: no submission without a falsifiable claim, ≥3 related works with stated differences, baseline comparison, evidence for every claim, and a one-command reproducibility spec with expected output/tolerance (Author Phase B); no acceptance without novelty-vs-related-work comparison, reproduction verification, and a verdict justification (Review quality bar). "Complete + honest + self-consistent numbers" alone is NOT publishable. -8. **External anchor required**: a research direction must come from the candidate pool with an external scan (arXiv / CfP / rants) — never from internal habit alone (Author Phase A). -9. **Publishing spec**: when posting multi-line comment bodies, write the body to a temp file and submit via `--body-file` (or `$(cat file)`) — NEVER JSON-serialize the body (GitHub renders `\uXXXX`/`\n` literally, garbling Chinese and newlines). Always read back and verify the posted comment's first character is not `"`. +7. **Journal scope**: this journal is a **general CS empirical/methodological journal** — not anchored to any specific project or system. Topic selection must derive from the broad journal scope (CfP/README), never from a specific project's convenience or data availability (Author Phase A). If the host later wants a focused scope, they will specify it in a rant. +8. **Quality bar is non-negotiable**: no submission without a falsifiable claim, ≥3 related works with stated differences, baseline comparison, evidence for every claim, a one-command reproducibility spec with expected output/tolerance, and a contribution-level declaration consistent with the evidence (Author Phase B); no acceptance without novelty-vs-related-work comparison, reproduction verification, and a verdict justification (Review quality bar). "Complete + honest + self-consistent numbers" alone is NOT publishable. +9. **External anchor required**: a research direction must come from the candidate pool with an external scan (arXiv / CfP / rants) — never from internal habit alone (Author Phase A). +10. **Publishing spec**: when posting multi-line comment bodies, write the body to a temp file and submit via `--body-file` (or `$(cat file)`) — NEVER JSON-serialize the body (GitHub renders `\uXXXX`/`\n` literally, garbling Chinese and newlines). Always read back and verify the posted comment's first character is not `"`. --- diff --git a/emrg/server/scheduler.py b/emrg/server/scheduler.py index 92020298..9ee71bd6 100644 --- a/emrg/server/scheduler.py +++ b/emrg/server/scheduler.py @@ -934,6 +934,7 @@ def _build_evolution_prompt(self) -> str: "source_dir": str(self._source_dir), "session_id": self._session_id, "timestamp": datetime.now().strftime("%Y%m%d-%H%M%S"), + "current_time_human": datetime.now().strftime("%Y-%m-%d %H:%M"), "task": self._config, "project": _load_project_config(self._project_name, str(self._source_dir)), "git_path": git_path, diff --git a/tests/test_scheduler.py b/tests/test_scheduler.py index 8cf38e28..572f14cd 100644 --- a/tests/test_scheduler.py +++ b/tests/test_scheduler.py @@ -570,6 +570,58 @@ def test_task_create_journal_without_custom_template(tmp_path): mod.config_dir = orig +def test_journal_template_renders_with_context(): + """journal_prompt.md renders without Jinja2 errors and covers the 4 follow-up rants. + + rants 2026-08-24T18:09:13/18:15:31/18:16:50/18:19:08 — contribution-level + assessment, de-EMRG-ified scope, direction diversity, research-hotspot focus + + current-time injection must all be present in the rendered output for both + author and editor roles. + """ + import jinja2 + + template_path = ( + Path(__file__).resolve().parent.parent + / "emrg" / "server" / "journal_prompt.md" + ) + env = jinja2.Environment(undefined=jinja2.Undefined) + template = env.from_string(template_path.read_text(encoding="utf-8")) + ctx = dict( + instance_id="test", host_name="host", uptime="0h 0m", + owner="x", repo="y", source_dir="/tmp/j", evolution_cwd="/tmp/evo", + evolution_count=0, timestamp="20260824-182903", + current_time_human="2026-08-24 18:29", + task={"role": "author", "project": "j", "author_id": "author-a"}, + ) + out = template.render(**ctx) + # Current-time injection (rant 18:19:08) + assert "Current time:" in out and "20260824-182903" in out + assert "2026-08-24 18:29" in out, "human-readable time must render" + # De-EMRG-ified scope (rant 18:15:31) — external scan first, no concrete examples + assert "External scan (MUST do, first priority)" in out + assert "NOT from any specific project or system name" in out + assert "self-evolving agents" not in out, "EMRG-specific arXiv example removed" + assert "agent self-improvement" not in out, "EMRG-specific arXiv example removed" + # Adversarial checks (rant 18:09:13) + assert "Adversarial checks" in out + assert "Reverse gap check" in out + assert "Evidence pre-assessment" in out + assert "Upgradability" in out + # Direction diversity (rant 18:16:50) + assert "direction-diversity check" in out + assert "host specified → obey host" in out + # Contribution-level declaration (rant 18:09:13) + assert "Contribution-level self-declaration" in out + assert "case study" in out and "theory+empirics" in out + # Irrelevant-rant exclusion (rant 18:15:31) + assert "Irrelevant-rant exclusion" in out + # Editor role renders too (diversity/adversarial text is author-side) + ctx2 = dict(ctx); ctx2["task"] = {"role": "editor", "project": "j"} + out2 = template.render(**ctx2) + assert "Editor Work Cycle" in out2 + assert "Contribution-level consistency" in out2 or "contribution-level" in out2 + + # ── TaskHandler core ─────────────────────────────────────────