Skip to content

Blackbird prompts: pitch-only / incubator refactor (prompts + docs) - #34

Open
andrewsu wants to merge 17 commits into
blackbirdfrom
blackbird-prompt-refactor
Open

Blackbird prompts: pitch-only / incubator refactor (prompts + docs)#34
andrewsu wants to merge 17 commits into
blackbirdfrom
blackbird-prompt-refactor

Conversation

@andrewsu

Copy link
Copy Markdown
Member

What this PR does

Reframes the coPI agent prompts — and their Blackbird-facing docs — from the original
lab↔lab collaboration model to a pitch-only / incubator model: each lab pitches one
idea to BlackbirdBot, which interviews it against a rubric and files an Opportunity
Assessment. Labs never talk to each other.

Scope is prompts + docs. The only code touched is prompt-string content in
thread_guidance.py (Baltimore removal). The engine still implements parts of the
collaboration model — see Remaining tension below, which is intended for a separate
reviewed change.

Commits (6)

  • a376d44 docs(specs): reframe the two prompt-set docs for the end-user audience
  • 55e89f8 feat(prompts): apply the incubator reframe to the on-disk prompt files
  • dcc5212 refactor(prompts): drop Baltimore location gating and trim redundancy
  • 96e2522 refactor(prompts): make intake pitch-only on both sides
  • fc8d8a9 docs(specs): add hub↔lab flow schematic; trim dormant Phase 2 from the prompt-set docs
  • 66ed1b8 docs(prompts): reframe the PI "about a paper" guidance for pitch-only

Blackbird-facing review docs


Remaining tension: pitch model vs. collaboration model

TL;DR

This PR converts the prompts and docs to a pitch-only / incubator model, but large
parts of the engine still implement the original lab↔lab collaboration model. The prompts
now tell each bot to do one thing while some of the code still drives the other. CI is red
until the code and the golden-master snapshots are reconciled.

The two models

  • Collaboration model (original, what the code was built for): a lab↔lab mesh. Labs
    discover each other's posts, propose collaborations, exchange :memo: Summary proposals,
    confirm with , plus a funding/GrantBot/FOA track. Post types include paper,
    help_wanted, introduction, idea_crosslab, funding_collab.
  • Pitch model (intended, what this PR describes): a star. Labs never talk to each
    other; each lab pitches one idea to BlackbirdBot, which interviews it against a
    rubric and files an Opportunity Assessment. No collaboration, no :memo:, no , no
    scouting.

Already converted (prompts + docs only)

  • All prompt .md files for both roles → pitch-only (system prompt, phase-4 interview,
    phase-5 post, phase-2 turned into no-ops).
  • Baltimore location gating removed everywhere (including _SCOUT_HUB guidance in code).
  • Both Blackbird-facing docs streamlined + a flow schematic added.
  • The hub side is essentially consistent (its _SCOUT_HUB guidance was already
    incubator-aligned; its role.toml already restricts posts to opportunity_assessment).

Where the collaboration model still lives — the conflicts to resolve

#LocationProblemSeverity
1src/agent/thread_guidance.py_PI_LAB (≈lines 23–49)The lab bot's actual interview guidance still says: ask about "the other lab's capabilities," look for "complementarity," name "a first experiment," "build toward a :memo: Summary proposal," and confirm a peer's :memo: with . This is the opposite of the pitch-only prompt wrapping it. Blocked by CLAUDE.md: those pi_lab strings are declared byte-identical/immutable and snapshot-pinned — rewording them needs sign-off + the snapshot regen. The intended replacement text already exists as §4 of the PI doc.Highest — without this the lab bot behaves collaboration-style regardless of every other prompt.
2src/agent/post_types.pyDEFAULT_POST_TYPES (≈line 91)Still the pre-reframe six types (paper, help_wanted, introduction, idea_crosslab, pitch, funding_collab). Since pi_lab has no role.toml, this tuple is the lab bot's menu, so the "authoritative" {post_type_menu} still offers collaboration/paper posts the prompts say don't exist. Should narrow to (pitch,). Unit-test deps: test_phase5_terminal_posts.py posts a paper; test_agent_prompts.py iterates the tuple.High
3simulation.py:memo:→Proposal path + prompts/email-reply-classify.mdThe collaboration-proposal lifecycle (a :memo: confirmed by creates a Proposal row) still exists. The prompts forbid it, so it's dormant once #1 lands — but it's collaboration-model residue to remove or consciously keep.Medium
4tests/characterization/__snapshots__/test_agent_turn_gm.ambr + test_phase5_prompt_gmGolden masters are pinned to the old prompts (old system prompt, _PI_LAB, six-type menu). This PR's .md changes already diverged them, so these tests fail today and ci.sh is red. They must be regenerated as a reviewed diff once #1/#2 land.Gate to merge
5Phase 2 scan/pruneNo-ops now, but still called every turn (no phase-skip guard). Tracked in #33.Low (cost only)
6GrantBot/FOA residue{foa_number} still substituted into the phase-4 template as literal "none"; #funding-opportunities still a universal channel; ## Your Private Instructions still injected by agent.py; GrantBot process/funding_collab/retrieve_foa still present.Low / separate scope

Consequences right now

  • The running lab bot is told pitch-only by its system prompt but told to broker
    collaborations by its phase guidance (Co pi podcast #1) — an internal contradiction in production behavior.
  • The lab bot's post menu can still offer paper/help_wanted/etc. (future ideas #2).
  • CI is red until the snapshots are regenerated (New LLM harness for making calls #4).

Suggested resolution order

  1. Decide Co pi podcast #1 (reword _PI_LAB) — the crux; needs a call on overriding the CLAUDE.md
    "do not reword" guard. Target text is PI-doc §4.
  2. future ideas #2 narrow DEFAULT_POST_TYPES to (pitch,) + fix the two unit tests.
  3. Turn based blocking and blank LLM response #3 decide: delete the :memo://Proposal lifecycle, or leave dormant.
  4. New LLM harness for making calls #4 regenerate golden masters as one reviewed diff (only after Co pi podcast #1Turn based blocking and blank LLM response #3 stabilize).
  5. investigate bad paper links #5 (Remove the dormant Phase 2 (scan + prune) prompts under pitch-only intake #33) add the Phase-2 skip guard + delete the four dormant files.
  6. deal with waitlist spam #6 scope the GrantBot/private-instructions cleanup separately.

andrewsuand others added 6 commits August 7, 2026 17:22
These are meant for Blackbird employees to comment on the prompt-driven
behavior of the bots, not for a developer tracking a change. Strip the
developer meta accordingly: the "Revision N" changelogs, the
PROPOSED/UNCHANGED labels, the prompt-assembly/code mechanics with file
and line references, and the code-change appendices. Retitle the
file-path section headers to plain-language names, keep every prompt body
in full, and add a per-section "Source:" line pointing at the file each
block quotes.
Note: the quoted blocks still reflect the proposed prompt text, which does
not yet match the on-disk prompt files verbatim; reconciling that is a
follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Overwrite the PI (pi_lab / global) and hub (scout_hub) system and phase
prompts with the proposed text documented in
docs/specs/2026-08-07-{pi,hub}-bot-prompts.md. This moves both roles off
the old lab-to-lab scientific-collaboration model and onto the Blackbird
incubator/investor model: GrantBot/FOA and funding threads removed, no
lab-to-lab collaboration or brokered introductions, private instructions
dropped, and the screening rubric folded into the scout_hub system prompt.
identity.md (both) and the eight prompts/specialists/*.md were already
identical and are left untouched. The two "interview phase guidance"
sections are sourced from src/agent/thread_guidance.py (Python, not an md
file) and are out of scope here.
These prompts assume accompanying code changes not made in this commit
(e.g. the Phase-4 {foa_number} substitution, the PI post-type list, and
the #funding-opportunities universal channel).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Blackbird will evaluate the Baltimore-anchoring commitment outside this
system, so remove it as a screening dimension everywhere it drives bot
behavior:
- PI + hub agent-system / phase4 / phase5 prompts: the gating criterion,
the interview questions, the red flag, the decision-heuristic clause,
the deferral examples, and the `baltimore_commitment` key in the
assessment JSON skeleton. Dropping the key is safe — `gating` is plain
JSONB filtered by `_normalize_gating`, and no schema column or consumer
reads it.
- src/agent/thread_guidance.py: the `_SCOUT_HUB` DECIDE guidance (the one
remaining live "ask whether they'd anchor in Baltimore" instruction).
The snapshot-pinned `_PI_LAB` strings have no Baltimore and are left
byte-identical.
- Both companion docs: §6 phase-guidance blocks stripped, and the six
changed prompt blocks resynced so the docs still mirror the files.
The Maryland non-dilutive funding programs (TEDCO/MSCRF/MII/QOF) are kept
— they are capital sources, not location gating.
Also removes a few clear redundancies: the PI "Who You Can Reach" section
(covered by Core Rule 4 + Post Labels; its one unique point folded into
Core Rule 4), the redundant "never ask to be introduced" conclusion
bullet, and the hub Post-Labels pitch-intake paragraph (restated
Interview Structure).
Note: test_agent_turn_gm's golden-master snapshot still holds the
pre-reframe collaboration prompts (it diverged at 55e89f8); regenerating
it as a reviewed diff is deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A lab agent's only top-level post is now a 💡 pitch, and the hub only
ever interviews pitches — it no longer scouts unsolicited results.
Lab agent (prompts/agent-system.md, phase5-new-post.md):
- Drop the 📰 Result post type from Post Labels, "how an
interview starts", and Phase 5. If the lab cannot turn something into a
pitch, it does not post.
Hub (prompts/roles/scout_hub/*):
- agent-system: an interview begins only when a PI opens a 💡 pitch
(auto-routed as a thread); the hub does not scout and never opens a
thread at a lab itself.
- phase2 scan + prune: now no-ops — with labs posting only pitches (which
reach the hub as Phase 3 threads, not this list) and the hub's own 🔍
assessments excluded, there is nothing to select or keep.
- phase4: interviews are unchanged; the "about a paper" note reframed
from result-scouting to a pitch that builds on published work.
- phase5: the "reply to an interesting post" option is gone (nothing
populates the interesting list); the assessment is now Option A and skip
Option B, and the action JSON drops "reply"/target_post_id.
Docs mirror the files. Prompt-only change: DEFAULT_POST_TYPES
(post_types.py) still lists `paper` and the other pre-reframe types, so
the rendered menu and the golden-master snapshot regen remain deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e prompt-set docs
- Add docs/specs/2026-08-07-hub-lab-flow.md — a Mermaid schematic for the
Blackbird-staff audience showing the pitch → interview → assessment
cycle, the per-turn phase pipeline, and the interview's message-count
progression.
- Streamline both prompt-set docs so reviewers see only the prompts that
actually drive an exchange: drop the two dormant Phase 2 sections
(Scanning / Trimming — no-ops under pitch-only intake), renumber the
remaining sections, and reframe the intros away from "full set" /
"scanning".
The Phase 2 prompt files still exist on disk; skipping the phase and
deleting them is tracked in #33. These docs are now curated
"used-prompts" references, not complete on-disk mirrors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Under pitch-only intake an interview starts from the lab's own pitch, not
from the hub scouting results. Rewrite the PI phase-4 "about a paper"
section from result-scouting framing ("the hub reads results looking for
something worth screening") to "if your pitch builds on one of your lab's
papers", matching the hub-side reframe. Mirrored into the PI doc.
The deeper PI-side gap — the _PI_LAB phase guidance in
src/agent/thread_guidance.py still uses the collaboration model
(:memo:/complementarity/first-experiment) and is snapshot-pinned — is
left for a separate reviewed change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ahueband others added 11 commits August 12, 2026 11:00
…ocs branch, engine branch)
Approved design from the 2026-08-12 brainstorm: strict hub-and-spoke topology,
Approach C auto-activation intake, GrantBot/funding removal, memo-lifecycle
deletion, phase-2 code guard, one-pitch-per-day pacing, and the two-branch
packaging (this branch = instruction text for PR34; engine reconciliation as a
stacked draft PR).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ase guidance
Replaces the collaboration-era _PI_LAB strings (:memo:/✅ mandates) with the
pitch-model text from the PI doc §4, and ports the hub doc's four §4 rewordings
into _SCOUT_HUB (including 'in your rubric' for the dangling private-instructions
pointer). Golden masters are regenerated in the engine-reconciliation branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…le; align interview-start with hub auto-open
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…fidentiality; resolve the unsatisfiable ❓ label
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e-note length band
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ontext block
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…em in the prompt-set docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ion intake, disabled-in-code phase 2, one pitch per day
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ment lead-in, P3 activation node
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e-1 table label
Final-review fixes: the hub prune prompt (and its doc mirror) no longer
asserts the retired do-not-scout model; the flow-doc phase-1 row header now
matches its corrected cell; the Key-rules 💡 is backticked (deliberate
deviation from the plan's bullet text, for consistent literal rendering);
the Reading-it paragraph is re-wrapped with no word changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ahueb

Copy link
Copy Markdown
Collaborator

Prompts+docs reconciliation landed on this branch

Following an adversarial audit of this PR and a design review (see
docs/plans/2026-08-12-pr34-pitch-only-reconciliation-design.md, committed here), this
branch now carries the full instruction-text reconciliation:

  • thread_guidance.py strings adopted from the docs' §4_PI_LAB is now the
    pitch-model text (removing the live MUST-vs-NEVER :memo:/ contradiction this PR
    introduced when it deleted the old scouting-hub exception paragraph), and _SCOUT_HUB
    picked up the hub doc's rewordings, including "in your rubric" for the dangling
    private-instructions pointer. Both docs' §4 sections are now true.
  • Confidentiality rule restored (own private instructions) in both system prompts —
    the old Rule 3 protection had been dropped while ## Your Private Instructions is
    still injected.
  • Intake reframed for auto-activation: every lab post opens a hub thread, mentioned
    or not; the hub may open/join a thread at a lab. (Engine change lands in the stacked
    reconciliation PR.)
  • Fixes: unsatisfiable :question: label resolved; gating skeleton demonstrates all
    three tri-state values; visible-note band made absolute (4-8 sentences);
    one-pitch-per-day documented; phase-2 prompts marked disabled-in-code and reproduced
    in the docs; flow doc corrected (phase-1 label, intake, phase-2 status).

CI remains red on this branch by design — tests and golden-master regeneration land
in a stacked engine-reconciliation draft PR (topology enforcement, post-type narrowing
to pitch/assessment, GrantBot/funding removal, memo-lifecycle deletion, phase-2 code
guard, PI-tag repurpose, per-role pitch cap). Golden masters will be regenerated last as
a single reviewed diff.

Flagged for the engine-reconciliation branch (out of this branch's scope): the final
review found mesh-model residue in two prompt files outside this branch's file list —
prompts/private-profile-synthesis.md (generates "Collaboration Preferences" into private
profiles that are injected as ## Your Private Instructions, contradicting the new
no-joint-work rules) and prompts/pi-dm-classify.md (a "look for opportunities with the
Wiseman lab" example). The reconciliation branch should sanitize both and add
forbidden-phrase assertions (e.g. "do not scout", "only way an interview") to the planned
doc-sync test.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@andrewsu@ahueb