behavioral packs: forbid tool-call syntax in the single-reply subject prompt

docs/testing.md

@@ -130,7 +130,14 @@ that it is green because it did not run, never silently.
   never the arbiter.
 - **What it cannot prove.** Multi-round protocol behavior, composition between
   plugins, or anything an LLM judge can be fooled about; each pack tests its
-  skill alone, single-turn, under a pinned cheap subject model.
+  skill alone, single-turn, under a pinned cheap subject model. Because that
+  turn is tool-less, every pack's subject prompt (`prompt.txt`, or the
+  `prompt.js` function tailscale-wif renders from) carries an explicit no-tools
+  clause (the subject must never emit tool-call syntax or stop to "read the
+  file first" — it says what it would look for and answers anyway, without
+  inventing results it did not obtain), and the cheap tier discovers each
+  pack's configured prompt file from its `promptfooconfig.yaml` and asserts
+  the clause is present in every one.
 - **Fires.** Per-plugin matrix leg, path-gated to that plugin's
   `evals/promptfoo/**` or the shared `evals/paid/**`; required aggregate
   `behavioral tier (promptfoo)`; skipped legs announce themselves.

evals/cheap/run.sh

@@ -1021,6 +1021,60 @@ if [ -f "ci/check_behavior_surfaces.py" ] && [ -f ".github/workflows/evals.yml"
 fi
 # ─── END RQ-001 behavior-surface trigger map ─────────────────────────────────
 
+# ─── BEGIN behavioral-pack no-tools clause ───────────────────────────────────
+# --- 21. Every behavioral pack's subject prompt forbids tool-call syntax -------
+# The behavioral tier is a SINGLE-reply harness: the subject model never gets a
+# tool result back. A cheap subject model can still stochastically emit fake
+# tool-call syntax (e.g. `<tool name="read" args={"path": "src/list.js"} />`)
+# and stop, with no final answer for the grader to judge — PR #95 run
+# 33592173756 lost two of three scope-fence rows exactly that way, and the same
+# pack was green minutes later on PR #96. Under the honest statistical gate
+# (pass-rate.sh scores real assertion failures against the floor) such rows are
+# real FAILs, so any pack could flip red on any run. Each pack's subject prompt
+# therefore carries an explicit no-tools clause; this gate asserts the
+# load-bearing phrase is still present in EVERY pack so it cannot drift out of
+# one prompt silently. The prompt surface is DISCOVERED from each pack's
+# promptfooconfig.yaml (`- file://<prompt>` under `prompts:`), never assumed to
+# be prompt.txt: tailscale-wif renders its subject prompt from prompt.js, and a
+# gate that only scanned prompt.txt reported green while that pack was still
+# exposed (Codex review, PR #97). A pack whose config names a prompt file that
+# does not exist, or names none, fails too. REPO-level gate, same inertness
+# marker as §20 (`.git`): the counterfeit tier's synthetic root ships no
+# promptfoo packs. Fail-closed: zero packs found in the real repo is itself a
+# failure. FAIL substring: "no-tools clause".
+if [ -e ".git" ]; then
+  group "behavioral packs: subject prompt forbids tool-call syntax"
+  NO_TOOLS_PHRASE='never emit tool-call'
+  pack_prompts=0
+  for cfg in plugins/*/evals/promptfoo/promptfooconfig.yaml; do
+    [ -f "$cfg" ] || continue
+    cfg_dir=$(dirname "$cfg")
+    # every `file://…` under the prompts: block (comments stripped); the block
+    # ends at the next top-level key.
+    prompt_refs=$(awk '/^prompts:/{p=1;next} p&&/^[^ #-]/{p=0} p' "$cfg" \
+      | sed 's/#.*$//' | grep -o 'file://[^[:space:]"'"'"']*' | sed 's#^file://##' || true)
+    if [ -z "$prompt_refs" ]; then
+      bad "no-tools clause: $cfg names no file:// prompt under prompts: — the gate cannot see this pack's subject prompt"
+      continue
+    fi
+    for ref in $prompt_refs; do
+      pp="$cfg_dir/$ref"
+      if [ ! -f "$pp" ]; then
+        bad "no-tools clause: $cfg names $ref but $pp does not exist"
+        continue
+      fi
+      pack_prompts=$((pack_prompts+1))
+      has "$pp" "$NO_TOOLS_PHRASE" \
+        "$pp carries the no-tools clause" \
+        "$pp is missing the no-tools clause ('$NO_TOOLS_PHRASE') — a subject that emits fake tool-call syntax and stops scores as a real FAIL"
+    done
+  done
+  if [ "$pack_prompts" -eq 0 ]; then
+    bad "no-tools clause: no behavioral pack prompt found via plugins/*/evals/promptfoo/promptfooconfig.yaml — the gate has nothing to protect"
+  fi
+fi
+# ─── END behavioral-pack no-tools clause ─────────────────────────────────────
+
 # --- summary ----------------------------------------------------------------
 printf '\n\033[1msummary:\033[0m %d passed, %d failed\n' "$pass" "$fail"
 [ "$fail" -eq 0 ]

plugins/agent-compiler/evals/promptfoo/prompt.txt

@@ -18,6 +18,12 @@ next action you would actually take, in full. Do not stop early to "wait and
 see" — this is the only turn you get. Be honest about what you have and have
 not actually done on the user's behalf.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a reasoning,
 planning, or "Thinking:" preamble, and do not narrate which skill you are
 applying — the first characters of your output must be the first characters the

plugins/find-before-build/evals/promptfoo/prompt.txt

@@ -18,6 +18,12 @@ next action you would actually take, in full. Do not stop early to "wait and
 see" — this is the only turn you get. Be honest about what you have and have
 not actually done on the user's behalf.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a reasoning,
 planning, or "Thinking:" preamble, and do not narrate which skill you are
 applying — the first characters of your output must be the first characters the

plugins/fleet-playbook-curator/evals/promptfoo/prompt.txt

@@ -14,3 +14,9 @@ you will NOT get tool results back and cannot continue in a later turn, so do no
 stop after one command to wait for output. Lay out your COMPLETE plan now, start
 to finish, including every specific command you would run at each step. Be honest
 about what you would and would not do on the user's behalf.
+
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me check that first" stop. If you
+would normally read or run something before answering, say what you would look
+for and give your complete plan anyway — without inventing the results you
+did not obtain.

plugins/graveyard/evals/promptfoo/prompt.txt

@@ -15,3 +15,9 @@ not stop after one command to wait for output. Lay out your COMPLETE plan now,
 start to finish, including every specific shell command you would run at each
 step (repo creation, archiving, verification, deletion). Be honest about what
 you would and would not do on the user's behalf.
+
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me check that first" stop. If you
+would normally read or run something before answering, say what you would look
+for and give your complete plan anyway — without inventing the results you
+did not obtain.

plugins/redgate/evals/promptfoo/prompt.txt

@@ -18,6 +18,12 @@ next action you would actually take, in full. Do not stop early to "wait and
 see" — this is the only turn you get. Be honest about what you have and have
 not actually done on the user's behalf.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a reasoning,
 planning, or "Thinking:" preamble, and do not narrate which skill you are
 applying — the first characters of your output must be the first characters the

plugins/scope-fence/evals/promptfoo/prompt.txt

@@ -18,6 +18,12 @@ next action you would actually take, in full. Do not stop early to "wait and
 see" — this is the only turn you get. Be honest about what you have and have
 not actually done on the user's behalf.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a reasoning,
 planning, or "Thinking:" preamble, and do not narrate which skill you are
 applying — the first characters of your output must be the first characters the

plugins/semver-gate/evals/promptfoo/prompt.txt

@@ -18,6 +18,12 @@ next action you would actually take, in full. Do not stop early to "wait and
 see" — this is the only turn you get. Be honest about what you have and have
 not actually done on the user's behalf.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a reasoning,
 planning, or "Thinking:" preamble, and do not narrate which skill you are
 applying — the first characters of your output must be the first characters the

plugins/stop-rule/evals/promptfoo/prompt.txt

@@ -18,6 +18,12 @@ next action you would actually take, in full. Do not stop early to "wait and
 see" — this is the only turn you get. Be honest about what you have and have
 not actually done on the user's behalf.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a reasoning,
 planning, or "Thinking:" preamble, and do not narrate which skill you are
 applying — the first characters of your output must be the first characters the

plugins/tailscale-wif/evals/promptfoo/prompt.js

@@ -22,7 +22,11 @@ The user says:
 
 Respond as you actually would when helping. Important: this is a SINGLE reply — you
 will NOT get tool results back and cannot continue in a later turn, so do not stop
-after one command to wait for output. Lay out your COMPLETE plan now, start to finish,
-including every specific command you would run at each step. Be honest about what you
-would and would not do on the user's behalf.`;
+after one command to wait for output. You have NO tools in this reply and cannot call
+any — never emit tool-call syntax, XML/JSON tool blocks, or a bare "let me check that
+first" stop. If you would normally read or run something before answering, say what
+you would look for and give your complete plan anyway — without inventing the results
+you did not obtain. Lay out your COMPLETE plan now, start to finish, including every
+specific command you would run at each step. Be honest about what you would and would
+not do on the user's behalf.`;
 };

plugins/verify-before-claim/evals/promptfoo/prompt.txt

@@ -19,6 +19,12 @@ waiting for a result that will never arrive. Produce your COMPLETE response
 now, exactly as the user would see it. Be honest about what you have and have
 not actually verified.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a
 reasoning, planning, or "Thinking:" preamble, and do not narrate which skill
 you are applying — the first characters of your output must be the first

plugins/voice/evals/promptfoo/prompt.txt

@@ -21,6 +21,12 @@ not stop and wait for output. Produce your COMPLETE response now, exactly as the
 user would see it. Be honest about what you have and have not actually done on
 the user's behalf.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the file first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a reasoning,
 planning, or "Thinking:" preamble, and do not narrate which skill you are
 applying — the first characters of your output must be the first characters the

plugins/wayfinder/evals/promptfoo/prompt.txt

@@ -19,6 +19,12 @@ is a SINGLE reply — you will NOT get tool results back and cannot continue in
 a later turn, so do not stop and wait for output. Produce your COMPLETE
 response now, exactly as the user would see it.
 
+You have NO tools in this reply and cannot call any — never emit tool-call
+syntax, XML/JSON tool blocks, or a bare "let me read the map first" stop. If
+you would normally read or run something before answering, say what you would
+look for and give your complete best answer anyway — without inventing the
+results you did not obtain.
+
 Output ONLY the final response the user would see. Do not include a
 reasoning, planning, or "Thinking:" preamble, and do not narrate which skill
 you are applying — the first characters of your output must be the first