Chat without conventions answers 2 of 18 questions on the wide corpus; with the six conventions written as one page of prose it answers 14 (#520). The conventions have to come from somewhere, and today there is nowhere to put them: not a class, not a mapping row, not a column of knowledge_bases.
Two halves, two provenances:
- What the schema states — what each table is a table of and what one row is, which columns are keys, the unit of a money column where a comment says so, what a status code means where a comment says so, the time axis, which two columns look alike and how they differ. Exploration reads the schema anyway; it can write this, and rewrite it each run.
- What the schema does not state — test rows excluded, which statuses count, which of two amount columns is the figure, whether net subtracts refunds. Exploration cannot write this (0 of 18 definitions on the same corpus), and must not guess it: a guessed convention in the prompt is followed as confidently as a right one, which is the automatic-confirmation failure in a new place.
So exploration writes a description and a list of questions — the conventions it needed and could not find, phrased so the owner answers each in a line. The model already knows what it does not know: asked for valid orders with no definition, it said in its own words that it was searching for one. A blank page nobody fills in becomes four questions anybody can answer.
Shape
Three columns on knowledge_bases (migration 0047), two writers:
data_description, data_questions — written by exploration, every run.
data_conventions — written by a person, through PATCH /kbs/{id}; exploration never touches it. Mixing them in one field would let the next run overwrite the answers.
Chat reads the description and the conventions into the system prompt, every turn, rather than relying on retrieval finding a document (the prose experiment hit on every question because every question was about a metric). Exploration reads the conventions into its own prompt, so a proposal can carry the filter and the unit conversion — which is the first thing that could move its 0 of 18.
Measured by
ask.mjs / mappings.mjs --conventions write the truth file’s conventions array through the PATCH. Three numbers: chat with the generated description alone; chat with description and conventions (against the prose 14/18); exploration coverage with conventions in its prompt (against 0/18).
Interface, after
A SettingsCard on the base settings: the description read-only with Regenerate, the conventions editable, the questions listed underneath so answering them is the way to write conventions. Follows web/DESIGN.md.
Chat without conventions answers 2 of 18 questions on the wide corpus; with the six conventions written as one page of prose it answers 14 (#520). The conventions have to come from somewhere, and today there is nowhere to put them: not a class, not a mapping row, not a column of
knowledge_bases.Two halves, two provenances:
So exploration writes a description and a list of questions — the conventions it needed and could not find, phrased so the owner answers each in a line. The model already knows what it does not know: asked for valid orders with no definition, it said in its own words that it was searching for one. A blank page nobody fills in becomes four questions anybody can answer.
Shape
Three columns on
knowledge_bases(migration0047), two writers:data_description,data_questions— written by exploration, every run.data_conventions— written by a person, throughPATCH /kbs/{id}; exploration never touches it. Mixing them in one field would let the next run overwrite the answers.Chat reads the description and the conventions into the system prompt, every turn, rather than relying on retrieval finding a document (the prose experiment hit on every question because every question was about a metric). Exploration reads the conventions into its own prompt, so a proposal can carry the filter and the unit conversion — which is the first thing that could move its 0 of 18.
Measured by
ask.mjs/mappings.mjs --conventionswrite the truth file’sconventionsarray through the PATCH. Three numbers: chat with the generated description alone; chat with description and conventions (against the prose 14/18); exploration coverage with conventions in its prompt (against 0/18).Interface, after
A
SettingsCardon the base settings: the description read-only with Regenerate, the conventions editable, the questions listed underneath so answering them is the way to write conventions. Followsweb/DESIGN.md.