Skip to content

Exploration describes the data and asks what it cannot tell #570

Description

@WaylandYang

Chat without conventions answers 2 of 18 questions on the wide corpus; with the six conventions written as one page of prose it answers 14 (#520). The conventions have to come from somewhere, and today there is nowhere to put them: not a class, not a mapping row, not a column of knowledge_bases.

Two halves, two provenances:

  • What the schema states — what each table is a table of and what one row is, which columns are keys, the unit of a money column where a comment says so, what a status code means where a comment says so, the time axis, which two columns look alike and how they differ. Exploration reads the schema anyway; it can write this, and rewrite it each run.
  • What the schema does not state — test rows excluded, which statuses count, which of two amount columns is the figure, whether net subtracts refunds. Exploration cannot write this (0 of 18 definitions on the same corpus), and must not guess it: a guessed convention in the prompt is followed as confidently as a right one, which is the automatic-confirmation failure in a new place.

So exploration writes a description and a list of questions — the conventions it needed and could not find, phrased so the owner answers each in a line. The model already knows what it does not know: asked for valid orders with no definition, it said in its own words that it was searching for one. A blank page nobody fills in becomes four questions anybody can answer.

Shape

Three columns on knowledge_bases (migration 0047), two writers:

  • data_description, data_questions — written by exploration, every run.
  • data_conventions — written by a person, through PATCH /kbs/{id}; exploration never touches it. Mixing them in one field would let the next run overwrite the answers.

Chat reads the description and the conventions into the system prompt, every turn, rather than relying on retrieval finding a document (the prose experiment hit on every question because every question was about a metric). Exploration reads the conventions into its own prompt, so a proposal can carry the filter and the unit conversion — which is the first thing that could move its 0 of 18.

Measured by

ask.mjs / mappings.mjs --conventions write the truth file’s conventions array through the PATCH. Three numbers: chat with the generated description alone; chat with description and conventions (against the prose 14/18); exploration coverage with conventions in its prompt (against 0/18).

Interface, after

A SettingsCard on the base settings: the description read-only with Regenerate, the conventions editable, the questions listed underneath so answering them is the way to write conventions. Follows web/DESIGN.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions