Uh oh!
There was an error while loading. Please reload this page.
internal: stop toposort panicking when a dependency key is repeated - #6250
internal: stop toposort panicking when a dependency key is repeated#6250prql-bot wants to merge 3 commits into
toposort panicking when a dependency key is repeated#6250Conversation
prql-bot
left a comment
There was a problem hiding this comment.
Traced the fix against the merged tree and it holds up. index collapses repeated keys to the last declaration while dag keeps one entry per element of dependencies, so nodes was the only vector sized off the deduplicated count — the panic and the while spin both follow from that single mismatch, and sizing off dependencies.len() closes both. With the bound corrected the while wrapper really is dead: 0..dependencies.len() reaches every node and visit short-circuits on done, so order lands at exactly dependencies.len() in one pass. repeated_key's expected ["a", "c", "a"] matches what the code produces (dag == [[2], [], []], order [2, 0, 1]), and the reasoning about why is in the test's own comment rather than left for the next reader to re-derive — good.
Ran cargo test -p prqlc --lib -- utils::toposort: 6 passed. The full check rollup on this commit is green.
One adjacent observation, not blocking and not necessarily this PR's job — see the inline note.
| order: Vec::with_capacity(dependencies.len()), | ||
| }; | ||
| if let Some(start) = start.map(|s| index.get(s).unwrap()) { |
There was a problem hiding this comment.
This line has the same shape as the bug being fixed: a pub function that panics on an input that violates an undocumented precondition. index.get(s).unwrap() aborts when start names a key that isn't in dependencies, which is the same class of caller-visible panic the PR body argues matters here (utils is pub under the default-on cli feature).
I'm deliberately not attaching a suggestion, because the fix isn't mechanical. Propagating with ? would return None, which the function already uses to mean "there's a cycle" — and toposort_tables does toposort(...).unwrap(), so an unknown start key would still panic, just with a worse message than today's. So the choice is between widening the return type, documenting the precondition and keeping the panic as an assertion, or accepting the conflation. That's a semantic call worth a maintainer's opinion rather than something to fold in silently.
Related, and cheaper: toposort has no doc comment at all, and this PR makes its contract subtler — repeated keys are now permitted, later declarations win for edge resolution, and the output has one entry per input element, so it can contain the same key twice. Worth a few lines on the function, since the only place that's written down right now is inside a test.
There was a problem hiding this comment.
Took the doc half in 4f0d4a2 — toposort now carries a doc comment stating the contract this PR changes: one output element per input element rather than per distinct key, repeated keys permitted with the last declaration winning for edge resolution, dependencies on undeclared keys ignored, and start limiting the output to what's reachable from that key. That was only recorded inside repeated_key's comment before.
Left the semantic call alone, as you framed it. The doc comment records the precondition under a # Panics section and notes why it can't currently be reported through the return type (None already means "there's a cycle"), so the option is documented rather than decided — widening the return type stays open and would edit the same two places.
The one thing I did change on that line is the panic's message. The bare unwrap aborted with the generic "called Option::unwrap() on a None value", which says nothing about what went wrong; it's now an expect reading:
`start` must name a key declared in `dependencies`
Behaviour is identical, only the message differs. Key has no Debug bound, so the offending key itself can't be included without widening the signature.
Verified with cargo test -p prqlc --lib -- utils::toposort (6 passed), cargo clippy -p prqlc --lib -- -D warnings (clean), and cargo fmt -p prqlc -- --check.
Repeated keys are now permitted, later declarations win for edge resolution, and the output has one entry per input element — none of which was written down outside a test. Also labels the existing `start` panic with an `expect` message instead of a bare `unwrap`; the behaviour is unchanged.
prql-bot
left a comment
There was a problem hiding this comment.
Checked the new doc comment claim by claim against the code, since a contract statement on a pub function is the kind of prose that goes stale silently. Three of the four hold: edges resolving to the last declaration follows from index being a HashMap collected in order (later inserts overwrite); dependencies on undeclared keys are dropped by the flat_map over index.get(d); and start limiting the output to the reachable entries is what with_root already pins.
The fourth — None if the dependencies contain a cycle — isn't true when start is Some. Inline suggestion for it.
The expect message reads well and stays out of second person. Leaving the semantic question on that line where you put it — it's still a maintainer call, and restating it here wouldn't move it.
Uh oh!
There was an error while loading. Please reload this page.
With `start` set, `visit` is only entered from that node, so a cycle in an unreachable part of `dependencies` is never seen and the function returns `Some`. The doc comment claimed `None` for any cycle. Adds a test pinning the behaviour.
toposortbuildsdagwith one node per element ofdependencies, but sizednodesbyindex.len()— the count of distinct keys. The two are equal only when every key is unique. When a key repeats,nodesis short, andvisit'sself.nodes.get_mut(n).unwrap()panics on any node index past the end. A different arrangement of the same input hangs instead: the driver loop spunwhile order.len() < dependencies.len()over a range that could never push that many entries, so it retried a completed sort forever.The fix sizes
nodesandorderbydependencies.len(), matchingdag. With the bound corrected, one pass over every node completes the sort —visitreturns immediately for a node already marked done — so thewhilewrapper is redundant and is dropped.toposort_tablesinsemantic/lowering.rsbuilds its input from aHashMapand always passesSome(main_table), so no compilation path reaches this. It matters becauseutilsispubunder theclifeature, which is on by default, makingprqlc::utils::toposortpublic API of the crate as published.Follow-up from review:
toposortnow also carries a doc comment stating its contract — one output element per input element rather than per distinct key, repeated keys permitted with the last declaration winning for edge resolution, undeclared dependency keys ignored, and thestartprecondition under# Panics. The bareunwrapon an unknownstartkey keeps its behaviour but gains anexpectmessage; whether that panic should instead be reportable through the return type is left as an open question on the review thread. A later review round found the doc's cycle claim too strong — withstartset, only the reachable portion is visited, so a cycle elsewhere goes undetected — so the sentence is qualified andcycle_unreachable_from_rootpins it.Regression test
repeated_keypanics attoposort.rs:61onmainand passes with the fix.cargo test -p prqlcis green (608 tests), as iscargo clippy -p prqlc --lib -- -D warnings. No changelog entry — no compiler behaviour changes, matching #6190.Why the test expects `["a", "c", "a"]`
For
[("c", vec!["a"]), ("a", vec![]), ("a", vec![])],indexmapsato the last entry that declares it, index 2, sodagis[[2], [], []]. Visiting node 0 recurses into node 2 first, giving order[2, 0, 1]— that is, theathatcactually resolves to, thenc, then the shadoweda, which nothing depends on. Every input entry appears exactly once and the resolved dependency precedes its dependent.The hang variant isn't covered by a test on purpose: reproducing it means a test that never returns, which would tie up a CI job rather than fail it. Removing the
whilemakes it unreachable, andrepeated_keycovers the loop bound that caused both.task prqlc:pull-request, the gateCLAUDE.mdnames, could not run —cargo-instais blocked in the tend sandbox. This is the known problem from #6235;cargo test -p prqlcand the clippy run above are what was actually executed.