Document dGPredictor KEGG carry-over defect in the NAR 2026 Methods drafts

Papers/NAR_Update_2026/MANUSCRIPT.md

@@ -97,7 +97,9 @@ The 2020 release combined two thermodynamics sources: eQuilibrator as the primar
 
 **Group Contribution rebuilt from MFAToolkit.** The 2020 fallback GC method has been rebuilt from the MFAToolkit implementation against the current compound set, so the group definitions and parameter fits reflect the enlarged compound base rather than the 2020 snapshot. Coverage: [TBD].
 
-**dGpredictor retrained on ModelSEED.** dGpredictor (an ML-based ΔG estimator) has been retrained using the ModelSEED-curated compound structures as training data. This produces a ModelSEED-native estimator rather than one trained on a competing biochemistry database, and lets us report ΔG estimates for compounds the other three methods cannot cover. Training procedure and hyperparameters: [TBD]. Coverage: [TBD].
+**dGpredictor (original, KEGG-mediated): identifier carry-over defect discovered and corrected.** The original dGpredictor (Wang et al. 2021) integration, staged in 2023, mapped ModelSEED reactions to KEGG reaction equations via their KEGG aliases. An audit for this update found a stale-variable bug in the generating notebook: the alias-extraction loop never reset its holding variable between reactions, so every reaction without a KEGG alias silently inherited the id of the nearest preceding reaction that had one. Of 27,715 reactions carrying a stored dGpredictor record, 17,271 (62%) hold an estimate for a KEGG id absent from the reaction's aliases; a database-wide fingerprint test confirmed the carry-over mechanism for 100.0% of contaminated entries (zero exceptions), e.g. rxn00019 (2-nitropropane dioxygenase) carrying RuBisCO's R00024 energy. The contaminated subset shows null correlation with eQuilibrator and Group Contribution (r ≈ 0.00 vs 0.61/0.74 for the correctly-mapped subset), and 7,466 contaminated values had been promoted into canonical `deltag`/`deltagerr`/`reversibility` — favored by the promotion policy's lowest-uncertainty rule because they carry the copied reaction's deceptively tight error. The correction: extraction rewritten with a per-reaction reset, staging regenerated [TBD], an alias-validation guard added to the ingest script, and the 7,466 canonical triples re-derived from the repaired records. Full account in Methods draft `drafts/methods_multi_source_thermodynamics.md`.
+
+**dGpredictor retrained on ModelSEED.** dGpredictor (an ML-based ΔG estimator) has additionally been retrained using the ModelSEED-curated compound structures as training data. This produces a ModelSEED-native estimator rather than one trained on a competing biochemistry database, and lets us report ΔG estimates for compounds the other three methods cannot cover. It is keyed directly by ModelSEED reaction id, with no KEGG-alias detour, and was therefore unaffected by the carry-over defect above. Training procedure and hyperparameters: [TBD]. Coverage: [TBD].
 
 **OpenTECR integration.** OpenTECR is a community effort to standardize experimentally-measured thermodynamic constants; we integrate its values where a mapping between OpenTECR reaction identifiers and ModelSEED reaction IDs exists. Coverage: [TBD].
 
@@ -207,7 +209,7 @@ For protein-carrier cofactor standardisation, an earlier commit in this release
 
 *(Source: drafts/methods_multi_source_thermodynamics.md, Results portion — pending numbers.)*
 
-**[NUMBERS PENDING]** Per-source coverage (compounds with ΔfG′, reactions with accepted ΔrG′) across eQuilibrator, rebuilt Group Contribution, dGpredictor retrained on ModelSEED, and OpenTECR. Head-to-head ΔrG′ comparison (correlation, uncertainty distributions) mirroring the 2020 Figure 2. How per-reaction-class heuristic overlays change direction assignments relative to the source values alone.
+**[NUMBERS PENDING]** Per-source coverage (compounds with ΔfG′, reactions with accepted ΔrG′) across eQuilibrator, rebuilt Group Contribution, dGpredictor retrained on ModelSEED, and OpenTECR. Head-to-head ΔrG′ comparison (correlation, uncertainty distributions) mirroring the 2020 Figure 2 — for the original dGpredictor this must be computed from the post-repair records (see the KEGG carry-over correction in Methods); the pre-repair values understate the method's agreement because 62% of its records carried another reaction's energy. How per-reaction-class heuristic overlays change direction assignments relative to the source values alone.
 
 ### Reaction similarity
 

Papers/NAR_Update_2026/drafts/methods_multi_source_thermodynamics.md

@@ -2,7 +2,7 @@
 
 **Target section:** Materials and Methods, "Multi-source thermodynamics" (see `PAPER_2026_SKELETON.md`).
 **Guide reference:** `PAPER_2026_GUIDE.md` §6.
-**Status:** GC subsection filled in against the committed rebuild; eQuilibrator / dGpredictor / OpenTECR versions still `[TBD]`; direction-heuristic rule set still `[TBD]`.
+**Status:** GC subsection filled in against the committed rebuild; dGpredictor (original) subsection filled in against the 2026-08 KEGG carry-over audit — corrected regeneration not yet landed; eQuilibrator / dGpredictor / OpenTECR versions still `[TBD]`; direction-heuristic rule set still `[TBD]`.
 
 ---
 
@@ -20,7 +20,17 @@ The 2020 release combined two thermodynamics sources: eQuilibrator as the primar
 
 *Coverage.* [TBD compounds with numeric GC Δ<sub>f</sub>G; TBD reactions with numeric GC Δ<sub>r</sub>G′] — to be filled from the current DB snapshot.
 
-**dGpredictor retrained on ModelSEED.** dGpredictor (an ML-based Δ<sub>f</sub>G estimator) has been retrained using the ModelSEED-curated compound structures as training data. This produces a ModelSEED-native estimator rather than one trained on a competing biochemistry database, and lets us report Δ<sub>f</sub>G estimates for compounds the other three methods cannot cover. Training procedure and hyperparameters: [TBD]. Coverage: [TBD]. A quinone/quinol audit is outstanding: dGpredictor reportedly carries extreme error on these pairs (see `data/quinone_quinol_investigation_[TBD].md`), and the paper will note whether audit results warrant a correction pass or a per-class error flag.
+**dGpredictor (original, KEGG-mediated): a systematic identifier carry-over defect, discovered and corrected.** The database has carried Δ<sub>r</sub>G′ estimates from the original dGpredictor (Wang et al. 2021, *Metab Eng*, <a href="https://doi.org/10.1016/j.ymben.2021.10.013">10.1016/j.ymben.2021.10.013</a>) since 2023. Because the published dGpredictor operates on KEGG reaction equations, the integration was mediated by identifier mapping: each ModelSEED reaction's KEGG alias was extracted, the KEGG equation was scored, and the result was staged as `rxn → KEGG-R-id → {dG_mean, dG_uncer}` (kJ/mol) under `Biochemistry/Thermodynamics/dGPredictor/json_files/`, from which `Update_Reaction_dGPredictor_Energies.py` averages across a reaction's KEGG ids, converts to kcal/mol, and stores an additive `dGPredictor [energy, error, operator]` record. An audit for this update found that most of these staged mappings were wrong, in a way that was systematic, silent, and self-perpetuating; we document the defect, its forensic confirmation, and the correction here because 7,466 of the affected values had been promoted into the canonical `deltag` / `deltagerr` / `reversibility` fields.
+
+*The defect finding.* Of the 27,715 reactions carrying a stored dGpredictor record, 17,271 (62%) hold an estimate computed for a KEGG reaction id that is not among the reaction's aliases — neither on the reaction record nor in `Biochemistry/Aliases/Unique_ModelSEED_Reaction_Aliases.txt`, which agree on all 17,271 cases. The root cause is a stale-variable bug in the notebook that generated the 2023 staging files: the KEGG-alias extraction loop assigned its holding variable inside the per-alias scan but never reset it between reactions, so every reaction *without* a KEGG alias silently inherited the id of the most recent preceding reaction that had one (a bare `except` masked all but the first failure). The result is characteristic runs of contamination: rxn00019 (2-nitropropane dioxygenase, no KEGG alias) inherited R00024 — RuBisCO — from rxn00018 and stores its Δ<sub>r</sub>G′ of −8.63 ± 0.04 kcal/mol; rxn13478–rxn13481, four consecutive KEGG-less reactions, all inherited R09245 from rxn13477 and carry byte-identical −67.33 ± 2.29 values.
+
+*Forensic confirmation.* The carry-over mechanism was confirmed as the sole cause by a database-wide fingerprint test: for 100.0% of the 17,784 contaminated staged entries (zero exceptions), the staged KEGG id exactly equals the KEGG alias of the nearest preceding KEGG-bearing reaction in file order. A correlation split provides independent confirmation and quantifies the damage: the correctly-mapped subset agrees with the other estimators as expected (Pearson r = 0.61 vs eQuilibrator, n = 9,783; r = 0.74 vs Group Contribution, n = 9,278), while the contaminated subset shows the null correlation of randomly-assigned energies (r = −0.001 and −0.006 respectively) — explaining the method's previously puzzling low overall agreement with the other thermodynamic sources.
+
+*Why it persisted.* The defect survived every pipeline re-run because the staged files are the interface: `Update_Reaction_dGPredictor_Energies.py` (invoked by `Rerun_Thermodynamics.sh`) consumes the staged `rxn → KEGG-id` mapping as ground truth and never re-validates the staged id against the reaction's aliases, so each rerun faithfully re-propagated the 2023 contamination. Worse, the contaminated records carry the deceptively small uncertainties of whatever reaction they were copied from, so the promotion policy's within-tier lowest-uncertainty rule (see `Promote_Reaction_Thermodynamics_to_Canonical.py`) systematically favored them: 7,466 contaminated dGpredictor values were promoted into the canonical `deltag` / `deltagerr` / `reversibility` of reactions whose canonical energy was otherwise undefined.
+
+*The correction.* The extraction was rewritten with a per-reaction reset of the alias-holding variable (`dG_prediction_modelseed_dev_branch_file_run.py` in the dGPredictor repository), which restricts staged entries to reactions that genuinely carry the KEGG alias. The staging JSONs are being regenerated under the corrected extraction [TBD: land regenerated files and updated coverage], `Update_Reaction_dGPredictor_Energies.py` gains an alias-validation guard so a staged KEGG id not present among the reaction's aliases is dropped rather than ingested, and the 7,466 contaminated canonical triples are re-derived by re-running the promotion pass over the repaired per-source records [TBD: confirm post-repair promotion counts]. Corrected coverage: [TBD].
+
+**dGpredictor retrained on ModelSEED.** dGpredictor (an ML-based Δ<sub>f</sub>G estimator) has additionally been retrained using the ModelSEED-curated compound structures as training data. This produces a ModelSEED-native estimator rather than one trained on a competing biochemistry database, and lets us report Δ<sub>f</sub>G estimates for compounds the other three methods cannot cover. Because the retrained model is keyed directly by ModelSEED reaction id (`modelseed_retrained_dG.json`) with no KEGG-alias detour, it is structurally immune to — and was unaffected by — the identifier carry-over defect above; the two dGpredictor variants are stored as separate per-source records (`dGPredictor` and `dGPredictor-ModelSEED`). Training procedure and hyperparameters: [TBD]. Coverage: [TBD]. A quinone/quinol audit is outstanding: dGpredictor reportedly carries extreme error on these pairs (see `data/quinone_quinol_investigation_[TBD].md`), and the paper will note whether audit results warrant a correction pass or a per-class error flag.
 
 **OpenTECR integration.** OpenTECR is a community effort to standardize experimentally-measured thermodynamic constants; we integrate its values where a mapping between OpenTECR reaction identifiers and ModelSEED reaction IDs exists. Coverage: [TBD].
 
@@ -38,3 +48,5 @@ The 2020 release combined two thermodynamics sources: eQuilibrator as the primar
 - The heuristic overlay rule set needs codifying as a repo script (probably a YAML or TSV file listing patterns and their direction assignments) before the paper's claim that overlays are "applied globally" is defensible. The electron-carrier universal per-electron anchor prototype (Fdx / Trx / Flx / Grx, see `Scripts/Thermodynamics/Update_Compound_GroupContribution_Energies.py` future commits) is one candidate rule-set component.
 - Whether OpenTECR gets its own Methods paragraph or is folded into the eQuilibrator refresh is an open decision (see `PAPER_2026_PLAN.md` open decision #2).
 - Quinone/quinol dGpredictor error audit needs execution before we can defensibly report that source's coverage numbers for reactions using these carriers.
+- The dGpredictor KEGG carry-over correction is only partially landed: the corrected extraction script exists (in the dGPredictor repo), but the regenerated staging JSONs, the alias-validation guard in `Update_Reaction_dGPredictor_Energies.py`, and the re-promotion pass repairing the 7,466 contaminated canonical triples all still need to land in `dev` before the Methods text's "corrected" claims are defensible. A regression test pinning a few known carry-over cases (e.g. rxn00019 must not carry R00024's energy) would guard against a re-upload of stale staging files.
+- The Results head-to-head Δ<sub>r</sub>G′ comparison (correlation figure mirroring 2020 Figure 2) must be computed from the *repaired* dGpredictor records; the pre-repair correlations quoted in this section (0.61/0.74 clean vs ≈0 contaminated) are audit evidence, not the publishable coverage comparison.