Deprecate SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS settings by Copilot · Pull Request #1094 · ActivitySim/activitysim

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Copilot · 2026-07-30T21:05:44Z

SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS were memory hacks that manually pruned chooser tables before merging with alternatives. util.drop_unused_columns (on by default via compute_settings.drop_unused_columns) now does this automatically from the spec, so the manual lists are redundant — and a well-known gotcha, since omitting a column used by a spec silently breaks the model.

Settings

Both settings are now typed Any | None with a mode="before" field validator that emits a DeprecationWarning and discards the value. Existing configs continue to load rather than tripping extra="forbid".
Applies to TourLocationComponentSettings, TourModeComponentSettings, TourSchedulingSettings, and SchoolEscortSettings.

Component code

Removed the # FIXME - MEMORY HACK column subsetting (and the associated # Drop this when PR #1017 is mergedhousehold_id workarounds) from location_choice, util/tour_destination, util/tour_od, util/tour_scheduling, and school_escorting.
Deleted the now-unreachable helpers logsums.filter_chooser_columns and expressions.filter_chooser_columns.

Configs / docs / tests

Stripped both settings from the 46 bundled example config files.
New activitysim/core/test/test_deprecated_chooser_columns.py covering the warning per settings class and the no-warning-when-omitted case; updated the two existing tests that supplied the setting.
Change log entry under "Upcoming Changes".

>>>TourModeComponentSettings.model_validate(
... {"SPEC": "tour_mode_choice.csv", "LOGSUM_CHOOSER_COLUMNS": ["age"]}
... ).LOGSUM_CHOOSER_COLUMNSDeprecationWarning: LOGSUM_CHOOSER_COLUMNSisdeprecatedandnolongerused,
unusedcolumnsarenowdroppedautomaticallyNone

Users should remove these keys from their configs; no other action is required.

FixesSimulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016

jpn-- · 2026-07-30T21:36:59Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T21:38:07Z

@copilot, address review comments on this PR

Copilot · 2026-07-30T21:42:57Z

activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During househ...

Fixed in 71f88ba by taking the second option: _interaction_sample no longer bypasses pruning when trace targets exist. It now always runs util.drop_unused_columns on both choosers and alternatives when compute_settings.drop_unused_columns is enabled, adding the trace slicer id columns (person_id, proto_person_id, proto_household_id; household_id is already always retained by drop_unused_columns) to the protected list so interaction_trace_rows can still identify traced rows.

The full, unpruned choosers and alternatives tables are still written to the trace output earlier in the function, so nothing is lost from the trace except extra unused columns in the interaction_df dump. Estimation paths are untouched — interaction_sample passes estimator=None to utility evaluation and the other call sites that guard on estimator is None were not changed.

jpn--

• activitysim/abm/models/util/tour_destination.py:697 — The change merges all persons_merged columns into tours instead of retaining the previous narrow chooser-column selection. During household tracing, interaction_sample disables automatic chooser-column pruning, then cross-joins these wide choosers with alternatives. Because persons_merged can include household, land-use, accessibility, and disaggregate-accessibility fields, destination sampling can regress to a large chooser×alternative memory footprint and potentially OOM realistic traced runs.

Retain a safe explicit chooser-column subset for trace/estimation paths, or change pruning to preserve only trace-required fields rather than bypassing pruning completely.

jpn-- · 2026-07-30T23:41:54Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn-- · 2026-07-30T23:42:36Z

+ This setting is now obsolete and does nothing. Its functionality has been
+ replaced by :func:`activitysim.core.util.drop_unused_columns`.
+
+ .. deprecated:: 1.4


The current release is v1.5, so this will be officially deprecated as of the next release (1.6)

jpn--

This looks good to me. The SANDAG test is failing due to the bug in at-work tour logsums.

jpn-- · 2026-08-04T01:50:14Z

- choosers[ORIG_TAZ] = network_los.map_maz_to_taz(choosers[orig_maz])
+ # This is the TAZ for the configured tour origin. A wider chooser table
+ # may already contain a same-named home TAZ, which is incorrect for models
+ # such as at-work subtour destination choice.


This looks like the AI (here, OpenAI's Sol) found and fixed a mostly unrelated bug.

jpn-- · 2026-08-04T02:05:57Z

@dhensle, @janzill I don't think a comprehensive review from both of you here is totally necessary, but I know each of you has been in some of these files recently (and this is a re-do of David's original PR from some time ago) so I want to give you an opportunity to comment / review before I finalize and merge this.

janzill

this all looks good, no suggestions but one comment: Sol did well finding that at-work origin bug.

dhensle

This looks good to me too. Glad to see copilot kept the traceable id columns which I think has previously been an issue that was also fixed as part of this PR. This is not tested as part of this, but we should really add some better tracing test coverage. I think that is better done in a separate PR.

Initial plan
21254e5

CopilotAI assigned Copilot and jpn--Jul 30, 2026

Copilotstarted work on behalf of jpn--July 30, 2026 21:06 View session

CopilotAI linked an issue Jul 30, 2026 that may be closed by this pull request
Simulate Chooser Columns and Logsum Chooser Columns Settings Not needed #1016
Closed

Deprecate SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS settings
9935b74

CopilotAI changed the title ~~[WIP] Deprecate SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS settings~~Deprecate SIMULATE_CHOOSER_COLUMNS and LOGSUM_CHOOSER_COLUMNS settingsJul 30, 2026

CopilotAI requested a review from jpn--July 30, 2026 21:18

Copilotfinished work on behalf of jpn--July 30, 2026 21:18

Copilotstarted work on behalf of jpn--July 30, 2026 21:38 View session

Drop unused columns in interaction_sample even when tracing
c53002f

Copilotfinished work on behalf of jpn--July 30, 2026 21:43

jpn-- requested changes Jul 30, 2026
View reviewed changes

Fix chooser column deprecation regressions
0f5234e

jpn-- approved these changes Aug 4, 2026
View reviewed changes

jpn-- requested review from dhensle and janzill August 4, 2026 02:03

jpn-- marked this pull request as ready for review August 4, 2026 02:06

jpn-- added this to Phase 11 Aug 4, 2026

jpn-- moved this to Under Review in Phase 11 Aug 4, 2026

This was referenced Aug 4, 2026
Incorrect origin TAZ for non-home tours in 2 zone models #1099
Open
Access to global constants and removing chooser col filtering #1017
Closed

janzill approved these changes Aug 5, 2026
View reviewed changes

Merge branch 'main' into copilot/deprecate-chooser-columns-settings
315dabc

jpn-- mentioned this pull request Aug 13, 2026
2026-08-13 Engineering Team ActivitySim/meeting-notes#129
Open

dhensle approved these changes Aug 13, 2026
View reviewed changes

jpn-- mentioned this pull request Aug 13, 2026
Add tracing regression coverage for automatic chooser-column pruning #1105
Open

jpn-- merged commit 131f09a into mainAug 13, 2026
1 check passed

github-project-automationBot moved this from Under Review to Done in Phase 11 Aug 13, 2026

jpn-- deleted the copilot/deprecate-chooser-columns-settings branch August 13, 2026 19:34

jpn-- mentioned this pull request Aug 19, 2026
Update regressions for corrected at-work origin TAZ ActivitySim/sandag-abm3-example#46
Merged