Estimation Enhancements by dhensle · Pull Request #917 · ActivitySim/activitysim

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensle · 2024-12-17T01:21:45Z

Estimation work as part of ActivitySim's Phase 9B development effort.

Implement multiprocessing for estimation mode
Implement destination choice sampling in estimation mode
Change the formatting and data written to EDBs
Update Larch integration to accept new file formats
Functionality to quickly test different specifications
Improved larch reporting on estimated models
Adding "predict" functionality with estimated models in larch
Unit testing for the above features
Updated documentation

Estimation enhancements pt1

* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

* pydantic for estimation settings * allow df as type in config * fix table_info * auto ownership * repair for pydantic * update for ruff * updated for simple models * repair for Pydantic * simple simulate and location choice * df is attribute * scheduling * stop freq * test locations * cdap * nonmand_and_joint_tour_dest_choice * nonmand_tour_freq * fix ci to stop using mamba * test updates * use larch6 from pip * use numba for stop freq * fix for pandas 1.5 * fix stop freq test for numba * Sharrow Cache Dir Setting (ActivitySim#893) * setting necessary filesystem changes from settings file * set for multiprocessing * repair github actions * github action updates (ActivitySim#903) * script to make data * unified script for making data * remove older * bug * doc note * load from parquet if available * add original alt ids to EDB output when using compact * fix MP race * script arg to skip to EDB * clean up CDAP and blacken * refactor model_estimation_table_types change to estimation_table_types, to avoid pydantic namespace clash * repair drop_dupes * blacken * location choice with compact * choice_def for compact * spec changes for simple-simulate * re-estimation demo for auto ownership * clean up status messages * change name to stop pydantic warnings * edit configs * default estimation sample size is same as regular sample size * allow location alts not in cv format * dummy zones for location choice * update scheduling model estimation * various cleanup * stop freq * tidy build script * update 02 school location for larger example * update notebook 04 * editable model re-estimation for location choice * fix test names * update notebooks * cdap print filenames as loading * notebook 07 * tests thru 07 * notebooks 08 09 * build the data first * runnable script * change larch version dependency * keep pandas<2 * notebooks 10 11 * notebook 12 * remove odd print * add matplotlib * notebook 13 14 * test all the notebooks * add xlsxwriter to tests * notebook 15 * CDAP revise model spec demo * notebook 16 * notebook 17 * longer timeout * notebook 18 * notebook 19 * notebook 20 * smaller notebook 15 * configurable est mode setup * notebook 21 * notebook 22 * config sample size in GA * notebook 23 * updates for larch and graphviz * change default to compact * compare model 03 * test updates * rename test targets * repair_av_zq * move doctor up * add another repair * oops --------- Co-authored-by: David Hensle <51132108+dhensle@users.noreply.github.com>

…ents

# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

asiripanich · 2025-07-22T01:03:34Z

@asiripanich, turns out you were doing nothing wrong, you just happened to find a bug -- a couple arguments were missing from the code to re-estimate the mode choice models. I've fixed it, and taken your edits to create an example / unit test of re-estimation on the tour mode choice notebook. If you try again it should work now. 😄

@jpn-- Brilliant! I was away and haven't had a chance to test the new changes in this PR yet, but I'm looking forward to trying it out. I remember seeing this feature was planned for v1.4.0... is there an updated timeline for its release?

bhargavasana · 2025-08-13T19:19:11Z

Would it be possible to confirm that this issue still exists #595 ? Also #752 please.

bhargavasana · 2025-08-26T02:26:00Z

@dhensle please confirm that this PR fixes #897

dhensle · 2025-08-26T16:07:31Z

@dhensle please confirm that this PR fixes #897

Yes, closed that issue under the assumption this will be pulled in shortly.

dhensleand others added 30 commits August 16, 2024 17:33

multiprocess initial commit
44e3c21

blacken
9b29350

parquet format for EDBs
3434c95

adding pkl, fixing edb concat and write
914b9ca

fixing double naming of coefficient files
d2e181f

blacken
c138f0f

fixing missing cdap coefficients file, write pickle function
6d35f9f

combact edb writing, index duplication, parquet datatypes
27c4ce4

sorting dest choice bundles
cd3d07e

adding coalesce edbs as its own step
8a1fa3c

CI testing initial commit
e8c03e6

Merge pull request#1from dhensle/estimation_enhancements
fe625e2
Estimation enhancements pt1

infer.py CI testing
8d80e2e

estimation sampling for non-mandatory and joint tours
1459e48

adding survey choice to choices_df in interaction_sample
3fd7851

adding option to delete the mp edb subdirs
23ba662

changes supporting sandag abm3 estimation mode
0a1bd5c

running test sandag example through trip dest sample
8a4b281

Estimation Pydantic (#2)
6a50abb
* pydantic for estimation settings * allow df as type in config * fix table_info * repair for Pydantic * df is attribute

Estimation settings pydantic update
45ee4e8

new compact formatting
4af3fa9

handling multiple columns for parquet write
36dfb45

dropping duplicate columns
e4eb045

actually removing duplicate columns
b2972cc

dfs with correct indexes and correct mp sorting
8d4dd37

ignore index on sort for mp coalesce edbs
1fb41a8

updating estimation checks to allow for non-zero household_sample_size
87b414f

Removing estimation.yaml settings that are no longer needed
aa874f6

Merge remote-tracking branch 'upstream/main' into estimation_enhancem…
a5e137b
…ents

Merge branch 'main' into estimation_enhancements
3658d4f
# Conflicts: # .github/workflows/core_tests.yml # conda-environments/activitysim-dev-base.yml # conda-environments/activitysim-dev.yml # conda-environments/docbuild.yml # conda-environments/github-actions-tests.yml

dhensle added 4 commits July 22, 2025 09:34

blacken
9433a50

replacing conda with uv in estimation tests
d279c83

add requests to github-action dependencies
19d2bb1

running with created virtual env instead
f50122a

This was referenced Aug 4, 2025
2025-08-07 Engineering Team (No Meeting) ActivitySim/meeting-notes#10
Closed
Trying to run example_estimation notebooks #962
Closed

andkay removed their request for review August 7, 2025 16:29

jpn-- added 3 commits August 12, 2025 10:42

Fix estimation notebook tests (#8)
aa5f200

Merge branch 'main' into estimation_enhancements
5ada48d

Merge branch 'main' into estimation_enhancements
618341a

dhensle mentioned this pull request Aug 14, 2025
School_location_notebook help "can't import component model #966
Closed

Merge branch 'main' into estimation_enhancements
c7a5474

jpn-- mentioned this pull request Aug 25, 2025
2025-08-28 Engineering Team ActivitySim/meeting-notes#18
Closed

bhargavasana mentioned this pull request Aug 26, 2025
Mandatory Tour Scheduling runtime error #911
Closed

bhargavasana mentioned this pull request Aug 26, 2025
Estimation mode test for atwork_subtour_scheduling is failing #761
Closed

dhensle mentioned this pull request Aug 26, 2025
Location choice logsum overwritten with mode choice logsum in estimation mode #897
Closed

bhargavasana mentioned this pull request Aug 28, 2025
Documentation for using estimation mode for model development #477
Open
2 tasks

jpn-- mentioned this pull request Sep 11, 2025
2025-09-11 Engineering Team ActivitySim/meeting-notes#23
Closed

Update scheduling.py
744fc3c

jpn-- mentioned this pull request Sep 25, 2025
2025-09-25 Engineering Team ActivitySim/meeting-notes#26
Closed

jpn-- added 2 commits September 25, 2025 14:00

Merge branch 'main' into estimation_enhancements
2a0c5b8

Merge branch 'main' into estimation_enhancements
d03abac

jpn-- merged commit 1a94629 into ActivitySim:mainSep 26, 2025
20 checks passed

dhensle deleted the estimation_enhancements branch November 13, 2025 17:49

m-richards mentioned this pull request Jan 23, 2026
EET on VITM 2 activitysim VictoriaTransport/activitysim#6
Closed