Explicit chunking on all interaction simulate models

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.

[dhensle]

Adds the option of explicit chunking to all interaction simulate models that were not already hooked-up. These include destination choice, location choice, and scheduling.

Also implemented a feature where the explicit_chunk setting can be less than 1. If less than one, it specifies the fraction. So explicit_chunk: 0.1 would mean that there would be 10 chunks. If greater than 1, explicit_chunk remains the total number of rows in the chooser table.

[dhensle]

Sharing some testing results for posterity.

Used TransLink's model at a 10% sample size. No chunking looks like this:

I then set chunk_training to explicit with the following explicit_chunk settings for submodels:

• workplace_location: 0.5
• mandatory_tour_scheduling: 0.25
• non_mandatory_tour_destination: 0.5
• non_mandatory_tour_scheduling: 0.5
• trip_destination: 0.5

(notice the y-axis scale difference from the above plot)

Run time for no chunking was 113.9 minutes and for explicit chunking was 115 minutes -- very minimal increase in runtime.

[jpn--]

This looks great. Couple minor changes to simplify.

[jpn--]

The value of explicit_and_odd_num_choosers is not needed. It's perfectly fine for some multiple of rows_per_chunk to overrun the end of the choosers by a bit, slicing beyond the end of the range. If it were needed, checking for odd wouldn't be enough, we'd need to adjust based on the inverse of the number of chunks (e.g. 0.25 won't line up unless the total is divisible by 4 not 2)

[dhensle]

Please take a look at the changes in this commit: dhensle@1ed94c5

I was hitting the assert statement for the alts, (not the choosers) which prompted me to try to adjust the rows_per_chunk. But good point about odd not being good enough. I removed this functionality and replace the assert statement with a simple check on overflow of the alt index. I think the solution in the above commit fixes the issue (it runs successfully), but suggest you take a look. Thanks!

[jpn--]

We don't need to update the rows_per_chunk here, as noted above we can overrun the end of the choosers and be fine.

[dhensle]

see above response.