External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jpn-- wants to merge 8 commits into from
Closed

External regions #858

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
915e8d7
test external regional model examples
jpn-- Apr 23, 2024
3038256
cache buster
jpn-- Apr 23, 2024
3bdc1a4
optional variable doc
jpn-- Apr 24, 2024
b876e9c
fix conda cache dirs
jpn-- Apr 24, 2024
f578e72
Merge branch 'main' into external-regions
jpn-- Apr 24, 2024
9c282f0
Merge branch 'main' into external-regions
jpn-- Apr 24, 2024
e41054f
Merge branch 'main' into external-regions
jpn-- Apr 25, 2024
79cd6a2
Merge branch 'main' into external-regions
jpn-- May 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 46 additions & 63 deletions .github/workflows/core_tests.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
- '*'

env:
CACHE_NUMBER: 0 # increase to reset cache manually
CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
name: linux-64-py${{ matrix.python-version }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup Mambaforge
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
- name: Set cache date for year and month
run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

- uses: actions/cache@v3
- uses: actions/cache@v4
with:
path: /usr/share/miniconda3/envs/asim-test
key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
- name: Update environment
run: |
mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
mamba install --yes \
"psutil=5.9.5" \
"pydantic=2.6.1" \
"pypyr=5.8.0" \
"pytables=3.6.1" \
"pytest-cov" \
"pytest-regressions=2.5.0" \
"scikit-learn=1.2.2" \
"sharrow>=2.6.0" \
"simwrapper=1.8.5" \
"xarray=2023.2.0" \
"zarr=2.14.2" \
"zstandard=0.21.0"
if: steps.cache.outputs.cache-hit != 'true'

- name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
include:
- os: macos-latest
label: macOS
prefix: /Users/runner/miniconda3/envs/asim-test
python-version: "3.10"

- os: windows-latest
label: win-64
prefix: C:\Miniconda3\envs\asim-test
python-version: "3.10"

defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
name: ${{ matrix.label }}
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup Mambaforge
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
- name: Set cache date for year and month
run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

- uses: actions/cache@v3
- uses: actions/cache@v4
with:
path: ${{ matrix.prefix }}
path: ${{ env.CONDA }}/envs
key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
builtin_regional_models:
needs: foundation
env:
mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
python-version: "3.10"
label: linux-64
strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
name: ${{ matrix.region }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup Mambaforge
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
- name: Set cache date for year and month
run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

- uses: actions/cache@v3
- uses: actions/cache@v4
with:
path: ${{ env.mamba-env-prefix }}
path: ${{ env.CONDA }}/envs
key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
external_regional_models:
needs: foundation
env:
mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
python-version: "3.10"
label: linux-64
strategy:
matrix:
region:
- prototype_mtc
- prototype_psrc_in_development
include:
- region: Standard 1-Zone Example (MTC)
region-org: ActivitySim
region-repo: activitysim-prototype-mtc
region-branch: extended
- region: Standard 2-Zone Example (SANDAG)
region-org: ActivitySim
region-repo: sandag-abm3-example
region-branch: main
fail-fast: false
defaults:
run:
shell: bash -l {0}
name: ${{ matrix.region }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Checkout ActivitySim
uses: actions/checkout@v4

- name: Setup Mambaforge
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
- name: Set cache date for year and month
run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

- uses: actions/cache@v3
- uses: actions/cache@v4
with:
path: |
${{ env.mamba-env-prefix }}
${{ env.CONDA }}/envs
~/.cache/ActivitySim
key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
id: cache

- name: Update environment
run: |
mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
mamba install --yes \
"psutil=5.9.5" \
"pydantic=2.6.1" \
"pypyr=5.8.0" \
"pytables=3.6.1" \
"pytest-cov" \
"pytest-regressions=2.5.0" \
"scikit-learn=1.2.2" \
"sharrow>=2.6.0" \
"simwrapper=1.8.5" \
"xarray=2023.2.0" \
"zarr=2.14.2" \
"zstandard=0.21.0"
if: steps.cache.outputs.cache-hit != 'true'

- name: Install activitysim
# installing without dependencies is faster, we trust that all needed dependencies
# are in the conda environment defined above. Also, this avoids pip getting
# confused and reinstalling tables (pytables).
run: |
python -m pip install -e . --no-deps
python -m pip install . --no-deps

- name: Conda checkup
run: |
mamba info -a
mamba list

- name: Checkout Example
uses: actions/checkout@v4
with:
repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
ref: '${{ matrix.region-branch }}'
path: '${{ matrix.region-repo }}'

- name: Test ${{ matrix.region }}
run: |
python -m activitysim test ${{ matrix.region }}
cd ${{ matrix.region-repo }}/test
python -m pytest .

random_seed_generation:
needs: foundation
env:
mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
python-version: "3.10"
label: linux-64
defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup Mambaforge
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
- name: Set cache date for year and month
run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

- uses: actions/cache@v3
- uses: actions/cache@v4
with:
path: ${{ env.mamba-env-prefix }}
path: ${{ env.CONDA }}/envs
key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
estimation_mode:
needs: foundation
env:
mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
python-version: "3.10"
label: linux-64
defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
name: estimation_mode_test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup Mambaforge
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
- name: Set cache date for year and month
run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

- uses: actions/cache@v3
- uses: actions/cache@v4
with:
path: ${{ env.mamba-env-prefix }}
path: ${{ env.CONDA }}/envs
key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
run:
shell: bash -l {0}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: "3.10"
- name: Install dependencies
uses: conda-incubator/setup-miniconda@v2
uses: conda-incubator/setup-miniconda@v3
with:
miniforge-variant: Mambaforge
miniforge-version: latest
Expand Down
12 changes: 11 additions & 1 deletion docs/dev-guide/using-sharrow.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
`@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

`@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

jpn--
wants to merge
8
commits into

from

 Closed


External regions
#858
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
915e8d7

test external regional model examples


jpn-- Apr 23, 2024
3038256

cache buster


jpn-- Apr 23, 2024
3bdc1a4

optional variable doc


jpn-- Apr 24, 2024
b876e9c

fix conda cache dirs


jpn-- Apr 24, 2024
f578e72

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
9c282f0

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
e41054f

Merge branch 'main' into external-regions


jpn-- Apr 25, 2024
79cd6a2

Merge branch 'main' into external-regions


jpn-- May 21, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


109 changes: 46 additions & 63 deletions
.github/workflows/core_tests.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
 - '*'

env:
 CACHE_NUMBER: 0 # increase to reset cache manually
 CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
 foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
 name: linux-64-py${{ matrix.python-version }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: /usr/share/miniconda3/envs/asim-test
 key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
 include:
 - os: macos-latest
 label: macOS
 prefix: /Users/runner/miniconda3/envs/asim-test
 python-version: "3.10"

 - os: windows-latest
 label: win-64
 prefix: C:\Miniconda3\envs\asim-test
 python-version: "3.10"

 defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
 name: ${{ matrix.label }}
 runs-on: ${{ matrix.os }}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ matrix.prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
 builtin_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
 external_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
 matrix:
 region:
 - prototype_mtc
 - prototype_psrc_in_development
 include:
 - region: Standard 1-Zone Example (MTC)
 region-org: ActivitySim
 region-repo: activitysim-prototype-mtc
 region-branch: extended
 - region: Standard 2-Zone Example (SANDAG)
 region-org: ActivitySim
 region-repo: sandag-abm3-example
 region-branch: main
 fail-fast: false
 defaults:
 run:
 shell: bash -l {0}
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - name: Checkout ActivitySim
 uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: |
 ${{ env.mamba-env-prefix }}
 ${{ env.CONDA }}/envs
 ~/.cache/ActivitySim
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
 # installing without dependencies is faster, we trust that all needed dependencies
 # are in the conda environment defined above. Also, this avoids pip getting
 # confused and reinstalling tables (pytables).
 run: |
 python -m pip install -e . --no-deps
 python -m pip install . --no-deps

 - name: Conda checkup
 run: |
 mamba info -a
 mamba list

 - name: Checkout Example
 uses: actions/checkout@v4
 with:
 repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
 ref: '${{ matrix.region-branch }}'
 path: '${{ matrix.region-repo }}'

 - name: Test ${{ matrix.region }}
 run: |
 python -m activitysim test ${{ matrix.region }}
 cd ${{ matrix.region-repo }}/test
 python -m pytest .

 random_seed_generation:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
 estimation_mode:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
 name: estimation_mode_test
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
 run:
 shell: bash -l {0}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4
 with:
 fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
 - name: Set up Python 3.10
 uses: actions/setup-python@v2
 with:
 python-version: "3.10"
 - name: Install dependencies
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand Down

12 changes: 11 additions & 1 deletion
docs/dev-guide/using-sharrow.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
 `@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

 `@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

jpn--
wants to merge
8
commits into

from

 Closed


External regions
#858
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
915e8d7

test external regional model examples


jpn-- Apr 23, 2024
3038256

cache buster


jpn-- Apr 23, 2024
3bdc1a4

optional variable doc


jpn-- Apr 24, 2024
b876e9c

fix conda cache dirs


jpn-- Apr 24, 2024
f578e72

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
9c282f0

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
e41054f

Merge branch 'main' into external-regions


jpn-- Apr 25, 2024
79cd6a2

Merge branch 'main' into external-regions


jpn-- May 21, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


109 changes: 46 additions & 63 deletions
.github/workflows/core_tests.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
 - '*'

env:
 CACHE_NUMBER: 0 # increase to reset cache manually
 CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
 foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
 name: linux-64-py${{ matrix.python-version }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: /usr/share/miniconda3/envs/asim-test
 key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
 include:
 - os: macos-latest
 label: macOS
 prefix: /Users/runner/miniconda3/envs/asim-test
 python-version: "3.10"

 - os: windows-latest
 label: win-64
 prefix: C:\Miniconda3\envs\asim-test
 python-version: "3.10"

 defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
 name: ${{ matrix.label }}
 runs-on: ${{ matrix.os }}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ matrix.prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
 builtin_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
 external_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
 matrix:
 region:
 - prototype_mtc
 - prototype_psrc_in_development
 include:
 - region: Standard 1-Zone Example (MTC)
 region-org: ActivitySim
 region-repo: activitysim-prototype-mtc
 region-branch: extended
 - region: Standard 2-Zone Example (SANDAG)
 region-org: ActivitySim
 region-repo: sandag-abm3-example
 region-branch: main
 fail-fast: false
 defaults:
 run:
 shell: bash -l {0}
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - name: Checkout ActivitySim
 uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: |
 ${{ env.mamba-env-prefix }}
 ${{ env.CONDA }}/envs
 ~/.cache/ActivitySim
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
 # installing without dependencies is faster, we trust that all needed dependencies
 # are in the conda environment defined above. Also, this avoids pip getting
 # confused and reinstalling tables (pytables).
 run: |
 python -m pip install -e . --no-deps
 python -m pip install . --no-deps

 - name: Conda checkup
 run: |
 mamba info -a
 mamba list

 - name: Checkout Example
 uses: actions/checkout@v4
 with:
 repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
 ref: '${{ matrix.region-branch }}'
 path: '${{ matrix.region-repo }}'

 - name: Test ${{ matrix.region }}
 run: |
 python -m activitysim test ${{ matrix.region }}
 cd ${{ matrix.region-repo }}/test
 python -m pytest .

 random_seed_generation:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
 estimation_mode:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
 name: estimation_mode_test
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
 run:
 shell: bash -l {0}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4
 with:
 fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
 - name: Set up Python 3.10
 uses: actions/setup-python@v2
 with:
 python-version: "3.10"
 - name: Install dependencies
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand Down

12 changes: 11 additions & 1 deletion
docs/dev-guide/using-sharrow.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
 `@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

 `@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

jpn--
wants to merge
8
commits into

from

 Closed


External regions
#858
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
915e8d7

test external regional model examples


jpn-- Apr 23, 2024
3038256

cache buster


jpn-- Apr 23, 2024
3bdc1a4

optional variable doc


jpn-- Apr 24, 2024
b876e9c

fix conda cache dirs


jpn-- Apr 24, 2024
f578e72

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
9c282f0

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
e41054f

Merge branch 'main' into external-regions


jpn-- Apr 25, 2024
79cd6a2

Merge branch 'main' into external-regions


jpn-- May 21, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


109 changes: 46 additions & 63 deletions
.github/workflows/core_tests.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
 - '*'

env:
 CACHE_NUMBER: 0 # increase to reset cache manually
 CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
 foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
 name: linux-64-py${{ matrix.python-version }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: /usr/share/miniconda3/envs/asim-test
 key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
 include:
 - os: macos-latest
 label: macOS
 prefix: /Users/runner/miniconda3/envs/asim-test
 python-version: "3.10"

 - os: windows-latest
 label: win-64
 prefix: C:\Miniconda3\envs\asim-test
 python-version: "3.10"

 defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
 name: ${{ matrix.label }}
 runs-on: ${{ matrix.os }}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ matrix.prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
 builtin_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
 external_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
 matrix:
 region:
 - prototype_mtc
 - prototype_psrc_in_development
 include:
 - region: Standard 1-Zone Example (MTC)
 region-org: ActivitySim
 region-repo: activitysim-prototype-mtc
 region-branch: extended
 - region: Standard 2-Zone Example (SANDAG)
 region-org: ActivitySim
 region-repo: sandag-abm3-example
 region-branch: main
 fail-fast: false
 defaults:
 run:
 shell: bash -l {0}
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - name: Checkout ActivitySim
 uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: |
 ${{ env.mamba-env-prefix }}
 ${{ env.CONDA }}/envs
 ~/.cache/ActivitySim
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
 # installing without dependencies is faster, we trust that all needed dependencies
 # are in the conda environment defined above. Also, this avoids pip getting
 # confused and reinstalling tables (pytables).
 run: |
 python -m pip install -e . --no-deps
 python -m pip install . --no-deps

 - name: Conda checkup
 run: |
 mamba info -a
 mamba list

 - name: Checkout Example
 uses: actions/checkout@v4
 with:
 repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
 ref: '${{ matrix.region-branch }}'
 path: '${{ matrix.region-repo }}'

 - name: Test ${{ matrix.region }}
 run: |
 python -m activitysim test ${{ matrix.region }}
 cd ${{ matrix.region-repo }}/test
 python -m pytest .

 random_seed_generation:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
 estimation_mode:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
 name: estimation_mode_test
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
 run:
 shell: bash -l {0}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4
 with:
 fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
 - name: Set up Python 3.10
 uses: actions/setup-python@v2
 with:
 python-version: "3.10"
 - name: Install dependencies
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand Down

12 changes: 11 additions & 1 deletion
docs/dev-guide/using-sharrow.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
 `@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

 `@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

jpn--
wants to merge
8
commits into

from

 Closed


External regions
#858
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
915e8d7

test external regional model examples


jpn-- Apr 23, 2024
3038256

cache buster


jpn-- Apr 23, 2024
3bdc1a4

optional variable doc


jpn-- Apr 24, 2024
b876e9c

fix conda cache dirs


jpn-- Apr 24, 2024
f578e72

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
9c282f0

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
e41054f

Merge branch 'main' into external-regions


jpn-- Apr 25, 2024
79cd6a2

Merge branch 'main' into external-regions


jpn-- May 21, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


109 changes: 46 additions & 63 deletions
.github/workflows/core_tests.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
 - '*'

env:
 CACHE_NUMBER: 0 # increase to reset cache manually
 CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
 foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
 name: linux-64-py${{ matrix.python-version }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: /usr/share/miniconda3/envs/asim-test
 key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
 include:
 - os: macos-latest
 label: macOS
 prefix: /Users/runner/miniconda3/envs/asim-test
 python-version: "3.10"

 - os: windows-latest
 label: win-64
 prefix: C:\Miniconda3\envs\asim-test
 python-version: "3.10"

 defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
 name: ${{ matrix.label }}
 runs-on: ${{ matrix.os }}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ matrix.prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
 builtin_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
 external_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
 matrix:
 region:
 - prototype_mtc
 - prototype_psrc_in_development
 include:
 - region: Standard 1-Zone Example (MTC)
 region-org: ActivitySim
 region-repo: activitysim-prototype-mtc
 region-branch: extended
 - region: Standard 2-Zone Example (SANDAG)
 region-org: ActivitySim
 region-repo: sandag-abm3-example
 region-branch: main
 fail-fast: false
 defaults:
 run:
 shell: bash -l {0}
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - name: Checkout ActivitySim
 uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: |
 ${{ env.mamba-env-prefix }}
 ${{ env.CONDA }}/envs
 ~/.cache/ActivitySim
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
 # installing without dependencies is faster, we trust that all needed dependencies
 # are in the conda environment defined above. Also, this avoids pip getting
 # confused and reinstalling tables (pytables).
 run: |
 python -m pip install -e . --no-deps
 python -m pip install . --no-deps

 - name: Conda checkup
 run: |
 mamba info -a
 mamba list

 - name: Checkout Example
 uses: actions/checkout@v4
 with:
 repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
 ref: '${{ matrix.region-branch }}'
 path: '${{ matrix.region-repo }}'

 - name: Test ${{ matrix.region }}
 run: |
 python -m activitysim test ${{ matrix.region }}
 cd ${{ matrix.region-repo }}/test
 python -m pytest .

 random_seed_generation:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
 estimation_mode:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
 name: estimation_mode_test
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
 run:
 shell: bash -l {0}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4
 with:
 fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
 - name: Set up Python 3.10
 uses: actions/setup-python@v2
 with:
 python-version: "3.10"
 - name: Install dependencies
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand Down

12 changes: 11 additions & 1 deletion
docs/dev-guide/using-sharrow.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
 `@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

 `@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

jpn--
wants to merge
8
commits into

from

 Closed


External regions
#858
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
915e8d7

test external regional model examples


jpn-- Apr 23, 2024
3038256

cache buster


jpn-- Apr 23, 2024
3bdc1a4

optional variable doc


jpn-- Apr 24, 2024
b876e9c

fix conda cache dirs


jpn-- Apr 24, 2024
f578e72

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
9c282f0

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
e41054f

Merge branch 'main' into external-regions


jpn-- Apr 25, 2024
79cd6a2

Merge branch 'main' into external-regions


jpn-- May 21, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


109 changes: 46 additions & 63 deletions
.github/workflows/core_tests.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
 - '*'

env:
 CACHE_NUMBER: 0 # increase to reset cache manually
 CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
 foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
 name: linux-64-py${{ matrix.python-version }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: /usr/share/miniconda3/envs/asim-test
 key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
 include:
 - os: macos-latest
 label: macOS
 prefix: /Users/runner/miniconda3/envs/asim-test
 python-version: "3.10"

 - os: windows-latest
 label: win-64
 prefix: C:\Miniconda3\envs\asim-test
 python-version: "3.10"

 defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
 name: ${{ matrix.label }}
 runs-on: ${{ matrix.os }}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ matrix.prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
 builtin_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
 external_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
 matrix:
 region:
 - prototype_mtc
 - prototype_psrc_in_development
 include:
 - region: Standard 1-Zone Example (MTC)
 region-org: ActivitySim
 region-repo: activitysim-prototype-mtc
 region-branch: extended
 - region: Standard 2-Zone Example (SANDAG)
 region-org: ActivitySim
 region-repo: sandag-abm3-example
 region-branch: main
 fail-fast: false
 defaults:
 run:
 shell: bash -l {0}
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - name: Checkout ActivitySim
 uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: |
 ${{ env.mamba-env-prefix }}
 ${{ env.CONDA }}/envs
 ~/.cache/ActivitySim
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
 # installing without dependencies is faster, we trust that all needed dependencies
 # are in the conda environment defined above. Also, this avoids pip getting
 # confused and reinstalling tables (pytables).
 run: |
 python -m pip install -e . --no-deps
 python -m pip install . --no-deps

 - name: Conda checkup
 run: |
 mamba info -a
 mamba list

 - name: Checkout Example
 uses: actions/checkout@v4
 with:
 repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
 ref: '${{ matrix.region-branch }}'
 path: '${{ matrix.region-repo }}'

 - name: Test ${{ matrix.region }}
 run: |
 python -m activitysim test ${{ matrix.region }}
 cd ${{ matrix.region-repo }}/test
 python -m pytest .

 random_seed_generation:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
 estimation_mode:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
 name: estimation_mode_test
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
 run:
 shell: bash -l {0}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4
 with:
 fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
 - name: Set up Python 3.10
 uses: actions/setup-python@v2
 with:
 python-version: "3.10"
 - name: Install dependencies
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand Down

12 changes: 11 additions & 1 deletion
docs/dev-guide/using-sharrow.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
 `@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

 `@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

jpn--
wants to merge
8
commits into

from

 Closed


External regions
#858
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
915e8d7

test external regional model examples


jpn-- Apr 23, 2024
3038256

cache buster


jpn-- Apr 23, 2024
3bdc1a4

optional variable doc


jpn-- Apr 24, 2024
b876e9c

fix conda cache dirs


jpn-- Apr 24, 2024
f578e72

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
9c282f0

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
e41054f

Merge branch 'main' into external-regions


jpn-- Apr 25, 2024
79cd6a2

Merge branch 'main' into external-regions


jpn-- May 21, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


109 changes: 46 additions & 63 deletions
.github/workflows/core_tests.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
 - '*'

env:
 CACHE_NUMBER: 0 # increase to reset cache manually
 CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
 foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
 name: linux-64-py${{ matrix.python-version }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: /usr/share/miniconda3/envs/asim-test
 key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
 include:
 - os: macos-latest
 label: macOS
 prefix: /Users/runner/miniconda3/envs/asim-test
 python-version: "3.10"

 - os: windows-latest
 label: win-64
 prefix: C:\Miniconda3\envs\asim-test
 python-version: "3.10"

 defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
 name: ${{ matrix.label }}
 runs-on: ${{ matrix.os }}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ matrix.prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
 builtin_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
 external_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
 matrix:
 region:
 - prototype_mtc
 - prototype_psrc_in_development
 include:
 - region: Standard 1-Zone Example (MTC)
 region-org: ActivitySim
 region-repo: activitysim-prototype-mtc
 region-branch: extended
 - region: Standard 2-Zone Example (SANDAG)
 region-org: ActivitySim
 region-repo: sandag-abm3-example
 region-branch: main
 fail-fast: false
 defaults:
 run:
 shell: bash -l {0}
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - name: Checkout ActivitySim
 uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: |
 ${{ env.mamba-env-prefix }}
 ${{ env.CONDA }}/envs
 ~/.cache/ActivitySim
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
 # installing without dependencies is faster, we trust that all needed dependencies
 # are in the conda environment defined above. Also, this avoids pip getting
 # confused and reinstalling tables (pytables).
 run: |
 python -m pip install -e . --no-deps
 python -m pip install . --no-deps

 - name: Conda checkup
 run: |
 mamba info -a
 mamba list

 - name: Checkout Example
 uses: actions/checkout@v4
 with:
 repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
 ref: '${{ matrix.region-branch }}'
 path: '${{ matrix.region-repo }}'

 - name: Test ${{ matrix.region }}
 run: |
 python -m activitysim test ${{ matrix.region }}
 cd ${{ matrix.region-repo }}/test
 python -m pytest .

 random_seed_generation:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
 estimation_mode:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
 name: estimation_mode_test
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
 run:
 shell: bash -l {0}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4
 with:
 fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
 - name: Set up Python 3.10
 uses: actions/setup-python@v2
 with:
 python-version: "3.10"
 - name: Install dependencies
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand Down

12 changes: 11 additions & 1 deletion
docs/dev-guide/using-sharrow.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
 `@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

 `@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
External regions#858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

jpn--
wants to merge
8
commits into

from

 Closed


External regions
#858
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
915e8d7

test external regional model examples


jpn-- Apr 23, 2024
3038256

cache buster


jpn-- Apr 23, 2024
3bdc1a4

optional variable doc


jpn-- Apr 24, 2024
b876e9c

fix conda cache dirs


jpn-- Apr 24, 2024
f578e72

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
9c282f0

Merge branch 'main' into external-regions


jpn-- Apr 24, 2024
e41054f

Merge branch 'main' into external-regions


jpn-- Apr 25, 2024
79cd6a2

Merge branch 'main' into external-regions


jpn-- May 21, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


109 changes: 46 additions & 63 deletions
.github/workflows/core_tests.yml

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -10,7 +10,7 @@ on:
 - '*'

env:
 CACHE_NUMBER: 0 # increase to reset cache manually
 CACHE_NUMBER: 2 # increase to reset cache manually

jobs:
 foundation:
Expand All@@ -24,10 +24,10 @@ jobs:
 name: linux-64-py${{ matrix.python-version }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -38,7 +38,7 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: /usr/share/miniconda3/envs/asim-test
 key: linux-64-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
Expand All@@ -47,19 +47,6 @@ jobs:
 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
Expand DownExpand Up@@ -109,12 +96,10 @@ jobs:
 include:
 - os: macos-latest
 label: macOS
 prefix: /Users/runner/miniconda3/envs/asim-test
 python-version: "3.10"

 - os: windows-latest
 label: win-64
 prefix: C:\Miniconda3\envs\asim-test
 python-version: "3.10"

 defaults:
Expand All@@ -124,10 +109,10 @@ jobs:
 name: ${{ matrix.label }}
 runs-on: ${{ matrix.os }}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -138,9 +123,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ matrix.prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ matrix.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -199,7 +184,6 @@ jobs:
 builtin_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
Expand All@@ -221,10 +205,10 @@ jobs:
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -235,9 +219,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -301,25 +285,31 @@ jobs:
 external_regional_models:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 strategy:
 matrix:
 region:
 - prototype_mtc
 - prototype_psrc_in_development
 include:
 - region: Standard 1-Zone Example (MTC)
 region-org: ActivitySim
 region-repo: activitysim-prototype-mtc
 region-branch: extended
 - region: Standard 2-Zone Example (SANDAG)
 region-org: ActivitySim
 region-repo: sandag-abm3-example
 region-branch: main
 fail-fast: false
 defaults:
 run:
 shell: bash -l {0}
 name: ${{ matrix.region }}
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - name: Checkout ActivitySim
 uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -330,52 +320,46 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: |
 ${{ env.mamba-env-prefix }}
 ${{ env.CONDA }}/envs
 ~/.cache/ActivitySim
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

 - name: Update environment
 run: |
 mamba env update -n asim-test -f conda-environments/github-actions-tests.yml
 mamba install --yes \
 "psutil=5.9.5" \
 "pydantic=2.6.1" \
 "pypyr=5.8.0" \
 "pytables=3.6.1" \
 "pytest-cov" \
 "pytest-regressions=2.5.0" \
 "scikit-learn=1.2.2" \
 "sharrow>=2.6.0" \
 "simwrapper=1.8.5" \
 "xarray=2023.2.0" \
 "zarr=2.14.2" \
 "zstandard=0.21.0"
 if: steps.cache.outputs.cache-hit != 'true'

 - name: Install activitysim
 # installing without dependencies is faster, we trust that all needed dependencies
 # are in the conda environment defined above. Also, this avoids pip getting
 # confused and reinstalling tables (pytables).
 run: |
 python -m pip install -e . --no-deps
 python -m pip install . --no-deps

 - name: Conda checkup
 run: |
 mamba info -a
 mamba list

 - name: Checkout Example
 uses: actions/checkout@v4
 with:
 repository: '${{ matrix.region-org }}/${{ matrix.region-repo }}'
 ref: '${{ matrix.region-branch }}'
 path: '${{ matrix.region-repo }}'

 - name: Test ${{ matrix.region }}
 run: |
 python -m activitysim test ${{ matrix.region }}
 cd ${{ matrix.region-repo }}/test
 python -m pytest .

 random_seed_generation:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -385,10 +369,10 @@ jobs:
 runs-on: ubuntu-latest

 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -399,9 +383,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -442,7 +426,6 @@ jobs:
 estimation_mode:
 needs: foundation
 env:
 mamba-env-prefix: /usr/share/miniconda3/envs/asim-test
 python-version: "3.10"
 label: linux-64
 defaults:
Expand All@@ -451,10 +434,10 @@ jobs:
 name: estimation_mode_test
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4

 - name: Setup Mambaforge
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand All@@ -465,9 +448,9 @@ jobs:
 - name: Set cache date for year and month
 run: echo "DATE=$(date +'%Y%m')" >> $GITHUB_ENV

 - uses: actions/cache@v3
 - uses: actions/cache@v4
 with:
 path: ${{ env.mamba-env-prefix }}
 path: ${{ env.CONDA }}/envs
 key: ${{ env.label }}-conda-${{ hashFiles('conda-environments/github-actions-tests.yml') }}-${{ env.DATE }}-${{ env.CACHE_NUMBER }}
 id: cache

Expand DownExpand Up@@ -517,15 +500,15 @@ jobs:
 run:
 shell: bash -l {0}
 steps:
 - uses: actions/checkout@v3
 - uses: actions/checkout@v4
 with:
 fetch-depth: 0 # get all tags, lets setuptools_scm do its thing
 - name: Set up Python 3.10
 uses: actions/setup-python@v2
 with:
 python-version: "3.10"
 - name: Install dependencies
 uses: conda-incubator/setup-miniconda@v2
 uses: conda-incubator/setup-miniconda@v3
 with:
 miniforge-variant: Mambaforge
 miniforge-version: latest
Expand Down

12 changes: 11 additions & 1 deletion
docs/dev-guide/using-sharrow.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -193,13 +193,23 @@ in several examples:
 `@np.log1p(size_terms.get(df.alt_dest, df.purpose)) # sharrow: np.log1p(size_terms['sizearray'])`

Here, `size_terms` is a DataFrameMatrix class instance, a special class written into
ActivitySim to facilitate reading from a DataFrame as it it were a 2-d array. As it
ActivitySim to facilitate reading from a DataFrame as if it were a 2-d array. As it
is a special purpose class written in Python, the numba compiler cannot handle it directly.
Fortunately, sharrow provides an alternative: passing the size terms as a xarray `DataArray`.
This has a slightly different interface, so the sharrow and legacy evaluation modes require
different expressions. The switching expression is used to handle the DataFrameMatrix
on the left (before "# sharrow:") and the DataArray on the right.

### Optional Variables

In some cases, a variable may be used where it is available, but is not strictly
necessary for the model to run. For example, a model may have a reference to
mode choice logsums, but the model can still run without them, if it is being used
prior to when logsums are calculated. In this case, the variable can be accessed
using the `get` method, which allows for a default value if the variable is not found.

 `@df.get('mode_choice_logsum', 0)`

### Performance Considerations

Sharrow is usually expected to bring significant performance gains to ActivitySim.
Expand Down