Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Adding Logsums

bstabler edited this page Jan 15, 2018 · 18 revisions

Scope of Work

Getting Started

  • Will start by implementing just the work location choice model logsum needs in order to draft a vectorized approach
  • Will create two new model steps and revise the existing step:
    • workplace_location_sample - will use a similar expression file as today (but without the logsum expression), but will build a table of workers * all zones in order to select a sample of alternative work locations for the next model. This selects 30 locations from the 1450 zones. It does 30 picks and returns the pick count as well, since this is used in the later model step as an alternative correction factor.
    • workplace_location_logsums - will start with the workers * sampled alt zones table output above and will add home taz, alt taz, depart period, end period, and will add a logsum column (see issue below). The existing tour mode choice expression file is used with the following skim lookups:
      • @odt_skims - home taz, work alternative taz, time period = 8am
      • @dot_skims - work alternative taz, home taz, time period = 5pm
    • workplace_location (existing) - will start with the table output above and the existing expression file in order to choose a work location like before. This step selects one location from the sample of 30, this time with the mode choice logsum included. The 30 alternatives are collapsed into the unique set of alternatives and the pick count (transformed into a sample correction factor) is added as a utility term.
  • The mode choice model will be revised to add new capabilities to get logsums in addition to making choices (#164). The existing mode.tour_mode_choice_simulate method calls _mode_choice_simulate, which calls asim.simple_simulate, which calls either eval_mnl or eval_nl, which then calculates utilities and calls the logit class to calculate probabilities and make a choice. This needs to be revised as follows:
    • The logit class gets a new method, get_logsum, which returns the logsum (composite utility across all modes).
    • Then asim.simple_simulate and mode_choice_simulate need to be revised to support two use cases: calculating utilities and making choice. Setting up and calculating utilities is required in both cases.
    • Finally, the mode class needs to be revised to support calculating work location mode logsums using the workplace_location_logsums table calculated above.

Sample Correction Factor

The sample correction factor is calculated as follows:

freq = how often an alternative is sampled (i.e. the pick_count)
prob = probability of the alternative
correction_factor = log(freq/prob)

For example:

freq 1.00	2.00	3.00	4.00	5.00
prob 0.30	0.30	0.30	0.30	0.30
correction factor 1.20	1.90	2.30	2.59	2.81

As the alternative is oversampled, its utility goes up for final selection. The unique set of alternatives is passed to the final choice model and the correction factor is included in the utility. The correction factor is capped by the min(correction_factor,60) expression.

In order to make the vectorized implementation easier, a good solution is to keep the full set of alternatives (including duplicates), put the correction factor on each alternative (record), and then NA the utility of the duplicate alternatives before calculating the probability. Adding an isDuplicate attribute to each alternative allows us to add an expression to the final choice model like: isDuplicate * -99999 in order to make the alternative unavailable. This makes it so we don't have to modify the correction factor and/or the probabilities, while allowing us to keep the existing assumed fixed set of alternatives.

Clone this wiki locally