traveled graph implementation (Python) by yakra · Pull Request #201 · TravelMapping/DataProcessing

yakra · 2019-03-29T16:53:51Z

Implements #196 in Python

in preparation for "traveled" graph implementation

* Separate collection of "traveled" edges implemented. * fmt_mask allows new collapsed/traveled edges to be constructed together or separately as needed. * HighwayGraph.matching_vertices calculates & returns # of vertices for traveled graph, though this info is not used yet. * Traveled graph partially implemented: for now, just a glorified collapsed graph with a few extra vertices & edges. * Traveler numbers and hex codes are not yet implemented. * Subgraphs are not yet implemented..

* Subgraphs partially implemented: for now, just glorified collapsed graphs with a few extra vertices & edges. * Traveler numbers and hex codes are not yet implemented.

TravelMapping#199

yakra · 2019-03-31T04:06:32Z

@jteresco, you only said "a new version number in the header"; I made the "2.0" assumption. Do you prefer a different number, or is 2.0 fine?
Thoughts on adding "-collapsed" to TMG filenames, E.G. AB-region-collapsed.tmg? More descriptive / less ambiguous compared to a vanilla filename, and more convenient if I want to mv *collapsed.tmg foo/, or rm *collapsed.tmg, etc.
- On the one hand, the links on the graph download page would be broken.
- OTOH, it'll have to be revamped eventually.

jteresco · 2019-03-31T13:16:43Z

I like 2.0 and I like adding the format to every filename. I can easily enough revamp the download page. I think HDX would be good, it would just start loading up the -collapsed graphs from the list in the DB. More interesting will be to bring in user stats there for display and use in the algorithms.

yakra · 2019-03-31T15:29:32Z

OK. For now, the existing filenames are retained in this PR.

yakra · 2019-04-07T18:35:17Z

When revamping the graph download page, thoughts on including links to tmg2html "HDX-Lite" pages?
(Semi-related: #95)
(I'd have to first update tmg2html to read TMG 2.0 traveled files. Relatively easy to do; just hinges on how #199 is resolved.)

jteresco · 2019-04-08T12:00:57Z

Makes sense to me to link to HDX-Lite pages. I think that whole page needs a little thought anyway now with 3 graph formats per subgraph.

In theory, saves a tiny bit of time. In practice, not a noticeable amount. :)

(if -> elif) into tmg2_python

(if -> elif) into RelevantTrav

(minor datacheck speed improvements) into tmg2_python

(minor datacheck speed improvements) into RelevantTrav

(duplicate label comments) into tmg2_Python

(duplicate label comments) into RelevantTrav

jteresco · 2019-04-23T00:07:30Z

I am about 2 weeks away from what I hope is a chance to merge this in and make corresponding changes to get the new format graphs listed on the web site and to get the format documented for potential users.

yakra · 2019-04-23T19:56:39Z

I am about 2 weeks away from what I hope is a chance to merge this in

Thanks, good to know.

and make corresponding changes to get the new format graphs listed on the web site

Should I go ahead and make the change to FOO-bar-collapsed.tmg style filenames in anticipation of this, then?

and to get the format documented for potential users.

Let's get #199 sorted before moving ahead. Two questions, should I:

Use a single-0 hex code for each edge in zero-traveler graphs? (flagged as "Most likely do")
Add the number of travelers to the second header line? (flagged as "To do or not to do?")

terminal LF -> tmg2_Python

terminal LF -> RelevantTrav

jteresco · 2019-04-23T20:20:31Z

and make corresponding changes to get the new format graphs listed on the web site
Should I go ahead and make the change to FOO-bar-collapsed.tmg style filenames in anticipation of this, then?

Yes, let's go for it all in one shot.

and to get the format documented for potential users.
Let's get #199 sorted before moving ahead. Two questions, should I:
* Use a single-`0` hex code for each edge in zero-traveler graphs? _(flagged as "Most likely do")_
* Add the number of travelers to the second header line? _(flagged as "To do or not to do?")_

I think yes to both with an eye toward ease of parsing for potential users.

TravelMapping#199 TravelMapping#201

RelevantTrav into tmg2_Python

yakra · 2019-04-23T21:31:16Z

Yes, let's go for it all in one shot.
...
I think yes to both with an eye toward ease of parsing for potential users.

All three changes are now live in this branch.

TravelMapping#199 TravelMapping#201

* 12% speedup Augmenting travelers for detected concurrent segments * 34% speedup Creating per-traveler stats log entries and augmenting data * 1.7% speedup for subgraph generation. Maybe.

yakra · 2019-05-02T06:43:29Z

LOL, why not a benchmark.
BiggaTomato / UserData @ 207c8eb / HighwayData @ 0c0573b
Subgraph generation time increased from 99.9s to 159.3s.
No doubt most of that comes from hex code generation.

And nothing wrong with that, traveled graphs will be super neat to have. :)

Also added a commit to address #203.

jteresco · 2019-05-08T02:45:42Z

The time has come. I'm bringing this in. I'm going to try it out on my Mac and will make corresponding changes to the graph downloads page (TravelMapping/Web#299) before putting to use for the production site updates on noreaster.

yakra added 10 commits March 23, 2019 14:14

rename classes & variables
616ee5d
in preparation for "traveled" graph implementation

use one class for simple & collapsed edge objects
15c7bd6

concurrent travelers sanity check
84ddc30

HGEdge: store canonical HighwaySegment instead of region
4669828

comment: region -> segment
71bbd81

HGVertex: is_hidden -> visibility
e0bc96c

traveled subgraph partial implementation
55bc552
* Subgraphs partially implemented: for now, just glorified collapsed graphs with a few extra vertices & edges. * Traveler numbers and hex codes are not yet implemented.

full traveled graph implementation
cedc846

remove redundant function
d49c2d4

yakra added enhancement graph generation labels Mar 29, 2019

yakra mentioned this pull request Mar 30, 2019
traveled graphs: record only relevant travelers #199
Closed

traveled graphs: record only relevant travelers
5b8b8d7
TravelMapping#199

yakra mentioned this pull request Apr 4, 2019
Graph generation: how many/which users have clinched each segment #196
Closed

yakra mentioned this pull request Apr 19, 2019
C++: traveled graphs, and region-splitting support #206
Merged

yakra added 9 commits April 22, 2019 15:17

canonical_waypoint_name: if -> elif
5635614
In theory, saves a tiny bit of time. In practice, not a noticeable amount. :)

Merge pull request#74from yakra/Python_devel
3faa153
(if -> elif) into tmg2_python

Merge pull request#75from yakra/Python_devel
f4a8a63
(if -> elif) into RelevantTrav

minor datacheck speed improvements
b2de340

Merge pull request#76from yakra/Python_devel
bb0ddbf
(minor datacheck speed improvements) into tmg2_python

Merge pull request#77from yakra/Python_devel
0dcab3f
(minor datacheck speed improvements) into RelevantTrav

comments for duplicate label datacheck
95969e1

Merge pull request#78from yakra/Python_devel
0e7b5c5
(duplicate label comments) into tmg2_Python

Merge pull request#79from yakra/Python_devel
db94c0b
(duplicate label comments) into RelevantTrav

yakra added 3 commits April 23, 2019 16:02

terminal LF in traveled graphs (Python)
48ed3df

Merge pull request#82from yakra/Python_devel
013d7ed
terminal LF -> tmg2_Python

Merge pull request#83from yakra/Python_devel
065c17d
terminal LF -> RelevantTrav

yakra added 2 commits April 23, 2019 17:18

finalizing traveled graphs
e815c31
TravelMapping#199 TravelMapping#201

Merge pull request#84from yakra/RelevantTrav
e542667
RelevantTrav into tmg2_Python

yakra added a commit to yakra/DataProcessing that referenced this pull request Apr 23, 2019
C++: finalizing traveled graphs
652efe3
TravelMapping#199 TravelMapping#201

yakra changed the title ~~traveled graph implementation~~traveled graph implementation (Python)Apr 24, 2019

yakra mentioned this pull request Apr 28, 2019
Python: remove unused ClinchedSegmentEntry class #216
Merged

TravelerList.clinched_by: list -> set
0ede2eb
* 12% speedup Augmenting travelers for detected concurrent segments * 34% speedup Creating per-traveler stats log entries and augmenting data * 1.7% speedup for subgraph generation. Maybe.

jteresco mentioned this pull request May 8, 2019
Graph index page updates for traveled graphs TravelMapping/Web#299
Closed

jteresco merged commit c492192 into TravelMapping:masterMay 8, 2019

yakra deleted the tmg2_Python branch June 19, 2019 21:59

Navigation Menu

Uh oh!

traveled graph implementation (Python) - #201

traveled graph implementation (Python)#201
jteresco merged 26 commits into
TravelMapping:masterfrom
yakra:tmg2_Python

yakra commented Mar 29, 2019

Uh oh!

yakra commented Mar 31, 2019

Uh oh!

jteresco commented Mar 31, 2019

Uh oh!

yakra commented Mar 31, 2019

Uh oh!

yakra commented Apr 7, 2019

Uh oh!

jteresco commented Apr 8, 2019

Uh oh!

jteresco commented Apr 23, 2019

Uh oh!

yakra commented Apr 23, 2019•
edited
Loading

Uh oh!

jteresco commented Apr 23, 2019

Uh oh!

yakra commented Apr 23, 2019

Uh oh!

yakra commented May 2, 2019•
edited
Loading

Uh oh!

jteresco commented May 8, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

yakra commented Mar 29, 2019

Uh oh!

yakra commented Mar 31, 2019

Uh oh!

jteresco commented Mar 31, 2019

Uh oh!

yakra commented Mar 31, 2019

Uh oh!

yakra commented Apr 7, 2019

Uh oh!

jteresco commented Apr 8, 2019

Uh oh!

jteresco commented Apr 23, 2019

Uh oh!

yakra commented Apr 23, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jteresco commented Apr 23, 2019

Uh oh!

yakra commented Apr 23, 2019

Uh oh!

yakra commented May 2, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jteresco commented May 8, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yakra commented Apr 23, 2019•
edited
Loading

yakra commented May 2, 2019•
edited
Loading