unprocessedwpts.log order

[michihdeu]

The list is ordered by region but the routes of the region are not sorted, e.g.:

hwy_data/ROU/roudj/rou.dj174c.wpt
hwy_data/ROU/roudj/rou.dj108h.wpt
hwy_data/ROU/roudj/rou.dj108i.wpt
hwy_data/ROU/roudj/rou.dj108k.wpt
hwy_data/ROU/roudj/rou.dj190.wpt
hwy_data/ROU/roudj/rou.dj191.wpt
hwy_data/ROU/roudj/rou.dj191a.wpt
hwy_data/ROU/roudj/rou.dj191b.wpt
hwy_data/ROU/roudj/rou.dj191f.wpt
hwy_data/ROU/roudj/rou.dj709amas.wpt
hwy_data/ROU/roudj/rou.dj709e.wpt
hwy_data/ROU/roudj/rou.dj102i.wpt
hwy_data/ROU/roudj/rou.dj102p.wpt

They should also be sorted alphabetically.

[yakra]

The all_wpt_files list (and thus our unprocessed wpts) are stored in a set, rather than a list, so the order they're stored in memory isn't necessarily the same as the order in which they're added to the container.

Sorting asciibetically is easy enough. We have a couple options.

• Just store them in a list instead of a set. There shouldn't be a risk of anything being added to the list twice. This may slow down C++ when removing .wpts that are processed from the list. (Python may not be as bad in this regard?) Building the list in the first place will be faster though.
• Continue to use a set, and just sort the unprocessed wpts before writing unprocessedwpts.log. Easily done in Python; just takes 8 more characters. C++ will take ~2-3 more lines, but is still easy enough. Slower building the initial set, faster removing items from it.

The 2nd option will definitely destroy csv order, I.E. me.us001altban before me.us001altpor. Order of the final unprocessedwpts.log should be the same between Python & C++ flavors.
The 1st option will rely more on the whims of how the directory listings are read in from the OS. Order of the final unprocessedwpts.log may differ between Python & C++ flavors.

[yakra]

The all_wpt_files list (and thus our unprocessed wpts) are stored in a set, rather than a list, so the order they're stored in memory isn't necessarily the same as the order in which they're added to the container.

What was I talking about? This is, and always has been, a list:
https://github.com/TravelMapping/DataProcessing/blame/d00dff8b94b3eb178db362c390252fb46f0ce26b/siteupdate/python-teresco/siteupdate.py#L2311
Must have been only thinking of C++...

[yakra]

The 2nd option will definitely destroy csv order, I.E. ...

Again, what was I talking about? Unprocessed WPTs are by definition not in any CSVs...