Data check for systemupdates.csv

[michihdeu]

The file should be checked for invalid entries.
For instance, that the "systemName" column contains an existing path.

[yakra]

Would this issue be better suited to TravelMapping/Web instead? I think handling it in the PHP may be the way to go...
There are many entries with systemNames that are no longer valid. Most of these are for systems merged into larger active systems, E.G. engb, irlr4, usaky8, etc. It makes sense to keep these around for historical reference purposes.
Note that for a merged system, only the raw text is provided in the System Code column, without a link to the HB.

It makes sense IMO to detect entries where a system with a statusChange to preview later has a statusChange to merged, and similarly disable the HB link.

Or simpler yet, the HB link could simply be disabled for any systemName that is no longer valid. Simpler to code, and this can catch cases other than just preview->merged.

[michihdeu]

It's here because I wanna be able to check it before the site update is executed.

I generally think that all dead links should be eliminated. The entire entries (or minimum the systemName) could be removed. I don't think that a complex check is required nor reasonable.

[jteresco]

I'm definitely in favor of keeping the entries for historical interest, and agree that we should avoid invalid links. I'm also thinking it is better done on the PHP side to avoid bad links, but that the site update process (including the soon-to-exist highway data check version) should clearly report malformed entries.

[yakra]

Drilling down a bit farther, looking for invalid system codes besides merged cases...

1. cannb

asiahr and asiahp are also listed with a statusChange of split, but cannb is the only split system with a code that's no longer valid. It should be apparent to anyone reading updates.php that was split into cannba, cannbc and cannbl.

2. mnem2

I assume this was superseded by 2018-01-05;Montenegro;mnem;Montenegro Magistralni Put;re-entered

3. mkdm

Macedonia Magistrale are later listed with the mkda system code. Either the code was changed at some point, or mkdm was a typo or error.

4. serba

This looks like a typo for srba.

5. nldn0

Netherlands Niet-Autosnelwegen 1-99. It's not clear from looking at systemupdates.csv, or the commit history of systems.csv, what became of this system. I also see nldn1 thru nldn9 here. Nothing in systemupdates.csv says they ever hit preview; I've not yet looked forward in the commit history of systems.csv to investigate their disappearance.

6. hgkrt

typo for hkgrt

7. gbna1

gbna1 is listed as extended and merged on adjacent lines, both for 2015-12-30. This along with gbna6, gbna8 & gbna9 merged on the same date. The top entry, for Great Britain A Roads, should probably be for vanilla gbna instead?

[michihdeu]

5. nldn0

It was changed to nldr

[yakra]

It was changed to nldr

On 2016-12-14?

[michihdeu]

yep! It was initially planned to have all N routes in one system but the 3-digit routes are maintained by the provinces. The other routes are maintained by the state.

@yakra I guess you wanna clean the file?

[yakra]

@yakra I guess you wanna clean the file?

A backdated note such as
2016-12-14;Netherlands;nldr;Netherlands Niet-Autosnelwegen 1-99;renamed
could satisfy the historically curious. :)

As for the rest, *shrug* -- ping @si404?

[michihdeu]

The actual entries are:

2016-12-14;Netherlands;nldr;Netherlands Rijkswegen;active
2016-01-15;Netherlands;nldn0;Netherlands Niet-Autosnelwegen 1-99;preview

Option 1 (yakra's idea, note: it was actually 4 days earlier; and w/o link in first entry):

2016-12-14;Netherlands;nldr;Netherlands Rijkswegen;active
2016-12-10;Netherlands;nldr;Netherlands Niet-Autosnelwegen 1-99;renamed
2016-01-15;Netherlands;;Netherlands Niet-Autosnelwegen 1-99;preview

Option 2 (only changing the link):

2016-12-14;Netherlands;nldr;Netherlands Rijkswegen;active
2016-01-15;Netherlands;nldr;Netherlands Niet-Autosnelwegen 1-99;preview

Option 3 (option 1 but using new system name):

2016-12-14;Netherlands;nldr;Netherlands Rijkswegen;active
2016-12-10;Netherlands;nldr;Netherlands Rijkswegen;re-entered
2016-01-15;Netherlands;;Netherlands Niet-Autosnelwegen 1-99;preview

Option 4 (option 3 but with the new system link in first entry):

2016-12-14;Netherlands;nldr;Netherlands Rijkswegen;active
2016-12-10;Netherlands;nldr;Netherlands Rijkswegen;re-entered
2016-01-15;Netherlands;nldr;Netherlands Niet-Autosnelwegen 1-99;preview

I don't have a favorite. Option 2, 3 or 4 are fine. It should be similar to the other changes required (especially w/ or w/o new system link).

[yakra]

A backdated note such as
2016-12-14;Netherlands;nldr;Netherlands Niet-Autosnelwegen 1-99;renamed
could satisfy the historically curious. :)

I mistyped that. :( I meant to say:
2016-12-14;Netherlands;nldn0;Netherlands Niet-Autosnelwegen 1-99;renamed

Does that change your options?

[michihdeu]

nldn0 is a dead link. That's exactly what I don't like!

[jteresco]

I'd deal with dead links in the Web code, so I'd prefer to keep them in the CSV, again for the vague historical interest.