strip .list comments before counting fields

[yakra]

Removing .list line comments before checking the number of fields will make life easier for implementing #58:

line = line.strip().rstrip('\x00')
# ignore empty or "comment" lines
if len(line) == 0 or line.startswith("#"):
 continue

->

# ignore comments, leading/trailing whitespace, trailing null zeros
line = line.split('#')[0].strip()
# ignore empty lines
if len(line) == 0:
 continue

This stays the same:
fields = re.split('[ \t]+',line)

The payoff is when we get here:

if len(fields) != 4:
 # OK if 5th field exists and starts with #
 if len(fields) < 5 or not fields[4].startswith("#"):
 self.log_entries.append("Incorrect format line: " + line)
 continue

->

if len(fields) == 4:
 # Do what we're already doing
elif len(fields) == 6:
 # https://github.com/TravelMapping/DataProcessing/issues/58
else:
 self.log_entries.append("Incorrect format line: " + line)
 continue

...rather than something a lot more complex.

[yakra]

OTOH, this would adversely affect performance. Worst case or even pretty close to average case (as most lines don't contain comments), we'd be scanning each line twice:

• once for # to remove comments
• again for whitespace to split into fields

A more complex solution may be in order.

[yakra]

This is easy:

• Start off as we already do:
  • Ignore empty or "comment" lines
  • Split line into fields
• Iterate thru all fields or first 7, whichever's less
  • If we find a field starting with #, truncate the fields container to before that one, and break.
• IOW, "Removing .list line comments before checking the number of fields".
• Proceed as in the OP.

I'll close this issue for housekeeping purposes, and let this be implemented as part of #58.

[yakra]

• Iterate thru all fields or first 7, whichever's less
  • If we find a field starting with #, truncate the fields container to before that one, and break.

This makes sense from a Python perspective, the way the split function is already used to automate things.

• In C++, we already have to populate the fields vector element-by-element, so it makes sense to not even add in fields beginning with #.
  • Slight speedup due to not having to process the whole line when splitting.
    Or not. 0.07s slower on lab1, within margin of error. Inline comments are relatively rare, making the gains negligible. Also, there's a small amount more overhead now, checking whether each field starts with #.
• Trying to mimic C++ style fields construction in Python is likely not worth the hassle, and may even be counter-productive performance-wise.

[yakra]

HighwayData @ 27a625459516a8f5c53f73a24ce51416ab991677
UserData @ 3ec63996d33e499027a3d4c8f47abc0d3ce7659f

formulanone.log:
13c13
< Waypoint label(s) not found in line: GA GA1LpRom # "GA746">GA20-AL9
---
> Incorrect format line: GA GA1LpRom # "GA746">GA20-AL9

Old version happens because 4 fields suggest a valid format .list line, and we attempt to parse both # and "GA746">GA20-AL9 as waypoint labels.
New version happens because we truncate everything starting at the #, leaving only 2 fields, yielding an incorrect format line.

[yakra]

Addressed on BiggaTomato's local fork in 265c35d13a6a14e66da1d6cee134a3a31c038427.
This commit will most likely not make it to GitHub, as I plan to squash commits.