datacheck.sh: Relevant logs should be stored in a specific folder

[michihdeu]

The relevant files are currently hidden within all the user logs et cetera.

I think it would be fine to have two directories:

• userlogs
• logs

The location, currently "/home/michih/DataProcessing/siteupdate/python-teresco/logs/", should be output in Bash at the end of the run:

Data check successful!
Log file location: "/home/michih/DataProcessing/siteupdate/python-teresco/logs/"
Total run time: [417.2]
Tue May 22 12:47:20 EDT 2018
datacheck.sh: complete

[yakra]

The relevant files are currently hidden within all the user logs et cetera.

I think it would be fine to have two directories:

• userlogs
• logs

Seconded.

[jteresco]

I agree on this, but will hold off for a little while with all of the server changes and coming switch to Leaflet, since this also would require some changes on the Web side.

[jteresco]

When the update running on the new server right now completes, we should have site update logs in logs, user logs in logs/users .

I'll update the link on user pages as soon as it finishes and I see the files are there.

[michihdeu]

I run it but had some problem.

First, I know that existing log files used to be not removed. For that reason, I deleted my UserData folder (/Home/michih/UserData).
Then, I run git pull, no changes.

I run sh datacheck.sh but error, siteupdate.sh not found. It was really not there. I tried git pull again but it didn't work. I've deleted the whole /Home/michih/DataProcessing folder , git clone https://github.com/TravelMapping/DataProcessing.git and I could run it.

It worked, many logs are created but Bash outputs an error:

[349.6] Creating per-traveler stats log entries and augmenting data structure.........................................................................................................................................................................................................!
[418.7] Writing traveler list logs.
Traceback (most recent call last):
File "./siteupdate.py", line 2955, in <module>
t.write_log(args.logfilepath+"/users")
File "./siteupdate.py", line 1044, in write_log
logfile = open(path+"/"+self.traveler_name+".log","wt",encoding='UTF-8')
FileNotFoundError: [Errno 2] No such file or directory: 'logs/users/1.log'
Wed May 30 11:10:13 EDT 2018
datacheck.sh: complete

It's correct, files are not created. Why?

Another issue:

/home/michih/DataProcessing/siteupdate/python-teresco/logs/siteupdate.log ends with:

[418.7] Writing traveler list logs. but the error is not logged!

And:

Where is the datacheck log file?

[yakra]

'logs/users/1.log' ?

[michihdeu]

The file is not existing and not generated (there's a user called "1", no. 193 with 2210mi traveled)

[jteresco]

Just forgot to edit one line in datacheck.sh that creates the directory for user logs. Should be good if you pull down the latest DataProcessing changes.

All log files other that those generated per-user should still end up in logs, while each user's log will end up in logs/users .

[michihdeu]

Works now. Thanks 👍