ComputeStatsThread parallelism

[yakra]

Separate ComputeStatsThread into:

• CompStatsRThread
  • Mutexes per route rather than per segment.
• CompStatsTThread
  • A small increase in overhead during ConcAugThread. Should be well worth it.
  • Iterate by TravelerList, not HighwaySystem->Route-HighwaySegment.
  • No mutexes!
  • More refactoring (Python speedup too)!
• How do the various alternatives perform with threading disabled?
  • _cs1s2lt: 8.66s
  • _cs1s2lhr: 7.26s
  • _cs1s1l: 7.18s

---

• "compute lots of stats" comment can be cleaned up & clarified in Python too

[yakra]

# user logs
for u in *; do
 echo $u
 diff <(egrep -c 'System .+ \(active) overall: [0-9]+.[0-9]{2} of [0-9]+.[0-9]{2} mi \(100.00%\)' $u) <(tail -n 2 $u | head -n 1 | cut -f7 -d' ')
 diff <(egrep -c 'System .+ \(preview) overall: [0-9]+.[0-9]{2} of [0-9]+.[0-9]{2} mi \(100.00%\)' $u) <(tail -n 1 $u | cut -f7 -d' ')
done

---

DataProcessing/siteupdate/cplusplus/classes/TravelerList/userlog.cpp

Lines 48 to 50 in a20ee6d

	if (t_system_overall == h->total_mileage())
	if (h->active())
	active_systems_clinched++;

How did calculation of t_system_overall & h->total_mileage() change?

• h->total_mileage()

  unchanged

• t_system_overall

  old	new
  TravelerList::system_region_miles(h)	TravelerList::system_region_miles(h)
  sum of doubles in system_region_mileages	sum of doubles in system_region_mileages
  length/system_concurrency_count, system -> route -> segment	length/system_concurrency_count, TravelerList -> clinched_segments

• convert double to float for comparison

[yakra]

cd tm/UserData/
git show -s
cd ../HighwayData
git show -s
cd ~/ytm/DataProcessing/siteupdate/cplusplus/
git branch
git checkout -- `git status | grep modified | sed 's~.*:~~'`
git pull
rm classes/TravelerList/compute_stats_t.cpp threads/CompStatsTThread.cpp
git checkout 258b6130f2ea997f2c5779efa44751cb70f6fa75; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1tn
git checkout ee6d5304ce88fd52a67a9b2aa6903bacb05ea69f; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1ti #cs1t
git checkout 7629ee13d4892be464e05eb02ed242005b9f6520; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1sn
git checkout 3e391a77f202a1431089e47020ddfe110cff2c81; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1si #cs1s
git checkout 8efec7a9a29dba14416d9dfaf25191d3cf9c1340; g++ siteupdate.cpp -std=c++11 -pthread -o siteupdate_cs1x #cs1x
emacs check10.sh; ./m1.sh _cs1x

# _cs1tn _cs1ti _cs1sn _cs1si _cs1x

[yakra]

31ccd30 ComputeStatsThread parallelism phase 1

squashed on BiggaTomato:

head	msg
7629ee1	_cs1sn (#366)
3e391a7	_cs1si (#366)
92b0d9552007341be28432b97eeed466dd0ba77b	ST ConcAug fix; ComputeStats -> CompStatsR
56d47eaed1161ff7d72b18314f0eccabe73f6d8e	_cs1s2lhr
2daec1a10ef19b5565de09112be4aaac9c6ce312	_cs1s1l
7e78da0c769b0e4971bac4ed41fffa6082ed5d12	CompStatsRThread comments
5aced65db3df159e97036e49b5524df3351b506c	remove extraneous active/preview checks
278f5b56e4eaa536a64d68095da87f572d6e9c3c	clean up "compute lots of stats" comment