use clang optimization flag

[yakra]

Be sure to make clean before you make again (or gmake as the case may be).

Re the change to WaypointQuadtree.cpp, I'm surprised I never got linking errors before.

---

Wow, FreeBSD is a top performer here. Note the small bump in processing time when we run out of physical cores & start using hyperthreaded virtual cores.
Less apparent on lab3 & especially lab4.

The "default" line for FreeBSD was thrown off by a couple big outliers.

Ouch, another horrible performer under FreeBSD.

Same thing, zoomed in to just the Linux machines.

Another win for FreeBSD.


A couple outliers in FreeBSD's "default" line again.

For the 24-thread machines, the results of 5 passes were averaged together instead of the usual 10, because I wanted to do these tests in a finite amount of time. :) With this task being so fast, the noise is a much larger piece of the pie, resulting in fairly jagged lines.


Another case where FreeBSD has trouble scaling beyond just a few threads, though this isn't as bad as the others.
There are several potential changes that could maybe help out, and as we saw in #517, sometimes a little tweak goes a long way.
Good news is, noreaster scales up to 8 threads. This looks like a memory bandwidth issue, and shouldn't be as tough to tackle as nmp_merged or subgraphs.

The somewhat jagged lines could be a result of small sample size again.
Why does BiggaTomato stay flat between 3 & 4 threads? The DB file is being written in the background. So for a little while, siteupdate actually runs numthreads+1 threads. It's just easiest to do this way. It usually finishes up at some point during subgraph generation.
Well, that would be the case, except that it already stays flat between 2 & 3 threads. Running out of memory bandwidth I guess.

This is just painful. :( A nice bump at 1 thread, and then... nothing. This may take a lot of work to see an improvement...

Same thing, zoomed in to just the Linux machines.
At 4 threads on BT & 8 on lab1 & lab5, that's probably the DB file slowing down graph generation again.

---

single-threaded tasks in multi-threaded siteupdate

top: default / -O0
bottom:-O3

BT	lab1	lab5	lab2	lab3	lab4	bsdlab
Reading region, country, and continent descriptions	.0040 .0011	.0030 .0010	.0020 .0005	.0025 .0009	.0035 .0012	.0041 .0014
Reading systems list	.6388 .3934	.4503 .2826	.3640 .1867	.5357 .3225	.7169 .4374	.6730 .3402
Finding all .wpt files...* files found	.4979 .4396	.2295 .2011	.2554 .2184	.2889 .2574	.3922 .3455	.3910 .4357
Finding unprocessed wpt files	.0142 .0083	.0106 .0062	.0130 .0055	.0133 .0073	.0123 .0074	.0165 .0094
Near-miss point log and tm-master.nmp file	.7112 .4870	.4151 .3019	.4220 .2427	.5105 .3579	.6826 .4707	.8405 .5181
Concurrent segment detection	.7570 .3647	.5378 .3069	.4489 .1751	.6300 .3385	.9040 .5067	.8745 .3218
Reading updates file	.0396 .0389	.0285 .0281	.0258 .0254	.0329 .0331	.0454 .0404	.0288 .0276
Reading systemupdates file	.0006 .0005	.0007 .0006	.0004 .0003	.0006 .0005	.0009 .0007	.0005 .0003
Processed .* traveler list files. Sorting and numbering	.0004 .0002	.0002 .0001	.0004 .0001	.0002 .0002	.0008 .0003	.0008 .0002
Clearing route & label hash tables	.2294 .1442	.1344 .1124	.1085 .0546	.1528 .1254	.2949 .2118	.2064 .0877
Writing route and label logs	.4222 .1759	.2305 .1294	.2367 .0626	.2755 .1440	.4296 .2428	.4933 .1117
Writing to concurrencies.log	.3050 .2727	.1868 .1702	.1423 .1130	.1901 .1699	.3045 .2853	.2629 .1984
Writing highway data stats log file	.1931 .1519	.1890 .1623	.1097 .0863	.2021 .1777	.2686 .2245	.1911 .1336
Writing stats csv files	1.7749 1.1743	.6701 .4540	.6512 .4462	.9172 .6325	1.9110 1.2893	1.6824 1.1524
Reading datacheckfps.csv	.0101 .0075	.0133 .0107	.0361 .0342	.0118 .0104	.0473 .0321	.0318 .0243
Setting up for graphs of highway data	.8726 .4447	.5071 .2352	.6122 .2345	.6084 .2732	1.2327 .6286	1.1912 .5611
Creating edges	1.9794 1.1674	.9023 .6282	1.0598 .5988	1.1242 .7827	2.1535 1.4456	2.0437 1.2360
Compressing collapsed edges	1.3646 .6540	.6600 .2882	.7712 .2588	.7981 .3424	1.8995 1.0588	1.7339 1.1830
Writing graph waypoint simplification log	.1299 .1224	.1049 .0983	.0774 .0611	.1303 .1220	.3482 .3014	.2775 .2297
Setting up subgraphs	1.3351 1.1321	.4590 .3721	.4671 .3752	.3070 .2353	.3702 .2936	.3194 .2444
Clearing HighwayGraph contents from memory	1.4703 .9588	.7784 .4951	.7830 .3693	.9857 .6038	1.8046 1.3576	1.0112 .7988
Marking datacheck false positives	1.8045 .5343	1.7090 .9466	1.2997 .4812	2.2242 1.3515	2.2488 1.7582	2.0259 .5065
Found .* datacheck errors and matched .* FP entries	0 0	.0010 .0010	0 0	.0010 .0011	.0038 .0047	0 0
Writing log of unmatched datacheck FP entries	.0001 .0001	.0002 .0002	.0001 .0001	.0002 .0002	.0004 .0004	.0001 .0001
Writing datacheck.log	.0070 .0055	.1155 .3964	.0034 .2120	.1442 .6740	.0103 .3064	.0074 .1270
Resume writing database file	.0096 .0076	.0093 .0096	.0060 .0065	.0110 .0125	.0120 .0267	.0097 .0118
Processed .* highway systems	.0149 .0070	.0731 .0663	.0073 .0034	.0834 .0756	.2571 .2821	.0554 .0415
WaypointQuadtree contains .* total nodes	.0031 .0026	.0020 .0011	.0009 .0005	.0020 .0012	.0035 .0025	.0016 .0008
Computing waypoint colocation stats, reporting all with .* or more colocations:	.7188 .5349	.4668 .3135	.4378 .2668	.5654 .3715	.8830 .6405	.8500 .6161

---

siteupdateST

top: default / -O0
bottom:-O3

BT	lab1	lab5	lab2	lab3	lab4	bsdlab
Reading region, country, and continent descriptions	.0036 .0013	.0040 .0012	.0029 .0012	.0027 .0010	.0039 .0015	.0031 .0009
Reading systems list	.6397 .3909	.4737 .2852	.3530 .1856	.5383 .3127	.7112 .4353	.6564 .3072
Finding all .wpt files. .* files found	.4961 .4385	.2337 .2036	.2552 .2217	.2928 .2605	.3907 .3362	.3948 .4494
Reading waypoints for all routes	11.9922 6.8816	6.9573 3.7160	6.9757 3.2935	9.2052 4.9499	11.7993 6.4641	10.9858 5.2846
Sorting waypoints in Quadtree	3.7683 1.2658	2.1093 1.1123	2.2986 .6838	2.5753 1.3455	3.4226 1.9014	3.5320 1.0321
Finding unprocessed wpt files	.0125 .0065	.0052 .0029	.0083 .0034	.0069 .0038	.0086 .0049	.0126 .0055
Near-miss point log and tm-master.nmp file	.6412 .4123	.3592 .2466	.4130 .2276	.4491 .3044	.6321 .4230	.6227 .3447
Writing near-miss point merged wpt files	3.0926 2.8874	1.7041 1.5618	1.6677 1.5484	2.1109 1.9298	2.7429 2.5422	2.5316 2.3818
Concurrent segment detection	.6867 .3152	.5042 .2661	.4466 .1698	.5980 .3108	.7718 .4466	.6731 .2637
Creating label hashes and checking route integrity	1.1644 .5300	.6740 .3712	.6442 .2768	.8472 .4538	1.1056 .5828	.9839 .4165
Reading updates file	.0275 .0260	.0172 .0171	.0153 .0141	.0216 .0212	.0282 .0276	.0233 .0218
Reading systemupdates file	.0007 .0005	.0004 .0004	.0003 .0002	.0005 .0005	.0008 .0007	.0005 .0004
Processing traveler list files:	13.6179 6.3435	7.7015 3.9958	8.3551 3.6810	9.1902 4.6261	11.6469 6.0119	12.4801 5.2961
Processed .* traveler list files. Sorting and numbering	.0004 .0002	.0002 .0001	.0003 .0001	.0002 .0001	.0003 .0001	.0004 .0001
Clearing route & label hash tables	.1888 .1021	.1163 .0831	.0994 .0467	.1227 .0948	.1763 .1263	.1464 .0646
Writing route and label logs	.3466 .0973	.2894 .1868	.2360 .0575	.3332 .2024	.4273 .2578	.3565 .0862
Augmenting travelers for detected concurrent segments	5.2394 3.2560	3.6148 2.2776	3.2230 1.7555	3.8917 2.4635	4.9246 3.1934	4.5379 2.4245
Performing per-route data checks and computing stats	6.3980 2.1974	3.3031 1.4002	3.8125 1.2916	4.0901 1.6904	5.2259 2.1860	5.7971 1.9178
Writing highway data stats log file	.1821 .1441	.1383 .1176	.1062 .0789	.1641 .1419	.2205 .1810	.1587 .1211
Creating per-traveler stats logs and augmenting data structure	33.0327 17.6778	22.0984 9.1960	22.8136 8.8497	25.5185 10.8630	31.0281 13.3940	32.5740 12.3124
Writing allbyregionactiveonly.csv	.5265 .3999	.2662 .2005	.2447 .1808	.3266 .2459	.4264 .3194	.3739 .2817
Writing allbyregionactivepreview.csv	.7377 .5503	.3683 .2768	.3391 .2477	.4527 .3389	.5878 .4419	.5149 .3764
Writing per-system stats csv files	.5858 .4284	.3042 .2198	.2875 .1998	.3793 .2707	.4877 .3503	.4395 .3032
Reading datacheckfps.csv	.0105 .0082	.0076 .0060	.0062 .0044	.0093 .0073	.0140 .0112	.0117 .0081
Setting up for graphs of highway data	.8156 .3793	.4780 .2131	.5889 .2225	.5792 .2507	.7701 .3280	.9136 .3393
Creating unique names and vertices	2.4403 1.4896	1.8919 1.2981	1.5840 .8554	2.2505 1.5051	2.9548 2.0886	2.4563 1.3441
Creating edges	1.4579 .8779	.8655 .5821	.9989 .5205	1.0688 .7136	1.3600 .9372	1.5436 .8290
Compressing collapsed edges	1.0787 .4365	.6181 .2622	.7423 .2459	.7520 .3064	.9315 .3788	1.1292 .3642
Writing graph waypoint simplification log	.0825 .0754	.0581 .0532	.0483 .0360	.0675 .0598	.0908 .0798	.0748 .0549
Writing master TM graph files	4.8957 4.0317	3.1352 2.6169	3.0301 2.5080	3.7402 3.1063	4.7022 3.9244	4.5410 3.7398
Creating continent graphs	10.2184 7.3478	7.0612 5.5430	7.3678 4.9343	8.2043 6.1743	10.2388 7.6388	10.8380 7.0800
Creating multisystem graphs	2.1258 1.6187	1.5534 1.3049	1.5238 1.1716	1.7180 1.4066	2.0785 1.6995	2.1780 1.6317
Creating system data graphs	2.3439 1.8517	1.7230 1.4693	1.6274 1.3050	1.8858 1.5847	2.2696 1.8999	2.3144 1.8196
Creating country graphs	7.1299 5.5012	5.1263 4.2029	4.9214 3.6760	5.6060 4.4994	7.0072 5.6147	7.0674 5.1529
Creating multiregion graphs	2.4047 1.9080	1.7733 1.4371	1.6478 1.2322	1.8688 1.4885	2.3483 1.8703	2.3331 1.6975
Creating regional data graphs	8.1909 6.5444	5.1538 4.1925	4.9798 3.8385	5.2359 4.1994	6.8920 5.4836	6.9225 5.1910
Creating area data graphs	.4980 .3532	.3075 .2414	.3101 .2271	.3401 .2599	.4312 .3278	.4553 .3193
Clearing HighwayGraph contents from memory	1.0810 .5862	.6474 .3921	.7220 .2973	.7885 .4555	1.0236 .5783	1.1198 .4625
Marking datacheck false positives	2.3850 .9341	1.6721 .8743	1.8886 .8538	2.3581 1.2464	3.0368 1.6190	3.3338 1.5672
Found .* datacheck errors and matched .* FP entries	0 0	.0009 .0008	0 0	.0004 .0004	.0006 .0007	0 0
Writing log of unmatched datacheck FP entries	.0001 .0001	.0001 .0001	.0001 .0001	.0002 .0001	.0002 .0002	.0002 .0002
Writing datacheck.log	.0055 .0043	.0041 .0032	.0036 .0025	.0041 .0030	.0055 .0044	.0051 .0035
...continents, countries, regions	.0003 .0001	.0002 .0001	.0002 .0001	.0002 .0002	.0003 .0003	.0003 .0002
...systems	.0005 .0003	.0004 .0003	.0003 .0002	.0005 .0004	.0006 .0006	.0005 .0003
...routes	.1487 .1259	.0988 .0924	.0949 .0864	.1206 .1117	.1526 .1496	.1422 .1289
...connectedRoutes	.1245 .1049	.0832 .0754	.0767 .0677	.0996 .0909	.1253 .1165	.1164 .1013
...connectedRouteRoots	.0064 .0025	.0030 .0013	.0027 .0014	.0034 .0014	.0048 .0020	.0041 .0018
...waypoints	2.9989 2.8391	1.7540 1.7053	1.6367 1.5768	2.1351 2.0774	2.8348 2.7818	2.5627 2.4527
...segments	7.1758 4.8770	5.7852 3.9942	4.5624 2.9252	6.8010 4.6551	8.6883 6.0985	6.8235 4.4124
...clinched$	.6012 .4295	.3699 .2406	.4440 .4050	.4548 .2971	.5945 .3825	.6760 .6096
...overallMileageByRegion	.0010 .0010	.0006 .0006	.0005 .0005	.0007 .0007	.0009 .0009	.0009 .0008
...systemMileageByRegion	.0024 .0022	.0014 .0014	.0014 .0013	.0017 .0017	.0022 .0021	.0025 .0020
...clinchedOverallMileageByRegion	.0393 .0358	.0228 .0213	.0227 .0211	.0279 .0260	.0356 .0331	.0343 .0318
...clinchedSystemMileageByRegion	.0750 .0690	.0445 .0430	.0441 .0425	.0538 .0518	.0691 .0659	.0680 .0642
...clinchedConnectedRoutes	.4465 .4123	.2657 .2507	.2624 .2546	.3266 .3048	.4095 .3907	.3995 .3831
...clinchedRoutes	.5297 .4961	.3104 .3073	.3025 .2957	.3802 .3715	.4887 .4683	.4547 .4427
...updates	.0088 .0037	.0066 .0035	.0054 .0020	.0083 .0047	.0107 .0062	.0082 .0031
...systemUpdates	.2458 .2145	.1511 .0997	.1342 .0855	.1910 .1307	.2484 .1715	.2016 .1261
...datacheckErrors	.0089 .0073	.0096 .0083	.0068 .0069	.0106 .0094	.0130 .0118	.0099 .0114
...graphs	.0013 .0007	.0010 .0007	.0009 .0005	.0012 .0008	.0015 .0011	.0014 .0008
Processed .* highway systems	.0149 .0070	.0690 .0614	.0073 .0033	.0780 .0687	.0913 .0974	.0105 .0046
WaypointQuadtree contains .* total nodes	.0031 .0027	.0019 .0011	.0009 .0005	.0020 .0012	.0025 .0015	.0009 .0005
Computing waypoint colocation stats, reporting all with .* or more colocations:	.5647 .3766	.4279 .2671	.4094 .2311	.5119 .3075	.6299 .3902	.5973 .3392

[yakra]

Forgot to mention -- These speed tests are set up to write all output files except for siteupdate.log to /dev/null.
This way, we can see what the program is capable of without a disk access bottleneck as a factor.
Relevant for subgraphs and especially nmp_merged files, the latter of which hasn't changed substantially in a while.
This will explain differences in benchmarks from older charts, such as those in yakra#188.