Fix race in parallel adjacent_find - #816
Conversation
Check cancellation status _before_ getting a chunk of work to process, instead of after. If we check after and a result was found in a later block than the one we were assigned, we won't notice a better result in our block.
Billy O'Neal (BillyONeal)
left a comment
There was a problem hiding this comment.
Also checked the other find algorithms and they already did this.
Stephan T. Lavavej (StephanTLavavej)
left a comment
There was a problem hiding this comment.
Great catch! This looks correct to me, but Miya Natsuhara (@mnatsuhara) is more qualified to provide final signoff.
After this change, I observe that we now have 8 occurrences of the exact same pattern:
Lines 1291 to 1298 in 11150ca
(
_Static_partitioned_mismatch2::_Process_chunk() is very slightly different, testing _Results._Storage._Complete(), but it could be made completely uniform by adding a helper function to _Static_partitioned_mismatch_results.)
Can/should we centralize this pattern? It looks like intrusively modifying _Static_partition_team::_Get_next_key() to take _Results and inspect _Results._Complete() would disrupt other _Get_next_key() callsites and mix in something that the _Static_partition_team shouldn't be concerned with, but could we add a non-member function that takes both _Results and _Team? The callers will still need if (!_Key) { return _Cancellation_status::_Canceled; } but this would give us an opportunity to add a centralized comment about the surprisingly subtle control flow here (otherwise only captured in your PR description) and would make it structurally easier to do the right thing in the future (for new algorithms or maintenance of existing algorithms).
I spent about an hour tracing through code looking for the race, maybe two minutes figuring out how to fix it, roughly a minute discovering that all of the other
Latest changes include a non-member function template I'll reset reviews since this is a much more substantial change than the initial bugfix. |
This reverts commit 23f932c.
This is once again unresolved: I reverted my attempt after discovering detrimental codegen. |
Stephan T. Lavavej (StephanTLavavej)
left a comment
There was a problem hiding this comment.
Thanks for double-checking the codegen!
|
Thanks race condition! for fixing this |
Check cancellation status before getting a chunk of work to process, instead of after. If we check after and a result was found in a later block than the one we were assigned, we won't notice a better result in our block.
I think this is the cause of the extremely infrequent test failures we're seeing, e.g. https://dev.azure.com/vclibs/STL/_build/results?buildId=2983&view=ms.vss-test-web.build-test-results-tab&runId=1006240&resultId=115817&paneView=debug.