Uh oh!
There was an error while loading. Please reload this page.
fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868 - #6192
Conversation
d492d1c to
9e50226CompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
… run multiprocess on CI d5a944d Revert: fix: missing changes from bitcoin#19267 - run multiprocess on CI (pasta) Pull request description: ## Issue being fixed or feature implemented See dashpay#6192 for a discussion around this: for now simply make CI happy again, by not running for multiprocessor (since this is new, and unstable / dev feature, and not something shipped). ## What was done? Revert commit which reintroduces multiprocessor CI while we investigate proper fix ## How Has This Been Tested? CI ## Breaking Changes ## Checklist: _Go over all the following points, and put an `x` in all the boxes that apply._ - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ Top commit has no ACKs. Tree-SHA512: d878b02efc72019e09b09e2482e63dca276824b66272135e00ff340ba60b5b58e5f5ae3feff3b894f08958ca8fc6ce71f91c926f6aaf29a459d6230f129316f6
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…#22937 and ipc/process 540f687 fix: lock `::cs_main` before accessing `ChainstateManager::m_best_header` (Kittywhiskers Van Gogh) aafded6 fix: compilation error due to rebase error between bitcoin#22937 and ipc/process (Kittywhiskers Van Gogh) Pull request description: ## Issue being fixed or feature implemented CI failure for multiprocess ## What was done? It resolve compilation failure on develop (apply changes from 22937 to ipc/process) See alternate solutions: - #6192 - #6195 ## How Has This Been Tested? Run build locally: ``` make MULTIPROCESS=1 -j10 ./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings --enable-werror --enable-debug --enable-crash-hooks --enable-maintainer-mode --enable-stacktraces --enable-multi-process ``` ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: PastaPastaPasta: utACK 540f687 Tree-SHA512: 75b675e93763e8e53086a10b93516b8ec94418902f9be2de738517176cc835c0dad25c286426089a5327a9c13d1787b89debda2c6025cb598489204d7d5af2cf
fb186ef to
dc0fbcdCompareUdjinM6
commented
Aug 12, 2024
my bad, should've run linter locally 🙈 pls also check 2d6612a |
e95dd63 to
1c55375CompareUh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
should we use std::for_each here?
There was a problem hiding this comment.
We would not be able to do the below snippet because the function isn't taking in a pair but one only one half of it
std::for_each(m_wallet_manager_map.begin(), m_wallet_manager_map.end(), func);There was a problem hiding this comment.
could do something ```suggestion
ranges::for_each(m_wallet_manager_map, [&](auto [_, clientman]){func(clientman)});
There was a problem hiding this comment.
can't use structured bindings there, would be more like 019a780 instead
There was a problem hiding this comment.
I don't like to use std::for_each if it's not super trivial.
ranges::for_each(m_wallet_manager_map, [&](auto [_, clientman]){func(clientman)});
it looks like extra more difficult to read, I disagree with this suggestment.
For-each has been a great option when there was no range-loop in C++! But in this particular case it looks difficult to read, difficult to understand. That's a case when 2 lines better than one!
using any_of below makes sense, but not here.
PastaPastaPasta
commented
Aug 20, 2024
Can you squash these two commits? |
can be fixed by backporting bitcoin#22868 or via a temporary fix |
Avoid TSan-reported data race ``` WARNING: ThreadSanitizer: data race (pid=374820) Read of size 8 at 0x7b140002ce10 by thread T12: #0 _M_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:154:42 (dashd+0xb58e08) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408) #1 get /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:361:21 (dashd+0xb58e08) #2 operator-> /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:355:9 (dashd+0xb58e08) #3 CoinJoinWalletManager::DoMaintenance() /src/dash/src/coinjoin/client.cpp:1907:9 (dashd+0xb58e08) [...] Previous write of size 8 at 0x7b140002ce10 by thread T17 (mutexes: write M0): #0 operator new(unsigned long) <null> (dashd+0x162657) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408) #1 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/ext/new_allocator.h:114:27 (dashd+0xb772b4) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408) #2 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/alloc_traits.h:443:20 (dashd+0xb772b4) #3 _M_get_node /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_tree.h:580:16 (dashd+0xb772b4) [...] dashpay#8 CoinJoinWalletManager::Add(CWallet&) /src/dash/src/coinjoin/client.cpp:1898:26 (dashd+0xb58c73) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408) [...] SUMMARY: ThreadSanitizer: data race [...] ```
excludes changes in `wallet/context.h` due to `::vpwallets` (and thus, `cs_wallets`) not being deglobalized yet
…#22937 and ipc/process bdce842 fix: lock `::cs_main` before accessing `ChainstateManager::m_best_header` (Kittywhiskers Van Gogh) 42ad8d1 fix: compilation error due to rebase error between bitcoin#22937 and ipc/process (Kittywhiskers Van Gogh) Pull request description: ## Issue being fixed or feature implemented CI failure for multiprocess ## What was done? It resolve compilation failure on develop (apply changes from 22937 to ipc/process) See alternate solutions: - dashpay/dash#6192 - dashpay/dash#6195 ## How Has This Been Tested? Run build locally: ``` make MULTIPROCESS=1 -j10 ./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings --enable-werror --enable-debug --enable-crash-hooks --enable-maintainer-mode --enable-stacktraces --enable-multi-process ``` ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: PastaPastaPasta: utACK bdce842 Tree-SHA512: 75b675e93763e8e53086a10b93516b8ec94418902f9be2de738517176cc835c0dad25c286426089a5327a9c13d1787b89debda2c6025cb598489204d7d5af2cf
… with Mutex, avoid data race, partial bitcoin#22868 50547cc partial bitcoin#22868: Call load handlers without cs_wallet locked (Kittywhiskers Van Gogh) 84f1113 coinjoin: protect `m_wallet_manager_map` with `cs_wallet_manager_map` (Kittywhiskers Van Gogh) Pull request description: ## Additional Information This pull request aims to deal with regressions ([build](https://gitlab.com/dashpay/dash/-/jobs/7550859495)) spotted in `develop` after the merger of [dash#6143](dashpay/dash#6143), namely, a failure to build the multiprocess variant and two data races, one involving `ChainstateManager::m_best_header` and another involving `CoinJoinWalletManager::m_wallet_manager_map`. * Fixes for the build failure and the first data race (b136742f98c6792735232cc170123f849fd13bad and 9e7c6850139e69cb64cfe89813e6f0e4b1e626e1), have been spun off into [dash#6199](dashpay/dash#6199) and merged * The second data race is being avoided by protected `m_wallet_manager_map` with a new `RecursiveMutex`, `cs_wallet_manager_map` (the contents of this PR). ~~A version of these changes are available using a regular `Mutex` but prove far more cumbersome ([source](dashpay/dash@b89457d)) when taking practicalities into account ([comment](dashpay/dash#6192 (comment) A `Mutex` version is now available courtesy of UdjinM6 (see [patch](dashpay/dash@ecf5c1b)), thanks! ## Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)** - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation **(note: N/A)** - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: UdjinM6: light-ACK 50547cc Tree-SHA512: 52545a1547357a45fb6bf4d3948fc75a5e88f1d86e9809b1060007cd74dd383249691d8d9777f647c7534d458fe126ae818fa2b47f0e5b0cee78a469af1ed0c9
Additional Information
This pull request aims to deal with regressions (build) spotted in
developafter the merger of dash#6143, namely, a failure to build the multiprocess variant and two data races, one involvingChainstateManager::m_best_headerand another involvingCoinJoinWalletManager::m_wallet_manager_map.Fixes for the build failure and the first data race (b136742 and 9e7c685), have been spun off into dash#6199 and merged
The second data race is being avoided by protected
m_wallet_manager_mapwith a newRecursiveMutex,cs_wallet_manager_map(the contents of this PR).A version of these changes are available using a regularAMutexbut prove far more cumbersome (source) when taking practicalities into account (comment).Mutexversion is now available courtesy of UdjinM6 (see patch), thanks!Checklist: