Skip to content

fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868 - #6192

Merged
PastaPastaPasta merged 2 commits into
dashpay:developfrom
kwvg:6143_fixup
Aug 29, 2024
Merged

fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868#6192
PastaPastaPasta merged 2 commits into
dashpay:developfrom
kwvg:6143_fixup

Conversation

@kwvg

@kwvgkwvg commented Aug 10, 2024

Copy link
Copy Markdown
Collaborator

Additional Information

This pull request aims to deal with regressions (build) spotted in develop after the merger of dash#6143, namely, a failure to build the multiprocess variant and two data races, one involving ChainstateManager::m_best_header and another involving CoinJoinWalletManager::m_wallet_manager_map.

  • Fixes for the build failure and the first data race (b136742 and 9e7c685), have been spun off into dash#6199 and merged

  • The second data race is being avoided by protected m_wallet_manager_map with a new RecursiveMutex, cs_wallet_manager_map (the contents of this PR). A version of these changes are available using a regular Mutex but prove far more cumbersome (source) when taking practicalities into account (comment). A Mutex version is now available courtesy of UdjinM6 (see patch), thanks!

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas (note: N/A)
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation (note: N/A)
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

@kwvgkwvg added this to the 21.2 milestone Aug 10, 2024
@kwvgkwvg changed the title fix: resolve compile failure in multiprocess build due to missing std::filesystem preparation changesfix: resolve compile failure in multiprocess build due to missing std::filesystem preparation changes, data racesAug 10, 2024
@kwvg
kwvg marked this pull request as ready for review August 10, 2024 21:22
@kwvg
kwvgforce-pushed the 6143_fixup branch 3 times, most recently from d492d1c to 9e50226CompareAugust 11, 2024 08:34
Comment threadsrc/coinjoin/client.cpp Outdated
Comment threadsrc/rpc/coinjoin.cpp Outdated
PastaPastaPasta added a commit to PastaPastaPasta/dash that referenced this pull request Aug 11, 2024
… run multiprocess on CI
d5a944d Revert: fix: missing changes from bitcoin#19267 - run multiprocess on CI (pasta)
Pull request description:
## Issue being fixed or feature implemented
See dashpay#6192 for a discussion around this: for now simply make CI happy again, by not running for multiprocessor (since this is new, and unstable / dev feature, and not something shipped).
## What was done?
Revert commit which reintroduces multiprocessor CI while we investigate proper fix
## How Has This Been Tested?
CI
## Breaking Changes
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
Top commit has no ACKs.
Tree-SHA512: d878b02efc72019e09b09e2482e63dca276824b66272135e00ff340ba60b5b58e5f5ae3feff3b894f08958ca8fc6ce71f91c926f6aaf29a459d6230f129316f6
Comment threadsrc/coinjoin/interfaces.cpp Outdated
Comment threadsrc/wallet/wallet.cpp Outdated
Comment threadsrc/dsnotificationinterface.cpp
Comment threadsrc/coinjoin/client.cpp Outdated
Comment threadsrc/coinjoin/client.cpp Outdated
Comment threadsrc/rpc/coinjoin.cpp Outdated
PastaPastaPasta added a commit that referenced this pull request Aug 12, 2024
…#22937 and ipc/process
540f687 fix: lock `::cs_main` before accessing `ChainstateManager::m_best_header` (Kittywhiskers Van Gogh)
aafded6 fix: compilation error due to rebase error between bitcoin#22937 and ipc/process (Kittywhiskers Van Gogh)
Pull request description:
## Issue being fixed or feature implemented
CI failure for multiprocess
## What was done?
It resolve compilation failure on develop (apply changes from 22937 to ipc/process)
See alternate solutions:
- #6192
- #6195
## How Has This Been Tested?
Run build locally:
```
make MULTIPROCESS=1 -j10
./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings --enable-werror --enable-debug --enable-crash-hooks --enable-maintainer-mode --enable-stacktraces --enable-multi-process
```
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
PastaPastaPasta:
utACK 540f687
Tree-SHA512: 75b675e93763e8e53086a10b93516b8ec94418902f9be2de738517176cc835c0dad25c286426089a5327a9c13d1787b89debda2c6025cb598489204d7d5af2cf
@kwvg
kwvg marked this pull request as draft August 12, 2024 12:45
@kwvgkwvg changed the title fix: resolve compile failure in multiprocess build due to missing std::filesystem preparation changes, data racesfix: protect CoinJoinWalletManager::m_wallet_manager_map with RecursiveMutex, avoid data raceAug 12, 2024
@kwvg
kwvgforce-pushed the 6143_fixup branch 2 times, most recently from fb186ef to dc0fbcdCompareAugust 12, 2024 19:06
@UdjinM6

Copy link
Copy Markdown

my bad, should've run linter locally 🙈 pls also check 2d6612a

@kwvg
kwvgforce-pushed the 6143_fixup branch 2 times, most recently from e95dd63 to 1c55375CompareAugust 12, 2024 19:33
@kwvgkwvg changed the title fix: protect CoinJoinWalletManager::m_wallet_manager_map with RecursiveMutex, avoid data racefix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data raceAug 13, 2024
@kwvg
kwvg marked this pull request as ready for review August 15, 2024 11:44
Comment threadsrc/coinjoin/client.h Outdated
Comment threadsrc/coinjoin/client.h Outdated
Comment on lines 101 to 104

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use std::for_each here?

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would not be able to do the below snippet because the function isn't taking in a pair but one only one half of it

std::for_each(m_wallet_manager_map.begin(), m_wallet_manager_map.end(), func);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could do something ```suggestion
ranges::for_each(m_wallet_manager_map, [&](auto [_, clientman]){func(clientman)});

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@UdjinM6, thoughts?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't use structured bindings there, would be more like 019a780 instead

@knstknstAug 26, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like to use std::for_each if it's not super trivial.

ranges::for_each(m_wallet_manager_map, [&](auto [_, clientman]){func(clientman)});

it looks like extra more difficult to read, I disagree with this suggestment.

For-each has been a great option when there was no range-loop in C++! But in this particular case it looks difficult to read, difficult to understand. That's a case when 2 lines better than one!

using any_of below makes sense, but not here.

@PastaPastaPasta

Copy link
Copy Markdown
Member

Can you squash these two commits?

@UdjinM6

UdjinM6 commented Aug 25, 2024

Copy link
Copy Markdown
POTENTIAL DEADLOCK DETECTED
Previous lock order was:
(2) 'walletInstance->cs_wallet' in wallet/wallet.cpp:4952 (in thread 'qt-init')
(1) 'cs_wallet_manager_map' in coinjoin/client.cpp:1903 (in thread 'qt-init')
Current lock order is:
(1) 'cs_wallet_manager_map' in coinjoin/client.cpp:1913 (in thread 'scheduler')
'cs_deqsessions' in coinjoin/client.cpp:998 (in thread 'scheduler')
(2) 'm_wallet.cs_wallet' in coinjoin/client.cpp:804 (in thread 'scheduler')

can be fixed by backporting bitcoin#22868 or via a temporary fix

diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp
index f648fc2ced..6a33585ede 100644
--- a/src/wallet/wallet.cpp
+++ b/src/wallet/wallet.cpp
@@ -4949,8 +4949,6 @@ std::shared_ptr<CWallet> CWallet::Create(interfaces::Chain* chain, interfaces::C
// Try to top up keypool. No-op if the wallet is locked.
walletInstance->TopUpKeyPool();
- LOCK(walletInstance->cs_wallet);
-
if (chain && !AttachChain(walletInstance, *chain, error, warnings)) {
return nullptr;
}
@@ -4959,6 +4957,8 @@ std::shared_ptr<CWallet> CWallet::Create(interfaces::Chain* chain, interfaces::C
coinjoin_loader->AddWallet(*walletInstance);
}
+ LOCK(walletInstance->cs_wallet);
+
{
LOCK(cs_wallets);
for (auto& load_wallet : g_load_wallet_fns) {

kwvg added 2 commits August 26, 2024 15:55
Avoid TSan-reported data race
```
WARNING: ThreadSanitizer: data race (pid=374820)
Read of size 8 at 0x7b140002ce10 by thread T12:
#0 _M_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:154:42 (dashd+0xb58e08) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
#1 get /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:361:21 (dashd+0xb58e08)
#2 operator-> /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:355:9 (dashd+0xb58e08)
#3 CoinJoinWalletManager::DoMaintenance() /src/dash/src/coinjoin/client.cpp:1907:9 (dashd+0xb58e08)
[...]
Previous write of size 8 at 0x7b140002ce10 by thread T17 (mutexes: write M0):
#0 operator new(unsigned long) <null> (dashd+0x162657) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
#1 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/ext/new_allocator.h:114:27 (dashd+0xb772b4) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
#2 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/alloc_traits.h:443:20 (dashd+0xb772b4)
#3 _M_get_node /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_tree.h:580:16 (dashd+0xb772b4)
[...]
dashpay#8 CoinJoinWalletManager::Add(CWallet&) /src/dash/src/coinjoin/client.cpp:1898:26 (dashd+0xb58c73) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
[...]
SUMMARY: ThreadSanitizer: data race [...]
```
excludes changes in `wallet/context.h` due to `::vpwallets` (and thus,
`cs_wallets`) not being deglobalized yet
@kwvgkwvg changed the title fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data racefix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868Aug 26, 2024

@UdjinM6UdjinM6 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (to be merged via merge commit)

light-ACK 87d775d

@PastaPastaPasta
PastaPastaPasta merged commit 6dbed2a into dashpay:developAug 29, 2024
@UdjinM6UdjinM6 modified the milestones: 21.2, 22Oct 29, 2024
ab4cus pushed a commit to ab4cus/e4Coin-core that referenced this pull request Jul 8, 2026
…#22937 and ipc/process
bdce842 fix: lock `::cs_main` before accessing `ChainstateManager::m_best_header` (Kittywhiskers Van Gogh)
42ad8d1 fix: compilation error due to rebase error between bitcoin#22937 and ipc/process (Kittywhiskers Van Gogh)
Pull request description:
## Issue being fixed or feature implemented
CI failure for multiprocess
## What was done?
It resolve compilation failure on develop (apply changes from 22937 to ipc/process)
See alternate solutions:
- dashpay/dash#6192
- dashpay/dash#6195
## How Has This Been Tested?
Run build locally:
```
make MULTIPROCESS=1 -j10
./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings --enable-werror --enable-debug --enable-crash-hooks --enable-maintainer-mode --enable-stacktraces --enable-multi-process
```
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
PastaPastaPasta:
utACK bdce842
Tree-SHA512: 75b675e93763e8e53086a10b93516b8ec94418902f9be2de738517176cc835c0dad25c286426089a5327a9c13d1787b89debda2c6025cb598489204d7d5af2cf
ab4cus pushed a commit to ab4cus/e4Coin-core that referenced this pull request Jul 8, 2026
… with Mutex, avoid data race, partial bitcoin#22868
50547cc partial bitcoin#22868: Call load handlers without cs_wallet locked (Kittywhiskers Van Gogh)
84f1113 coinjoin: protect `m_wallet_manager_map` with `cs_wallet_manager_map` (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
This pull request aims to deal with regressions ([build](https://gitlab.com/dashpay/dash/-/jobs/7550859495)) spotted in `develop` after the merger of [dash#6143](dashpay/dash#6143), namely, a failure to build the multiprocess variant and two data races, one involving `ChainstateManager::m_best_header` and another involving `CoinJoinWalletManager::m_wallet_manager_map`.
* Fixes for the build failure and the first data race (b136742f98c6792735232cc170123f849fd13bad and 9e7c6850139e69cb64cfe89813e6f0e4b1e626e1), have been spun off into [dash#6199](dashpay/dash#6199) and merged
* The second data race is being avoided by protected `m_wallet_manager_map` with a new `RecursiveMutex`, `cs_wallet_manager_map` (the contents of this PR). ~~A version of these changes are available using a regular `Mutex` but prove far more cumbersome ([source](dashpay/dash@b89457d)) when taking practicalities into account ([comment](dashpay/dash#6192 (comment) A `Mutex` version is now available courtesy of UdjinM6 (see [patch](dashpay/dash@ecf5c1b)), thanks!
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation **(note: N/A)**
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
light-ACK 50547cc
Tree-SHA512: 52545a1547357a45fb6bf4d3948fc75a5e88f1d86e9809b1060007cd74dd383249691d8d9777f647c7534d458fe126ae818fa2b47f0e5b0cee78a469af1ed0c9
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@kwvg@UdjinM6@PastaPastaPasta@knst