Skip to content

Fix: Prevent deadlocks during mtime/size/etag propagation - #34302

Merged
icewind1991 merged 2 commits into
masterfrom
fix/propagation_deadlock_handling
Oct 3, 2022
Merged

Fix: Prevent deadlocks during mtime/size/etag propagation#34302
icewind1991 merged 2 commits into
masterfrom
fix/propagation_deadlock_handling

Conversation

@Raudius

@RaudiusRaudius commented Sep 28, 2022•
edited
Loading

Copy link
Copy Markdown
Contributor

When uploading many files in one go deadlocks can occur when propagating the changes. This PR performs two changes which should help prevent deadlocks:

  • Combines the 2 propagation queries into 1 which halves the amount of queries that can cause the deadlock to occur
  • Adds a retry loop in case the deadlock occurs

@RaudiusRaudius added bug 2. developing Work in progress labels Sep 28, 2022
@Raudius
Raudiusforce-pushed the fix/propagation_deadlock_handling branch 3 times, most recently from ff32a09 to 372e8a9CompareSeptember 28, 2022 10:21
@Raudius

Raudius commented Sep 28, 2022

Copy link
Copy Markdown
ContributorAuthor

Wondering whether we could move the retry logic to \OC\DB\QueryBuilder\QueryBuilder::executeStatement

@szaimenszaimen added this to the Nextcloud 26 milestone Sep 28, 2022
@szaimen
szaimen requested review from a team, CarlSchwan and juliusknorr and removed request for a teamSeptember 28, 2022 11:42
@RaudiusRaudius added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Sep 28, 2022
icewind1991
icewind1991 reviewed Sep 29, 2022
View reviewed changes
Comment threadlib/private/Files/Cache/Propagator.php
@Raudius
Raudiusforce-pushed the fix/propagation_deadlock_handling branch 3 times, most recently from 6c69d3c to 2176849CompareSeptember 29, 2022 17:02
CarlSchwan
CarlSchwan reviewed Sep 30, 2022
View reviewed changes
Comment threadlib/private/Files/Cache/Propagator.php Outdated
@Raudius
Raudiusforce-pushed the fix/propagation_deadlock_handling branch 2 times, most recently from 5491c55 to 3db1f6dCompareSeptember 30, 2022 10:09
icewind1991
icewind1991 reviewed Oct 1, 2022
View reviewed changes
Comment threadlib/private/Files/Cache/Propagator.php Outdated
@Raudius
Fix: Prevent deadlocks during mtime/size/etag propagation
45a1ebc
Signed-off-by: raul <raul@nextcloud.com>
@Raudius
Fix unencrypted size calculation for files created before the Encrypt…
95bc770
…ion storage is enabled
Signed-off-by: raul <raul@nextcloud.com>
@Raudius
Raudiusforce-pushed the fix/propagation_deadlock_handling branch from fa454bb to 95bc770CompareOctober 3, 2022 11:44
@juliusknorr

juliusknorr commented Oct 3, 2022

Copy link
Copy Markdown
Member

/backport to stable25

@juliusknorr

juliusknorr commented Oct 3, 2022

Copy link
Copy Markdown
Member

/backport to stable24

juliusknorr
juliusknorr reviewed Oct 3, 2022
View reviewed changes
Comment threadlib/private/Files/Cache/Propagator.php
@icewind1991
icewind1991 merged commit 27720d0 into masterOct 3, 2022
@icewind1991
icewind1991 deleted the fix/propagation_deadlock_handling branch October 3, 2022 14:23
@welcome

welcomeBot commented Oct 3, 2022

Copy link
Copy Markdown

Thanks for your first pull request and welcome to the community! Feel free to keep them coming! If you are looking for issues to tackle then have a look at this selection: https://github.com/nextcloud/server/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22

@backportbot-nextcloudbackportbot-nextcloudBot mentioned this pull request Oct 3, 2022
Merged
@backportbot-nextcloud

backportbot-nextcloudBot commented Oct 3, 2022

Copy link
Copy Markdown

The backport to stable24 failed. Please do this backport manually.

@RaudiusRaudius mentioned this pull request Oct 6, 2022
Merged
@solracsf

solracsf commented Oct 7, 2022•
edited
Loading

Copy link
Copy Markdown
Member

@Raudius there is a bunch of issues about deadlocks; can you please check if this PR can help on some of them? 👍
https://github.com/nextcloud/server/search?q=deadlock&state=open&type=issues

@RaudiusRaudius mentioned this pull request Oct 17, 2022
Closed
8 tasks
@RaudiusRaudius mentioned this pull request Oct 27, 2022
Merged
@juliusknorrjuliusknorr mentioned this pull request Jun 12, 2023
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CarlSchwanCarlSchwanCarlSchwan left review comments
@icewind1991icewind1991icewind1991 approved these changes
@juliusknorrjuliusknorrjuliusknorr approved these changes

Assignees

No one assigned

Labels

3. to reviewWaiting for reviewsbug

Projects

None yet

Milestone

Nextcloud 26

Development

Successfully merging this pull request may close these issues.

7 participants

@Raudius@juliusknorr@solracsf@icewind1991@CarlSchwan@skjnldsv@szaimen