Skip to content

Restrict query when searching for versions of trashbin files - #23097

Merged
MorrisJobke merged 1 commit into
masterfrom
bugfix/noid/trashbin-query-versions
Oct 14, 2020
Merged

Restrict query when searching for versions of trashbin files#23097
MorrisJobke merged 1 commit into
masterfrom
bugfix/noid/trashbin-query-versions

Conversation

@juliusknorr

Copy link
Copy Markdown
Member

The current way of obtaining the versions from files in the trashbin is quite slow since it does a like operation on the unindexed name column. Since we know the full path of the file in the trashbin we can limit the search to the parent id. This should drastically speed up query times on large file cache tables.

We can ignore other things than single file restores since all usages have a separate code path for that.

@ChristophWurstChristophWurst left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 🐘

@rullzer

Copy link
Copy Markdown
Member

nice one. but I'd really love the input from our storage overloard @icewind1991

@juliusknorr

Copy link
Copy Markdown
MemberAuthor

Yes, I'd highly appreciate a closer look from @icewind1991 as well.

Some more comments on my validation paths since I also don't know that code very well.

I've tested this to still work when:

  • Restore a file with versions
  • Restore a file with versions from within a deleted folder
  • Restore a deleted folder containing files with versions

View::searchCommon also includes mountpoints in the search, so this is no longer the case however I assumed there should never be any mountpoints inside of the /[uid]/files_trashbin/versions fake root.

$mounts = Filesystem::getMountManager()->findIn($this->fakeRoot);
foreach ($mountsas$mount) {
$mountPoint = $mount->getMountPoint();
$storage = $mount->getStorage();
if ($storage) {
$cache = $storage->getCache('');
$relativeMountPoint = substr($mountPoint, $rootLength);
$results = call_user_func_array([$cache, $method], $args);
if ($results) {
foreach ($resultsas$result) {
$internalPath = $result['path'];
$result['path'] = rtrim($relativeMountPoint . $result['path'], '/');
$path = rtrim($mountPoint . $internalPath, '/');
$owner = \OC::$server->getUserManager()->get($storage->getOwner($internalPath));
$files[] = newFileInfo($path, $storage, $internalPath, $result, $mount, $owner);
}
}
}

@rullzer

Copy link
Copy Markdown
Member

mmmm so for some reason the static analysis complains. Lets summon the wizards: @MorrisJobke@kesselb

@kesselb

kesselb commented Oct 3, 2020

Copy link
Copy Markdown
Contributor

Oh. I'm not sure what Psalm is complaining about 🙈

  • array_map always returns an array hence the is_array check is not longer required (but that should be a different line).
  • $matches = array_map there are two spaces before array_map (but this is also another line - and a job for php-cs-fixer anyway)
  • $ma inside the foreach is resolved (by PHPStorm) to mixed|OC\Files\Cache\CacheEntry.

image

Mind to try the above version? I'm not sure if Psalm is reading the inline /** @var **/ annotation. The above code works at least with PHPStrom and $ma is resolved correctly.

@rullzerrullzer added this to the Nextcloud 21 milestone Oct 4, 2020
Comment threadapps/files_trashbin/lib/Trashbin.php
@juliusknorr
juliusknorrforce-pushed the bugfix/noid/trashbin-query-versions branch from b2b655f to 62403c7CompareOctober 7, 2020 09:20
@juliusknorr

Copy link
Copy Markdown
MemberAuthor

Let's see if psalm is happy with that

@MorrisJobke

Copy link
Copy Markdown
Member

Let's see if psalm is happy with that

Looks good 👍 Even one error is "fixed". 😆

@MorrisJobkeMorrisJobke left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code itself makes sense 👍

@juliusknorr
juliusknorrforce-pushed the bugfix/noid/trashbin-query-versions branch from 87a0e0c to 7e6f0dfCompareOctober 14, 2020 12:18
Comment threadapps/files_trashbin/lib/Trashbin.php
@juliusknorr
juliusknorrforce-pushed the bugfix/noid/trashbin-query-versions branch from 7e6f0df to 2243f58CompareOctober 14, 2020 14:38
Signed-off-by: Julius Härtl <jus@bitgrid.net>
@juliusknorr
juliusknorrforce-pushed the bugfix/noid/trashbin-query-versions branch from 2243f58 to 2616a78CompareOctober 14, 2020 14:38
@faily-bot

faily-botBot commented Oct 14, 2020

Copy link
Copy Markdown

🤖 beep boop beep 🤖

Here are the logs for the failed build:

Status of 34140: failure

mariadb10.1-php7.3

mysql8.0-php7.4

@juliusknorrjuliusknorr added 4. to release Ready to be released and/or waiting for tests to finish and removed 3. to review Waiting for reviews labels Oct 14, 2020
@MorrisJobke
MorrisJobke merged commit 02fa6a7 into masterOct 14, 2020
@MorrisJobke
MorrisJobke deleted the bugfix/noid/trashbin-query-versions branch October 14, 2020 20:50
@MorrisJobke

Copy link
Copy Markdown
Member

@juliushaertl Backport or not?

@MorrisJobke

Copy link
Copy Markdown
Member

ping @juliushaertl

@juliusknorr

Copy link
Copy Markdown
MemberAuthor

/backport to stable20

@juliusknorr

Copy link
Copy Markdown
MemberAuthor

/backport to stable19

@juliusknorr

Copy link
Copy Markdown
MemberAuthor

/backport to stable18

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4. to releaseReady to be released and/or waiting for tests to finishbug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@juliusknorr@rullzer@kesselb@MorrisJobke@nickvergessen@icewind1991@ChristophWurst