Skip to content

Use paginated search for contacts - #18816

Merged
blizzz merged 1 commit into
masterfrom
bugfix/noid/paginate-contacts-search
Apr 23, 2020
Merged

Use paginated search for contacts#18816
blizzz merged 1 commit into
masterfrom
bugfix/noid/paginate-contacts-search

Conversation

@juliusknorr

Copy link
Copy Markdown
Member

This PR drastically improves the performance of searching for sharees when the system addressbook contains a lot of users. Without it all matching contact entries would be queried from the database for both SHARE_TYPE_REMOTE and SHARE_TYPE_EMAIL which led to quite bad performance when searching with queries that matched a lot of entries (1-3 characters)

@juliusknorrjuliusknorr added bug 3. to review Waiting for reviews labels Jan 10, 2020
@juliusknorrjuliusknorr added this to the Nextcloud 19 milestone Jan 10, 2020
@juliusknorr
juliusknorrforce-pushed the bugfix/noid/paginate-contacts-search branch from 6f5ba0d to c41a99bCompareJanuary 10, 2020 15:24
Comment threadapps/dav/lib/CardDAV/CardDavBackend.php Outdated
// Search in contacts
//@todo Pagination missing
$addressBookContacts = $this->contactsManager->search($search, ['CLOUD', 'FN']);
$addressBookContacts = $this->contactsManager->search($search, ['CLOUD', 'FN'], ['limit' => $limit, 'offset' => $offset]);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is, if you e.g. match the local domain, you will get all system address book entries.
But they are skipped with the first if in the loop, but also no further contacts will be found, although you would have matches.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contacts from other addressbooks should still be found, since the search is executed with the limit on each addressbook individually: https://github.com/nextcloud/server/pull/18816/files#diff-905974f5907f69fe7586a80ea67117a6L48

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stil concerns @nickvergessen ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets try it

This was referenced Apr 4, 2020
@nickvergessen
nickvergessen requested review from nickvergessen and removed request for nickvergessenApril 15, 2020 10:34
@nickvergessennickvergessen added 4. to release Ready to be released and/or waiting for tests to finish and removed 3. to review Waiting for reviews labels Apr 15, 2020
@nickvergessen

Copy link
Copy Markdown
Member

Needs a rebase

This was referenced Apr 15, 2020
@MorrisJobke
MorrisJobkeforce-pushed the bugfix/noid/paginate-contacts-search branch from 043c013 to 8fd2031CompareApril 17, 2020 07:09
@MorrisJobke

Copy link
Copy Markdown
Member

Needs a rebase

Done.

@blizzz

Copy link
Copy Markdown
Member

CI looks quite red

@rullzerrullzer mentioned this pull request Apr 23, 2020
11 tasks
@juliusknorr
juliusknorrforce-pushed the bugfix/noid/paginate-contacts-search branch from 8fd2031 to 17104b5CompareApril 23, 2020 14:27
Signed-off-by: Julius Härtl <jus@bitgrid.net>
@juliusknorr
juliusknorrforce-pushed the bugfix/noid/paginate-contacts-search branch from 17104b5 to 6709833CompareApril 23, 2020 17:08
@juliusknorr

Copy link
Copy Markdown
MemberAuthor

Aw, isset is no function so using \isset was failing 🙈

@blizzz
blizzz merged commit 038045d into masterApr 23, 2020
@blizzz
blizzz deleted the bugfix/noid/paginate-contacts-search branch April 23, 2020 20:21
@nickvergessen

Copy link
Copy Markdown
Member

An exception occurred while executing "SELECT c.carddata, c.uri FROM oc_cardsc WHERE c.id IN (SELECT DISTINCT cp.cardid FROM oc_cards_propertiescp WHERE (cp.addressbookid = ?) AND ((cp.name = ?) OR (cp.name = ?)) AND (cp.value COLLATE utf8mb4_general_ci LIKE ?) LIMIT 10)" with params ["1", "EMAIL", "FN", "%tes%"]:\n\nSQLSTATE[42000]: Syntax error or access violation: 1235 This version of MySQL doesn"t yet support "LIMIT & IN/ALL/ANY/SOME subquery"

:(

@J0WI

J0WI commented Apr 29, 2020

Copy link
Copy Markdown
Contributor
current syntax
SELECT`c`.`carddata`, `c`.`uri`FROM`oc_cards``c`WHERE`c`.`id`IN (
SELECT DISTINCT`cp`.`cardid`FROM`oc_cards_properties``cp`WHERE (`cp`.`addressbookid`= ?)
AND ((`cp`.`name`= ?)
OR (`cp`.`name`= ?))
AND (`cp`.`value` COLLATE utf8mb4_general_ci LIKE ?)
LIMIT10
);
@nickvergessen AFAIK you can just use an additional dummy subquery as a workaround. Can you try something like:
SELECT`c`.`carddata`, `c`.`uri`FROM`oc_cards``c`WHERE`c`.`id`IN (
SELECT`cardid`FROM (
SELECT DISTINCT`cp`.`cardid`FROM`oc_cards_properties``cp`WHERE (`cp`.`addressbookid`= ?)
AND ((`cp`.`name`= ?)
OR (`cp`.`name`= ?))
AND (`cp`.`value` COLLATE utf8mb4_general_ci LIKE ?)
LIMIT10
)
AS compatlayer
);

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4. to releaseReady to be released and/or waiting for tests to finishbugperformance 🚀

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@juliusknorr@nickvergessen@MorrisJobke@blizzz@J0WI@ChristophWurst@kesselb