fix(systemtags): Add missing systemtags index

[marcelklehr]

• Resolves: Systemtag DB slowness

Summary

MariaDB [nextcloud]> explain select * from oc_systemtag_object_mapping where systemtagid = 808;
+------+-------------+-----------------------------+-------+---------------+---------+---------+------+-------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------------+-------+---------------+---------+---------+------+-------+--------------------------+
| 1 | SIMPLE | oc_systemtag_object_mapping | index | NULL | PRIMARY | 524 | NULL | 38418 | Using where; Using index |
+------+-------------+-----------------------------+-------+---------------+---------+---------+------+-------+--------------------------+
1 row in set (0.006 sec)
MariaDB [nextcloud]> CREATE INDEX marcels_systag_idx on oc_systemtag_object_mapping (systemtagid, objecttype);
Query OK, 0 rows affected (0.087 sec)
Records: 0 Duplicates: 0 Warnings: 0
MariaDB [nextcloud]> explain select * from oc_systemtag_object_mapping where systemtagid = 808;
+------+-------------+-----------------------------+------+---------------------+---------------------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------------+------+---------------------+---------------------+---------+-------+------+-------------+
| 1 | SIMPLE | oc_systemtag_object_mapping | ref | marcels_systag_idx | marcels_systag_idx | 8 | const | 626 | Using index |
+------+-------------+-----------------------------+------+---------------------+---------------------+---------+-------+------+-------------+

Before:

• rows: 38418

After:

• rows: 626

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[marcelklehr]

Can we backport this?

[nickvergessen]

Can we backport this?

I guess we should

[blizzz]

Looks promising also on postgres:

master=# explain analyze select * from oc_systemtag_object_mapping where systemtagid = 27;
QUERY PLAN ---------------------------------------------------------------------------------------------------------------------------
Seq Scan on oc_systemtag_object_mapping (cost=0.00..42.60 rows=169 width=21) (actual time=0.025..0.771 rows=145 loops=1)
Filter: (systemtagid = 27)
Rows Removed by Filter: 2063
Planning Time: 0.265 ms
Execution Time: 0.816 ms
(5 rows)
master=# CREATE INDEX marcels_systag_idx on oc_systemtag_object_mapping (systemtagid, objecttype);
CREATE INDEX
QUERY PLAN -----------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on oc_systemtag_object_mapping (cost=5.59..22.70 rows=169 width=21) (actual time=0.174..0.303 rows=145 loops=1)
Recheck Cond: (systemtagid = 27)
Heap Blocks: exact=12
-> Bitmap Index Scan on marcels_systag_idx (cost=0.00..5.55 rows=169 width=0) (actual time=0.143..0.144 rows=145 loops=1)
Index Cond: (systemtagid = 27)
Planning Time: 0.184 ms
Execution Time: 0.380 ms
(7 rows)

[nickvergessen]

The autoloaders are not up to date
Please run: bash build/autoloaderchecker.sh
And commit the result

[marcelklehr]

Do we actually need this migration or is the missing indices stuff enough? Or should I put this into the original v13... migration that added the systemtags table?

[nickvergessen]

yeah, actually it should be in the original one, otherwise it still performs the index creation on the update and that would nullify the rest of this PR.

[marcelklehr]

/backport to stable27

[marcelklehr]

/backport to stable26

[marcelklehr]

/backport to stable25

[marcelklehr]

Backports didn't happen?

[backportbot-nextcloud]

The backport to stable27 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable27
git pull origin stable27
# Create the new backport branch
git checkout -b fix/foo-stable27
# Cherry pick the change from the commit sha1 of the change against the default branch# This might cause conflicts. Resolve them.
git cherry-pick abc123
# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable27

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

[backportbot-nextcloud]

The backport to stable26 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable26
git pull origin stable26
# Create the new backport branch
git checkout -b fix/foo-stable26
# Cherry pick the change from the commit sha1 of the change against the default branch# This might cause conflicts. Resolve them.
git cherry-pick abc123
# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable26

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

[backportbot-nextcloud]

The backport to stable25 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable25
git pull origin stable25
# Create the new backport branch
git checkout -b fix/foo-stable25
# Cherry pick the change from the commit sha1 of the change against the default branch# This might cause conflicts. Resolve them.
git cherry-pick abc123
# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable25

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

[solracsf]

/backport to stable27

[solracsf]

/backport to stable26

[solracsf]

/backport to stable25

[backportbot-nextcloud]

The backport to stable27 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable27
git pull origin stable27
# Create the new backport branch
git checkout -b fix/foo-stable27
# Cherry pick the change from the commit sha1 of the change against the default branch# This might cause conflicts. Resolve them.
git cherry-pick abc123
# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable27

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

[backportbot-nextcloud]

The backport to stable26 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable26
git pull origin stable26
# Create the new backport branch
git checkout -b fix/foo-stable26
# Cherry pick the change from the commit sha1 of the change against the default branch# This might cause conflicts. Resolve them.
git cherry-pick abc123
# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable26

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

[backportbot-nextcloud]

The backport to stable25 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable25
git pull origin stable25
# Create the new backport branch
git checkout -b fix/foo-stable25
# Cherry pick the change from the commit sha1 of the change against the default branch# This might cause conflicts. Resolve them.
git cherry-pick abc123
# Push the cherry pick commit to the remote repository and open a pull request
git push origin fix/foo-stable25

More info at https://docs.nextcloud.com/server/latest/developer_manual/getting_started/development_process.html#manual-backport

[ernolf]

I don't know if this is the right place but I just updated 3 instances, two with MySQL, one with PostgreSQL to 27.0.1 RC1

The /settings/admin/overview throws this message:

but on all of the three instances

occ db:add-missing-indices

does not make any changes at all nor do i get the sql query with

occ db:add-missing-indices --dry-run

and the warning does not disapear.

MariaDB [nextcloud]> explain select * from oc_systemtag_object_mapping where systemtagid = 568;
+------+-------------+-----------------------------+-------+---------------+---------+---------+------+--------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------------------------+-------+---------------+---------+---------+------+--------+--------------------------+
| 1 | SIMPLE | oc_systemtag_object_mapping | index | NULL | PRIMARY | 524 | NULL | 143547 | Using where; Using index |
+------+-------------+-----------------------------+-------+---------------+---------+---------+------+--------+--------------------------+
1 row in set (0.000 sec)
MariaDB [nextcloud]> DESCRIBE oc_systemtag_object_mapping;
+-------------+---------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------------+------+-----+---------+-------+
| objectid | varchar(64) | NO | PRI | | |
| objecttype | varchar(64) | NO | PRI | | |
| systemtagid | bigint(20) unsigned | NO | PRI | 0 | |
+-------------+---------------------+------+-----+---------+-------+
3 rows in set (0.001 sec)
MariaDB [nextcloud]> SHOW INDEXES FROM oc_systemtag_object_mapping;
+-----------------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| oc_systemtag_object_mapping | 0 | PRIMARY | 1 | objecttype | A | 2 | NULL | NULL | | BTREE | | |
| oc_systemtag_object_mapping | 0 | PRIMARY | 2 | objectid | A | 143547 | NULL | NULL | | BTREE | | |
| oc_systemtag_object_mapping | 0 | PRIMARY | 3 | systemtagid | A | 143547 | NULL | NULL | | BTREE | | |
+-----------------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
3 rows in set (0.001 sec)

[blizzz]

@ernolf best is to open a new issue. Turned your comment into one now. Thanks for reporting!