Add MariaDB 10.6 support (drop row_format=compressed)

[solracsf]

Keeping MariaDB 10.4 too as both versions have some BC breaks, so tests will run on both (for now).

[solracsf]

Should be good now, but regarding e49233a should the conversion just be dropped (as on the commit) or should we convert actual COMPRESSED tables to DYNAMIC? 🤔

[solracsf]

We can also recommend (documentation only) to use https://mariadb.com/kb/en/innodb-page-compression/#configuring-the-innodb-page-compression-algorithm

for newly created tables
https://mariadb.com/kb/en/innodb-page-compression/#enabling-innodb-page-compression-by-default

[mariadb]
...
innodb_compression_default=ON (defaults to OFF)

as recommended by MariaDB : https://mariadb.com/kb/en/innodb-page-compression/#comparison-with-the-compressed-row-format

[mariadb]
...
innodb_compression_algorithm=zlib (defaults to zlib, others available)

https://mariadb.com/kb/en/innodb-system-variables/#innodb_compression_algorithm

[obel1x]

there are two different points:

1. how to handle existing tables which have compressed row_format for existing installations
2. how to setup new tables in new installations

imo for 1. they should be converted as soon as possible. i personally would like to get hints if my db is not configured for best nextcloud- performance e.g. by stating, that the "global innodb_compression_default" has not been set up best. this should be a hint, not a must. maybe before converting all tables so that the converted tables are best running after conversion. If the db-admin chooses not to do so, its up to him.

for the 2nd i than would not do any coding concerning row_format if not really needed as there may be reasons for doing it in a different way. @acsfer the line had been removed and after that replaced by dnamic setting? why have you done so? is there a need for this?
Thank you for getting those things together here :)

[solracsf]

@obel1x

1. See Add MariaDB 10.6 support (drop row_format=compressed) #30129 (comment)
2. Tables would be DYNAMIC because it is the default of MySQL/MariaDB. Simple ;)

Points at #30129 (comment) are purely documentation related (as any other "tuning related" settings).

[obel1x]

2. Tables would be DYNAMIC because it is the default of MySQL/MariaDB. Simple ;)

If thats the reason, please remove it. It is only a source for errors in the future and redundant with db-settings.

Points at #30129 (comment) are purely documentation related (as any other "tuning related" settings).

Thats fine, there are morer important issues
Ty!

[MichaIng]

Did anyone actually test how well a common Nextcloud database compresses on a common filesystem, let's say ext4 (sparse file support!), with the default compression format zlib? I mean it's obviously a trade-off for space vs R/W performance/system load, while space indirectly affects performance as of buffer sizes. My point is that Nextcloud shouldn't make any unverified universal recommendations. When it depends much on the individual case, then it's better do not set or recommend anything but leave it to database engine/system defaults or admin choice and at best leave a link in the docs that compression may be something for admins to look into.

On the other hand, the size of the oc_filescache table is already a major reason for database issues, risking to worsen this is probably not advised 😄.

[Nuadh]

Did anyone actually test how well a common Nextcloud database compresses on a common filesystem, let's say ext4 (sparse file support!), with the default compression format zlib? I mean it's obviously a trade-off for space vs R/W performance/system load, while space indirectly affects performance as of buffer sizes. My point is that Nextcloud shouldn't make any unverified universal recommendations. When it depends much on the individual case, then it's better do not set or recommend anything but leave it to database engine/system defaults or admin choice and at best leave a link in the docs that compression may be something for admins to look into.

On the other hand, the size of the oc_filescache table is already a major reason for database issues, risking to worsen this is probably not advised smile.

My nextcloud db is 340MB, but takes only 91MB on a lz4 compressed ZFS filesystem. It's a small instance, but it shows the effect of compressing. I would guess you could get fairly similar results with row_format=compressed. It is not worth enforcing, but is a tuning-tip worth mentioning

[MichaIng]

Jep, should be similar with MariaDB internal compression then, even better with gzip, though slower as well. The other question is how this affects average query speeds.

Through with factor 4 for InnoDB cache size this is a pretty major factor. Usually the database server consumes most RAM of the whole stack (at least in all cases I've seen with different webservers and PHP-FPM), and the InnoDB cache size again is most responsible for MariaDB RAM usage. Not much point in ~doubling RAM for a server only to disable database compression for a theoretical query speedup 😄.

However, not enforcing it then has exactly this benefit that admins can choose freely whether and which compression format (or is that already possible?) and on which level they want to do it, like in your case native filesystem compression instead. Btw, how did you actually achieve this when, as I understand, ROW_FORMAT=COMPRESSED is forcefully used by Nextcloud currently? Did you patch it out manually?

[Nuadh]

Jep, should be similar with MariaDB internal compression then, even better with gzip, though slower as well. The other question is how this affects average query speeds.

Through with factor 4 for InnoDB cache size this is a pretty major factor. Usually the database server consumes most RAM of the whole stack (at least in all cases I've seen with different webservers and PHP-FPM), and the InnoDB cache size again is most responsible for MariaDB RAM usage. Not much point in ~doubling RAM for a server only to disable database compression for a theoretical query speedup smile.

However, not enforcing it then has exactly this benefit that admins can choose freely whether and which compression format (or is that already possible?) and on which level they want to do it, like in your case native filesystem compression instead. Btw, how did you actually achieve this when, as I understand, ROW_FORMAT=COMPRESSED is forcefully used by Nextcloud currently? Did you patch it out manually?

That was my bad.. I "manually" changed it ages ago, but didn't check.. The updater seems to force it to compressed. Double compression is twice as good, no?

I now changed it back to DYNAMIC, and the on disk usage is 77MB, actual size is 290MB.

[MichaIng]

That was my bad.. I "manually" changed it ages ago, but didn't check.. The updater seems to force it to compressed. Double compression is twice as good, no?

How dumb questions can solve problem unexpectedly 😄. Moreless same compression ratio is quite surprising in case of lz4 vs gzip, did you actively convert all tables to DYNAMIC? Ah wait, since innodb_compression_algorithm=zlib is the default since MariaDB 10.2.4, your tables are probably still doubled compressed without explicitly setting innodb_compression_algorithm=none? However, it's details while I think the argument/benefit of not forcing compression within Nextcloud is clear.

[Nuadh]

That was my bad.. I "manually" changed it ages ago, but didn't check.. The updater seems to force it to compressed. Double compression is twice as good, no?

How dumb questions can solve problem unexpectedly smile. Moreless same compression ratio is quite surprising in case of lz4 vs gzip, did you actively convert all tables to DYNAMIC? Ah wait, since innodb_compression_algorithm=zlib is the default since MariaDB 10.2.4, your tables are probably still doubled compressed without explicitly setting innodb_compression_algorithm=none? However, it's details while I think the argument/benefit of not forcing compression within Nextcloud is clear.

as Innodb_num_pages_page_compressed is 0, and innodb_compression_default is OFF, I think I'm fine, until the NC updater makes it compressed again:) InnoDB has moved to page compression instead of row compression in MariaDB 10.6. I added innodb_compression_default = OFF in my server config, to avoid sudden changes in that default^^

[solracsf]

Guys, is it possible to discuss the PR here only, and move further discussions to the forum? Thanks for understanding. 😉

[obel1x]

Sorry, wanted to change the line, but dont' know how to do this here in git.
Please remove the line completely. We already see this is a source for future errors and it is redundant with db- default already set in mariadb

[MichaIng]

@obel1x
You mean 'row_format' => 'compressed',? It is already removed now, so no compression forced in createConnectionParams() and no forced compression done when converting tables to utf8mb4. LGTM: It doesn't change something for existing instances (no conversion back to DYNAMIC) and also for new instances nothing changes thanks to the defaults, as long as MariaDB defaults have not been explicitly changed by admin (or package maintainer in theory).

[MichaIng]

MariaDB v10.6 is hitting more and more repositories, v10.7 in RC state. It would be great to have this merged with NC24, so I'm giving this hereby a push for another review 🙂.

[skjnldsv]

Please update other pipelines if possible and update the documentation :)

[Nascire]

Out of curiousity - this change only affects new installations it seems? What about existing installations with COMPRESS format?

[nickvergessen]

Before updating to 10.6 you need put Nextcloud in maintenance mode, revert the compress and then update

[Nascire]

I managed to change them myself - but I think most people out there would appreciate an official way ...

[nickvergessen]

Right, we could make lib/private/Repair/Collation.php change it to dynamic or whatever other value is better.

[MichaIng]

Right, we could make lib/private/Repair/Collation.php change it to dynamic or whatever other value is better.

I thought the same, though not sure whether this could take too long, so that it is better to show a notice in the admin panel instead, like for adding missing table indices and such? Would then require a related CLI command.

EDIT: Ah wait, this is done at line 81 already: https://github.com/nextcloud/server/blob/e49233a/lib/private/Repair/Collation.php#L81
Pretty inefficient how it was, first everything converted to DYNAMIC, then in case of utf8mb4 (default on modern MySQL/MariaDB) converted again to COMPRESSED 🤔.

EDIT2: Since this function only loops through non-UTF8 tables, a dedicated repair step/CLI command could do the COMPRESSED => DYNAMIC conversion for MariaDB 10.6 support.

[MichaIng]

Since MariaDB 10.6 was just merged to Debian Bookworm/testing, I converted my tables manually and can verify that all is working well after upgrading MariaDB 🙂:

foriin$(mysql -e 'show tables from nextcloud;'| tail -n +2)do
mysql -e "alter table nextcloud.$i ROW_FORMAT=DYNAMIC;"done

[obel1x]

What is needed to get the Changes backported into NC 23 as 24 is not determined so that 23 will be the stable one for some time?

[bdrewery]

MariaDB no longer plans to drop support for row_format=compressed. So there are no upgrade concerns for Nextcloud users in terms of row_format.

From the maintainer of the feature:

Based on community feedback to MDEV-23497, and because MDEV-27058 already addressed the main technical motivation of removing the write support for ROW_FORMAT=COMPRESSED, I no longer think that we need to remove the write support in the foreseeable future.

• https://jira.mariadb.org/browse/MDEV-22367?focusedCommentId=213009&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-213009

MDEV-27736 in MariaDB Server 10.6.6 essentially reverted this change, by changing the default value of innodb_read_only_compressed to OFF. The plan to remove support for InnoDB ROW_FORMAT=COMPRESSED tables (MDEV-22367) has been cancelled.

• https://jira.mariadb.org/browse/MDEV-23497?focusedCommentId=214134&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-214134