Allow Longer keys upto 600 chars

[G8XSU]

Motivation: Earlier we were conservative about our key-size since it is easier to increase but difficult to increase in backward compatible fashion. Now with introduction of sub_namespace and directly allowing upto 360char keys in LDK, we need support for longer key lengths.

Earlier it was 120 based on uuid length.

[jkczyz]

Motivation: Earlier we were conservative about our key-size since it is easier to increase but difficult to increase in backward compatible fashion. Now with introduction of sub_namespace and directly allowing upto 360char keys in LDK, we need support for longer key lengths.

Earlier it was 120 based on uuid length.

What's the math for 360 characters? I'm getting a significantly less number (112), e.g., monitor_updates/deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef_4294967295/18446744073709551615

[tnull]

Motivation: Earlier we were conservative about our key-size since it is easier to increase but difficult to increase in backward compatible fashion. Now with introduction of sub_namespace and directly allowing upto 360char keys in LDK, we need support for longer key lengths.
Earlier it was 120 based on uuid length.

What's the math for 360 characters? I'm getting a significantly less number (112), e.g., monitor_updates/deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef_4294967295/18446744073709551615

We guarantee a max length of 120 chars for each of namespace, sub-namespace, and key (cf. https://github.com/lightningdevkit/rust-lightning/blob/ce7463486ee1ae61e9af439c3d34d00244248ee9/lightning/src/util/persist.rs#L35).

[G8XSU]

Yes each of namespace subnamespace key can be up to 120 char each.
so it would be around 120*3 + delimiters, but we allow for longer than that since vss is not only meant for ldk.

But I don't think anyone else will use keys longer than ldk. So i will be fine with tighter bound as well.

Current number is from 120*5 = 600 (considering some buffer and if some other application wants to use longer keys.)

We could do around 400, as well, shouldn't matter much.

[jkczyz]

Ah, thanks for explaining. Couple questions:

• Can a db's schema be changed after the fact to increase the character limit?
• Any reason why we shouldn't use text? (https://www.postgresql.org/docs/current/datatype-character.html)

[jkczyz]

Answering one of my own questions, ALTER COLUMN can update the type and would cast the data.

https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-DESC-SET-DATA-TYPE

I don't have a strong preference on text vs character varying, but we should document the choice of the value in the commit message.

[G8XSU]

text vs character varying

I think it is basic sql that text is for unlimited size strings, and varchar is for limiting size of column.
It would be wrong usage of sql itself if a primary key column is text.

If we were using text, we owe an explanation. by default everything should be varchar if possible.

[jkczyz]

FWIW, I'm seeing opposite advice but it doesn't touch on primary keys, FWIW.

https://wiki.postgresql.org/wiki/Don't_Do_This#Text_storage

Could you update the commit message to say why 600 was chosen?

[G8XSU]

Added explanation for both in description.

Mainly:

• We use varchar over text as it is sql standard
  and can be indexed in all sql-implementations while limiting
  key size.

For example MySQL doesn't allow indexes on text, postgres i will have to check. Hence we shouldn't use them as primary keys.
Text is also a non-standard across different sql implementations.
And depending on size of value in text field, it can get stored in separate file other than main database file. (Similar to blob).

Varchar and a text with constraint have similar characterstics apart from storage, indexing and sql-standardness.

[jkczyz]

Not sure if it makes sense to use a power of two for any efficiency gains, but otherwise LGTM.

[G8XSU]

Acc. to stackoverflow it doesnt matter. In reality, even much longer string lengths have comparable performance.

https://stackoverflow.com/questions/593364/varchar-fields-is-a-power-of-two-more-efficient
https://stackoverflow.com/questions/13233495/using-power-of-two-numbers-for-length-of-database-columns