Hardcoded length of PEM-encoding causes interoperability issues

[Eusebius1920]

When creating ssh-keys with terraform

resource "tls_private_key" "test" {
 algorithm = "ED25519"
}

// using: tls_private_key.test.private_key_openssh leads to a private key like this

/*
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtz
c2gtZWQyNTUxOQAAACBsiA4fEAFmnafsQ3/nfd0U3OfqD05qhIKNaaIecZVBNgAA
AIirzgwtq84MLQAAAAtzc2gtZWQyNTUxOQAAACBsiA4fEAFmnafsQ3/nfd0U3Ofq
D05qhIKNaaIecZVBNgAAAEDwfMr2enmcI1eTBwBgJ6DyKFFiKE/rmVkRNz97QHWF
BWyIDh8QAWadp+xDf+d93RTc5+oPTmqEgo1poh5xlUE2AAAAAAECAwQF
-----END OPENSSH PRIVATE KEY-----
*/

each line of the private key has 64 characters. A similar thing occurs when creating a private key with pythons cryptography library:

from cryptography.hazmat.primitives.asymmetric import ed25519
from cryptography.hazmat.primitives.serialization import Encoding
from cryptography.hazmat.primitives.serialization import PrivateFormat
from cryptography.hazmat.primitives.serialization import NoEncryption

key = ed25519.Ed25519PrivateKey.generate()

key.private_bytes(
 encoding=Encoding.PEM,
 format=PrivateFormat.OpenSSH,
 encryption_algorithm=NoEncryption(),
 )

# leads to
"""
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZWQyNTUx
OQAAACBbsSlccs9iGtCYk98px79gJ2yifXxJqM/hBqky1Uy06QAAAIgmw48RJsOPEQAAAAtzc2gt
ZWQyNTUxOQAAACBbsSlccs9iGtCYk98px79gJ2yifXxJqM/hBqky1Uy06QAAAEA/6vNn5cXUt1Ws
/YuqOmJUvs4NyxeI143TpJdZs9Y6nluxKVxyz2Ia0JiT3ynHv2AnbKJ9fEmoz+EGqTLVTLTpAAAA
AAECAwQF
-----END OPENSSH PRIVATE KEY-----
"""

where every line has 76 characters.

I dug into the RFC for PEM-encoding and found nothing about the line width. Both of those private keys are not readable by this library, because it assumes a fixed line width of 70 characters which is the same as ssh-keygen would produce (as suggested by the comment inside the linked file). Dedcoding one of the private key files directly above then yields to an Error-Result:

Encoding(Pem(Base64(InvalidEncoding)))

So to maintain interoperability one has to reformat all private keys to a fixed line-length of 70. Can we change that?

[tarcieri]

Due to the nature of the PEM decoder used (the pem-rfc7468 crate), which is designed to be able to decode documents in constant-time (so as to prevent timing sidechannels when decoding private key files), it would actually be somewhat difficult to support arbitrary line widths.

The decoder works based on advance knowledge of the line width. We could potentially fall back to 64 if 70 fails, but supporting arbitrary line widths would require fairly significant changes to the way the decoder works which would likely make it very difficult if not impossible to support constant-time operation, which is its main design goal (aside from adhering to RFC7468-style encoding, which technically does allow for variable line widths if the encoder chooses to implement them).

It is somewhat unfortunate the Python encoder chose something as strange as 76. At least 64 follows PEM conventions.

[tarcieri]

There is a fairly simple way to implement a lax mode which could be used as fallback if the line width is other than 70: it could scan the line width of the first line after the encapsulation boundary, and then attempt to interpret the rest of the key based on that line width, which should work with any line width so long as it's consistent.

This would require data-dependent branching on the key material in the first line, but it could be avoided for the rest of the key, and only in the event a non-standard line width is used (as it were, the parser does have some timing variability in the event of an error anyway)

[Eusebius1920]

Thanks for elaborating on the problem!

So the default line width of 70 is not RFC 7468 conform, but 64 would be. Am I getting this right? Do you know why OpenSSH is using 70 chars per line?

Additional ideas to help with the interoperability:

• Maybe it would suffice to expose the line width as a setting / flag to the library-user? So as long as you are dealing with keys from the same generator-source you can work around the problems by changing the default of 70 to something else. As long as the config is constant for the use-case the timing will stay intact.
• Create a normalization method that can be (optionally) applied to incoming keys that formats them to the desired 70 chars per line

*Edit:
The python implementation is using 76 chars per line, because it is relying on the base64-implementation of the python stdlib: https://docs.python.org/3/library/base64.html#base64.encode

But I can imagine that there are more different line widths out in the wild.

[tarcieri]

It's illegal for an RFC 7468-compliant generator to generate the line width of 70, however OpenSSH doesn't claim to conform to RFC 7468. Technically this is a violation of a MUST NOT in pem-rfc7468 but we support it in order to generate keys identical to OpenSSH. I don't know why OpenSSH chose 70 chars: it's a very annoying size to support since it doesn't evenly divide by 4, which is the basic encoding unit of Base64.

I think it's possible to support arbitrary line widths specifically for OpenSSH, and that's probably the best thing to do since OpenSSH supports these alternate line widths.

[ZenenTreadwell]

Chiming in to say that I also ran into this problem with my 64 character width PEM files. I don't imagine it would be a difficult fix but I'm not a rust programmer.