signature: add MultipartSigner and MultipartVerifier

[daxpedda]

This PR adds new traits for multipart messages: MultipartSigner, RandomizedMultipartSigner, RandomizedMultipartSignerMut and MultipartVerifier.

The idea here is to allow non-contiguous bytes to be passed, which is necessary when the message has to be constructed from multiple sources without wanting to allocate memory for a contiguous message. E.g. for no_std environments or when the message is rather big but pre-hashing is not applicable, e.g. PureEdDSA, ML-DSA or SLH-DSA.

I know this is a rather big breaking change, so let me know what you think!

These new traits can be implemented by a bunch of crates:

• ecdsa: Implement MultipartSigner/Verifier signatures#982
• ml-dsa: Implement MultipartSigner/Verifier signatures#982
• slh-dsa: Implement MultipartSigner/Verifier signatures#982
• bign256: Implement MultipartSigner/Verifier elliptic-curves#1221
• sm2: Implement MultipartSigner/Verifier elliptic-curves#1221
• k256: Implement MultipartSigner/Verifier elliptic-curves#1221
• dsa: Implement MultipartSigner/Verifier signatures#982
• lms: Implement MultipartSigner/Verifier signatures#982
• rsa: Implement MultipartSign/Verify RSA#525
• ed25519-dalek: ed25519: implement MultipartSigner/Verifier dalek-cryptography/curve25519-dalek#764

ResolvesRustCrypto/signatures#959.

[newpavlov]

If we are to support such messages, we probably should do it in separate methods or maybe even in a separate trait.

[daxpedda]

If we are to support such messages, we probably should do it in separate methods or maybe even in a separate trait.

I will go ahead and move it into a separate method then. I don't think a separate trait is necessary as the &[u8] method will just use &[&[u8]] method. But let me know if you have more thoughts on this.

[daxpedda]

I went ahead and added a sign/verify_segments_* trait method to every relevant trait.
Now that its separate, I also went ahead and used Iterator<Item: AsRef<[u8]>>, because the more convenient method is still present.

Will update the other PRs today and see how it looks before moving this out of draft again.

[tarcieri]

This adds a lot of code surface but really doesn’t add any new functionality.

We already support IUF signing via DigestSigner and PrehashSigner. So this would be yet another API for such cases.

Where I guess this could maybe help are for constructions that don’t fit into a strictly IUF API like streaming Ed25519 (non-Ed25519ph) or ML-DSA with external mu. But it’s unclear to me we can build traits which actually abstract over such APIs. If we could, I guess this would be close, but that should probably be the design rationale.

[tarcieri]

Also note that the original proposed &[&[u8]] is probably the safest approach here. Ed25519 needs to do multiple passes over the input message to compute a signature, and the input message can't change. I'm not sure how you'd express that with Iterator

[tarcieri]

FWIW, here's discussion on this sort of API for Ed25519. It was actually removed from this PR, but prior to that, it was suggested to use &[&[u8]] for the API:

https://github.com/dalek-cryptography/curve25519-dalek/pull/735/files/bfdaa86b1820596d893f69390a6204f19afde306#diff-bc1db46f3164930c55100377fea321e44637c4ca4575d3960c179f4ffbaa6703

[daxpedda]

Where I guess this could maybe help are for constructions that don’t fit into a strictly IUF API like streaming Ed25519 (non-Ed25519ph) or ML-DSA with external mu. But it’s unclear to me we can build traits which actually abstract over such APIs. If we could, I guess this would be close, but that should probably be the design rationale.

The intention is not a streaming interface, but a streaming interface would address the problem here indeed as well.

Also note that the original proposed &[&[u8]] is probably the safest approach here. Ed25519 needs to do multiple passes over the input message to compute a signature, and the input message can't change. I'm not sure how you'd express that with Iterator

Ah right ... I forgot why I had planned it with &[&[u8]] to begin with.

---

So if the approach with &[&[u8]] is acceptable, I would prefer to move forward with that. If not, we can continue with discussing a potential streaming API.

Just to clarify: the intention here is to support non-contiguous byte slices for Ed25519/448, ML-DSA and SLH-DSA.

[tarcieri]

I agree with the earlier comments that this should be a new trait, like MultipartSigner. Implementing it requires support from the underlying signature algorithm, which doesn't currently exist in ed25519-dalek or in ml-dsa, for example.

I think it would probably also be good to get everything prototyped in PRs to those crates first before we merge, so we know it's actually a good abstraction.

[daxpedda]

I agree with the earlier comments that this should be a new trait, like MultipartSigner. Implementing it requires support from the underlying signature algorithm (which doesn't currently exist in ed25519-dalek, for example, or in ml-dsa)

I added support for ML-DSA and SLH-DSA in RustCrypto/signatures#981. I'm happy to make a PR to the curve25519-dalek repo as well.

Just my 2¢: if you take a look at RustCrypto/signatures#981, the changes are pretty minimal. I really don't believe that going from &[u8] to &[&[u8]] is worth a separate trait. However, in the meantime I will move forward with MultipartSigner, let me know if you change your mind though, no problem at all.

[tarcieri]

@daxpedda if you look at dalek-cryptography/curve25519-dalek#735, the same cannot be said of Ed25519.

I don't want to force signature implementations to have to implement this interface as their only possible choice. We can't even guarantee this interface is possible for all signature algorithms (at least in an unbuffered, no_std-friendly manner)

[tarcieri]

Alright, that was easier than expected, so let me say this:

• Most people don't need or want this functionality
• It's incredibly uncommon to define APIs this way, IMO
• Changing this will break all current users of these traits

I still think it should be a separate trait to keep it out of the way.

[daxpedda]

Sounds good, on it!

[daxpedda]

Let me know what you think.

We may want to implement traits on demand, for OPAQUE I will only need MultiPartSigner, RandomizedMultiPartSigner and MultiPartVerifier.

[daxpedda]

Should I add a multipart_* pre-fix to the methods to prevent conflict?

[daxpedda]

I went ahead and did both: removed MultiPartSigner & co. and added prefixed method names with multi_part.

[daxpedda]

Implementation for ML-DSA and SLH-DSA done in RustCrypto/signatures#982.

I hope that dalek-cryptography/curve25519-dalek#763 was sufficient, so I will wait until we are ready to upgrade RustCrypto dependencies there before I make a PR. Probably dalek-cryptography/curve25519-dalek#735 will be relevant as well, as that could be used to implement the MultiPart traits.

Let me know if there is something else you want me to implement/explore/discuss.

[tarcieri]

@daxpedda regarding rsa, it's another case like ecdsa where it already impls DigestSigner/PrehashSigner and needs whatever glue (macro or blanket impl) we choose to use there

[daxpedda]

FYI: the blanket implementation didn't work because I can't bind D to the MultipartSigner trait. And I feel like the implementation is so short it doesn't deserve a macro, but let me know if you think otherwise (and where the macro should live in that case).

Will go ahead and make the PR in rsa.

[tarcieri]

Yeah, just go ahead and hand write impls for now

[daxpedda]

Done: RustCrypto/signatures#982.

Let me know if you want me to implement it for any other specific crate or for the complete rest.

[daxpedda]

I went ahead and implemented it for the remaining crates.

[tarcieri]

Can tentatively approve this, but it would be good to see it integrated into ed25519-dalek as well before a final release. It doesn't look like it will be a problem.

[daxpedda]

Done in dalek-cryptography/curve25519-dalek#764.

[vdods]

I appreciate the work done here, as well as generally for the signature and related crates.

I'm curious why an implementation that accepted std::io::Read or some other kind of iterator wasn't chosen? Is it to avoid dependence on std? Being able to avoid an unbounded allocation to produce an ed25519 signature seems worthwhile. There are situations where one can't choose to use ed25519ph -- for example, producing a JWS with alg:EdDSA and crv:Ed25519 over a detached payload which could be arbitrarily large and therefore potentially impossible (or if not, definitely not recommended) to load into memory. Thoughts?

[daxpedda]

This PR specifically implements a non-streaming API, but doesn't preclude one either.

I would also definitely be interested in a streaming interface, but it would probably need some design work on how exactly we are going to take that input. As you mentioned, std::io::Read is probably undesirable because it lacks no_std compatibility. Because we need the same input twice, Iterator won't work, we would need IntoIterator or something else. We also have no way to tell the user via the type system that we need the same data twice.

See ed25519_dalek::hazmat::raw_sign_byupdate(), which might already address your use-case.

[tarcieri]

@vdods yes, what @daxpedda said.

Many signature algorithms implement either DigestSigner or PrehashSigner, where the Digest supports an Initialize-Update-Finalize API which can be computed in a "streaming" manner. So for people who need to sign large messages, that's one way. And if you're dead set on using the std::io API, there's https://github.com/RustCrypto/utils/tree/master/digest-io but literally all of our crates except for that one are designed to support no_std.

That said, not all signature algorithms support such an API, notably neither EdDSA or ML-DSA work with such an API. EdDSA needs two passes over the input message as @daxpedda mentioned, and ML-DSA uses an "external mu" approach where the algorithm controls the digest prefix, both aimed at hardening against collision attacks against the underlying hash function. I think a slight tweak to the design of DigestSigner would allow us to support external mu: #2003

[vdods]

Ahh, I think I missed that in EdDSA, the first hash gets fed into the second hash before the message does, and therefore intrinsically doesn't support streaming.

If I understand it correctly, the message-signing and digest-signing versions of EdDSA are intrinsically different and the digest version can't be used to produce the equivalent of signing an arbitrary message. But if one were able to choose which algorithm to use, then the digest-signing version would indeed be a good choice to keep the digest process in a fixed buffer of memory and avoid allocations.

Here's a dumb and simple (but not elegant) idea about how to support streaming without the need for a "clonable" iterator: Have the caller simply provide two streams/iterators (however that's implemented) that they guarantee to be identical. One stream gets fed into each of the two hashes used in EdDSA. There could even be a debug mode check that uses a "tee" writer to feed each stream en route into a hasher, and then the hash values checked for equality (verifying with negligible error that the streams were indeed identical).

Anyway, I get the difficulty now, and am less sure that it would be a good idea.

[tarcieri]

Now that I think about it, the DigestSigner shape proposed in #2003 could work with EdDSA too, the same way ed25519_dalek::hazmat::raw_sign_byupdate is implemented: with the expectation that multiple invocations of the provided Fn apply the same updates to the digest.

Though, having some way of hazmat-gating it (possibly via a special newtype located in the hazmat module?) still seems like a good idea.

[vdods]

Nice, that approach seems good.