Checksum support for CLI/WebDAV?

[teras]

Hello!

After an invitation from your tech support, I am writing this missing feature request. Since the idea is to use it as a backup with rclone, there's no guarantee that the uploaded file has indeed the correct content, in case it was uploaded with an error (even if file size matches). We are talking about backup - a sensitive topic and keeping track of the uploaded file's checksum is crucial. By uploading a file, expecting it to be correct, but when you need it to find out that it is corrupted, is worse than not having a backup at all.

I know that other WebDAV implementations, like the one from NextCloud, do support this.Are there any plans on supporting this (or any other verification method) ?

Even if WebDAV is not supporting this, being able to support that on a CLI tool, it would be awesome. I would even create a simple script to send (reliable) file(s) to the cloud, even without the luxury of a proper UI tool. Otherwise the real value of internxt as a reliable backup place will be practically non existent.

Thank you and wishing for a positive answer.

[larryrider]

Hey @teras!! Thanks a lot for the detailed feedback!

You're absolutely right that some WebDAV implementations like Nextcloud support checksum validation. However, it's worth noting that those implementations doesn't perform client-side encryption when files are uploaded via WebDAV. This means the server receives and stores the file in plain (unencrypted) form, which allows it to generate and validate checksums directly on the raw file content.

In contrast, Internxt is built around zero-knowledge, end-to-end encryption. Files are encrypted on your device before being uploaded, and the encrypted data is what gets stored in the cloud. Because of this, any checksum verification would need to be performed on the encrypted version of the file, not the original.

Now, even if we implemented a checksum mechanism on the encrypted file (for example, in the CLI), this wouldn't fully eliminate the risks. If something went wrong during the encryption process itself (e.g. a hardware or memory issue), the resulting encrypted file might be corrupted, and any checksum generated at that point would still match the corrupted file. So the checksum would confirm the integrity of the encrypted blob, but not the original file's integrity.

To put it simply: once the file is encrypted, any checksum reflects the encrypted content, not the original data. So if encryption fails silently, both the encrypted file and its checksum will be wrong, giving a false sense of security.

That said, you're right that verifying the uploaded file's integrity is still very useful. We're actively discussing ways to make this possible in a reliable way that integrates with our encryption flow, while still keeping the zero-knowledge and the privacy guarantees that Internxt offers.

[teras]

Thank you again for the thorough answer.

I agree and totally applause with your zero-knowledge policy. But checksumming even the encrypted data would make a huge impact, IMHO. Having a RAM or any hardware issue is still possible (I was personally having such issues recently due to buggy BIOS upgrades), but these are far more uncommon than sending a file online, where too many things can go wrong and checksums is usually the only weapon we have. Online transfer is a untrustful media and, if possible, please at least do something for the weakest part of the chain.

To cover the whole story, for rclone/restic itself, encryption is not required at all, since it is doing its own stuff but that's irrelevant. Keeping the files encrypted is the way to go.

[dwillitzer]

+1 — and it is more than a nice-to-have. We independently hit exactly the failure this asks about while moving large datasets: a file big enough to require an S3 multipart upload can come back with the correct size but wrong content from byte 0, silently, with the upload reporting success.

We traced it: on a corrupt run the client uploads every part correctly (each part's ETag matches the MD5 of the bytes sent) and sends a correctly-ordered (PartNumber, ETag) list to /files/finish — yet the assembled object is wrong. The corruption is introduced server-side during multipart completion. It correlates with upload speed (a fast link that pushes both parts within ~1 minute corrupts far more often than a heavily throttled one), which points at a completion/consistency race.

The bridge's only post-completion check (validateObjectInStorage) compares size only — no content hash — so a same-size/wrong-content object passes by construction. That is exactly why a checksum/content-verification step is needed, ideally server-side.

For now we work around it client-side: after every multipart upload we re-download the object, hash the decrypted content against the source, and retry on mismatch (never reporting success on unverified data). It works but doubles transfer time on large files. Server-side content verification at completion would fix this properly. Happy to share reproduction details if useful.