Two related things, one a set of protocol defects in the NBD server this library ships, the other a use for it.
The NBD export does not hold to what it tells a client
Reading Sources/.../NBD against the protocol document turned up five places where the server tells a client one thing and does another. https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md
- The command flags are read past. They sit in the two bytes after the request magic, and are skipped, so an export that advertises force unit access replies to a command carrying it before what it wrote is durable. A command with that flag has to wait.
- A read-only export does not turn writes away, which the protocol requires of it. The pod volume API already offers to ask for read-only, so a caller can ask today and not get it.
- A request reaching past the end is not refused as invalid.
- Structured replies are not offered. A client that asks for replies carrying their own framing does not get them, and that framing is what lets an export be asked how it is laid out: a read answered that way names the offset it covers and carries a message with any error rather than a bare number.
NBD_OPT_LIST does not name the export on offer, and flush is not stated to cover what other connections wrote, which is the condition for telling a client it may spread its work across several connections. Every connection here serves the one store behind the export, so the guarantee holds and is simply not stated.
These are worth fixing whether or not anything below lands, so I can send them as their own pull request.
Letting an export keep its blocks somewhere other than a file
The export writes through to a file. Letting the store behind it be chosen makes a memory-backed store possible, which is what I wanted it for: swapping a container to an area held in host memory rather than on disk. With the block-descriptor context above, a store in memory knows which chunks it holds and a file is asked through the same seeks a sparse copy uses, so the holes a client is told about are the ones really there.
Testing note
The memory-backed test asserts the release as well as the high water mark: the filler exits while the guest is still up, freeing its swap slots, and a freed cluster is discarded, so the export drops to little more than the swap header. The discards trail the exit, so the test watches the export while the machine is still up rather than sampling it once. A test that samples after shutdown measures nothing.
Question
Is a memory-backed swap area something you want in the library, or is the useful half here just the protocol fixes plus the pluggable store? I am happy to send the first two sections alone.
Depends on the swap work in #877.
Two related things, one a set of protocol defects in the NBD server this library ships, the other a use for it.
The NBD export does not hold to what it tells a client
Reading
Sources/.../NBDagainst the protocol document turned up five places where the server tells a client one thing and does another. https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.mdNBD_OPT_LISTdoes not name the export on offer, and flush is not stated to cover what other connections wrote, which is the condition for telling a client it may spread its work across several connections. Every connection here serves the one store behind the export, so the guarantee holds and is simply not stated.These are worth fixing whether or not anything below lands, so I can send them as their own pull request.
Letting an export keep its blocks somewhere other than a file
The export writes through to a file. Letting the store behind it be chosen makes a memory-backed store possible, which is what I wanted it for: swapping a container to an area held in host memory rather than on disk. With the block-descriptor context above, a store in memory knows which chunks it holds and a file is asked through the same seeks a sparse copy uses, so the holes a client is told about are the ones really there.
Testing note
The memory-backed test asserts the release as well as the high water mark: the filler exits while the guest is still up, freeing its swap slots, and a freed cluster is discarded, so the export drops to little more than the swap header. The discards trail the exit, so the test watches the export while the machine is still up rather than sampling it once. A test that samples after shutdown measures nothing.
Question
Is a memory-backed swap area something you want in the library, or is the useful half here just the protocol fixes plus the pluggable store? I am happy to send the first two sections alone.
Depends on the swap work in #877.