A container whose workload exceeds its memory limit meets the out of memory killer, with nothing between the two. A guest can be given a swap area to reclaim to instead, and nothing in Containerization asks for one today.
I have this implemented and would like to check the design against what the project wants before opening the pull request.
What it does
LinuxContainer takes a swap size beside the memory limit it plays against. At bootstrap the area is a raw file on the host, attached to the guest as a block device; vminitd writes a swap header to it and enables it while mounting the container's filesystems. A container that asks for no swap gets none, which is what a container expected to stay within its memory wants.
A block device rather than a file inside the guest, because that is how a guest gets swap it can write to when its own root is read only.
The area holds nothing that outlives the container, so it is made afresh at each bootstrap.
Two details worth stating, because both are easy to get silently wrong
The area is enabled with discard, and the backing device is marked non rotational first. virtio block devices are rotational by default, and the kernel only tracks a swap area in clusters when its device is non rotational, so without that the discard flags are accepted and no discard is ever issued: the sparse host file then holds the area's high water mark for as long as the container runs, and nothing says why.
The host file is sparse. The guest reaches the area as a block device, which the kernel takes as a single extent without consulting the host's layout, so the hole-free requirement that binds a swap file inside a guest does not bind the host file backing a swap device. Sparse backing therefore costs the host only the pages the guest has actually swapped out, and gives them back on discard. Reference for the in-guest requirement this is distinguished from: https://github.com/torvalds/linux/blob/master/mm/swapfile.c
Questions for the maintainers
- Should the size count swap alone, or the memory-plus-swap total that the OCI runtime spec's
memory.swap carries? I count swap alone, so that a value reads against the memory limit rather than including it, and convert at the spec boundary. The spec's own spelling is the total. - Is the block-device form the one you want, or would you rather the area lived inside the guest's writable layer where one exists?
- Default zero, as here, or a proportion of the memory limit?
Adjacent: #653 proposed a custom OOM killer for the same underlying situation, and was closed.
I will follow up with the pull request; happy to reshape it against whatever you prefer here.
A container whose workload exceeds its memory limit meets the out of memory killer, with nothing between the two. A guest can be given a swap area to reclaim to instead, and nothing in Containerization asks for one today.
I have this implemented and would like to check the design against what the project wants before opening the pull request.
What it does
LinuxContainertakes a swap size beside the memory limit it plays against. At bootstrap the area is a raw file on the host, attached to the guest as a block device;vminitdwrites a swap header to it and enables it while mounting the container's filesystems. A container that asks for no swap gets none, which is what a container expected to stay within its memory wants.A block device rather than a file inside the guest, because that is how a guest gets swap it can write to when its own root is read only.
The area holds nothing that outlives the container, so it is made afresh at each bootstrap.
Two details worth stating, because both are easy to get silently wrong
The area is enabled with discard, and the backing device is marked non rotational first. virtio block devices are rotational by default, and the kernel only tracks a swap area in clusters when its device is non rotational, so without that the discard flags are accepted and no discard is ever issued: the sparse host file then holds the area's high water mark for as long as the container runs, and nothing says why.
The host file is sparse. The guest reaches the area as a block device, which the kernel takes as a single extent without consulting the host's layout, so the hole-free requirement that binds a swap file inside a guest does not bind the host file backing a swap device. Sparse backing therefore costs the host only the pages the guest has actually swapped out, and gives them back on discard. Reference for the in-guest requirement this is distinguished from: https://github.com/torvalds/linux/blob/master/mm/swapfile.c
Questions for the maintainers
memory.swapcarries? I count swap alone, so that a value reads against the memory limit rather than including it, and convert at the spec boundary. The spec's own spelling is the total.Adjacent: #653 proposed a custom OOM killer for the same underlying situation, and was closed.
I will follow up with the pull request; happy to reshape it against whatever you prefer here.