VAAPI HEVC Main10 hardware depth encoding example

[bmegli]

High level overview

Add link in "Community Projects" section pointing to realsense-depth-to-vaapi-hevc10.

The example is added in the middle, just after realsense-ir-to-vaapi-h264, where it fits best logically.

The project

realsense-depth-to-vaapi-hevc10

The warning - the repository itself warns about using video codec for depth map encoding, and points to HEVC 3D extension reference software encoder.

The project is example of hardware encoding D400 depth stream to HEVC Main10 (10 bit) profile using VAAPI (at least Kaby Lake).

This is a purely hardware based pipeline (no depth processing on the host).

The encoding

The encoding takes advantage of:

• VAAPI P010LE pixel format
• the fact that Realsense Z16 depth can be interpreted as P016LE pixel format Y plane
• the fact that P016LE and P010LE are binary compatible (discarding precision)
• the fact that Realsense Depth Units may be used to control precision/range trade-off

Pixel format	Side												Side
	MSB	15	14	...	7	6	5	4	3	2	1	0	LSB
P016LE		x	x	...	x	x	x	x	x	x	x	x
P010LE		x	x	...	x	x	0	0	0	0	0	0

The P010LE has data only in 10 MSB, zeroes in 6 LSB (ignored values).

Realsense Depth Units control precision/range trade-off. We are losing 6 bits of precision in the process of encoding. Example trade-offs (with current production firmware):

• the best precision/worst range is 6.4 mm/6.5472 m (for depth units 0.0001)
• the worst precission/best range is 64 cm/654.72 m (for depth units 0.01)
• all trade-offs in between are possible with purely hardware based encoding pipeline
• other trade-offs are discussed in Realsense hardware accelerated depth encoding and depth units #5587

For range: 65472 in binary is 10 ones followed by 6 zeroes.
For precision: we discard 6 bits of data (64 times worse precision than set in Depth Units).

Rationale

Depth encoding with video codec will be far from perfect, with possible artifacts. Nonetheless this pipeline allows real-time encoding practically using no CPU resources. The data may be streamed using low bandwidth wireless connection.

[dorodnic]

Thanks @bmegli
Very interesting