add interface for modifier query

[XinfengZhang]

1. this interface is for modifier query/negotiation. it could query the modifier list for a given format under a config
2. it support different modifier list for different format. for example: ARGB maybe just support linear, RGB maybe support linear only, NV12 support tile4 etc.

[HeJunyan]

Maybe, I think, we just need to extend the "vaQuerySurfaceAttributes" a little.

For example, we define a new type VASurfaceAttribDRMFormatModifiersForFormat after VASurfaceAttribDRMFormatModifiers.

The VASurfaceAttribDRMFormatModifiers return all possible modifiers for a config, while the VASurfaceAttribDRMFormatModifiersForFormat just return the modifiers for some specific format of that config.

And we define a struct, such as:
struct {
uint32_t format;
VADRMFormatModifierList drm_list;
};
as the VAGenericValue.p of that VASurfaceAttrib query. The format is input and drm_list is the output?

Then we can avoid define a new API.

[ceyusa]

Maybe, I think, we just need to extend the "vaQuerySurfaceAttributes" a little.

But that's already in place #505

Sadly, media-driver, neither mesa still no have an implementation for vaQuerySurfaceAttributes. Mesa only implemented the vaCreateSurface part.

[alatteri]

any chance to get this mergered? The capability would help us a lot with color accuracy.

[nowrep]

Sadly, media-driver, neither mesa still no have an implementation for vaQuerySurfaceAttributes. Mesa only implemented the vaCreateSurface part.

That's because it can't be implemented, the modifier query only makes sense when format is specified which is not the case with vaQuerySurfaceAttributes.

So we would need this PR (vaGetSurfaceFormatAttributes) to implement it in Mesa.

[any1]

Does there exist a mesa merge request for this?

[nowrep]

Does there exist a mesa merge request for this?

No, but I can make one if that can help getting this merged.

[any1]

Does there exist a mesa merge request for this?

No, but I can make one if that can help getting this merged.

Well, if you're in the mood for it, I'd say: go for it. ;)

[nowrep]

Mesa impl: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32053

Test app to print all modifiers: https://gist.github.com/nowrep/4c3eab86ad77dcfa605f516b567a66c1

[anonymix007]

@XinfengZhang Are there any blockers for this PR? Some use-cases like DMA-Buf sharing are basically impossible to get working reliably without this feature. So it would be great if this could be merged and implemented in all the major drivers.

[XinfengZhang]

so, comparing with #590 , you prefer this interface?

[nowrep]

so, comparing with #590 , you prefer this interface?

Yes, #590 doesn't have the format parameter which is important.

[XinfengZhang]

TBH, this interface is complex than #590 , especially for driver side.

[nowrep]

#590 is basically the same as vaQuerySurfaceAttributes that is already available. The issue is that with only VAConfig we know the profile and possibly RT format. But we need to query modifiers for the surface format (fourcc).

Only this interface gives the driver the actual format.

[XinfengZhang]

I am not sure other vendor, from Intel side, almost all format support same modifiers on one platform, for example, https://github.com/torvalds/linux/blob/master/include/uapi/drm/drm_fourcc.h#L717
LNL: I915_FORMAT_MOD_4_TILED_LNL_CCS, I915_FORMAT_MOD_4_TILED & DRM_FORMAT_MOD_LINEAR

[nowrep]

In Mesa AMD we support different modifiers for single and multi-plane formats. In addition, the subsampled formats (yuyv, ...) are linear only. This is not vaapi driver specific and applies to EGL and Vulkan as well.

For the encode/decode entrypoints depending on the hw generation there are further restrictions.

All other APIs that support modifiers provide per-format query, so it would be good if VAAPI did the same.

[HeJunyan]

We did not pay enough attention to this API before because we assume when all libva surfaces export as DMA, its modifiers can be accepted by EGL module(at least for Intel's platform). I think it's time to implement this, so that we can avoid unneeded CSC.

[DatCaptainHorse]

Just giving my support for this to be merged, or something similar so things can get a move on 😅

On the end-developer side of things, Intel's way of handling modifiers has honestly felt like a mess and just a constant pain, making hacks for them is not fun..

I hope this PR gets merged and further improvements can finally progress around the space.

Off-topic feedback/comment: despite Intel claiming they're open-source and all.. what's with the hundreds of unlooked PRs around every Intel repo? It's good if PRs even by your own developers get merged, but that's not nature of open-source at all. It's like saying your farm is all-inclusive, but then you don't let any horses in.