[vlc-devel] [PATCH] [RFC] direct3d9: disable YUV pixel formats using full range on nVidia

[Steve Lhomme]

We need to do a range conversion or use a full range format knowingly.

We should check that the source is actually in limited range (otherwise it's fine)
We need to check what range other manufacturers assume for each pixel format
We need to check on more NVIDIA GPU/Drivers to see if it's common to all of them.
---
 modules/video_output/win32/direct3d9.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/modules/video_output/win32/direct3d9.c b/modules/video_output/win32/direct3d9.c
index d3def81868..a49e40f4f9 100644
--- a/modules/video_output/win32/direct3d9.c
+++ b/modules/video_output/win32/direct3d9.c
@@ -990,6 +990,10 @@ static const d3d9_format_t *Direct3DFindFormat(vout_display_t *vd, vlc_fourcc_t
     if( !hardware_scale_ok )
         msg_Warn( vd, "Disabling hardware chroma conversion due to odd dimensions" );
 
+    D3DADAPTER_IDENTIFIER9 identifier;
+    HRESULT hr = IDirect3D9_GetAdapterIdentifier(sys->hd3d.obj, sys->d3d_dev.adapterId, 0, &identifier);
+    bool isNvidia = identifier.VendorId == GPU_MANUFACTURER_NVIDIA;
+
     for (unsigned pass = 0; pass < 2; pass++) {
         const vlc_fourcc_t *list;
         const vlc_fourcc_t dxva_chroma[] = {chroma, 0};
@@ -1010,6 +1014,25 @@ static const d3d9_format_t *Direct3DFindFormat(vout_display_t *vd, vlc_fourcc_t
                 if (format->fourcc != list[i])
                     continue;
 
+                if (isNvidia)
+                {
+                    bool canDisplayLimitedRange = true;
+                    switch (format->fourcc)
+                    {
+                    case VLC_CODEC_YV12:
+                    case VLC_CODEC_I420:
+                    case VLC_CODEC_NV12:
+                    case VLC_CODEC_D3D9_OPAQUE:
+                    case VLC_CODEC_D3D9_OPAQUE_10B:
+                    case VLC_CODEC_UYVY:
+                    case VLC_CODEC_YUYV:
+                        canDisplayLimitedRange = false;
+                        break;
+                    }
+                    if (!canDisplayLimitedRange)
+                        continue;
+                }
+
                 msg_Warn(vd, "trying surface pixel format: %s",
                          format->name);
                 if (!Direct3D9CheckConversion(vd, format->format, target)) {
-- 
2.17.0