libavutil/hwcontext_dxva2: Add check for possible errors from GetAdapterDisplayModeEx
authorHumberto Ribeiro <mont3z.claro5@gmail.com>
Wed, 3 Jan 2018 23:22:15 +0000 (15:22 -0800)
committerwm4 <nfxjfg@googlemail.com>
Thu, 4 Jan 2018 14:41:40 +0000 (15:41 +0100)
This prevents a possible crash in CreateDeviceEx when using faulty
response from GetAdapterDisplayModeEx and allows ffmpeg to fallback to
classic d3d9.

Signed-off-by: wm4 <nfxjfg@googlemail.com>
libavutil/hwcontext_dxva2.c

index 2ddd4be..44ebdbc 100644 (file)
@@ -485,7 +485,11 @@ static int dxva2_device_create9ex(AVHWDeviceContext *ctx, UINT adapter)
     if (FAILED(hr))
         return AVERROR_UNKNOWN;
 
-    IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL);
+    hr = IDirect3D9Ex_GetAdapterDisplayModeEx(d3d9ex, adapter, &modeex, NULL);
+    if (FAILED(hr)) {
+        IDirect3D9Ex_Release(d3d9ex);
+        return AVERROR_UNKNOWN;
+    }
 
     d3dpp.BackBufferFormat = modeex.Format;