vaapi_encode_h264: Reduce SAR to valid range
authorMark Thompson <sw@jkqxz.net>
Sun, 28 Oct 2018 17:13:53 +0000 (17:13 +0000)
committerMark Thompson <sw@jkqxz.net>
Sun, 18 Nov 2018 21:34:04 +0000 (21:34 +0000)
commita830056b32feb2db463e062d4ec2b6fb47ec70ee
treed1e1236cfeb4bced022a7ee82aa137ba7d78e170
parentead0d2eb7686c41f2aac3052412755d827d525f2
vaapi_encode_h264: Reduce SAR to valid range

The SAR of the input could have a numerator or denominator greater than
2^16 which would then be truncated to a 16-bit integer when written to
the VUI parameters, giving a random result.  Instead, reduce the SAR to
the nearest representable fraction.

Fixes #7502.
libavcodec/vaapi_encode_h264.c