avcodec/wmavoice: Fix rounding and integer anomalies in calc_input_response()
authorMichael Niedermayer <michael@niedermayer.cc>
Sat, 14 Dec 2019 14:27:44 +0000 (15:27 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Sun, 2 Feb 2020 23:11:18 +0000 (00:11 +0100)
commit38d37584448731f90977132b838d50ff1a28811b
tree4fe6953cfbeaca7b9a221719c707b243ba1d47a7
parent6847e22c8c85b80bf1d25ec66f77f7ccbcf43aed
avcodec/wmavoice: Fix rounding and integer anomalies in calc_input_response()

Fixes: out of array access
Fixes: inf is outside the range of representable values of type 'int'
Fixes: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long'
Fixes: 19316/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WMAVOICE_fuzzer-5677369365102592

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/wmavoice.c