avcodec/apedec: Fix multiple integer overflows and undefined behaviorin filter_3800()
authorMichael Niedermayer <michael@niedermayer.cc>
Sun, 16 Jun 2019 09:39:15 +0000 (11:39 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Wed, 31 Jul 2019 18:35:07 +0000 (20:35 +0200)
commit267eb2ab7f87696e1a156ca9a5ff1b1628d170c1
tree50c29453969cb7c419299b49740323a3f2bc17a0
parentda5039415c2bd625085d15e6c92e0b64eefddcbf
avcodec/apedec: Fix multiple integer overflows and undefined behaviorin filter_3800()

Fixes: left shift of negative value -4
Fixes: signed integer overflow: -15091694 * 167 cannot be represented in type 'int'
Fixes: signed integer overflow: 1898547155 + 453967445 cannot be represented in type 'int'
Fixes: 15258/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5759095564402688
Fixes: signed integer overflow: 962196438 * 31 cannot be represented in type 'int'
Fixes: 15364/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5718799845687296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/apedec.c