avformat/rmdec: Check old_format len for overflow
authorMichael Niedermayer <michael@niedermayer.cc>
Mon, 26 Apr 2021 20:35:37 +0000 (22:35 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Fri, 18 Jun 2021 16:58:25 +0000 (18:58 +0200)
commit06d174e289eb185f03a34a738965f0042f39c038
tree0da75ec4dd102670d4091ab8eda3014d9dcce86f
parentfe12aa689003db9b07a6e1b837031dcc57a71435
avformat/rmdec: Check old_format len for overflow

Maybe such large values could be disallowed earlier and closer to where
they are set.

Fixes: signed integer overflow: 538976288 * 8224 cannot be represented in type 'int'
Fixes: 29102/clusterfuzz-testcase-minimized-ffmpeg_dem_RM_fuzzer-6704350354341888

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavformat/rmdec.c