avcodec/shorten: Fix signed 32bit overflow in shift in shorten_decode_frame()
authorMichael Niedermayer <michael@niedermayer.cc>
Sun, 12 Aug 2018 21:06:55 +0000 (23:06 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Wed, 15 Aug 2018 22:55:39 +0000 (00:55 +0200)
Fixes: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 9480/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-6647324284551168 -rss_limit_mb=2000

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/shorten.c

index 054494f..1ffb7d8 100644 (file)
@@ -715,7 +715,7 @@ static int shorten_decode_frame(AVCodecContext *avctx, void *data,
                 if (s->version < 2)
                     s->offset[channel][s->nmean - 1] = sum / s->blocksize;
                 else
-                    s->offset[channel][s->nmean - 1] = s->bitshift == 32 ? 0 : (sum / s->blocksize) * (1 << s->bitshift);
+                    s->offset[channel][s->nmean - 1] = s->bitshift == 32 ? 0 : (sum / s->blocksize) * (1LL << s->bitshift);
             }
 
             /* copy wrap samples for use with next block */