avcodec/mjpegdec: Fix integer overflow in ljpeg_decode_rgb_scan()
authorMichael Niedermayer <michael@niedermayer.cc>
Tue, 5 Jun 2018 00:17:24 +0000 (02:17 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Wed, 6 Jun 2018 14:58:08 +0000 (16:58 +0200)
Fixes: signed integer overflow: 32768 + 2147450880 cannot be represented in type 'int'
Fixes: 7885/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_THP_fuzzer-5298834394578944

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/mjpegdec.c

index 22ca69f..d1dca84 100644 (file)
@@ -1078,7 +1078,7 @@ static int ljpeg_decode_rgb_scan(MJpegDecodeContext *s, int nb_components, int p
                     return -1;
 
                 left[i] = buffer[mb_x][i] =
-                    mask & (pred + (dc * (1 << point_transform)));
+                    mask & (pred + (unsigned)(dc * (1 << point_transform)));
             }
 
             if (s->restart_interval && !--s->restart_count) {