avcodec/diracdec: Correct max pixels check
authorMichael Niedermayer <michael@niedermayer.cc>
Mon, 25 Feb 2019 20:37:45 +0000 (21:37 +0100)
committerMichael Niedermayer <michael@niedermayer.cc>
Sun, 3 Mar 2019 12:17:02 +0000 (13:17 +0100)
Dirac internally allocates 5 images per plane and frame currently. One being the actual
image the other 4 being filtered for motion compensation.

Fixes: Out of memory
Fixes: 12870/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5684825871089664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/diracdec.c

index db010b4..30b4bfa 100644 (file)
@@ -2137,7 +2137,7 @@ static int dirac_decode_data_unit(AVCodecContext *avctx, const uint8_t *buf, int
             return ret;
         }
 
-        if (CALC_PADDING((int64_t)dsh->width, MAX_DWT_LEVELS) * CALC_PADDING((int64_t)dsh->height, MAX_DWT_LEVELS) > avctx->max_pixels)
+        if (CALC_PADDING((int64_t)dsh->width, MAX_DWT_LEVELS) * CALC_PADDING((int64_t)dsh->height, MAX_DWT_LEVELS) * 5LL > avctx->max_pixels)
             ret = AVERROR(ERANGE);
         if (ret >= 0)
             ret = ff_set_dimensions(avctx, dsh->width, dsh->height);