avcodec/truemotion2: Fix several integer overflows in tm2_low_res_block()
authorMichael Niedermayer <michael@niedermayer.cc>
Sat, 26 Oct 2019 19:39:41 +0000 (21:39 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Fri, 1 Nov 2019 19:12:14 +0000 (20:12 +0100)
Fixes: signed integer overflow: 1077952576 + 1355863565 cannot be represented in type 'int'
Fixes: 16196/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5679842317565952

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/truemotion2.c

index c736320..fbdb657 100644 (file)
@@ -581,10 +581,10 @@ static inline void tm2_low_res_block(TM2Context *ctx, AVFrame *pic, int bx, int
     deltas[10] = GET_TOK(ctx, TM2_L_LO);
 
     if (bx > 0)
-        last[0] = (last[-1] - ctx->D[0] - ctx->D[1] - ctx->D[2] - ctx->D[3] + last[1]) >> 1;
+        last[0] = (int)((unsigned)last[-1] - ctx->D[0] - ctx->D[1] - ctx->D[2] - ctx->D[3] + last[1]) >> 1;
     else
-        last[0] = (last[1]  - ctx->D[0] - ctx->D[1] - ctx->D[2] - ctx->D[3])>> 1;
-    last[2] = (last[1] + last[3]) >> 1;
+        last[0] = (int)((unsigned)last[1]  - ctx->D[0] - ctx->D[1] - ctx->D[2] - ctx->D[3])>> 1;
+    last[2] = (int)((unsigned)last[1] + last[3]) >> 1;
 
     t1 = ctx->D[0] + ctx->D[1];
     ctx->D[0] = t1 >> 1;