avcodec/agm: Fix overflow of signed shift
authorMichael Niedermayer <michael@niedermayer.cc>
Sun, 30 Jun 2019 17:45:29 +0000 (19:45 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Fri, 19 Jul 2019 19:42:38 +0000 (21:42 +0200)
Fixes: left shift of 1 by 31 places cannot be represented in type 'int'
Fixes: 15328/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5637545171353600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavcodec/agm.c

index a499c09..2c4c980 100644 (file)
@@ -918,7 +918,7 @@ static void get_tree_codes(uint32_t *codes, Node *nodes, int idx, uint32_t pfx,
         codes[idx] = pfx;
     } else if (idx >= 0) {
         get_tree_codes(codes, nodes, nodes[idx].child[0], pfx + (0 << bitpos), bitpos + 1);
-        get_tree_codes(codes, nodes, nodes[idx].child[1], pfx + (1 << bitpos), bitpos + 1);
+        get_tree_codes(codes, nodes, nodes[idx].child[1], pfx + (1U << bitpos), bitpos + 1);
     }
 }