avfilter/vf_xbr: Fix left shift of negative number
authorAndreas Rheinhardt <andreas.rheinhardt@gmail.com>
Sat, 28 Sep 2019 02:25:56 +0000 (04:25 +0200)
committerMichael Niedermayer <michael@niedermayer.cc>
Sat, 28 Sep 2019 15:24:32 +0000 (17:24 +0200)
Affected every usage of vf_xbr, e.g. the FATE-tests filter-2xbr,
filter-3xbr, filter-4xbr.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
libavfilter/vf_xbr.c

index 2c71871..87810c8 100644 (file)
@@ -395,7 +395,7 @@ static int init(AVFilterContext *ctx)
             int startg = FFMAX3(-bg, -rg, 0);
             int endg = FFMIN3(255-bg, 255-rg, 255);
             uint32_t y = (uint32_t)(( 299*rg + 1000*startg + 114*bg)/1000);
-            c = bg + (rg<<16) + 0x010101 * startg;
+            c = bg + rg * (1 << 16) + 0x010101 * startg;
             for (g = startg; g <= endg; g++) {
                 s->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v;
                 c+= 0x010101;