swscale: fix internal rgb->yv12 chroma conversion used by the main scaler path
authorRamiro Polla <ramiro.polla@gmail.com>
Mon, 6 Sep 2010 02:02:53 +0000 (02:02 +0000)
committerRamiro Polla <ramiro.polla@gmail.com>
Mon, 6 Sep 2010 02:02:53 +0000 (02:02 +0000)
The shift must be applied before the masking.

Originally committed as revision 32051 to svn://svn.mplayerhq.hu/mplayer/trunk/libswscale

libswscale/swscale.c

index 118816a..6ceddfd 100644 (file)
@@ -1069,9 +1069,9 @@ static inline void name(uint8_t *dstU, uint8_t *dstV, const uint8_t *src, const
 {\
     int i;\
     for (i=0; i<width; i++) {\
-        int b= (((const type*)src)[i]&maskb)>>(shb+shp);\
-        int g= (((const type*)src)[i]&maskg)>>(shg+shp);\
-        int r= (((const type*)src)[i]&maskr)>>(shr+shp);\
+        int b= ((((const type*)src)[i]>>shp)&maskb)>>shb;\
+        int g= ((((const type*)src)[i]>>shp)&maskg)>>shg;\
+        int r= ((((const type*)src)[i]>>shp)&maskr)>>shr;\
 \
         dstU[i]= ((RU)*r + (GU)*g + (BU)*b + (257<<((S)-1)))>>(S);\
         dstV[i]= ((RV)*r + (GV)*g + (BV)*b + (257<<((S)-1)))>>(S);\