h264: check the pixel format directly and force a reinit on mismatches.
authorMichael Niedermayer <michaelni@gmx.at>
Tue, 29 Jan 2013 03:17:48 +0000 (04:17 +0100)
committerMichael Niedermayer <michaelni@gmx.at>
Tue, 5 Feb 2013 15:50:52 +0000 (16:50 +0100)
The existing checks are insufficient to detect a pixel format
changes in case of some damaged streams.
Fixes inconsistency and later out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11c99c78bafa77f679a1a3ba06ad00984b9a4cae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
libavcodec/h264.c

index 7ae778a..1152258 100644 (file)
@@ -2765,7 +2765,8 @@ static int decode_slice_header(H264Context *h, H264Context *h0)
                      || s->avctx->bits_per_raw_sample != h->sps.bit_depth_luma
                      || h->cur_chroma_format_idc != h->sps.chroma_format_idc
                      || av_cmp_q(h->sps.sar, s->avctx->sample_aspect_ratio)));
-
+    if (h0->s.avctx->pix_fmt != get_pixel_format(h0))
+        must_reinit = 1;
 
     s->mb_width  = h->sps.mb_width;
     s->mb_height = h->sps.mb_height * (2 - h->sps.frame_mbs_only_flag);