avformat/movenc: Check pal_size before use