fix qtrle encoding when previous frame linesize differs, fix #998