Do not reuse the rbsp de-escape buffer if both