Quant changes only once per MB so move the corresponding scale factor assignment