rv10/20: Fix a buffer overread caused by losing track of the remaining buffer size.