hevc: Fix modulo operations