swr: Remember previously set int_sample_format from user