lavf: Don't drop both pts and dts if timestamps are invalid