avcodec/extract_extradata: return an error when buffer allocation fails