Wait for CreateMPU before returning from put_object

[passaro]

S3CrtClient::put_object was originally implemented so that it would complete immediately and return a PutObjectRequest implementation. Any error from the S3 request would only be returned on calling write or complete on the PutObjectRequest. With this change, we modify put_object to await for the initial CreateMultipartUpload request to complete and only then either return a PutObjectRequest or propagate the error from the request. This is analogous to what done for get_object in #1171 and addresses an issue where errors were not propagated correctly (#1007).

At the file handle level, however, we still want the open operation to complete quickly, without waiting for CreateMultipartUpload to complete. In order to preserve the previous behavior, upload::atomic was adapted to spawn a concurrent task in the background when calling put_object.

Does this change impact existing behavior?

Yes.

Does this change need a changelog entry?

Yes, for mountpoint-s3-client. No user-visible changes in mountpoint-s3.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

[monthonk]

lgtm, nice that we can maintain the same behavior for open.

need to look at the failing test