Core: Seeing unexpected exception during upload to GCS with Java client

[coopsplace]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/java-storage/issues
• Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform

If you are still having issues, please include as much information as possible:

Environment details

1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
   General, Core, and Other are also allowed as types
2. OS type and version: Linux x64 docker container unsure of specifics
3. Java version: 11
4. version(s): GCS Library version 2.38.0

Steps to reproduce

1. Happens intermittently on upload.

Code example

        final long ret =
                isArchive ? uploadFile.length() : FileUtils.sizeOfDirectory(uploadFile);
        if (isArchive) {
            BlobId blobId = BlobId.of(s3BucketName, s3Path);
            BlobInfo blobInfo = BlobInfo.newBuilder(blobId)
                    .setStorageClass(infrequentAccess
                            ? com.google.cloud.storage.StorageClass.NEARLINE
                            : com.google.cloud.storage.StorageClass.STANDARD).build();
            storage.createFrom(blobInfo, uploadFile.toPath());
        } else {
            try (Stream<Path> stream = Files.walk(uploadFile.toPath())) {
                forkJoinPool.submit(() ->
                        stream.parallel().filter(x -> !x.toFile().isDirectory())
                                .forEach(filePath -> {
                                    try {
                                        uploadFile(filePath.toFile(), s3BucketName,
                                                Path.of(s3Path, uploadFile.toPath()
                                                                .relativize(filePath)
                                                                .toString())
                                                        .toString(),
                                                infrequentAccess, true, sse);
                                    } catch (Exception e) {
                                        throw new RuntimeException(e);
                                    }
                                })).get();
            }
        }
        return ret;

Stack trace

com.google.cloud.storage.StorageException: Client side data loss detected. Attempt to append to a resumable session with an offset higher than the backend has 
|> PUT https://storage.googleapis.com/upload/storage/v1/b/prd-us-psc-lc-shared-infra-indexes-gcs-green/o?name=class%3Ddefault/org%3DN88FD8JR/partition%3Dd/type%3Draw_current/tstamp%3D0/size%3D0/partition-55&uploadType=resumable&upload_id=AD-8ljvvtif0xx7yz_pwkGcUQbPSH3b96EmAPBDcvZNDJHFHoZOTV6D9F5vM6qeP29LraFm2k5T2_APXqhM8V3ctwROvtRb6F4ALqoQ_L6mZD-Qd2A 
|> content-range: bytes 1003487232-2090235391/1086748160 
|> x-goog-gcs-idempotency-token: e5ebfb44-2ad2-4a60-a45a-9e4c121af017 
| 
|< HTTP/1.1 400 Bad Request 
|< content-length: 37 
|< content-type: text/plain; charset=utf-8 
|< 
|< Failed to parse Content-Range header. 
| 
  at com.google.cloud.storage.ResumableSessionFailureScenario.toStorageException(ResumableSessionFailureScenario.java:305) at com.google.cloud.storage.ResumableSessionFailureScenario.toStorageException(ResumableSessionFailureScenario.java:133) 
  at com.google.cloud.storage.JsonResumableSessionPutTask.call(JsonResumableSessionPutTask.java:210) 
  at com.google.cloud.storage.JsonResumableSession.lambda$put$0(JsonResumableSession.java:81) 
  at com.google.cloud.storage.Retrying.lambda$run$0(Retrying.java:102) 
  at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103) 
  at com.google.cloud.RetryHelper.run(RetryHelper.java:76) 
  at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) 
  at com.google.cloud.storage.Retrying.run(Retrying.java:99) 
  at com.google.cloud.storage.JsonResumableSession.put(JsonResumableSession.java:68) 
  at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:272) 
  at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:235)

External references such as API reference guides

• ?

Any additional information below

Case 53758543 opened with support, they suggested I also open an issue here.

Following these steps guarantees the quickest resolution possible.

Thanks!

[BenWhitehead]

Hi thanks for opening the issue and providing the full exception. A proposed fix is in #2771.

[coopsplace]

That's great thank you.