fix: update gRPC object list implementation to include synthetic directories #1824
Conversation
product-auto-label
bot
added
size: l
BenWhitehead
force-pushed
the
grpc/obj/list/directories
branch
from
a639a23 to
c5aa5d0
Compare
| @@ -186,46 +179,6 @@ public void testGetBlobKmsKeyNameField() { | |||
| assertNull(remoteBlob.getContentType()); | |||
| } | |||
|
|
|||
| @Test(timeout = 5000) | |||
| public void testListBlobsKmsKeySelectedFields() throws InterruptedException { | |||
There was a problem hiding this comment.
This test was "testing" if an object with a kms key would have said kms key populated in a list response. Field masks and mapping are verified elsewhere, deleting here.
| Storage.BlobListOption.fields(BlobField.METADATA)); | ||
| // Listing blobs is eventually consistent, we loop until the list is of the expected size. The | ||
| // test fails if timeout is reached. |
There was a problem hiding this comment.
Listing objects is strongly consistent[1], we don't need the manual retry loop here.
[1] https://cloud.google.com/storage/docs/consistency#strongly_consistent_operations
| Storage.BlobListOption.prefix("test-list-blobs-empty-selected-fields-blob"), | ||
| Storage.BlobListOption.fields()); | ||
| // Listing blobs is eventually consistent, we loop until the list is of the expected size. The | ||
| // test fails if timeout is reached. |
There was a problem hiding this comment.
Listing objects is strongly consistent[1], we don't need the manual retry loop here.
[1] https://cloud.google.com/storage/docs/consistency#strongly_consistent_operations
| Storage.BlobListOption.prefix("test-list-blobs-current-directory/"), | ||
| Storage.BlobListOption.currentDirectory()); | ||
| // Listing blobs is eventually consistent, we loop until the list is of the expected size. The | ||
| // test fails if timeout is reached. |
There was a problem hiding this comment.
Listing objects is strongly consistent[1], we don't need the manual retry loop here.
[1] https://cloud.google.com/storage/docs/consistency#strongly_consistent_operations
| } | ||
|
|
||
| @Test | ||
| public void testListBlobsMultiplePages() { |
There was a problem hiding this comment.
Since we're having to walk the pages ourselves now, add a test to verify we are doing it accurately.
| started = true; | ||
| action.accept(seed); | ||
| prev = seed; | ||
| private static <T> Stream<T> streamIterate( |
There was a problem hiding this comment.
No actual code change to this helper method, simply un-nested it from TransformingPageDecorator
| @@ -63,7 +57,6 @@ | |||
| @CrossRun( | |||
| transports = {Transport.HTTP, Transport.GRPC}, | |||
| backends = Backend.PROD) | |||
| @ParallelFriendly | |||
There was a problem hiding this comment.
Yes, I found a time where this tests was not parallel friendly as some object names weren't unique across CrossRun's and one deleted resources of the other. We can bring this back once we have hermetic object names.
…ctories Rework ITObjectTest.testListBlobsCurrentDirectory to have a more descriptive name along with a more straight forward implementation and assertions. (Removed all the listing retry stuff, object list is strongly consistent in GCS.) * test: remove obsolete test which is validating field mask which is tested separately
BenWhitehead
force-pushed
the
grpc/obj/list/directories
branch
from
892ec4e to
c5b9dd0
Compare
Rework list centric tests in ITObjectTest to have more straightforward implementations and assertions, and in some cases a more clear name of what is being validated. (Removed all the list retry stuff, object list is strongly consistent in GCS.)