GH-44308: [C++][FS][Azure] Implement SAS token authentication #45021
Conversation
…n. This avoids cheating by using the account key again to generate SAS tokens in tests
cpp/src/arrow/filesystem/azurefs.cc
Outdated
| // Assume these are part of a SAS token. Its not ideal to make such an assumption | ||
| // but given that a SAS token is a complex set of URI parameters, that could be | ||
| // tricky to exhaustively list I think its the best option. | ||
| credential_kind = CredentialKind::kSasToken; |
There was a problem hiding this comment.
We don't have the SAS token specification that includes parameter names used by a SAS token, right?
There was a problem hiding this comment.
Yeah, I had a quick search and couldn't find what we need. If you think it's important I can try a bit harder. The closest I found seemed to be unabbreviated versions of what actually appears in the sas token.
There was a problem hiding this comment.
There are many parameters but can we check them...?
github-actions
bot
added
awaiting changes
github-actions
bot
added
awaiting change review
|
Wait... I might have just accidentally worked out how to avoid any of the special authentication stuff for copying... |
| @@ -311,6 +321,15 @@ Status AzureOptions::ConfigureAccountKeyCredential(const std::string& account_ke | |||
| return Status::OK(); | |||
| } | |||
|
|
|||
| Status AzureOptions::ConfigureSasCredential(const std::string& sas_token) { | |||
There was a problem hiding this comment.
| Status AzureOptions::ConfigureSasCredential(const std::string& sas_token) { | |
| Status AzureOptions::ConfigureSASCredential(const std::string& sas_token) { |
| @@ -690,6 +690,36 @@ class TestAzureOptions : public ::testing::Test { | |||
| ASSERT_EQ(options.credential_kind_, AzureOptions::CredentialKind::kEnvironment); | |||
| } | |||
|
|
|||
| void TestFromUriCredentialSasToken() { | |||
There was a problem hiding this comment.
| void TestFromUriCredentialSasToken() { | |
| void TestFromUriCredentialSASToken() { |
| // We use StartCopyFromUri instead of CopyFromUri because it supports blobs larger | ||
| // than 256 MiB and it doesn't require generating a SAS token to authenticate | ||
| // reading a source blob in the same storage account. | ||
| auto copy_operation = dest_blob_client.StartCopyFromUri(src_url); |
There was a problem hiding this comment.
Wow! I should have found it when I implement this...
| // than 256 MiB and it doesn't require generating a SAS token to authenticate | ||
| // reading a source blob in the same storage account. | ||
| auto copy_operation = dest_blob_client.StartCopyFromUri(src_url); | ||
| copy_operation.PollUntilDone(std::chrono::milliseconds(1000)); |
There was a problem hiding this comment.
Could you add a comment what std::chrono::milliseconds(1000) means?
Rationale for this change
SAS token auth is sometimes useful and it the last one we haven't implemented.
What changes are included in this PR?
ConfigureSasCredentialAzureOptions::FromUriso that simply appending a SAS token to a blob storage URI works. e.g.AzureOptions::FromUri("abfs://[email protected]/?se=2024-12-12T18:57:47Z&sig=pAs7qEBdI6sjUhqX1nrhNAKsTY%2B1SqLxPK%2BbAxLiopw%3D&sp=racwdxylti&spr=https,http&sr=c&sv=2024-08-04")CopyFileto use StartCopyFromUri instead of CopyFromUriAre these changes tested?
Yes
CopyFileAzureOptions::FromUriwith a SAS token.I also made sure to run the tests which connect to real blob storage.
Are there any user-facing changes?
AzureOptions::FromUriinstead of failing fast. IMO this is a regression but still the best option to support SAS token.