Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-45015: [C++][Parquet] Allow configuring the default footer read size #45016

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

zeroshade
Copy link
Member

@zeroshade zeroshade commented Dec 12, 2024

Rationale for this change

Reading the footer for a parquet file whose file metadata is >64KB can require multiple round trips to a high latency file system like S3. Allowing this default read size to be configurable allows for reducing the round trips if it's known up front that the file might potentially have a large amount of metadata.

What changes are included in this PR?

A footer_read_size_ property is added to parquet ReaderProperties along with a getter/setter. This is then utilized in the file reader's GetFooterReadSize method.

@zeroshade zeroshade requested a review from lidavidm December 12, 2024 20:46
@zeroshade zeroshade requested a review from wgtmac as a code owner December 12, 2024 20:46
Copy link

⚠️ GitHub issue #45015 has been automatically assigned in GitHub to PR creator.

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting committer review Awaiting committer review labels Dec 13, 2024
@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting merge Awaiting merge labels Dec 13, 2024
@github-actions github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes awaiting change review Awaiting change review labels Dec 13, 2024
@github-actions github-actions bot added awaiting changes Awaiting changes awaiting change review Awaiting change review and removed awaiting changes Awaiting changes awaiting change review Awaiting change review labels Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants