Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task]: Implement CsvIO.Read #24552

Closed
1 of 6 tasks
damondouglas opened this issue Dec 6, 2022 · 2 comments
Closed
1 of 6 tasks

[Task]: Implement CsvIO.Read #24552

damondouglas opened this issue Dec 6, 2022 · 2 comments

Comments

@damondouglas
Copy link
Contributor

damondouglas commented Dec 6, 2022

What work does this Task describe?

This task addresses support for transforming CSV records into schema aware PCollection elements.

What value may result from this Task's output?

Some legacy systems only support CSV export and having a mechanism for parsing files in CSV format may help users.

Ready and Done Measures

Ready

Done

The following are measures for marking this Task complete.

Tests detect errors in:

  • validating an output Schema
  • converting PCollection<String> to PCollection<Row>
  • converting PCollection<String> to PCollection<T> where T is a user class type
  • mapping a header to an input Schema field names, included and positioned appropriately for each sharded CSV file
  • PBegin source with multiple headers map to the appropriate Schema

Issue Priority

Priority: 3

Issue Component

Component: beam-community

@twosom
Copy link
Contributor

twosom commented Oct 8, 2024

Hi. is this task done?

I think it overlaps with the issue #31877

@liferoad
Copy link
Collaborator

@damondouglas Please check.

@damondouglas damondouglas closed this as not planned Won't fix, can't repro, duplicate, stale Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants