Skip to content

Commit

Permalink
Add standard {ReadFrom,WriteTo}{Csv,Json} yaml endpoints. (#28268)
Browse files Browse the repository at this point in the history
  • Loading branch information
robertwb authored Sep 8, 2023
1 parent 19160f4 commit ee8e53b
Showing 1 changed file with 29 additions and 0 deletions.
29 changes: 29 additions & 0 deletions sdks/python/apache_beam/yaml/standard_io.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,32 @@
'ReadFromBigQuery': 'apache_beam.yaml.yaml_io.read_from_bigquery'
# Disable until https://github.com/apache/beam/issues/28162 is resolved.
# 'WriteToBigQuery': 'apache_beam.yaml.yaml_io.write_to_bigquery'

# Declared as a renaming transform to avoid exposing all
# (implementation-specific) pandas arguments and aligning with possible Java
# implementation.
# Invoking these directly as a PyTransform is still an option for anyone wanting
# to use these power-features in a language-dependent manner.
- type: renaming
transforms:
'ReadFromCsv': 'ReadFromCsv'
'WriteToCsv': 'WriteToCsv'
'ReadFromJson': 'ReadFromJson'
'WriteToJson': 'WriteToJson'
config:
mappings:
'ReadFromCsv':
path: 'path'
'WriteToCsv':
path: 'path'
'ReadFromJson':
path: 'path'
'WriteToJson':
path: 'path'
underlying_provider:
type: python
transforms:
'ReadFromCsv': 'apache_beam.io.ReadFromCsv'
'WriteToCsv': 'apache_beam.io.WriteToCsv'
'ReadFromJson': 'apache_beam.io.ReadFromJson'
'WriteToJson': 'apache_beam.io.WriteToJson'

0 comments on commit ee8e53b

Please sign in to comment.