Skip to content

Commit

Permalink
Allow JSON type in TableSchema for WriteToBigQuery with FILE_LOAD met…
Browse files Browse the repository at this point in the history
…hod in Python SDK (#30340)

* remove JSON type checking and raise error in WriteoToBigQuery with FILE_LOADS method

* Changing to warning instead

* update warning message. Nw should be the same with Java sdk

* fix lie length

* fixing formatting
  • Loading branch information
sapphire008 authored Feb 20, 2024
1 parent 4120041 commit e00a81f
Showing 1 changed file with 5 additions and 7 deletions.
12 changes: 5 additions & 7 deletions sdks/python/apache_beam/io/gcp/bigquery.py
Original file line number Diff line number Diff line change
Expand Up @@ -2178,13 +2178,11 @@ def expand(self, pcoll):
def find_in_nested_dict(schema):
for field in schema['fields']:
if field['type'] == 'JSON':
raise ValueError(
'Found JSON type in table schema. JSON data '
'insertion is currently not supported with '
'FILE_LOADS write method. This is supported with '
'STREAMING_INSERTS. For more information: '
'https://cloud.google.com/bigquery/docs/reference/'
'standard-sql/json-data#ingest_json_data')
logging.warning(
'Found JSON type in TableSchema for "File_LOADS" write '
'method. Make sure the TableSchema field is a parsed '
'JSON to ensure the read as a JSON type. Otherwise it '
'will read as a raw (escaped) string.')
elif field['type'] == 'STRUCT':
find_in_nested_dict(field)

Expand Down

0 comments on commit e00a81f

Please sign in to comment.