Skip to content

Commit

Permalink
Update releases 2.33.0 and 2.34.0 to include broken BQ copy jobs in b…
Browse files Browse the repository at this point in the history
…atch as a known issue (#27563)

* update release docs

* add error description and workaround

* update

* add java

* fix whitespace
  • Loading branch information
ahmedabu98 authored Jul 20, 2023
1 parent f1c2eb6 commit 3d501ee
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 0 deletions.
3 changes: 3 additions & 0 deletions website/www/site/content/en/blog/beam-2.33.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,9 @@ notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527
* Spark 2.x users will need to update Spark's Jackson runtime dependencies (`spark.jackson.version`) to at least version 2.9.2, due to Beam updating its dependencies.
* See a full list of open [issues that affect](https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20affectedVersion%20%3D%202.33.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC) this version.
* Go SDK jobs may produce "Failed to deduce Step from MonitoringInfo" messages following successful job execution. The messages are benign and don't indicate job failure. These are due to not yet handling PCollection metrics.
* Large Java BigQueryIO writes with the FILE_LOADS method will fail in batch mode (specifically, when copy jobs are used).
This results in the error message: `IllegalArgumentException: Attempting to access unknown side input`.
Please upgrade to a newer version (> 2.34.0) or use another write method (e.g. `STORAGE_WRITE_API`).

## List of Contributors

Expand Down
6 changes: 6 additions & 0 deletions website/www/site/content/en/blog/beam-2.34.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,12 @@ notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527
* Fixed error when importing the DataFrame API with pandas 1.0.x installed ([BEAM-12945](https://issues.apache.org/jira/browse/BEAM-12945)).
* Fixed top.SmallestPerKey implementation in the Go SDK ([BEAM-12946](https://issues.apache.org/jira/browse/BEAM-12946)).

### Known Issues

* Large Java BigQueryIO writes with the FILE_LOADS method will fail in batch mode (specifically, when copy jobs are used).
This results in the error message: `IllegalArgumentException: Attempting to access unknown side input`.
Please upgrade to a newer version (> 2.34.0) or use another write method (e.g. `STORAGE_WRITE_API`).

## List of Contributors

According to git shortlog, the following people contributed to the 2.34.0 release. Thank you to all contributors!
Expand Down

0 comments on commit 3d501ee

Please sign in to comment.