Skip to content

Commit

Permalink
Add back cloud profiler (apache#26700)
Browse files Browse the repository at this point in the history
  • Loading branch information
tvalentyn authored and cushon committed May 24, 2024
1 parent f9b0333 commit 21fcc55
Show file tree
Hide file tree
Showing 7 changed files with 217 additions and 202 deletions.
5 changes: 5 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,11 @@

* BigQuery sink in STORAGE_WRITE_API mode in batch pipelines could result in data consistency issues during the handling of other unrelated transient errors for Beam SDKs 2.35.0 - 2.46.0 (inclusive). For more details see: https://github.com/apache/beam/issues/26521

## Known Issues

* The google-cloud-profiler dependency was accidentally removed from Beam's Python Docker
Image [#26998](https://github.com/apache/beam/issues/26698). [Dataflow Docker images](https://cloud.google.com/dataflow/docs/concepts/sdk-worker-dependencies) still preinstall this dependency.

# [2.46.0] - 2023-03-10

## Highlights
Expand Down
2 changes: 2 additions & 0 deletions sdks/python/container/base_image_requirements_manual.txt
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ cython<1
future
# TODO: Uncomment python version once guppy supports it.
# https://github.com/apache/beam/issues/25984
# TODO(https://github.com/GoogleCloudPlatform/cloud-profiler-python/issues/127): Unpin python version.
google-cloud-profiler;python_version<"3.11"
guppy3;python_version<"3.11" # Memory profiler
mmh3 # Optimizes execution of some Beam codepaths. TODO: Make it Beam's dependency.
nltk # Commonly used for natural language processing.
Expand Down
84 changes: 43 additions & 41 deletions sdks/python/container/py310/base_image_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,132 +23,134 @@

absl-py==1.4.0
astunparse==1.6.3
attrs==22.2.0
beautifulsoup4==4.12.0
attrs==23.1.0
beautifulsoup4==4.12.2
bs4==0.0.1
cachetools==4.2.4
certifi==2022.12.7
cachetools==5.3.0
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==3.1.0
click==8.1.3
cloudpickle==2.2.1
crcmod==1.7
cryptography==40.0.1
cryptography==40.0.2
Cython==0.29.34
deprecation==2.1.0
dill==0.3.1.1
dnspython==2.3.0
docker==6.0.1
docker==6.1.2
docopt==0.6.2
exceptiongroup==1.1.1
execnet==1.9.0
fastavro==1.7.3
fastavro==1.7.4
fasteners==0.18
flatbuffers==23.3.3
flatbuffers==23.5.9
freezegun==1.2.2
future==0.18.3
gast==0.4.0
google-api-core==2.11.0
google-api-python-client==2.86.0
google-apitools==0.5.31
google-auth==2.17.1
google-auth==2.18.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud-bigquery==3.9.0
google-cloud-bigquery==3.10.0
google-cloud-bigquery-storage==2.19.1
google-cloud-bigtable==2.17.0
google-cloud-core==2.3.2
google-cloud-datastore==2.15.1
google-cloud-datastore==2.15.2
google-cloud-dlp==3.12.1
google-cloud-language==2.9.1
google-cloud-pubsub==2.15.2
google-cloud-pubsublite==1.8.1
google-cloud-profiler==4.0.0
google-cloud-pubsub==2.17.0
google-cloud-pubsublite==1.8.2
google-cloud-recommendations-ai==0.10.3
google-cloud-spanner==3.30.0
google-cloud-spanner==3.33.0
google-cloud-videointelligence==2.11.1
google-cloud-vision==3.4.1
google-crc32c==1.5.0
google-pasta==0.2.0
google-resumable-media==2.4.1
google-resumable-media==2.5.0
googleapis-common-protos==1.59.0
greenlet==2.0.2
grpc-google-iam-v1==0.12.6
grpcio==1.53.0
grpcio-status==1.53.0
guppy3==3.1.2
grpcio==1.54.2
grpcio-status==1.54.2
guppy3==3.1.3
h5py==3.8.0
hdfs==2.7.0
httplib2==0.21.0
hypothesis==6.70.2
httplib2==0.22.0
hypothesis==6.75.3
idna==3.4
iniconfig==2.0.0
jax==0.4.8
jax==0.4.10
joblib==1.2.0
keras==2.12.0
libclang==16.0.0
Markdown==3.4.3
MarkupSafe==2.1.2
ml-dtypes==0.0.4
ml-dtypes==0.1.0
mmh3==3.1.0
mock==5.0.1
mock==5.0.2
nltk==3.8.1
nose==1.3.7
numpy==1.23.5
oauth2client==4.1.3
oauthlib==3.2.2
objsize==0.6.1
opt-einsum==3.3.0
orjson==3.8.9
orjson==3.8.12
overrides==6.5.0
packaging==23.0
packaging==23.1
pandas==1.5.3
parameterized==0.8.1
parameterized==0.9.0
pluggy==1.0.0
proto-plus==1.22.2
protobuf==4.22.1
protobuf==4.22.5
psycopg2-binary==2.9.6
pyarrow==11.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser==2.21
pydot==1.4.2
PyHamcrest==1.10.1
PyHamcrest==2.0.4
pymongo==4.3.3
PyMySQL==1.0.3
pyparsing==3.0.9
pytest==7.2.2
pytest==7.3.1
pytest-timeout==2.1.0
pytest-xdist==3.2.1
pytest-xdist==3.3.0
python-dateutil==2.8.2
python-snappy==0.6.1
pytz==2023.3
PyYAML==6.0
regex==2023.3.23
requests==2.28.2
regex==2023.5.5
requests==2.30.0
requests-mock==1.10.0
requests-oauthlib==1.3.1
rsa==4.9
scikit-learn==1.2.2
scipy==1.10.1
six==1.16.0
sortedcontainers==2.4.0
soupsieve==2.4
SQLAlchemy==1.4.47
soupsieve==2.4.1
SQLAlchemy==1.4.48
sqlparse==0.4.4
tenacity==8.2.2
tensorboard==2.12.1
tensorboard==2.12.3
tensorboard-data-server==0.7.0
tensorboard-plugin-wit==1.8.1
tensorflow==2.12.0
tensorflow-estimator==2.12.0
tensorflow-io-gcs-filesystem==0.32.0
termcolor==2.2.0
termcolor==2.3.0
testcontainers==3.7.1
threadpoolctl==3.1.0
tomli==2.0.1
tqdm==4.65.0
typing_extensions==4.5.0
uritemplate==4.1.1
urllib3==1.26.15
websocket-client==1.5.1
Werkzeug==2.2.3
Werkzeug==2.3.4
wrapt==1.14.1
zstandard==0.20.0
zstandard==0.21.0
79 changes: 39 additions & 40 deletions sdks/python/container/py311/base_image_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,128 +23,127 @@

absl-py==1.4.0
astunparse==1.6.3
attrs==22.2.0
beautifulsoup4==4.12.0
attrs==23.1.0
beautifulsoup4==4.12.2
bs4==0.0.1
cachetools==4.2.4
certifi==2022.12.7
cachetools==5.3.0
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==3.1.0
click==8.1.3
cloudpickle==2.2.1
crcmod==1.7
cryptography==40.0.1
cryptography==40.0.2
Cython==0.29.34
deprecation==2.1.0
dill==0.3.1.1
dnspython==2.3.0
docker==6.0.1
docker==6.1.2
docopt==0.6.2
execnet==1.9.0
fastavro==1.7.3
fastavro==1.7.4
fasteners==0.18
flatbuffers==23.3.3
flatbuffers==23.5.9
freezegun==1.2.2
future==0.18.3
gast==0.4.0
google-api-core==2.11.0
google-apitools==0.5.31
google-auth==2.17.1
google-auth==2.18.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud-bigquery==3.9.0
google-cloud-bigquery==3.10.0
google-cloud-bigquery-storage==2.19.1
google-cloud-bigtable==2.17.0
google-cloud-core==2.3.2
google-cloud-datastore==2.15.1
google-cloud-datastore==2.15.2
google-cloud-dlp==3.12.1
google-cloud-language==2.9.1
google-cloud-pubsub==2.15.2
google-cloud-pubsublite==1.8.1
google-cloud-pubsub==2.17.0
google-cloud-pubsublite==1.8.2
google-cloud-recommendations-ai==0.10.3
google-cloud-spanner==3.30.0
google-cloud-spanner==3.33.0
google-cloud-videointelligence==2.11.1
google-cloud-vision==3.4.1
google-crc32c==1.5.0
google-pasta==0.2.0
google-resumable-media==2.4.1
google-resumable-media==2.5.0
googleapis-common-protos==1.59.0
greenlet==2.0.2
grpc-google-iam-v1==0.12.6
grpcio==1.53.0
grpcio-status==1.53.0
grpcio==1.54.2
grpcio-status==1.54.2
h5py==3.8.0
hdfs==2.7.0
httplib2==0.21.0
hypothesis==6.70.2
httplib2==0.22.0
hypothesis==6.75.3
idna==3.4
iniconfig==2.0.0
jax==0.4.8
jax==0.4.10
joblib==1.2.0
keras==2.12.0
libclang==16.0.0
Markdown==3.4.3
MarkupSafe==2.1.2
ml-dtypes==0.0.4
ml-dtypes==0.1.0
mmh3==3.1.0
mock==5.0.1
mock==5.0.2
nltk==3.8.1
nose==1.3.7
numpy==1.23.5
oauth2client==4.1.3
oauthlib==3.2.2
objsize==0.6.1
opt-einsum==3.3.0
orjson==3.8.9
orjson==3.8.12
overrides==6.5.0
packaging==23.0
packaging==23.1
pandas==1.5.3
parameterized==0.8.1
parameterized==0.9.0
pluggy==1.0.0
proto-plus==1.22.2
protobuf==4.22.1
protobuf==4.22.5
psycopg2-binary==2.9.6
pyarrow==11.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser==2.21
pydot==1.4.2
PyHamcrest==1.10.1
PyHamcrest==2.0.4
pymongo==4.3.3
PyMySQL==1.0.3
pyparsing==3.0.9
pytest==7.2.2
pytest==7.3.1
pytest-timeout==2.1.0
pytest-xdist==3.2.1
pytest-xdist==3.3.0
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0
regex==2023.3.23
requests==2.28.2
regex==2023.5.5
requests==2.30.0
requests-mock==1.10.0
requests-oauthlib==1.3.1
rsa==4.9
scikit-learn==1.2.2
scipy==1.10.1
six==1.16.0
sortedcontainers==2.4.0
soupsieve==2.4
SQLAlchemy==1.4.47
soupsieve==2.4.1
SQLAlchemy==1.4.48
sqlparse==0.4.4
tenacity==8.2.2
tensorboard==2.12.1
tensorboard==2.12.3
tensorboard-data-server==0.7.0
tensorboard-plugin-wit==1.8.1
tensorflow==2.12.0
tensorflow-estimator==2.12.0
tensorflow-io-gcs-filesystem==0.32.0
termcolor==2.2.0
termcolor==2.3.0
testcontainers==3.7.1
threadpoolctl==3.1.0
tqdm==4.65.0
typing_extensions==4.5.0
urllib3==1.26.15
websocket-client==1.5.1
Werkzeug==2.2.3
Werkzeug==2.3.4
wrapt==1.14.1
zstandard==0.20.0
zstandard==0.21.0
Loading

0 comments on commit 21fcc55

Please sign in to comment.