Skip to content

Commit

Permalink
Add links to colab/github repo in notebook (#24886)
Browse files Browse the repository at this point in the history 
* Add links to colab/github repo in notebook

* Align links

* Fix 2 missed notebooks
  • Loading branch information
@damccorm
damccorm authored Jan 4, 2023
1 parent 0a17788 commit 12f1072
Show file tree
Hide file tree
Showing 9 changed files with 143 additions and 1 deletion.
16 changes: 16 additions & 0 deletions examples/notebooks/beam-ml/custom_remote_inference.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,22 @@
"source": [
"# Remote inference in Apache Beam\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/custom_remote_inference.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/custom_remote_inference.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GNbarEZsalS2"
},
"source": [
"This example demonstrates how to implement a custom inference call in Apache Beam using the Google Cloud Vision API.\n",
"\n",
"The prefered way to run inference in Apache Beam is by using the [RunInference API](https://beam.apache.org/documentation/sdks/python-machine-learning/). \n",
Expand Down
16 changes: 16 additions & 0 deletions examples/notebooks/beam-ml/dataframe_api_preprocessing.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,22 @@
"\n",
"[Pandas DataFrames](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html) is one of the most common tools used for data exploration and preprocessing. Pandas is popular because of its ease of use. It has intuitive methods to perform common analytical tasks and data preprocessing. \n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/dataframe_api_preprocessing.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/dataframe_api_preprocessing.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
],
"metadata": {
"id": "A8xNRyZMW1yK"
}
},
{
"cell_type": "markdown",
"source": [
"For rapid execution, Pandas loads all of the data into memory on a single machine (one node). This configuration works well when dealing with small-scale datasets. However, many projects involve datasets that are too big to fit in memory. These use cases generally require parallel data processing frameworks, such as Apache Beam.\n",
"\n",
"Beam DataFrames provide a Pandas-like\n",
Expand Down
16 changes: 16 additions & 0 deletions examples/notebooks/beam-ml/run_custom_inference.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,22 @@
"source": [
"# Bring your own machine learning (ML) model to Beam RunInference\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_custom_inference.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_custom_inference.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "A8xNRyZMW1yK"
},
"source": [
"This notebook demonstrates how to run inference on your custom framework using the\n",
"[ModelHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelHandler) class.\n",
"\n",
Expand Down
11 changes: 10 additions & 1 deletion examples/notebooks/beam-ml/run_inference_multi_model.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,16 @@
{
"cell_type": "markdown",
"source": [
"# Ensemble model using an image captioning and ranking example"
"# Ensemble model using an image captioning and ranking example",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_multi_model.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_multi_model.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>"
],
"metadata": {
"id": "gPCMXWgOMt_0"
Expand Down
17 changes: 17 additions & 0 deletions examples/notebooks/beam-ml/run_inference_pytorch.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,23 @@
},
"source": [
"# Apache Beam RunInference for PyTorch\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "A8xNRyZMW1yK"
},
"source": [
"This notebook demonstrates the use of the RunInference transform for PyTorch. Apache Beam includes implementations of the [ModelHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelHandler) class for [users of PyTorch](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.pytorch_inference.html). For more information about the RunInference API, see [Machine Learning](https://beam.apache.org/documentation/sdks/python-machine-learning) in the Apache Beam documentation.\n",
"\n",
"\n",
Expand Down
17 changes: 17 additions & 0 deletions examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,23 @@
},
"source": [
"# Use RunInference in Apache Beam\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "A8xNRyZMW1yK"
},
"source": [
"You can use Apache Beam versions 2.40.0 and later with the [RunInference API](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.RunInference) for local and remote inference with batch and streaming pipelines.\n",
"The RunInference API leverages Apache Beam concepts, such as the `BatchElements` transform and the `Shared` class, to support models in your pipelines that create transforms optimized for machine learning inference.\n",
"\n",
Expand Down
17 changes: 17 additions & 0 deletions examples/notebooks/beam-ml/run_inference_sklearn.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,23 @@
},
"source": [
"# Apache Beam RunInference for scikit-learn\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_sklearn.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_sklearn.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "A8xNRyZMW1yK"
},
"source": [
"This notebook demonstrates the use of the RunInference transform for [scikit-learn](https://scikit-learn.org/), also called sklearn.\n",
"Apache Beam [RunInference](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.RunInference) has implementations of the [ModelHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelHandler) class prebuilt for scikit-learn. For more information about the RunInference API, see [Machine Learning](https://beam.apache.org/documentation/sdks/python-machine-learning) in the Apache Beam documentation.\n",
"\n",
Expand Down
17 changes: 17 additions & 0 deletions examples/notebooks/beam-ml/run_inference_tensorflow.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,23 @@
"cell_type": "markdown",
"source": [
"# Apache Beam RunInference with TensorFlow\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "A8xNRyZMW1yK"
},
"source": [
"This notebook demonstrates the use of the RunInference transform for [TensorFlow](https://www.tensorflow.org/).\n",
"Beam [RunInference](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.RunInference) accepts a ModelHandler generated from [`tfx-bsl`](https://github.com/tensorflow/tfx-bsl) using `CreateModelHandler`.\n",
"\n",
Expand Down
17 changes: 17 additions & 0 deletions examples/notebooks/beam-ml/run_inference_tensorflow_hub.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,23 @@
"source": [
"# RunInference with Sentence-T5 (ST5) model\n",
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow_hub.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/colab_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow_hub.ipynb\"><img src=\"https://raw.githubusercontent.com/google/or-tools/main/tools/github_32px.png\" />View source on GitHub</a>\n",
" </td>\n",
"</table>\n"
],
"id": "3ac8fc4a-a0ef-47b9-bd80-10801eebe13e"
},
{
"cell_type": "markdown",
"metadata": {
"id": "3402ecc9-28d6-4226-99b1-147a2d23b7a0"
},
"source": [
"This example demonstrates the use of the RunInference transform with the pre-trained [ST5 text encoder model](https://tfhub.dev/google/sentence-t5/st5-base/1) from TensorFlow Hub. The transform runs locally using the [Interactive Runner](https://beam.apache.org/releases/pydoc/2.11.0/apache_beam.runners.interactive.interactive_runner.html)."
],
"id": "3ac8fc4a-a0ef-47b9-bd80-10801eebe13e"
Expand Down

0 comments on commit 12f1072

Please sign in to comment.