Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up Java Tests GitHub workflow #33354

Merged
merged 3 commits into from
Dec 13, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 2 additions & 70 deletions .github/workflows/java_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,7 @@
name: Java Tests
on:
workflow_dispatch:
inputs:
runDataflow:
description: 'Type "true" if you want to run Dataflow tests (GCP variables must be configured, check CI.md)'
default: 'false'
required: false

schedule:
- cron: '10 2 * * *'
push:
Expand All @@ -33,8 +29,7 @@ on:
pull_request:
branches: ['master', 'release-*']
tags: ['v*']
paths: ['sdks/java/**', 'model/**', 'runners/**', 'examples/java/**',
'examples/kotlin/**', 'release/**', 'buildSrc/**']
paths: ['sdks/java/**', 'model/**', 'runners/**', 'examples/java/**', 'examples/kotlin/**', 'release/**', 'buildSrc/**']
# This allows a subsequently queued workflow run to interrupt previous runs
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.issue.number || github.event.pull_request.head.label || github.sha || github.head_ref || github.ref }}-${{ github.event.schedule || github.event.comment.id || github.event.sender.login}}'
Expand All @@ -44,26 +39,6 @@ env:
GRADLE_ENTERPRISE_CACHE_USERNAME: ${{ secrets.GE_CACHE_USERNAME }}
GRADLE_ENTERPRISE_CACHE_PASSWORD: ${{ secrets.GE_CACHE_PASSWORD }}
jobs:
check_gcp_variables:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this PR proposes removal of the Dataflow WordCount Job, we do not need this check_gcp_variables Job.

timeout-minutes: 5
name: "Check GCP variables set"
runs-on: [self-hosted, ubuntu-20.04, main]
outputs:
gcp-variables-set: ${{ steps.check_gcp_variables.outputs.gcp-variables-set }}
steps:
- name: Check out code
uses: actions/checkout@v4
- name: "Check are GCP variables set"
run: "./scripts/ci/ci_check_are_gcp_variables_set.sh"
id: check_gcp_variables
env:
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
GCP_SA_EMAIL: ${{ secrets.GCP_SA_EMAIL }}
GCP_SA_KEY: ${{ secrets.GCP_SA_KEY }}
GCP_TESTING_BUCKET: ${{ secrets.GCP_TESTING_BUCKET }}
GCP_REGION: "not-needed-here"
GCP_PYTHON_WHEELS_BUCKET: "not-needed-here"

java_unit_tests:
name: 'Java Unit Tests'
runs-on: ${{ matrix.os }}
Expand Down Expand Up @@ -152,46 +127,3 @@ jobs:
with:
name: java_wordcount_direct_runner-${{matrix.os}}
path: examples/java/build/reports/tests/integrationTest

java_wordcount_dataflow:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have GitHub workflow that test Java WordCount on Dataflow. It was skipped for so long and seems just another thing to maintain in our codebase.

name: 'Java Wordcount Dataflow'
needs:
- check_gcp_variables
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [[self-hosted, ubuntu-20.04, main], windows-latest]
# TODO(https://github.com/apache/beam/issues/31848) run on Dataflow after fixes credential on macOS/win GHA runner
if: |
needs.check_gcp_variables.outputs.gcp-variables-set == 'true' &&
(github.event_name == 'workflow_dispatch' && github.event.inputs.runDataflow == 'true')
steps:
- name: Check out code
uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Setup environment
uses: ./.github/actions/setup-environment-action
with:
java-version: 11
go-version: default
- name: Authenticate on GCP
uses: google-github-actions/auth@v1
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
project_id: ${{ secrets.GCP_PROJECT_ID }}
- name: Run WordCount
uses: ./.github/actions/gradle-command-self-hosted-action
with:
gradle-command: integrationTest
arguments: -p examples/ --tests org.apache.beam.examples.WordCountIT
-DintegrationTestPipelineOptions=[\"--runner=DataflowRunner\",\"--project=${{ secrets.GCP_PROJECT_ID }}\",\"--tempRoot=gs://${{ secrets.GCP_TESTING_BUCKET }}/tmp/\"]
-DintegrationTestRunner=dataflow
- name: Upload test logs
uses: actions/upload-artifact@v4
if: always()
with:
name: java_wordcount_dataflow-${{matrix.os}}
path: examples/java/build/reports/tests/integrationTest
Loading