Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix XVR_Direct time out #25247

Merged
merged 1 commit into from
Feb 1, 2023
Merged

Fix XVR_Direct time out #25247

merged 1 commit into from
Feb 1, 2023

Conversation

Abacn
Copy link
Contributor

@Abacn Abacn commented Feb 1, 2023

follow up of #25243

the test had chance of timeout for a while. This change reduced duplicate task for XVR postcommit tests and bump the timeout for XVR_Direct.

  • Bump XVR_Direct timeout to 2h

  • Only run GoUsingJava test suite once

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI.

@Abacn
Copy link
Contributor Author

Abacn commented Feb 1, 2023

run seed job

* Bump XVR_Direct timeout to 2h

* Only run GoUsingJava test suite once
@Abacn
Copy link
Contributor Author

Abacn commented Feb 1, 2023

Run XVR_Direct PostCommit

@codecov
Copy link

codecov bot commented Feb 1, 2023

Codecov Report

Merging #25247 (59654c7) into master (b2d500f) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master   #25247   +/-   ##
=======================================
  Coverage   72.97%   72.97%           
=======================================
  Files         743      743           
  Lines       99037    99037           
=======================================
+ Hits        72273    72275    +2     
+ Misses      25398    25396    -2     
  Partials     1366     1366           
Flag Coverage Δ
python 82.51% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ython/apache_beam/runners/interactive/sql/utils.py 76.09% <100.00%> (ø)
...eam/runners/portability/fn_api_runner/execution.py 92.49% <0.00%> (-0.64%) ⬇️
sdks/python/apache_beam/runners/direct/executor.py 96.46% <0.00%> (-0.55%) ⬇️
...ks/python/apache_beam/runners/worker/sdk_worker.py 89.24% <0.00%> (ø)
...hon/apache_beam/runners/direct/test_stream_impl.py 94.02% <0.00%> (+0.74%) ⬆️
...ks/python/apache_beam/runners/worker/data_plane.py 89.26% <0.00%> (+1.69%) ⬆️
sdks/python/apache_beam/utils/interactive_utils.py 97.56% <0.00%> (+2.43%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@github-actions
Copy link
Contributor

github-actions bot commented Feb 1, 2023

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @damccorm for label python.
R: @damccorm for label build.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Comment on lines +2542 to +2543
cleanupTask.configure { mustRunAfter goTask }
config.cleanupJobServer.configure { mustRunAfter goTask }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to special case these as well so that they run even if we're not depending on the goTask?

Copy link
Contributor Author

@Abacn Abacn Feb 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, besides being a dependency to validatesCrossLanguageRunner, validatesCrossLanguageRunnerGoUsingJava task can be triggered alone, where we are doing this for Dataflow XVR postcommits. Then clean up tassks should also be configured to run after it.

Copy link
Contributor Author

@Abacn Abacn Feb 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, answered wrong the yes/no question. I mean run after requirements are always configured not relevant to the dependency of validatesCrossLanguageRunnerGoUsingJava -> validatesCrossLanguageRunner, so I consider the answer is no

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see - thanks!

@Abacn
Copy link
Contributor Author

Abacn commented Feb 1, 2023

Run Java PreCommit

@Abacn
Copy link
Contributor Author

Abacn commented Feb 1, 2023

Run CommunityMetrics PreCommit

@Abacn
Copy link
Contributor Author

Abacn commented Feb 1, 2023

Run Java_Kafka_IO_Direct PreCommit

@Abacn
Copy link
Contributor Author

Abacn commented Feb 1, 2023

Run Java_Pulsar_IO_Direct PreCommit

@Abacn
Copy link
Contributor Author

Abacn commented Feb 1, 2023

different test failures in two run

Java_PreCommit

first run:

org.apache.beam.runners.flink.FlinkRequiresStableInputTest.testParDoRequiresStableInputPortable
org.apache.beam.runners.flink.FlinkRequiresStableInputTest.testParDoRequiresStableInputStatefulPortable

second run:

org.apache.beam.runners.fnexecution.ServerFactoryTest.urlFactoryWithPortSupplier
org.apache.beam.runners.fnexecution.ServerFactoryTest.defaultServerWithPortSupplier

Kafka_IO_Direct: org.apache.beam.sdk.io.kafka.KafkaIOIT.testWatermarkUpdateWithSparseMessages known flake, entered #25261

otherwise merging for now

@Abacn Abacn merged commit fde7909 into apache:master Feb 1, 2023
@Abacn Abacn deleted the xvrdirectruntime branch February 1, 2023 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants