Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit tests randomly fail to connect to ephemeral Prefect API server #16397

Open
dgarros opened this issue Dec 16, 2024 · 1 comment
Open

Unit tests randomly fail to connect to ephemeral Prefect API server #16397

dgarros opened this issue Dec 16, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@dgarros
Copy link

dgarros commented Dec 16, 2024

Bug summary

After upgrading to prefect 3.1.5 from 3.0.3, we have noticed that the ephemeral server is failing to start fairly often in our CI (1 out of 3 or 5 times on average)

When the issue happen, all tests fail with the following error message.

RuntimeError: Timed out while attempting to connect to ephemeral Prefect API server.

Here is how the fixture is defined

@pytest.fixture(scope="session")
def prefect_test_fixture():
    with prefect_test_harness():
        yield

The issue happens often in CI but it's been hard to reproduce it locally, not sure if this is related but in our CI we are using pytest-xdist to run

here is an example of a failed run in Github Action
https://github.com/opsmill/infrahub/actions/runs/12347503261/job/34454717567

The same job test was working reliably in 3.0.3

Version info

Version:             3.1.5
Python version:      3.12.7

List of pytest plugins we are using

pytest                                   7.4.4           pytest: simple pow...
pytest-asyncio                           0.21.2          Pytest support for...
pytest-benchmark                         4.0.0           A ``pytest`` fixtu...
pytest-clarity                           1.0.1           A plugin providing...
pytest-codspeed                          2.2.1           Pytest plugin to c...
pytest-cov                               4.1.0           Pytest plugin for ...
pytest-env                               1.1.3           pytest plugin that...
pytest-httpx                             0.30.0          Send responses to ...
pytest-timeout                           2.3.1           pytest plugin to a...
pytest-xdist                             3.4.0           pytest xdist plugi...
@dgarros dgarros added the bug Something isn't working label Dec 16, 2024
@dgarros
Copy link
Author

dgarros commented Dec 18, 2024

For now we've reverted to 3.0.11 where the issue is not present

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant