Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove integrationTestProvider option #1213

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

blampe
Copy link
Contributor

@blampe blampe commented Dec 11, 2024

This is currently unused, and we don't want to encourage two separate places for integration tests. If a provider includes e2e tests in the provider directory, the recommendation is to consolidate those under examples (or any other path).

@blampe blampe force-pushed the blampe/integration-test-provider branch from 589bbee to f2f1e65 Compare December 11, 2024 21:21
Copy link
Member

@danielrbradley danielrbradley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to delete this if we really don't need it, but I'd question the direction on requiring all e2e tests to be in the examples module.

We've moved some e2e tests into the provider module in Azure Native and Azure Classic because it allows the tests to be much more productive for repro'ing issues. This is because the test is in the same module as the provider code and so when attaching the debugger and using the provider debug attach option, it lets you step right through from the test into the provider code.

That said, to make this work well, we do need to integrate these tests better into our CI flow as they're just just run along with the provider unit tests right now which can slow down the whole pipeline.

@blampe
Copy link
Contributor Author

blampe commented Dec 12, 2024

We've moved some e2e tests into the provider module in Azure Native and Azure Classic because it allows the tests to be much more productive for repro'ing issues. This is because the test is in the same module as the provider code and so when attaching the debugger and using the provider debug attach option, it lets you step right through from the test into the provider code.

Yeah, this is understandable. Was talking about this with Florian this morning here.

The integration tests should absolutely be in the same module as the provider code, and it's our fault for over-complicating things by creating more than one module. The best and simplest fix is to consolidate everything under a root module, which is straightforward -- move provider/go.mod to the root, copy any dependencies from examples/go.mod into it, fix replace statements, and rewrite imports. This doesn't impact SDKs since they already have their own module.

In other words this

.
├── examples
│   ├── go.mod
│   └── go.sum
├── provider
│   ├── go.mod
│   └── go.sum
└── test

becomes

.
├── examples
├── go.mod
├── go.sum
├── provider
└── test

and now it really doesn't matter where your integration tests live.

Here's an example pulumi/pulumi-aws#4919. There a couple things we need to tweak, but it's mostly there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants