[Epic] Standardize on provider-ci for all providers #1101
Comments
|
Why does the title say Standardize on CI-mgmt for all providers and the overview says "Bridged providers currently display unusable or misleading results in pulumi preview." ? |
|
Probably because I failed a copy-paste! thanks for the catch! |
|
Looks like this probably supersedes: Most of that issue is complete at this point. The only dangling issues are: |
danielrbradley
changed the title
|
This week we pulled most of the differences in the generic template into the bridged one #1140. In particular sharding and some workflow consolidation that makes subsequent changes easier. We've also cleaned up a bunch of instances where providers had been using custom workflows:
In each case it's been relatively straightforward to translate the GH action into some additional test logic. This is not only simpler but it also moves us in a direction where things are more easily reproducible locally. Remaining I think we're actually much closer to #936 now. There's still probably some need for the generic template, but there's definitely a path towards consolidating. As a next step we need to rebase the generic template on top of all the recent work. We've onboarding the pulumiservice provider and can revisit EKS. I'd also like to look into getting k8s onto it -- that has a number of exceptions (for example it spins up its own k8s cluster) which will need codifying, but I don't think it will be too bad. |
) In pulumi/pulumi-aws-native#1919 I want to run an actual end-2-end test with `pulumitest` in AWS Native, which requires AWS credentials. In understand that a larger consolidation may be coming with #1101, but this change would allow me to land my test ahead of that. Please let me know if there is a better way of achieving the same outcome.
We have a problem right now where automated dependency upgrades can change our generated code, and this causes tests to fail because we expect no changes in CI. The "right" way to do this is with a hermetic `make renovate` step which can re-generate this code during the Renovate job, as @t0yv0 has started doing for EKS (e.g. pulumi/pulumi-eks#1552). But we don't have hermetic builds so we don't yet have an easy way to get this to work across all providers. This PR implements a suggestion from Daniel to help address the long tail. Essentially right before we fail a build for containing SDK changes, we will commit those changes and push them back to the PR -- _but only if it came from Renovate._ The build will still fail, but tests will be re-tried against the updated SDK. Renovate has already been configured to "ignore" updates from pulumi-bot (pulumi/renovate-config@82931a6), so if tests pass it will still automatically squash and merge. Here's an example where I manually modified the SDK and the PR was automatically corrected:  Refs #1101 Refs #936
Overview
Only bridged providers benefit from the consistency and automatic rollout of workflows generated from provider-ci today. This creates a lot of work when we have to make a change to all providers. Native provider ci also helps automate changes for some non-bridged providers but has also varied in structure to provider-ci. We'd like to drive greater consistency for across the whole fleet for things like package publish to gain the benefits of investment here across the whole fleet.
Key KPIs
Key Stakeholders
Key Deliverables
References 📔
The text was updated successfully, but these errors were encountered: