Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add check for orphaned PR environments #709

Open
lorenyu opened this issue Jul 25, 2024 · 6 comments · May be fixed by navapbc/platform-test#143
Open

Add check for orphaned PR environments #709

lorenyu opened this issue Jul 25, 2024 · 6 comments · May be fixed by navapbc/platform-test#143
Labels
scope: ci/cd GitHub actions workflows type: bug Something isn't working

Comments

@lorenyu
Copy link
Contributor

lorenyu commented Jul 25, 2024

I notice in platform-test that there are some orphaned PR environments for closed PRs. I don't know if this is the right solution for this issue, but we could add a check for PR environments that map to orphaned PRs.

Also not sure whether to classify this as a bug or an enhancement

@lorenyu
Copy link
Contributor Author

lorenyu commented Jul 25, 2024

@rocketnova @doshitan thoughts on this problem/solution?

@lorenyu lorenyu added type: bug Something isn't working scope: ci/cd GitHub actions workflows labels Jul 25, 2024
@rocketnova
Copy link
Contributor

Hmmm. Here are some initial thoughts:

  1. Maybe we could start with defining some guidelines around how long we want to keep environments open on platform-test? I know I've got a few open PRs on platform-test. Some of them are not exactly "orphaned" because I do want to return to the core idea in them, but I am open to us establishing a guideline of not leaving resources running untouched for more than X days (maybe a week?).
  2. We could run aws-nuke or cloud-nuke on a regular basis to enforce deletion of resources.
  3. If we want to add a check for PR environments that map to orphaned PRs, we would need to establish guidelines on how to map a PR to an environment and define what constitutes an orphaned PR. I'm not opposed to this, but it does seem like more overhead than some of the other ideas we haven't tried yet.

@lorenyu
Copy link
Contributor Author

lorenyu commented Jul 26, 2024

  • I realized most of the orphaned environments were mine so I cleaned those up.
  • I did notice one orphaned workspace t-ptwknr that looks related to the idp work. Can you look into that?

re: 3, I was thinking that an orphaned PR environment is an environment for a PR that's closed. The destroy PR environment script should have destroyed it but it failed for some reason. I noticed a few of those based on my PRs.

@lorenyu
Copy link
Contributor Author

lorenyu commented Jul 26, 2024

I think you're talking about something more complex, which is the concept of a stale PR. I think that's a useful discussion too, but perhaps out of scope for this ticket.

@rocketnova
Copy link
Contributor

Ohh great clarification. How would you map closed PRs to terraform environments?

@lorenyu
Copy link
Contributor Author

lorenyu commented Jul 29, 2024

The PR environment lives in a separate workspace in the service layer with the name p-<PR number>, so I'm thinking this workflow could do something like:

  1. workspaces = terraform -chdir=infra/<APP_NAME>/service workspace list
  2. pr_workspaces = filter workspaces to those matching p-(###)
  3. orphaned_pr_workspaces = for each PR workspace, use gh pr view <PR_NUMBER> --json state | jq --raw-output .state to see if it is "CLOSED" and get only the closed ones
  4. either automatically clean up orphaned_pr_workspaces or notify dev team that these exist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope: ci/cd GitHub actions workflows type: bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants