Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create stub platform test action #10

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Test Platform

on:
workflow_dispatch

env:
GH_TOKEN: ${{secrets.PLATFORM_TESTER_GITHUB_TOKEN}}

jobs:
test:
name: Test Platform
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Create test repository
run: |
REPO_ID=${{ env.GITHUB_RUN_NUMBER }} make test-repo
- name: Delete test repository
run: |
REPO_ID=${{ env.GITHUB_RUN_NUMBER }} make delete-test-repo
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.envrc
.env
6 changes: 6 additions & 0 deletions .markdownlint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Allow creation of collapsable sections in Markdown
# (See https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-collapsed-sections)
MD033:
allowed_elements:
- summary
- details
23 changes: 23 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
.PHONY = \
test-repo \
delete-test-repo

TEST_REPO_NAME=$(REPO_ID)

# From
# https://stackoverflow.com/questions/10858261/how-to-abort-makefile-if-variable-not-set
check_defined = \
$(strip $(foreach 1,$1, \
$(call __check_defined,$1,$(strip $(value 2)))))
__check_defined = \
$(if $(value $1),, \
$(error Undefined $1$(if $2, ($2))$(if $(value @), \
required by target `$@')))

test-repo:
@:$(call check_defined, REPO_ID, number to append to test repo name)
gh repo create "navapbc/platform-test-$(REPO_ID)" --add-readme --public

delete-test-repo:
@:$(call check_defined, REPO_ID, number to append to test repo name)
gh repo delete "navapbc/platform-test-$(REPO_ID)" --confirm
85 changes: 85 additions & 0 deletions docs/decisions/0002-automate-testing-of-platform.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Approach for Automated Testing of the Platform

* Status: proposed
* Deciders: [list everyone involved in the decision] <!-- optional -->
* Date: [YYYY-MM-DD when the decision was last updated] <!-- optional -->

Technical Story: Spike: Automated testing for platform [[#6](https://github.com/navapbc/platform/issues/6)

## Context and Problem Statement

Testing the platform currently requires multiple time-consuming steps, involving
creating a new repo, copying over template code, creating infrastructure from
template code, and verifying that CI and CD works. Furthermore, splitting up the
platform into separate template-infra and template-application-* repositories
increases the importance of testing the pieces together in an integrated
fashion. In order to sustainably maintain development of the platform, we need a
way to test the platform in an automated fashion.

## Proposal

### Overview

* Have a CI GitHub Action in `platform` repo that runs regularly (e.g. weekly or daily)
* The `platform` CI will:
1. Create a test repo `platform-test-[WORKFLOW_RUN_ID]`
2. Install the `template-infra` into the test repo
3. Install `template-application-nextjs` into the test repo
4. Run through the steps to set up the terraform backend
5. Spin up the application's infrastructure
6. Push up a test commit
7. Watch the GitHub action workflow runs for CI and CD
8. Check that CI and CD both complete successfully
9. Hit the deployed application's health check endpoint
10. Destroy the infrastructure
11. Delete the test repo

### Details

TBD

## Decision Outcome

TBD

### Positive Consequences

TBD

### Negative Consequences

TBD

## Pros and Cons of the Options <!-- optional -->

### [option 1]

[example | description | pointer to more information | …] <!-- optional -->

* Good, because [argument a]
* Good, because [argument b]
* Bad, because [argument c]
* … <!-- numbers of pros and cons can vary -->

### [option 2]

[example | description | pointer to more information | …] <!-- optional -->

* Good, because [argument a]
* Good, because [argument b]
* Bad, because [argument c]
* … <!-- numbers of pros and cons can vary -->

### [option 3]

[example | description | pointer to more information | …] <!-- optional -->

* Good, because [argument a]
* Good, because [argument b]
* Bad, because [argument c]
* … <!-- numbers of pros and cons can vary -->

## Links <!-- optional -->

* [Link type] [Link to ADR] <!-- example: Refined by [ADR-0005](0005-example.md) -->
* … <!-- numbers of links can vary -->
41 changes: 41 additions & 0 deletions docs/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#

## Set up tests in GitHub Actions

Set up `PLATFORM_TESTER_GITHUB_TOKEN` as secret. The token needs to have
`repo:public_repo` and `delete_repo` scopes.

## Run tests in GitHub Actions

Run the "Test Platform" workflow. `GH_TOKEN` needs to be set up as an
environment secret and it needs to be unexpired.

## Setup tests locally

Copy `template.envrc` to `.envrc`

```bash
cp template.envrc .envrc
```

Create a [GitHub token](https://github.com/settings/tokens) with
`repo:public_repo` and `delete_repo` access and update `GH_TOKEN` in `.envrc`
to that token value.

## Run tests locally

Run `make test-repo` with a unique `REPO_ID` (e.g. use your username and/or a
unique number). For example:

```bash
REPO_ID=lorenyu-001 make test-repo
```

## Clean up after running tests

Run `make delete-test-repo` with the same `REPO_ID` that you used to create
the repo to delete the repo.

```bash
REPO_ID=lorenyu-001 make delete-test-repo
```
1 change: 1 addition & 0 deletions template.envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export GH_TOKEN=[token with repo:public_repo and delete_repo scopes]