Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clean up build matrix with tox-gh-actions #59

Merged
merged 4 commits into from
Oct 13, 2023
Merged

clean up build matrix with tox-gh-actions #59

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
374ad73
clean up build matrix with tox-gh-actions
chris48s Sep 21, 2023
472a73c
fixup
chris48s Sep 21, 2023
9e60b7d
Merge branch 'main' into 55-tox-gha
chris48s Sep 23, 2023
e3b9175
make ci requirements and extras group
chris48s Sep 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 9 additions & 34 deletions {{ cookiecutter.__project_name_kebab }}/.github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,52 +36,28 @@ jobs:
strategy:
matrix:
python: ['3.8', '3.9', '3.10', '3.11']
django: ['3.2', '4.0', '4.1', '4.2']
wagtail: ['4.1', '4.2', '5.0', '5.1']
exclude:
- django: '4.0'
wagtail: '5.0'
- django: '4.0'
wagtail: '5.1'
- django: '4.2'
wagtail: '4.1'
- django: '4.2'
wagtail: '4.2'

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python }}
- name: Install Tox
- name: Install
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install --upgrade tox
python -m pip install .[ci]
- name: Test
run: |
tox
run: tox
env:
TOXENV: python${{ matrix.python }}-django${{ matrix.django }}-wagtail${{ matrix.wagtail }}-sqlite
DB: sqlite
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I learned the hard way that this instructs tox-gh-actions to take over. Needs to be in matrix as db: ['sqlite'] (and db: ['postgres'] accordingly)

Copy link
Contributor Author

@chris48s chris48s Oct 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Can you explain this? It has been a couple of weeks since I looked at this but from memory, setting this and adding

[gh-actions:env]
DB =
    sqlite: sqlite
    postgres: postgres

in tox.ini does what we want here: https://github.com/ymyzk/tox-gh-actions#factor-conditional-settings-environment-variable

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

torchbox/wagtail-grapple#368 and torchbox/wagtail-grapple#372 can explain it a tad better.

revisiting this (well, the tox-gh-actions readme), the "fix" is to leave the env line as is, but also include the db declaration in matrix because without it, toh-gh-actions seems to only run a subset of the tests rather than all for the matrix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I'll circle back to this one and re-test when I get a chance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so I just generated https://github.com/chris48s/wagtail-tox-gha-test-20231012 from this template.
Pushed it to run the CI.
Here's a link to the build: https://github.com/chris48s/wagtail-tox-gha-test-20231012/actions/runs/6499801023
Here is what tests ran:

test-sqlite (3.8)

  python3.8-django3.2-wagtail4.1-sqlite: OK (27.99=setup[25.89]+cmd[2.09] seconds)
  python3.8-django3.2-wagtail4.2-sqlite: OK (27.42=setup[25.56]+cmd[1.87] seconds)
  python3.8-django4.0-wagtail4.1-sqlite: OK (27.21=setup[25.36]+cmd[1.85] seconds)
  python3.8-django4.0-wagtail4.2-sqlite: OK (26.47=setup[24.62]+cmd[1.84] seconds)
  python3.8-django4.1-wagtail4.1-sqlite: OK (27.14=setup[25.29]+cmd[1.85] seconds)
  python3.8-django4.1-wagtail4.2-sqlite: OK (26.79=setup[24.98]+cmd[1.81] seconds)
  python3.8-django3.2-wagtail5.0-sqlite: OK (27.43=setup[25.52]+cmd[1.91] seconds)
  python3.8-django3.2-wagtail5.1-sqlite: OK (27.47=setup[25.62]+cmd[1.85] seconds)
  python3.8-django3.2-wagtailmain-sqlite: OK (36.20=setup[34.36]+cmd[1.84] seconds)
  python3.8-django4.1-wagtail5.0-sqlite: OK (27.01=setup[25.10]+cmd[1.91] seconds)
  python3.8-django4.1-wagtail5.1-sqlite: OK (27.55=setup[25.65]+cmd[1.90] seconds)
  python3.8-django4.1-wagtailmain-sqlite: OK (36.47=setup[34.57]+cmd[1.90] seconds)
  python3.8-django4.2-wagtail5.0-sqlite: OK (27.26=setup[25.31]+cmd[1.95] seconds)
  python3.8-django4.2-wagtail5.1-sqlite: OK (27.70=setup[25.78]+cmd[1.91] seconds)
  python3.8-django4.2-wagtailmain-sqlite: OK (36.24=setup[34.34]+cmd[1.90] seconds)
  congratulations :) (436.52 seconds)

test-sqlite (3.9)

  python3.9-django3.2-wagtail4.1-sqlite: OK (34.89=setup[31.80]+cmd[3.10] seconds)
  python3.9-django3.2-wagtail4.2-sqlite: OK (33.33=setup[31.07]+cmd[2.25] seconds)
  python3.9-django4.0-wagtail4.1-sqlite: OK (33.75=setup[31.42]+cmd[2.33] seconds)
  python3.9-django4.0-wagtail4.2-sqlite: OK (32.83=setup[30.53]+cmd[2.30] seconds)
  python3.9-django4.1-wagtail4.1-sqlite: OK (33.23=setup[30.94]+cmd[2.29] seconds)
  python3.9-django4.1-wagtail4.2-sqlite: OK (32.99=setup[30.69]+cmd[2.29] seconds)
  python3.9-django3.2-wagtail5.0-sqlite: OK (33.39=setup[31.12]+cmd[2.26] seconds)
  python3.9-django3.2-wagtail5.1-sqlite: OK (34.15=setup[31.79]+cmd[2.35] seconds)
  python3.9-django3.2-wagtailmain-sqlite: OK (46.29=setup[43.92]+cmd[2.37] seconds)
  python3.9-django4.1-wagtail5.0-sqlite: OK (33.73=setup[31.42]+cmd[2.32] seconds)
  python3.9-django4.1-wagtail5.1-sqlite: OK (34.80=setup[32.39]+cmd[2.42] seconds)
  python3.9-django4.1-wagtailmain-sqlite: OK (47.02=setup[44.69]+cmd[2.33] seconds)
  python3.9-django4.2-wagtail5.0-sqlite: OK (32.85=setup[30.55]+cmd[2.29] seconds)
  python3.9-django4.2-wagtail5.1-sqlite: OK (34.02=setup[31.72]+cmd[2.30] seconds)
  python3.9-django4.2-wagtailmain-sqlite: OK (45.14=setup[42.67]+cmd[2.47] seconds)
  congratulations :) (542.64 seconds)

test-sqlite (3.10)

  python3.10-django3.2-wagtail4.1-sqlite: OK (27.33=setup[25.16]+cmd[2.17] seconds)
  python3.10-django3.2-wagtail4.2-sqlite: OK (26.22=setup[24.50]+cmd[1.72] seconds)
  python3.10-django4.0-wagtail4.1-sqlite: OK (26.16=setup[24.45]+cmd[1.71] seconds)
  python3.10-django4.0-wagtail4.2-sqlite: OK (25.57=setup[23.86]+cmd[1.71] seconds)
  python3.10-django4.1-wagtail4.1-sqlite: OK (25.95=setup[24.23]+cmd[1.71] seconds)
  python3.10-django4.1-wagtail4.2-sqlite: OK (25.67=setup[23.94]+cmd[1.72] seconds)
  python3.10-django3.2-wagtail5.0-sqlite: OK (26.21=setup[24.49]+cmd[1.72] seconds)
  python3.10-django3.2-wagtail5.1-sqlite: OK (26.43=setup[24.70]+cmd[1.74] seconds)
  python3.10-django3.2-wagtailmain-sqlite: OK (35.94=setup[34.15]+cmd[1.78] seconds)
  python3.10-django4.1-wagtail5.0-sqlite: OK (25.56=setup[23.81]+cmd[1.75] seconds)
  python3.10-django4.1-wagtail5.1-sqlite: OK (26.08=setup[24.29]+cmd[1.79] seconds)
  python3.10-django4.1-wagtailmain-sqlite: OK (36.03=setup[34.24]+cmd[1.79] seconds)
  python3.10-django4.2-wagtail5.0-sqlite: OK (25.82=setup[24.07]+cmd[1.75] seconds)
  python3.10-django4.2-wagtail5.1-sqlite: OK (26.42=setup[24.65]+cmd[1.78] seconds)
  python3.10-django4.2-wagtailmain-sqlite: OK (36.05=setup[34.27]+cmd[1.78] seconds)
  congratulations :) (421.71 seconds)

test-sqlite (3.11)

  python3.11-django3.2-wagtail4.1-sqlite: OK (31.44=setup[28.45]+cmd[2.99] seconds)
  python3.11-django3.2-wagtail4.2-sqlite: OK (29.79=setup[27.80]+cmd[1.99] seconds)
  python3.11-django4.0-wagtail4.1-sqlite: OK (29.59=setup[27.53]+cmd[2.06] seconds)
  python3.11-django4.0-wagtail4.2-sqlite: OK (29.57=setup[27.48]+cmd[2.09] seconds)
  python3.11-django4.1-wagtail4.1-sqlite: OK (29.44=setup[27.45]+cmd[1.99] seconds)
  python3.11-django4.1-wagtail4.2-sqlite: OK (28.98=setup[26.99]+cmd[1.99] seconds)
  python3.11-django3.2-wagtail5.0-sqlite: OK (29.36=setup[27.36]+cmd[2.00] seconds)
  python3.11-django3.2-wagtail5.1-sqlite: OK (30.12=setup[28.05]+cmd[2.07] seconds)
  python3.11-django3.2-wagtailmain-sqlite: OK (41.03=setup[38.92]+cmd[2.11] seconds)
  python3.11-django4.1-wagtail5.0-sqlite: OK (29.53=setup[27.49]+cmd[2.04] seconds)
  python3.11-django4.1-wagtail5.1-sqlite: OK (30.41=setup[28.26]+cmd[2.15] seconds)
  python3.11-django4.1-wagtailmain-sqlite: OK (42.57=setup[40.50]+cmd[2.07] seconds)
  python3.11-django4.2-wagtail5.0-sqlite: OK (29.45=setup[27.33]+cmd[2.11] seconds)
  python3.11-django4.2-wagtail5.1-sqlite: OK (30.31=setup[28.20]+cmd[2.11] seconds)
  python3.11-django4.2-wagtailmain-sqlite: OK (40.27=setup[38.00]+cmd[2.27] seconds)
  congratulations :) (481.99 seconds)

test-postgres (3.8)

  python3.8-django3.2-wagtail4.1-postgres: OK (39.37=setup[36.12]+cmd[3.25] seconds)
  python3.8-django3.2-wagtail4.2-postgres: OK (27.83=setup[25.94]+cmd[1.89] seconds)
  python3.8-django4.0-wagtail4.1-postgres: OK (27.95=setup[26.04]+cmd[1.91] seconds)
  python3.8-django4.0-wagtail4.2-postgres: OK (27.22=setup[25.36]+cmd[1.86] seconds)
  python3.8-django4.1-wagtail4.1-postgres: OK (27.82=setup[25.83]+cmd[1.99] seconds)
  python3.8-django4.1-wagtail4.2-postgres: OK (27.17=setup[25.29]+cmd[1.88] seconds)
  python3.8-django3.2-wagtail5.0-postgres: OK (27.56=setup[25.73]+cmd[1.83] seconds)
  python3.8-django3.2-wagtail5.1-postgres: OK (27.50=setup[25.65]+cmd[1.85] seconds)
  python3.8-django3.2-wagtailmain-postgres: OK (36.34=setup[34.43]+cmd[1.91] seconds)
  python3.8-django4.1-wagtail5.0-postgres: OK (27.35=setup[25.40]+cmd[1.95] seconds)
  python3.8-django4.1-wagtail5.1-postgres: OK (28.11=setup[26.15]+cmd[1.96] seconds)
  python3.8-django4.1-wagtailmain-postgres: OK (37.03=setup[35.07]+cmd[1.96] seconds)
  python3.8-django4.2-wagtail5.0-postgres: OK (27.53=setup[25.58]+cmd[1.95] seconds)
  python3.8-django4.2-wagtail5.1-postgres: OK (28.13=setup[26.13]+cmd[2.00] seconds)
  python3.8-django4.2-wagtailmain-postgres: OK (37.61=setup[35.54]+cmd[2.07] seconds)

test-postgres (3.9)

  python3.9-django3.2-wagtail4.1-postgres: OK (36.21=setup[34.09]+cmd[2.12] seconds)
  python3.9-django3.2-wagtail4.2-postgres: OK (26.85=setup[25.02]+cmd[1.82] seconds)
  python3.9-django4.0-wagtail4.1-postgres: OK (27.25=setup[25.39]+cmd[1.86] seconds)
  python3.9-django4.0-wagtail4.2-postgres: OK (26.63=setup[24.83]+cmd[1.80] seconds)
  python3.9-django4.1-wagtail4.1-postgres: OK (27.02=setup[25.17]+cmd[1.85] seconds)
  python3.9-django4.1-wagtail4.2-postgres: OK (26.81=setup[24.88]+cmd[1.93] seconds)
  python3.9-django3.2-wagtail5.0-postgres: OK (26.96=setup[25.11]+cmd[1.85] seconds)
  python3.9-django3.2-wagtail5.1-postgres: OK (27.47=setup[25.58]+cmd[1.88] seconds)
  python3.9-django3.2-wagtailmain-postgres: OK (38.39=setup[36.53]+cmd[1.85] seconds)
  python3.9-django4.1-wagtail5.0-postgres: OK (26.79=setup[24.92]+cmd[1.87] seconds)
  python3.9-django4.1-wagtail5.1-postgres: OK (27.81=setup[25.94]+cmd[1.87] seconds)
  python3.9-django4.1-wagtailmain-postgres: OK (36.34=setup[34.45]+cmd[1.89] seconds)
  python3.9-django4.2-wagtail5.0-postgres: OK (26.94=setup[25.05]+cmd[1.89] seconds)
  python3.9-django4.2-wagtail5.1-postgres: OK (27.53=setup[25.62]+cmd[1.90] seconds)
  python3.9-django4.2-wagtailmain-postgres: OK (36.17=setup[34.25]+cmd[1.92] seconds)
  congratulations :) (445.31 seconds)

test-postgres (3.10)

  python3.10-django3.2-wagtail4.1-postgres: OK (47.51=setup[44.48]+cmd[3.03] seconds)
  python3.10-django3.2-wagtail4.2-postgres: OK (35.46=setup[33.19]+cmd[2.27] seconds)
  python3.10-django4.0-wagtail4.1-postgres: OK (35.75=setup[33.29]+cmd[2.46] seconds)
  python3.10-django4.0-wagtail4.2-postgres: OK (34.35=setup[32.04]+cmd[2.31] seconds)
  python3.10-django4.1-wagtail4.1-postgres: OK (34.49=setup[32.25]+cmd[2.24] seconds)
  python3.10-django4.1-wagtail4.2-postgres: OK (34.55=setup[32.07]+cmd[2.49] seconds)
  python3.10-django3.2-wagtail5.0-postgres: OK (34.90=setup[32.51]+cmd[2.39] seconds)
  python3.10-django3.2-wagtail5.1-postgres: OK (35.11=setup[32.54]+cmd[2.57] seconds)
  python3.10-django3.2-wagtailmain-postgres: OK (46.82=setup[44.52]+cmd[2.30] seconds)
  python3.10-django4.1-wagtail5.0-postgres: OK (33.53=setup[31.10]+cmd[2.43] seconds)
  python3.10-django4.1-wagtail5.1-postgres: OK (35.39=setup[32.88]+cmd[2.51] seconds)
  python3.10-django4.1-wagtailmain-postgres: OK (47.80=setup[45.39]+cmd[2.42] seconds)
  python3.10-django4.2-wagtail5.0-postgres: OK (34.22=setup[31.92]+cmd[2.30] seconds)
  python3.10-django4.2-wagtail5.1-postgres: OK (35.27=setup[32.78]+cmd[2.49] seconds)
  python3.10-django4.2-wagtailmain-postgres: OK (47.05=setup[44.66]+cmd[2.39] seconds)
  congratulations :) (572.43 seconds)

test-postgres (3.11)

  python3.11-django3.2-wagtail4.1-postgres: OK (34.27=setup[32.48]+cmd[1.78] seconds)
  python3.11-django3.2-wagtail4.2-postgres: OK (25.36=setup[23.75]+cmd[1.61] seconds)
  python3.11-django4.0-wagtail4.1-postgres: OK (24.98=setup[23.36]+cmd[1.62] seconds)
  python3.11-django4.0-wagtail4.2-postgres: OK (24.56=setup[22.92]+cmd[1.64] seconds)
  python3.11-django4.1-wagtail4.1-postgres: OK (25.11=setup[23.49]+cmd[1.62] seconds)
  python3.11-django4.1-wagtail4.2-postgres: OK (24.60=setup[23.00]+cmd[1.60] seconds)
  python3.11-django3.2-wagtail5.0-postgres: OK (25.56=setup[23.93]+cmd[1.63] seconds)
  python3.11-django3.2-wagtail5.1-postgres: OK (25.39=setup[23.74]+cmd[1.65] seconds)
  python3.11-django3.2-wagtailmain-postgres: OK (34.77=setup[33.11]+cmd[1.65] seconds)
  python3.11-django4.1-wagtail5.0-postgres: OK (24.81=setup[23.16]+cmd[1.65] seconds)
  python3.11-django4.1-wagtail5.1-postgres: OK (25.39=setup[23.71]+cmd[1.69] seconds)
  python3.11-django4.1-wagtailmain-postgres: OK (34.81=setup[33.12]+cmd[1.69] seconds)
  python3.11-django4.2-wagtail5.0-postgres: OK (25.20=setup[23.53]+cmd[1.67] seconds)
  python3.11-django4.2-wagtail5.1-postgres: OK (25.46=setup[23.78]+cmd[1.68] seconds)
  python3.11-django4.2-wagtailmain-postgres: OK (35.01=setup[33.33]+cmd[1.68] seconds)
  congratulations :) (415.35 seconds)

To me, that looks like all the tests we expect to run were excuted.

What am I missing?

Copy link
Collaborator

@zerolab zerolab Oct 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure, to be honest. Let's leave it as is as your additional due dilligence proves this works.
It was probably PEBCAK on my side, so I'm sorry for the time waste


test-postgres:
runs-on: ubuntu-latest
needs: lint
strategy:
matrix:
python: ['3.8', '3.9', '3.10', '3.11']
django: ['3.2', '4.0', '4.1', '4.2']
wagtail: ['4.1', '4.2', '5.0', '5.1']
experimental: [false]
exclude:
- django: '4.0'
wagtail: '5.0'
- django: '4.0'
wagtail: '5.1'
- django: '4.2'
wagtail: '4.1'
- django: '4.2'
wagtail: '4.2'

services:
postgres:
Expand All @@ -98,13 +74,12 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python }}
- name: Install Tox
- name: Install
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install --upgrade tox
python -m pip install .[ci]
- name: Test
run: |
tox
run: tox
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/{% endraw %}{{ cookiecutter.__project_name_snake }}{% raw %}
TOXENV: python${{ matrix.python }}-django${{ matrix.django }}-wagtail${{ matrix.wagtail }}-postgres{% endraw %}
DATABASE_URL: postgres://postgres:postgres@localhost:5432/{% endraw %}{{ cookiecutter.__project_name_snake }}
DB: postgres
4 changes: 4 additions & 0 deletions {{ cookiecutter.__project_name_kebab }}/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ testing = [
"dj-database-url==2.1.0",
"pre-commit==3.4.0"
]
ci = [
"tox==4.11.3",
"tox-gh-actions==3.1.3"
]

[project.urls]
Home = "https://github.com/{{ cookiecutter.github_username }}/{{ cookiecutter.__project_name_kebab }}"
Expand Down
12 changes: 12 additions & 0 deletions {{ cookiecutter.__project_name_kebab }}/tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ envlist =
python{3.8,3.9,3.10,3.11}-django{3.2,4.0,4.1}-wagtail{4.1,4.2}-{sqlite,postgres}
python{3.8,3.9,3.10,3.11}-django{3.2,4.1,4.2}-wagtail{5.0,5.1,main}-{sqlite,postgres}

[gh-actions]
python =
3.8: python3.8
3.9: python3.9
3.10: python3.10
3.11: python3.11

[gh-actions:env]
DB =
sqlite: sqlite
postgres: postgres

[testenv]
install_command = pip install -e ".[testing]" -U {opts} {packages}
commands = coverage run testmanage.py test --deprecation all {posargs: -v 2}
Expand Down