Skip to content

Commit

Permalink
[ci/cd] Split Test pipeline according to runtime-benchmarks feature (
Browse files Browse the repository at this point in the history
…#48)

## Summary

Based on original [issue](paritytech/ci_cd#864
) from @gilescope related to the monorepo:

> Currently the tests have started to be run with the
`runtime-benchmarks` feature enabled ( E.g.
https://github.com/paritytech/polkadot-sdk/blob/aabed6757e26e21ce06dd59946a17b72f349185e/.gitlab/pipeline/test.yml#L37
). This was done so that the autogenerated tests that the benchmark
macros create get run (but it seems that we already have a
short-benchmark CI job... - do the additional tests give us anything
extra? EDIT: the short-benchmarks only test 3 relay chains so not
comprehensive enough).

> However, this is inherently dangerous as not everything will be in a
production like state when it is being tested.

> For safety's sake we should be running without the
`runtime-benchmarks` feature enabled. Assuming the CI system can take
it, we should set up another job that runs the tests with the
runtime-benchmarks feature enabled. (if we did not do this we would
catch any problems later when the weights are rerun for the runtime
release builds).

## Proposed solution

Split [test
pipeline](https://github.com/polkadot-fellows/runtimes/blob/main/.github/workflows/test.yml#L44)
to the two pipelines:
- one without `runtime-benchmarks` feature
- one with `runtime-benchmarks` feature

Co-authored-by: Bastian Köcher <[email protected]>
  • Loading branch information
bkontur and bkchr authored Dec 7, 2023
1 parent b8df4b0 commit 2b30243
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,14 @@ jobs:
- name: Fetch cache
uses: Swatinem/rust-cache@a95ba195448af2da9b00fb742d14ffaaf3c21f43 # v2.7.0

- name: Test
run: cargo test --workspace --release --locked -q --features=runtime-metrics,try-runtime
env:
RUSTFLAGS: "-C debug-assertions -D warnings"
SKIP_WASM_BUILD: 1

- name: Test all features
run: cargo test --workspace --release --locked -q --features=runtime-benchmarks,runtime-metrics,try-runtime
env:
RUSTFLAGS: "-C debug-assertions -D warnings"
SKIP_WASM_BUILD: 1
SKIP_WASM_BUILD: 1

0 comments on commit 2b30243

Please sign in to comment.