Improve features dev-ex #1831

ggwpez · 2023-10-09T19:49:01Z

Adds a config file that allows to run zepter without any arguments in the workspace to address all issues.
A secondary workflow for the CI is provided as zepter run check. Both the formatting and linting are now in one check for efficiancy.

The latest version also detects some more things that featalign was already showing.

Error message in the CI now looks like this:

...
crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml)
  feature 'std'
    must propagate to:
      parity-scale-codec
Found 55 issues (run with --fix to fix).
Error: Command 'lint propagate-feature' failed with exit code 1

Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration.
It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter`.
Otherwise please ask directly in the Merge Request, GitHub Discussions or on Matrix Chat, thank you.

For more information, see:
  - https://github.com/paritytech/polkadot-sdk/issues/1831
  - https://github.com/ggwpez/zepter

TODO:

Check that CI fails correctly

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

bkontur · 2023-10-09T20:46:11Z

Maybe unrelated question, but should dev-dependencies be added to std / try-runtime / runtime-benchmarks features?

E.g. here https://github.com/paritytech/polkadot-sdk/pull/1801/files#diff-57e966c3ef58bb810065f964f9d2d0c5ef141de756082abd9b0f877de98f78c3R25-L34
pallet-balances = { path = "../balances" } and it was unnecessary added to std as pallet-balances/std.
But on the other hand, pallet-balances has to be added to try-runtime as pallet-balances/try-runtime, is that ok?

ggwpez · 2023-10-10T09:35:56Z

Maybe unrelated question, but should dev-dependencies be added to std / try-runtime / runtime-benchmarks features?

I am not really sure how to handle dev and build dependencies. Normally we dont disable the default features for dev dependencies, so we dont need to add them to std. But for runtime-benchmarks and try-runtime it would make sense.

liamaharon

🎉

substrate/primitives/wasm-interface/Cargo.toml

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Seems like a better spot than the .cargo folder. Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Otherwise it wont find the config file. Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez · 2023-10-16T12:13:43Z

IIUC #1870 moved some deps upstream. Now the problem is that the introduced dependency sp-bls12-381 pulls in sp-crypto-ec-utils from the SDK, which in term wants sp-std v11.
Since we did not backport the version bumps, the repo has version 8 and now pulls that stuff in twice - resulting in a build error here in the CI.

bkchr · 2023-10-16T12:50:50Z

Does cargo unify deps which have a path and a version? So, would that be really fixed by merging the version bumps back to master?

Changes: - Add missing crate to the workspace - Remove versions from local dependency links Maybe it is finally worth it to add this scrip to the CI to find these things earlier: [check-deps.py](https://github.com/ggwpez/substrate-scripts/blob/master/import-runtime-repos/check-deps.py). @paritytech/ci what would be the best location for that check? It takes only a second to run, so maybe we can squeeze it into one of the existing checks? Otherwise creating a new GH workflow feels a bit wasteful... maybe i can group it with #1831 --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez · 2023-10-17T16:19:38Z

Does cargo unify deps which have a path and a version? So, would that be really fixed by merging the version bumps back to master?

It does not seem to quite fix it, since we use one GH dependency here which then pulls in duplicate versions again.
I locally removed that dep and it worked, so hopefully it would be fixed if the bandersnatch also uses all the latest versions and gets published.

Changes: - Add missing crate to the workspace - Remove versions from local dependency links Maybe it is finally worth it to add this scrip to the CI to find these things earlier: [check-deps.py](https://github.com/ggwpez/substrate-scripts/blob/master/import-runtime-repos/check-deps.py). @paritytech/ci what would be the best location for that check? It takes only a second to run, so maybe we can squeeze it into one of the existing checks? Otherwise creating a new GH workflow feels a bit wasteful... maybe i can group it with #1831 --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* master: Removed TODO from test-case for hard-coded delivery fee estimation (#2042) Expose collection attributes from `Inspect` trait (#1914) `polkadot-parachain-primitives` should not depend on `frame-support`. (#1897) [testnet] Align testnet system parachain runtimes using `RelayTreasuryLocation` and `SystemParachains` in the same way (#2023) Sort the benchmarks before listing them (#2026) publish pallet-root-testing (#2017) Contracts: Add benchmarks to include files (#2022) Small optimisation to `--profile dev` wasm builds (#1851) basic-authorship: Improve time recording and logging (#2010) Application Crypto and BEEFY Support for paired (ECDSA,BLS) crypto (#1815) [ci] Run check-rust-feature-propagation in pr and master (#2012) Improve features dev-ex (#1831) Remove obsolete comment. (#2008)

* tsv-disabling: Removed TODO from test-case for hard-coded delivery fee estimation (#2042) Expose collection attributes from `Inspect` trait (#1914) `polkadot-parachain-primitives` should not depend on `frame-support`. (#1897) [testnet] Align testnet system parachain runtimes using `RelayTreasuryLocation` and `SystemParachains` in the same way (#2023) Sort the benchmarks before listing them (#2026) publish pallet-root-testing (#2017) Contracts: Add benchmarks to include files (#2022) Small optimisation to `--profile dev` wasm builds (#1851) basic-authorship: Improve time recording and logging (#2010) Application Crypto and BEEFY Support for paired (ECDSA,BLS) crypto (#1815) [ci] Run check-rust-feature-propagation in pr and master (#2012) Improve features dev-ex (#1831) Remove obsolete comment. (#2008)

* tsv-disabling: (36 commits) Removed TODO from test-case for hard-coded delivery fee estimation (#2042) Expose collection attributes from `Inspect` trait (#1914) `polkadot-parachain-primitives` should not depend on `frame-support`. (#1897) [testnet] Align testnet system parachain runtimes using `RelayTreasuryLocation` and `SystemParachains` in the same way (#2023) Sort the benchmarks before listing them (#2026) publish pallet-root-testing (#2017) Contracts: Add benchmarks to include files (#2022) Small optimisation to `--profile dev` wasm builds (#1851) basic-authorship: Improve time recording and logging (#2010) Application Crypto and BEEFY Support for paired (ECDSA,BLS) crypto (#1815) [ci] Run check-rust-feature-propagation in pr and master (#2012) Improve features dev-ex (#1831) Remove obsolete comment. (#2008) Refactor candidates test in paras_inherent (#2004) PVF: Add worker check during tests and benches (#1771) Bump actions/setup-node from 3.8.1 to 4.0.0 (#1997) polkadot: enable tikv-jemallocator/unprefixed_malloc_on_supported_platforms (#2002) Make `IdentityInfo` generic in `pallet-identity` (#1661) Ensure correct variant count in `Runtime[Hold/Freeze]Reason` (#1900) `CheckWeight`: Add more logging (#1996) ...

Adds a config file that allows to run `zepter` without any arguments in the workspace to address all issues. A secondary workflow for the CI is provided as `zepter run check`. Both the formatting and linting are now in one check for efficiancy. The latest version also detects some more things that `featalign` was already showing. Error message [in the CI](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3916205) now looks like this: ```pre ... crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml) feature 'std' must propagate to: parity-scale-codec Found 55 issues (run with --fix to fix). Error: Command 'lint propagate-feature' failed with exit code 1 Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration. It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter`. Otherwise please ask directly in the Merge Request, GitHub Discussions or on Matrix Chat, thank you. For more information, see: - #1831 - https://github.com/ggwpez/zepter ``` TODO: - [x] Check that CI fails correctly --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Adds a config file that allows to run `zepter` without any arguments in the workspace to address all issues. A secondary workflow for the CI is provided as `zepter run check`. Both the formatting and linting are now in one check for efficiancy. The latest version also detects some more things that `featalign` was already showing. Error message [in the CI](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3916205) now looks like this: ```pre ... crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml) feature 'std' must propagate to: parity-scale-codec Found 55 issues (run with --fix to fix). Error: Command 'lint propagate-feature' failed with exit code 1 Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration. It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter`. Otherwise please ask directly in the Merge Request, GitHub Discussions or on Matrix Chat, thank you. For more information, see: - paritytech/polkadot-sdk#1831 - https://github.com/ggwpez/zepter ``` TODO: - [x] Check that CI fails correctly --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Changes: - Add missing crate to the workspace - Remove versions from local dependency links Maybe it is finally worth it to add this scrip to the CI to find these things earlier: [check-deps.py](https://github.com/ggwpez/substrate-scripts/blob/master/import-runtime-repos/check-deps.py). @paritytech/ci what would be the best location for that check? It takes only a second to run, so maybe we can squeeze it into one of the existing checks? Otherwise creating a new GH workflow feels a bit wasteful... maybe i can group it with paritytech#1831 --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Adds a config file that allows to run `zepter` without any arguments in the workspace to address all issues. A secondary workflow for the CI is provided as `zepter run check`. Both the formatting and linting are now in one check for efficiancy. The latest version also detects some more things that `featalign` was already showing. Error message [in the CI](https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/3916205) now looks like this: ```pre ... crate 'test-parachains' (/Users/vados/Documents/work/polkadot-sdk/polkadot/parachain/test-parachains/Cargo.toml) feature 'std' must propagate to: parity-scale-codec Found 55 issues (run with --fix to fix). Error: Command 'lint propagate-feature' failed with exit code 1 Polkadot-SDK uses the Zepter CLI to detect abnormalities in the feature configuration. It looks like one more more checks failed; please check the console output. You can try to automatically address them by running `zepter`. Otherwise please ask directly in the Merge Request, GitHub Discussions or on Matrix Chat, thank you. For more information, see: - paritytech#1831 - https://github.com/ggwpez/zepter ``` TODO: - [x] Check that CI fails correctly --------- Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* use GitLab env vars to get git commit * compile_error to test it * Revert "compile_error to test it" This reverts commit 67d4782298d3cdfbe1a28231042bba6444316e8a.

ggwpez added 4 commits October 9, 2023 21:48

Add zepter config

7b02fa5

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Use valid semver

9415fe1

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Better help

ee7f1ec

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Fix remaining

288cca4

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez added the R0-silent Changes should not be mentioned in any release notes label Oct 9, 2023

ggwpez marked this pull request as ready for review October 9, 2023 20:06

ggwpez requested review from a team October 9, 2023 20:06

ggwpez requested review from acatangiu and jsidorenko as code owners October 9, 2023 20:06

ggwpez requested a review from a team October 9, 2023 20:06

ggwpez requested review from koute and a team as code owners October 9, 2023 20:06

paritytech-ci requested review from a team October 9, 2023 20:08

bkontur approved these changes Oct 9, 2023

View reviewed changes

bkchr approved these changes Oct 10, 2023

View reviewed changes

Merge branch 'master' into oty-zepter-devex

4c1cb4b

sergejparity approved these changes Oct 12, 2023

View reviewed changes

paritytech-ci requested a review from a team October 12, 2023 23:35

sam0x17 approved these changes Oct 13, 2023

View reviewed changes

liamaharon approved these changes Oct 13, 2023

View reviewed changes

Merge branch 'master' into oty-zepter-devex

2ba6f0f

mateo-moon approved these changes Oct 14, 2023

View reviewed changes

bkchr reviewed Oct 14, 2023

View reviewed changes

substrate/primitives/wasm-interface/Cargo.toml Outdated Show resolved Hide resolved

Unify optional dep enable

c1ed484

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez mentioned this pull request Oct 15, 2023

Unify features ggwpez/zepter#39

Open

ggwpez added 2 commits October 15, 2023 19:27

Move config file to .config

27589a5

Seems like a better spot than the .cargo folder. Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Use version 0.13.1

4f69a30

Otherwise it wont find the config file. Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez enabled auto-merge (squash) October 15, 2023 17:29

bkchr approved these changes Oct 15, 2023

View reviewed changes

ggwpez added 2 commits October 16, 2023 12:05

Merge remote-tracking branch 'origin/master' into oty-zepter-devex

f9a5ada

Update Cargo.lock after merge

73e7699

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez mentioned this pull request Oct 16, 2023

Workspace maintenance #1884

Merged

Omit bandersnatch std

cfe437f

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

ggwpez mentioned this pull request Oct 24, 2023

Add --ignore-missing-propagate arg ggwpez/zepter#42

Merged

ggwpez added 3 commits October 24, 2023 12:39

Ignore missing bandersnatch_vrfs/std in sp-core

ed9b07b

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Merge remote-tracking branch 'origin/master' into oty-zepter-devex

f173468

Fix features after merge

d464a84

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

juangirini approved these changes Oct 24, 2023

View reviewed changes

ggwpez merged commit 4a44356 into master Oct 24, 2023
10 checks passed

ggwpez deleted the oty-zepter-devex branch October 24, 2023 15:59

ggwpez mentioned this pull request Oct 24, 2023

CI check features #1708

Merged

kiltbot mentioned this pull request Nov 7, 2023

[AUTOMATIC] Update Polkadot dependencies from 1.2.0 to 1.3.0 KILTprotocol/kilt-node#575

Closed

bkchr pushed a commit that referenced this pull request Apr 10, 2024

Use GitLab env vars to get git commit (#1831)

947a0a6

* use GitLab env vars to get git commit * compile_error to test it * Revert "compile_error to test it" This reverts commit 67d4782298d3cdfbe1a28231042bba6444316e8a.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve features dev-ex #1831

Improve features dev-ex #1831

ggwpez commented Oct 9, 2023 •

edited

Loading

bkontur commented Oct 9, 2023

ggwpez commented Oct 10, 2023

liamaharon left a comment

ggwpez commented Oct 16, 2023 •

edited

Loading

bkchr commented Oct 16, 2023

ggwpez commented Oct 17, 2023

Improve features dev-ex #1831

Improve features dev-ex #1831

Conversation

ggwpez commented Oct 9, 2023 • edited Loading

bkontur commented Oct 9, 2023

ggwpez commented Oct 10, 2023

liamaharon left a comment

Choose a reason for hiding this comment

ggwpez commented Oct 16, 2023 • edited Loading

bkchr commented Oct 16, 2023

ggwpez commented Oct 17, 2023

ggwpez commented Oct 9, 2023 •

edited

Loading

ggwpez commented Oct 16, 2023 •

edited

Loading