Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add development chain-spec file for minimal/parachain templates for Omni Node compatibility #6529

Open
wants to merge 68 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
3743b9f
Add development chain spec config
EleisonC Nov 16, 2024
1e833b4
Add path chain_spec_path
EleisonC Nov 16, 2024
3345130
Add path chain_spec_path
EleisonC Nov 16, 2024
97a1cc2
Add development chain spec for parachain template
EleisonC Nov 18, 2024
80669c1
Update chain spec path
EleisonC Nov 18, 2024
7a060c5
Merge branch 'master' into add-dev-chain-spec
EleisonC Nov 18, 2024
91e0ba8
Merge branch 'add-dev-chain-spec' of https://github.com/EleisonC/polk…
EleisonC Nov 18, 2024
d97247d
Revert "Update chain spec path"
EleisonC Nov 18, 2024
301ab59
update chain_spec_path
EleisonC Nov 18, 2024
541b524
Merge branch 'master' into add-dev-chain-spec
EleisonC Nov 18, 2024
02e431e
Merge branch 'master' into add-dev-chain-spec
EleisonC Nov 21, 2024
0146534
remove chain spec path not required
EleisonC Nov 21, 2024
14a32f1
Add runtime test for minimal dev chain spec
EleisonC Nov 27, 2024
2c318e3
Merge branch 'master' into add-dev-chain-spec
EleisonC Nov 27, 2024
cd66bbe
Update templates/minimal/node/tests/validate_dev_chain_spec.rs
EleisonC Nov 28, 2024
419a9d8
Update templates/minimal/node/tests/validate_dev_chain_spec.rs
EleisonC Nov 28, 2024
6197599
Add new spec file Change tests dir to runtime.
EleisonC Nov 29, 2024
92c47be
Merge branch 'master' into add-dev-chain-spec
EleisonC Nov 29, 2024
46b99d7
Merge branch 'master' into add-dev-chain-spec
EleisonC Nov 29, 2024
172f411
added modified dev chain spec. build minimal/runtime without --release
EleisonC Nov 30, 2024
4971b76
Merge remote-tracking branch 'origin/add-dev-chain-spec' into add-dev…
EleisonC Nov 30, 2024
8342ccf
Merge branch 'master' into add-dev-chain-spec
EleisonC Nov 30, 2024
94d1885
Merge remote-tracking branch 'origin/add-dev-chain-spec' into add-dev…
EleisonC Nov 30, 2024
d98b6f6
match test chain spec builder to exisitng one
EleisonC Dec 1, 2024
f748e0b
add new dev chain spec
EleisonC Dec 1, 2024
04904b2
remove unused code
EleisonC Dec 1, 2024
64303ef
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 3, 2024
8564533
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 5, 2024
5af21e6
add dev chain spec srtool-runtime
EleisonC Dec 5, 2024
a6ae597
automate spec chain validation for minimal template
EleisonC Dec 9, 2024
1d0b042
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 9, 2024
d98a9c2
parachain template spec chain verification
EleisonC Dec 9, 2024
94e4d62
Merge branch 'add-dev-chain-spec' of https://github.com/EleisonC/polk…
EleisonC Dec 9, 2024
99c2084
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 9, 2024
d0a31f5
update the template/minimal/ readme.md file
EleisonC Dec 9, 2024
82b5708
update the verification workflows
EleisonC Dec 10, 2024
cb3839e
refactor remove unused crates & crate-fts
EleisonC Dec 10, 2024
d783e47
refactor remove test
EleisonC Dec 10, 2024
bd09d21
refactor minimal readme.md
EleisonC Dec 10, 2024
9b34aef
refactor minimal & parachain readme.md files
EleisonC Dec 10, 2024
9d62db2
refactor dev spec verification files
EleisonC Dec 10, 2024
960fc22
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 10, 2024
0b84817
add custom workflow minimal/parachain templates
EleisonC Dec 13, 2024
0e948e8
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 13, 2024
860d655
remove seprate workflows
EleisonC Dec 13, 2024
a2804ae
Merge branch 'add-dev-chain-spec' of https://github.com/EleisonC/polk…
EleisonC Dec 13, 2024
dbad177
add fix for missing dep
EleisonC Dec 13, 2024
b2c44a8
fix download artifact version
EleisonC Dec 13, 2024
6b7bf34
Update .github/workflows/veriy-minimal-parachain-dev-spec-chains.yml
EleisonC Dec 13, 2024
402a0fe
Update .github/workflows/veriy-minimal-parachain-dev-spec-chains.yml
EleisonC Dec 13, 2024
84d46da
Update .github/workflows/veriy-minimal-parachain-dev-spec-chains.yml
EleisonC Dec 13, 2024
09eac57
implement feedback
EleisonC Dec 13, 2024
fda6b31
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 13, 2024
2af2180
fix linter issues readme
EleisonC Dec 13, 2024
4f385c8
add fix for workflow
EleisonC Dec 13, 2024
ef4c303
add fix for workflow
EleisonC Dec 13, 2024
a35001e
remove actions write
EleisonC Dec 13, 2024
8d7a25b
minor fix
EleisonC Dec 13, 2024
546ba56
timeout increase
EleisonC Dec 14, 2024
d04e9f6
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 18, 2024
11053f9
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 19, 2024
142f249
upload new parachain dev spec
EleisonC Dec 19, 2024
318609b
concurency Prioritize CI Workflow Execution for Faster Feedback
EleisonC Dec 19, 2024
b66e6b4
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 19, 2024
278324c
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 19, 2024
f0a3f18
Merge branch 'master' into add-dev-chain-spec
EleisonC Dec 19, 2024
63ffef7
fix dev chain spec diff
EleisonC Dec 20, 2024
da812f5
Merge branch 'add-dev-chain-spec' of https://github.com/EleisonC/polk…
EleisonC Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 137 additions & 0 deletions .github/workflows/veriy-minimal-parachain-dev-spec-chains.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
name: veriy-minimal-parachain-dev-spec-chains.yml
EleisonC marked this conversation as resolved.
Show resolved Hide resolved
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
workflow_dispatch:
EleisonC marked this conversation as resolved.
Show resolved Hide resolved

permissions:
contents: read
actions: write
alvicsam marked this conversation as resolved.
Show resolved Hide resolved

jobs:
setup:
timeout-minutes: 30
runs-on: ubuntu-latest
strategy:
matrix:
template:
- package_name: 'parachain-template-runtime'
runtime_path: './templates/parachain/runtime'
artifact_name: 'wasm-parachain-template-runtime'
chain_spec_output: 'parachain_test_chain_spec'
runtime_wasm_path: 'parachain-template-runtime/parachain_template_runtime.compact.compressed.wasm'
- package_name: 'minimal-template-runtime'
runtime_path: './templates/minimal/runtime'
artifact_name: 'wasm-minimal-template-runtime'
chain_spec_output: 'minimal_test_chain_spec'
runtime_wasm_path: 'minimal-template-runtime/minimal_template_runtime.compact.compressed.wasm'
iulianbarbu marked this conversation as resolved.
Show resolved Hide resolved

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Install Rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
alvicsam marked this conversation as resolved.
Show resolved Hide resolved

- name: Install srtool-cli
run: |
cargo install --git https://github.com/chevdor/srtool-cli
echo "Using srtool-cli Version:"
srtool --version

- name: Pull srtool docker image
run: |
srtool pull

- name: Prepare build directories
run: |
sudo mkdir -p ${{ matrix.template.runtime_path }}/target
sudo chmod -R 777 ${{ matrix.template.runtime_path }}/target

- name: Build runtime binary
run: |
srtool build --package ${{ matrix.template.package_name }} --runtime-dir ${{ matrix.template.runtime_path }} --root

- name: Upload runtime artifact
uses: actions/[email protected]
with:
name: ${{ matrix.template.artifact_name }}
path: ${{ matrix.template.runtime_path }}/target/srtool/release/wbuild/${{ matrix.template.runtime_wasm_path }}

- name: Install staging-chain-spec-builder and deps
run: |
# Install protobuf compiler
sudo apt-get update
sudo apt-get install -y protobuf-compiler

# Verify protoc installation
protoc --version

cargo install --path substrate/bin/utils/chain-spec-builder

- name: Create chain spec
run: |
chain-spec-builder create \
EleisonC marked this conversation as resolved.
Show resolved Hide resolved
--relay-chain "dev" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For parachain-template this should be set to rococo-local.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks let me fix that

--para-id 1000 \
--runtime "${{ matrix.template.runtime_path }}/target/srtool/release/wbuild/${{ matrix.template.runtime_wasm_path }}" \
named-preset development > ${{ matrix.template.chain_spec_output }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output is written in chain_spec.json file by default, so no need to redirect here. I suggested above a way to write the chain spec to a certain file


- name: Upload chain spec artifact
uses: actions/[email protected]
with:
name: ${{ matrix.template.chain_spec_output }}
path: ${{ matrix.template.chain_spec_output }}

verify-template-chain-specs:
if: success()
needs: setup
runs-on: ubuntu-latest
strategy:
matrix:
template:
- runtime_path: './templates/parachain'
chain_spec_output: 'parachain_test_chain_spec'
- runtime_path: './templates/minimal'
chain_spec_output: 'minimal_test_chain_spec'

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Download generated spec
uses: actions/[email protected]
with:
name: ${{ matrix.template.chain_spec_output }}
path: ./downloaded_spec

- name: Compare chain specs
run: |
sudo apt-get update && sudo apt-get install -y jq
# Check if files exist
if [ ! -f "${{ matrix.template.runtime_path }}/dev_chain_spec.json" ]; then
echo "❌ Reference spec not found at ${{ matrix.template.runtime_path }}/dev_chain_spec.json"
exit 1
fi

if [ ! -f "./downloaded_spec/${{ matrix.template.chain_spec_output }}" ]; then
echo "❌ Generated spec not found at ./downloaded_spec/${{ matrix.template.chain_spec_output }}"
exit 1
fi

jq -S . ${{ matrix.template.runtime_path }}/dev_chain_spec.json > existing_spec_sorted.json
jq -S . ./downloaded_spec/${{ matrix.template.chain_spec_output }} > generated_spec_sorted.json

if diff existing_spec_sorted.json generated_spec_sorted.json; then
echo "✅ Spec files match perfectly"
else
echo "❌ Spec files differ:"
diff existing_spec_sorted.json generated_spec_sorted.json
exit 1
fi
24 changes: 21 additions & 3 deletions templates/minimal/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,26 @@
- 🤏 This template is a minimal (in terms of complexity and the number of components)
template for building a blockchain node.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file has a #### Update zombienet-omni-node.toml with a valid chain spec path section. It would need to be updated given it can use directly the dev_chain_spec.json. Same for parachain/README.md file.



- 🔧 To simplify the process of using Zombienet with Omni Node, we've added a pre-configured development chain spec (dev_chain_spec.json) to the minimal template. Additionally, the zombienet-omni-node.toml file has been updated to include a valid chain spec path pointing to dev_chain_spec.json


- 🔧 Its runtime is configured with a single custom pallet as a starting point, and a handful of ready-made pallets
such as a [Balances pallet](https://paritytech.github.io/polkadot-sdk/master/pallet_balances/index.html).


- 👤 The template has no consensus configured - it is best for experimenting with a single node network.


- ⚠️ The README files for the templates in this repository are currently speculative. A stable release addressing this is expected in December.

- Follow solely the templates' READMEs and templates themselves from polkadot-sdk master branch, but ignore the references to the dedicated templates directory.
For the most accurate chain spec generation, `staging-chain-spec-builder` or other mentioned binaries and runtimes, please try to build them based on the polkadot-sdk most updated master branch an example:
```bash
cargo install --path substrate/bin/utils/chain-spec-builder
```


## Template Structure

A Polkadot SDK based project such as this one consists of:
Expand All @@ -61,7 +76,7 @@ compiled unless building the entire workspace).
- 🛠️ Depending on your operating system and Rust version, there might be additional
packages required to compile this template - please take note of the Rust compiler output.

Fetch minimal template code:
Fetch minimal template code (Note: The template repositories are outdated, and their READMEs are not synced. Please ignore this step for now.):

```sh
git clone https://github.com/paritytech/polkadot-sdk-minimal-template.git minimal-template
Expand All @@ -81,15 +96,18 @@ be used to run the minimal template's runtime. `polkadot-omni-node` binary crate

Please see installation section on [crates.io/omni-node](https://crates.io/crates/polkadot-omni-node).

#### Build `minimal-template-runtime`
#### Build `minimal-template-runtime` (A reminder to build on the most updated version of polkadot-sdk master)

```sh
cargo build -p minimal-template-runtime --release
```

#### Install `staging-chain-spec-builder`
```bash
cargo install --path substrate/bin/utils/chain-spec-builder
```

Please see the installation section at [`crates.io/staging-chain-spec-builder`](https://crates.io/crates/staging-chain-spec-builder).
Or Please see the installation section at [`crates.io/staging-chain-spec-builder`](https://crates.io/crates/staging-chain-spec-builder).

#### Use chain-spec-builder to generate the chain_spec.json file

Expand Down
85 changes: 85 additions & 0 deletions templates/minimal/dev_chain_spec.json

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions templates/minimal/zombienet-omni-node.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
[relaychain]
default_command = "polkadot-omni-node"
chain = "dev"
chain_spec_path = "<path/to/chain_spec.json>"
default_args = ["--dev"]
chain_spec_path = "./dev_chain_spec.json"
default_args = ["--dev-block-time 3000"]

[[relaychain.nodes]]
name = "alice"
Expand Down
17 changes: 17 additions & 0 deletions templates/parachain/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,31 @@

- ⏫ This template provides a starting point to build a [parachain](https://wiki.polkadot.network/docs/learn-parachains).


- 🔧 To simplify the process of using Zombienet with Omni Node, we’ve added a pre-configured development chain spec (dev_chain_spec.json) to the parachain template. Additionally, the zombienet-omni-node.toml file has been updated to include a valid chain spec path pointing to dev_chain_spec.json


- ☁️ It is based on the
[Cumulus](https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/polkadot_sdk/cumulus/index.html) framework.


- 🔧 Its runtime is configured with a single custom pallet as a starting point, and a handful of ready-made pallets
such as a [Balances pallet](https://paritytech.github.io/polkadot-sdk/master/pallet_balances/index.html).


- 👉 Learn more about parachains [here](https://wiki.polkadot.network/docs/learn-parachains)


- ⚠️ The README files for the templates in this repository are currently speculative. A stable release addressing this is expected in December.


- Follow solely the templates' READMEs and templates themselves from polkadot-sdk master branch, but ignore the references to the dedicated templates directory.
For the most accurate chain spec generation, `staging-chain-spec-builder` or other mentioned binaries and runtimes, please try to build them based on the polkadot-sdk most updated master branch an example:
```bash
cargo install --path substrate/bin/utils/chain-spec-builder
```


## Template Structure

A Polkadot SDK based project such as this one consists of:
Expand Down
108 changes: 108 additions & 0 deletions templates/parachain/dev_chain_spec.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion templates/parachain/zombienet-omni-node.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ ws_port = 9955

[[parachains]]
id = 1000
chain_spec_path = "<path/to/chain_spec.json>"
chain_spec_path = "./dev_chain_spec.json"

[parachains.collator]
name = "charlie"
Expand Down
Loading