Enable Coretime on Polkadot Relay #401

tdimitrov · 2024-07-25T14:42:57Z

Redo of #354 on top of #381

TODOs:

Check migrations with chopsticks
Regenerate weights

relay/polkadot/src/lib.rs

tdimitrov · 2024-07-26T07:21:09Z

relay/polkadot/src/lib.rs

+				// TODO: Fix this key for Polkadot!
+				pallet_custom_origins::Origin::AuctionAdmin.into(),
+				[
+					0x5c, 0x68, 0xbf, 0x0c, 0x2d, 0x11, 0x04, 0x91, 0x6b, 0xa5, 0xa4, 0xde, 0xe6,


I assume this key is for scheduler -> lookup, right?

How can I understand which is the correct key?

I have the right key already.

I found it by searching for Origins: AuctionAdmin. It is:

[ 0x87, 0xa8, 0x71, 0xb4, 0xd6, 0x21, 0xf0, 0xb9, 0x73, 0x47, 0x5a, 0xaf, 0xcc, 0x32, 0x61, 0x0b, 0xd7, 0x68, 0x8f, 0x15, 0x02, 0x33, 0x8a, 0xcd, 0x00, 0xee, 0x48, 0x8a, 0xc3, 0x62, 0x0f, 0x4c, ],

I'll provide a PR with this and other changes against your PR.

Feel free to commit in this PR directly (I've invited you as a collaborator).

CHANGELOG.md

s0me0ne-unkn0wn · 2024-07-26T07:52:40Z

relay/kusama/src/lib.rs

@@ -1677,7 +1677,7 @@ construct_runtime! {
 		ParaSessionInfo: parachains_session_info = 61,
 		ParasDisputes: parachains_disputes = 62,
 		ParasSlashing: parachains_slashing = 63,
-		OnDemandAssignmentProvider: parachains_assigner_on_demand = 64,
+		OnDemand: parachains_assigner_on_demand = 64,


This renaming requires a storage migration using https://github.com/paritytech/polkadot-sdk/blob/604f56f03db847a90aa4fdb13be6b80482a4dcd6/substrate/frame/support/src/storage/migration.rs#L323
As storage keys will change

@tdimitrov Can you look into that please?

eskimor · 2024-07-26T09:33:19Z

Collecting things to test/check after that PR is ready: https://github.com/orgs/paritytech/projects/119/views/20?pane=issue&itemId=66200512

tdimitrov · 2024-07-26T13:28:26Z

relay/polkadot/src/lib.rs

+			move_storage_from_pallet(b"Pallet", b"OnDemandAssignmentProvider", b"OnDemand");
+			move_storage_from_pallet(b"ParaIdAffinity", b"OnDemandAssignmentProvider", b"OnDemand");
+			move_storage_from_pallet(b"QueueStatus", b"OnDemandAssignmentProvider", b"OnDemand");
+			move_storage_from_pallet(b"FreeEntries", b"OnDemandAssignmentProvider", b"OnDemand");
+			move_storage_from_pallet(
+				b"AffinityEntries",
+				b"OnDemandAssignmentProvider",
+				b"OnDemand",
+			);
+			move_storage_from_pallet(b"Revenue", b"OnDemandAssignmentProvider", b"OnDemand");


@s0me0ne-unkn0wn does this look right to you?

As far as I understand I need to call move_storage_from_pallet for each storage item in the pallet?
Are the pallet names correct?

Ideally I should use get_storage_prefix() here but the storage items are not accessible here so I hardcoded them 🥇

@s0me0ne-unkn0wn does this look right to you?

Well, it looks totally fine to me, but it should be taken into account that I didn't ever do it myself as well tbh 😆

As far as I understand I need to call move_storage_from_pallet for each storage item in the pallet?
Are the pallet names correct?

My understanding is the same, and the naming seems to be fine. I still hope to get some reviews from guys who have closer experience 🙂

Applying polkadot-fellows#354 on top of 1.14 release Co-authored-by: eskimor <[email protected]>

eskimor · 2024-08-01T09:44:02Z

relay/polkadot/src/lib.rs

+
+	// Migrate storage for pallet rename `OnDemandAssignmentProvider` -> `OnDemand`
+	pub struct OnDemandRename;
+	impl OnRuntimeUpgrade for OnDemandRename {


Two things here:

For Polkadot no migration should be needed as onDemand was newly introduced in this PR.

Do we need any protection here from preventing this migration to be run multiple times?

I've extracted the pallet rename on Kusama in a separate PR: #417

seadanda

Looks good

relay/polkadot/constants/src/lib.rs

seadanda · 2024-08-02T11:35:56Z

relay/polkadot/src/lib.rs

+parameter_types! {
+	pub const BrokerId: u32 = system_parachain::BROKER_ID;
+	pub const BrokerPalletId: PalletId = PalletId(*b"py/broke");
+	pub MaxXcmTransactWeight: Weight = Weight::from_parts(constants::WEIGHT_REF_TIME_PER_MILLIS, 20 * constants::WEIGHT_PROOF_SIZE_PER_KB);


Same comment as here: do we need such a huge buffer? Or maybe the other way round, do we care that we have such a big buffer? If not then I'd increase the Coretime Chain side too.

I took this from Kusama. If the value is not appropriate we should change it there too.

https://github.com/polkadot-fellows/runtimes/blob/main/relay/kusama/src/lib.rs#L1288

My bad, I probably took the values from master instead of from the release branch. Fixed.

CHANGELOG.md

relay/polkadot/src/lib.rs

Co-authored-by: Dónal Murray <[email protected]>

sandreim · 2024-08-07T09:29:36Z

relay/polkadot/src/weights/runtime_parachains_coretime.rs

+// ./target/production/polkadot
+// benchmark
+// pallet
+// --chain=./kusama-chain-spec.json


Same question as above

sandreim · 2024-08-07T09:30:21Z

relay/polkadot/src/weights/runtime_parachains_assigner_on_demand.rs

+// ./target/production/polkadot
+// benchmark
+// pallet
+// --chain=./kusama-chain-spec.json


Shouldn't this be generated with --chain=./polkadot-chain-spec.json ?

There should be another pre-release PR that regenerates all the weights. Usually @bkontur does it (although would be good to improve/automate that process)

relay/polkadot/src/lib.rs

eskimor · 2024-08-08T07:05:30Z

/merge

fellowship-merge-bot · 2024-08-08T07:07:12Z

Enabled auto-merge in Pull Request

Available commands

/merge: Enables auto-merge for Pull Request
/merge cancel: Cancels auto-merge for Pull Request
/merge help: Shows this menu

For more information see the documentation

Add the Polkadot Coretime chain in advance of the 1.3.0 release. This uses the new Price adapter which has been running on Kusama now for two sales cycles, and includes the mechanism to burn revenue. TODO: - [x] Add Transact tests for hardcoded weights after #401 is merged - [ ] Rerun benchmarks and check hardcoded weights after merge The genesis chain-spec is developing on seadanda#4. This can be used as a merge target for any community boot nodes who would like to be included at genesis and will be separately merged to `main`. --------- Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: fellowship-merge-bot[bot] <151052383+fellowship-merge-bot[bot]@users.noreply.github.com>

…chains (#4) * Add kusama live preset and chain spec to builder * Add coretime-polkadot chainspec * Make clippy happy * Remove unnecessary dep * Use presets for new chainspecs * Add live to preset names * Add stake plus bootnodes to coretime polkadot at release. (#5) * Add the Polkadot Coretime runtime (polkadot-fellows#410) Add the Polkadot Coretime chain in advance of the 1.3.0 release. This uses the new Price adapter which has been running on Kusama now for two sales cycles, and includes the mechanism to burn revenue. TODO: - [x] Add Transact tests for hardcoded weights after polkadot-fellows#401 is merged - [ ] Rerun benchmarks and check hardcoded weights after merge The genesis chain-spec is developing on #4. This can be used as a merge target for any community boot nodes who would like to be included at genesis and will be separately merged to `main`. --------- Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: fellowship-merge-bot[bot] <151052383+fellowship-merge-bot[bot]@users.noreply.github.com> * Update changelog * all runtimes: remove already applied migrations (polkadot-fellows#420) Remove migrations already **applied on-chain**. - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Adrian Catangiu <[email protected]> Co-authored-by: joe petrowski <[email protected]> --------- Signed-off-by: Adrian Catangiu <[email protected]> Co-authored-by: Tom <[email protected]> Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Branislav Kontur <[email protected]> Co-authored-by: fellowship-merge-bot[bot] <151052383+fellowship-merge-bot[bot]@users.noreply.github.com> Co-authored-by: Adrian Catangiu <[email protected]> Co-authored-by: joe petrowski <[email protected]>

…ration (#426) While testing #401 I noticed that there are two parachains with gaps in their leases which are not migrated to coretime correctly. These are para ids 3359 and 3388. The problem is that the migration obtains the list of parachain ids which should be migrated from `paras::Parachains` ([here](https://github.com/paritytech/polkadot-sdk/blob/1f49358db0033e57a790eac6daccc45beba81863/polkadot/runtime/parachains/src/coretime/migration.rs#L114)) and there are no entries for para ids which are not active at the moment. Paras 3359 and 3388 are not active in the current lease period (excerpt from `slots::leases()`: ``` [ 3,359 ] [ null [ 14xQXJdUDC1pzyt8y3z27ANiUBgP7zTSaYutaLELJoyQrdLP 2,010,000,000,000 ] [ 14xQXJdUDC1pzyt8y3z27ANiUBgP7zTSaYutaLELJoyQrdLP 2,010,000,000,000 ] <skipped> ``` and: ``` [ 3,388 ] [ null [ 1TpMimWf8NvrusAQ36hVdgMpYhhtojg5Nw41CuVxG2zHPDg 31,000,000,000,000 ] [ 1TpMimWf8NvrusAQ36hVdgMpYhhtojg5Nw41CuVxG2zHPDg 31,000,000,000,000 ] <skipped> ``` And they have got no entries in `paras::parachains()`: ``` [ 1,000 1,001 1,002 1,004 2,000 2,002 2,004 2,006 2,008 2,012 2,013 2,025 2,026 2,030 2,031 2,032 2,034 2,035 2,037 2,040 2,043 2,046 2,048 2,051 2,053 2,056 2,086 2,090 2,091 2,092 2,093 2,094 2,101 2,104 2,106 3,333 3,338 3,340 3,344 3,345 3,346 3,354 3,358 3,366 3,367 3,369 3,370 3,373 3,375 3,377 ] ``` As a result the migration skips them which is wrong. A proper fix for this issue will require a new polkadot-sdk release and a bump in the runtimes repo which requires time and effort. To avoid this the PR moves the coretime migration from polkadot-sdk to the fellowship repo. --------- Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Dónal Murray <[email protected]> Co-authored-by: fellowship-merge-bot[bot] <151052383+fellowship-merge-bot[bot]@users.noreply.github.com>

tdimitrov force-pushed the tsv-polkadot-coretime branch from f42562b to 8463811 Compare July 25, 2024 14:43

alindima reviewed Jul 26, 2024

View reviewed changes

relay/polkadot/src/lib.rs Outdated Show resolved Hide resolved

tdimitrov commented Jul 26, 2024

View reviewed changes

alindima reviewed Jul 26, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

s0me0ne-unkn0wn reviewed Jul 26, 2024

View reviewed changes

eskimor approved these changes Jul 26, 2024

View reviewed changes

tdimitrov commented Jul 26, 2024

View reviewed changes

ggwpez deleted the branch polkadot-fellows:main July 29, 2024 23:53

ggwpez closed this Jul 29, 2024

ggwpez reopened this Jul 30, 2024

tdimitrov changed the base branch from oty-update-1-14 to main July 30, 2024 12:57

tdimitrov changed the base branch from main to oty-update-1-14 July 30, 2024 12:57

tdimitrov changed the base branch from oty-update-1-14 to main July 30, 2024 12:59

tdimitrov and others added 11 commits July 30, 2024 16:28

Enable Coretime on Polkadot

edfd658

Applying polkadot-fellows#354 on top of 1.14 release Co-authored-by: eskimor <[email protected]>

Dummy weights (copy-pasted from Kusama)

d726aa4

ElasticScaling is already enabled - remove it from migrations

1ec40cf

Also remove from migrations

f33e546

Rename, auction key, parameter set

4111d36

And remove elastic scaling from the changelog

3f14519

Note on demand renaming in changelog

f762442

Add OnSwap

b1d09a9

Don't drop leases with holes.

db45f01

Add missing import and format

4727265

OnDemandRename - first try

a274576

tdimitrov force-pushed the tsv-polkadot-coretime branch from a6794d1 to a274576 Compare July 30, 2024 14:01

fmt

0a53150

tdimitrov marked this pull request as ready for review July 30, 2024 14:04

fmt

c7acb91

eskimor suggested changes Aug 1, 2024

View reviewed changes

tdimitrov mentioned this pull request Aug 2, 2024

Rename OnDemandAssignmentProvider to OnDemand on Kusama #417

Closed

2 tasks

tdimitrov added 3 commits August 2, 2024 11:16

Undo parachains_assigner_on_demand pallet rename on Kusama

76fc5ed

Fix benches

088cec4

Fix benches for real

0d6aa9e

seadanda approved these changes Aug 3, 2024

View reviewed changes

tdimitrov and others added 3 commits August 5, 2024 10:04

Apply suggestions from code review

50c8256

Co-authored-by: Dónal Murray <[email protected]>

Update MaxXcmTransactWeight

766a188

Merge branch 'main' into tsv-polkadot-coretime

fa1b781

tdimitrov requested a review from eskimor August 6, 2024 06:49

This was referenced Aug 6, 2024

1.3.0 Release #186

Closed

Add the Polkadot Coretime runtime #410

Merged

eskimor approved these changes Aug 7, 2024

View reviewed changes

sandreim reviewed Aug 7, 2024

View reviewed changes

acatangiu approved these changes Aug 7, 2024

View reviewed changes

joepetrowski approved these changes Aug 7, 2024

View reviewed changes

relay/polkadot/src/lib.rs Outdated Show resolved Hide resolved

joepetrowski added 2 commits August 7, 2024 18:49

Update relay/polkadot/src/lib.rs

e8ca5b9

Merge branch 'main' into tsv-polkadot-coretime

47fda1d

fellowship-merge-bot bot merged commit b52f7de into polkadot-fellows:main Aug 8, 2024
45 checks passed

tdimitrov mentioned this pull request Aug 12, 2024

Proper handling for legacy parachain leases with gaps in coretime migration #426

Merged

tdimitrov deleted the tsv-polkadot-coretime branch August 19, 2024 08:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Coretime on Polkadot Relay #401

Enable Coretime on Polkadot Relay #401

tdimitrov commented Jul 25, 2024 •

edited

Loading

tdimitrov Jul 26, 2024

eskimor Jul 26, 2024

eskimor Jul 26, 2024

eskimor Jul 26, 2024

tdimitrov Jul 26, 2024

s0me0ne-unkn0wn Jul 26, 2024

eskimor Jul 26, 2024

eskimor commented Jul 26, 2024 •

edited

Loading

tdimitrov Jul 26, 2024

s0me0ne-unkn0wn Jul 26, 2024

eskimor Aug 1, 2024

tdimitrov Aug 2, 2024

seadanda left a comment

seadanda Aug 2, 2024

tdimitrov Aug 5, 2024

seadanda Aug 5, 2024

tdimitrov Aug 5, 2024

sandreim Aug 7, 2024

sandreim Aug 7, 2024

joepetrowski Aug 7, 2024

eskimor commented Aug 8, 2024

fellowship-merge-bot bot commented Aug 8, 2024

Enable Coretime on Polkadot Relay #401

Enable Coretime on Polkadot Relay #401

Conversation

tdimitrov commented Jul 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eskimor commented Jul 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seadanda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eskimor commented Aug 8, 2024

fellowship-merge-bot bot commented Aug 8, 2024

tdimitrov commented Jul 25, 2024 •

edited

Loading

eskimor commented Jul 26, 2024 •

edited

Loading