Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add storage bounds for pallet staking and clean up deprecated non paged exposure storages #6445

Open
wants to merge 135 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 122 commits
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
6c808b6
add MaxInvulnerables
re-gius Nov 11, 2024
1b18bb1
add `MaxActiveValidators`
re-gius Nov 13, 2024
0ac60af
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 13, 2024
4edb8e1
fix + start working on migrations
re-gius Nov 13, 2024
65c7de6
nit (typo)
re-gius Nov 13, 2024
35069c9
fix migrations for `BoundedVec`
re-gius Nov 14, 2024
d4b15af
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 14, 2024
eb93a70
Migrate `ErasRewardPoints`
re-gius Nov 14, 2024
58cf45d
small fixes - `BoundedBTreeMap`
re-gius Nov 15, 2024
292c5d0
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 15, 2024
74f2055
fix
re-gius Nov 15, 2024
63f2c5e
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 18, 2024
b249a67
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 22, 2024
362f505
Adapt to new recent `DisabledValidators` change
re-gius Nov 22, 2024
4f21d53
Change `MaxActiveValidators` to `MaxWinners`
re-gius Nov 25, 2024
e258baf
undo change from another PR
re-gius Nov 25, 2024
4b7d896
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 25, 2024
01e2233
small fix for `MaxWinners`
re-gius Nov 25, 2024
243bec8
small fixes
re-gius Nov 25, 2024
a599afc
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 25, 2024
9a25fbc
Remove deprecated `ErasStakers` and `ErasStakersClipped`
re-gius Nov 25, 2024
98daabc
Fixes from #5986
re-gius Nov 25, 2024
aec5087
nit
re-gius Nov 25, 2024
36204da
remove unused import
re-gius Nov 25, 2024
18e9bee
fix `Invulnerables` migration
re-gius Nov 25, 2024
cbfd4a5
Undo migrations change
re-gius Nov 26, 2024
b5cbfec
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 26, 2024
9ca25a1
debug
re-gius Nov 26, 2024
10e7295
Increase `MaxInvulnerables` to 20 + fix migrations msg
re-gius Nov 26, 2024
b3186b1
fix - `MaxInvulnerables` = 20
re-gius Nov 26, 2024
336137a
Fix tests
re-gius Nov 26, 2024
c1911e1
Fix impl
re-gius Nov 26, 2024
2ac794e
Fix `Invulnerables` limit in benchmarking
re-gius Nov 26, 2024
583e46d
Bound `ExposurePage.others` with `MaxExposurePageSize`
re-gius Nov 26, 2024
6347a8c
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 26, 2024
9e82b85
fix `TypeInfo` error
re-gius Nov 27, 2024
1db7f39
fix benchmarking limit
re-gius Nov 27, 2024
f996a97
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 27, 2024
e030506
bound `BondedEras` Vec
re-gius Nov 27, 2024
3ab45d9
Add migration for `BondedEras`
re-gius Nov 27, 2024
a980daa
fix tests attempt
re-gius Nov 27, 2024
827e898
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 28, 2024
c4d5142
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 29, 2024
df8e567
bound `ClaimedRewards`
re-gius Nov 29, 2024
c066632
Use `WeakBoundedVec` for `EraStakersPaged` paged exposures
re-gius Nov 29, 2024
686b6b3
Turn more `BoundedVec` into `WeakBoundedVec`
re-gius Nov 29, 2024
9cf2263
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Nov 29, 2024
21ad9ea
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 2, 2024
ca881ac
add import
re-gius Dec 2, 2024
e986264
Migrate `MaxValidatorsCount` from storage to dynamic param + fixes
re-gius Dec 2, 2024
ea59383
Use `MaxValidatorsCount` to bound `ErasRewardPoints` `BoundedVec`
re-gius Dec 2, 2024
9a21b8f
remove unused imports
re-gius Dec 3, 2024
47863bb
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 3, 2024
50ce1a1
Use `MaxDisabledValidators` to bound disabled validators
re-gius Dec 3, 2024
40ccdec
fix `Vec`/`BoundedVec`
re-gius Dec 3, 2024
c3365a6
Fix `WeakBoundedVec` and `MaxValidatorsCount`
re-gius Dec 3, 2024
584fe53
fix
re-gius Dec 3, 2024
5333a7d
fix
re-gius Dec 3, 2024
31a9300
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 3, 2024
898d469
`BoundedVec` fix in tests
re-gius Dec 3, 2024
32df05d
fix
re-gius Dec 4, 2024
7bc9b98
Use `MaxValidatorsCount` for limit checks
re-gius Dec 4, 2024
9c4d874
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 4, 2024
536d5c5
undo wrong changes
re-gius Dec 9, 2024
608ec88
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 9, 2024
76469ea
Increase benchmarking `MaxValidators` limit
re-gius Dec 9, 2024
fb1a4fa
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 9, 2024
181fe4d
Remove `MaxValidators` from benchmarking config and use `MaxValidator…
re-gius Dec 9, 2024
b1f5f97
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 9, 2024
9c002c1
fix
re-gius Dec 9, 2024
503a7d9
Use `MaxValidatorsCount` for slashing pallet
re-gius Dec 9, 2024
63a18b1
Fix migration comments
re-gius Dec 9, 2024
49059c0
Restore benchmarking for slashing
re-gius Dec 10, 2024
efde7a7
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 10, 2024
8186680
fix
re-gius Dec 10, 2024
3fb6ca7
fix
re-gius Dec 10, 2024
66923a4
fix benchmarking config
re-gius Dec 10, 2024
bec0544
fix
re-gius Dec 10, 2024
3708085
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 10, 2024
73af22a
Remove `unbounded` for `ClaimedRewards`
re-gius Dec 10, 2024
1139689
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 11, 2024
2bda8eb
Update from re-gius running command 'prdoc --bump major --audience ru…
Dec 11, 2024
fe3147b
fix prdoc
re-gius Dec 11, 2024
cd8a0c3
fix prdoc
re-gius Dec 11, 2024
5b15e24
debug failing benchmark
re-gius Dec 11, 2024
9d055a8
benchmarking error debug
re-gius Dec 11, 2024
07b021b
undo debug changes
re-gius Dec 11, 2024
87e0e46
debug 1
re-gius Dec 11, 2024
473e0ff
debug 1
re-gius Dec 11, 2024
eb64ffc
debug 1
re-gius Dec 11, 2024
74d6c7a
debug dynamic param
re-gius Dec 11, 2024
94f5b31
restore after debugging
re-gius Dec 11, 2024
7b1fc5e
Fix bound comparison to `MaxValidatorsCount`
re-gius Dec 12, 2024
cf9d8c3
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 12, 2024
c265974
Use `MaxValidatorsCount` limit to check validators
re-gius Dec 16, 2024
bb5229c
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 16, 2024
6588714
fix staking tests
re-gius Dec 16, 2024
4754629
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 16, 2024
76a2d4b
Fix staking tests
re-gius Dec 17, 2024
67c41fa
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 17, 2024
bd28db5
fix benchmark
re-gius Dec 17, 2024
043b7d6
comment nit
re-gius Dec 17, 2024
68481f4
Always link `BondingDuration` to `MaxBondedEras` in tests
re-gius Dec 17, 2024
312904e
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 17, 2024
d508f8d
Remove old `MaxValidatorCount` from expected default genesis config
re-gius Dec 18, 2024
529fa0f
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 18, 2024
17eeef7
Update cumulus/parachains/integration-tests/emulated/chains/relays/we…
re-gius Dec 18, 2024
21a12d1
Fix PR title in prdoc and migrations for old deprecated storage items
re-gius Dec 18, 2024
e2fa0ca
Update substrate/frame/staking/src/lib.rs
re-gius Dec 18, 2024
4bf3cd6
add debug msg
re-gius Dec 18, 2024
f7c1fe4
fix migrations
re-gius Dec 18, 2024
c881bb0
fix comments + CR
re-gius Dec 18, 2024
306cd41
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 18, 2024
2da736a
Add more debugging `expect(...)` in tests
re-gius Dec 18, 2024
831f1b0
fix migrations
re-gius Dec 18, 2024
3702ce5
prefix with `_` unused variables
re-gius Dec 19, 2024
489fc9d
remove unused `page_size` fn arg + fix tests
re-gius Dec 19, 2024
65fc95b
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 19, 2024
9c88d39
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 19, 2024
a5e98e6
fix merge
re-gius Dec 19, 2024
7b82520
fix merge
re-gius Dec 19, 2024
30520af
fix merge
re-gius Dec 19, 2024
0d47dc5
expand errors msg for invulnerables
re-gius Dec 19, 2024
4b58f0a
using `default()` to create empty `WeakBoundedVec`
re-gius Dec 19, 2024
33e59b2
migration fix attempt
re-gius Dec 19, 2024
c889905
nit
re-gius Dec 19, 2024
300b7f9
migration fix
re-gius Dec 19, 2024
fc63dc7
fix migrations
re-gius Dec 20, 2024
720c9eb
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 20, 2024
a065a91
Set `MaxValidatorsCount` equal to `MaxAuthorities`
re-gius Dec 20, 2024
483eb03
Update polkadot/runtime/test-runtime/src/lib.rs
re-gius Dec 20, 2024
7aeb8a5
Merge branch 'master' into re-gius/staking/add-storage-bounds
re-gius Dec 20, 2024
c51d09b
fix comment
re-gius Dec 20, 2024
1a9be6b
fix
re-gius Dec 20, 2024
235a260
Update pr_6445.prdoc from semver CI
re-gius Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ use sp_authority_discovery::AuthorityId as AuthorityDiscoveryId;
use sp_consensus_babe::AuthorityId as BabeId;
use sp_consensus_beefy::ecdsa_crypto::AuthorityId as BeefyId;
use sp_core::storage::Storage;
use sp_runtime::Perbill;
use sp_runtime::{BoundedVec, Perbill};

// Polkadot
use polkadot_primitives::{AssignmentId, ValidatorId};
Expand Down Expand Up @@ -86,7 +86,13 @@ pub fn genesis() -> Storage {
.iter()
.map(|x| (x.0.clone(), x.1.clone(), STASH, pallet_staking::StakerStatus::Validator))
.collect(),
invulnerables: validators::initial_authorities().iter().map(|x| x.0.clone()).collect(),
invulnerables: BoundedVec::try_from(
validators::initial_authorities()
.iter()
.map(|x| x.0.clone())
.collect::<Vec<_>>(),
)
.expect("Limit for staking invulnerables must be less than initial authorities."),
force_era: pallet_staking::Forcing::ForceNone,
slash_reward_fraction: Perbill::from_percent(10),
..Default::default()
Expand Down
1 change: 0 additions & 1 deletion polkadot/runtime/common/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,6 @@ impl<T: pallet_session::Config> OneSessionHandler<T::AccountId>
/// A reasonable benchmarking config for staking pallet.
pub struct StakingBenchmarkingConfig;
impl pallet_staking::BenchmarkingConfig for StakingBenchmarkingConfig {
type MaxValidators = ConstU32<1000>;
type MaxNominators = ConstU32<1000>;
}

Expand Down
2 changes: 1 addition & 1 deletion polkadot/runtime/common/src/try_runtime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ where

let all_stakers = Ledger::<T>::iter().map(|(ctrl, l)| (ctrl, l.stash)).collect::<BTreeSet<_>>();
let mut all_exposed = BTreeSet::new();
ErasStakers::<T>::iter().for_each(|(_, val, expo)| {
ErasStakersPaged::<T>::iter().for_each(|((_era, val, _page), expo)| {
all_exposed.insert(val);
all_exposed.extend(expo.others.iter().map(|ie| ie.who.clone()))
});
Expand Down
14 changes: 0 additions & 14 deletions polkadot/runtime/parachains/src/disputes/slashing.rs
Original file line number Diff line number Diff line change
Expand Up @@ -83,17 +83,6 @@ const DEFENSIVE_PROOF: &'static str = "disputes module should bail on old sessio
#[cfg(feature = "runtime-benchmarks")]
pub mod benchmarking;

/// The benchmarking configuration.
pub trait BenchmarkingConfiguration {
const MAX_VALIDATORS: u32;
}

pub struct BenchConfig<const M: u32>;

impl<const M: u32> BenchmarkingConfiguration for BenchConfig<M> {
const MAX_VALIDATORS: u32 = M;
}

/// An offence that is filed when a series of validators lost a dispute.
#[derive(TypeInfo)]
#[cfg_attr(feature = "std", derive(Clone, PartialEq, Eq))]
Expand Down Expand Up @@ -400,9 +389,6 @@ pub mod pallet {

/// Weight information for extrinsics in this pallet.
type WeightInfo: WeightInfo;

/// Benchmarking configuration.
type BenchmarkingConfig: BenchmarkingConfiguration;
}

#[pallet::pallet]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,6 @@ use sp_session::MembershipProof;
// Candidate hash of the disputed candidate.
const CANDIDATE_HASH: CandidateHash = CandidateHash(Hash::zero());

// Simplify getting the value in the benchmark
pub const fn max_validators_for<T: super::Config>() -> u32 {
<<T>::BenchmarkingConfig as BenchmarkingConfiguration>::MAX_VALIDATORS
}

pub trait Config:
pallet_session::Config
+ pallet_session::historical::Config
Expand Down Expand Up @@ -148,7 +143,9 @@ mod benchmarks {
use super::*;

#[benchmark]
fn report_dispute_lost_unsigned(n: Linear<4, { max_validators_for::<T>() }>) {
fn report_dispute_lost_unsigned(
n: Linear<4, { <T as pallet_staking::Config>::MaxValidatorsCount::get() - 1 }>,
) {
let (session_index, key_owner_proof, validator_id) = setup_validator_set::<T>(n);

// submit a single `ForInvalid` dispute for a past session.
Expand Down
1 change: 0 additions & 1 deletion polkadot/runtime/rococo/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1184,7 +1184,6 @@ impl parachains_slashing::Config for Runtime {
ReportLongevity,
>;
type WeightInfo = parachains_slashing::TestWeightInfo;
type BenchmarkingConfig = parachains_slashing::BenchConfig<200>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overall observation is that we had to add temporarily a lot of bounds to pallets just for benchmarks, but as staking has now proper bounds for all its storage, they are not needed ✅

}

parameter_types! {
Expand Down
7 changes: 6 additions & 1 deletion polkadot/runtime/test-runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,7 @@ parameter_types! {
pub storage SessionsPerEra: SessionIndex = 6;
// 28 eras for unbonding (7 days).
pub storage BondingDuration: sp_staking::EraIndex = 28;
pub storage MaxBondedEras: u32 = (BondingDuration::get() as u32) + 1;
re-gius marked this conversation as resolved.
Show resolved Hide resolved
// 27 eras in which slashes can be cancelled (a bit less than 7 days).
pub storage SlashDeferDuration: sp_staking::EraIndex = 27;
pub const RewardCurve: &'static PiecewiseLinear<'static> = &REWARD_CURVE;
Expand Down Expand Up @@ -376,6 +377,7 @@ impl pallet_staking::Config for Runtime {
type Reward = ();
type SessionsPerEra = SessionsPerEra;
type BondingDuration = BondingDuration;
type MaxBondedEras = MaxBondedEras;
type SlashDeferDuration = SlashDeferDuration;
type AdminOrigin = frame_system::EnsureNever<()>;
type SessionInterface = Self;
Expand All @@ -396,6 +398,10 @@ impl pallet_staking::Config for Runtime {
type EventListeners = ();
type WeightInfo = ();
type DisablingStrategy = pallet_staking::UpToLimitWithReEnablingDisablingStrategy;
type MaxInvulnerables = ConstU32<20>;
type MaxRewardPagesPerValidator = ConstU32<20>;
type MaxValidatorsCount = ConstU32<300>;
type MaxDisabledValidators = ConstU32<100>;
}

parameter_types! {
Expand Down Expand Up @@ -550,7 +556,6 @@ impl parachains_slashing::Config for Runtime {
ReportLongevity,
>;
type WeightInfo = parachains_disputes::slashing::TestWeightInfo;
type BenchmarkingConfig = parachains_slashing::BenchConfig<1000>;
}

impl parachains_paras_inherent::Config for Runtime {
Expand Down
12 changes: 9 additions & 3 deletions polkadot/runtime/westend/src/genesis_config_presets.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ use sp_consensus_grandpa::AuthorityId as GrandpaId;
use sp_core::{crypto::get_public_from_string_or_panic, sr25519};
use sp_genesis_builder::PresetId;
use sp_keyring::Sr25519Keyring;
use sp_runtime::Perbill;
use sp_runtime::{BoundedVec, Perbill};
use westend_runtime_constants::currency::UNITS as WND;

/// Helper function to generate stash, controller and session key from seed
Expand Down Expand Up @@ -202,7 +202,10 @@ fn westend_testnet_genesis(
.iter()
.map(|x| (x.0.clone(), x.0.clone(), STASH, StakerStatus::<AccountId>::Validator))
.collect::<Vec<_>>(),
invulnerables: initial_authorities.iter().map(|x| x.0.clone()).collect::<Vec<_>>(),
invulnerables: BoundedVec::try_from(
initial_authorities.iter().map(|x| x.0.clone()).collect::<Vec<_>>()
)
.expect("Too many invulnerable validators!"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Error message can be a bit more informative, hinting at which config should be tweaked if this expect is ever reached.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error message improved in 0d47dc5

force_era: Forcing::NotForcing,
slash_reward_fraction: Perbill::from_percent(10),
},
Expand Down Expand Up @@ -373,7 +376,10 @@ fn westend_staging_testnet_config_genesis() -> serde_json::Value {
.iter()
.map(|x| (x.0.clone(), x.0.clone(), STASH, StakerStatus::<AccountId>::Validator))
.collect::<Vec<_>>(),
invulnerables: initial_authorities.iter().map(|x| x.0.clone()).collect::<Vec<_>>(),
invulnerables: BoundedVec::try_from(
initial_authorities.iter().map(|x| x.0.clone()).collect::<Vec<_>>()
)
.expect("Too many invulnerable validators!"),
force_era: Forcing::ForceNone,
slash_reward_fraction: Perbill::from_percent(10),
},
Expand Down
17 changes: 16 additions & 1 deletion polkadot/runtime/westend/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,14 @@ pub mod dynamic_params {
#[codec(index = 4)]
pub static UseAuctionSlots: bool = false;
}

#[dynamic_pallet_params]
#[codec(index = 1)]
pub mod staking {
/// Maximum number of validators allowed.
#[codec(index = 0)]
pub static MaxValidatorsCount: u32 = 300;
}
}

#[cfg(feature = "runtime-benchmarks")]
Expand Down Expand Up @@ -319,6 +327,7 @@ impl EnsureOriginWithArg<RuntimeOrigin, RuntimeParametersKey> for DynamicParamet

match key {
Inflation(_) => frame_system::ensure_root(origin.clone()),
Staking(_) => frame_system::ensure_root(origin.clone()),
}
.map_err(|_| origin)
}
Expand Down Expand Up @@ -716,6 +725,7 @@ parameter_types! {
pub const SessionsPerEra: SessionIndex = prod_or_fast!(6, 1);
// 2 eras for unbonding (12 hours).
pub const BondingDuration: sp_staking::EraIndex = 2;
pub const MaxBondedEras: u32 = (BondingDuration::get() as u32) + 1;
// 1 era in which slashes can be cancelled (6 hours).
pub const SlashDeferDuration: sp_staking::EraIndex = 1;
pub const MaxExposurePageSize: u32 = 64;
Expand All @@ -738,6 +748,7 @@ impl pallet_staking::Config for Runtime {
type Reward = ();
type SessionsPerEra = SessionsPerEra;
type BondingDuration = BondingDuration;
type MaxBondedEras = MaxBondedEras;
type SlashDeferDuration = SlashDeferDuration;
type AdminOrigin = EitherOf<EnsureRoot<AccountId>, StakingAdmin>;
type SessionInterface = Self;
Expand All @@ -756,6 +767,10 @@ impl pallet_staking::Config for Runtime {
type EventListeners = (NominationPools, DelegatedStaking);
type WeightInfo = weights::pallet_staking::WeightInfo<Runtime>;
type DisablingStrategy = pallet_staking::UpToLimitWithReEnablingDisablingStrategy;
type MaxInvulnerables = frame_support::traits::ConstU32<20>;
type MaxRewardPagesPerValidator = frame_support::traits::ConstU32<20>;
type MaxValidatorsCount = dynamic_params::staking::MaxValidatorsCount;
type MaxDisabledValidators = ConstU32<100>;
}

impl pallet_fast_unstake::Config for Runtime {
Expand Down Expand Up @@ -1417,7 +1432,6 @@ impl parachains_slashing::Config for Runtime {
ReportLongevity,
>;
type WeightInfo = weights::polkadot_runtime_parachains_disputes_slashing::WeightInfo<Runtime>;
type BenchmarkingConfig = parachains_slashing::BenchConfig<300>;
}

parameter_types! {
Expand Down Expand Up @@ -1841,6 +1855,7 @@ pub mod migrations {
parachains_shared::migration::MigrateToV1<Runtime>,
parachains_scheduler::migration::MigrateV2ToV3<Runtime>,
pallet_staking::migrations::v16::MigrateV15ToV16<Runtime>,
pallet_staking::migrations::v17::MigrateV16ToV17<Runtime>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can check the logs for this migration in this CI job:

https://github.com/paritytech/polkadot-sdk/actions/runs/12412281493/job/34651846644?pr=6445

there seems to be some error-ish logs in there:

💸 Migration failed for ClaimedRewards from v16 to v17.

Copy link
Contributor Author

@re-gius re-gius Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solved with force_from(...) for WeakBoundedVec. I noticed that old individual validator exposures have up to 205 pages, while we would expect them to be at most 20 in the future @Ank4n
What do you think is the best approach: keep 20 as a limit for MaxRewardPagesPerValidator for future items and force old items in a WeakBoundedVec or increase the MaxRewardPagesPerValidator limit to something like 250 to accomodate old items?

EDIT: I can also see a third option: merge pages for old items until they are less than 20.

// permanent
pallet_xcm::migration::MigrateToLatestXcmVersion<Runtime>,
);
Expand Down
2 changes: 0 additions & 2 deletions polkadot/runtime/westend/src/weights/pallet_fast_unstake.rs
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,6 @@ impl<T: frame_system::Config> pallet_fast_unstake::WeightInfo for WeightInfo<T>
/// Proof Skipped: ElectionProviderMultiPhase CurrentPhase (max_values: Some(1), max_size: None, mode: Measured)
/// Storage: Staking CurrentEra (r:1 w:0)
/// Proof: Staking CurrentEra (max_values: Some(1), max_size: Some(4), added: 499, mode: MaxEncodedLen)
/// Storage: Staking ErasStakers (r:257 w:0)
/// Proof Skipped: Staking ErasStakers (max_values: None, max_size: None, mode: Measured)
/// The range of component `v` is `[1, 256]`.
/// The range of component `b` is `[1, 64]`.
fn on_idle_check(v: u32, b: u32, ) -> Weight {
Expand Down
2 changes: 0 additions & 2 deletions polkadot/runtime/westend/src/weights/pallet_staking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -488,8 +488,6 @@ impl<T: frame_system::Config> pallet_staking::WeightInfo for WeightInfo<T> {
/// Proof: `Staking::Bonded` (`max_values`: None, `max_size`: Some(72), added: 2547, mode: `MaxEncodedLen`)
/// Storage: `Staking::Ledger` (r:65 w:65)
/// Proof: `Staking::Ledger` (`max_values`: None, `max_size`: Some(1091), added: 3566, mode: `MaxEncodedLen`)
/// Storage: `Staking::ErasStakersClipped` (r:1 w:0)
/// Proof: `Staking::ErasStakersClipped` (`max_values`: None, `max_size`: None, mode: `Measured`)
/// Storage: `Staking::ErasStakersOverview` (r:1 w:0)
/// Proof: `Staking::ErasStakersOverview` (`max_values`: None, `max_size`: Some(92), added: 2567, mode: `MaxEncodedLen`)
/// Storage: `Staking::ClaimedRewards` (r:1 w:1)
Expand Down
51 changes: 51 additions & 0 deletions prdoc/pr_6445.prdoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
title: Add storage bounds for pallet `staking` and clean up deprecated non paged exposure storages
doc:
- audience: Runtime Dev
description: |-
This is part of #6289 and necessary for the Asset Hub migration.

Building on the observations and suggestions from #255 .

**Changes**

- Add `MaxInvulnerables` to bound `Invulnerables` Vec -> `BoundedVec`.
- Set to constant 20 in the pallet (must be >= 17 for backward compatibility of runtime `westend`).
- Add `MaxDisabledValidators` to bound `DisabledValidators` Vec -> `BoundedVec`
- Set to constant 100 in the pallet (it should be <= 1/3 * `MaxValidatorsCount` according to the current disabling strategy)
- Remove `ErasStakers` and `ErasStakersClipped` (see #433 )
- They were deprecated in v14 and could have been removed since staking era 1504 (now it's > 1600)
- Completing the task from #5986
Comment on lines +15 to +17
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be included in the title, there might be other chain using the staking pallet,
It could also be nice to know if a migration is needed in case a chain didn't yet removed them, or they need to wait?

Copy link
Contributor Author

@re-gius re-gius Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Included in the title + killed deleted storage items (thanks!) in 21a12d1 + f7c1fe4

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it lacks guideline in regards to the clean up of deprecated non paged exposure storage items.

The issue #433 state that 84 eras must pass for the migration to finish.

I think the prdoc should mention what are the consequences of this cleanup.
I would imagine a message like this:

Non paged exposure logic is removed. The migration from non paged exposure to paged exposure was implemented as a slow migration that spans on multiple eras. Any runtime using the pallet staking must ensure that the migration from non paged to paged exposure is handled correctly before using this new version.

I don't have the exact details of the consequences of not waiting for the 84 eras. But if you know what are the consequences then you can also write it down.

We should keep in mind that prdoc is used by other projects using the staking pallets not only polkadot/kusama.

- Use `MaxExposurePageSize` to bound `ErasStakersPaged` mapping to exposure pages: each `ExposurePage.others` Vec is turned into a `WeakBoundedVec` to allow easy and quick changes to this bound
- Add `MaxBondedEras` to bound `BondedEras` Vec -> `BoundedVec`
- Set to `BondingDuration::get() + 1` everywhere to include both time interval endpoints in [`current_era - BondingDuration::get()`, `current_era`]. Notice that this was done manually in every test and runtime.
- Add `MaxRewardPagesPerValidator` to bound `ClaimedRewards` Vec of pages -> `WeakBoundedVec`
- Set to constant 20 in the pallet. The vector of pages is now a `WeakBoundedVec` to allow easy and quick changes to this parameter
- Remove `MaxValidatorsCount` optional storage item to add `MaxValidatorsCount` mandatory config parameter
- Using it to to bound `EraRewardPoints.individual` BTreeMap -> `BoundedBTreeMap`;
- Set to dynamic parameter in runtime westend so that changing it should not require migrations for it

crates:
- name: pallet-staking
bump: major
- name: westend-runtime
bump: minor
- name: sp-staking
bump: minor
- name: polkadot-runtime-common
bump: patch
- name: pallet-fast-unstake
bump: patch
- name: pallet-babe
bump: patch
- name: pallet-beefy
bump: patch
- name: pallet-delegated-staking
bump: patch
- name: pallet-grandpa
bump: patch
- name: pallet-root-offences
bump: patch
- name: polkadot-runtime-parachains
bump: major
- name: rococo-runtime
bump: patch
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
"stakers": [],
"minNominatorBond": 0,
"minValidatorBond": 0,
"maxValidatorCount": null,
"maxNominatorCount": null
},
"session": {
Expand Down
7 changes: 6 additions & 1 deletion substrate/bin/node/runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,7 @@ pallet_staking_reward_curve::build! {
parameter_types! {
pub const SessionsPerEra: sp_staking::SessionIndex = 6;
pub const BondingDuration: sp_staking::EraIndex = 24 * 28;
pub const MaxBondedEras: u32 = (BondingDuration::get() as u32) + 1;
pub const SlashDeferDuration: sp_staking::EraIndex = 24 * 7; // 1/4 the bonding duration.
pub const RewardCurve: &'static PiecewiseLinear<'static> = &REWARD_CURVE;
pub const MaxNominators: u32 = 64;
Expand All @@ -708,7 +709,6 @@ const MAX_QUOTA_NOMINATIONS: u32 = 16;
pub struct StakingBenchmarkingConfig;
impl pallet_staking::BenchmarkingConfig for StakingBenchmarkingConfig {
type MaxNominators = ConstU32<1000>;
type MaxValidators = ConstU32<1000>;
}

impl pallet_staking::Config for Runtime {
Expand All @@ -722,6 +722,7 @@ impl pallet_staking::Config for Runtime {
type Reward = (); // rewards are minted from the void
type SessionsPerEra = SessionsPerEra;
type BondingDuration = BondingDuration;
type MaxBondedEras = MaxBondedEras;
type SlashDeferDuration = SlashDeferDuration;
/// A super-majority of the council can cancel the slash.
type AdminOrigin = EitherOfDiverse<
Expand All @@ -745,6 +746,10 @@ impl pallet_staking::Config for Runtime {
type WeightInfo = pallet_staking::weights::SubstrateWeight<Runtime>;
type BenchmarkingConfig = StakingBenchmarkingConfig;
type DisablingStrategy = pallet_staking::UpToLimitWithReEnablingDisablingStrategy;
type MaxInvulnerables = ConstU32<20>;
type MaxRewardPagesPerValidator = ConstU32<20>;
type MaxValidatorsCount = ConstU32<300>;
Copy link
Contributor

@kianenigma kianenigma Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other pallets like session, babe, grandpa and such also have a notion of MaxValidators (expressed as MaxAuthorities) that should be equal to the max validators of this pallet.

within this file, you can use one pub const MaxValidators: u32 = 300 everywhere to unify it.

Taking it a step further, you can expose this as a part of trait SessionManager. this trait can expose a trat SessionManager { type MaxAuthorities = <set-by-staking-pallet> }. Then, within the pallet-session, who consumes SessionManager, you can do:

fn integroty_check() { 
    /// A way to express within pallet-session that whoever implements `SessionManager` should have a compatible `MaxAuthorities`. 
    assert!(T::SessionManager::MaxAuthoritiet::get() >= T::MaxAuthorities::get())
}

This might be too much for this PR, but good for you to be familiar with the pattern.

Whenever they are multiple parameters within two pallets that have a logical dependency (they have to be equal, or one has to be larger than the other), you can remove the implicitness like this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed in a065a91

Opened issue #6972 to unify MaxValidators limit (I also see a MaxActiveValidators)

type MaxDisabledValidators = ConstU32<100>;
}

impl pallet_fast_unstake::Config for Runtime {
Expand Down
5 changes: 3 additions & 2 deletions substrate/bin/node/testing/src/genesis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ use kitchensink_runtime::{
RuntimeGenesisConfig, SessionConfig, SocietyConfig, StakerStatus, StakingConfig,
};
use sp_keyring::Ed25519Keyring;
use sp_runtime::Perbill;
use sp_runtime::{BoundedVec, Perbill};

/// Create genesis runtime configuration for tests.
pub fn config() -> RuntimeGenesisConfig {
Expand Down Expand Up @@ -65,7 +65,8 @@ pub fn config_endowed(extra_endowed: Vec<AccountId>) -> RuntimeGenesisConfig {
validator_count: 3,
minimum_validator_count: 0,
slash_reward_fraction: Perbill::from_percent(10),
invulnerables: vec![alice(), bob(), charlie()],
invulnerables: BoundedVec::try_from(vec![alice(), bob(), charlie()])
.expect("Too many invulnerable validators!"),
..Default::default()
},
society: SocietyConfig { pot: 0 },
Expand Down
6 changes: 4 additions & 2 deletions substrate/frame/babe/src/mock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ use sp_runtime::{
impl_opaque_keys,
testing::{Digest, DigestItem, Header, TestXt},
traits::{Header as _, OpaqueKeys},
BuildStorage, Perbill,
BoundedVec, BuildStorage, Perbill,
};
use sp_staking::{EraIndex, SessionIndex};

Expand Down Expand Up @@ -141,6 +141,7 @@ pallet_staking_reward_curve::build! {
parameter_types! {
pub const SessionsPerEra: SessionIndex = 3;
pub const BondingDuration: EraIndex = 3;
pub const MaxBondedEras: u32 = (BondingDuration::get() as u32) + 1;
pub const RewardCurve: &'static PiecewiseLinear<'static> = &REWARD_CURVE;
pub static ElectionsBounds: ElectionBounds = ElectionBoundsBuilder::default().build();
}
Expand All @@ -160,6 +161,7 @@ impl pallet_staking::Config for Test {
type Currency = Balances;
type SessionsPerEra = SessionsPerEra;
type BondingDuration = BondingDuration;
type MaxBondedEras = MaxBondedEras;
type AdminOrigin = frame_system::EnsureRoot<Self::AccountId>;
type SessionInterface = Self;
type UnixTime = pallet_timestamp::Pallet<Test>;
Expand Down Expand Up @@ -342,7 +344,7 @@ pub fn new_test_ext_raw_authorities(authorities: Vec<AuthorityId>) -> sp_io::Tes
validator_count: 8,
force_era: pallet_staking::Forcing::ForceNew,
minimum_validator_count: 0,
invulnerables: vec![],
invulnerables: BoundedVec::new(),
..Default::default()
};

Expand Down
Loading
Loading