Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

[NPoS] Implements dynamic number of nominators #12970

Merged
merged 96 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
8cb3404
Implements dynamic nominations per nominator
gpestana Dec 16, 2022
c9b8a17
Adds SnapshotBounds and ElectionSizeTracker
gpestana Dec 19, 2022
fcc7657
Changes the ElectionDataProvider interface to receive ElectionBounds …
gpestana Dec 19, 2022
3ae3e29
Implements get_npos_voters with ElectionBounds
gpestana Dec 19, 2022
98f6e82
Implements get_npos_targets with ElectionBounds
gpestana Dec 19, 2022
5434b47
Adds comments
gpestana Dec 19, 2022
0a7b714
tests
gpestana Dec 21, 2022
1c347f9
Truncates nomninations that exceed nominations quota; Old tests passing
gpestana Dec 26, 2022
9bf66c3
Uses DataProviderBounds and ElectionBounds (to continue)
gpestana Dec 27, 2022
cdaeb7c
Finishes conversions - tests passing
gpestana Dec 27, 2022
4e38839
Refactor staking in babe mocks
gpestana Dec 27, 2022
11bd646
Replaces MaxElectableTargets and MaxElectingVoters with ElectionBound…
gpestana Dec 27, 2022
3fe1076
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Dec 27, 2022
a1f77ad
Fixes nits; node compiling
gpestana Dec 28, 2022
905b8e0
bechmarks
gpestana Dec 29, 2022
775fe25
removes nomination_quota extrinsic to request the nomination quota
gpestana Jan 3, 2023
eb2456b
Lazy quota check, ie. at nominate time only
gpestana Jan 3, 2023
b0b21a4
remove non-working test (for now)
gpestana Jan 3, 2023
82ac2db
tests lazy nominations quota when quota is lower than current number …
gpestana Jan 4, 2023
8218e17
Adds runtime API and custom RPC call for clients to query the nominat…
gpestana Jan 4, 2023
af0c52b
removes old rpc
gpestana Feb 28, 2023
5b5231a
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Feb 28, 2023
8e7ecbb
Cosmetic touches
gpestana Feb 28, 2023
bfbb91b
All mocks working
gpestana Feb 28, 2023
2c44857
Fixes benchmarking mocks
gpestana Feb 28, 2023
1bf29f7
nits
gpestana Feb 28, 2023
125ff54
more tests
gpestana Mar 1, 2023
7d9fdfc
renames trait methods
gpestana Mar 1, 2023
8fddf41
nit
gpestana Mar 1, 2023
805e3f7
".git/.scripts/commands/fmt/fmt.sh"
Mar 1, 2023
3317bd7
Fix V2 PoV benchmarking (#13485)
ggwpez Feb 28, 2023
1c20979
Move BEEFY code to consensus (#13484)
davxy Feb 28, 2023
8ba9e1e
chore: move genesis block builder to chain-spec crate. (#13427)
yjhmelody Feb 28, 2023
761d7ae
Speed up storage iteration from within the runtime (#13479)
koute Mar 1, 2023
3136966
Make unbounded channels size warning exact (part 1) (#13490)
dmitry-markin Mar 1, 2023
8d78bee
Removal of Prometheus alerting rules deployment in cloud-infra (#13499)
lazam Mar 1, 2023
397ce89
sp-consensus: remove unused error variants (#13495)
andresilva Mar 1, 2023
1349e6a
Expose `ChargedAmount` (#13488)
pmikolajczyk41 Mar 1, 2023
92b1231
sc-consensus-beefy: fix metrics: use correct names (#13494)
acatangiu Mar 1, 2023
42eeb97
clippy fix
gpestana Mar 1, 2023
5a29c9c
Merge remote-tracking branch 'origin/master' into gpestana/staking-dy…
Mar 1, 2023
f51083b
Merge remote-tracking branch 'origin/master' into gpestana/staking-dy…
Mar 6, 2023
d067a1c
removes NominationsQuotaExceeded event
gpestana Mar 6, 2023
422393c
Update frame/staking/src/lib.rs
gpestana Mar 16, 2023
fb418a9
adds back the npos_max_iter
gpestana Mar 16, 2023
b786d33
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Mar 16, 2023
af3b8ba
remove duplicate imports added after merge
gpestana Mar 16, 2023
7d8ffda
fmt
gpestana Mar 16, 2023
1246c73
Adds comment in public struct; Refactors CountBound and SizeCount to …
gpestana Mar 21, 2023
325c91f
addresses various pr comments
gpestana Mar 21, 2023
1f8f502
PR comment reviews
gpestana Mar 23, 2023
67c4734
Fixes on-chain election bounds and related code
gpestana Mar 24, 2023
70d8579
EPM checks the size of the voter list returned by the data provider
gpestana Mar 24, 2023
a68a37d
cosmetic changes
gpestana Mar 24, 2023
849a2e0
updates e2e tests mock
gpestana Mar 24, 2023
306668e
Adds more tests for size tracker and refactors code
gpestana Mar 25, 2023
7ec6305
Adds back only_iterates_max_2_times_max_allowed_len test
gpestana Mar 25, 2023
e8576b7
Refactor
gpestana Mar 25, 2023
e12f287
removes unecessary dependency
gpestana Mar 25, 2023
c7acde4
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Mar 25, 2023
61a8ca5
empty commit -- restart all stuck CI jobs
gpestana Mar 27, 2023
5c8a034
restarts ci jobs
gpestana Mar 31, 2023
187ce07
Renames ElectionBounds -> Bounds in benchmarking mocks et al
gpestana Apr 2, 2023
9023997
updates mocks
gpestana Apr 2, 2023
ee5c863
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Apr 2, 2023
c3b4375
Update frame/election-provider-support/src/lib.rs
gpestana Apr 11, 2023
9f0319d
Update frame/staking/src/pallet/impls.rs
gpestana Apr 11, 2023
bece9d5
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Apr 11, 2023
8bd5ce9
Update frame/election-provider-support/src/lib.rs
gpestana Apr 11, 2023
7970ade
Update frame/staking/src/tests.rs
gpestana Apr 11, 2023
208ee20
more checks in api_nominations_quota in tests
gpestana Apr 11, 2023
6c5b3b1
Improves docs
gpestana Apr 11, 2023
6619b5d
fixes e2e tests
gpestana Apr 12, 2023
02d8e78
Uses size_hint rather than mem::size_of in size tracker; Refactor siz…
gpestana Apr 12, 2023
4805516
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Apr 12, 2023
d18b175
nits from reviews
gpestana Apr 13, 2023
31ed378
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Apr 19, 2023
4769857
Refactors bounds to own module; improves docs
gpestana Apr 19, 2023
a5cd416
More tests and docs
gpestana Apr 19, 2023
742f07d
fixes docs
gpestana Apr 19, 2023
fe61184
Fixes benchmarks
gpestana Apr 20, 2023
87945f6
Fixes rust docs
gpestana Apr 20, 2023
75be862
fixes bags-list remote-ext-tests
gpestana Apr 20, 2023
b3aab85
Simplify bound checks in create_snapshot_external
gpestana Apr 20, 2023
bc0e8bd
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana May 17, 2023
aab74d6
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Jun 3, 2023
fe2dfe3
Adds target size check in get_npos_targets
gpestana Jun 4, 2023
491ddfc
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Aug 5, 2023
8bfd731
".git/.scripts/commands/fmt/fmt.sh"
Aug 5, 2023
747aa52
restart ci
gpestana Aug 5, 2023
0bda7a4
rust doc fixes and cosmetic nits
gpestana Aug 6, 2023
157e3a7
Merge remote-tracking branch 'origin/master' into gpestana/staking-dy…
Aug 8, 2023
3c3fc5a
Merge remote-tracking branch 'origin/master' into gpestana/staking-dy…
Aug 9, 2023
86ab41c
rollback upgrade on parity-scale-codec version (unecessary)
gpestana Aug 9, 2023
e4f7f9f
reset cargo lock, no need to update it
gpestana Aug 9, 2023
07c4a7d
Merge branch 'master' into gpestana/staking-dynamic-nominators
gpestana Aug 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions bin/node/runtime/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@

use codec::{Decode, Encode, MaxEncodedLen};
use frame_election_provider_support::{
bounds::{ElectionBounds, ElectionBoundsBuilder},
onchain, BalancingConfig, ElectionDataProvider, SequentialPhragmen, VoteWeight,
};
use frame_support::{
Expand Down Expand Up @@ -562,14 +563,16 @@ parameter_types! {
pub HistoryDepth: u32 = 84;
}

/// Upper limit on the number of NPOS nominations.
const MAX_QUOTA_NOMINATIONS: u32 = 16;
gpestana marked this conversation as resolved.
Show resolved Hide resolved

pub struct StakingBenchmarkingConfig;
impl pallet_staking::BenchmarkingConfig for StakingBenchmarkingConfig {
type MaxNominators = ConstU32<1000>;
type MaxValidators = ConstU32<1000>;
}

impl pallet_staking::Config for Runtime {
type MaxNominations = MaxNominations;
type Currency = Balances;
type CurrencyBalance = Balance;
type UnixTime = Timestamp;
Expand All @@ -594,6 +597,7 @@ impl pallet_staking::Config for Runtime {
type ElectionProvider = ElectionProviderMultiPhase;
type GenesisElectionProvider = onchain::OnChainExecution<OnChainSeqPhragmen>;
type VoterList = VoterList;
type NominationsQuota = pallet_staking::FixedNominationsQuota<MAX_QUOTA_NOMINATIONS>;
// This a placeholder, to be introduced in the next PR as an instance of bags-list
type TargetList = pallet_staking::UseValidatorsMap<Self>;
type MaxUnlockingChunks = ConstU32<32>;
Expand Down Expand Up @@ -647,17 +651,20 @@ frame_election_provider_support::generate_solution_type!(
VoterIndex = u32,
TargetIndex = u16,
Accuracy = sp_runtime::PerU16,
MaxVoters = MaxElectingVoters,
MaxVoters = MaxElectingVotersSolution,
>(16)
gpestana marked this conversation as resolved.
Show resolved Hide resolved
);

parameter_types! {
// Note: the EPM in this runtime runs the election on-chain. The election bounds must be
// carefully set so that an election round fits in one block.
pub ElectionBoundsMultiPhase: ElectionBounds = ElectionBoundsBuilder::default()
.voters_count(10_000.into()).targets_count(1_500.into()).build();
pub ElectionBoundsOnChain: ElectionBounds = ElectionBoundsBuilder::default()
.voters_count(5_000.into()).targets_count(1_250.into()).build();

pub MaxNominations: u32 = <NposSolution16 as frame_election_provider_support::NposSolution>::LIMIT as u32;
pub MaxElectingVoters: u32 = 40_000;
pub MaxElectableTargets: u16 = 10_000;
// OnChain values are lower.
pub MaxOnChainElectingVoters: u32 = 5000;
pub MaxOnChainElectableTargets: u16 = 1250;
pub MaxElectingVotersSolution: u32 = 40_000;
// The maximum winners that can be elected by the Election pallet which is equivalent to the
// maximum active validators the staking pallet can have.
pub MaxActiveValidators: u32 = 1000;
Expand Down Expand Up @@ -712,8 +719,7 @@ impl onchain::Config for OnChainSeqPhragmen {
type DataProvider = <Runtime as pallet_election_provider_multi_phase::Config>::DataProvider;
type WeightInfo = frame_election_provider_support::weights::SubstrateWeight<Runtime>;
type MaxWinners = <Runtime as pallet_election_provider_multi_phase::Config>::MaxWinners;
type VotersBound = MaxOnChainElectingVoters;
type TargetsBound = MaxOnChainElectableTargets;
type Bounds = ElectionBoundsOnChain;
}

impl pallet_election_provider_multi_phase::MinerConfig for Runtime {
Expand Down Expand Up @@ -761,9 +767,8 @@ impl pallet_election_provider_multi_phase::Config for Runtime {
type GovernanceFallback = onchain::OnChainExecution<OnChainSeqPhragmen>;
type Solver = SequentialPhragmen<AccountId, SolutionAccuracyOf<Self>, OffchainRandomBalancing>;
type ForceOrigin = EnsureRootOrHalfCouncil;
type MaxElectableTargets = MaxElectableTargets;
type MaxWinners = MaxActiveValidators;
type MaxElectingVoters = MaxElectingVoters;
type ElectionBounds = ElectionBoundsMultiPhase;
type BenchmarkingConfig = ElectionProviderBenchmarkConfig;
type WeightInfo = pallet_election_provider_multi_phase::weights::SubstrateWeight<Self>;
}
Expand Down
12 changes: 8 additions & 4 deletions frame/babe/src/mock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,16 @@

use crate::{self as pallet_babe, Config, CurrentSlot};
use codec::Encode;
use frame_election_provider_support::{onchain, SequentialPhragmen};
use frame_election_provider_support::{
bounds::{ElectionBounds, ElectionBoundsBuilder},
onchain, SequentialPhragmen,
};
use frame_support::{
parameter_types,
traits::{ConstU128, ConstU32, ConstU64, KeyOwnerProofSystem, OnInitialize},
};
use pallet_session::historical as pallet_session_historical;
use pallet_staking::FixedNominationsQuota;
use sp_consensus_babe::{AuthorityId, AuthorityPair, Randomness, Slot, VrfSignature};
use sp_core::{
crypto::{KeyTypeId, Pair, VrfSecret},
Expand Down Expand Up @@ -161,6 +165,7 @@ parameter_types! {
pub const SlashDeferDuration: EraIndex = 0;
pub const RewardCurve: &'static PiecewiseLinear<'static> = &REWARD_CURVE;
pub const OffendingValidatorsThreshold: Perbill = Perbill::from_percent(16);
pub static ElectionsBounds: ElectionBounds = ElectionBoundsBuilder::default().build();
}

pub struct OnChainSeqPhragmen;
Expand All @@ -170,12 +175,10 @@ impl onchain::Config for OnChainSeqPhragmen {
type DataProvider = Staking;
type WeightInfo = ();
type MaxWinners = ConstU32<100>;
type VotersBound = ConstU32<{ u32::MAX }>;
type TargetsBound = ConstU32<{ u32::MAX }>;
type Bounds = ElectionsBounds;
}

impl pallet_staking::Config for Test {
type MaxNominations = ConstU32<16>;
type RewardRemainder = ();
type CurrencyToVote = ();
type RuntimeEvent = RuntimeEvent;
Expand All @@ -197,6 +200,7 @@ impl pallet_staking::Config for Test {
type GenesisElectionProvider = Self::ElectionProvider;
type VoterList = pallet_staking::UseNominatorsAndValidatorsMap<Self>;
type TargetList = pallet_staking::UseValidatorsMap<Self>;
type NominationsQuota = FixedNominationsQuota<16>;
type MaxUnlockingChunks = ConstU32<32>;
type HistoryDepth = ConstU32<84>;
type EventListeners = ();
Expand Down
12 changes: 10 additions & 2 deletions frame/bags-list/remote-tests/src/snapshot.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@

//! Test to execute the snapshot using the voter bag.

use frame_election_provider_support::SortedListProvider;
use frame_election_provider_support::{
bounds::{CountBound, DataProviderBounds},
SortedListProvider,
};
use frame_support::traits::PalletInfoAccess;
use remote_externalities::{Builder, Mode, OnlineConfig};
use sp_runtime::{traits::Block as BlockT, DeserializeOwned};
Expand Down Expand Up @@ -62,8 +65,13 @@ where
<Runtime as pallet_staking::Config>::VoterList::count(),
);

let bounds = match voter_limit {
None => DataProviderBounds::default(),
Some(v) => DataProviderBounds { count: Some(CountBound(v as u32)), size: None },
};

let voters =
<pallet_staking::Pallet<Runtime> as ElectionDataProvider>::electing_voters(voter_limit)
<pallet_staking::Pallet<Runtime> as ElectionDataProvider>::electing_voters(bounds)
.unwrap();

let mut voters_nominator_only = voters
Expand Down
11 changes: 7 additions & 4 deletions frame/beefy/src/mock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@

use std::vec;

use frame_election_provider_support::{onchain, SequentialPhragmen};
use frame_election_provider_support::{
bounds::{ElectionBounds, ElectionBoundsBuilder},
onchain, SequentialPhragmen,
};
use frame_support::{
construct_runtime, parameter_types,
sp_io::TestExternalities,
Expand Down Expand Up @@ -184,6 +187,7 @@ parameter_types! {
pub const BondingDuration: EraIndex = 3;
pub const RewardCurve: &'static PiecewiseLinear<'static> = &REWARD_CURVE;
pub const OffendingValidatorsThreshold: Perbill = Perbill::from_percent(17);
pub static ElectionsBoundsOnChain: ElectionBounds = ElectionBoundsBuilder::default().build();
}

pub struct OnChainSeqPhragmen;
Expand All @@ -193,12 +197,10 @@ impl onchain::Config for OnChainSeqPhragmen {
type DataProvider = Staking;
type WeightInfo = ();
type MaxWinners = ConstU32<100>;
type VotersBound = ConstU32<{ u32::MAX }>;
type TargetsBound = ConstU32<{ u32::MAX }>;
type Bounds = ElectionsBoundsOnChain;
}

impl pallet_staking::Config for Test {
type MaxNominations = ConstU32<16>;
type RewardRemainder = ();
type CurrencyToVote = ();
type RuntimeEvent = RuntimeEvent;
Expand All @@ -220,6 +222,7 @@ impl pallet_staking::Config for Test {
type GenesisElectionProvider = Self::ElectionProvider;
type VoterList = pallet_staking::UseNominatorsAndValidatorsMap<Self>;
type TargetList = pallet_staking::UseValidatorsMap<Self>;
type NominationsQuota = pallet_staking::FixedNominationsQuota<16>;
type MaxUnlockingChunks = ConstU32<32>;
type HistoryDepth = ConstU32<84>;
type EventListeners = ();
Expand Down
6 changes: 4 additions & 2 deletions frame/election-provider-multi-phase/src/benchmarking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
use super::*;
use crate::{unsigned::IndexAssignmentOf, Pallet as MultiPhase};
use frame_benchmarking::account;
use frame_election_provider_support::bounds::DataProviderBounds;
use frame_support::{
assert_ok,
traits::{Hooks, TryCollect},
Expand Down Expand Up @@ -270,8 +271,9 @@ frame_benchmarking::benchmarks! {

// we don't directly need the data-provider to be populated, but it is just easy to use it.
set_up_data_provider::<T>(v, t);
let targets = T::DataProvider::electable_targets(None)?;
let voters = T::DataProvider::electing_voters(None)?;
// default bounds are unbounded.
let targets = T::DataProvider::electable_targets(DataProviderBounds::default())?;
let voters = T::DataProvider::electing_voters(DataProviderBounds::default())?;
let desired_targets = T::DataProvider::desired_targets()?;
assert!(<MultiPhase<T>>::snapshot().is_none());
}: {
Expand Down
Loading