This repository has been archived by the owner on Nov 15, 2023. It is now read-only.
Refactored block body database scheme #10779
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
arkpar
added
B5-clientnoteworthy
C1-low
This was referenced Feb 2, 2022
cheme
reviewed
Feb 3, 2022
There was a problem hiding this comment.
Looks like a really good change.
IIReadC this change set, the storage chain mode was merely enforcing fast sync, and removing this explicit mode makes sense. (also having rc on the transaction column 👍 )
Got a few small question after first reading (in comments).
cheme
approved these changes
Feb 3, 2022
Without having reviewed the pr. Why do we not just have a |
|
@bkchr |
Ahh makes sense 🤦 I was confused because of "The headers are written to the BODY_INDEX column". Do you mean something different here with "headers"?
Why can we not "unify" this? Aka write a migration to move everything over to "one" format. Like 2 columns, BODY and TRANSACTIONS. While BODY only holds "references" (probably hashes) of the transactions? |
|
Right, headers refer to extrinsic header. Each extrinsic may have an indexed "transaction" portion. The rest is considered to be a header. Current format makes more sense when the blockchain does not have that many indexed extrinsics. Storing each extrinsic as a separate database entry will increase space requirements for e.g. kusama by quite a bit. That's why the new encoding scheme is only used for blocks that actually have indexed content. The new encoding format actually allows for each individual extrinsic to be either embedded in the |
Merged
bkchr
approved these changes
Mar 3, 2022
client/db/src/parity_db.rs
Outdated
| }, | ||
| } | ||
|
|
||
| if upgrade { | ||
| log::info!("Upgrading metadata"); |
There was a problem hiding this comment.
Do we really want this to be an info log?
If yes, we should probably make the message more expressive.
| @@ -38,20 +38,22 @@ pub fn open<H: Clone + AsRef<[u8]>>( | |||
| path: &std::path::Path, | |||
| db_type: DatabaseType, | |||
| create: bool, | |||
| upgrade: bool, | |||
There was a problem hiding this comment.
I would propose some simple enum:
enum Upgrade { Yes, No }
to make it a little bit more expressive.
client/db/src/lib.rs
Outdated
| }, | ||
| Err(err) => | ||
| return Err(sp_blockchain::Error::Backend(format!( |
There was a problem hiding this comment.
Suggested change
| return Err(sp_blockchain::Error::Backend(format!( | |
| Err(sp_blockchain::Error::Backend(format!( |
|
This bumps db version implying one-way db migration. Does that imply |
grishasobol
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Mar 28, 2022
* Refactored tx storage database scheme * Bump parity-db * fmt * Fix handling invalid index size + test * Removed superflous result * Minor changes * fmt
grishasobol
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Mar 28, 2022
* Refactored tx storage database scheme * Bump parity-db * fmt * Fix handling invalid index size + test * Removed superflous result * Minor changes * fmt
grishasobol
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Mar 28, 2022
* Refactored tx storage database scheme * Bump parity-db * fmt * Fix handling invalid index size + test * Removed superflous result * Minor changes * fmt
10 tasks
8 tasks
driemworks
added a commit
to ideal-lab5/substrate-offchain-ipfs
that referenced
this pull request
May 9, 2022
* Make some UI test expectations crisper (#10791) * Make some UI test expectations crisper * Update frame/support/test/tests/construct_runtime_ui/undefined_genesis_config_part.rs * Update UI test expectations * Missing newlines * More whitespace issues * contracts: `is_contract(address)` and `caller_is_origin()` are added to API (#10789) * is_contract() and caller_is_origin() added to Ext API * is_contract() exposed in wasm runtime.rs * + test for is_contract() * + seal_is_contract benchmark * caller_is_origin() exposed to wasm/runtime.rs and covered by a test * + seal_caller_is_origin benchmark * Update frame/contracts/src/exec.rs Co-authored-by: Alexander Theißen <[email protected]> * Update frame/contracts/src/exec.rs Co-authored-by: Alexander Theißen <[email protected]> * Update frame/contracts/src/exec.rs Co-authored-by: Alexander Theißen <[email protected]> * Update frame/contracts/src/wasm/runtime.rs Co-authored-by: Alexander Theißen <[email protected]> * Update frame/contracts/src/wasm/runtime.rs Co-authored-by: Alexander Theißen <[email protected]> * Update frame/contracts/src/wasm/runtime.rs Co-authored-by: Alexander Theißen <[email protected]> * Update frame/contracts/src/exec.rs Co-authored-by: Alexander Theißen <[email protected]> * identation fix for benchmark macroses; test cosmetic improvement * benchmark fix * + is_contract() wasm test * + caller_is_origin() wasm test * Apply suggestions from code review Co-authored-by: Alexander Theißen <[email protected]> * is_contract() to borrow param instead of taking ownership * phrasing improved Co-authored-by: Hernando Castano <[email protected]> * fixed wasm tests according to @athei feedback * dead code warnings suppressed by unstable-interface attributes * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs Co-authored-by: Alexander Theißen <[email protected]> Co-authored-by: Hernando Castano <[email protected]> Co-authored-by: Parity Bot <[email protected]> * maybe_total_supply in pallet-assets (#10799) * maybe_total_supply in pallet-assets * Update frame/assets/src/functions.rs Co-authored-by: Bastian Köcher <[email protected]> * Referenda and Conviction Voting pallets (#10195) * Initial draft of new referendum state machine. * Docs * Fixes * Fixes * Add conviction-voting pallet * Basic build * Building * Some TODOs * Tests building * Add missing file * Basic lifecycle test * Add couple of tests * Another test * More tests * Fixes * Fixes * Formatting * Fixes * Tests * Fixes * Fixes * More tests * Formatting * First few benchmarks * First few benchmarks * Defered queue servicing * More testing * Benchmarks * Fiddly benchmark * Final nudge benchmarks * Formatting * Formatting * Finished up benchmarks * cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_referenda --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/referenda/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Events finished * Missing file * No GenesisConfig for Referenda * Formatting * Docs * Docs * Docs * Per-class conviction voting * New test & mock utils * More tests * Tests * Tests finished 🎉 * Benchmarking stuff * Fixes * Test harness * Test harness * Benchmarks for Conviction=Voting * Benchmarking pipeline complete * Docs * Formatting * Remove unneeded warning * Fix UI tests * cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_conviction_voting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/conviction-voting/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Docs * Update frame/conviction-voting/src/vote.rs Co-authored-by: Shawn Tabrizi <[email protected]> * update sp-runtime version * MEL Fixes for Referenda and Conviction Voting (#10725) * free maxencodedlen * more maxencodedlen * more MEL * more mel * disable storage info * More Referenda Patches (#10760) * basic fixes * fix benchmarking * fix license * prevent panic in curve math * fmt * bump crate versions * Update mock.rs Co-authored-by: Parity Bot <[email protected]> Co-authored-by: Shawn Tabrizi <[email protected]> * More efficient WASM instance memory decommit on macos (#10801) * More efficient WASM instance memory decommit on macos * Apply suggestions from code review Co-authored-by: Alexander Theißen <[email protected]> * Updated error message Co-authored-by: Alexander Theißen <[email protected]> * make inner field of IdentityFields pub (#10773) * Corrected description of clear_attribute extrinsic (#10729) * block validators after X sessions, update mock runtime for test * sp-maybe-compressed-blob: reduce boilerplate code (#10814) Signed-off-by: koushiro <[email protected]> * `seal_delegate_call` api function (support for library contracts) (#10617) * seal_call_code implementation - tests - benchmark * Addressing @xgreenx's comments * Fix test-linux-stable-int * Rename seal_call_code to seal_delegate_call * Pass value unchanged into lib contract * Address @athei's comments - whitespace .wat issues - wrong/missing .wat comments - redundant .wat calls/declarations - change order of functions (seal_delegate_call right after seal_call) in decls, tests, benchmark - fix comments, move doc comments to enum variants - remove unnecessary empty lines - rename runtime cost DelegateCall to DelegateCallBase - do not set CallFlags::ALLOW_REENTRY for delegate_call * Do not pass CallFlags::ALLOWS_REENTRY for delegate_call * Update comment for seal_delegate_call and CallFlags * Addressing @athei's comments (minor) * Allow reentry for a new frame after delegate_call (revert) * Same seal_caller and seal_value_transferred for lib contract - test - refactor frame args due to review - logic for seal_caller (please review) * Put caller on frame for delegate_call, minor fixes * Update comment for delegate_call * Addressing @athei's comments * Update weights generated by benchmark * Improve comments * Address @HCastano's comments * Update weights, thanks @joao-paulo-parity * Improve InvalidCallFlags error comment * Add Storage Info to Various Pallets (#10810) * atomic swap * bounties * bounties fmt * gilt * indices * nicks * randomness-collective-flip * recovery * reuse maxapprovals * Update tests.rs * Update frame/randomness-collective-flip/src/lib.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * use the correct bound * complete recovery * use `bounded_vec` macro * Update tests.rs * transaction payment * uniques * mmr * example offchain worker * beefy-mmr * Update frame/recovery/src/lib.rs Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Use BoundedVec instead of a type-parameterized BoundedString * cargo fmt * Update frame/atomic-swap/src/lib.rs * use config const * Update lib.rs * update mel_bound * fmt Co-authored-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Keith Yeung <[email protected]> * fixed regression in inline docs (#10819) * try-runtime-cli: Add to docs (#10822) * try-runtime-cli: Add to docs of pre/post hooks * Add note about --ws-max-out-buffer-capacity 1000 * Bump serde from 1.0.132 to 1.0.136 (#10816) Bumps [serde](https://github.com/serde-rs/serde) from 1.0.132 to 1.0.136. - [Release notes](https://github.com/serde-rs/serde/releases) - [Commits](https://github.com/serde-rs/serde/compare/v1.0.132...v1.0.136) --- updated-dependencies: - dependency-name: serde dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * uodate tests, ipfs tests failing * --dev implies --tmp (#10828) * Add a new host function for reporting fatal errors; make WASM backtraces readable when printing out errors (#10741) * Add a new host function for reporting fatal errors * Fix one of the wasmtime executor tests * Have `#[runtime_interface(wasm_only)]` actually mean WASM-only, and not no_std-only * Print out errors through `Display` instead of `Debug` * Switch one more trait to require `Error` for its error instead of only `Debug` * Align to review comments * specify ipfs version, update tests * Add db params for export-state CLI command (#10830) * update tests, cleanup * Allow `SetBalance` to handle error when trying to kill acount with reference counter. (#10826) * bug found * fix logic * a little simpler * add test * UncheckedExtrinsic: Harden decode and clarify `EXTRINSIC_FORMAT_VERSION` (#10829) * UncheckedExtrinsic: Harden decode and clarify `EXTRINSIC_FORMAT_VERSION` * Apply suggestions from code review * fix typo * pallet-staking: Add extrinsic `force_apply_min_commission` (#10786) * pallet-staking: Add extrinsic `force_apply_min_commission` * Add benchmarks * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Bound iteration by max_validator_count * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Only apply to 1 validator * Update doc comments * Uncomment tests * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Accept signed origins * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Remove contains_key check * Add test for try_mutate_exists * Impove try_mutate_exists docs * Delete redundant try_mutate_exists tests; * Delete residual from removed test * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Return an error when the stash does not exist * Update try_mutate_exist doc wording * Update frame/staking/src/pallet/mod.rs * Apply suggestions from code review Co-authored-by: Parity Bot <[email protected]> Co-authored-by: Shawn Tabrizi <[email protected]> * Upgradable contracts using `set_code` function (#10690) * poc logic * set_code_hash impl, tests, benchmark * Address @xgreenx's comments * Move func defs closer to set_storage * Check if code exists - increment/decrement codehash refcount * Document error for non-existing code hash * Revert unrelated change * Changes due to @athei's review * Fix error handling - comment errors: ReturnCodes - update mock ext implementation - return Error::CodeNotFound when no code for such hash * Emit ContractCodeUpdated when setting new code_hash * Address @athei's comments * Move related defs to the bottom * Minor comment update Co-authored-by: Alexander Theißen <[email protected]> * Improve docs * Improve docs * Update frame/contracts/src/wasm/runtime.rs Co-authored-by: Alexander Theißen <[email protected]> * Refactor set_code_hash test * Minor change to benchmark Co-authored-by: Alexander Theißen <[email protected]> * Minor change to benchmark Co-authored-by: Alexander Theißen <[email protected]> * Minor comment refactor Co-authored-by: Alexander Theißen <[email protected]> * Address @HCastano's comments * Update seal_set_code_hash comment Co-authored-by: Hernando Castano <[email protected]> * Move set_code_hash after delegate_call * Move function to the bottom * Moved and changed banchmark, added verify block * Bring back previous benchmark * Remove skip_meta for seal_set_code_hash * Bring back skip_meta for seal_set_storage_per_new_kb * Apply weights Co-authored-by: Alexander Theißen <[email protected]> Co-authored-by: Hernando Castano <[email protected]> * Separate wasmi and wasmer sandbox implementations into their own modules (#10563) * Moves wasmi specific `ImportResolver` and `MemoryTransfer` impls to submodule * Splits context store environmental, moves impl `Externals` to wasmi backend * Adds wasmer sandbox backend stub module * Move sandbox impl code to backend specific modules * Moves wasmi stuff * Fixes value conversion * Makes it all compile * Remove `with_context_store` * Moves `WasmerBackend` to the impl * Reformat the source * Moves wasmer MemoryWrapper * Reformats the source * Fixes mutability * Moves backend impls to a submodule * Fix visibility * Reformat the source * Feature gate wasmer backend module * Moves wasmi memory allocation to backend module * Rename WasmerBackend to Backend * Refactor dispatch result decoding, get rid of Wasmi types in common sandbox code * Reformat the source * Remove redundant prefixes in backend functions * Remove wasmer-sandbox from default features * Post-review changes * Add conversion soundness proof * Remove redundant prefix * Removes now redundant clone_inner * Add `Error::SandboxBackend`, refactor invoke result * Fix comments * Rename `Error::SandboxBackend` to `Sandbox` * Simplifies logic in `wasmer_backend::invoke` * Fixes memory management * Show Network ID when creating and inspecting (#10838) Signed-off-by: Antonio Yang <[email protected]> * Reduce overhead of generating network event metrics (#10839) * Fix reentrancy of FrozenBalance::died hook (#10473) * assets: execute `died` hook outside of mutate Signed-off-by: Oliver Tale-Yazdi <[email protected]> * assets: extend tests for `died` hook Signed-off-by: Oliver Tale-Yazdi <[email protected]> * assets: update doc of FrozenBalance::died Signed-off-by: Oliver Tale-Yazdi <[email protected]> * assets: review fixes - fix cases where `died` should not have been called - use `Option<DeadConsequence>` instead of `DeadConsequence` Signed-off-by: Oliver Tale-Yazdi <[email protected]> * assets: update comment in mock.rs Signed-off-by: Oliver Tale-Yazdi <[email protected]> * assets: return `Remove` in dead_account The return value is ignored in the only case that it is produced by a call, but having it this way makes it more understandable. Signed-off-by: Oliver Tale-Yazdi <[email protected]> * allow trailing comma (#10841) * build-script-utils: allow reading the git commit hash from env var (#10845) * build-script-utils: allow reading the git commit hash from env var * build-script-utils: make the env var name substrate specific * Don't create DB leaves when syncing historic blocks (#10844) * Don't create DB leaves when syncing historic blocks * Changed leaves check and added test * fmt * cleanup, use unsigned tx when appropriate * add weights * fix broken link (#10846) * subkey: Support `--version` cli command (#10853) * subkey: Support `--version` cli command * FMT :facepalm: * Remove `u32_trait` (#10850) * Remove `u32_trait` This trait only existed because there wasn't any const generic support at time of creation. However, we now have support for it :) * FMT * wasm-builder: Support latest nightly (#10837) * wasm-builder: Support latest nightly With latest nightly, aka rust version 1.60+ namespaced features are added. This changes the handling of optional dependencies. We currently have features that enable optional dependencies when `std` is enabled. This was before no problem, but now the wasm-builder detects them as enabled. To support the transition period until 1.60 is released as stable, this pr adds an heuristic to not enable these optional crates in the wasm build when they are enabled in the `std` feature. This heuristic fails when someones enables these optional dependencies from the outside as well as via the `std` feature, however we hope that no one is doing this at the moment. When namespaced features are enabled, these dependencies needs to be enabled using `dep:dependency-name` to solve this properly. https://doc.rust-lang.org/cargo/reference/unstable.html#namespaced-features * Remove accidentally added features * update tests, remove unneeded params * min authorities check before removal * contracts: Fix `seal_call` weights (#10796) * Fix call weights * Fix instantiate benchmark * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Remove stale and superflous comments * `decrement_refcount` should be infallible * Don't hardcode increment_refcount, decrement_refcount * Rename CopyIn/CopyOut * Fix warning in tests Co-authored-by: Parity Bot <[email protected]> * Enable the `parking_lot` feature for `tokio` (#10855) * [ci] fix publish-rustdocs (#10858) * Fix beefy mock ecdsa keys (#10854) Compressed ECDSA keys requires to have 0x02 or 0x03 as their first byte in order to allow public key recovery. Nevertheless the test was working because of the `unwrap_or_default()` at the end of the conversion routine (i.e. the invalid keys were converted to an empty vector). * Mark sync_state_genSyncSpec JSON-RPC as safe (#10832) * Mark sync_state_genSyncSpec JSON-RPC as safe * Note that parameter is unused * Ideally I'd wait for compilation to be finished before pushing, but it's really taking a long time * Remove deny_unsafe parameter * Remove unused dependency * Reduce CPU overhead of gossip (#10859) * sp-core-hashing: use the `digest::Digest` trait to handle the hashing function uniformly (#10835) * sp-core-hashing: use sha3 instead of tiny-keccak Signed-off-by: koushiro <[email protected]> * use blake2 instead of blake2-rfc Signed-off-by: koushiro <[email protected]> * improve som hashing of sp-core and sp-api-proc-macro Signed-off-by: koushiro <[email protected]> * Some nits Signed-off-by: koushiro <[email protected]> * cargo fmt Signed-off-by: koushiro <[email protected]> * Use fully qualified sytnax for `retain_mut` (#10865) * add llvm (#10864) * refactor election score (#10834) * refactor election score * Test for ord * remove reference * vec -> slice * change iter to iter_by_significance * improve doc * fix typo * add explanation about [u128; 3] * consolidate threshold and epsilon * random fixes * rename * remove Into * make iter_by_sig private * remove vec * Fix tests * Measure per byte and not kb for certain benchmarks (#10863) * Tidy Democracy (#10867) * add test * Assorted refactorings * complete test * saturating math * final check * use `default` Co-authored-by: Gav Wood <[email protected]> * cleanup storage maps, cleanup unused imports * sc-executor: Improve logging (#10869) Improves the logging by switching to `tracing` for a better log output. Besides that, it also adds a trace for the function being executed. * fix test issues * Track allowed requests for state/warp sync (#10843) * Track allowed requests for state/warp sync * Added missing allowed_requests resets * Apply suggestions from code review Co-authored-by: Bastian Köcher <[email protected]> * fmt Co-authored-by: Bastian Köcher <[email protected]> * Slots: Ensure that a race betwen finalized and best number is taken care off (#10876) * Remove old lock file (#10879) * Further reduce the CPU overhead of networking metrics (#10875) * Simplify `num_connected_peers` * Track requested peer counts * Revert "Track requested peer counts" This reverts commit 9f1c8704353df6afc17ed7e9f4ab8d8e29466ae4. * Remove `substrate_sub_libp2p_peerset_num_requested` metric * Remove two unused functions that I forgot to get rid of in previous commit * Introduce `BoundedVec::iter_mut` (#10884) * Introduce iters into BoundedVec * Fix * Remove unneeded funcs * Update frame/support/src/storage/bounded_vec.rs * Update frame/support/src/storage/bounded_vec.rs Co-authored-by: Bastian Köcher <[email protected]> * Slots: Log total proposing duration as milliseconds (#10886) Parachains have currently a total proposing time of 500ms, so it this currently always prints `0`. While actually the value is not `0` ;) * staking: Clarify reward calc docs (#10890) * tracing: Adds `init_for_tests` (#10893) This function is useful for tests. It will enable `TRACE` logging and also uses the libtest aware writer. * make submissions pub (#10899) * contracts: Allow stack height metering to be disabled (#10877) * Allow stack height metering to be disabled * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs Co-authored-by: Parity Bot <[email protected]> * Rename Uniques Error::Unknown to something more sensible (#10895) * Rename Uniques Error::Unknown to something more sensible * Typos * Typos * fmt Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fix tests Signed-off-by: Oliver Tale-Yazdi <[email protected]> * fmt Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Oliver Tale-Yazdi <[email protected]> * state-machine: Move all functionality from trie backend to the essence (#10904) * state-machine: Move all functionality from trie backend to the essence This is required for some future changes of me and it also makes more sense to have all the functionality inside the essence. Besides that it changes the child root cache to directly cache the hash. * Update primitives/state-machine/src/trie_backend_essence.rs Co-authored-by: cheme <[email protected]> * FMT Co-authored-by: cheme <[email protected]> * staking: Remove `SessionInterface` supertrait (#10901) * consensus-slots: cleanup SlotDuration config (#10878) * consensus-slots: cleanup the SlotDuration config * fix tests * address review comments * add contracts pallet + rpc to runtime * Replace libsecp256k1 with k256 in FRAME related code (#10883) * Replace libsecp256k1 with k256 in beefy-mmr * Port of FRAME `contracts` benchmarking from `libsecp256k1` to `k256` * Newtype to allow `Pcg32` rng usage with `k256` in contracts benchmarks * Use `sp-io::crypto` to generate dummy keys in `contracts` bechmarks * More compact code * Cargo fmt * Build `sp-keystore` only for dev profile * Move public key generation back to the `map` * Clean obsolete BABE's weight data (#10748) * Clean obsolete BABE weight data * Take out test assertion from check closure * Optimize metadata access using `HeaderMetadata` trait * Apply suggestions from code review * Introduce finalize and import pre-commit synchronous actions * Do not hold locks between internal methods calls * Remove unused generic bound * Apply suggestions from code review * Register BABE's pre-commit actions on `block_import` instead of `start_babe` * PreCommit actions should be `Fn` instead of `FnMut` * More robust safenet in case of malformed finality notifications Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: André Silva <[email protected]> * Factor DB weights out into their own files (#10908) * Factor DB weights out into their own files Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Review fixes Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fix CI Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Weights in own mod Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Get rid of unnecessary use of `async-std` in non-test code (#10891) * No longer generate specs with consensus_engine field (#10346) * No longer generate specs with consensus_engine field * #[allow(unused)] * Upgrading parity-scale-codec to v3 (#10825) * Upgraded dependencies * Adapting code to scale v3 * Empty commit to trigger CI * Triggering CI * Fixing UI test * Remove superfluous dev-dep added by #9228 * Cryout for CI * sc-cli: Fix bugs after switching to clap3 (#10920) * sc-cli: Fix bugs after switching to clap3 Before switching to clap3 we support cli options like `--reserved-nodes A B` and after you needed to pass `--reserved-nodes` cli option multiple times `--reserved-nodes A --reserved-nodes B`. This is fixed by setting `multiple_occurrences(true)` option. This also done for all the other `Vec` cli options in `sc-cli`. Besides that `--sync` wasn't supporting case insensitive parsing of the value. This is now also supported. For both regressions a test is added. Besides that the pr removes all the `rename_all = PascalCase` attributes, because they are not needed. All other `ArgEnum`s were checked and all are already using `ignore_case(true)`. * Bring back `PascalCase`, because otherwise it falls back to `kebab-case`... * Storage benchmarking (#10897) * WIP Signed-off-by: Oliver Tale-Yazdi <[email protected]> * WIP: DB benchmarking Signed-off-by: Oliver Tale-Yazdi <[email protected]> * WIP Signed-off-by: Oliver Tale-Yazdi <[email protected]> * WIP Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Simplify code Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Remove old files Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Remove old files Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Minimize changes Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Add license Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Remove dependencies Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Extend template Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Linter Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Linter Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Beauty fixes Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Remove default Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Add feature Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Remove seed Signed-off-by: Oliver Tale-Yazdi <[email protected]> * CI wakeup Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fmt Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Review fixes Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Adding doc Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Adding doc Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Improve template Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Do not expose columns Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fix ColumnId Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Nicer template prints Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Cleanup Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fix json path Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Simplify `bench_write` logic Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Invert tx before the second commit Signed-off-by: Oliver Tale-Yazdi <[email protected]> * add transfer assets extrinsic * add chain ext * Minor improvements to `bounded_vec` and `defensive`. (#10873) * Fix a few things in bounded_vec * add test for try_extend * Update frame/support/src/storage/bounded_vec.rs Co-authored-by: Bastian Köcher <[email protected]> * some review comments * use swap * remove clone * use pop instead of truncate * remove warn * review comments * Update frame/support/src/storage/bounded_vec.rs Co-authored-by: Bastian Köcher <[email protected]> * fix rustdoc * fix links * undo link Co-authored-by: Bastian Köcher <[email protected]> * Replace libsecp256k1 with secp256k1 (#10798) * Replace libsecp256k1 with secp256k1 * Wipe ecdsa secret key from memory on drop * Some comments for a known issue * Safer core crypto primitives `from_slice` constructor Previous version panics if slice lenght is not the expected one. * Unit test fix * Enable use of global secp256k1 context * Better comments for ecdsa `Pair` drop * Replace `libsecp256k1` with `seco256k1` in `beefy-mmr` Used to convert ecdsa public key to ETH address * Replace `libsecp256k1` with `secp256k1` in FRAME `contracts`benchmarks * Temporary rollback of `beefy-mmr` to libsecp256k1 Check for detected build issues * Cargo fmt * Rollback of FRAME `contracts` benchmarks to `libsecp256k1` * Rollback for unrelated changes * Typo fix * Add comments for deprecated `ecdsa_verify` and `secp256k1_ecdsa_recover` * Integrate try-runtime into substrate node template (#10909) * [10892-integrate-try-runtime-into-node-template] - Integrated try-runtime into node template * [10892-integrate-try-runtime-into-node-template] Added match arms for try-runtime in command.rs * [10892-integrate-try-runtime-into-node-template] Added match arms for try-runtime in command.rs * Added feature flag for try-runtime in node-template/node and enabled try-runtime for node-template/runtime * Added missing type annotations for try-runtime SubCommand in node-template * Added missing type annotations for try-runtime SubCommand in node-template * Implemented frame_try_runtime::TryRuntime<Block> for the node-template Runtime * doc corrections (#10936) * #10576: generic utility to unsubscribe from broadcast upon drop of the rx-side. (#10708) * #10576: refactor `sc-utils::notification` and `sc-client-api::notifications`, so that they use common subscribe/unsubscribe routines * Add some docs. Reorganise `sc-utils::notification` * `sc-clent-api::notifications` and `sc-utils::notification` — ensure the SubscriptionGuard is dropped before the Rx-channel * `sc-utils::pubsub::SubscriptionGuard` make it a bit more ergonomic. Let the `Rx` to be put inside of the `SubscriptionGuard`, so that the latter shall guarantee the order: - first unsubscribe; - then drop the `Rx`. * Being less zealous with splitting the modules into little pieces * rework pubsub: the concrete usage should only define a good registry type * sc-client-api::notifications: make it comply with the reworked pubsub * cargo fmt * make sc-client-api tests work * Address the review notes * cargo fmt * Describe the behaviour of pubsub registry * Doc-comments for module `sc-utils::pubsub` * Fix: it used to send notifications regardless of the filter setup during subscription * `sc-client-api::StorageNotifications` the API does not have to require mut-self-reference. As a result `sc-service::Client` does not have to wrap its `storage_notifications` into a Mutex. * cargo fmt * Several changes addressing the notes by @bckhr. - Remove the `impl Default for StorageNotifications<Block>`; - no need for groupping the `remove_from` and `listen_from` into a separate `helpers` module; - remove unnecessary import `use registry::SubscribeOp`. * Add a doc-comment to the `sc-client::notifications::SubscribeOp` * As per @bkchr note on the unproven assertion: behave gracefully upon receiving a duplicate subscription-ID. * sc-utils::pubsub: log when a registry yields an ID that does point to an existing sink * `sc-utils::notifications`: payload materialized lazily * Update Cargo.lock (after adding `log` as a dependency to the `sc-utils`) * `sc-client-api::notifications`: introduce a struct (instead of a type def) for the notification message * Get rid of `sc-utils::pubsub::Channel` trait (instead just use the `sc-utils::mpsc`) * The SubsID is no more generic: the fact it is a `Copy` is known — no need to pass it by ref * sc-utils::pubsub internals do not have to be generic over the channel type * Rename Hub::dispatch into Hub::send * That method was unnecessary (`SubscriberSink::render_notification`) * cargo fmt * No need for a separate UnsubscribeGuard type * Ditch the type-def of SubsID in the sc-utils::pubsub, instead — just use the crate::id_sequence::SeqID * Return the <Registry as Dispatch>::Ret when sending an item * Make the `Hub<M, R>::lock_registry(...)` method more ergonomic * cargo doc links * cargo doc links * Use a simpler name for the type * cargo doc links * Derive `Default` rather than implement it * Derive `Default` rather than implement it * Remove an unnecessary usage of type_name * Define a more cautious order between sinks.remove->registry.unsubscribe and registry.subscribe->sinks.insert * Hub: lock_registry_for_tests->map_registry_for_tests — a safer choice for a public API * Replace Mutex over the shared Registry with a ReentrableMutex+RefCell * sc-utils::pubsub: add tests for a panicking registry * Add the missing copyright headers * Arc<Vec<_>> -> Arc<[_]> * Bring back MaxNominations as a metadata constant (#10947) * chain ext: parse params and call transfer func * Bump futures from 0.3.16 to 0.3.19 (#10930) Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.19. - [Release notes](https://github.com/rust-lang/futures-rs/releases) - [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md) - [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.19) --- updated-dependencies: - dependency-name: futures dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Clean up extra_constant renaming. (#10935) Co-authored-by: Xiankun Cheng <[email protected]> * Bump smallvec from 1.7.0 to 1.8.0 (#10949) Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.7.0 to 1.8.0. - [Release notes](https://github.com/servo/rust-smallvec/releases) - [Commits](https://github.com/servo/rust-smallvec/compare/v1.7.0...v1.8.0) --- updated-dependencies: - dependency-name: smallvec dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * new pallet: whitelist pallet (#10159) * pallet whitelist * refactor a bit * fmt * address audit * improve tests * return Ok + refund * add test for dispatching failing * add dispatch_whitelisted_call_with_preimage * fmt * better name * Consume all data on decode Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Add error docs Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Remove phantom data Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Use rust 2021 Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Update crate features Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Fmt Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Make compile Signed-off-by: Oliver Tale-Yazdi <[email protected]> * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Bump Preimage max size Signed-off-by: Oliver Tale-Yazdi <[email protected]> * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_whitelist --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/whitelist/src/weights.rs --template=./.maintain/frame-weight-template.hbs * Fmt Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Parity Bot <[email protected]> * Release primitive crates (#10937) * cargo set-version --bump major -p pallet-contracts-primitives * cargo set-version --bump major -p sp-core * cargo set-version --bump major -p sp-runtime-interface * cargo set-version --bump major -p sp-wasm-interface * cargo set-version --bump major -p sp-runtime * cargo set-version --bump major -p sp-storage * cargo set-version --bump major -p sp-rpc * cargo set-version --bump major -p sp-io * cargo set-version --bump major -p sp-trie * cargo set-version -p sp-state-machine -- 0.12.0 * cargo set-version -p sp-externalities -- 0.12.0 * cargo set-version -p sp-keystore -- 0.12.0 * cargo set-version --bump major -p sp-keyring * cargo set-version --bump major -p sp-version * cargo set-version --bump major -p sp-tracing * cargo set-version --bump major -p sp-application-crypto * cargo set-version --bump major -p sp-arithmetic * cargo unleash version bump-major -p sp-runtime-interface-proc-macro * Add codec max-encoded-len feature to sp-arithmetic * cargo unleash version bump-major -p sp-core-hashing-proc-macro * Update new whitelist pallet to new primitive versions (#10953) * sp-trie: Switch to thiserror and some other small cleanups (#10954) * sp-trie: Switch to thiserror and some other small cleanups * Add some extra method for converting a compact proof to a memory db * Only maintain at most 1 `UnlockChunk` per era (#10670) * Only maintain at most 1 `UnlockChunk` per era * Bound `unlocking` * Run cargo +nightly-2021-10-29 fmt * Make benchmarks stuff compile * Update frame/staking/src/lib.rs Co-authored-by: Kian Paimani <[email protected]> * Remove DerefMut; Implement neccesary methods directly * Doc comments for new BoundedVec methods * Fix benchmarks * wip bonded_vec macro * Correct rust doc * Apply suggestions from code review Co-authored-by: Kian Paimani <[email protected]> * Update staking::Config impls * Add MaxUnlockingChunks to more places * Use defensive saturating add * FMT Co-authored-by: Kian Paimani <[email protected]> * staking: Expose `MaxUnlockingChunks` in metadata as a const (#10958) * Allow trailing commas for `bounded_vec!` (#10959) * SimpleSlotWorker: Do not implement `SlotWorker` for all types implementing `SimpleSlotWorker` (#10934) Because Rust currently doesn't support specialization, it prevents users from implementing `SlotWorker` for their own types. This pr solves this by removing the generic implementation of `SlotWorker` for `SimpleSlotWorker` and providing some wrapper type for that. * Bump digest from 0.10.2 to 0.10.3 (#10960) Bumps [digest](https://github.com/RustCrypto/traits) from 0.10.2 to 0.10.3. - [Release notes](https://github.com/RustCrypto/traits/releases) - [Commits](https://github.com/RustCrypto/traits/compare/digest-v0.10.2...digest-v0.10.3) --- updated-dependencies: - dependency-name: digest dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump scale-info from 2.0.0 to 2.0.1 (#10965) Bumps [scale-info](https://github.com/paritytech/scale-info) from 2.0.0 to 2.0.1. - [Release notes](https://github.com/paritytech/scale-info/releases) - [Changelog](https://github.com/paritytech/scale-info/blob/master/CHANGELOG.md) - [Commits](https://github.com/paritytech/scale-info/commits) --- updated-dependencies: - dependency-name: scale-info dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add pr-custom-review config (#10968) * Revise how staking configurations are set (#10955) * Revise how staking configurations are set fixes #10938 * Fix and add additional tests * Format * Formatting * Add doc Co-authored-by: Kian Paimani <[email protected]> * Update frame/staking/src/tests.rs Co-authored-by: Kian Paimani <[email protected]> * Format * Fix build * Update weights.rs * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/staking/src/weights.rs --template=./.maintain/frame-weight-template.hbs Co-authored-by: Shawn Tabrizi <[email protected]> Co-authored-by: Kian Paimani <[email protected]> Co-authored-by: Parity Bot <[email protected]> * Bump libc from 0.2.112 to 0.2.119 (#10967) Bumps [libc](https://github.com/rust-lang/libc) from 0.2.112 to 0.2.119. - [Release notes](https://github.com/rust-lang/libc/releases) - [Commits](https://github.com/rust-lang/libc/compare/0.2.112...0.2.119) --- updated-dependencies: - dependency-name: libc dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump parity-db from 0.3.5 to 0.3.8 (#10970) Bumps [parity-db](https://github.com/paritytech/parity-db) from 0.3.5 to 0.3.8. - [Release notes](https://github.com/paritytech/parity-db/releases) - [Commits](https://github.com/paritytech/parity-db/commits/v0.3.8) --- updated-dependencies: - dependency-name: parity-db dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [EPM pallet]: remove `number of signed submissions` (#10945) * [EPM pallet]: remove `number of signed submissions` Closing #9229 * fix tests * remove needless assert * Update frame/election-provider-multi-phase/src/lib.rs * cargo fmt Signed-off-by: Niklas <[email protected]> * fix grumbles * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs * drop `num_signed_submissions` in WeightInfo too * fix build Co-authored-by: Parity Bot <[email protected]> * Refactored block body database scheme (#10779) * Refactored tx storage database scheme * Bump parity-db * fmt * Fix handling invalid index size + test * Removed superflous result * Minor changes * fmt * Deprecate "paritydb-experimental" CLI in favour or "paritydb" (#10975) * Deprecate paritydb-experimental * Updated comment * [contracts] Stabilize "seal0" `is_contract` and `caller_is_origin` (#10971) * stabilize `seal_is_contract` * stabilize `seal_caller_is_origin` * Trie version migration pallet (#10073) * starting * Updated from other branch. * setting flag * flag in storage struct * fix flagging to access and insert. * added todo to fix * also missing serialize meta to storage proof * extract meta. * Isolate old trie layout. * failing test that requires storing in meta when old hash scheme is used. * old hash compatibility * Db migrate. * runing tests with both states when interesting. * fix chain spec test with serde default. * export state (missing trie function). * Pending using new branch, lacking genericity on layout resolution. * extract and set global meta * Update to branch 4 * fix iterator with root flag (no longer insert node). * fix trie root hashing of root * complete basic backend. * Remove old_hash meta from proof that do not use inner_hashing. * fix trie test for empty (force layout on empty deltas). * Root update fix. * debug on meta * Use trie key iteration that do not include value in proofs. * switch default test ext to use inner hash. * small integration test, and fix tx cache mgmt in ext. test failing * Proof scenario at state-machine level. * trace for db upgrade * try different param * act more like iter_from. * Bigger batches. * Update trie dependency. * drafting codec changes and refact * before removing unused branch no value alt hashing. more work todo rename all flag var to alt_hash, and remove extrinsic replace by storage query at every storage_root call. * alt hashing only for branch with value. * fix trie tests * Hash of value include the encoded size. * removing fields(broken) * fix trie_stream to also include value length in inner hash. * triedbmut only using alt type if inner hashing. * trie_stream to also only use alt hashing type when actually alt hashing. * Refactor meta state, logic should work with change of trie treshold. * Remove NoMeta variant. * Remove state_hashed trigger specific functions. * pending switching to using threshold, new storage root api does not make much sense. * refactoring to use state from backend (not possible payload changes). * Applying from previous state * Remove default from storage, genesis need a special build. * rem empty space * Catch problem: when using triedb with default: we should not revert nodes: otherwhise thing as trie codec cannot decode-encode without changing state. * fix compilation * Right logic to avoid switch on reencode when default layout. * Clean up some todos * remove trie meta from root upstream * update upstream and fix benches. * split some long lines. * UPdate trie crate to work with new design. * Finish update to refactored upstream. * update to latest triedb changes. * Clean up. * fix executor test. * rust fmt from master. * rust format. * rustfmt * fix * start host function driven versioning * update state-machine part * still need access to state version from runtime * state hash in mem: wrong * direction likely correct, but passing call to code exec for genesis init seem awkward. * state version serialize in runtime, wrong approach, just initialize it with no threshold for core api < 4 seems more proper. * stateversion from runtime version (core api >= 4). * update trie, fix tests * unused import * clean some TODOs * Require RuntimeVersionOf for executor * use RuntimeVersionOf to resolve genesis state version. * update runtime version test * fix state-machine tests * TODO * Use runtime version from storage wasm with fast sync. * rustfmt * fmt * fix test * revert useless changes. * clean some unused changes * fmt * removing useless trait function. * remove remaining reference to state_hash * fix some imports * Follow chain state version management. * trie update, fix and constant threshold for trie layouts. * update deps * Update to latest trie pr changes. * fix benches * Verify proof requires right layout. * update trie_root * Update trie deps to latest * Update to latest trie versioning * Removing patch * update lock * extrinsic for sc-service-test using layout v0. * Adding RuntimeVersionOf to CallExecutor works. * fmt * error when resolving version and no wasm in storage. * use existing utils to instantiate runtime code. * migration pallet * Patch to delay runtime switch. * Revert "Patch to delay runtime switch." This reverts commit d35f273b7d67b1b85a9e72973cab13c5c156c1d3. * fix test * fix child migration calls. * useless closure * remove remaining state_hash variables. * Fix and add more tests * Remove outdated comment * useless inner hash * fmt * remote tests * finally ksm works * batches are broken * clean the benchmarks * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <[email protected]> * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <[email protected]> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Joshy Orndorff <[email protected]> * Update frame/state-trie-migration/src/lib.rs * brand new version * fix build * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Guillaume Thiolliere <[email protected]> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Guillaume Thiolliere <[email protected]> * Update primitives/storage/src/lib.rs Co-authored-by: cheme <[email protected]> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <[email protected]> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <[email protected]> * fmt and opt-in feature to apply state change. * feature gate core version, use new test feature for node and test node * Use a 'State' api version instead of Core one. * fix merge of test function * use blake macro. * Fix state api (require declaring the api in runtime). * Opt out feature, fix macro for io to select a given version instead of latest. * run test nodes on new state. * fix * new test structure * new testing stuff from emeric * Add commit_all, still not working * Fix all tests * add comment * we have PoV tracking baby * document stuff, but proof size is still wrong * FUCK YEAH * a big batch of review comments * add more tests * tweak test * update config * some remote-ext stuff * delete some of the old stuff * sync more files with master to minimize the diff * Fix all tests * make signed migration a bit more relaxed * add witness check to signed submissions * allow custom migration to also go above limit * Fix these pesky tests * ==== removal of the unsigned stuff ==== * Make all tests work again * separate the tests from the logic so it can be reused easier * fix overall build * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <[email protected]> * Update frame/state-trie-migration/src/lib.rs Co-authored-by: cheme <[email protected]> * Slightly better termination * some final tweaks * Fix tests * Restrict access to signed migrations * address most of the review comments * fix defensive * New simplified code * Fix weights * fmt * Update frame/state-trie-migration/src/lib.rs Co-authored-by: Bastian Köcher <[email protected]> * make the tests correctly fail * Fix build * Fix build * try and fix the benchmarks * fix build * Fix cargo file * Fix runtime deposit * make rustdoc happy * cargo run --quiet --profile=production --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_state_trie_migration --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/state-trie-migration/src/weights.rs --template=./.maintain/frame-weight-template.hbs Co-authored-by: cheme <[email protected]> Co-authored-by: Guillaume Thiolliere <[email protected]> Co-authored-by: Joshy Orndorff <[email protected]> Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Parity Bot <[email protected]> * BEEFY and GRANDPA protocol names should use full genesis hash (#10974) std::fmt::Display shows formats as reduced hash (e.g. 0xb0a8…dafe) Use hex::encode to format full hash. Signed-off-by: acatangiu <[email protected]> * Move weight constants to own mod (#10980) * Move block+ext weights to own mod Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Unused import Signed-off-by: Oliver Tale-Yazdi <[email protected]> * add block usage logs to system pallet (#10940) * add block usage logs to system pallet * add Debug * use % instead of default Debug impl * change formatting * revert * update runtime storage, update chain extension * cleanup chain ext * Fix the undeterministic storage proof recorded for the same execution (#10915) * Add a test case for the determinism of recorded proof * Replace HashMap with BTreeMap for the actual proof records * cargo +nightly fmt --all * Store the trie nodes in BTreeSet for StorageProof * Nit * Revert the BTreeMap changes and sort when converting to storage proof * Remove PartialEq from StorageProof * Remove unnecessary change * Add `compare` method to StorageProof * FMT * Dummy change to trigger CI * Use `BTreeSet` for StorageProof and keep using `Vec` for CompactProof * Update comment on `iter_nodes` * Revert `PartialEq` removal * create iris ledger pallet * ledger funcs in chain ext * rename Iris -> IrisAssets, update readmes * Minor Uniques pallet improvements and XCM v3 preparations (#10896) * Introduce Helper to Uniques for benchmark stuff * Fixes * Formatting * Featuregate the Helper, include ContainsPair * Introduce & use EnsureOriginWithArg * Benchmarking * Docs * More ContainsBoth helpers * Formatting * Formatting * Fixes Co-authored-by: Shawn Tabrizi <[email protected]> * election provider support: Update some test only types (#10983) * Feedback from @XLC for Referenda Pallet (#10991) * feedback from @xlc * english * fmt Signed-off-by: Oliver Tale-Yazdi <[email protected]> Co-authored-by: Oliver Tale-Yazdi <[email protected]> * Make bags-list generic over node value and instantiable (#10997) * make instantiable * update * cargo fmt * Clean up * bags-list: Make it generic over node value * Respond to some feedback * Apply suggestions from code review Co-authored-by: Kian Paimani <[email protected]> * Add back default impl for weight update worst case * Update to Score in more places' * Use VoteWeight, not u64 to reduce test diff * FMT * FullCodec implies Codec * formatting * Fixup bags list remote test Co-authored-by: doordashcon <[email protected]> Co-authored-by: Doordashcon <[email protected]> Co-authored-by: Kian Paimani <[email protected]> * sc-finality-grandpa: use the #[from] attriute to remove boilerplate code (#11003) Signed-off-by: koushiro <[email protected]> * contracts: Add test to verify unique trie ids (#10914) * Add test to verify unique trie ids * Rename trie_seed to nonce * Rename AccountCounter -> Nonce * fmt * sp-core: `full_crypto` doesn't imply `std` (#11006) * sp-core: `full_crypto` doesn't imply `std` This pr changes the feature set of `secp256k1` to not use `global-context` when only the `full_crypto` is enabled. It will be slower when the `std` feature is not enabled as the context always needs to be recreated, but that is fine. * Update client/cli/src/arg_enums.rs Co-authored-by: Davide Galassi <[email protected]> Co-authored-by: Davide Galassi <[email protected]> * Remove unused `parent_hash` in `OverlayedChanges::into_storage_changes` (#11011) Ref https://github.com/paritytech/substrate/pull/10922#issuecomment-1064258443 CC @cheme * iris ledger pallet -> transfer not yet functioning * sp-api: Don't be dirty (#11015) Ensure that the sp api macros don't use functions without providing the full path to the function. This hygiene ensures that we don't actually try to call a method of an imported trait for example. * unlock and transfer works * Update clap to the latest version (#11017) * Update clap to the latest version Besides that it also removes some `structopt` leftovers from some docs. * Fix compile errors * More fixes * Move scripts used in CI to the new location (#11008) Move scripts used in CI to the new location - **./scripts/ci/** * Move github scripts * Move more files * Move ci scripts and fix dependencies * Update docs/node-template-release.md Co-authored-by: João Paulo Silva de Souza <[email protected]> * Remove Cargo.lock * Apply suggestions from code review Co-authored-by: Denis Pisarev <[email protected]> * Make more paths uniform Co-authored-by: João Paulo Silva de Souza <[email protected]> Co-authored-by: Denis Pisarev <[email protected]> * Move `sp-npos-elections-solution-type` to `frame-election-provider-support` (#11016) * Move `sp-npos-elections-solution-type` to `frame-election-provider-support` First stab at it, will need to amend some more stuff * Fixing tests * Fixing tests * Fixing cargo.toml for std configuration * fmt * Committing suggested changes renaming, and re exporting macro. * Removing unneeded imports * shell.nix: Update to a newer nightly (#11028) * update chain ext * update naming, formatting, start tests * Update lockfile (#11035) Signed-off-by: Oliver Tale-Yazdi <[email protected]> * Additional `benchmark-storage` flags (#11004) * Fix typos * Enable overwriting handlebars template * Optionally name json output or disable json altogether * Don't write to json by default * Include block id in handlebars output * Include warmups for write benchmarks * PR comments * Drop unnecessary file extension * Use more appropriate types * Use more appropriate error message * More use of more appropriate types * Rework write benchmark warmups * Run same benchmark for both read and write * SharedData: Update locks to mention possible deadlocks (#11034) * SharedData: Update locks to mention possible deadlocks * Update `Cargo.lock` * Stabilize `seal_delegate_call` (#11037) * add tests * re-enumerate chain ext func ids * Bump names from 0.12.0 to 0.13.0 (#11047) Bumps [names](https://github.com/fnichol/names) from 0.12.0 to 0.13.0. - [Release notes](https://github.com/fnichol/names/releases) - [Changelog](https://github.com/fnichol/names/blob/main/CHANGELOG.md) - [Commits](https://github.com/fnichol/names/compare/v0.12.0...v0.13.0) --- updated-dependencies: - dependency-name: names dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * State migration rpc (#10981) * setting flag * flag in storage struct * fix flagging to access and insert. * added todo to fix * also missing serialize meta to storage proof * extract meta. * Isolate old trie layout. * failing test that requires storing in meta when old hash scheme is used. * old hash compatibility * Db migrate. * runing tests with both states when interesting. * fix chain spec test with serde default. * export state (missing trie function). * Pending using new branch, lacking genericity on layout resolution. * extract and set global meta * Update to branch 4 * fix iterator with root flag (no longer insert node). * fix trie root hashing of root * complete basic backend. * Remove old_hash meta from proof that do not use inner_hashing. * fix trie test for empty (force layout on empty deltas). * Root update fix. * debug on meta * Use trie key iteration that do not include value in proofs. * switch default test ext to use inner hash. * small integration test, and fix tx cache mgmt in ext. test failing * Proof scenario at state-machine level. * trace for db upgrade * try different param * act more like iter_from. * Bigger batches. * Update trie dependency. * drafting codec changes and refact * before removing unused branch no value alt hashing. more work todo rename all flag var to alt_hash, and remove extrinsic replace by storage query at every storage_root call. * alt hashing only for branch with value. * fix trie tests * Hash of value include the encoded size. * removing fields(broken) * fix trie_stream to also include value length in inner hash. * triedbmut only using alt type if inner hashing. * trie_stream to also only use alt hashing type when actually alt hashing. * Refactor meta state, logic should work with change of trie treshold. * Remove NoMeta variant. * Remove state_hashed trigger specific functions. * pending switching to using threshold, new storage root api does not make much sense. * refactoring to use state from backend (not possible payload changes). * Applying from previous state * Remove default from storage, genesis need a special build. * rem empty space * Catch problem: when using triedb with default: we should not revert nodes: otherwhise thing as trie codec cannot decode-encode without changing state. * fix compilation * Right …
ark0f
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Feb 27, 2023
* Refactored tx storage database scheme * Bump parity-db * fmt * Fix handling invalid index size + test * Removed superflous result * Minor changes * fmt
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR changes the way block bodies are stored in the database. It is backward compatible with the current plain body storage, but breaks compatibility with existing storage chains. Since there are no actual storage chains deployed, this is non-issue.
Motivation:
--storage-chainCLI flag is removed.Summary of changes:
Currently there are two incompatible storage scheme selected with with the
--storage-chainflag. When the flag is not specified, block bodies are stored as blobs in the theBODYcolumn and indexed extrinsic are not allowed. In storage chain mode each extrinsic is split into indexed portion (which may be empty) and the header, that contains the rest of the extrinsic. A list of headers is inserted intoBODYcolumn. Indexed parts are inserted intoTRANSACTIONcolumn with reference counting.With this PR there's just single mode.
Blocks that don't contain a single indexed extrinsic are saved in the
BODYcolumn, maintaining compatible with current format.Block that have indexed content are split. The headers are written to the
BODY_INDEXcolumn and indexed parts are inserted intoTRANSACTIONS.polkadot companion: paritytech/polkadot#4837
cumulus companion: paritytech/cumulus#949