refactor: Remove commitment key copy out of instance #4893

lucasxia01 · 2024-03-01T19:11:20Z

This PR removes the duplicated commitment key or PCS verification key in the prover instance and verifier instance. It instead moves the commitment keys to the proving/verification keys. As a result, the recursive verification keys must be updated to actually be used, while they were previously unused and uninstantiated.

when folding the proving key, the commitment key is uninitialized

AztecBot · 2024-03-01T21:56:09Z

Benchmark results

No metrics with a significant change found.

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit c9eb856a and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	64 txs
l1_rollup_calldata_size_in_bytes	5,700	18,884	36,452
l1_rollup_calldata_gas	66,108	239,096	469,820
l1_rollup_execution_gas	194,056	500,402	909,078
l2_block_processing_time_in_ms	1,202 (-1%)	4,546	9,024 (-1%)
note_successful_decrypting_time_in_ms	202	542	993
note_trial_decrypting_time_in_ms	8.90 (-15%)	66.9 (+5%)	130 (-2%)
l2_block_building_time_in_ms	15,970	63,901 (-1%)	127,394 (-2%)
l2_block_rollup_simulation_time_in_ms	12,144	48,699 (-1%)	97,042 (-2%)
l2_block_public_tx_process_time_in_ms	3,796 (+1%)	15,130 (-2%)	30,201 (-2%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	14,428 (-4%)	27,396 (-2%)
note_history_successful_decrypting_time_in_ms	1,240 (-8%)	2,416 (-1%)
note_history_trial_decrypting_time_in_ms	100 (+8%)	142 (-8%)
node_database_size_in_bytes	18,788,432	35,250,256 (-1%)
pxe_database_size_in_bytes	29,923	59,478

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit	circuit_simulation_time_in_ms	circuit_input_size_in_bytes	circuit_output_size_in_bytes
private-kernel-init	247 (-2%)	44,736	28,001
private-kernel-ordering	178 (-1%)	52,625	14,627
base-rollup	1,287 (-1%)	177,932	933
root-rollup	70.4 (-1%)	4,192	825
private-kernel-inner	315 (-1%)	73,715	28,001
public-kernel-app-logic	192 (-1%)	32,254	25,379
merge-rollup	5.75 (-1%)	2,712	933

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	2 leaves	8 leaves	16 leaves	32 leaves	64 leaves	128 leaves	512 leaves	1024 leaves	2048 leaves	4096 leaves
batch_insert_into_append_only_tree_16_depth_ms	9.84	10.3 (-1%)	12.5 (-10%)	16.4	22.1 (-13%)	35.6	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.9	17.5	23.0	31.6	47.0	79.0	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.570	0.573 (-1%)	0.529 (-10%)	0.503 (-1%)	0.463 (-12%)	0.442	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	N/A	N/A	N/A	45.8	72.4 (-1%)	231 (-1%)	448	872 (-1%)	1,729
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	N/A	N/A	N/A	96.0	159	543	1,055	2,079	4,127
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	0.469	0.446 (-1%)	0.421 (-1%)	0.418	0.415 (-1%)	0.413
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	N/A	N/A	N/A	55.3 (+3%)	108 (-1%)	339 (-1%)	665	1,312 (-1%)	2,618 (-1%)
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	N/A	N/A	N/A	104	207	691	1,363	2,707	5,395
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	0.490 (+2%)	0.483 (-1%)	0.458 (-1%)	0.456	0.455 (-1%)	0.455 (-1%)
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	N/A	61.9	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	N/A	109	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	N/A	0.539	N/A	N/A	N/A	N/A	N/A	N/A

Miscellaneous

Transaction sizes based on how many contracts are deployed in the tx.

Metric	0 deployed contracts
tx_size_in_bytes	19,179

Transaction processing duration by data writes.

Metric	0 new note hashes	1 new note hashes
tx_pxe_processing_time_ms	2,620 (+1%)	1,355 (-1%)

Metric	0 public data writes	1 public data writes
tx_sequencer_processing_time_ms	0.0310 (+4%)	469 (-1%)

…c-packages into lx/ck-out-of-instance

lucasxia01 · 2024-03-04T19:13:03Z

barretenberg/cpp/src/barretenberg/flavor/goblin_ultra_recursive.hpp

@@ -41,7 +41,7 @@ template <typename BuilderType> class GoblinUltraRecursiveFlavor_ {
    using NativeVerificationKey = NativeFlavor::VerificationKey;

    // Note(luke): Eventually this may not be needed at all


not sure what this comment means, but this PR makes sure that we use it

lucasxia01 · 2024-03-04T19:15:25Z

barretenberg/cpp/src/barretenberg/flavor/goblin_ultra_recursive.hpp

@@ -41,7 +41,7 @@ template <typename BuilderType> class GoblinUltraRecursiveFlavor_ {
    using NativeVerificationKey = NativeFlavor::VerificationKey;

    // Note(luke): Eventually this may not be needed at all
-    using VerifierCommitmentKey = bb::VerifierCommitmentKey<Curve>;


previously was using the wrong Curve. We want to use the native curve here.

lucasxia01 · 2024-03-04T19:16:09Z

barretenberg/cpp/src/barretenberg/flavor/goblin_ultra_recursive.hpp

@@ -114,6 +114,7 @@ template <typename BuilderType> class GoblinUltraRecursiveFlavor_ {
         */
        VerificationKey(CircuitBuilder* builder, const std::shared_ptr<NativeVerificationKey>& native_key)


Question: is there any point in keeping the other constructor here? It seems dangerous to not set up the verification key with the pcs_verification_key...

Feel free to investigate and resolve here though it may make sense to resolve in a broader PR related to the non-use of copy constructors in the Pg code.

lucasxia01 · 2024-03-04T19:28:45Z

barretenberg/cpp/src/barretenberg/protogalaxy/decider_prover.cpp

@@ -17,7 +17,7 @@ DeciderProver_<Flavor>::DeciderProver_(const std::shared_ptr<Instance>& inst,
                                       const std::shared_ptr<Transcript>& transcript)
    : accumulator(std::move(inst))
    , transcript(transcript)
-    , commitment_key(inst->commitment_key)
+    , commitment_key(inst->proving_key->commitment_key)


storing commitment_key is really only useful because now we don't have to write instance->proving_key->commitment_key everywhere we want to use the commitment_key, but this should probably be changed to a reference instead of a shared_ptr

Makes sense, feel free resolve in a future PR.

lucasxia01 · 2024-03-04T20:04:43Z

...etenberg/cpp/src/barretenberg/stdlib/recursion/honk/verifier/recursive_verifier_instance.hpp

+            inst_vk = vk.get_value();
+        }
+
+        VerifierInstance inst(inst_verification_key);
        inst.pub_inputs_offset = pub_inputs_offset;
        inst.public_input_size = public_input_size;
        inst.log_instance_size = log_instance_size;


not sure if some of these are unnecessary or not

codygunton

LGTM

codygunton · 2024-03-05T22:52:05Z

barretenberg/cpp/src/barretenberg/flavor/goblin_ultra_recursive.hpp

@@ -114,6 +114,7 @@ template <typename BuilderType> class GoblinUltraRecursiveFlavor_ {
         */
        VerificationKey(CircuitBuilder* builder, const std::shared_ptr<NativeVerificationKey>& native_key)


Feel free to investigate and resolve here though it may make sense to resolve in a broader PR related to the non-use of copy constructors in the Pg code.

codygunton · 2024-03-05T22:52:37Z

barretenberg/cpp/src/barretenberg/protogalaxy/decider_prover.cpp

@@ -17,7 +17,7 @@ DeciderProver_<Flavor>::DeciderProver_(const std::shared_ptr<Instance>& inst,
                                       const std::shared_ptr<Transcript>& transcript)
    : accumulator(std::move(inst))
    , transcript(transcript)
-    , commitment_key(inst->commitment_key)
+    , commitment_key(inst->proving_key->commitment_key)


Makes sense, feel free resolve in a future PR.

codygunton · 2024-03-05T22:54:47Z

...etenberg/cpp/src/barretenberg/stdlib/recursion/honk/verifier/recursive_verifier_instance.hpp

+            inst_vk = vk.get_value();
+        }
+
+        VerifierInstance inst(inst_verification_key);
        inst.pub_inputs_offset = pub_inputs_offset;
        inst.public_input_size = public_input_size;
        inst.log_instance_size = log_instance_size;


🤖 I have created a release *beep* *boop* --- <details><summary>aztec-package: 0.26.1</summary> ## [0.26.1](aztec-package-v0.26.0...aztec-package-v0.26.1) (2024-03-06) ### Miscellaneous * **aztec-package:** Synchronize aztec-packages versions </details> <details><summary>barretenberg.js: 0.26.1</summary> ## [0.26.1](barretenberg.js-v0.26.0...barretenberg.js-v0.26.1) (2024-03-06) ### Miscellaneous * **barretenberg.js:** Synchronize aztec-packages versions </details> <details><summary>aztec-cli: 0.26.1</summary> ## [0.26.1](aztec-cli-v0.26.0...aztec-cli-v0.26.1) (2024-03-06) ### Miscellaneous * **aztec-cli:** Synchronize aztec-packages versions </details> <details><summary>aztec-packages: 0.26.1</summary> ## [0.26.1](aztec-packages-v0.26.0...aztec-packages-v0.26.1) (2024-03-06) ### Features * Adding fr compatibility to smt variables api ([#4884](#4884)) ([c085cbb](c085cbb)) * **avm-simulator:** Implement EMITUNENCRYPTEDLOG ([#4926](#4926)) ([5f3304e](5f3304e)) * Choose constructor method in Contract.deploy ([#4939](#4939)) ([e899e56](e899e56)) * Indirect mem flag deserialisation ([#4877](#4877)) ([4c6820f](4c6820f)) ### Miscellaneous * Add missing jobs to CI end ([#4963](#4963)) ([ff4110e](ff4110e)) * **avm-simulator:** Better type env getters ([#4950](#4950)) ([8f97977](8f97977)) * **avm-simulator:** Revive field comparison ([#4957](#4957)) ([ee21374](ee21374)) * **avm-simulator:** Test improvements ([#4946](#4946)) ([f74e6a1](f74e6a1)) * Fix CCI config ([#4974](#4974)) ([40178f0](40178f0)) * Remove commitment key copy out of instance ([#4893](#4893)) ([6eb6778](6eb6778)) * **vscode:** Add avm-transpiler to vscode rust-analyzer settings ([#4952](#4952)) ([db915e5](db915e5)) </details> <details><summary>barretenberg: 0.26.1</summary> ## [0.26.1](barretenberg-v0.26.0...barretenberg-v0.26.1) (2024-03-06) ### Features * Adding fr compatibility to smt variables api ([#4884](#4884)) ([c085cbb](c085cbb)) * Indirect mem flag deserialisation ([#4877](#4877)) ([4c6820f](4c6820f)) ### Miscellaneous * Remove commitment key copy out of instance ([#4893](#4893)) ([6eb6778](6eb6778)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-package: 0.26.1</summary> ## [0.26.1](AztecProtocol/aztec-packages@aztec-package-v0.26.0...aztec-package-v0.26.1) (2024-03-06) ### Miscellaneous * **aztec-package:** Synchronize aztec-packages versions </details> <details><summary>barretenberg.js: 0.26.1</summary> ## [0.26.1](AztecProtocol/aztec-packages@barretenberg.js-v0.26.0...barretenberg.js-v0.26.1) (2024-03-06) ### Miscellaneous * **barretenberg.js:** Synchronize aztec-packages versions </details> <details><summary>aztec-cli: 0.26.1</summary> ## [0.26.1](AztecProtocol/aztec-packages@aztec-cli-v0.26.0...aztec-cli-v0.26.1) (2024-03-06) ### Miscellaneous * **aztec-cli:** Synchronize aztec-packages versions </details> <details><summary>aztec-packages: 0.26.1</summary> ## [0.26.1](AztecProtocol/aztec-packages@aztec-packages-v0.26.0...aztec-packages-v0.26.1) (2024-03-06) ### Features * Adding fr compatibility to smt variables api ([#4884](AztecProtocol/aztec-packages#4884)) ([c085cbb](AztecProtocol/aztec-packages@c085cbb)) * **avm-simulator:** Implement EMITUNENCRYPTEDLOG ([#4926](AztecProtocol/aztec-packages#4926)) ([5f3304e](AztecProtocol/aztec-packages@5f3304e)) * Choose constructor method in Contract.deploy ([#4939](AztecProtocol/aztec-packages#4939)) ([e899e56](AztecProtocol/aztec-packages@e899e56)) * Indirect mem flag deserialisation ([#4877](AztecProtocol/aztec-packages#4877)) ([4c6820f](AztecProtocol/aztec-packages@4c6820f)) ### Miscellaneous * Add missing jobs to CI end ([#4963](AztecProtocol/aztec-packages#4963)) ([ff4110e](AztecProtocol/aztec-packages@ff4110e)) * **avm-simulator:** Better type env getters ([#4950](AztecProtocol/aztec-packages#4950)) ([8f97977](AztecProtocol/aztec-packages@8f97977)) * **avm-simulator:** Revive field comparison ([#4957](AztecProtocol/aztec-packages#4957)) ([ee21374](AztecProtocol/aztec-packages@ee21374)) * **avm-simulator:** Test improvements ([#4946](AztecProtocol/aztec-packages#4946)) ([f74e6a1](AztecProtocol/aztec-packages@f74e6a1)) * Fix CCI config ([#4974](AztecProtocol/aztec-packages#4974)) ([40178f0](AztecProtocol/aztec-packages@40178f0)) * Remove commitment key copy out of instance ([#4893](AztecProtocol/aztec-packages#4893)) ([6eb6778](AztecProtocol/aztec-packages@6eb6778)) * **vscode:** Add avm-transpiler to vscode rust-analyzer settings ([#4952](AztecProtocol/aztec-packages#4952)) ([db915e5](AztecProtocol/aztec-packages@db915e5)) </details> <details><summary>barretenberg: 0.26.1</summary> ## [0.26.1](AztecProtocol/aztec-packages@barretenberg-v0.26.0...barretenberg-v0.26.1) (2024-03-06) ### Features * Adding fr compatibility to smt variables api ([#4884](AztecProtocol/aztec-packages#4884)) ([c085cbb](AztecProtocol/aztec-packages@c085cbb)) * Indirect mem flag deserialisation ([#4877](AztecProtocol/aztec-packages#4877)) ([4c6820f](AztecProtocol/aztec-packages@4c6820f)) ### Miscellaneous * Remove commitment key copy out of instance ([#4893](AztecProtocol/aztec-packages#4893)) ([6eb6778](AztecProtocol/aztec-packages@6eb6778)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

lucasxia01 marked this pull request as draft March 1, 2024 19:11

lucasxia01 self-assigned this Mar 1, 2024

Base automatically changed from cg/ck-out-of-composer to master March 1, 2024 21:00

lucasxia01 added 2 commits March 1, 2024 21:15

initial fix attempt

4af6511

fixed?

a8ca2ae

when folding the proving key, the commitment key is uninitialized

lucasxia01 force-pushed the lx/ck-out-of-instance branch from 8babc5b to a8ca2ae Compare March 1, 2024 21:16

lucasxia01 added 2 commits March 1, 2024 22:26

my prayers were answered

db915c0

Merge remote-tracking branch 'origin/master' into lx/ck-out-of-instance

6d58cd0

lucasxia01 marked this pull request as ready for review March 4, 2024 14:54

lucasxia01 added 3 commits March 4, 2024 10:51

Merge branch 'master' into lx/ck-out-of-instance

4e87f7e

cleaned up

5e21259

Merge branch 'lx/ck-out-of-instance' of github.com:AztecProtocol/azte…

d6ff97f

…c-packages into lx/ck-out-of-instance

lucasxia01 commented Mar 4, 2024

View reviewed changes

lucasxia01 added 2 commits March 4, 2024 19:57

removing resolving issue 881 to reduce scope

bc4c1e2

Merge branch 'master' into lx/ck-out-of-instance

c00520a

lucasxia01 commented Mar 4, 2024

View reviewed changes

unnecessary change

9c07446

lucasxia01 requested a review from codygunton March 4, 2024 21:05

codygunton approved these changes Mar 5, 2024

View reviewed changes

codygunton merged commit 6eb6778 into master Mar 5, 2024
83 checks passed

codygunton deleted the lx/ck-out-of-instance branch March 5, 2024 22:55

AztecBot mentioned this pull request Mar 5, 2024

chore(master): Release 0.26.1 #4964

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: Remove commitment key copy out of instance #4893

refactor: Remove commitment key copy out of instance #4893

lucasxia01 commented Mar 1, 2024 •

edited

Loading

AztecBot commented Mar 1, 2024 •

edited

Loading

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

lucasxia01 Mar 4, 2024

lucasxia01 Mar 4, 2024

lucasxia01 Mar 4, 2024

codygunton Mar 5, 2024

lucasxia01 Mar 4, 2024

codygunton Mar 5, 2024

lucasxia01 Mar 4, 2024

codygunton Mar 5, 2024

codygunton left a comment

codygunton Mar 5, 2024

codygunton Mar 5, 2024

codygunton Mar 5, 2024

		@@ -41,7 +41,7 @@ template <typename BuilderType> class GoblinUltraRecursiveFlavor_ {
		using NativeVerificationKey = NativeFlavor::VerificationKey;

		// Note(luke): Eventually this may not be needed at all

		@@ -114,6 +114,7 @@ template <typename BuilderType> class GoblinUltraRecursiveFlavor_ {
		*/
		VerificationKey(CircuitBuilder* builder, const std::shared_ptr<NativeVerificationKey>& native_key)

refactor: Remove commitment key copy out of instance #4893

refactor: Remove commitment key copy out of instance #4893

Conversation

lucasxia01 commented Mar 1, 2024 • edited Loading

AztecBot commented Mar 1, 2024 • edited Loading

Benchmark results

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codygunton left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lucasxia01 commented Mar 1, 2024 •

edited

Loading

AztecBot commented Mar 1, 2024 •

edited

Loading