keyless exebench in ci #15559
keyless exebench in ci #15559
Conversation
⏱️ 2h 29m total CI duration on this PR
|
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
testsuite/single_node_performance.py
Outdated
| @@ -64,6 +64,8 @@ class Flow(Flag): | |||
| ] | |||
| ) | |||
| MAIN_SIGNER_ACCOUNTS = 2 * MAX_BLOCK_SIZE | |||
| NUM_KEYLESS_ACCOUNTS = 20000 # Creating 20k keyless accounts should take ~1 min. | |||
| MAX_BLOCK_SIZE_KEYLESS = 10000 | |||
There was a problem hiding this comment.
why do you need different value here?
testsuite/single_node_performance.py
Outdated
| @@ -64,6 +64,8 @@ class Flow(Flag): | |||
| ] | |||
| ) | |||
| MAIN_SIGNER_ACCOUNTS = 2 * MAX_BLOCK_SIZE | |||
| NUM_KEYLESS_ACCOUNTS = 20000 # Creating 20k keyless accounts should take ~1 min. | |||
testsuite/single_node_performance.py
Outdated
| key=RunGroupKey("keyless-coin-transfer"), | ||
| key_extra=RunGroupKeyExtra( | ||
| txn_auth_mode="keyless", | ||
| transaction_type_override="non-conflicting-coin-transfer" |
There was a problem hiding this comment.
to have a best comparison, let's use no-op or apt-fa-transfer - so we can compare keyless speed to no-keyless directly?
if we use no-op, then we could have only 10k accounts, and have additional_dst_pool_accounts=0? so that might be best?
testsuite/single_node_performance.py
Outdated
| create_db_command = f"RUST_BACKTRACE=1 {BUILD_FOLDER}/aptos-executor-benchmark --block-executor-type aptos-vm-with-block-stm --block-size {MAX_BLOCK_SIZE} --execution-threads {NUMBER_OF_EXECUTION_THREADS} {DB_CONFIG_FLAGS} {DB_PRUNER_FLAGS} create-db {FEATURE_FLAGS} --data-dir {tmpdirname}/db --num-accounts {NUM_ACCOUNTS}" | ||
| output = execute_command(create_db_command) | ||
|
|
||
| results = [] | ||
| print(f"Warmup - creating another DB with {NUM_KEYLESS_ACCOUNTS} keyless accounts") | ||
| create_db_command_keyless = f"RUST_BACKTRACE=1 {BUILD_FOLDER}/aptos-executor-benchmark --block-executor-type aptos-vm-with-block-stm --block-size {MAX_BLOCK_SIZE_KEYLESS} --execution-threads {NUMBER_OF_EXECUTION_THREADS} {DB_CONFIG_FLAGS} {DB_PRUNER_FLAGS} --use-keyless-accounts create-db {FEATURE_FLAGS} --data-dir {keyless_tmp_dir}/db --num-accounts {NUM_KEYLESS_ACCOUNTS}" |
There was a problem hiding this comment.
wondering if we can have a single db with both ?
also - based on FLOW - keyless run might be requested or not needed. We don't need to warmup keyless if keyless is not detected. we can have bool - keyless_warmed_up, and first time we need keyless, initialize those accounts.
There was a problem hiding this comment.
wondering if we can have a single db with both ?
it requires more refactoring of executor benchmark... better be a separate pr?
There was a problem hiding this comment.
maybe for now leave it as separate DB. but make the change to only create it if keyless test is being run
| if test.key_extra.txn_auth_mode == "keyless": | ||
| data_dir = keyless_tmp_dir | ||
| keyless_flags = "--use-keyless-accounts" | ||
| additional_dst_pool_accounts = 0 |
There was a problem hiding this comment.
"if keyless run, set additional_dst_pool_accounts = 0"
@igor-aptos you were suggesting this in another comment, right?
testsuite/single_node_performance.py
Outdated
| ] + [ | ||
| RunGroupConfig( | ||
| expected_tps=7600, | ||
| key=RunGroupKey("keyless-coin-transfer"), |
Description
Add initial TPS expectation in CI for non-conflicting coin transfers between 20k keyless accounts.
How Has This Been Tested?
Manual CI runs.
Key Areas to Review
n/a
Type of Change
Which Components or Systems Does This Change Impact?
n/a
Checklist