Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add solo smoke test to test flow #905

Merged
merged 32 commits into from
Dec 13, 2024
Merged

Conversation

JeffreyDallas
Copy link
Contributor

@JeffreyDallas JeffreyDallas commented Dec 2, 2024

Description

This pull request changes the following:

  • Added a flag generateEcdsaKey so user can use solo account create to create account with auto generated ECDSA key, no need to provide a EDCSA key string to solo
  • Added a flag createAmount so solo can continuously generating many account, so we can use the feature to generate background transactions when testing relay node. Cannot use original standalone solo account create since it is too slow can only finish one account in about 9~10 seconds
  • Retrieve and display EVM address for user with solo account get command

Related Issues

Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
@JeffreyDallas JeffreyDallas requested review from a team and leninmehedy as code owners December 2, 2024 22:40
@JeffreyDallas JeffreyDallas requested a review from dalvizu December 2, 2024 22:40
Copy link
Contributor

github-actions bot commented Dec 2, 2024

Unit Test Results - Linux

  1 files  ±0   57 suites  +4   3s ⏱️ -1s
220 tests +4  220 ✅ +4  0 💤 ±0  0 ❌ ±0 
230 runs  +4  230 ✅ +4  0 💤 ±0  0 ❌ ±0 

Results for commit dd01b20. ± Comparison against base commit 83fb584.

♻️ This comment has been updated with latest results.

@JeffreyDallas JeffreyDallas marked this pull request as draft December 2, 2024 22:42
Copy link
Contributor

github-actions bot commented Dec 2, 2024

Unit Test Results - Windows

  1 files  ±0   57 suites  +4   16s ⏱️ +3s
220 tests +4  220 ✅ +4  0 💤 ±0  0 ❌ ±0 
230 runs  +4  230 ✅ +4  0 💤 ±0  0 ❌ ±0 

Results for commit dd01b20. ± Comparison against base commit 83fb584.

♻️ This comment has been updated with latest results.

Signed-off-by: Jeffrey Tang <[email protected]>
…solo-smoke-test

Signed-off-by: Jeffrey Tang <[email protected]>

# Conflicts:
#	src/commands/account.ts
#	src/commands/flags.ts
Copy link

codecov bot commented Dec 3, 2024

Codecov Report

Attention: Patch coverage is 77.14286% with 16 lines in your changes missing coverage. Please review.

Project coverage is 84.98%. Comparing base (83fb584) to head (dd01b20).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
src/commands/flags.ts 70.58% 10 Missing ⚠️
src/commands/account.ts 83.33% 5 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #905      +/-   ##
==========================================
- Coverage   85.01%   84.98%   -0.03%     
==========================================
  Files          71       71              
  Lines       19142    19221      +79     
  Branches     1631     1684      +53     
==========================================
+ Hits        16273    16335      +62     
+ Misses       2764     2730      -34     
- Partials      105      156      +51     
Files with missing lines Coverage Δ
src/commands/account.ts 75.86% <83.33%> (+0.19%) ⬆️
src/commands/flags.ts 78.51% <70.58%> (-0.34%) ⬇️

... and 17 files with indirect coverage changes

Impacted file tree graph

@JeffreyDallas JeffreyDallas marked this pull request as ready for review December 3, 2024 19:16
Signed-off-by: Jeffrey Tang <[email protected]>
…solo-smoke-test

Signed-off-by: Jeffrey Tang <[email protected]>

# Conflicts:
#	src/commands/account.ts
#	src/commands/flags.ts
Signed-off-by: Jeffrey Tang <[email protected]>
Signed-off-by: Jeffrey Tang <[email protected]>
@jeromy-cannon jeromy-cannon marked this pull request as draft December 9, 2024 14:28
@JeffreyDallas JeffreyDallas marked this pull request as ready for review December 9, 2024 15:14
Copy link
Contributor

@jeromy-cannon jeromy-cannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error handling needs to be fixed in the script. You have a test case error, but it ate the error and showed a success: https://github.com/hashgraph/solo/actions/runs/12238568891/job/34136963069?pr=905#step:9:518

@JeffreyDallas
Copy link
Contributor Author

JeffreyDallas commented Dec 10, 2024

error handling needs to be fixed in the script. You have a test case error, but it ate the error and showed a success: https://github.com/hashgraph/solo/actions/runs/12238568891/job/34136963069?pr=905#step:9:518

This is the issue I mentioned before that with linux github runner,
port-forward randomly stop itself without any errors or logs,
so I wrapped the test within a retry function, try five times.

also there is a background thread kept re-enable port-forward.

from log you can see results of ps -ef |grep port-forwar
the svc/relay-node1-hedera-json-rpc-relay was there before the test,
after test failed ( because relay port forward stopped), it was no longer there,
after reenabled and retry the test, test would pass

runner      1522       1  0 15:18 ?        00:00:00 kubectl port-forward -n solo-e2e svc/haproxy-node1-svc 50211:50211
runner      1725       1  0 15:19 ?        00:00:00 kubectl port-forward -n solo-e2e svc/hedera-explorer 8080:80
runner      1729       1  0 15:19 ?        00:00:00 kubectl port-forward svc/mirror-grpc -n solo-e2e 5600:5600
runner      1916       1  0 15:20 ?        00:00:00 kubectl port-forward -n solo-e2e svc/relay-node1-hedera-json-rpc-relay 7546:7546
runner      3011    2692  0 15:22 ?        00:00:00 grep port-forward
Enable port forward round 4
Function start_contract_test failed with return code 1
Retry start_contract_test in 2 seconds
-----------------------
Wait a few seconds for background transactions to start
Run smart contract test

> @hashgraph/[email protected] hh:test
> hardhat test --grep "@OZERC20 Test Suite"



  @OZERC20 Test Suite
    1) "before all" hook for "should be able to execute name()"


  0 passing (96ms)
  1 failing
....
/home/runner/_work/solo
-----------------------
runner      1522       1  0 15:18 ?        00:00:00 kubectl port-forward -n solo-e2e svc/haproxy-node1-svc 50211:50211
runner      1725       1  0 15:19 ?        00:00:00 kubectl port-forward -n solo-e2e svc/hedera-explorer 8080:80
runner      1729       1  0 15:19 ?        00:00:00 kubectl port-forward svc/mirror-grpc -n solo-e2e 5600:5600
runner      3096    2692  0 15:22 ?        00:00:00 grep port-forward
Enable port forward round 5
runner      1522       1  0 15:18 ?        00:00:00 kubectl port-forward -n solo-e2e svc/haproxy-node1-svc 50211:50211
runner      1725       1  0 15:19 ?        00:00:00 kubectl port-forward -n solo-e2e svc/hedera-explorer 8080:80
runner      1729       1  0 15:19 ?        00:00:00 kubectl port-forward svc/mirror-grpc -n solo-e2e 5600:5600
runner      3099    2692  3 15:22 ?        00:00:00 kubectl port-forward -n solo-e2e svc/relay-node1-hedera-json-rpc-relay 7[546](https://github.com/hashgraph/solo/actions/runs/12238568891/job/34136963069?pr=905#step:9:547):7546
runner      3190    2692  0 15:22 ?        00:00:00 grep port-forward

@JeffreyDallas
Copy link
Contributor Author

This is the same issue repeated on macOS

line 285 we can see all port forwards are still active
after a few seconds, test failed,
line 332 show relay port forward is gone.

smoke_test_log.txt

mishomihov00
mishomihov00 previously approved these changes Dec 11, 2024
Copy link
Contributor

@mishomihov00 mishomihov00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review applies to:
.github/workflows/flow-task-test.yaml

@jeromy-cannon
Copy link
Contributor

My point, is that the workflow should be showing a failure, but it is showing a success.

@JeffreyDallas
Copy link
Contributor Author

My point, is that the workflow should be showing a failure, but it is showing a success.

What shown in the log is first try of test is failed due to port-forward is stopped,
but after re-enable port-forward, test itself is passed.

The workflow is passed because test success with the retry less than 5 times.

@JeffreyDallas
Copy link
Contributor Author

After some experiment from last night, I found if port-forward were called within Task,
they might randomly stopped.

but if port-forward were called outside Task, they would be consistently working.

I did not have an explanation for this.
But did find a workaround in test script:

  1. kill port-forward first to remove those Task command enabled port forward
  2. restart port-foward within test script

Copy link

codacy-production bot commented Dec 11, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.02% (target: -1.00%) 78.57%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (83fb584) 19114 16438 86.00%
Head commit (dd01b20) 19221 (+107) 16527 (+89) 85.98% (-0.02%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#905) 70 55 78.57%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

@jeromy-cannon
Copy link
Contributor

that makes sense, thank you for the explanation, approved

@jeromy-cannon jeromy-cannon merged commit 79410f0 into main Dec 13, 2024
38 of 41 checks passed
@jeromy-cannon jeromy-cannon deleted the 00876-D-solo-smoke-test branch December 13, 2024 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consolidate smoke test of smart contract and javascript sdk with solo
3 participants