Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go-algorand 3.7.0-stable Release PR #4071

Merged
merged 39 commits into from
Jun 6, 2022

Conversation

Algo-devops-service
Copy link
Contributor

@Algo-devops-service Algo-devops-service commented Jun 3, 2022

GitHub Logo

Overview

Bandwidth optimization for non-participating nodes and golang 1.17 upgrade.

What's New

  • Non-participating nodes do not receive gossiped proposed transactions by default
  • Building from source now supported for M1 Mac
  • Golang 1.17 upgrade

Note

A new flag was added to update.sh that will enable checksum and signature verification on the downloaded updater archive with the goal that at some point in the future, this will be the default. This requires that gpg and sha256sum are available on the system to be able to perform the checks. To start using this, pass -verify when calling the script. If -verify is not explicitly passed, update.sh will function the same as before.

Changelog

New Features

  • Build: Bump golang to 1.17.9/Mac M1 Support (#3919)

Enhancements

  • Algod: Add a sourcemap flag for compile endpoint (#3938)
  • Algod API: Remove errant base64 docs reference (#3982)
  • Algokey: Add -v flag to get the version (#4004)
  • Build: Add install_buildtools.sh to check_deps.sh error message (#3921)
  • Ci: use tag to determine channel if possible (#4017)
  • Ci: add simple codecov config (#3997)
  • Cleanup: replace crypto.HashObj(Transaction) with Transaction.ID() (#3958)
  • Devops: Use Cloudflare API token instead of auth key (#4039)
  • Dryrun: Return EvalDeltas for failed executions in Dryrun (#3929)
  • Metrics: collect and report Go runtime.metrics (#4041)
  • Network: Non-participating Nodes Request Tx Gossip Only If Forcefetchtransactions: true (#3918)
  • Tealdbg: Add hidden "generate-docs" command (#3830)
  • Tests: Support e2e-go tests when repo is not installed in GOPATH. (#3949)
  • Tools: Add benchmark warnings for PRs and push graphs for commits into master (#3998)
  • Transaction Sync: Transition test (#3882)

Bugfixes

  • Agreement: fix typo in bandwidthFilter_test (#4028)
  • Ci: ensure msgp generator has been run and is clean (#3978)
  • Developer Tools: Add manjaro support to build script (#3893)
  • Dryrun: Split dryrun cost field into BudgetConsumed and BudgetAdded (#3957)
  • E2e Tests: Fixes for e2e test runs (#3948)
  • Fix: place updater in same directory as update.sh (#3983)
  • Metrics: update node_exporter to support Linux diskstats (#4047)
  • Metrics: make metrics easier to use with prometheus (#4020)
  • Race Condition: Add lock around catchpointsMu to avoid race condition (#3944)

Protocol Upgrade

This release does not contain a protocol upgrade.

vFuture

vFuture changes are not yet available in MainNet or TestNet but can be used in private networks.

New Features

  • AVM: Add bn256 pairing opcodes experimentally (#4013)

Enhancements

  • AVM: base64_decode can decode padded or unpadded encodings (#4015)
  • AVM: Allow immutable access to foreign app accounts (#3994)
  • AVM: Unify inner transaction ID calculation (#3927)
  • Stateproofs: New block header field - SHA256 merkle root of the transactions (#3829)
  • Stateproofs: fix GetProof endpoint response (#3985)

Additional Resources

Algo-devops-service and others added 30 commits March 23, 2022 17:22
While upgrading to golang 1.17.9 a couple of race conditions have been detected during E2E tests.
This fixes catchpoint label assignment.
In e2e.sh when we fail to set the E2E_PLATFORM variable, the shell will complain
that we are using a binary operator (==) on a single argument.
Setting the left side of the test statement to a string with variable interpolation fixes this.

The e2e python client runner does not initialize the stdout and stderr variables
which causes an exception in reportcomms when the except block runs after an exception.
…3949)

Running e2e go tests fails when the repo is not cloned inside the GOPATH.
This change looks up the current directory using runtime.Caller
as an alternative to hard coding the GOPATH location.
Unifies the inner transaction ID calculation with a new consensus parameter, UnifyInnerTxIDs.

Co-authored-by: Ben Guidarelli <[email protected]>
Bump go version to 1.17.9
Also includes //+build => //go:build generated via make sanity
Fixed some data races discovered by 1.17 race detector
…actions (algorand#3829)

Currently, the TxnRoot block header contains the root of the merkle tree
built from the transactions in the block, using the SHA512_256 hash function.
Since the Ethereum VM (and others) does not support SHA512_256 natively,
we have added a new header, which will be used by the Light Clients
deployed on other networks in order to verify Algorand blocks.

Co-authored-by: algoidan <[email protected]>
)

Return evaldeltas for failed execution and check global state
Test for a network in transition.
Runs a 6 node network with the following topology:

New Leaf 1                           New Leaf 2
           > New Relay - Old Relay <
Old Leaf 1                           Old Leaf 2
…lgorand#3958)

* replace crypto.HashObj(Transaction) with Transaction.ID()
This should ensure make msgp has been run for changes
that impact msgp serialization on CI builds.

It looks like algorand#3829 was merged but missed changes to agreement/msgp_gen.go
and gci updates from algorand/msgp#14 were not incorporated into algorand#3919.
* add simple codecov config

* add patch to .codecov.yml
* allow foreign app accounts to be accessed (immutably)
* add bn256 add, scalar multiply and pairing opcode
* replace with gnark bn254 and bench
* update opcost for bn256 according to benchmark


Some doc tweaks, and moved implementation to pairing.go

These opcodes should stay in vFuture until

1. We consider the serialization format
2. We have unit tests
3. We consider BLS 12-381 (and the opcodes of eip 2537)
4. Audit of gnark-crypto library

Co-authored-by: Bo Yao <[email protected]>
Co-authored-by: Bo Yao <[email protected]>
Co-authored-by: bo-abstrlabs <[email protected]>
Co-authored-by: chris erway <[email protected]>
* make TagCounter metrics easier to use with prometheus

* ensure 0 counters are logged

* allow for pre-declaring TagCounter tags for use with prometheus

* fix expected in TestTagCounterWriteMetric

* deregister counter used in test

* fix lint warning

* CR comment

* Log incorrect metrics for debugging test failures

* deregister more counters and tagcounters used by tests

* remove unused Segment
…Transactions: true (algorand#3918)

Save bandwidth by having non-participating non-relay nodes
opt-out of TX transaction gossip traffic using message-of-interest.
To enable set localConfig.ForceFetchTransactions = true

Manual testing has started local private networks to ensure
that the new message-of-interest propagated.
Cluster tests were run to check bandwidth usage.

Co-authored-by: cce <[email protected]>
When generating the CLI documentation for the Developer Portal
it was noticed that tealdbg was missing. Much like goal, kmd, and algokey,
added the "generate-docs" command option to generate the markdown output.
Aharonee and others added 9 commits May 26, 2022 09:21
* add metrics.RuntimeMetrics
* hook up EnableRuntimeMetrics as new config variable
* set EnableRuntimeMetrics: true in netgoal-generated configs
* add EnableRuntimeMetrics to bootstrappedScenario and custom recipes
* check that all defaultRuntimeMetrics are supported by current version of go
* add TestSanitizePrometheusName
* add partitiontest to new tests
* use algod_go_ prefix for prometheus runtime metrics
@codecov
Copy link

codecov bot commented Jun 3, 2022

Codecov Report

Merging #4071 (1657db4) into rel/stable (025f642) will increase coverage by 4.72%.
The diff coverage is 55.90%.

@@              Coverage Diff               @@
##           rel/stable    #4071      +/-   ##
==============================================
+ Coverage       49.81%   54.53%   +4.72%     
==============================================
  Files             409      391      -18     
  Lines           68897    48665   -20232     
==============================================
- Hits            34320    26541    -7779     
+ Misses          30874    19896   -10978     
+ Partials         3703     2228    -1475     
Impacted Files Coverage Δ
agreement/gossip/network.go 54.83% <ø> (ø)
agreement/pseudonode.go 70.07% <ø> (ø)
cmd/goal/clerk.go 8.93% <0.00%> (-0.06%) ⬇️
cmd/tealdbg/main.go 20.33% <0.00%> (-1.09%) ⬇️
config/localTemplate.go 42.85% <ø> (ø)
config/version.go 9.09% <ø> (ø)
crypto/secp256k1/panic_cb.go 0.00% <ø> (ø)
crypto/secp256k1/scalar_mult_cgo.go 77.77% <ø> (ø)
crypto/secp256k1/secp256.go 42.25% <ø> (ø)
daemon/algod/api/server/v1/handlers/handlers.go 0.63% <0.00%> (ø)
... and 64 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 025f642...1657db4. Read the comment docs.

@algojohnlee algojohnlee merged commit 79a08c8 into algorand:rel/stable Jun 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.