Semantic Blackbox Testing #249

tzaffi · 2022-03-18T10:01:46Z

Subroutine Blackbox - E2E Testing using the TEAL Blackbox Toolkit

Graviton introduces the TEAL Blackbox Toolkit.

This PR extends the functionality to PyTEAL. In particular:

Start with a pure @Subroutine and stack a @Blackbox decorator on top to specifying its input_types
Generate an approval program for that subroutine using blackbox_pyteal() that is ready to compile to either a logic sig or an app
Use the TEAL Blackbox Toolkit to make semantic assertions and generate reports

See graviton_test examples 1-3 for detailed usage examples.

Summary of Changes

.github/workflows/build.yml - introduce MakerXStudio sandbox dev repo's prebuilt docker image to allow for integration tests in github actions (and this is blazingly fast - 2 minutes with caching)
Makefile - adding:
- make integration-test
- local-gh-simulate
- sandbox-dev-up/down
README.md - Instructions for running integration test. Additionally, explain how to use the dockerhub algod image.
new file docker-compose.yml (derived from MakerX) that lets us quickly stand up an algod for integration tests on github
examples/signature/factorizer_game.py - pyteal parametrized family of logic sig contract accounts. I used this in my last demo which created and executed more than 10K dry runs on 100+ logic sigs in around 3 minutes.
pyteal/ast/subroutine.py - adding the ability to specify the TealType of @Subroutine parameters
setup.py - requiring the following for development:
- graviton@🦙 - for blackbox testing
- pytest-xdist - for asynchronous testing capability
- tabulate - required by graviton when generating reports
tests/
- blackbox.py- utility dry-running@Subroutines. In particular, an algod_client()is provided together withblackbox_pyteal()which takes as an input a@subroutine+ aModeand outputs compilablePyTeal` ready to dryrun, inspect and assert against.
- integration/ - introducing the integrations tests directory
  - graviton_test.py - New integration test that shows how to graviton can be leveraged to gain confidence in pyteal correctness
  - user_guide_test.py - tests for examples that end up in the user guide including 3 graviton examples. This also shows how to leverage graviton, but using a more straightforward approach than graviton_test.py
- unit/ - new folder which contains the pure unit tests compile_test.py, module_test.py, pre_v6_test.py, and pass_by_ref_test.py
  - blackbox_test.py - companion unit test for blackbox.py
  - pre_v6_test.py - new e2e test that freezes some pre-v6 pyteal's and asserts that their generated TEAL doesn't change
  - pass_by_ref_test.py - add more test asserting that illegal programs don't compile

Timeless Quandary: Do we need caching?

ANSWER: NO! (in this case)

While I have your attention, compare what happens

when miss the docker cache for the sandbox:

compared to what happens when we don't:

Action Items

…drails

…ss-by-ref-guardrails

examples/signature/factorizer_game.py

pyteal/ast/subroutine.py

jasonpaulos · 2022-04-27T21:40:46Z

pyteal/ast/subroutine.py

+
+        self.__name: str = nameStr if nameStr else self.implementation.__name__
+
+    def validate(


Can you prefix it with an underscore to explicitly indicate this is not meant to be called by users?

pyteal/ast/subroutine.py

pyteal/ast/subroutine_test.py

setup.py

jasonpaulos · 2022-04-27T21:53:13Z

setup.py

@@ -21,10 +21,13 @@
            "black==22.3.0",
            "flake8==4.0.1",
            "flake8-tidy-imports==4.6.0",
+            "graviton@git+https://github.com/algorand/graviton@🦙",


What is the plan for publishing graviton to pypi?

I don't have a roadmap for this. Probably worth a discussion. I'm not sure how to gauge the importance of publishing. This is the current popularity of the repo:

tests/unit/module_test.py

jasonpaulos · 2022-04-27T22:04:19Z

This got lost from my review comment:

It seems like utils is only used for tests, so can it be moved to test/utils? As a general rule we should keep the top level directory as simple as possible

tzaffi · 2022-04-28T04:43:50Z

This got lost from my review comment:

It seems like utils is only used for tests, so can it be moved to test/utils? As a general rule we should keep the top level directory as simple as possible

Ok. I've moved utils back under tests

Makefile

pyteal/ast/subroutine.py

tests/unit/blackbox_test.py

tzaffi · 2022-04-28T05:16:37Z

@jasonpaulos - I've either implemented your suggestions or responded to your questions. FYI, regarding the privatizing of BlackboxWrapper and SubroutineDefinition.validate(). I did add an underscore prefix, but I wonder if that really makes sense, given that that tests/unit/blackbox.py::blackbox_pyteal() uses BlackboxWrapper and BlackboxWrapper uses SubroutineDefinition.validate()

LMK

jasonpaulos · 2022-04-28T18:33:01Z

@tzaffi thanks for the changes. Since BlackboxWrapper is not in user-consumable code, I see no reason for it to have an underscore prefix (not to mention I've never seen it added to a class before). SubroutineDefinition.validate() is in user-consumable code, so the underscore communicates that it's not meant to be directly used by consumers and shouldn't be relied upon. However, this is purely informational, so it's perfectly fine if our code continues to use the _validate() method.

Once #249 (comment) is resolved I'll approve.

tzaffi · 2022-04-28T19:13:05Z

@tzaffi thanks for the changes. Since BlackboxWrapper is not in user-consumable code, I see no reason for it to have an underscore prefix (not to mention I've never seen it added to a class before). SubroutineDefinition.validate() is in user-consumable code, so the underscore communicates that it's not meant to be directly used by consumers and shouldn't be relied upon. However, this is purely informational, so it's perfectly fine if our code continues to use the _validate() method.

Once #249 (comment) is resolved I'll approve.

Ok, I believe we should be good to go @jasonpaulos . LMK if you spot anything else.

…iton Logic Sig Example (#322) # Blackbox Testing ABI Functionality This PR brings in dry-run testing of ABI types and subroutines. The space of ABI-Functionality that needs testing is large, and this PR doesn't purport to provide full coverage. However, the basic tools are provided to obtain greater coverage in the future. The following specific ABI use cases are tested: * abi_sum() which computes the sume of a DynamicArray * integers and complex integer number (Gassian integers) are built up from basic types and basic operations are validated * round-trip encoding and decoding of a number of ABI types # Testing Logic Sigs which interact with their Payment Transaction #249 introduced examples/signature/factorizer_game.py which includes a family of logic sigs that provide a payout that depends on the solution to an algebra puzzle. With a recent improvement to graviton we can now run blackbox tests on this logic sig family, and gain confidence in its correctness. See tests/integration/pure_logicsig_test.py for how this is done.

* ABI Types (#154) * initial commit to be used for reference and comparison * Fmt * move abi to ast * make tuple return the deocded Tuple * rename collections, fixed some of the mypy issues * Fix mypy issues * remove Literal * add str method for abi type names * fix names * make imports nice * make imports nice * make collections generic * rm param spec * lie to mypy * make dynamic array work * [WIP] Make Add and Mul NaryExprs * ABI refactor * Implement decoding * Make arrays encode to a single value * Make types mutable * Proof of concept map implementation * Remove old comment * Update Uint16 limit check Co-authored-by: Giorgio Ciotti <[email protected]> * provide a method to know the type name (#176) * Refactor array implementation * Add ComputedType and fix bug in map * Add more int types and bool support * Add string method * Remove files for features that won't make it into this PR * Remove Array.map 😢 * Make decode more powerful * Uint bounds checking and tests * Remove random line * Bool tests and bugfixes * Allow uint and bool set to copy another type * Begin tuple and array tests, improve dynamic tuple encoding, and remove Array.forEach * Test indexTuple and fix bugs * Fix field shadowing * Finish tuple tests and type tests * Implement static and dynamic array tests * Finish testing abi types and fix tuple encoding bug * Add generic param to new instances of arrays * Address feedback * Fix tuple decoding issue * Fix uint64 decoding test * minor improvement to distinguish abi.Byte and abi.Uint8 (#194) * Add docstrings to abi.Type * Respond to feedback * Respond to feedback & fix bug * Fix indexTuple bug * Add docs for TealBlock methods * ABI Array docstrings and comments (#216) Co-authored-by: Jason Paulos <[email protected]> Co-authored-by: Giorgio Ciotti <[email protected]> Co-authored-by: Hang Su <[email protected]> * Merge with master and regenerate `__index__.pyi` (#224) * Fix build script invocation (#223) * Regen __init__.pyi * Reorder CI build steps to change source files before running tests (#225) Co-authored-by: Michael Diamant <[email protected]> * Merge #227 into feature/abi (#231) * Fix build script invocation (#223) * Bring #225 to master (#227) * Ignore tests generating TEAL file outputs used for expected comparisons (#228) * Fix typo in CONTRIBUTING.md (#229) Co-authored-by: Jason Paulos <[email protected]> * Make ABI types fully specified (#222) * Convert Type and Tuple to indexable types * Convert and combine uint types * Fix type and unit tests * Convert Bool * Fix Tuple getitem * Convert arrays and array tests * Fix tuple tests * [WIP] satisfy mypy * Refactor ABI types * Update docs, add Tuple4 and Tuple5, and change type_spec method name * No longer test on Python 3.6 and 3.7 * No longer test on Python 3.6 and 3.7 * Remove accidentally included file * Add another literal test case * Debugging through CI * blacken * Remove problematic test case * Minor test coverage improvements * Unit test to make sure type_spec_from_annotation always works on all types * Partially address feedback * Make uint8 != byte and use typing.Final in a few constructors * Undo custom set methods * Fix typo Co-authored-by: Michael Diamant <[email protected]> Co-authored-by: Michael Diamant <[email protected]> * Passing ABI type arguments to Subroutine (#263) - passing ABI variables into subroutine - subroutine type annotations with ABI types * ABI Method Return (#175) Encode ABI variable and concat with `RETURN_METHOD_SELECTOR` and log it. * Allow `ComputedType` to be passed to the `set` method on ABI types (#267) * update minor support for setting ComputedType * minor tweak * backwards for 3.8 on subscriptable type * attempt to test uint set * minor * cleanup * update arrays, tuple to test * tuple testing * update documents * resolving comments * py3.10 less go * python typing annotations * Merge branch 'master' into feature/abi (#284) * Move to pyteal as pt in ABI tests with concise prefix (#286) * ABI Strings (#278) * Move to pyteal as pt in #278 (#287) * Merge absolute imports into feature/abi (#288) * Remove temporary I252 ignore on pyteal.ast.abi (#290) * Fix abi import (#303) * Fix abi import * ignore flake8 * Feature/abi merge master again (#310) Following major changes: * Merge in Blackbox testing * replace `SubroutineDefinition. _arg_types_and_by_refs()` by `SubroutineDefinition._validate()` * add abi cases to `subroutine_test.py::test_validate()` * merge conflicts * merge again * wip * pass all tests after resolving conflicts * Add `set` on address and string, change array init to accept typespec (#289) * adding set on String and Address types * Add convenience `make` method for ABI types (#326) * Add make method for abi types * Respondn to feedback * ABI Type subroutine return (#256) * wtf * update * update to f-str * define void type * update instantiated computed type returnedType for ABI return in subroutine * minor * update stuffs to help infer type annotation of return ABI * minor * minor * minor * minor * minor * try my best to save stuffs * simplify decorator to single function * tear it down * minor * sheeeesh emacs * updates * minor, renaming something * new design, start over * updates * abi fn wrapper for now * minor * minor update on subroutine def * minor fixes * minor fixes * changes * more constraint on void ret * update comment examples * import from abi * update some error msg and comments * testcases partial * upgrade testscripts * Bundle optional refactorings to subroutine.py (#308) * Bundle optional refactorings to subroutine.py * Refactor to remove branching * storing local changes * pr review partly * pr review partly * update test script * Abi subroutine feature merge (#315) * resolve conflicts * lint * minor reconstruct * missing imports * missing requirements from bad merge * cooperate with typespec n var def change * update comments * some comments resolving? * trim * update some comments * bring testcase back * restriction on output kwarg name * stop ci! i am reformatting * simplify name constrain * resolving comments * Use deferred subroutine expression for ABI returns (#328) * Add subroutine deferred expr * Allow multiple deferred blocks * Remove error * flake8 * Make test more realistic * Add second test * clean up code and comments * remove deferred blocks function * Add coverage for multiple ops in block error * hack return from abi var * a hack * better returned type * simplify * add some compiler test * flake break no way * PR comment resolve partial * polishing Co-authored-by: Michael Diamant <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> Co-authored-by: Jason Paulos <[email protected]> * adding byte type hint to DyanamicArray parent class (#339) * adding byte type hint to DyanamicArray parent class * appease mypy * specify Byte in set method instead of T * Define level of support provided for ABIReturnSubroutine (#341) * Testing ABI Functionality via Graviton / Dry-runs + Txn Friendly Graviton Logic Sig Example (#322) # Blackbox Testing ABI Functionality This PR brings in dry-run testing of ABI types and subroutines. The space of ABI-Functionality that needs testing is large, and this PR doesn't purport to provide full coverage. However, the basic tools are provided to obtain greater coverage in the future. The following specific ABI use cases are tested: * abi_sum() which computes the sume of a DynamicArray * integers and complex integer number (Gassian integers) are built up from basic types and basic operations are validated * round-trip encoding and decoding of a number of ABI types # Testing Logic Sigs which interact with their Payment Transaction #249 introduced examples/signature/factorizer_game.py which includes a family of logic sigs that provide a payout that depends on the solution to an algebra puzzle. With a recent improvement to graviton we can now run blackbox tests on this logic sig family, and gain confidence in its correctness. See tests/integration/pure_logicsig_test.py for how this is done. * abi_roundtrip_test::test_abi_types_comprehensive() (#355) * adding sizeof util (#358) * adding size_of util * Reference Types (#361) * adding reference types to ABI types * Clearer Comment (#364) * Clearer Comment and link comment to issue #184 * Bug: Graviton's report no longer has last_rows (#365) * adding deref type to get the underlying value of the reference type (#372) * `ABI Router` implementation and test (#170) * init commit * minor * minor * rm decorator * update ABI router design, seems we are still working on previous design * minor * update condition parser here * skeleton for handler wrapping, need to tweak on subroutine to expose more message * module * use subroutine fn wrapper * update dummy method return class * unify ast construct way * minor * update notes for tasks and todos * update handler wrapping for bare app call * minor * update redirecting args to method registered * minor * minor * update router src * update program node * add questions in build prog * update executing method branch * minor * minor * minor * hide other methods, set MethodAppArgNumLimit, keyword args on exposed register func, skeleton for arg de-tuple * minor * update notes * update * add a default for no registered ast gen * update ast construction to conds * need some documentations * move abi-router in abi dir * minor * wtf * update * update to f-str * define void type * update instantiated computed type returnedType for ABI return in subroutine * minor * update stuffs to help infer type annotation of return ABI * minor * minor * minor * minor * minor * try my best to save stuffs * simplify decorator to single function * tear it down * minor * sheeeesh emacs * update with latest abi impl * minors * minor * updates * minor, renaming something * new design, start over * updates * abi fn wrapper for now * minor * minor update on subroutine def * minor fixes * minor fixes * changes * more constraint on void ret * update comment examples * import from abi * update some error msg and comments * testcases partial * upgrade testscripts * merging feature/abi to abi-router (#305) * merging feature/abi * missed one * make router use absolute imports * linter happy? * Export Router (#306) * Merge branch 'master' into feature/abi (#284) * Move to pyteal as pt in ABI tests with concise prefix (#286) * ABI Strings (#278) * Move to pyteal as pt in #278 (#287) * Merge absolute imports into feature/abi (#288) * Remove temporary I252 ignore on pyteal.ast.abi (#290) * Fix abi import (#303) * Fix abi import * ignore flake8 * move router to ast Co-authored-by: Michael Diamant <[email protected]> Co-authored-by: Jason Paulos <[email protected]> * fix spacing issue * Bundle optional refactorings to subroutine.py (#308) * Bundle optional refactorings to subroutine.py * Refactor to remove branching * storing local changes * pr review partly * pr review partly * update test script * Abi subroutine feature merge (#315) * resolve conflicts * lint * minor reconstruct * missing imports * missing requirements from bad merge * cooperate with typespec n var def change * update comments * some comments resolving? * trim * update some comments * bring testcase back * restriction on output kwarg name * stop ci! i am reformatting * squash merge abi subroutine atm, merge again after abi subroutine into feature/abi * conform to pep8, reconstructing * update json generator * simplify name constrain * resolving comments * Use deferred subroutine expression for ABI returns (#328) * Add subroutine deferred expr * Allow multiple deferred blocks * Remove error * flake8 * Make test more realistic * Add second test * clean up code and comments * remove deferred blocks function * Add coverage for multiple ops in block error * updating wrap handler * change how to de-tuple args * update checkbox * minor * bug fixes * Method sig for router (#340) * adding method_signature to ABIReturnSubroutine * minor, renaming * minor * simplify * compiler test adding... * documentation * start testcases * update router testcase * docstring tweak * update testcases * update testcases * formatting * branch cond testing * router contract test * subroutine branch wrapper bare call case tested * more testcases to barecall * minor, need to work on method call wrapper * half done on method call wrapper * update on abi method wrapper with return * detuple testcase * compiler test need to see csp * rm redundant type annotation * more error msg * renaming * renaming is_registrable -> is_abi_routable * reording imports, eliminate anti patterns * remove anti-pattern * section it out * non_empty_power_set -> power_set * add conflict detection to protect AST from overshadowing * error message * shrinked commits on pr review * add compile_program * fixup subroutine testcase * remove method signature specifying, directly infer from ABIReturnSubroutine * remove method sign field, directly infer from ABIReturnSubroutine * relaxation of bare app call arg num restriction * add barecall class, api change following * rename barecall to OnCompleteAction * remove outdated comments * start changing router api * eliminate contradiction for oc and creation * remove restriction on cond * new interface: add bare-calls on init, register method-call more fine-grained * documentation for new interface from last discussion * Refactor #170 to reduce visibility and mutation (#362) * Refactor to use partition * Designate _oc_under_call_config as private * hide wrap handler method * adding comment changes * pr review partly * partition into two methods * hide all these methods * arc4 compliant CallConfigs classmethod * arc4 compliant CallConfigs documentation * disclaimer comments * update new abi return hash prefix * review comments partly * OnCompleteActions -> BareCallActions * disclaimer change, rename CallConfigs -> MethodConfig * disclaimer move to docstring * better naming for naming prefix * new ast builder * review comments * An attempt to add decorator syntax to abi-router (#370) * an attempt to use decorator * no return, or we will have python-level issue? * minor fix * per pr comment on on-complete-action * eliminate potential issue * update call config testcase * add testcase for on complete action * per comments * Add a check in `method_signature` to disallow reference return types (#368) * new ast builder * adding check in method_signature to raise error if its attempted to be called on method with reference type as return value * use type spec instead of str * adding recursive type checking method and using it in subroutine method signature * Adding one more test case for extra nesting * appease the linter Co-authored-by: Hang Su <[email protected]> * add new compiler tests for router * fix one test for wrapping bare calls * comments fixing * simplified oc-action post_init check * naming variables * comments * comments * testcase for method config update * add router method only in clearstate prog * closeout -> clearstate * removing is_arc4_compliant * update add method handler and method interface * remove arc4_compliant * Add manual test for 15+ args in abi router (#375) Co-authored-by: Ben Guidarelli <[email protected]> Co-authored-by: Michael Diamant <[email protected]> Co-authored-by: Jason Paulos <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> * Adding method_spec to ABIReturnSubroutine (#380) * adding method_spec to ABIReturnSubroutine * Txn types (#376) * Add transaction abi types * Blackbox testing unifying abi-return-subroutine and subroutine input handling (#374) * unify subroutine case and abi-returning-subroutine * reconstruct code on argument prep and call * an example subroutine taking in abi arguments * update comment example and explanation * per pr comment * Changes to router clear state (#386) * define reference type spec (#383) * define reference type spec Co-authored-by: Jason Paulos <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> * Rename decode parameters to abide by PEP8 conventions (#389) * More abi graviton tests (#391) * test_conditional_factorial() * Router return subroutine (#422) * Adding return value to add_method_call * Method call itxn (#387) * adding itxn method to call ABI method * __str__ formatting (#411) * standardize str formatting * Check transaction type arguments (#427) * Check transaction type arguments * Rename Transaction.set to be private * Check `Expr` length when converting to `abi.Address` (#432) * Fix on previous address length check (#433) * minor fix * minor * per Jason comment * minor, formatting * unit -> uint (#434) * Support declaring compatible PyTeal version in source code (#429) * Add semantic version dependency * Add `pragma` to enforce compatible compiler versions * Raise `TealInternalError` instead of `TealInputError` * Switch back to `pkg_resources` and convert PEP 440 to semantic version 2.0.0 * Fix linter errors * Make pep 440 converter private * Add `Pragma` expression * Refactor pragma functions from compiler to ast to avoid circular dependencies * Add `is_valid_compiler_version` check helper for the `Pragma` expression * Use `TealProgramError` instead of `TealInternalError` * Test underlying TEAL is unchanged * Refactor underlying pragma methods to parent directory * Inherit from `Expr` instead * Fix unclear docstring * Document init Document init * Add caret compiler version tests * Ignore unused fixture imports * Consistently use `start_index` and `end_index` (#436) * Consistently use start_ and end_index * format * allow passing router description to Contract constructor (#448) * allow passing router description to Contract constructor * Document new ABI features (#400) * Initial commit * Fix abi-docs tests (#402) * Fix abi-docs Sphinx warnings (#401) * Extend abi-docs with experimental design language (#403) * Add type fundamentals section * Add basic type usage and some docstrings for referenced methods * Finish documenting set and fix overloaded method docs * Add docstrings for get and __getitem__ * Add reference type docs * Add txn type examples * Fix errors * Make ComputedValue parameter type covariant * ComputedValue and subroutine sections * ...wasn't included in previous commit * Add bare app call and method registration examples * Add router e2e example and compilation explanation * Fix post-merge linter/test failures * Add calling documentation * Partially address feedback * Respond to feedback * Resolve TODOs * Add pragma references * didn't make it into the previous commit * Fix Bool.__module__ * Mention stack size limit * More pragma documentation * Address other feedback * Warn about reference type limits * **cannot** Co-authored-by: Michael Diamant <[email protected]> * Feature/abi merge master (#457) * Clarify minimum Python version management docs (#435) * Foreign prefix on App and Asset arrays (#440) * replacing foreignapps with applications * fix assets as well * Add Execute Method (#444) * adding execute method to allow omission of begin/submit for common use case * exec docstring * update testcase Co-authored-by: Hang Su <[email protected]> Co-authored-by: Michael Diamant <[email protected]> Co-authored-by: Ben Guidarelli <[email protected]> Co-authored-by: Hang Su <[email protected]> * `abi.Bool` improvements (#453) * abi.Bool improvements * Don't need Seq anymore * nor its import * No black, that was not a tuple * modify integreation tests * missed one * Missed a file * Rename pyteal/ast/abi/bool.py functions to follow PEP 8 conventions (#459) * Specify dependency versions (#464) * Rename pyteal/ast/abi/util.py functions to follow PEP 8 conventions (#460) * Rename pyteal/ast/abi/tuple.py functions to follow PEP 8 conventions (#461) * Fix typos in pyteal/ast/abi/util.py (#467) * Rename encoded_string to demarcate internal usage (#466) * Rename pyteal/ast/abi method parameters to follow PEP 8 conventions (#468) Co-authored-by: Ben Guidarelli <[email protected]> Co-authored-by: Jason Paulos <[email protected]> Co-authored-by: Giorgio Ciotti <[email protected]> Co-authored-by: Hang Su <[email protected]> Co-authored-by: Hang Su <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> Co-authored-by: Jacob Daitzman <[email protected]>

Zeph Grunschlag and others added 30 commits March 3, 2022 13:22

start with more test cases

9e2bc7d

This shall morph into the guardrails PR

3f6d775

ignore vscode as well

27d708c

explicit teal version in test functions

75e24c6

refactor to separate old from new

cc1d581

Merge remote-tracking branch 'origin' into recursive-pass-by-ref-guar…

b404552

…drails

wip

b68fb85

why is pyteal compilation flakey?

8a62325

Merge branch 'master' of github.com:algorand/pyteal into recursive-pa…

d37a5d9

…ss-by-ref-guardrails

inline comment per issue #242

5195a3f

wip... but will need to clean up these experiments

2a75511

subroutine input_types

f8e2055

semantic tests starter. next: call dryryn

60f0458

dry run working - next up assertions

03d60c7

passing local tests

5a9c94f

wip

0bed137

getting read for CSV

06d83b5

if error, use it instead of teal in the last trace line

cfc068d

ignore CSV's

c926518

adding a Makefile with pytest coverate command

6a6410e

part 1. refactor to use pysdk as much as possible

6147636

Update .gitignore

b275ad1

Update Makefile

da5e026

Update pyteal/compiler/scratchslots.py

e3e439d

Update tests/semantic_test.py

836249e

try installing git on github!!!

db5836d

rm sudo

b4aca8b

apt -y

e824211

refactor github actions into Makefile

f63d9f5

mypy

bdf8c45

jasonpaulos reviewed Apr 27, 2022

View reviewed changes

Zeph Grunschlag added 6 commits April 27, 2022 22:23

per CR comment + pytest xdist n=auto

ae9067a

multithread as many units as possible as well

06529ac

revert per CR comment

6cc99b3

per CR suggestion

1679ba4

per CR suggestion: sayonara utils - you're just a bunch of tests

853d7c0

privatize validate & BlackboxWrapper + typehints in validate()

d5cb493

tzaffi commented Apr 28, 2022

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Update Makefile

c14e919

tzaffi commented Apr 28, 2022

View reviewed changes

pyteal/ast/subroutine.py Outdated Show resolved Hide resolved

modern union

7935fae

tzaffi commented Apr 28, 2022

View reviewed changes

tests/unit/blackbox_test.py Outdated Show resolved Hide resolved

Update tests/unit/blackbox_test.py

bc9d5dd

Zeph Grunschlag added 3 commits April 28, 2022 13:44

no underscore for BlackboxWrapper

843ed96

Merge remote-tracking branch 'origin' into subr-blackbox

7619773

as CR suggested, tabulate comes for free with graviton

cd1654b

jasonpaulos approved these changes Apr 28, 2022

View reviewed changes

tzaffi merged commit cc86572 into master Apr 28, 2022

tzaffi deleted the subr-blackbox branch April 28, 2022 19:47

This was referenced May 2, 2022

no utils pls #314

Merged

Use Docker Image for C.I. algorand/graviton#14

Closed

tzaffi mentioned this pull request May 17, 2022

Testing ABI Functionality via Graviton / Dry-runs + Txn Friendly Graviton Logic Sig Example #322

Merged

7 tasks

michaeldiamant mentioned this pull request Dec 22, 2022

Prevent blackbox_test_runner from passing without invariant assertions #621

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Semantic Blackbox Testing #249

Semantic Blackbox Testing #249

tzaffi commented Mar 18, 2022 •

edited

Loading

jasonpaulos Apr 27, 2022

jasonpaulos Apr 27, 2022

tzaffi Apr 28, 2022 •

edited

Loading

jasonpaulos commented Apr 27, 2022 •

edited

Loading

tzaffi commented Apr 28, 2022

tzaffi commented Apr 28, 2022 •

edited

Loading

jasonpaulos commented Apr 28, 2022

tzaffi commented Apr 28, 2022


		self.__name: str = nameStr if nameStr else self.implementation.__name__

		def validate(

Semantic Blackbox Testing #249

Semantic Blackbox Testing #249

Conversation

tzaffi commented Mar 18, 2022 • edited Loading

Subroutine Blackbox - E2E Testing using the TEAL Blackbox Toolkit

Summary of Changes

Timeless Quandary: Do we need caching?

ANSWER: NO! (in this case)

when miss the docker cache for the sandbox:

compared to what happens when we don't:

Action Items

jasonpaulos Apr 27, 2022

Choose a reason for hiding this comment

jasonpaulos Apr 27, 2022

Choose a reason for hiding this comment

tzaffi Apr 28, 2022 • edited Loading

Choose a reason for hiding this comment

jasonpaulos commented Apr 27, 2022 • edited Loading

tzaffi commented Apr 28, 2022

tzaffi commented Apr 28, 2022 • edited Loading

jasonpaulos commented Apr 28, 2022

tzaffi commented Apr 28, 2022

tzaffi commented Mar 18, 2022 •

edited

Loading

tzaffi Apr 28, 2022 •

edited

Loading

jasonpaulos commented Apr 27, 2022 •

edited

Loading

tzaffi commented Apr 28, 2022 •

edited

Loading