Add hypothesis tests #532
Add hypothesis tests #532
Conversation
This adds initial support for hypothesis tests. I've seeded this with a couple of tests for app.py. In addition I've added configuration such that more iterations of these tests are run on travis.
There was a problem hiding this comment.
Looks fine to me assuming the travis job passes. Out of curiosity what is the reasoning for splitting this out to another directory instead of putting it under something like under the unit test directory? There is not really a precedence in any of the other aws or boto org projects. So just want to make sure this is precedence we want to set if we expand hypothesis testing to the rest of the projects.
Codecov Report
@@ Coverage Diff @@
## master #532 +/- ##
=======================================
Coverage 94.39% 94.39%
=======================================
Files 18 18
Lines 3107 3107
Branches 397 397
=======================================
Hits 2933 2933
Misses 126 126
Partials 48 48Continue to review full report at Codecov.
|
|
Capturing offline discussion. It seems most projects don't separate property tests from their other tests, and the only reason I've done it this way in other projects is because I didn't also have to handle functional, integration tests etc. So I'll:
|
Based on review feedback. This also makes it easier to add property tests as functional and integration tests. I've set the default number of iterations to 10 by default. On travis, we bump that up to 2000 via the HYPOTHESIS_PROFILE env var.
|
@kyleknap ready to look at again. |
This adds initial support for hypothesis tests. I've seeded this
with a couple of tests for app.py. In addition I've added
configuration such that more iterations of these tests are run on
travis.
Originally mentioned in #506 (comment).