Add auto-generated test files

[wing328]

As part of #1921, we've add capabilities to generate test files for API and model files.

If anyone has cycle to add the feature to the following languages, please reply to let us know:

• Android
• Bash
• C++
• C# ([C#] Add unit tests for C# APIs and models #2007)
• Clojure
• Dart
• Elixir
• Flash
• Go (issue #1948, added testing code to test go api #2532, issue #2569: added model tests for Go #2577)
• Groovy
• Javascript
• Java
• ObjC
• Perl ([Perl] generate files for unit testing models and API files #2109)
• PHP ([PHP] generate files for unit testing models and API files #1921)
• Python
• Ruby ([Ruby] add auto-generated test cases (rspec) #1980)
• Swift
• Scala
• TypeScript

Keywords: unit tests, integration tests

[ePaul]

I don't quite understand: What is the purpose of those generated tests?
The PHP tests seem to be empty, so they are not yet testing anything – It looks as after generation they need to be edited to actually test something. But then the "don't edit this file" comment should go away.

But then what is being tested here? The getters/setters of the model objects? (That kind of test, if deemed necessary, should also be generated. Though I would (as a codegen-user, not as a maintainer of codegen) prefer if the generated code just works and I don't have to test it myself.)

What might be useful would be a test of the actual API methods (which is to be run from a test framework against an actual server) – generating some template as a start for your own written tests.

[wing328]

The purpose is to save developer's time in creating those test files from scratch. (I'll remove "don't edit this file" in my next PR)

And yes, ideally we should test getter/setter for each model. The work I've done is just a starting point. Ideally the test fils should leverage all available information provided in the spec to generate all possible test cases.

In my opinion, generating the test code makes the SDK/API client looks more mature. For example, if there are 2 open-source Ruby SDKs for Twitter API - one with test cases and another without any, I would tend to think the one with test cases is more mature. What do you think?

There are scaffolded test cases for API methods, e.g. https://github.com/swagger-api/swagger-codegen/blob/master/samples/client/petstore/csharp/SwaggerClientTest/Lib/SwaggerClient.Test/PetApiTests.cs. Is that what you're looking for?

Thanks again for the feedback. I think what I'll do is to update the wording to elaborate more on the purpose of the scaffolded test cases in the auto-generated test files.

[wing328]

@ePaul does it make sense to you?

Later if anyone from the community has cycle, we can update codegen to automatically generate test cases specified by vendor extension in the endpoint definition.

[wing328]

@ePaul FYI. #2301 to revise the wording has been merged into master.

[ePaul]

@wing328 Thanks, this way it is clearer.

[waiterZen]

Hi, I want to contribute javascript related test code. And I also want to add some feature after driven down the core if I find something can make improvement. If I choose JS, all the pr will commit to this reps?

[wing328]

@waiterZen that's a separate project. You can refer to the PRs above to get a better understanding of what the change (adding auto-generated test files) looks like

[mhuisman]

in java/retrofit2 there is sufficient functionality in ApiClient.mustache to merit adding a default template that offers unit tests for that functionality. I'm shoehorning it into into api_test.mustache but it feels imprecise/ugly. Would you be interested in a PR addressing adding ApiClient_test.mustache?

[wing328]

Would you be interested in a PR addressing adding ApiClient_test.mustache?

Definitely. Please kindly file a PR so that we can review the enhancement. Thanks.

[wing328]

@mhuisman if you need help with the PR, please reply to let us know.

[cbornet]

In that case please do it against branch 2.3.0 since there were some changes in the API.

[grokify]

Given the merged pull requests, what else is necessary for Go to get a ☑?

[ ] Go (#2532, #2577)