Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CodeGenerator unit test #117

Merged
merged 22 commits into from
Feb 11, 2021
Merged

CodeGenerator unit test #117

merged 22 commits into from
Feb 11, 2021

Conversation

nbaztec
Copy link
Contributor

@nbaztec nbaztec commented Dec 29, 2020

A follow up from #104 on #48

The approach is as follows:

  • Obtain FileDescriptorSet from proto compilation
    • Use protoc provided in PATH, if not present download it
    • Use resources/*.proto files to generate temporary descriptor_set
  • Parse the List<FileDescriptorProto> for CodeGenerator
  • Use KotlinCompilation to verify generated code

Open to suggestions.

@nbaztec nbaztec marked this pull request as ready for review December 31, 2020 18:43
@nbaztec
Copy link
Contributor Author

nbaztec commented Jan 6, 2021

@garyp any opinions?

Copy link
Collaborator

@garyp garyp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @nbaztec. Thanks! This will make it much easier to add unit tests of the code generator.

My only feedback is that it'd be better to do the protoc download and protoc execution as gradle tasks rather than as part of the unit test. That will allow the results to be cached properly by gradle rather than having to be rerun every time the tests run. Specifically:

  • Create a gradle task that uses ProtocTask to run protoc and generate the descriptor set into a file under the gradle build directory. The tests can then depend on this gradle task.

  • The current ProtocTask doesn't download protoc if it doesn't already exist. That has been a bit of an annoyance and it'd be nice if it downloaded protoc, though you don't have to implement it in this PR if you don't want to. If you do implement it, rather than manually fetching protoc from a URL I would suggest instead declaring a new gradle configuration and adding a dependency in that configuration on the protoc maven artifact. Then the gradle task (from the previous point) can depend on that configuration and gradle will take care of downloading and caching for you.

@nbaztec
Copy link
Contributor Author

nbaztec commented Jan 9, 2021

@garyp I tried to implement as per your suggestions, please take a look. My Gradle competencies are quite limited so feel free to suggest changes.

@nbaztec nbaztec requested a review from garyp January 9, 2021 18:04
Copy link
Collaborator

@garyp garyp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly what I had in mind. There are a few minor things to address, but then this should be good to merge.

I do think it'd make more sense to have DescriptorProtocTask extend the existing ProtocTask. Looking at the code you have here, it'd only take minimal changes to ProtocTask in order to make it useable for this use case. However, in the interest of iterative development, we can merge this version after you address the other comments, and then refactor DescriptorProtocTask to extend ProtocTask in a future PR.

buildSrc/src/main/kotlin/DescriptorProtocTask.kt Outdated Show resolved Hide resolved
protoc-gen-kotlin/lib/build.gradle.kts Outdated Show resolved Hide resolved
protoc-gen-kotlin/lib/build.gradle.kts Outdated Show resolved Hide resolved
protoc-gen-kotlin/lib/build.gradle.kts Outdated Show resolved Hide resolved
protoc-gen-kotlin/lib/build.gradle.kts Outdated Show resolved Hide resolved
protoc-gen-kotlin/lib/build.gradle.kts Outdated Show resolved Hide resolved
@nbaztec nbaztec requested a review from garyp January 17, 2021 14:21
Copy link
Collaborator

@garyp garyp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor nit, otherwise looks good.

@@ -1,6 +1,7 @@
plugins {
kotlin("multiplatform")
`maven-publish`
id("com.google.osdetector") version "1.6.2"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure the Gradle plugin versions are consistent across the entire project, we declare all plugins in the top-level build.gradle.kts file with apply false. Then we apply them in the individual Gradle subprojects as needed.

Copy link
Collaborator

@garyp garyp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for adding this!

@nbaztec
Copy link
Contributor Author

nbaztec commented Jan 19, 2021

I think the task compileTestJs has some issues. I've got it once before and re-running the build (with empty commit) worked:

> Task :runtime:compileTestKotlinJs
e: java.lang.StackOverflowError
	at org.jetbrains.kotlin.js.inline.clean.TemporaryVariableElimination$analyze$1.visitBinaryExpression(TemporaryVariableElimination.kt)
	at org.jetbrains.kotlin.js.backend.ast.JsBinaryOperation.accept(JsBinaryOperation.java:47)
	at org.jetbrains.kotlin.js.backend.ast.JsVisitor.accept(JsVisitor.kt:11)

Would it be fine if you re-ran it or should I add empty commits?

@garyp
Copy link
Collaborator

garyp commented Jan 19, 2021

@nbaztec I'm rerunning it. We've been seeing a lot more StackOverflowErrors in the compiler with Kotlin 1.4 😞 Do you remember if the last time you saw this error it was also in the compileTestKotlinJs task?

@nbaztec
Copy link
Contributor Author

nbaztec commented Jan 19, 2021

I think yes, here's the build https://github.com/streem/pbandk/runs/1673410304

@garyp garyp linked an issue Jan 19, 2021 that may be closed by this pull request
@garyp garyp merged commit 5ae59c9 into streem:master Feb 11, 2021
garyp added a commit that referenced this pull request Feb 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add the ability to unit test the output of protoc-gen-kotlin
2 participants