Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: search gapic additional protos in BUILD.bazel #2004

Merged
merged 9 commits into from
Sep 20, 2023

Conversation

JoeWang1127
Copy link
Collaborator

Fixes #1999 ☕️

@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Sep 18, 2023
@JoeWang1127
Copy link
Collaborator Author

The following list is used in testing:

google/cloud/clouddms/v1 google-cloud-clouddms-v1-java
google/cloud/discoveryengine/v1 google-cloud-discoveryengine-v1-java
google/cloud/edgecontainer/v1 google-cloud-edgecontainer-v1-java
google/cloud/networkconnectivity/v1 google-cloud-networkconnectivity-v1-java
google/cloud/retail/v2 google-cloud-retail-v2-java
google/privacy/dlp/v2 google-cloud-privacy-dlp-v2-java

@JoeWang1127 JoeWang1127 marked this pull request as ready for review September 18, 2023 19:44
@JoeWang1127 JoeWang1127 requested a review from a team as a code owner September 18, 2023 19:44
Comment on lines 94 to 107
### contains_iam_policy (optional)
Whether to include `google/iam/v1/iam_policy.proto` into the library.
The value is either `true` or `false`.
The default value is `false`.

Use `--contains_iam_policy` to specify the value.

### contains_locations (optional)
Whether to include `google/cloud/location/locations.proto` into the library.
The value is either `true` or `false`.
The default value is `false`.

Use `--contains_locations` to specify the value.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see this list growing over time. Would it make sense to just have additional_protos parameter?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean additional_protos as a string of protos?
For example:
additional_protos="google/iam/v1/iam_policy.proto google/cloud/location/locations.proto" or additional_protos="google/cloud/location/locations.proto"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to make it more generic. It would solve the problem for common shopping protos as well. I'm also curious why google/longrunning/operations.proto is not affected by this issue.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blakeli0 post some findings in the issue that answers your question.

Copy link
Collaborator Author

@JoeWang1127 JoeWang1127 Sep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blakeli0 @meltsufin I plan to name additional protos that pass to the generator gapic_additional_protos, WDYT?

So far I found google/cloud/location/locations.proto, google/iam/v1/iam_policy.proto and google/cloud/common_resources.proto may go to this list.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@sonarqubecloud
Copy link

[gapic-generator-java-root] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@sonarqubecloud
Copy link

[java_showcase_integration_tests] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@sonarqubecloud
Copy link

[java_showcase_unit_tests] Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@JoeWang1127 JoeWang1127 changed the title fix: search additional protos in BUILD.bazel fix: search gapic additional protos in BUILD.bazel Sep 19, 2023
@JoeWang1127 JoeWang1127 changed the title fix: search gapic additional protos in BUILD.bazel feat: search gapic additional protos in BUILD.bazel Sep 19, 2023
result="${if_match}"
fi
echo "${result}"
}

__get_iam_policy_from_BUILD() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to search for these protos from BUILD now that they are being passed in as parameters?
In addition, do we need to read anything else from BUILD file other than searching for common protos?

Copy link
Collaborator Author

@JoeWang1127 JoeWang1127 Sep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to search for these protos from BUILD now that they are being passed in as parameters?

All functions that read BUILD are used by generate_library_integration_test.sh for testing against googleapis-gen. They are not in test_utilities.sh because they are part of generate_sdk_libraries.sh later.

In addition, do we need to read anything else from BUILD file other than searching for common protos?

transport, rest_numeric_enums and include_samples are also read from BUILD for testing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All functions that read BUILD are used by generate_library_integration_test.sh for testing against googleapis-gen. They are not in test_utilities.sh because they are part of generate_sdk_libraries.sh later.

I'm not sure we need it in generate_sdk_libraries.sh either. We would still need to keep the bazel interface, so for a single client library, the flow would be bazel build -> java_gapic_assembly_gradle_pkg -> generate_library.sh, where java_gapic_assembly_gradle_pkg would pass all the info from bazel file to our shell script. So generate_sdk_libraries.sh would just be calling bazel build for each service in this case.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that's the case then we probably don't need these utility functions.

Right now it's only used in testing and can be removed without impacting generate_library.sh.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now it's only used in testing and can be removed without impacting generate_library.sh.

We can still keep them and move them to the test utils.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can still keep them and move them to the test utils.

How about I create another PR to refactor test utilities into test_utilities.sh. I think there are other functions can move into test_utilities.sh as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@JoeWang1127 JoeWang1127 merged commit ed16ac7 into main Sep 20, 2023
41 checks passed
@JoeWang1127 JoeWang1127 deleted the fix/search-addition-protos-in-BUILD branch September 20, 2023 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: m Pull request size is medium.
Projects
None yet
4 participants