googleapis / api-common-protos Goto Github PK

There are some examples inside the proto itself (resource.proto) which adds some references to a non-existent name_descriptor field.

Furthermore, the AIP establishes the use of a type & pattern properties to define the metadata associated to a specific resource. The comments make a plan to use those in a different way by wrapping it on a custom property (name_descriptor) and make it repeatable.

Is it possible to clarify this? Thanks!

@noahdietz @lukesneeringer as you were the ones involved in the MR.

The protos in google/api here are out of sync with https://github.com/googleapis/googleapis/tree/master/google/api

python microgenerator uses google.api python files from this repo, but generates code using protos from googleapis. The mismatch causes the generated unit tests fail to run.

Policy Bot found one or more issues with this repository.

• Default branch is 'main'
• Branch protection is enabled
• Renovate bot is enabled
• Merge commits disabled
• There is a CODEOWNERS file
• There is a valid LICENSE.md
• There is a CODE_OF_CONDUCT.md
• There is a CONTRIBUTING.md
• There is a SECURITY.md

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency io_bazel_rules_go to v0.47.0

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update alpine docker tag to v3.19
• chore(deps): update python docker tag to v3.12
• chore(deps): update ubuntu docker tag to v22.10
• chore(deps): update ubuntu docker tag to v24

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

The current field name of package for product metadata on the input-contract branch is problematic for Java/Kotlin because protoc does not check for reserved keywords.

Can we change the name before it goes mainstream? I did open a PR for an upstream fix, but changing the name is probably easier (maybe package_metadata?): protocolbuffers/protobuf#5709

cc/ @noahdietz @timburks - I'll update kiosk once this is resolved.

Hi,

Thanks for the C# Google.Api.CommonProtos NuGet package.
Could you also provide a vcpkg equivalent for C++? There is already a vcpkg for google-cloud-cpp, but this only includes the common protos as "third party" in vcpkg/buildtrees/google-cloud-cpp/src/v0.1.0-533022434b/third_party/googleapis/google/type/date.proto
With a dedicated vcpkg this would make the common protos available seamlessly to Visual Studio 2017 C++ projects from vcpkg/installed/x64-windows/include/google/type/date.proto
This is what the protobuf vcpkg provides (vcpkg/installed/x64-windows/include/google/protobuf/timestamp.proto)

Regards,

Trying to solve

Module not found: Error: Can't resolve './google/api/annotations_pb.js'

I've documented the use case here grpc-ecosystem/grpc-gateway#834
Another related issue protocolbuffers/protobuf-javascript#62

Error logging currently generates following warnings:

Google\Cloud\Logging\V2\LogSink_VersionFormat is deprecated and will be removed in the next major release. Use Google\Cloud\Logging\V2\LogSink\VersionFormat instead
 
-> app/vendor/google/cloud-logging/src/V2/LogSink_VersionFormat.php

Google\Rpc\QuotaFailure_Violation is deprecated and will be removed in the next major release. Use Google\Rpc\QuotaFailure\Violation instead

-> app/vendor/google/common-protos/src/Rpc/QuotaFailure_Violation.php 

Google\Rpc\PreconditionFailure_Violation is deprecated and will be removed in the next major release. Use Google\Rpc\PreconditionFailure\Violation instead

-> /app/vendor/google/common-protos/src/Rpc/PreconditionFailure_Violation.php

Google\Rpc\BadRequest_FieldViolation is deprecated and will be removed in the next major release. Use Google\Rpc\BadRequest\FieldViolation instead

-> /app/vendor/google/common-protos/src/Rpc/BadRequest_FieldViolation.php

Google\Rpc\Help_Link is deprecated and will be removed in the next major release. Use Google\Rpc\Help\Link instead

-> /app/vendor/google/common-protos/src/Rpc/Help_Link.php

This is thrown from generated PHP protobufs, but I assume that the root cause is outdated references in source protobuf definitions.
We are observing it while using the latest 0.1.0 version of google/common-protos.

After updating my google.Protobuf Library from 3.8.0 to 3.9.1 and GRPC Library from 1.22.0 to 2.25,0.
I am facing this runtime crash : "Could not load signature of Google.Type.CalendarPeriodReflection:get_Descriptor due to: Could not load file or assembly 'Google.Protobuf, Version=3.8.0.0, Culture=neutral, PublicKeyToken=a7d26565bac4d604' or one of its dependencies. assembly:Google.Protobuf, Version=3.8.0.0, Culture=neutral, PublicKeyToken=a7d26565bac4d604 type: member:(null) signature:"

Is there any missing class in google.Protobuf 3.9.1 or any compatibility issues.

Here is some date range related proto definitions. Do you have a plan to port these protos to this repository?

• https://github.com/googleapis/googleapis/tree/master/google/actions/type

Added to googleapis a few months ago.

Is the intention to keep this repo up to date, or is it wiser to depend on the googleapis repo? Cheers!

is there any difference or customization to the pregenerated client code you provide?

There is a bunch of invalid version matchers in setup.py. PEP 440 states:

The canonical public version identifiers MUST comply with the following scheme:
[N!]N(.N)*[{a|b|rc}N][.postN][.devN]

So you are missing a dot and a number in every version identifier that contains the string dev.

It is also considered bad practice to have an upper bound on package versions and installers like pip do not typically consider development versions in any case (unless explicitly told to).

See:

• googleapis/google-api-python-client#2151
• googleapis/python-api-common-protos#165
• googleapis/google-auth-library-python#1321
• googleapis/python-api-core#504

Edit: sorry - one too many.

Environment details

• Programming language: C++
• OS: Ubuntu 22.04
• Language runtime version: Protobuf 3.12.4
• Package version: 1.50.0

Steps to reproduce

1. Run protoc on all the .proto files
2. The following warnings are reported:

google/api/auth.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/api/billing.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/api/endpoint.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/api/logging.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/api/monitoring.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/api/quota.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/api/service.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/api/usage.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/iam/v1/logging/audit_data.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/iam/v1/policy.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/logging/type/http_request.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/logging/type/log_severity.proto:19:1: warning: Import google/api/annotations.proto is unused.

3. The generation then completes successfully

Discussion

These warnings don't break anything, but they are a false positive. They waste our and our user's time by bringing attention to a non-existent problem. We could disable printing of all Protobuf warnings in our build setup, but then we could miss some actually important issues.

I suggest to remove the specified unused includes from those files. Or is there a flag I can pass to protoc to silence specifically "unused import" warnings?

I was trying to define one parameter that could reference different types with no luck. The problem can be easily shown using IAM's GetRole method as an example:

// The request to get the definition of an existing role.
message GetRoleRequest {
  // The resource name of the role in one of the following formats:
  // `roles/{ROLE_NAME}`
  // `organizations/{ORGANIZATION_ID}/roles/{ROLE_NAME}`
  // `projects/{PROJECT_ID}/roles/{ROLE_NAME}`
  string name = 1;
}

In this case google.api.resource_reference cannot be used as only one type can be referenced.

I can think of two different ways to solve this:

• Make resource_reference.type a repeated field
• Use oneof in name

I think the first one would be the cleanest option but WDYT?

Any help would be much appreciated!

Is your feature request related to a problem? Please describe.

Modern python can be type-checked using mypy, but the python distribution of this package does not include the type specs, nor is there a companion type package.

Describe the solution you'd like

Types generated using e.g. https://pypi.org/project/mypy-protobuf/ included either directly in the python googleapis-common-protos package or a separate googleapis-common-protos-stubs package. A separate stubs package might help with avoiding accidental breakage.

Describe alternatives you've considered

I could check in the googleapi protos into the repository where I'm using them and add codegen from that.

Additional context
For the grpcio packages, there's stubs maintained by a third-party, https://pypi.org/project/grpc-stubs/. The protoc plugin https://pypi.org/project/mypy-protobuf/ codegens stubs for proto files (messages and grpc clients/handlers).

As mentioned by @jbolinger here, it seems useful to have something more than a commit hash to identify releases. A datestamp might be better than a semantic version number, but I don't really have a preference.

Policy Bot found one or more issues with this repository.

• Default branch is 'main'
• Branch protection is enabled
• Renovate bot is enabled
• Merge commits disabled
• There is a CODEOWNERS file
• There is a valid LICENSE.md
• There is a CODE_OF_CONDUCT.md
• There is a CONTRIBUTING.md
• There is a SECURITY.md

I had a quick look at a few protobuf files from googleapis and it seems like this repository is not being kept up-to-date with the changes made to protobuf files within googleapis. Is this repository still being migrated to, as detailed in googleapis/go-genproto#92 (comment)?

In the 2.0.1 jar, google/api/endpoint.proto contains a repeated string features = 4, however the corresponding Java class file doesn't have any of the methods corresponding to this field.

The Python package resulting from this repository and published on PyPI is not compatible with the new released versions of protobuf because it has an upper-limit on <5.0.0.dev0.

Hi, I cannot understand how to correctly use google/rpc/*.proto definitions in my project. Please explain the best practices here.

I copied google/rpc/*.proto into my repository with protos. Compiled them with buf tool and got all three go modules in one directory:
google/rpc/code.pb.go google/rpc/error_details.pb.go google/rpc/status.pb.go. Each module declares different package name:
package code, package errdetails, package status. This results in a compilaion errror, because Go requires that all modules in one same dir must declare the same one package, not three different ones.

How am I supposed to use those google/rpc/*.proto in my project, please give an advice.

Here OPTIONAL is used:

But minwindef.h unconditionally defines OPTIONAL to nothing and so generated code fails to build.
The include comes via grpc/include/grpc/impl/codegen/port_platform.h which is included into generated code.

The version numbers used for the compiled packages are not consistent across languages.

For example, the current C# package version is 2.2.0, the current Java package version is 2.0.1, and the current PyPI package version is 1.52.0.

When working in a multi-language environment, these discrepancies make it difficult to ensure compatibility between projects. Additionally, if you are using a local clone of this repo to compile your own proto files to be used in your application(s), then aligning the version of the language packages with the version of the protos found in the local clone is very difficult.

Would it be possible to align these version numbers in some way? Ideally, it would be great if the version number was tied to a release tag on this repo.

Related issue: #17

We are currently working using different languages, using protos to generate some of the classes (some of them have dependencies on protos defined in this project). To ship our Swift project we have to generate and compile some protos under google.rpc.*, while on Java we just add them as a dependency.

It would be great if you could provide Swift packages with already compiled protos just like in other languages.

According to the note in https://github.com/googleapis/api-common-protos#packages

Note that if using these packages, you will still need a local copy of these protos when using protoc, but you will not need to ship compiled versions of them. (This is consistent with protoc's default behavior of only providing compiled output for the files specifically requested, and not their imports.)

This does not seem right for js package.
I protoc compiled my proto file with those imports from google-proto-files.
Apparently I have to ship compiled version of imported protos too because the google-proto-files does not contain them (unlike the packages for other languages or imports from google/protobuf/).

If you unpack https://files.pythonhosted.org/packages/61/29/1549f61917eadd11650e42b78b4afcfe9cb467157af4510ab8cb59535f14/googleapis-common-protos-1.5.6.tar.gz, some files in googleapis_common_protos.egg-info directory have 640 permissions instead of 644.

I believe it should be 644.

Real-world impact is e.g. you cannot use it in AWS Lambda, as it's run from a different user, and will throw access denied exception on namespace_packages.txt file.

Similar to unrelated bug https://bugs.archlinux.org/task/30020

Environment details

• Programming language: Python
• OS: Ubuntu 20.04 LTS
• Language runtime version: 3.8.10
• Package version: 1.58.0

Steps to reproduce

1. Install googleapis-common-protos==1.58.0 into a virtual environment.
2. Create a file called test.py with the following contents:

import google.rpc

3. Run python -Wall test.py inside the virtual environment and observe the deprecation warnings:

.../lib/python3.8/site-packages/pkg_resources/__init__.py:2804: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
.../lib/python3.8/site-packages/pkg_resources/__init__.py:2804: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
.../lib/python3.8/site-packages/pkg_resources/__init__.py:2309: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)
.../lib/python3.8/site-packages/google/rpc/__init__.py:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
pkg_resources.declare_namespace(__name__)
.../lib/python3.8/site-packages/pkg_resources/__init__.py:2309: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(parent)

The issue seems to be coming primarily from the init files generated here:

In our test suites warnings are considered errors, and as google-api-core depends on googleapis-common-protos, this issue breaks every run where we import any Google API SDK package.

I'd like to import some of the google.api protos into my own proto, but I need to make sure I'm on the correct commit hash for the released PyPI package (e.g., 1.53.0) because my team depends on the PyPI package.

I searched the git log for 1.53.0 or some of the previous version numbers but found nothing.

Is there a way to identify the commits for released packages?

Can we also include google/protobuf/descriptor.proto here?

This not being synced causes go-genproto to ignore changes made internally googleapis/go-genproto#92, in turn causing grpc-ecosystem/grpc-gateway#712 to break because a valid change has been steamrolled by automation.

The protos in the googleapis/api-common-protos are not updated automatically and may be outdated. This issue tracks updating the README to direct users to the protos in googleapis/googleapis instead which are updated regularly.

I'm able to import the java artifact containing these protobuf models, but it's lacking the Kotlin generated code which makes the models easier to work with.

How can I import an artifact that contains the protobuf models with a Kotlin language target?

I apologize for writing this googleapis issue here, as googleapis has not enabled Issues.

I found a proto file that is not suitable for the Go language.
https://github.com/googleapis/googleapis/blob/master/google/ads/googleads/v10/resources/experiment_arm.proto

This file will be converted experiment_arm.pb.go by protoc-gen-go. However, the Go language compiler interprets the _arm suffix as an architecture type, so it cannot be compiled on non-ARM machines.
Of course, if you rename the file, there is no problem. I guess it's up to the person building the file to be careful, but it's a little uncool. Perhaps, it doesn't work well with automatic build system.

I think it would be better to avoid such file names.
What do you think?

Stackdriver monitoring has been updated and it will be very useful to update the protos here as well to reflect the latest changes in the api/distribution.proto.

See:
https://github.com/googleapis/api-common-protos/blob/master/google/api/distribution.proto
https://github.com/googleapis/googleapis/blob/master/google/api/distribution.proto