Giter Club home page Giter Club logo

api-compiler's Introduction

Google APIs

This repository contains the original interface definitions of public Google APIs that support both REST and gRPC protocols. Reading the original interface definitions can provide a better understanding of Google APIs and help you to utilize them more efficiently. You can also use these definitions with open source tools to generate client libraries, documentation, and other artifacts.

Building

Bazel

The recommended way to build the API client libraries is through Bazel >= 4.2.2.

First, install bazel.

To build all libraries:

bazel build //...

To test all libraries:

bazel test //...

To build one library in all languages:

bazel build //google/example/library/v1/...

To build the Java package for one library:

bazel build //google/example/library/v1:google-cloud-example-library-v1-java

Bazel packages exist in all the libraries for Java, Go, Python, Ruby, Node.js, PHP and C#.

Overview

Google APIs are typically deployed as API services that are hosted under different DNS names. One API service may implement multiple APIs and multiple versions of the same API.

Google APIs use Protocol Buffers version 3 (proto3) as their Interface Definition Language (IDL) to define the API interface and the structure of the payload messages. The same interface definition is used for both REST and RPC versions of the API, which can be accessed over different wire protocols.

There are several ways of accessing Google APIs:

  1. JSON over HTTP: You can access all Google APIs directly using JSON over HTTP, using Google API client library or third-party API client libraries.

  2. Protocol Buffers over gRPC: You can access Google APIs published in this repository through GRPC, which is a high-performance binary RPC protocol over HTTP/2. It offers many useful features, including request/response multiplex and full-duplex streaming.

  3. Google Cloud Client Libraries: You can use these libraries to access Google Cloud APIs. They are based on gRPC for better performance and provide idiomatic client surface for better developer experience.

Discussions

This repo contains copies of Google API definitions and related files. For discussions or to raise issues about Google API client libraries, GRPC or Google Cloud Client Libraries please refer to the repos associated with each area.

Repository Structure

This repository uses a directory hierarchy that reflects the Google API product structure. In general, every API has its own root directory, and each major version of the API has its own subdirectory. The proto package names exactly match the directory: this makes it easy to locate the proto definitions and ensures that the generated client libraries have idiomatic namespaces in most programming languages. Alongside the API directories live the configuration files for the GAPIC toolkit.

NOTE: The major version of an API is used to indicate breaking change to the API.

Generate gRPC Source Code

To generate gRPC source code for Google APIs in this repository, you first need to install both Protocol Buffers and gRPC on your local machine, then you can run make LANGUAGE=xxx all to generate the source code. You need to integrate the generated source code into your application build system.

NOTE: The Makefile is only intended to generate source code for the entire repository. It is not for generating linkable client library for a specific API. Please see other repositories under https://github.com/googleapis for generating linkable client libraries.

Go gRPC Source Code

It is difficult to generate Go gRPC source code from this repository, since Go has different directory structure. Please use this repository instead.

api-compiler's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

api-compiler's Issues

NPE when compiling an openapi spec to a service config

Using an openapi.yaml spec from a tutorial:

$ java -jar build/libs/api-compiler-with-deps-0.0.7.jar --openapi openapi.yaml --json_out api.json --bin_out api.bin 
Exception in thread "main" java.lang.NullPointerException
	at com.google.api.tools.framework.tools.ModelBuilder.parseFileAsDescriptorSet(ModelBuilder.java:134)
	at com.google.api.tools.framework.tools.ModelBuilder.parseFileDescriptors(ModelBuilder.java:100)
	at com.google.api.tools.framework.tools.ModelBuilder.setup(ModelBuilder.java:53)
	at com.google.api.tools.framework.tools.ToolDriverBase.setupModel(ToolDriverBase.java:81)
	at com.google.api.tools.framework.tools.ToolDriverBase.run(ToolDriverBase.java:72)
	at com.google.api.tools.framework.tools.configgen.ServiceConfigGeneratorTool.main(ServiceConfigGeneratorTool.java:44)

Publish releases as jars with dependencies

Please publish the released artifacts here on github.
The tool requires checking out the repo and is sensitive to which directory you run "run.sh".
It is a lot easier to just run java -jar api-compiler.jar.

JSON name of proto field unhelpful.

If an interface is defined with the following simple .proto:

message Metasyntactic {
    bool foobar = 1;
    bool fooBar = 2;
    bool foo_bar = 3;
}

...and compiled with protoc:

protoc --include_imports  \
  --include_source_info \
  --proto_path=. meta.proto \
  --descriptor_set_out out.pb

Google API Compiler can be used to generate the Service Configuration file:

$API_COMPILER_HOME./run.sh \ 
  --configs grpc-api-config.yaml \
  --descriptor out.pb \
  --json_out service-config.json

The resulting service configuration file defines a google.protobuf.type for each field.

  • foobar remains foobar
  • fooBar' has a JSON name of foo_bar`
  • foo_bar has a JSON name of fooBar

There are a few problems with this:

  • When an API deployed with this service configuration, any client must use a different naming convention from the server.
  • A Proto Descriptor file and gRPC API Configuration YAML file can be used to deploy a service (API) to Cloud without the need for an explicit service configuration file. This approach does not rename any proto message fields. Therefore, any client code must change significantly depending on how the API is deployed.
    • Use the API Manager and subsequent API configuraton file, and fooBar is received as foo_bar.
    • Use a proto descriptor and gRPC config file, and fooBar is received as fooBar.

Resources can't be found on Windows due to use of File.separator

Running the toolkit runCodeGen Gradle task on Windows, I get an error:

top level:1: cannot open source 'auth_document.snip': java.lang.IllegalArgumentException: resource com/google/api/tools/framework/aspects/authentication/snippet\auth_document.snip not found.

Note the backslash at the end. This is due to the use of File.separator in SnippetSet.resourceInputSupplier.

This should just use a / as that's always the separator in resource names - while a backslash may work while loading straight from the file system, it won't work when the resource is loaded from a jar file.

There are lots of uses of File.separator in this repo - we really need to check each and every one of them; if any of them are for resources rather than actual files, that's an issue.

Inconsistent use of File.pathSeparator

Location assumes a :-separated name:

* The name of the input. This is a ':'-separated list of file names,

SnippetParser.Input creates a name based on File.pathSeparator:

return parent.getPath() + File.pathSeparator + name;

The combination of these prevents the parser from working properly on Windows.

Enable master branch protection

      This repository does not seem to have master branch
      protection enabled, at least in the way I'm expecting.
      I was hoping for:

      - master branch protection
      - requiring at least one code reviewer
      - requiring at least two status checks
      - enforcing rules for admins

      Please turn it on!

C# docs give warnings

Now we're building XML docs from our APIs, we're getting a lot of warnings from the generated client code about a few things:

Ambiguous references

...where they refer to method groups with multiple overloads.

I don't believe there's any way of referring to a method group as such in C# XML docs at the moment. Options:

  • Pick one overload to use
  • Remove the <see> tag entirely.

No docs for FooClientImpl, the FooClientImpl constructor, the GrpcClient property, or the Response generated GetEnumerator methods

Just a matter of adding a canned bit of XML.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.