Giter Club home page Giter Club logo

python-grpc's Introduction

OpenTracing support for gRPC in Python

The grpcio-opentracing package makes it easy to add OpenTracing support to gRPC-based systems in Python.

Installation

pip install grpcio-opentracing

Getting started

See the below code for basic usage or examples/trivial for a complete example.

Client-side usage example

from grpc_opentracing import open_tracing_client_interceptor
from grpc_opentracing.grpcext import intercept_channel

tracer = # some OpenTracing Tracer instance
interceptor = open_tracing_client_interceptor(tracer)
channel = # the grpc.Channel you created to invoke RPCs
channel = intercept_channel(channel, interceptor)

# All future RPC activity involving `channel` will be automatically traced.

Server-side usage example

from grpc_opentracing import open_tracing_server_interceptor
from grpc_opentracing.grpcext import intercept_server

tracer = # some OpenTracing Tracer instance
interceptor = open_tracing_server_interceptor(tracer)
server = # the grpc.Server you created to receive RPCs
server = intercept_server(server, interceptor)

# All future RPC activity involving `server` will be automatically traced.

Integrating with other spans.

grpcio-opentracing provides features that let you connect its span with other tracing spans. On the client-side, you can write a class that derives from ActiveSpanSource and provide it when creating the interceptor.

class CustomActiveSpanSource(ActiveSpanSource):
  @classmethod
  def get_active_span(self):
    # your custom method of getting the active span
tracer = # some OpenTracing Tracer instance
interceptor = open_tracing_client_interceptor(
                  tracer,
                  active_span_source=CustomActiveSpanSource)
...

On the server-side, the context argument passed into your service methods packages the gRPC span created on the server-side.

class CustomRpcService(...):
  ...
  def Method1(self, request, context):
    span = context.get_active_span()
    ...

See examples/integration for a complete example.

python-grpc's People

Contributors

carlosalberto avatar crazed avatar cshowe avatar guilhermef avatar h3l avatar raybotha avatar rnburn avatar vpquant avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

python-grpc's Issues

No good way to get ServicerContext code/details from interceptor when replaced with _OpenTracingServicerContext

Hi, we've been using grpcext.intercept_server() UnaryServerInterceptor and inspecting the server's return code like this:

def intercept_unary(self, request, servicer_context, server_info, handler):
    try:
      handler(request, servicer_context)
    except:
      code = servicer_context._state.code
      ...

We know this is not great: technically _state is private, but the ServicerContext offers no other public API for getting the code out of it.

The problem comes when we ALSO try to use open_tracing_server_interceptor(). When we do that, the servicer_context is replaced with an _OpenTracingServicerContext. And _OpenTracingServicerContext does not expose _state, so the above code breaks.

Yes, we know it's our fault for relying on the private _state field. But would it be possible to request a way to read the code/details from within a UnaryServerInterceptor, that would work whether or not we're using open_tracing_server_interceptor()?

grpcext as a separate package?

We do not use opentracing yet, but we do use the wrapper for unary and streaming server interceptors.

The new grpcio lib has support for server interceptor, but we could not get it to do things with the response.

We ended up using grpcext from this lib. Is it possible to make it into a separate library?

Add Travis

Having CI going would be good here. I'll look into it.

Test collection hanging on openSUSE Tumbleweed x86_64

My builds halt at

[   30s] + /usr/bin/python3 setup.py test
[   31s] running pytest
[   31s] running egg_info
[   31s] writing grpcio_opentracing.egg-info/PKG-INFO
[   31s] writing dependency_links to grpcio_opentracing.egg-info/dependency_links.txt
[   31s] writing requirements to grpcio_opentracing.egg-info/requires.txt
[   31s] writing top-level names to grpcio_opentracing.egg-info/top_level.txt
[   31s] reading manifest file 'grpcio_opentracing.egg-info/SOURCES.txt'
[   31s] writing manifest file 'grpcio_opentracing.egg-info/SOURCES.txt'
[   31s] running build_ext
[   31s] ============================= test session starts ==============================
[   31s] platform linux -- Python 3.8.6, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
[   31s] rootdir: /home/abuild/rpmbuild/BUILD/python-grpc-1.1.4
[   31s] collected 43 items

Build project at https://build.opensuse.org/package/show/home:jayvdb:opentracing/python-grpcio-opentracing , where full logs can be obtained for the builds.

The only obvious difference is Python 3.8 instead of the 3.6 used locally here in CI.

cannot use intercept_channel with `with` statement

grpc channel is usually used with with statement like here -> https://github.com/grpc/grpc/blob/master/examples/python/helloworld/greeter_client_with_options.py#L31

but...

import opentracing
from grpc_opentracing import open_tracing_client_interceptor
from grpc_opentracing.grpcext import intercept_channel
from grpc import insecure_channel

ic = open_tracing_client_interceptor(opentracing.global_tracer())
with intercept_channel(insecure_channel("localhost:5000"), ic) as ch:
  pass  # blabla
Traceback (most recent call last):
  File "testwith.py", line 7, in <module>
    with intercept_channel(insecure_channel("localhost:5000"), ic) as ch:
AttributeError: __enter__

add wrapper for grpc.Server.wait_for_termination

Hi, grpc/grpc#19299 added a handy wait_for_termination method to the grpc.Server interface.
It would be cool to implement this in _InterceptorServer and pass through to the underlying server.
...
Though looking at it, I think #14 will resolve this since it removes the wrapped server entirely. Is there an ETA on that PR at all?

Adding protobuf as requirements

I have installed the requirements from examples/requirements.txt and when I am trying the trivial example, it's failing with the following error
Traceback (most recent call last): File "trivial_server.py", line 13, in <module> import command_line_pb2 File "/Users/mahendra/python-grpc/examples/trivial/command_line_pb2.py", line 6, in <module> from google.protobuf import descriptor as _descriptor ModuleNotFoundError: No module named 'google'

To resolve this error I had to install protobuf. It will be better if we can add it to our requirements file.
Thanks

TypeError instantiating client interceptor

My server-side tracing is working well, however testing a client-side interceptor fails immediately with this error:

Traceback (most recent call last):
  File "client.py", line 29, in <module>
    channel = intercept_channel(channel, interceptor)
  File "/home/ray/.virtualenvs/grpc-GRTZPqPV/lib/python3.6/site-packages/grpc_opentracing/grpcext/__init__.py", line 95, in intercept_channel
    return _interceptor.intercept_channel(channel, *interceptors)
  File "/home/ray/.virtualenvs/grpc-GRTZPqPV/lib/python3.6/site-packages/grpc_opentracing/grpcext/_interceptor.py", line 216, in intercept_channel
    result = _InterceptorChannel(result, interceptor)
TypeError: Can't instantiate abstract class _InterceptorChannel with abstract methods close

This is the code:

import grpc
import yaml
from jaeger_client import Config
from grpc_opentracing import open_tracing_client_interceptor
from grpc_opentracing.grpcext import intercept_channel
from calculator import CalculatorStub

SERVICE = "calculator-client"

with open("config.yaml") as file:
    config = yaml.load(file)


def init_tracer(service_name=SERVICE):
    tracer_config = Config(config=config["jaeger"],
                           service_name=service_name,
                           validate=True)
    tracer = tracer_config.initialize_tracer()
    return tracer


tracer = init_tracer()
interceptor = open_tracing_client_interceptor(tracer, log_payloads=True)
channel = grpc.insecure_channel("localhost:50051")
channel = intercept_channel(channel, interceptor)
client = CalculatorStub(channel)

This is my config:

jaeger:
  sampler:
    type: const
    param: 1
  local_agent:
    reporting_host: 127.0.0.1
    reporting_port: 5775
  logging: true
Python 3.6.5, opentracing==1.3.0, grpcio-opentracing==1.1.1

Thanks

Ongoing maintenance of this repo

My organization has been using a forked version of grpcio-opentracing to add tracing to one of our projects and it has been great! Thanks for sharing the code, @rnburn.

However, we'd like to propose some upstream changes (namely support for grpcio >1.80.0), but it looks like this repo isn't actively maintained. I just wanted to reach out to @rnburn and @carlosalberto before opening a PR to get a better idea of who has ownership over this project. If you need help with maintenance, I'd love to help keep things up to date!

cc @austinlparker @iredelmeier - Apologies for the random tag, but it looks like you might be admins of this org?

aligning opentracing.tracer with servicer_context.tracer

When I use the python-grpc module on a server, the only way I can access the active span in the code is using the RPC context (that includes the span). That's because the RPC context is the only place that keeps track of the scope manager and the current active span. However, if I'm not mistaken, it's a pretty common pattern that the general opentracing module would be aligned with the tracing activity that the framework-specific module is generating.

why?

  1. this would enable me to access the active span anywhere my the code by calling opentracing.tracer.active_span without being required to pass the context to every function I call from the RPC.
  2. that would enable me to generate more complex spans activity on top of the activity the python-grpc module generates. (for example, opening and closing spans for child functions of the RPC).
  3. that would enable third-party software products that uses or communicates with the opentracing data to be able to do so, by importing opentracing and using it anywhere it wants. (That's actually how I bumped into this issue, my company's agent is accessing the server's tracing data for advanced features)è

how?
#35

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.