instana / python-sensor Goto Github PK

View Code? Open in Web Editor NEW

64.0 39.0 28.0 1.78 MB

:snake: Python Distributed Tracing & Metrics Sensor for Instana

Home Page: https://www.instana.com/

License: MIT License

Python 99.73% Dockerfile 0.03% HTML 0.01% Shell 0.22%

instrumentation user-monitoring performance metrics metrics-gathering opentracing tracing apm distributed-tracing

python-sensor's Introduction

Instana

The instana Python package collects key metrics and distributed traces for Instana.

This package supports Python 3.7 or greater.

Any feedback is welcome. Happy Python visibility.

Installation

Instana remotely instruments your Python web servers automatically via Instana AutoTrace™️. To configure which Python processes this applies to, see the configuration page.

Manual Installation

If you wish to instrument your applications manually, you can install the package with the following into the virtualenv, pipenv, or container (hosted on PyPI):

pip install instana

or to alternatively update an existing installation:

pip install -U instana

Activating Without Code Changes

The Instana package can then be activated without any code changes required by setting the following environment variable for your Python application:

export AUTOWRAPT_BOOTSTRAP=instana

This will cause the Instana Python package to instrument your Python application automatically. Once it finds the Instana host agent, it will report Python metrics and distributed traces.

Activating via Import

Alternatively, if you prefer the manual method, import the instana package inside of your Python application:

import instana

See also our detailed installation document for additional information covering Django, Flask, End-user Monitoring (EUM), and more.

Documentation

You can find more documentation covering supported components and minimum versions in the Instana documentation portal.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/instana/python-sensor.

Want to instrument other languages? See our Node.js, Go, Ruby instrumentation or many other supported technologies.

python-sensor's People

Contributors

Stargazers

Watchers

python-sensor's Issues

Need a way to access active span of automatic instrumentation

If a user wants to sprinkle in OpenTracing among existing automatic instrumentation, they need a way to access the currently active span to inherit context from.

Infinite Recursion when making request

I am seeing an issue on 1.22.1. When making a request via a celery worker (which uses gevent) I get an infinite recursion, here is the callstack:

RecursionError: maximum recursion depth exceeded while calling a Python object
[Previous line repeated 457 more times]
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/local/lib/python3.7/ssl.py", line 518, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/local/lib/python3.7/ssl.py", line 518, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/local/lib/python3.7/ssl.py", line 518, in options
context.options |= options
File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 276, in create_urllib3_context
cert_reqs=resolve_cert_reqs(self.cert_reqs),
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 344, in connect
conn.connect()
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
self._validate_conn(conn)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
chunked=chunked,
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
return wrapped(*args, **kwargs)
**File "/usr/local/lib/python3.7/site-packages/instana/instrumentation/urllib3.py", line 67, in urlopen_with_instana**
timeout=timeout
File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send

I'm wondering if this has to do with recent changes to add gevent support, I found a similar issue resolved here:

gevent/gevent#1016

I will try to step back to an older version and see if I can find where it was introduced.

Pub/Sub emulator on v.2.3.0 has issues with SSL handshake

Issue here: googleapis/python-pubsub#290
Will be fixed soon.

In the meantime, I'll use v.2.2.0 or lower for the current tracing implementation.

Will affect: #297

pip & Python 2 support

Python 2 Support for pip

pip 20.3 was the last version of pip that supported Python 2. Bugs reported with pip which only occur on Python 2.7 will likely be closed as “won’t fix” issues by pip’s maintainers.

AttributeError when extracting tags from pika BlockingChannel

Typo in https://github.com/instana/python-sensor/blob/master/instana/instrumentation/pika.py#L121 causes AttributeError when setting span tags.

2022-03-10 15:47:48,593: 1 DEBUG instana: basic_consume_with_instana:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/instana/instrumentation/pika.py", line 121, in _cb_wrapper
    _extract_consumer_tags(scope.span,
  File "/usr/local/lib/python3.8/site-packages/instana/instrumentation/pika.py", line 33, in _extract_consumer_tags
    _extract_broker_tags(span, conn)
  File "/usr/local/lib/python3.8/site-packages/instana/instrumentation/pika.py", line 21, in _extract_broker_tags
    span.set_tag("address", "%s:%d" % (conn.params.host, conn.params.port))
AttributeError: 'BlockingConnection' object has no attribute 'params'

Pika's BlockingConnection uses composition instead of inheritance. Therefore, the params field is not available in its parent but rather in its _impl attribute. This is done for consume_with_instana wrapper but not for basic_consume_with_instana wrapper.

Is Python 3.10 supported?

Problem Description

The github documentation says "Python > 2.7 is supported", but https://www.ibm.com/docs/en/instana-observability/current?topic=package-python-supported-components-versions#python-runtimes limits this to
Python < 3.10.
Which one is correct? If Python 3.10 is not supported yet - do you have a timeline for that? Thanks!

Minimal, Complete, Verifiable, Example

No response

Python Version

3.10

Python Modules

na

Python Environment

na

Automatic memcached instrumentation

import opentracing
import os
import pylibmc
import schedule
import sys
import time
import wrapt

memcached_hostname = os.environ['MEMCACHED_HOSTNAME']
memcached_port = os.environ['MEMCACHED_PORT']

mc = pylibmc.Client([memcached_hostname])


@wrapt.decorator
def instana_pylibmc(wrapped, instance, args, kwargs):
    with opentracing.tracer.start_active_span('memcache') as batch_scope:
        batch_scope.span.set_tag('span.kind', 'exit')
        batch_scope.span.set_tag('memcached.connection', '%s:%s' % (memcached_hostname, memcached_port))
        batch_scope.span.set_tag('memcached.operation', 'get')
        batch_scope.span.set_tag('memcached.key', args[0])
        try:
            rv = wrapped(*args, **kwargs)
            is_cache_hit = "false" if rv is None else "false"
            batch_scope.span.set_tag('memcached.cache_hit', is_cache_hit)
            return rv
        except Exception as e:
            batch_scope.span.log_exception(e)


@instana_pylibmc
def get_key(my_key):
    value = mc[my_key]


def job():
    with opentracing.tracer.start_active_span('Scheduled Job') as batch_scope:
        batch_scope.span.set_tag('span.kind', 'entry')
        get_key('my_key')


def main():
    schedule.every(1).seconds.do(job)

    while True:
        schedule.run_pending()
        time.sleep(1)


if __name__ == "__main__":
    main()

Update and version limit wrapt for latest fixes

https://twitter.com/GrahamDumpleton/status/1085499228212719616

Django: Instrumentation can get activated even when django not in use

Just having the Django package in sys.modules (but without a running Django app) causes the following cosmetic error:

2019-10-17 12:24:28,705: 1 DEBUG instana: Instrumenting django
2019-10-17 12:24:28,944: 1 DEBUG instana: django.middleware:
Traceback (most recent call last):
  File "/tmp/instana/python/instana/instrumentation/django/middleware.py", line 133, in <module>
    wsgiapp = get_internal_wsgi_application()
  File "/usr/local/lib/python2.7/site-packages/django/core/servers/basehttp.py", line 42, in get_internal_wsgi_application
    app_path = getattr(settings, 'WSGI_APPLICATION')
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 39, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
ImproperlyConfigured: Requested setting WSGI_APPLICATION, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Add documentation for the end to end tracing capabilities

We are able to correlate end user traces to python backend calls. We should add this to the documentation

Doubt in instana.instrumentation.django.middleware module

It seems to use the below command to reload application.

from django.core.servers.basehttp import get_internal_wsgi_application
            wsgiapp = get_internal_wsgi_application()
            wsgiapp.load_middleware()

Below is from the get_internal_wsgi_application function definition from github.

Load and return the WSGI application as configured by the user in
``settings.WSGI_APPLICATION``. With the default ``startproject`` layout,
this will be the ``application`` object in ``projectname/wsgi.py``.
This function, and the ``WSGI_APPLICATION`` setting itself, are only useful
for Django's internal server (runserver); external WSGI servers should just
be configured to point to the correct application object directly.
If settings.WSGI_APPLICATION is not set (is ``None``), return
whatever ``django.core.wsgi.get_wsgi_application`` returns.

Does this sensor work only for runserver in django applications?
Also, How does this sensor reload running gunicorn processes? I could not find any steps to gracefully shut down gunicorn processes as recommended in this gunicorn documentation

Sensor breaks with pyasn1 package

Basically, you can find the suspect here:

meter.py

...
    def collect_modules(self):
        try:
            m = sys.modules
            r = {}
            for k in m:
                if m[k]:
                    d = m[k].__dict__
                    if "version" in d and d["version"]:
                        r[k] = self.jsonable(d["version"])
                    elif "__version__" in d and d["__version__"]:
                        r[k] = self.jsonable(d["__version__"])
                    else:
                        r[k] = "builtin"

            return r
        except Exception as e:
            l.error("collect_modules: ", str(e))

            return None
...

Following line of code fails if module pyasn1 is being processed:

if "version" in d and d["version"]

exception thrown is pyasn1.error.PyAsn1Error: Uninitialized ASN.1 value ("__nonzero__" attribute looked up)

This problem came from package pysnmp which imports pyasn1

Snippet to reproduce

You can reproduce this problem with a small snippet

import sys
from pysnmp.entity.rfc3413.oneliner import cmdgen

m = sys.modules
r = {}
for k in m:
    if m[k]:
        d = m[k].__dict__
        if "version" in d and d["version"]:
            r[k] = d["version"]
        elif "__version__" in d and d["__version__"]:
            r[k] = d["__version__"]
        else:
            r[k] = "builtin"
print(r)

use of requests package

Hi there as per the documentation here https://docs.instana.io/quick_start/api/#rest-api

from instana.api import APIClient
c = APIClient(base_url="https://<dashboard-url>", api_token='MY_API_TOKEN')

# Or, alternatively if you have the environment variables set
# c = APIClient()

# Retrieve the current application view
av = c.application_view()
av.json()

which currently breaks due to requests not being present in the calls, which make use of urllib3 and NOT the requests package for some reason?

The API currently uses the requests package to make the REST calls to the API.
As such, requests response objects are returned from API calls.

~~is this intended as such? or is requests simply a dependency that needs to be installed?~~ requests seems to be prepackaged but not implemented it seems that ob._body is necessary to be called..

or do you plan on using raw urllib3? in which case the docs need to be updated?

Need a way to set app wide service name

The sensor currently lacks a way for users to set a global service name for their application.

If the user instantiates a tracer themselves, they can then set the service name via Options for all traces generated with that tracer but there is no method for default sensor instantiation.

Current idea is to add two separate methods. One via environment variable and the second via code.

e.g. export INSTANA_SERVICE_NAME=myservice or via code instana.service_name = "myservice"

asyncio (libs) support

Hello,

I wanted to ask if you guys have planned to support any asyncio libraries out of the box? e.g., aiohttp (client and server) or sanic.

Flask: -m flask run breaks with sensor; `flask run` works

(DemoFlask) Hughs-MacBook-Pro:DemoFlask hughbrien$ python -m flask run
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 153, in _get_module_details
    code = loader.get_code(mod_name)
AttributeError: '_ImportHookChainedLoader' object has no attribute 'get_code'

[Bug]: Instana AutoTrace causes patroni process to hang

Problem Description

A database cluster built with Zalando Spilo (https://github.com/zalando/spilo) and postgres-operator (https://github.com/zalando/postgres-operator) uses patroni (https://github.com/zalando/patroni) for cluster state management . Running on Kubernetes.

When Instana python instrumentation is enabled, the patroni python process inside each 'postgresql' pod hangs a few seconds after startup.

The command line used to execute patroni via Spilo is available at https://github.com/zalando/spilo/blob/master/postgres-appliance/runit/patroni/run#L29 (used via runsv by spilo) - when the last exec in this changed to include INSTANA_DISABLE_AUTO_INSTR=true after env , patroni starts and runs normally.

The above was figured out after spending some time trying to figure out what is wrong. This traceback (which fortunately was printed due to a mistake in manually trying to get print a traceback from /usr/local/lib/python3.6/dist-packages/google/cloud/storage/batch.py) provided the hint of what is causing patroni to hang:

  File "<string>", line 1, in <module>
  File "/tmp/.instana/python/instana/__init__.py", line 206, in <module>
    boot_agent()
  File "/tmp/.instana/python/instana/__init__.py", line 155, in boot_agent
    from .instrumentation.google.cloud import storage
  File "/tmp/.instana/python/instana/instrumentation/google/cloud/storage.py", line 14, in <module>
    from google.cloud import storage
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/storage/__init__.py", line 35, in <module>
    from google.cloud.storage.batch import Batch
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/storage/batch.py", line 30, in <module>
    traceback.print_stack(file=sys.stdout)
NameError: name 'sys' is not defined

What pointed towards 'batch.py' was this traceback from Patroni when it got stuck (acquired via kill -SIGABRT when the process was started with PYTHONFAULTHANDLER=true)

Current thread 0x00007fbe8a4f0740 (most recent call first):
  File "/usr/lib/python3.6/re.py", line 182 in search
  File "/usr/lib/python3.6/ctypes/util.py", line 283 in _findSoname_ldconfig
  File "/usr/lib/python3.6/ctypes/util.py", line 313 in find_library
  File "/usr/lib/python3/dist-packages/asn1crypto/_perf/_big_num_ctypes.py", line 35 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/lib/python3/dist-packages/asn1crypto/_int.py", line 56 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/lib/python3/dist-packages/asn1crypto/_elliptic_curve.py", line 51 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/lib/python3/dist-packages/asn1crypto/keys.py", line 22 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/lib/python3/dist-packages/cryptography/x509/extensions.py", line 13 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/lib/python3/dist-packages/cryptography/x509/base.py", line 16 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/lib/python3/dist-packages/cryptography/x509/__init__.py", line 8 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/local/lib/python3.6/dist-packages/google/auth/crypt/_cryptography_rsa.py", line 27 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1023 in _handle_fromlist
  File "/usr/local/lib/python3.6/dist-packages/google/auth/crypt/rsa.py", line 20 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1023 in _handle_fromlist
  File "/usr/local/lib/python3.6/dist-packages/google/auth/crypt/__init__.py", line 43 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1023 in _handle_fromlist
  File "/usr/local/lib/python3.6/dist-packages/google/auth/_service_account_info.py", line 22 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1023 in _handle_fromlist
  File "/usr/local/lib/python3.6/dist-packages/google/oauth2/service_account.py", line 77 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1023 in _handle_fromlist
  File "/usr/local/lib/python3.6/dist-packages/google/auth/transport/requests.py", line 48 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/_helpers/__init__.py", line 31 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 678 in exec_module
  File "<frozen importlib._bootstrap>", line 665 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 955 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 971 in _find_and_load
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1023 in _handle_fromlist
  File "/usr/local/lib/python3.6/dist-packages/google/cloud/storage/batch.py", line 28 in <module>
  ...

While attempting to figure out what is wrong, based on the above stack trace, noticed that uninstalling 'google-cloud-storage' fixed the problem. The curious part was that I could not find anything in patroni that would end up calling 'google-cloud-storage. While patroni supports WAL backups to google via WAL-E, and hence installs the google-cloud-storage python module, our configuration did not use it. Turned out the problem was not that patroni was importing it, but instana was, at https://github.com/instana/python-sensor/blob/master/instana/instrumentation/google/cloud/storage.py#L14 and as a consequence the whole patroni process hangs.

As said, if running strace on the patroni pid (strace -f -v -s 128 -p <pid>) this does not occur and patroni runs fine. If there are other ways I could provide further debugging details, I will be glad to assist.

root@host# uname -a
Linux <redacted> 5.4.190-107.353.amzn2.x86_64 #1 SMP Wed Apr 27 21:16:35 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

root@host# cat /etc/issue
Ubuntu 18.04.6 LTS \n \l

Minimal, Complete, Verifiable, Example

setup kubernetes, install zalando-spilo using postgres-operator, enable instana instrumentation and attempt to launch a postgres cluster. After a few seconds (< 30), the patroni API at localhost:8008 (inside any of the postgresql cluster containers) stops answering requests.

(sorry, was not able to replicate this in any minimal nor complete way..)

Python Version

Python 3.6

Python Modules

`apt list '*python*' --installed `
libpython3-stdlib/now 3.6.7-1~18.04 amd64 [installed,local]
libpython3.6/now 3.6.9-1~18.04ubuntu1.7 amd64 [installed,local]
libpython3.6-minimal/now 3.6.9-1~18.04ubuntu1.7 amd64 [installed,local]
libpython3.6-stdlib/now 3.6.9-1~18.04ubuntu1.7 amd64 [installed,local]
postgresql-plpython3-10/now 10.21-1.pgdg18.04+1 amd64 [installed,local]
postgresql-plpython3-11/now 11.16-1.pgdg18.04+1 amd64 [installed,local]
postgresql-plpython3-12/now 12.11-1.pgdg18.04+1 amd64 [installed,local]
postgresql-plpython3-13/now 13.7-1.pgdg18.04+1 amd64 [installed,local]
postgresql-plpython3-14/now 14.4-1.pgdg18.04+1 amd64 [installed,local]
postgresql-plpython3-9.6/now 9.6.24-1.pgdg18.04+1 amd64 [installed,local]
python-babel-localedata/now 2.4.0+dfsg.1-2ubuntu1.1 all [installed,local]
python3/now 3.6.7-1~18.04 amd64 [installed,local]
python3-asn1crypto/now 0.24.0-1 all [installed,local]
python3-babel/now 2.4.0+dfsg.1-2ubuntu1.1 all [installed,local]
python3-blinker/now 1.4+dfsg1-0.1 all [installed,local]
python3-boto/now 2.44.0-1ubuntu2.18.04.1 all [installed,local]
python3-cachetools/now 2.0.0-2 all [installed,local]
python3-cdiff/now 1.0-1 all [installed,local]
python3-certifi/now 2018.1.18-2 all [installed,local]
python3-cffi/now 1.11.5-1 all [installed,local]
python3-cffi-backend/now 1.11.5-1 amd64 [installed,local]
python3-chardet/now 3.0.4-1 all [installed,local]
python3-click/now 6.7-3 all [installed,local]
python3-colorama/now 0.3.7-1 all [installed,local]
python3-consul/now 0.7.1-1 all [installed,local]
python3-cryptography/now 2.1.4-1ubuntu1.4 amd64 [installed,local]
python3-dateutil/now 2.6.1-1 all [installed,local]
python3-debtcollector/now 1.13.0-0ubuntu1 all [installed,local]
python3-dnspython/now 1.15.0-1 all [installed,local]
python3-docutils/now 0.14+dfsg-3 all [installed,local]
python3-etcd/now 0.4.3-2 all [installed,local]
python3-funcsigs/now 1.0.2-4 all [installed,local]
python3-gevent/now 1.2.2-2 amd64 [installed,local]
python3-greenlet/now 0.4.12-2 amd64 [installed,local]
python3-idna/now 2.6-1 all [installed,local]
python3-iso8601/now 0.1.11-1 all [installed,local]
python3-jwt/now 1.5.3+ds1-1 all [installed,local]
python3-kazoo/now 2.2.1-1ubuntu1 all [installed,local]
python3-keyring/now 10.6.0-1 all [installed,local]
python3-keystoneauth1/now 3.4.0-0ubuntu1 all [installed,local]
python3-keystoneclient/now 1:3.15.0-0ubuntu1 all [installed,local]
python3-lxml/now 4.2.1-1ubuntu0.6 amd64 [installed,local]
python3-meld3/now 1.0.2-2 amd64 [installed,local]
python3-minimal/now 3.6.7-1~18.04 amd64 [installed,local]
python3-monotonic/now 1.1-2 all [installed,local]
python3-msgpack/now 0.5.6-1 amd64 [installed,local]
python3-netaddr/now 0.7.19-1 all [installed,local]
python3-netifaces/now 0.10.4-0.1build4 amd64 [installed,local]
python3-oauthlib/now 2.0.6-1 all [installed,local]
python3-oslo.config/now 1:5.2.0-0ubuntu1 all [installed,local]
python3-oslo.i18n/now 3.19.0-0ubuntu1 all [installed,local]
python3-oslo.serialization/now 2.24.0-0ubuntu2 all [installed,local]
python3-oslo.utils/now 3.35.0-0ubuntu1.1 all [installed,local]
python3-pbr/now 3.1.1-3ubuntu3 all [installed,local]
python3-pkg-resources/now 39.0.1-2 all [installed,local]
python3-ply/now 3.11-1 all [installed,local]
python3-positional/now 1.1.1-3 all [installed,local]
python3-prettytable/now 0.7.2-3 all [installed,local]
python3-psutil/now 5.4.2-1ubuntu0.1 amd64 [installed,local]
python3-psycopg2/now 2.8.6-2~pgdg18.04+1 amd64 [installed,local]
python3-pyasn1/now 0.4.2-3 all [installed,local]
python3-pyasn1-modules/now 0.2.1-0.2 all [installed,local]
python3-pycparser/now 2.18-2 all [installed,local]
python3-pyparsing/now 2.2.0+dfsg1-2 all [installed,local]
python3-pystache/now 0.5.4-6 all [installed,local]
python3-requests/now 2.18.4-2ubuntu0.1 all [installed,local]
python3-rfc3986/now 0.3.1-2 all [installed,local]
python3-rsa/now 3.4.2-1 all [installed,local]
python3-six/now 1.11.0-2 all [installed,local]
python3-stevedore/now 1:1.28.0-0ubuntu1 all [installed,local]
python3-swiftclient/now 1:3.5.0-0ubuntu1 all [installed,local]
python3-tz/now 2018.3-2 all [installed,local]
python3-urllib3/now 1.22-1ubuntu0.18.04.2 all [installed,local]
python3-wrapt/now 1.9.0-3 amd64 [installed,local]
python3-yaml/now 3.12-1build2 amd64 [installed,local]
python3.6/now 3.6.9-1~18.04ubuntu1.7 amd64 [installed,local]
python3.6-minimal/now 3.6.9-1~18.04ubuntu1.7 amd64 [installed,local]

`pip3 list`
asn1crypto (0.24.0)
Babel (2.4.0)
blinker (1.4)
boto (2.44.0)
boto3 (1.23.10)
botocore (1.26.10)
cachetools (2.0.0)
cdiff (1.0)
certifi (2018.1.18)
cffi (1.11.5)
chardet (3.0.4)
click (6.7)
colorama (0.3.7)
cryptography (2.1.4)
debtcollector (1.13.0)
dnspython (1.15.0)
filechunkio (1.8)
funcsigs (1.0.2)
gevent (1.2.2)
google-api-core (2.8.2)
google-auth (2.8.0)
google-cloud-core (2.3.1)
google-cloud-storage (2.0.0)
google-crc32c (1.1.2)
google-resumable-media (2.3.3)
googleapis-common-protos (1.56.2)
greenlet (0.4.12)
idna (2.6)
iso8601 (0.1.11)
jmespath (0.10.0)
kazoo (2.2.1.dev0)
keystoneauth1 (3.4.0)
lxml (4.2.1)
meld3 (1.0.2)
monotonic (1.0)
msgpack (0.5.6)
netaddr (0.7.19)
netifaces (0.10.4)
oauthlib (2.0.6)
oslo.config (5.2.0)
oslo.i18n (3.19.0)
oslo.serialization (2.24.0)
oslo.utils (3.35.0)
patroni (2.1.4)
pbr (3.1.1)
pg-view (1.3.1)
pip (9.0.1)
ply (3.11)
positional (1.1.1)
prettytable (0.7.2)
protobuf (3.19.4)
psutil (5.4.2)
psycopg2 (2.8.6)
pyasn1 (0.4.2)
pyasn1-modules (0.2.1)
pycparser (2.18)
PyJWT (1.5.3)
pyparsing (2.2.0)
pystache (0.5.4)
python-consul (0.7.1)
python-dateutil (2.6.1)
python-etcd (0.4.3)
python-keystoneclient (3.15.0)
python-swiftclient (3.5.0)
pytz (2018.3)
PyYAML (3.12)
requests (2.18.4)
rfc3986 (0.3.1)
rsa (3.4.2)
s3transfer (0.5.2)
setuptools (59.6.0)
six (1.11.0)
stevedore (1.28.0)
urllib3 (1.22)
wal-e (1.1.1)
wrapt (1.9.0)
ydiff (1.2)

Python Environment

KUBERNETES_ROLE_LABEL=spilo-role
POD_IP=<redacted>
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://172.20.0.1:443
HOSTNAME=<redacted>
PGHOME=/home/postgres
KUBERNETES_PORT_443_TCP_ADDR=172.20.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/postgresql/14/bin
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP=tcp://172.20.0.1:443
KUBERNETES_SERVICE_PORT_HTTPS=443
POD_NAMESPACE=platform
KUBERNETES_SERVICE_HOST=172.20.0.1
LC_ALL=en_US.utf-8
HOME=/home/postgres

ModuleNotFoundError instrumentation.google on v1.26.0

Recently updated to 1.26.0 and started getting this error. Downgraded back to 1.25 and it's working again.

app.py:3: in <module>
    import instana  # noqa: F401
/home/nix/.local/share/virtualenvs/engine-ehS8vc03/lib/python3.7/site-packages/instana/__init__.py:181: in <module>
    boot_agent()
/home/nix/.local/share/virtualenvs/engine-ehS8vc03/lib/python3.7/site-packages/instana/__init__.py:141: in boot_agent
    from .instrumentation.google.cloud import storage
E   ModuleNotFoundError: No module named 'instana.instrumentation.google'

Agent host & port should be configurable via environment variables

Like our other sensors, the Python sensor should be configurable via environment variables.

INSTANA_AGENT_IP
INSTANA_AGENT_PORT

https://github.com/instana/python-sensor/blob/master/instana/agent.py#L35-L36

Instrumented uwsgi on OSX Mojave can crash post fork

OSX Mojave added some threading security features that broke a large number of various projects. This is likely not a direct Instana issue but we should investigate to know how to remedy.

https://bugs.python.org/issue33725

Syntax warning and deprecation warning in Python 3.8

Syntax warning due to comparison of literals using is where == or != should be used. Deprecation warnings are raised due to invalid escape sequences in Python 3.8 . Below is a log of the warnings raised during compiling all the python files. Using raw strings or escaping them will fix this issue.

 find . -iname '*.py'  | xargs -P 4 -I{} python -Wall -m py_compile {} 

./instana/util.py:230: DeprecationWarning: invalid escape sequence \s
  regexp_sql_values = re.compile('(\'[\s\S][^\']*\'|\d*\.\d+|\d+|NULL)')
./instana/util.py:283: DeprecationWarning: invalid escape sequence \.
  regexp_py = re.compile('\.py$')
./instana/util.py:19: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if sys.version_info.major is 2:
./instana/util.py:250: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if hip is not None and len(hip) is 8:
./instana/instrumentation/urllib3.py:21: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if args is not None and len(args) is 2:
./instana/instrumentation/urllib3.py:34: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if len(parts) is 2:
./instana/instrumentation/aiohttp/server.py:47: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if ec is 0:
./instana/instrumentation/django/middleware.py:58: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if ec is 0:
./instana/instrumentation/flask/vanilla.py:69: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if ec is 0:
./instana/instrumentation/flask/with_blinker.py:67: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if ec is 0:
./instana/instrumentation/sqlalchemy.py:13: DeprecationWarning: invalid escape sequence \/
  url_regexp = re.compile('\/\/(\S+@)')
./instana/instrumentation/tornado/server.py:82: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if ec is 0:
./instana/meter.py:191: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if self.agent.machine.fsm.current is "wait4init":
./instana/meter.py:194: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if self.agent.machine.fsm.current is not "good2go":
./instana/meter.py:219: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response.status_code is 200 and len(response.content) > 2:
./instana/agent.py:164: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response.status_code is 200:
./instana/agent.py:178: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response.status_code is 200:
./instana/agent.py:197: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response.status_code is 200:
./instana/agent.py:222: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response.status_code is 200:
./instana/fsm.py:159: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if response and (response.status_code is 200) and (len(response.content) > 2):
./instana/tracer.py:157: DeprecationWarning: invalid escape sequence \.
  re_tracer_frame = re.compile('/instana/.*\.py$')
./tests/test_id_management.py:8: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if sys.version_info.major is 2:
./tests/test_secrets.py:87: DeprecationWarning: invalid escape sequence \d
  kwlist = ['\d']
./tests/test_secrets.py:97: DeprecationWarning: invalid escape sequence \d
  kwlist = ['\d\d\d']

Check implication of init for rabbitmq span type

Specifically this piece of code:

python-sensor/instana/span.py

Lines 257 to 258 in f800501

 if "rabbitmq" in self.data and self.data["rabbitmq"]["sort"] == "consume": 

 self.k = 1 # entry

    def __init__(self, span, source, service_name, **kwargs):
        # pylint: disable=invalid-name
        super(RegisteredSpan, self).__init__(span, source, service_name, **kwargs)
        self.n = span.operation_name
        self.k = 1
        if span.operation_name in self.ENTRY_SPANS:
            # entry
            self._populate_entry_span_data(span)
            self.data["service"] = service_name
        elif span.operation_name in self.EXIT_SPANS:
            self.k = 2 # exit
            self._populate_exit_span_data(span)
        elif span.operation_name in self.LOCAL_SPANS:
            self.k = 3 # intermediate span
            self._populate_local_span_data(span)

        if "rabbitmq" in self.data and self.data["rabbitmq"]["sort"] == "consume":
            self.k = 1  # entry

Looks like, it has no effect.

Python 2.6.6: pkg_resources

We support Python 2.7 and greater but logging this Python 2.6.6 issue for posterity.

When instrumenting an older Python 2.6.6 application, the following error is hit:

Python 2.6.6 (r266:84292, Jun 20 2019, 14:14:55)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/instana/python/instana/__init__.py", line 25, in <module>
    import pkg_resources
  File "/tmp/instana/python/pkg_resources/__init__.py", line 959, in <module>
    class Environment:
  File "/tmp/instana/python/pkg_resources/__init__.py", line 963, in Environment
    self, search_path=None, platform=get_supported_platform(),
  File "/tmp/instana/python/pkg_resources/__init__.py", line 190, in get_supported_platform
    plat = get_build_platform()
  File "/tmp/instana/python/pkg_resources/__init__.py", line 393, in get_build_platform
    from sysconfig import get_platform
ImportError: No module named sysconfig
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/instana/python/instana/__init__.py", line 25, in <module>
    import pkg_resources
  File "/tmp/instana/python/pkg_resources/__init__.py", line 79, in <module>
    from . import py31compat
ImportError: cannot import name py31compat
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/instana/python/instana/__init__.py", line 25, in <module>
    import pkg_resources
  File "/tmp/instana/python/pkg_resources/__init__.py", line 79, in <module>
    from . import py31compat

No Urllib3 instrumentation with requests package

There are certain use cases where urllib3 calls are not correctly instrumented when using the requests package.

Custom Tracing: Warn if unsupported values are used for SetTag

Our Custom Tracing Best Practices document outlines the formats for common tags but if a user still attempts to set a value of something unsupported, (e.g. a Python object of some sort) we should log a more informational warning.

File "/usr/local/lib/python2.7/site-packages/instana/agent.py", line 91, in extractor
   return {k.lower(): v for k, v in o.__dict__.items() if v is not None}
AttributeError: 'datetime.datetime' object has no attribute '__dict__'

Under Django, snapshot data can be missing (app name shown as Unknown)

Investigate what happened with snapshot data.

ImportError: No module named 'tornado'

We started getting this error in our CI today, which is configured to get the latest version of the instana package.

  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/gunicorn/app/base.py", line 223, in run
    super(Application, self).run()
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/gunicorn/arbiter.py", line 60, in __init__
    self.setup(app)
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/gunicorn/arbiter.py", line 95, in setup
    self.log = self.cfg.logger_class(app.cfg)
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/gunicorn/glogging.py", line 200, in __init__
    self.setup(cfg)
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/gunicorn/glogging.py", line 256, in setup
    disable_existing_loggers=False)
  File "/opt/miniconda/envs/.venv/lib/python3.5/logging/config.py", line 76, in fileConfig
    formatters = _create_formatters(cp)
  File "/opt/miniconda/envs/.venv/lib/python3.5/logging/config.py", line 123, in _create_formatters
    c = _resolve(class_name)
  File "/opt/miniconda/envs/.venv/lib/python3.5/logging/config.py", line 94, in _resolve
    found = __import__(used)
  File "/usr/local/pythagoras/pythagoras/__init__.py", line 1, in <module>
    from flask import Flask
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/wrapt/importer.py", line 158, in load_module
    notify_module_loaded(module)
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/wrapt/decorators.py", line 440, in _synchronized
    return wrapped(*args, **kwargs)
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/wrapt/importer.py", line 136, in notify_module_loaded
    hook(module)
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/wrapt/importer.py", line 105, in import_hook
    __import__(entrypoint.module_name)
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/instana/__init__.py", line 90, in <module>
    boot_agent()
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/instana/__init__.py", line 65, in boot_agent
    import instana.singletons
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/instana/singletons.py", line 28, in <module>
    from opentracing.scope_managers.tornado import TornadoScopeManager
  File "/opt/miniconda/envs/.venv/lib/python3.5/site-packages/opentracing/scope_managers/tornado.py", line 24, in <module>
    import tornado.stack_context
ImportError: No module named 'tornado'

our conda-requirements.yaml file:

name: .venv
channels:
  - defaults
  - conda-forge
dependencies:
  - asn1crypto=0.24.0=py35_3
  - ca-certificates=2018.11.29=ha4d7672_0
  - certifi=2018.8.24=py35_1001
  - cffi=1.11.5=py35h5e8e0c9_1
  - chardet=3.0.4=py35_3
  - click=7.0=py_0
  - cryptography=2.3.1=py35hdffb7b8_0
  - cryptography-vectors=2.3.1=py35_0
  - fastcache=1.0.2=py35h470a237_1
  - flask=1.0.2=py_2
  - gevent=1.3.6=py35h470a237_0
  - gmp=6.1.2=hf484d3e_1000
  - gmpy2=2.0.8=py35_0
  - gunicorn=19.9.0=py35_1000
  - idna=2.7=py35_2
  - itsdangerous=1.1.0=py_0
  - jinja2=2.10=py_1
  # - libcxx=7.0.0=h6bb024c_1002
  - libedit
  - libffi=3.2.1=hf484d3e_1005
  - libgcc-ng=7.3.0=hdf63c60_0
  - libstdcxx-ng=7.3.0=hdf63c60_0
  - llvm-meta=7.0.0=0
  - markupsafe=1.0=py35h470a237_1
  - mpc=1.1.0=hb20f59a_1006
  - mpfr=4.0.1=ha14ba45_1000
  - mpmath=1.1.0=py_0
  - ncurses=6.1=hf484d3e_1002
  - openssl=1.0.2p=h14c3975_1002
  - pkg-config=0.29.2=h1bed415_8
  - pip=18.0=py35_1001
  - pycparser=2.19=py_0
  - pyopenssl=18.0.0=py35_0
  - pysocks=1.6.8=py35_2
  - pyyaml=3.13=py35h470a237_1
  - readline=7.0=hf8c457e_1001
  - requests=2.19.1=py35_1
  - setuptools=40.4.3=py35_0
  - six=1.11.0=py35_1
  - sqlite=3.26.0=h67949de_1000
  - structlog=19.1.0=py_0
  - sympy=1.1.1=py35_0
  - tk=8.6.9=h84994c4_1000
  - urllib3=1.23=py35_1
  - werkzeug=0.14.1=py_0
  - wheel=0.32.0=py35_1000
  - xz=5.2.4=h14c3975_1001
  - yaml=0.1.7=h14c3975_1001
  - zlib=1.2.11=h14c3975_1004
  - greenlet=0.4.14=py35h14c3975_0
  - python=3.5.6=hc3d631a_0
  - pip:
    - autowrapt==1.0
    - basictracer==3.0.0
    - fysom==2.1.5
    - instana
    - json-logging-py==0.2
    - opentracing==2.0.0
    - protobuf==3.7.0rc2
    - wrapt==1.11.1
prefix: /opt/miniconda/envs/.venv

Remove /sbin/ip dependency

We should instead read /proc/net/route

Not passing command line arguments to the script

I am trying to setup instana according to: https://www.instana.com/docs/ecosystem/python/ where I:

pip install instana
export AUTOWRAPT_BOOTSTRAP=instana

Everything seems to be working except that I am no longer able to pass command line arguments to python scripts.

Given a script file test.py:

import sys
print(sys.argv)

when I run from the command line:

python test.py a b c

the output is empty list instead of passed arguments.

If I don't set: AUTOWRAPT_BOOTSTRAP=instana instana is not starting and everything is working fine.

python version is 3.8 run as a virtualenv

threading.Thread.setDaemon has been deprecated in favor of setting daemon attribute directly in Python 3.10

Ref : python/cpython#25174

rg -t py -w 'currentThread|notifyAll|activeCount|isDaemon|setDaemon'                              
instana/collector/base.py
102:            self.reporting_thread.setDaemon(True)

Sqlalchemy will drop support for Python 2 after v.1.4

https://docs.sqlalchemy.org/en/14/changelog/migration_14.html#python-3-6-is-the-minimum-python-3-version-python-2-7-still-supported

Report proper Django name in snapshot w/ Installed Apps

We should report the proper Django app name with the following:

import django.core.handlers.base as base
name = os.path.basename(base.settings.BASE_DIR)
string.capitalize(name)

which would reveal the package name that this app is built with.

Then also report the list of installed apps:

>>> base.settings.INSTALLED_APPS
['polls.apps.PollsConfig', 'django.contrib.admin', 'django.contrib.auth', 
'django.contrib.contenttypes', 'django.contrib.sessions', 
'django.contrib.messages', 'django.contrib.staticfiles']

I can not get the Python sensor script to work

There are several module imports that fail.

[Bug]: Instana interferes with exception raised when connecting to database

Problem Description

When connecting to a database server using sqlalchemy usually an OperationalError is raised. However, Instana somehow intercepts this and instead raises an AttributeError (see stacktrace below).

This is an issue, since python-testcontainers since version 3.5 checks for explicit Exceptions to decide whether to wait or not and AttributeError is not one of them: testcontainers/testcontainers-python#193

self = <instana.instrumentation.pep0249.ConnectionFactory object at 0x10a3e0f40>
args = ()
kwargs = {'database': 'test', 'host': 'localhost', 'password': 'test', 'port': 58725, ...}
connect_params = ((), {'database': 'test', 'host': 'localhost', 'password': 'test', 'port': 58725, ...})

    def __call__(self, *args, **kwargs):
        connect_params = (args, kwargs) if args or kwargs else None
    
        return self._wrapper_ctor(
>           connection=self._connect_func(*args, **kwargs),
            module_name=self._module_name,
            connect_params=connect_params)

../../.venv/lib/python3.9/site-packages/instana/instrumentation/pep0249.py:133: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

dsn = 'host=localhost user=test password=test port=58725 dbname=test'
connection_factory = None, cursor_factory = None
kwargs = {'database': 'test', 'host': 'localhost', 'password': 'test', 'port': 58725, ...}
kwasync = {}

    def connect(dsn=None, connection_factory=None, cursor_factory=None, **kwargs):
        """
        Create a new database connection.
    
        The connection parameters can be specified as a string:
    
            conn = psycopg2.connect("dbname=test user=postgres password=secret")
    
        or using a set of keyword arguments:
    
            conn = psycopg2.connect(database="test", user="postgres", password="secret")
    
        Or as a mix of both. The basic connection parameters are:
    
        - *dbname*: the database name
        - *database*: the database name (only as keyword argument)
        - *user*: user name used to authenticate
        - *password*: password used to authenticate
        - *host*: database host address (defaults to UNIX socket if not provided)
        - *port*: connection port number (defaults to 5432 if not provided)
    
        Using the *connection_factory* parameter a different class or connections
        factory can be specified. It should be a callable object taking a dsn
        argument.
    
        Using the *cursor_factory* parameter, a new default cursor factory will be
        used by cursor().
    
        Using *async*=True an asynchronous connection will be created. *async_* is
        a valid alias (for Python versions where ``async`` is a keyword).
    
        Any other keyword parameter will be passed to the underlying client
        library: the list of supported parameters depends on the library version.
    
        """
        kwasync = {}
        if 'async' in kwargs:
            kwasync['async'] = kwargs.pop('async')
        if 'async_' in kwargs:
            kwasync['async_'] = kwargs.pop('async_')
    
        dsn = _ext.make_dsn(dsn, **kwargs)
>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       psycopg2.OperationalError: connection to server at "localhost" (127.0.0.1), port 58725 failed: server closed the connection unexpectedly
E       	This probably means the server terminated abnormally
E       	before or while processing the request.

../../.venv/lib/python3.9/site-packages/psycopg2/__init__.py:122: OperationalError

The above exception was the direct cause of the following exception:

pytestconfig = <_pytest.config.Config object at 0x1075b5610>

    @pytest.fixture(scope="session")
    def _db_container(pytestconfig: Any) -> PostgresContainer:
>       with _create_container(
            "timescale/timescaledb:latest-pg11",
            PostgresContainer,
        ) as container:

../conftest.py:107: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../.venv/lib/python3.9/site-packages/testcontainers/core/container.py:69: in __enter__
    return self.start()
../../.venv/lib/python3.9/site-packages/testcontainers/core/generic.py:50: in start
    self._connect()
../../.venv/lib/python3.9/site-packages/testcontainers/core/waiting_utils.py:49: in wrapper
    return wrapped(*args, **kwargs)
../../.venv/lib/python3.9/site-packages/testcontainers/core/generic.py:28: in _connect
    engine.connect()
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py:2263: in connect
    return self._connection_cls(self, **kwargs)
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py:104: in __init__
    else engine.raw_connection()
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py:2369: in raw_connection
    return self._wrap_pool_connect(
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py:2339: in _wrap_pool_connect
    Connection._handle_dbapi_exception_noconnection(
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py:1581: in _handle_dbapi_exception_noconnection
    util.raise_(newraise, with_traceback=exc_info[2], from_=e)
../../.venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py:182: in raise_
    raise exception
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py:2336: in _wrap_pool_connect
    return fn()
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py:304: in unique_connection
    return _ConnectionFairy._checkout(self)
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py:778: in _checkout
    fairy = _ConnectionRecord.checkout(pool)
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py:495: in checkout
    rec = pool._do_get()
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/impl.py:140: in _do_get
    self._dec_overflow()
../../.venv/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
../../.venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py:182: in raise_
    raise exception
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/impl.py:137: in _do_get
    return self._create_connection()
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py:309: in _create_connection
    return _ConnectionRecord(self)
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py:440: in __init__
    self.__connect(first_connect_check=True)
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py:661: in __connect
    pool.logger.debug("Error on connect(): %s", e)
../../.venv/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py:68: in __exit__
    compat.raise_(
../../.venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py:182: in raise_
    raise exception
../../.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py:656: in __connect
    connection = pool._invoke_creator(self)
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/strategies.py:114: in connect
    return dialect.connect(*cargs, **cparams)
../../.venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py:508: in connect
    return self.dbapi.connect(*cargs, **cparams)
../../.venv/lib/python3.9/site-packages/instana/instrumentation/pep0249.py:133: in __call__
    connection=self._connect_func(*args, **kwargs),
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

dsn = 'host=localhost user=test password=test port=58725 dbname=test'
connection_factory = None, cursor_factory = None
kwargs = {'database': 'test', 'host': 'localhost', 'password': 'test', 'port': 58725, ...}
kwasync = {}

    def connect(dsn=None, connection_factory=None, cursor_factory=None, **kwargs):
        """
        Create a new database connection.
    
        The connection parameters can be specified as a string:
    
            conn = psycopg2.connect("dbname=test user=postgres password=secret")
    
        or using a set of keyword arguments:
    
            conn = psycopg2.connect(database="test", user="postgres", password="secret")
    
        Or as a mix of both. The basic connection parameters are:
    
        - *dbname*: the database name
        - *database*: the database name (only as keyword argument)
        - *user*: user name used to authenticate
        - *password*: password used to authenticate
        - *host*: database host address (defaults to UNIX socket if not provided)
        - *port*: connection port number (defaults to 5432 if not provided)
    
        Using the *connection_factory* parameter a different class or connections
        factory can be specified. It should be a callable object taking a dsn
        argument.
    
        Using the *cursor_factory* parameter, a new default cursor factory will be
        used by cursor().
    
        Using *async*=True an asynchronous connection will be created. *async_* is
        a valid alias (for Python versions where ``async`` is a keyword).
    
        Any other keyword parameter will be passed to the underlying client
        library: the list of supported parameters depends on the library version.
    
        """
        kwasync = {}
        if 'async' in kwargs:
            kwasync['async'] = kwargs.pop('async')
        if 'async_' in kwargs:
            kwasync['async_'] = kwargs.pop('async_')
    
        dsn = _ext.make_dsn(dsn, **kwargs)
>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       AttributeError: 'NoneType' object has no attribute '_stan_scope'

../../.venv/lib/python3.9/site-packages/psycopg2/__init__.py:122: AttributeError

Minimal, Complete, Verifiable, Example

import sqlalchemy
from testcontainers.mysql import PostgresContainer

with PostgresContainer("dockette/postgres:latest") as container:
    engine = sqlalchemy.create_engine(container.get_connection_url())
    version, = engine.execute("select version()").fetchone()
    print(version)

Python Version

Python 3.9

Python Modules

requirements.txt


alembic==1.7.7; python_version >= "3.6"
argcomplete==2.0.0; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
ariadne==0.14.1
atomicwrites==1.4.0; python_version >= "3.7" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.7" and python_full_version >= "3.4.0"
attrs==21.4.0; python_full_version >= "3.6.1" and python_version >= "3.7" and python_full_version < "4.0.0" and python_version < "4.0"
autowrapt==1.0
basictracer==3.2.0
black==22.3.0; python_full_version >= "3.6.2"
certifi==2021.10.8; python_version >= "3.7" and python_full_version < "3.0.0" and python_version < "4.0" or python_full_version >= "3.6.0" and python_version < "4" and python_version >= "3.7"
cffi==1.15.0; python_version >= "3.6"
chardet==4.0.0; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
charset-normalizer==2.0.7; python_full_version >= "3.5.0"
click==8.1.2; python_version >= "3.7" and python_full_version >= "3.6.2" and python_full_version < "4.0.0" and python_version < "4.0"
colorama==0.4.4; sys_platform == "win32" and python_version >= "3.7" and python_full_version >= "3.6.2" and platform_system == "Windows" and (python_version >= "3.7" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.7" and python_full_version >= "3.5.0")
coverage==6.3.2; python_version >= "3.7"
cryptography==3.4.8; python_version >= "3.6"
datamodel-code-generator==0.11.20; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
deprecation==2.1.0; python_version >= "3.5"
dnspython==2.2.1; python_version >= "3.7" and python_version < "4.0" and python_full_version >= "3.6.1"
docker==5.0.3; python_version >= "3.6"
elasticsearch-dsl==7.4.0; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.4.0")
elasticsearch==7.17.2; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.4.0" and python_version < "4")
email-validator==1.1.3; python_full_version >= "3.6.1" and python_version >= "3.7" and python_version < "4.0"
fastapi==0.68.2; python_full_version >= "3.6.1"
flake8==4.0.1; python_version >= "3.6"
fysom==2.1.6
genson==1.2.2; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
graphql-core==3.1.7; python_version >= "3.6" and python_version < "4"
idna==3.3; python_full_version >= "3.6.1" and python_version >= "3.7" and python_version < "4.0"
importlib-metadata==4.2.0; python_version == "3.7" and python_full_version >= "3.6.2" and python_full_version < "4.0.0"
importlib-resources==5.6.0; python_version < "3.9" and python_version >= "3.7"
inflect==5.5.2; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
iniconfig==1.1.1; python_version >= "3.7"
instana==1.37.1
isodate==0.6.1; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
isort==5.10.1; python_full_version >= "3.6.1" and python_version < "4.0" and python_full_version < "4.0.0" and python_version >= "3.7"
jinja2==3.1.1; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
jsonschema==3.2.0; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
mako==1.2.0; python_version >= "3.7"
markupsafe==2.1.1; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
mccabe==0.6.1; python_version >= "3.6"
mockito==1.3.0
mypy-extensions==0.4.3; python_full_version >= "3.6.2" and python_version >= "3.7" and python_full_version < "4.0.0" and python_version < "4.0"
mypy==0.942; python_version >= "3.6"
openapi-schema-validator==0.1.6; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
openapi-spec-validator==0.3.3; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
opentracing==2.4.0
packaging==21.3; python_version >= "3.7"
pathspec==0.9.0; python_full_version >= "3.6.2" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
platformdirs==2.5.1; python_version >= "3.7" and python_full_version >= "3.6.2" and python_full_version < "4.0.0" and python_version < "4.0"
pluggy==1.0.0; python_version >= "3.7"
prance==0.21.8.0; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
protobuf==4.0.0rc2
psycopg2-binary==2.9.3; python_version >= "3.6"
py==1.11.0; python_version >= "3.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.7"
pycodestyle==2.8.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6"
pycparser==2.21; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "3.6"
pydantic==1.9.0; python_full_version >= "3.6.1"
pyflakes==2.4.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "3.6"
pyjwt==1.7.1
pyparsing==3.0.7; python_version >= "3.7"
pyrsistent==0.16.1; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
pysnooper==1.1.1; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
pytest-asyncio==0.18.3; python_version >= "3.7"
pytest-cov==3.0.0; python_version >= "3.6"
pytest==7.1.1; python_version >= "3.7"
python-dateutil==2.8.2; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.4.0"
python-json-logger==2.0.2; python_version >= "3.5"
pywin32==227; sys_platform == "win32" and python_version >= "3.6"
pyyaml==6.0; python_version >= "3.6"
requests==2.27.1; python_version >= "3.7" and python_version < "4.0" and python_full_version >= "3.6.1" and python_full_version < "4.0.0"
ruamel.yaml.clib==0.2.6; platform_python_implementation == "CPython" and python_version < "3.11" and python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7"
ruamel.yaml==0.17.21; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
semver==2.13.0; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
six==1.16.0; python_version >= "3.7" and python_version < "4.0" and python_full_version >= "3.6.1" and python_full_version < "4.0.0"
slacker==0.14.0
sqlalchemy==1.3.24; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.4.0")
starlette-context==0.3.3; python_version >= "3.7"
starlette==0.14.2; python_version >= "3.6"
testcontainers==3.5.3; python_version >= "3.6"
testfixtures==6.18.5
toml==0.10.2; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and python_version >= "3.7" and python_version < "4.0"
tomli==2.0.1; python_version < "3.11" and python_full_version >= "3.6.2" and python_version >= "3.7" and python_full_version < "4.0.0"
typed-ast==1.5.2
typing-extensions==4.1.1; python_version >= "3.7" and python_version < "3.8" and python_full_version >= "3.6.2" and python_full_version < "4.0.0"
urllib3==1.26.9; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0" and python_version < "4")
websocket-client==1.3.2; python_version >= "3.7"
wiremock==2.3.1; python_version >= "3.4"
wrapt==1.14.0; python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.5"
zipp==3.8.0; python_version < "3.8" and python_version >= "3.7"



### Python Environment

```shell
PYTHONPATH=.

`hip` referenced before assignment

Traceback (most recent call last):
 File "/home/julian/PycharmProjects/xxx-xxxx/venv/lib/python3.5/site-packages/instana/util.py", line 201, in get_default_gateway
   if hip is not None and len(hip) is 8:
UnboundLocalError: local variable 'hip' referenced before assignment

Code:

python-sensor/instana/util.py

Line 221 in 3c81c09

if hip is not None and len(hip) is 8:

In certain cases, package version numbers not collected

In some cases, more modules than desired fallback to builtin when they should really have an actual version. This example is from an app with Django 1.3.7. The real version should properly be detected.

urllib3 will drop support for Python 2.7 & 3.5 in v.2.0

⚠️ IMPORTANT: urllib3 v2.0 will drop support for Python 2: Read more in the v2.0 Roadmap

https://github.com/urllib3/urllib3/releases/tag/1.26.4

Importing ABC directly from collections has been removed in Python 3.10 in favor of collections.abc

python-sensor/instana/instrumentation/logging.py

Line 28 in 94fd99b

 if args and len(args) == 1 and isinstance(args[0], collections.Mapping) and args[0]: 

aiohttp middleware returns None if handler raises

https://github.com/instana/python-sensor/blob/master/instana/instrumentation/aiohttp/server.py#L40 calling the handler is wrapped in a try-except block, that logs and discards the exception: https://github.com/instana/python-sensor/blob/master/instana/instrumentation/aiohttp/server.py#L56

This causes issues with the utility exceptions provided in aiohttp.web_exceptions, and conflicts with the behaviour other middleware can implement based on exceptions a handler would raise.

Spyne 2.12.15 Breaks Tests

Recently released Spyne is causing these Travis tests to fail:

Collecting spyne>=2.9 (from instana==1.8.7->-r requirements-test.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/cb/73/67e489bdfdf8ed9f927001595a1e57891b12e580fcfa9aa64b3c4df16a8a/spyne-2.12.15.tar.gz (439kB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-kxoz8746/spyne/setup.py", line 51, in <module>
        LONG_DESC += u"\n\n" + open('CHANGELOG.rst', 'r').read().decode('utf8')
    AttributeError: 'str' object has no attribute 'decode'

Spyne is used in the test suite and not a general dependency in normal installs.

Limited the version for the time being.

Typo in Pika instrumentation

Line 101: https://github.com/instana/python-sensor/blob/master/instana/instrumentation/pika.py#L101

Should be properties

Creating this issue, so I fix it later

Add support of TEXT_MAP format for injection/extraction of the span data

User story

As an application developer who collects tracing data in a very tricky way
I want Instana tracing library to support TEXT_MAP format for injection/extraction of the span data
So that I don't need to implement this standard way of injection/extraction myself

Explanation

Currently instana tracer is able to inject/extract span data using HTTPPropagator which works fine for tracing HTTP requests. However, in my case it's needed to work with data coming from a queue. And queue messages don't support HTTP headers :)

Another reason for adding support of TEXT_MAP format is that it's suggested by Opentracing standard: http://opentracing.io/documentation/pages/api/cross-process-tracing.html#required-injectextract-carrier-formats

Acceptance criteria

It is possible to extract span data with a following call:

tracer.extract(
                    format=Format.TEXT_MAP,
                    carrier=trace,
                )

The same goes for tracer.inject call

	if "rabbitmq" in self.data and self.data["rabbitmq"]["sort"] == "consume":
	self.k = 1 # entry

instana / python-sensor Goto Github PK

python-sensor's Introduction

Instana

Installation

Manual Installation

Activating Without Code Changes

Activating via Import

Documentation

Contributing

More

python-sensor's People

Contributors

Stargazers

Watchers

Forkers

python-sensor's Issues

Problem Description

Minimal, Complete, Verifiable, Example

Python Version

Python Modules

Python Environment

Snippet to reproduce

Problem Description

Minimal, Complete, Verifiable, Example

Python Version

Python Modules

Python Environment

Problem Description

Minimal, Complete, Verifiable, Example

Python Version

Python Modules

User story

Explanation

Acceptance criteria

Recommend Projects

Recommend Topics

Recommend Org