Giter Club home page Giter Club logo

thundra-agent-python's Introduction

Thundra Agent Python For Tracing

OpenTracing Badge Pyversions PyPI

Trace your marvelous python projects with async monitoring by Thundra!

Async Monitoring with Zero Overhead

By default, Thundra agent reports by making an HTTPS request. This adds an overhead to your lambda function.

Instead, you can setup async monitoring in 2 minutes and monitor your lambda functions with zero overhead!

Contents

Installation

Run this command from your project directory:

pip install thundra -t .

Configuration

You can configure Thundra using environment variables or module initialization parameters.

Environment variables have higher precedence over initialization parameters.

Check out the configuration part of our docs for more detailed information.

1. Most Useful Environment variables

Name Type Default Value
THUNDRA_APIKEY string -
THUNDRA_AGENT_APPLICATION_NAME string -
THUNDRA_AGENT_APPLICATION_STAGE string -
THUNDRA_AGENT_TRACE_DISABLE bool false
THUNDRA_AGENT_METRIC_DISABLE bool false
THUNDRA_AGENT_LOG_DISABLE bool false
THUNDRA_AGENT_TRACE_REQUEST_SKIP bool false
THUNDRA_AGENT_TRACE_RESPONSE_SKIP bool false
THUNDRA_AGENT_LAMBDA_TIMEOUT_MARGIN int 200
THUNDRA_AGENT_REPORT_REST_BASEURL string https://collector.thundra.io/v1
THUNDRA_AGENT_TEST_RUN_ID string -
THUNDRA_AGENT_TEST_PROJECT_ID string -
THUNDRA_AGENT_TEST_STATUS_REPORT_FREQUENCY int 30sec
THUNDRA_AGENT_TEST_LOG_COUNT_MAX int 100

2. Object initialization parameters

Name Type Default Value
api_key string -
disable_trace bool False
disable_metric bool False
disable_log bool False

Usage

No-Code Change Tracing

The simplest way to auto trace configuration of all your endpoints into project by setting following environment variables:

export THUNDRA_APIKEY=<your_thundra_api_key>
export THUNDRA_AGENT_APPLICATION_NAME=<your_application_name>
<python command>

For illustration of Fastapi:

export THUNDRA_APIKEY=<your_thundra_api_key>
export THUNDRA_AGENT_APPLICATION_NAME=fastapi-prod
uvicorn main:app --reload

For Dockerfile, you just replace export with ENV.

You can see the list of no-code change supported frameworks

In-Code Configuration Tracing

Another simple alternative is to copy the snippet into your code before your app created:

import thundra

thundra.configure(
     options={
         "config": {
             "thundra.apikey": <your_thundra_api_key>,
             "thundra.agent.application.name": <your_application_name>
         }
     }
)

To run on your framework please refer to supported frameworks

Decorator

Just import this module, pass your api key to it and use our decorator to wrap your handler:

import thundra

thundra.configure(
     options={
         "config": {
             "thundra.apikey": <your_thundra_api_key>,
             "thundra.agent.application.name": <your_application_name>
         }
     }
)

@thundra.<framework_wrapper>
def handler(event, context):
    print("Hello Thundra!")

OR

export THUNDRA_APIKEY=<your_thundra_api_key>
export THUNDRA_AGENT_APPLICATION_NAME=<your_application_name>
import thundra

@thundra.<framework_wrapper>
def handler(event, context):
    print("Hello Thundra!")

NOTES

  • To run on your framework please refer to supported frameworks
  • You should disable framework that you use by using environment variable or in-code configuration. Explanations can be found in corresponding framework section.

Request Body

Normally thundra does not get your project's requests' body. In order to see request body on tracing data, following environment variable shall be set besides api key and application name:

export THUNDRA_AGENT_TRACE_REQUEST_SKIP=True

OR

import thundra

thundra.configure(
     options={
         "config": {
             "thundra.apikey": <your_thundra_api_key>,
             "thundra.agent.application.name": <your_application_name>,
			 "thundra.agent.trace.request.skip": True
         }
     }
)

Frameworks

The following frameworks are supported by Thundra:

Framework Supported Version Auto-tracing Supported
AWS Lambda All
Django >=1.11
Flask >=0.5
Fastapi >=0.62.0
Chalice >=1.0.0
Tornado >=6.0.0

AWS Lambda

Tracing Lambda functions:

  1. Check our integration document on web site

NOTES

  • Make sure to choose just one of the methods from integration document above.
  • For decorator tracing method:
    1. Set THUNDRA_APIKEY and other configurations if needed as no-code change or in-code change sytle.
    2. Change @thundra.<framework_wrapper> by @thundra.lambda_wrapper.
  • In order to activate AWS Step Functions trace, THUNDRA_AGENT_LAMBDA_AWS_STEPFUNCTIONS environment variable should be set true.
  • In order to activate AWS AppSync trace, THUNDRA_AGENT_LAMBDA_AWS_APPSYNC environment variable should be set true.
  • For other integrations' configuration, please take a look environment variables table at the end.

Django

Tracing Django application:

  1. Auto trace by using one of the following methods:
  2. Trace specific endpoints by Decorator

Code snippet in In-Code Configuration Tracing should be added into settings.py file. To trace django database processes, following environment variables shall be set to false:

  • No-Code Change Tracing:

    export THUNDRA_AGENT_TRACE_INTEGRATIONS_DJANGO_ORM_DISABLE=false
    export THUNDRA_AGENT_TRACE_INTEGRATIONS_RDB_DISABLE=false
  • In-Code Configuration Tracing:

    import thundra
    
    thundra.configure(
        options={
            "config": {
                "thundra.apikey": <your_thundra_api_key>,
                "thundra.agent.application.name": <your_application_name>,
                "thundra.agent.trace.integrations.django.orm.disable": False,
                "thundra.agent.trace.integrations.rdb.disable": False
            }
        }
    )

NOTES

  • Make sure to choose just one of the methods.
  • For decorator tracing method:
    1. Disable thundra django instrumentation by using THUNDRA_AGENT_TRACE_INTEGRATIONS_DJANGO_DISABLE or its corresponding in-code configuration.
    2. Change @thundra.<framework_wrapper> by @thundra.django_wrapper.

Flask

Tracing Flask application:

  1. Auto trace by using one of the following methods:
  2. Trace specific endpoints by Decorator

Code snippet in In-Code Configuration Tracing should be added into app.py file before application is initialized.

NOTES

  • Make sure to choose just one of the methods.
  • For decorator tracing method:
    1. Disable thundra flask instrumentation by using THUNDRA_AGENT_TRACE_INTEGRATIONS_FLASK_DISABLE or its corresponding in-code configuration.
    2. Change @thundra.<framework_wrapper> by @thundra.flask_wrapper.

Fastapi

Tracing Fastapi application:

  1. Auto trace by using one of the following methods:
  2. Trace specific endpoints by Decorator by adding fastapi.Request as an parameter to your function if not exists.

NOTES

  • Make sure to choose just one of the methods.
  • For decorator tracing method:
    1. Disable thundra fastapi instrumentation by using THUNDRA_AGENT_TRACE_INTEGRATIONS_FASTAPI_DISABLE or its corresponding in-code configuration.
    2. Change @thundra.<framework_wrapper> by @thundra.fastapi_wrapper.
  • Fastapi has been supported for python 3.7 and above.

Chalice

Tracing Chalice application:

  1. Auto trace by using one of the following methods:
  2. Trace specific endpoints by Decorator

NOTES

  • Make sure to choose just one of the methods.
  • For decorator tracing method:
    1. Disable thundra chalice instrumentation by using THUNDRA_AGENT_TRACE_INTEGRATIONS_CHALICE_DISABLE or its corresponding in-code configuration.
    2. Change @thundra.<framework_wrapper> by @thundra.chalice_wrapper.

Tornado

Tracing Tornado application:

  1. Auto trace by using one of the following methods:
  2. Trace specific endpoints by Decorator

NOTES

  • Make sure to choose just one of the methods.
  • Tornado has been supported for python 3.7 and above.
  • For decorator tracing method:
    1. Disable thundra tornado instrumentation by using THUNDRA_AGENT_TRACE_INTEGRATIONS_TORNADO_DISABLE or its corresponding in-code configuration.
    2. Change @thundra.<framework_wrapper> by @thundra.tornado_wrapper.

Integrations

Thundra provides out-of-the-box instrumentation (tracing) for following libraries.

Library Supported Version
logging Fully supported
requests >=2.0.0
redis >=2.10.0
pymongo >=3.0.0
PyMySQL >=0.7.0
MySQLdb >=1.0.0
psycopg2 >=2.2.0
botocore (boto3) >=1.4.0
SQLAlchemy >=1.2.0
elasticsearch >=0.4.1
aiohttp >=1.1.5

Log Plugin

Log plugin is added by default, but in order to enable it, you should add ThundraLogHandler to your logger.

You can either add it via logging.conf or add during getting logger object.

Configure via logging.conf

An example of configuration file is as follows:

[loggers]
keys=root

[handlers]
keys=thundraHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=NOTSET
handlers=thundraHandler

[handler_thundraHandler]
class=ThundraLogHandler
level=NOTSET
formatter=simpleFormatter
args=()

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

Configure during getting logger object

You should add the followings after getting the logger object:

handler = ThundraLogHandler()
logger.addHandler(handler)
...
logger.removeHandler(handler)

Thundra Foresight

Foresight is a project powered by Thundra agent to show every detail for test runs. For know, it only supports pytest. Although Its automatically enabled after installing Thundra as 'pip install thundra', you need to set THUNDRA_APIKEY and THUNDRA_AGENT_TEST_PROJECT_ID as described below.

Activating Thundra Foresight

  • Firstly, setting up environment for Thundra. There are three ways to do it.

    1. Setting THUNDRA_APIKEY and THUNDRA_AGENT_TEST_PROJECT_ID as environment variable.
    export THUNDRA_APIKEY=<your_apikey>
    export THUNDRA_AGENT_TEST_PROJECT_ID=<your_test_project_id>
    1. "thundra_apikey " and "thundra_agent_test_project_id" in you .env file. Then, loaded them in conftest.py file like:
        thundra_apikey =<your_apikey>
        thundra_agent_test_project_id = <your_test_project_id>
        # This method requires python >= 3.5
        from pathlib import Path
    
        from dotenv import load_dotenv
    
        env_path = Path(<your_env_file_name>)
        load_dotenv(dotenv_path=env_path)
    1. Importing Thundra in conftest file and configure it.
        import thundra
    
        thundra.configure(
            options={
                "config": {
                    "thundra.apikey": <your_apikey>,
                    "thundra.agent.test.project.id": <your_test_project_id>
                }
            }
        )

Deactivating Thundra Foresight

  • There are three ways to deactivate Thundra Foresight for pytest:

    1. Run pytest with --thundra_disable command on terminal for specific pytest run.
        pytest --thundra_disable <your_tests_path>
    1. Modifying any configuration file read by pytest(pytest.ini, setup.cfg, pyproject.toml etc.) Please read carefully pytest official documentation for configuration files.
        [pytest]
        thundra_disable = 1

    or

        [tool.pytest.ini_options]
        thundra_disable = 1
    1. Setting THUNDRA_AGENT_TEST_DISABLE=True as environment variable.

NOTES

  • All Thundra agent features are valid in Foresight. It's default enabled. If you see more information about your test cases, you can visit Thundra APM. If you want to disable thundra agent for tracing, you can set "THUNDRA_AGENT_DISABLE" as environment variable, "thundra_agent_disable" in .env file or "thundra.agent.disable" into thundra.configure() to True as describing above.

Getting Help

If you have any issue around using the library or the product, please don't hesitate to:

  • Use the documentation.
  • Open an issue in GitHub.
  • Join our python slack channel.

Opening Issues

For any problem you encounter while using thundra, Please feel free to contact us via github issue or our python slack channel.

When opening a new issue, please provide as much information about the environment:

  • Library version, Python runtime version, dependencies, operation system with version etc.
  • Snippet of the usage.
  • A reproducible example can really help.

The GitHub issues are intended for bug reports and feature requests.

All Environment Variables

Following table shows all environment variables of No-Code Change Tracing and their In-Code Configuration Tracing equivalent.

Environment Variable Name In-Code Configuration Tracing Name
THUNDRA_APIKEY thundra.apikey
THUNDRA_AGENT_DEBUG_ENABLE thundra.agent.debug.enable
THUNDRA_AGENT_DISABLE thundra.agent.disable
THUNDRA_AGENT_TRACE_DISABLE thundra.agent.trace.disable
THUNDRA_AGENT_METRIC_DISABLE thundra.agent.metric.disable
THUNDRA_AGENT_LOG_DISABLE thundra.agent.log.disable
THUNDRA_AGENT_APPLICATION_ID thundra.agent.application.id
THUNDRA_AGENT_APPLICATION_INSTANCE_ID thundra.agent.application.instanceid
THUNDRA_AGENT_APPLICATION_NAME thundra.agent.application.name
THUNDRA_AGENT_APPLICATION_STAGE thundra.agent.application.stage
THUNDRA_AGENT_APPLICATION_DOMAIN_NAME thundra.agent.application.domainname
THUNDRA_AGENT_APPLICATION_CLASS_NAME thundra.agent.application.classname
THUNDRA_AGENT_APPLICATION_VERSION thundra.agent.application.version
THUNDRA_AGENT_APPLICATION_TAG_PREFIX thundra.agent.application.tag
THUNDRA_AGENT_APPLICATION_REGION thundra.agent.application.region
THUNDRA_AGENT_REPORT_REST_BASEURL thundra.agent.report.rest.baseurl
THUNDRA_AGENT_REPORT_CLOUDWATCH_ENABLE thundra.agent.report.cloudwatch.enable
THUNDRA_AGENT_REPORT_REST_COMPOSITE_BATCH_SIZE thundra.agent.report.rest.composite.batchsize
THUNDRA_AGENT_REPORT_CLOUDWATCH_COMPOSITE_BATCH_SIZE thundra.agent.report.cloudwatch.composite.batchsize
THUNDRA_AGENT_REPORT_REST_COMPOSITE_ENABLE thundra.agent.report.rest.composite.enable
THUNDRA_AGENT_REPORT_CLOUDWATCH_COMPOSITE_ENABLE thundra.agent.report.cloudwatch.composite.enable
THUNDRA_AGENT_REPORT_REST_LOCAL thundra.agent.report.rest.local
THUNDRA_AGENT_LAMBDA_HANDLER thundra.agent.lambda.handler
THUNDRA_AGENT_LAMBDA_WARMUP_WARMUPAWARE thundra.agent.lambda.warmup.warmupaware
THUNDRA_AGENT_LAMBDA_TIMEOUT_MARGIN thundra.agent.lambda.timeout.margin
THUNDRA_AGENT_LAMBDA_ERROR_STACKTRACE_MASK thundra.agent.lambda.error.stacktrace.mask
THUNDRA_AGENT_TRACE_REQUEST_SKIP thundra.agent.trace.request.skip
THUNDRA_AGENT_TRACE_RESPONSE_SKIP thundra.agent.trace.response.skip
THUNDRA_AGENT_LAMBDA_TRACE_KINESIS_REQUEST_ENABLE thundra.agent.lambda.trace.kinesis.request.enable
THUNDRA_AGENT_LAMBDA_TRACE_FIREHOSE_REQUEST_ENABLE thundra.agent.lambda.trace.firehose.request.enable
THUNDRA_AGENT_LAMBDA_TRACE_CLOUDWATCHLOG_REQUEST_ENABLE thundra.agent.lambda.trace.cloudwatchlog.request.enable
THUNDRA_AGENT_LAMBDA_AWS_STEPFUNCTIONS thundra.agent.lambda.aws.stepfunctions
THUNDRA_AGENT_LAMBDA_AWS_APPSYNC thundra.agent.lambda.aws.appsync
THUNDRA_AGENT_TRACE_INSTRUMENT_DISABLE thundra.agent.trace.instrument.disable
THUNDRA_AGENT_TRACE_INSTRUMENT_TRACEABLECONFIG thundra.agent.trace.instrument.traceableconfig
THUNDRA_AGENT_TRACE_SPAN_LISTENERCONFIG thundra.agent.trace.span.listenerconfig
THUNDRA_AGENT_SAMPLER_TIMEAWARE_TIMEFREQ thundra.agent.sampler.timeaware.timefreq
THUNDRA_AGENT_SAMPLER_COUNTAWARE_COUNTFREQ thundra.agent.sampler.countaware.countfreq
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SNS_MESSAGE_MASK thundra.agent.trace.integrations.aws.sns.message.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SNS_TRACEINJECTION_DISABLE thundra.agent.trace.integrations.aws.sns.traceinjection.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SQS_MESSAGE_MASK thundra.agent.trace.integrations.aws.sqs.message.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SQS_TRACEINJECTION_DISABLE thundra.agent.trace.integrations.aws.sqs.traceinjection.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_LAMBDA_PAYLOAD_MASK thundra.agent.trace.integrations.aws.lambda.payload.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_LAMBDA_TRACEINJECTION_DISABLE thundra.agent.trace.integrations.aws.lambda.traceinjection.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_DYNAMODB_STATEMENT_MASK thundra.agent.trace.integrations.aws.dynamodb.statement.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_DYNAMODB_TRACEINJECTION_ENABLE thundra.agent.trace.integrations.aws.dynamodb.traceinjection.enable
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_ATHENA_STATEMENT_MASK thundra.agent.trace.integrations.aws.athena.statement.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_BODY_MASK thundra.agent.trace.integrations.http.body.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_URL_DEPTH thundra.agent.trace.integrations.http.url.depth
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_TRACEINJECTION_DISABLE thundra.agent.trace.integrations.http.traceinjection.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_ERROR_STATUS_CODE_MIN thundra.agent.trace.integrations.http.error.status.code.min
THUNDRA_AGENT_TRACE_INTEGRATIONS_REDIS_COMMAND_MASK thundra.agent.trace.integrations.redis.command.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_RDB_STATEMENT_MASK thundra.agent.trace.integrations.rdb.statement.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_ELASTICSEARCH_BODY_MASK thundra.agent.trace.integrations.elasticsearch.body.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_ELASTICSEARCH_PATH_DEPTH thundra.agent.trace.integrations.elasticsearch.path.depth
THUNDRA_AGENT_TRACE_INTEGRATIONS_MONGODB_COMMAND_MASK thundra.agent.trace.integrations.mongodb.command.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_EVENTBRIDGE_DETAIL_MASK thundra.agent.trace.integrations.aws.eventbridge.detail.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SES_MAIL_MASK thundra.agent.trace.integrations.aws.ses.mail.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_SES_MAIL_DESTINATION_MASK thundra.agent.trace.integrations.aws.ses.mail.destination.mask
THUNDRA_AGENT_TRACE_INTEGRATIONS_HTTP_DISABLE thundra.agent.trace.integrations.http.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_RDB_DISABLE thundra.agent.trace.integrations.rdb.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_AWS_DISABLE thundra.agent.trace.integrations.aws.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_REDIS_DISABLE thundra.agent.trace.integrations.redis.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_ES_DISABLE thundra.agent.trace.integrations.elasticsearch.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_MONGO_DISABLE thundra.agent.trace.integrations.mongodb.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_SQLALCHEMY_DISABLE thundra.agent.trace.integrations.sqlalchemy.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_CHALICE_DISABLE thundra.agent.trace.integrations.chalice.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_DJANGO_DISABLE thundra.agent.trace.integrations.django.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_DJANGO_ORM_DISABLE thundra.agent.trace.integrations.django.orm.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_FLASK_DISABLE thundra.agent.trace.integrations.flask.disable
THUNDRA_AGENT_TRACE_INTEGRATIONS_FASTAPI_DISABLE thundra.agent.trace.integrations.fastapi.disable
THUNDRA_AGENT_LOG_CONSOLE_DISABLE thundra.agent.log.console.disable
THUNDRA_AGENT_LOG_LOGLEVEL thundra.agent.log.loglevel
THUNDRA_AGENT_LAMBDA_DEBUGGER_ENABLE thundra.agent.lambda.debugger.enable
THUNDRA_AGENT_LAMBDA_DEBUGGER_PORT thundra.agent.lambda.debugger.port
THUNDRA_AGENT_LAMBDA_DEBUGGER_LOGS_ENABLE thundra.agent.lambda.debugger.logs.enable
THUNDRA_AGENT_LAMBDA_DEBUGGER_WAIT_MAX thundra.agent.lambda.debugger.wait.max
THUNDRA_AGENT_LAMBDA_DEBUGGER_IO_WAIT thundra.agent.lambda.debugger.io.wait
THUNDRA_AGENT_LAMBDA_DEBUGGER_BROKER_PORT thundra.agent.lambda.debugger.broker.port
THUNDRA_AGENT_LAMBDA_DEBUGGER_BROKER_HOST thundra.agent.lambda.debugger.broker.host
THUNDRA_AGENT_LAMBDA_DEBUGGER_SESSION_NAME thundra.agent.lambda.debugger.session.name
THUNDRA_AGENT_LAMBDA_DEBUGGER_AUTH_TOKEN thundra.agent.lambda.debugger.auth.token
THUNDRA_AGENT_TEST_RUN_ID thundra.agent.test.run.id
THUNDRA_AGENT_TEST_PROJECT_ID thundra.agent.test.project.id
THUNDRA_AGENT_TEST_STATUS_REPORT_FREQUENCY thundra.agent.test.status.report.freq
THUNDRA_AGENT_TEST_LOG_COUNT_MAX thundra.agent.test.log.count.max
THUNDRA_AGENT_TEST_DISABLE thundra.agent.test.disable

thundra-agent-python's People

Contributors

adursun avatar bunyaminsg avatar default-usr avatar dependabot[bot] avatar dsalv avatar erdogan-yesil avatar faziletozer avatar gokhan721 avatar hamitb avatar kobalski avatar morellic avatar plazma-prizma avatar rwxdash avatar salihkardan avatar sevenones avatar sturmianseq avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

thundra-agent-python's Issues

Django server crashes on start

After adding thundra to my django settings, my container crashes on start.

From my settings:

import thundra

thundra.configure(
    options={"config": {"thundra.apikey": "xxxx",}}
)

The command it fails on:

python /code/manage.py rundebugserver 0.0.0.0:80

Trace:

Exception in thread django-main-thread:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", line 53, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 121, in inner_run
    self.check_migrations()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 459, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/loader.py", line 53, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/loader.py", line 216, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 55, in has_table
    with self.connection.cursor() as cursor:
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 259, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 235, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/base/base.py", line 200, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 206, in get_new_connection
    psycopg2.extras.register_default_jsonb(conn_or_curs=connection, loads=lambda x: x)
  File "/usr/local/lib/python3.7/site-packages/psycopg2/_json.py", line 156, in register_default_jsonb
    loads=loads, oid=JSONB_OID, array_oid=JSONBARRAY_OID, name='jsonb')
  File "/usr/local/lib/python3.7/site-packages/psycopg2/_json.py", line 125, in register_json
    register_type(JSON, not globally and conn_or_curs or None)
TypeError: argument 2 must be a connection, cursor or None

My dependencies:

python = "3.7.7"
argon2-cffi = "^20.1.0"
asgiref = "^3.2.10"
boto3 = "^1.14.45"
dj-database-url = "^0.5.0"
django = "^3.1.4"
django-axes = "^5.4.3"
django-memoize = "^2.2.1"
django-storages = "^1.9.1"
django-enhanced-emails = "^0.0.7"
django-ses = "^1.0.2"
djangorestframework = "^3.12.2"
djangorestframework_simplejwt = "^4.4.0"
djoser = "^2.0.3"
psycopg2-binary = "^2.8.6"
pytz = "^2020.1"
requests = "^2.24.0"
sentry-sdk = "^0.19.5"
sqlparse = "^0.3.1"
watchtower = "^0.8.0"
django-phonenumber-field = {version = "^4.0.0", extras = ["phonenumbers"]}
sodapy = "^2.1.0"
httpx = "^0.13.1"
pandas = "^1.1.1"
xlrd = "^1.2.0"
django-filter = "^2.2.0"
django-import-export = "^2.2.0"
scrapy = "^2.1.0"
scrapy_djangoitem = "^1.1.1"
django-dirtyfields = "^1.4.1"
thundra = "^2.5.7"

It also fails, with the same error, on a production gunicorn start.

Thundra causes exception in lambda that uses SSM

SSM is important for secrets management.
Thundra injects itself into botocore and tries to wrap functions. But if it doesn't have a subclass for the required function it just crashes.

File "/var/runtime/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/var/task/thundra/integrations/modules/botocore.py", line 22, in _wrapper
response = INTEGRATIONS[integration_name].create_span(
KeyError: 'ssm'

The issue didn't exist before adding thundra, and removing thundra makes the issue go away.

Note: this happens in a call to boto3 that occurs before thundra agent is loaded:

        ssm_client = boto3.client('ssm', region_name='us-east-1')
        response = ssm_client.get_parameter(parameter_name)

module 'botocore.vendored.requests' has no attribute 'Session'

Thundra setup is causing a following error on AWS Lambda:

module initialization error
module 'botocore.vendored.requests' has no attribute 'Session'

The recent updates in the botocore could be a reason to that: https://aws.amazon.com/blogs/developer/removing-the-vendored-version-of-requests-from-botocore/.

Might be related to the following line: https://github.com/thundra-io/thundra-lambda-agent-python/blob/master/thundra/integrations/modules/botocore.py#L40, couldn't find any other mentions of botocore.vendored.requests in the source code.

AutoTracing Support For Lambda -> Lambda Asynchronous Invocation

๐Ÿ‘‹ I have 2 simple lambdas setup using Asynchronous invocation, but they are not showing up as connected in the thundra UI.

I looked at the docs but I think I am missing something. Could you please advise on the best practice for propagating a trace to the asynchronously invoked lambda using thundra python client??

Thank you

Lambda 1 AWS Config Events

Screen Shot 2020-02-15 at 11 28 48 AM

Lambda 1

from thundra.thundra_agent import Thundra

thundra = Thundra()

@thundra
def handler(event, context):
    print(event, context)
    return {'message': 'hello'}

Lambda 2

from thundra.thundra_agent import Thundra

thundra = Thundra()

@thundra
def handler(event, context):
    print('received: {} {}'.format(event, context))
    return {'hi': 'hi'}

Architecture

Screen Shot 2020-02-15 at 11 30 44 AM

Time of availability ?

Hi guys,

It is not an issue, but I have a lot of serverless functions written in python, and I would like to know when do you think the python agent could be available ?

Thanks !
Best

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.