tyktechnologies / tyk-gateway-docker Goto Github PK

View Code? Open in Web Editor NEW

85.0 35.0 77.0 16.54 MB

Docker compose deployment to run Tyk OSS Gateway

JavaScript 100.00%

api-gateway gateway open-source opensource tyk tyk-gateway

tyk-gateway-docker's People

Stargazers

Watchers

tyk-gateway-docker's Issues

Docker-compose is broken as it expects "apps" to be a file, not a folder

Cloning this repo and running docker-compose up -d fails because:

ERROR: for tyk-gateway-docker_tyk-gateway_1  Cannot start service tyk-gateway: OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:495: container init caused: rootfs_linux.go:60: mounting "/host_mnt/Users/me/workspace/tyk/tyk-gateway-docker/apps" to rootfs at "/var/lib/docker/overlay2/685a030b0618a20ba4ad5810a0989ecec6748cde11bccb49280cf32844582d89/merged/opt/tyk-gateway/apps" caused: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

Investigating further, the issue is because the the path specified in docker-compose (/opt/tyk-gateway/apps) is actually a file inside of this container, not a folder. But "apps" is a folder when cloning this as-is.

Inside the docker container:

root@b1138c98a19b:/opt/tyk-gateway# ls -lart
total 51068
-rw-r--r-- 1 root root      888 May  6 07:42 tyk.conf
-rw-r--r-- 1 root root      826 May  6 07:42 apps
-rwxr-xr-x 1 root root 52244752 May  6 07:43 tyk
drwxr-xr-x 2 root root     4096 May  6 07:46 templates

Easy enough to fix, but the README documentation is not accurate.

Sample config files missing

In the docker repo description at https://hub.docker.com/r/tykio/tyk-gateway/ there is this info:

Run a standalone Tyk Gateway with your (modified) tyk.conf (see sample configs in our docker github repository):

I'm expecting to find the config files here in the repo, but they are not here.

Issue while deployment

Helo, so i constantly get following issue and i don't understand how i can resolve this, i found multiple tyk gateway images but all of them caused the same issue.

Started Container tyk-gateway-tyk-gateway-1
Starting Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/data/compose/41/tyk.standalone.conf" to rootfs at "/opt/tyk-gateway/tyk.conf": mount /data/compose/41/tyk.standalone.conf:/opt/tyk-gateway/tyk.conf (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown:
Are you trying to mount a directory onto a file (or vice-versa)?
Check if the specified host path exists and is the expected type

here my compose file:

version: '3.0'
services:
  tyk-gateway:
    image: tykio/tyk-gateway:v4.1.0
    #ports:
      #- 8080:8080
    networks:
      webproxy:
        ipv4_address: 172.26.0.17
    volumes:
      - ./tyk.standalone.conf:/opt/tyk-gateway/tyk.conf
      - ./apps:/opt/tyk-gateway/apps
      - ./middleware:/opt/tyk-gateway/middleware
      - ./certs:/opt/tyk-gateway/certs
    environment:
      - TYK_GW_SECRET=foo
    depends_on:
      - tyk-redis
      
  tyk-redis:
    image: redis:6.2.7-alpine
    networks:
      webproxy:
        ipv4_address: 172.26.0.205
    #ports:
      #- 6379:6379

networks:
  webproxy:
    driver: bridge
    external: true
    ```

how do I configure standalone Tyk?

https://hub.docker.com/r/tykio/tyk-gateway/ says to see this repo for how to configure tyk.conf for a standalone deployment (no dashboard). How can I do this?

Connectivity issue with gateway and dashboard

I'm setting up gateway v2.1.0.2 with dashboard v1.1.

I'm using the following configurations:

Gateway:
tyk.conf

{
    "listen_port": 8080,
    "secret": "352d20ee67be67f6340b4c0605b044b7",
    "node_secret": "352d20ee67be67f6340b4c0605b044b7",
    "template_path": "/opt/tyk-gateway/templates",
    "tyk_js_path": "/opt/tyk-gateway/js/tyk.js",
    "middleware_path": "/opt/tyk-gateway/middleware",
    "use_db_app_configs": true,
    "db_app_conf_options": {
        "connection_string": "http://1.1.1.1:3000",
        "node_is_segmented": false,
        "tags": ["test2"]
    },
    "app_path": "/opt/tyk-gateway/apps/",
    "storage": {
        "type": "redis",
        "host": "1.1.1.1",
        "port": 6379,
        "username": "",
        "password": "",
        "database": 0,
        "optimisation_max_idle": 100
    },
    "enable_analytics": true,
    "analytics_config": {
        "type": "mongo",
        "csv_dir": "/tmp",
        "mongo_url": "",
        "mongo_db_name": "",
        "mongo_collection": "",
        "purge_delay": -1,
        "ignored_ips": []
    },
    "health_check": {
        "enable_health_checks": true,
        "health_check_value_timeouts": 60
    },
    "optimisations_use_async_session_write": true,
    "enable_non_transactional_rate_limiter": true,
    "enable_sentinel_rate_limiter": false,
    "allow_master_keys": false,
    "policies": {
        "policy_source": "service",
        "policy_connection_string": "http://1.1.1.1:3000",
        "policy_record_name": "tyk_policies"
    },
    "hash_keys": true,
    "close_connections": true
}

Where 1.1.1.1 is the virtual IP to the cluster where we are installing tyk. Also, 1.1.1.1:3000 is dashboard container port 3000.

Dashboard:
tyk_analytics.conf

{
    "listen_port": 3000,
    "tyk_api_config": {
        "Host": "http://1.1.1.1",
        "Port": "3001",
        "Secret": "352d20ee67be67f6340b4c0605b044b7"
    },
    "mongo_url": "mongodb://1.1.1.1:27017/tyk_analytics",
    "page_size": 10,
    "admin_secret": "12345",
    "shared_node_secret": "352d20ee67be67f6340b4c0605b044b7",
    "redis_port": 6379,
    "redis_host": "1.1.1.1",
    "redis_password": "",
    "enable_cluster": false,
    "force_api_defaults": false,
    "notify_on_change": true,
    "license_key": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJhbGxvd2VkX25vZGVzIjoiZDE2ODBmYTEtMTJkYi00ZjdhLTU3ZTQtZDZlNDFjNWI0MTdmIiwiZXhwIjoxNDk1NzkzMTYyLCJvd25lciI6IjU3NGQ2MjA5NDVmOTJlNjY4OTAwMDA5ZCJ9.0E-xlcJXTntqqKLdnXiEpFnhtcApydBtERKbrEk-Xc5oc5-k9P_k7K7x2n79zQnz-5kKhERDXtoO9yoBamE4He8Gucw7K0tKQZ90NIem_I81FoXWeXunZAEWpWpie8Og9hL1Y2DoOXgCk-3HjmJB_4eyQjqPK8SLoGdoFKpj8ooDanQCkgVGPzLRIoAzHvl6vo6uJs1nUPbMyDhe6d6XeJtP9OnKeZdQiHmepa59gP9nxGSZ2V201U6Aja_iGKWlw2e0IBusjGcQljSg1CoAURVVC-RWL0LNvWelAYvLlPJwxxWJI7EYG04PGAW3r1GJHKjgzgGm7WNb-ABuMR-llQ",
    "redis_database": 0,
    "redis_hosts": null,
    "hash_keys": true,
    "email_backend": {
        "enable_email_notifications": false,
        "code": "",
        "settings": null,
        "default_from_email": "",
        "default_from_name": ""
    },
    "hide_listen_path": false,
    "sentry_code": "",
    "sentry_js_code": "",
    "use_sentry": false,
    "enable_master_keys": false,
    "enable_duplicate_slugs": true,
    "show_org_id": true,
    "host_config": {
        "enable_host_names": false,
        "disable_org_slug_prefix": true,
        "hostname": "",
        "override_hostname": "store-prod-agent-elb-665374692.ap-northeast-1.elb.amazonaws.com",
        "portal_domains": {},
        "portal_root_path": "/portal"
    },
    "http_server_options": {
        "use_ssl": false,
        "certificates": [
            {
                "domain_name": "",
                "cert_file": "",
                "key_file": ""
            }
        ],
        "min_version": 0
    },
    "ui": {
        "login_page": {},
        "nav": {},
        "uptime": {},
        "portal_section": null,
        "designer": {},
        "dont_show_admin_sockets": false,
        "dont_allow_license_management": false,
        "dont_allow_license_management_view": false
    },
    "home_dir": "/opt/tyk-dashboard",
    "identity_broker": {
        "enabled": false,
        "host": {
            "connection_string": "",
            "secret": ""
        }
    },
    "tagging_options": {
        "tag_all_apis_by_org": false
    }
}

Also, 1.1.1.1:3001 is virtual IP for gateway container port 8080.

Gateway stdout:

Registered docker executor on 10.0.2.139
Starting task tyk_gateway.9de57d0f-296f-11e6-9e3a-4eda5203db59:
Gateway stderr:

I0603 09:43:27.827817 23965 exec.cpp:143] Version: 0.28.1
I0603 09:43:27.831390 23994 exec.cpp:217] Executor registered on slave 851c0789-8d9b-4611-94f8-a5479a0f4f70-S81
time="Jun 3 09:43:28" level=info msg="Connection dropped, connecting.."
time="Jun 3 09:43:28" level=warning msg="Cache purging is no longer part of Tyk Gateway, please use Tyk-Pump."
time="Jun 3 09:43:28" level=info msg="Starting Poller"
time="Jun 3 09:43:28" level=info msg="Setting up Server"
time="Jun 3 09:43:28" level=info msg="--> Standard listener (http)"
time="Jun 3 09:43:28" level=info msg="Registering node."
time="Jun 3 09:43:28" level=error msg="Failed to register node, retrying in 5s"
time="Jun 3 09:43:33" level=error msg="Failed to register node, retrying in 5s"
time="Jun 3 09:43:38" level=error msg="Failed to register node, retrying in 5s"
time="Jun 3 09:43:43" level=error msg="Failed to register node, retrying in 5s"
time="Jun 3 09:43:48" level=error msg="Failed to register node, retrying in 5s"
time="Jun 3 09:43:53" level=error msg="Failed to register node, retrying in 5s"



Dashboard stderr:

I0603 09:38:39.017470 22689 exec.cpp:143] Version: 0.28.1
I0603 09:38:39.020246 22708 exec.cpp:217] Executor registered on slave 851c0789-8d9b-4611-94f8-a5479a0f4f70-S81

Dashboard stdout:
```Registered docker executor on 10.0.2.139
Starting task tyk_dashboard.ee2d829d-296e-11e6-9e3a-4eda5203db59
time="2016-06-03T09:38:39Z" level=info msg="Loading configuration from ./tyk_analytics.conf" 
time="2016-06-03T09:38:39Z" level=info msg="Connecting to redis on: 1.1.1.1:6379" 


�[1m�[32mTyk.io Analytics Dashboard v0.9.3�[0m�[0m
�[1m�[32m===============================�[0m�[0m
Copyright Jively Ltd. 2014
http://www.tyk.io

Listening on port:     3000
Tyk API at:            http://1.1.1.1:3001


[49 μs] 404 '/register/node'
[23 μs] 404 '/register/node'
[17 μs] 404 '/register/node'
[17 μs] 404 '/register/node'
[24 μs] 404 '/register/node'

Reduced docker image size

Wondering if tyk gateway container images be built from scratch image than ubuntu ?
If not cleaning up after installation does reduce the size from 1.5 GB to 830 MB.
One may want to check up the functionality.

FROM ubuntu:14.04
RUN apt-get update && apt-get install -y wget curl ca-certificates apt-transport-https curl && apt-get clean && apt-get autoremove
RUN curl https://packagecloud.io/gpg.key | apt-key add - && apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10 && apt-get update && apt-get install -y ca-certificates && apt-get install -y wget build-essential libluajit-5.1-2 luarocks lua-cjson python3-setuptools python3-dev libpython3.4 python3-pip && apt-get clean && apt-get autoremove
RUN wget https://github.com/google/protobuf/releases/download/v3.1.0/protobuf-python-3.1.0.tar.gz && tar -xvzf protobuf-python-3.1.0.tar.gz && cd protobuf-3.1.0/ && ./configure -prefix=/usr && make && make install && cd python && python3 setup.py build --cpp_implementation && python3 setup.py install --cpp_implementation && pip3 install grpcio && rm -fr /protobuf-3.1.0 /protobuf-python-3.1.0.tar.gz
RUN echo "deb https://packagecloud.io/tyk/tyk-gateway/ubuntu/ trusty main" | sudo tee /etc/apt/sources.list.d/tyk_tyk-gateway.list && echo "deb-src https://packagecloud.io/tyk/tyk-gateway/ubuntu/ trusty main" | sudo tee -a /etc/apt/sources.list.d/tyk_tyk-gateway.list && apt-get update && apt-get install -y tyk-gateway=2.3.1.2 && apt-get clean && apt-get autoremove

COPY ./tyk.standalone.conf /opt/tyk-gateway/tyk.conf
VOLUME ["/opt/tyk-gateway/"]

WORKDIR /opt/tyk-gateway
COPY entrypoint.sh /opt/tyk-gateway/entrypoint.sh

CMD ["./entrypoint.sh"]
EXPOSE 8080

Custom middleware in keyless-plugin-api.json does not work

Driver definition in keyless-plugin-api.json is at wrong place and the middleware does not work.

tyk-gateway-docker/apps/keyless-plugin-api.json

Line 28 in e44c765

"driver": "otto",

This should be under custom_middleware object like this:

    "custom_middleware": {
        "pre": [
            {
                "name": "testJSVMData",
                "path": "./middleware/injectHeader.js",
                "require_session": false,
                "raw_body_only": false
            }
        ],
        "driver": "otto"
    }

It's not clear which Dockerfile the automated build is based on

When checking

https://hub.docker.com/r/tykio/tyk-gateway/~/dockerfile/

on the right hand side the Github repo link is

https://github.com/lonelycode/tyk-gateway-docker

Following that link however redirects to this repo here

https://github.com/TykTechnologies/tyk-gateway-docker

The Dockerfile in here (https://github.com/TykTechnologies/tyk-gateway-docker/blob/master/Dockerfile) is not the same as is shown on https://hub.docker.com/r/tykio/tyk-gateway/~/dockerfile/ .

I hope the problem is clear.

title

body

"docker-compose up -d " will fail for image: "docker.tyk.io/tyk-gateway/tyk-gateway:v3.2.1 "

time="Oct 15 02:18:37" level=info msg="Tyk API Gateway v3.2.1" prefix=main
time="Oct 15 02:18:37" level=fatal msg="Error initialising system: open /opt/tyk-gateway/tyk.conf: permission denied" prefix=main

which made always fail in running the container.
solution: rebuild the iamge in Dockerfile, add following cmd makes it success.

COPY ./tyk.standalone.conf /opt/tyk-gateway/tyk.conf

env_vars: entrypoint.sh overrides environment variables

Documentation indicates which env vars are available to set for the gateway

https://tyk.io/docs/configure/gateway-env-variables/

listen_port	TYK_GW_LISTENPORT
secret	TYK_GW_SECRET

However entrypoint.sh overrides these variables

export TYK_GW_LISTENPORT="$TYKLISTENPORT"
export TYK_GW_SECRET="$TYKSECRET"

Either documentation should specify that if using standard docker installation, you need to set TYKLISTENPORT and TYKSECRET rather than TYK_GW_LISTENPORT and TYK_GW_SECRET.

entrypoint.sh should only set TYK_GW_LISTENPORT and TYK_GW_SECRET if they are not already set.

docker-compose image changed?

Trying to get the demo to work locally, it appears that the image tag pointing to docker.tyk.io does not work. Perhaps that docker registry has been deprecated?

tyk-gateway-docker/docker-compose.yml

Line 4 in 85b898f

image: docker.tyk.io/tyk-gateway/tyk-gateway:v4.1.0

I got it to work with the following tag:

image: tykio/tyk-gateway:v4.1.0

Python dispatcher can't initialise

When using a gateway image for v2.9.0 and later and configuring a Python plugin for an API, the gateway isn't able to initialise the co-process dispatcher.

Relevant logs:

time="Dec 18 21:26:15" level=info msg="----> Fetching Bundle: 60d3a06d-e4c2-45b1-90e8-1273cf98db8d" prefix=main
time="Dec 18 21:26:17" level=info msg="----> Loading bundle: 60d3a06d-e4c2-45b1-90e8-1273cf98db8d" prefix=main
time="Dec 18 21:26:17" level=info msg="----> Verifying bundle: 60d3a06d-e4c2-45b1-90e8-1273cf98db8d" prefix=main
time="Dec 18 21:26:17" level=info msg="----> Bundle is valid, adding to spec: 60d3a06d-e4c2-45b1-90e8-1273cf98db8d" prefix=main
time="Dec 18 21:26:17" level=error msg="Python version '' doesn't exist" error="No Python installations found"
time="Dec 18 21:26:17" level=error msg="Couldn't load Python dispatcher" error="python version '' doesn't exist" prefix=coprocess
time="Dec 18 21:26:17" level=error msg="Driver 'python' isn't loaded" prefix=coprocess

This is happening due to a missing python3-config executable on this image. The new dlpython component is using it in order to determine the version and paths to the Python library currently installed on the system.

The issue disappears when python3-dev package is installed on the container. This however adds significant size overhead to the image (TBD: how much exactly, the packages are about 80MB) as it also installs the development files (symbols, headers, etc.) for the Python lib + dependencies.

It might be better to have ability to force some library path for the gateway instead since we know that for each image.

Deprecate this repo

This repo is used by Hub to autobuild images but other base images are in the tyk repo

Use a Buddy pipeline to build the image.

Tyk container to accept configuration file path as environment variable

Hi, currently the tyk container expects the conf file to be at /opt/tyk-gateway/tyk.conf though the binary can accept a --conf command line switch. This restriction is imposed through hard wiring inside the entrypoint.sh. It'd be great if this can be considered from an environment variable, like the TYK_GW_SECRET currently does.

As seen in master branch (entrypoint.sh)

TYKCONF=/opt/tyk-gateway/tyk.conf

# for backwards compatibility if TYKSECRET is not empty, then set TYK_GW_SECRET to TYKSECRET
if [[ -n "${TYKSECRET}" ]]; then
  export TYK_GW_SECRET="${TYKSECRET}"
fi

Thank you

Tyk CE Headless cannot connect to redis using Docker-Compose and Helm

Reproducing

Docker Compose https://github.com/TykTechnologies/tyk-gateway-docker/blob/master/docker-compose.yml
Tyk Config https://github.com/TykTechnologies/tyk-gateway-docker/blob/master/tyk.standalone.conf

I cloned this repository and ran docker-compose up -d. I checked the logs and saw the following error

error="storage: Redis is either down or was not configured" prefix=main

This error is consistent with behavior observed with Tyk CE Headless on Kubernetes as well.

Logs

Tyk Gateway

2023-05-27 14:18:10 time="May 27 13:18:10" level=info msg="Tyk API Gateway 4.3.3" prefix=main
2023-05-27 14:18:10 time="May 27 13:18:10" level=warning msg="Insecure configuration allowed" config.allow_insecure_configs=true prefix=checkup
2023-05-27 14:18:10 time="May 27 13:18:10" level=error msg="Could not set version in versionStore" error="storage: Redis is either down or was not configured" prefix=main
2023-05-27 14:18:10 time="May 27 13:18:10" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
2023-05-27 14:18:10 time="May 27 13:18:10" level=info msg="Starting Poller" prefix=host-check-mgr
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="PIDFile location set to: /var/run/tyk/tyk-gateway.pid" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=warning msg="The control_api_port should be changed for production" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=error msg="Connection to Redis failed, reconnect in 10s" error="storage: Redis is either down or was not configured" prefix=pub-sub
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="--> [REDIS] Creating single-node client"
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="--> Standard listener (http)" port=":8080" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=warning msg="Starting HTTP server on:[::]:8080" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Initialising distributed rate limiter" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Starting gateway rate limiter notifications..."
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Tyk Gateway started (4.3.3)" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="--> Listening on address: (open interface)" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="--> Listening on port: 8080" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="--> PID: 1" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="--> [REDIS] Creating single-node client"
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="--> [REDIS] Creating single-node client"
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Loading policies" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Policies found (1 total):" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/client-mtls-api.json"
2023-05-27 14:18:11 time="May 27 13:18:11" level=error msg="error while closing file /opt/tyk-gateway/apps/client-mtls-api.json"
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/keyless-plugin-api.json"
2023-05-27 14:18:11 time="May 27 13:18:11" level=warning msg="Legacy path detected! Upgrade to extended."
2023-05-27 14:18:11 time="May 27 13:18:11" level=error msg="error while closing file /opt/tyk-gateway/apps/keyless-plugin-api.json"
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Loading API Specification from /opt/tyk-gateway/apps/protected-api.json"
2023-05-27 14:18:11 time="May 27 13:18:11" level=warning msg="Legacy path detected! Upgrade to extended."
2023-05-27 14:18:11 time="May 27 13:18:11" level=error msg="error while closing file /opt/tyk-gateway/apps/protected-api.json"
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Detected 3 APIs" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Loading API configurations." prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Tracking hostname" api_name="Tyk Test Keyless API" domain="(no host)" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Tracking hostname" api_name="Tyk Test API" domain="(no host)" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Tracking hostname" api_name=mTls domain="(no host)" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Initialising Tyk REST API Endpoints" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="API bind on custom port:0" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Checking security policy: Open" api_id=keyless api_name="Tyk Test Keyless API" org_id=default
2023-05-27 14:18:11 time="May 27 13:18:11" level=error msg="Unsupported driver ''" prefix=coprocess
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="API Loaded" api_id=keyless api_name="Tyk Test Keyless API" org_id=default prefix=gateway server_name=-- user_id=-- user_ip=--
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="API bind on custom port:0" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Checking security policy: Token" api_id=1 api_name="Tyk Test API" org_id=default
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="API Loaded" api_id=1 api_name="Tyk Test API" org_id=default prefix=gateway server_name=-- user_id=-- user_ip=--
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="API bind on custom port:0" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Checking security policy: Open" api_id=4a77d8bfe76f41ad7ae5875b2259df3f api_name=mTls org_id=5e9d9544a1dcd60001d0ed20
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="API Loaded" api_id=4a77d8bfe76f41ad7ae5875b2259df3f api_name=mTls org_id=5e9d9544a1dcd60001d0ed20 prefix=gateway server_name=-- user_id=-- user_ip=--
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Loading uptime tests..." prefix=host-check-mgr
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="Initialised API Definitions" prefix=main
2023-05-27 14:18:11 time="May 27 13:18:11" level=info msg="API reload complete" prefix=main

Redis

2023-05-27 14:18:10 1:C 27 May 2023 13:18:10.308 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2023-05-27 14:18:10 1:C 27 May 2023 13:18:10.308 # Redis version=6.2.7, bits=64, commit=00000000, modified=0, pid=1, just started
2023-05-27 14:18:10 1:C 27 May 2023 13:18:10.308 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
2023-05-27 14:18:10 1:M 27 May 2023 13:18:10.308 * monotonic clock: POSIX clock_gettime
2023-05-27 14:18:10 1:M 27 May 2023 13:18:10.309 # A key '__redis__compare_helper' was added to Lua globals which is not on the globals allow list nor listed on the deny list.
2023-05-27 14:18:10 1:M 27 May 2023 13:18:10.309 * Running mode=standalone, port=6379.
2023-05-27 14:18:10 1:M 27 May 2023 13:18:10.309 # Server initialized
2023-05-27 14:18:10 1:M 27 May 2023 13:18:10.311 * Ready to accept connections
2023-05-27 14:23:11 1:M 27 May 2023 13:23:11.102 * 100 changes in 300 seconds. Saving...
2023-05-27 14:23:11 1:M 27 May 2023 13:23:11.103 * Background saving started by pid 16
2023-05-27 14:23:11 16:C 27 May 2023 13:23:11.110 * DB saved on disk
2023-05-27 14:23:11 16:C 27 May 2023 13:23:11.111 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 14:23:11 1:M 27 May 2023 13:23:11.203 * Background saving terminated with success
2023-05-27 14:28:12 1:M 27 May 2023 13:28:12.069 * 100 changes in 300 seconds. Saving...
2023-05-27 14:28:12 1:M 27 May 2023 13:28:12.072 * Background saving started by pid 17
2023-05-27 14:28:12 17:C 27 May 2023 13:28:12.079 * DB saved on disk
2023-05-27 14:28:12 17:C 27 May 2023 13:28:12.080 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 14:28:12 1:M 27 May 2023 13:28:12.176 * Background saving terminated with success
2023-05-27 14:33:13 1:M 27 May 2023 13:33:13.023 * 100 changes in 300 seconds. Saving...
2023-05-27 14:33:13 1:M 27 May 2023 13:33:13.026 * Background saving started by pid 18
2023-05-27 14:33:13 18:C 27 May 2023 13:33:13.036 * DB saved on disk
2023-05-27 14:33:13 18:C 27 May 2023 13:33:13.036 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 14:33:13 1:M 27 May 2023 13:33:13.128 * Background saving terminated with success
2023-05-27 14:38:14 1:M 27 May 2023 13:38:14.090 * 100 changes in 300 seconds. Saving...
2023-05-27 14:38:14 1:M 27 May 2023 13:38:14.093 * Background saving started by pid 19
2023-05-27 14:38:14 19:C 27 May 2023 13:38:14.104 * DB saved on disk
2023-05-27 14:38:14 19:C 27 May 2023 13:38:14.105 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 14:38:14 1:M 27 May 2023 13:38:14.194 * Background saving terminated with success
2023-05-27 14:43:15 1:M 27 May 2023 13:43:15.028 * 100 changes in 300 seconds. Saving...
2023-05-27 14:43:15 1:M 27 May 2023 13:43:15.032 * Background saving started by pid 20
2023-05-27 14:43:15 20:C 27 May 2023 13:43:15.046 * DB saved on disk
2023-05-27 14:43:15 20:C 27 May 2023 13:43:15.047 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 14:43:15 1:M 27 May 2023 13:43:15.132 * Background saving terminated with success
2023-05-27 14:48:16 1:M 27 May 2023 13:48:16.057 * 100 changes in 300 seconds. Saving...
2023-05-27 14:48:16 1:M 27 May 2023 13:48:16.058 * Background saving started by pid 21
2023-05-27 14:48:16 21:C 27 May 2023 13:48:16.065 * DB saved on disk
2023-05-27 14:48:16 21:C 27 May 2023 13:48:16.065 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 14:48:16 1:M 27 May 2023 13:48:16.160 * Background saving terminated with success
2023-05-27 14:53:17 1:M 27 May 2023 13:53:17.019 * 100 changes in 300 seconds. Saving...
2023-05-27 14:53:17 1:M 27 May 2023 13:53:17.023 * Background saving started by pid 22
2023-05-27 14:53:17 22:C 27 May 2023 13:53:17.033 * DB saved on disk
2023-05-27 14:53:17 22:C 27 May 2023 13:53:17.034 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 14:53:17 1:M 27 May 2023 13:53:17.124 * Background saving terminated with success
2023-05-27 15:01:32 1:M 27 May 2023 14:01:32.856 * 100 changes in 300 seconds. Saving...
2023-05-27 15:01:32 1:M 27 May 2023 14:01:32.863 * Background saving started by pid 23
2023-05-27 15:01:32 23:C 27 May 2023 14:01:32.873 * DB saved on disk
2023-05-27 15:01:32 23:C 27 May 2023 14:01:32.873 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 15:01:32 1:M 27 May 2023 14:01:32.964 * Background saving terminated with success
2023-05-27 15:06:33 1:M 27 May 2023 14:06:33.058 * 100 changes in 300 seconds. Saving...
2023-05-27 15:06:33 1:M 27 May 2023 14:06:33.062 * Background saving started by pid 31
2023-05-27 15:06:33 31:C 27 May 2023 14:06:33.072 * DB saved on disk
2023-05-27 15:06:33 31:C 27 May 2023 14:06:33.072 * RDB: 0 MB of memory used by copy-on-write
2023-05-27 15:06:33 1:M 27 May 2023 14:06:33.163 * Background saving terminated with success

Any help will be appreciated.

Removal of components required for hybrid docker containers

As a separate docker image is available for hybrid gateway , can this one be updated to include only components required for standalone?

VOLUME used in Dockerfile prevents downstream modification of /opt/tyk-gateway/

Docker does not allow changes to a VOLUME'd directory after the VOLUME command in a Dockerfile.
This includes other Dockerfiles that inherit upon a base dockerfile.

This dockerfile has a VOLUME mount of /opt/tyk-gateway/

But this causes problems for me (I'm adding other files into /opt/tyk-gateway/ in my Dockerfile)

Links to relevant discussion on docker github page:
moby/moby#3639

optimised defaults for tyk.conf

We should set optimised defaults for tyk configuration file.

Changes should include

close_connections: false
max_idle_connections_per_host: 500

Reduce docker image size

We need to do some more work in order to reduce the docker image size for the gateway.

At the moment, it stands at 116mb

We currently ship with 3 tyk binaries. Each binary is approx 32mb - py, grpc & lua. These should be separated into separate builds. e.g. tyk-gateway:py-2.7.3, tyk-gateway:grpc-2.7.3 or tyk-gateway:lua-2.7.3
Currently we ship with base image debian:jessie-slim (30mb). We could look into using alpine which is a 2mb base image. If we require glibc https://github.com/sgerrand/alpine-pkg-glibc or if we may be better off using musl binaries.
Strip debug info from compiled binaries using linker flags -s -w. go build -ldflags="-s -w" . which shaves off circa 10mb from each binary.
Investigate using UPX to further pack the binaries - bringing binary size down to approx 5mb. https://github.com/upx/upx

Can't add `templates` directory

Attempting to create a POC with Tyk and mutate a request body to add a field before it is forwarded to the proxied URL - but I am getting an error from docker that isnt entirely clear when attempting to add templates to the root and as a volume in docker-compose.yml

My docker-compose file:

version: '3.3'
services:
  tyk-gateway:
    image: docker.tyk.io/tyk-gateway/tyk-gateway:v4.1.0
    ports:
      - 8080:8080
    networks:
      - tyk
    volumes:
      - ./tyk.standalone.conf:/opt/tyk-gateway/tyk.conf
      - ./apps:/opt/tyk-gateway/apps
      - ./middleware:/opt/tyk-gateway/middleware
      - ./templates:/opt/tyk-gateway/templates
      - ./certs:/opt/tyk-gateway/certs
    environment:
      - TYK_GW_SECRET=foo
    depends_on:
      - tyk-redis
  tyk-redis:
    image: redis:6.2.7-alpine
    networks:
      - tyk
    ports:
      - 6379:6379

networks:
  tyk:

./templates/add_app_id.tmpl:

{
    "to": "{{.to}}",
    "from": "{{.from}}",
    "text": "{{.text}}",
    "applicationId" : "abc123",  /* attempting to inject this string into every request body */
}

./apps/keyless-plugin-api.json:

{
  "name": "Tyk Test Keyless API",
  "api_id": "keyless",
  "org_id": "default",
  "definition": {
    "location": "header",
    "key": "version"
  },
  "use_keyless": true,
  "version_data": {
    "not_versioned": true,
    "versions": {
      "Default": {
        "name": "Default",
        "use_extended_paths": true,
        "extended_paths": {
          "transform": [
            {
              "path": "/keyless-test",
              "method": "POST",
              "template_data": {
                "template_mode": "file",
                "template_source": "./templates/add_app_id.tmpl",
                "input_type": "json",
                "enable_session": true
              }
            }
          ]
        }
      }
    }
  },
  "custom_middleware": {
    "pre": [
      {
        "name": "testJSVMData",
        "path": "./middleware/injectHeader.js",
        "require_session": false,
        "raw_body_only": false
      }
    ]
  },
  "driver": "otto",
  "proxy": {
    "listen_path": "/keyless-test",
    "target_url": "https://myService.m.pipedream.net/api/v1",
    "strip_listen_path": false
  }
}

Error from docker when running docker compose up -d:

time="Dec 19 21:24:03" level=info msg="Tyk API Gateway 4.1.0" prefix=main
time="Dec 19 21:24:03" level=warning msg="Insecure configuration allowed" config.allow_insecure_configs=true prefix=checkup
time="Dec 19 21:24:03" level=error msg="Could not set version in versionStore" error="storage: Redis is either down or was not configured" prefix=main
time="Dec 19 21:24:03" level=error msg="cannot set key in pollerCacheKey" error="storage: Redis is either down or was not configured"
time="Dec 19 21:24:03" level=info msg="Starting Poller" prefix=host-check-mgr
panic: html/template: pattern matches no files: `/opt/tyk-gateway/templates/error*`
goroutine 1 [running]:
html/template.Must(...)
/usr/local/go/src/html/template/template.go:372
github.com/TykTechnologies/tyk/gateway.(*Gateway).setupGlobals(0xc00012b000)
/go/src/github.com/TykTechnologies/tyk/gateway/server.go:374 +0x20c9
github.com/TykTechnologies/tyk/gateway.(*Gateway).initialiseSystem(0xc00012b000, 0x37c37275f7444d85, 0xc00013d620)
/go/src/github.com/TykTechnologies/tyk/gateway/server.go:1169 +0x44c
github.com/TykTechnologies/tyk/gateway.Start()
/go/src/github.com/TykTechnologies/tyk/gateway/server.go:1469 +0x296
main.main()
/__w/tyk/tyk/main.go:8 +0x25

tyktechnologies / tyk-gateway-docker Goto Github PK

tyk-gateway-docker's People

Stargazers

Watchers

Forkers

tyk-gateway-docker's Issues

Reproducing

Logs

Tyk Gateway

Redis

Recommend Projects

Recommend Topics

Recommend Org