vozlt / nginx-module-sts Goto Github PK

can't use server_traffic_status_dump

Hello, I'm curious how this indicator "nginx _sts_upstream _connects_total" is calculated when nginx proxy udp. As far as I know, udp does not have the concept of "connection"

Hi,

sorry for this, just new on nginx. when I tried to do "make" I got this error. Please help, what did I do wrong?

In file included from /tmp/nginx-modules/nginx-module-stream-sts//src/ngx_stream_server_traffic_status_module.c:9:0:
/tmp/nginx-modules/nginx-module-stream-sts//src/ngx_stream_server_traffic_status_module.h:13:24: fatal error: ngx_stream.h: No such file or directory
 #include <ngx_stream.h>

hi
"nginx_sts_upstream_connects_total", I rely on this indicator to count the cps of nginx. After testing, it is found that this counter does not increase by 1 when the tcp handshake succeeds, but only after the tcp wave ends. Can you change it? I don't think it makes sense to increase this counter after the connection is disconnected, because the TCP connection usually lasts for a long time.

test code:
tcp_server.py:

import socket
import threading
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('0.0.0.0', 4444))

s.listen(5)
print('Waiting for connection...')

def tcplink(sock, addr):
    print('Accept new connection from %s:%s...' % addr)
    while True:
        data = sock.recv(1024)
        if not data or data.decode('utf-8') == 'exit':
            break
        sock.send(('Hello, %s!' % data.decode('utf-8')).encode('utf-8'))
    sock.close()
    print('Connection from %s:%s closed.' % addr)

while True:
    sock, addr = s.accept()
    t = threading.Thread(target=tcplink, args=(sock, addr))
    t.start()

nginx conf:

daemon off;
worker_processes 2;
error_log /opt/openresty/logs/error.log error;
pid /opt/openresty/logs/nginx.pid;

events {
	worker_connections 50000;
}

http {
	server_tokens off;
	stream_server_traffic_status_zone;
	vhost_traffic_status_zone shared:vhost_traffic_status:32m;

	server {
		listen [::]:9103;
		listen 9103;

		location /metrics/4 {
			allow all;
			stream_server_traffic_status_display;
			stream_server_traffic_status_display_format prometheus;
		}

		location /metrics/7 {
			allow all;
			vhost_traffic_status_display;
			vhost_traffic_status_display_format prometheus;
		}
	}

}
stream {
	server_traffic_status_zone;
	upstream vs-qb4x1fv1vvxrsw {
		server 10.0.1.2:4444	weight=1 max_fails=1 fail_timeout=10s;
	}

	server {
		listen 5001 so_keepalive=60s::;
		proxy_connect_timeout 60s;	# dail timeout
		proxy_timeout 60s;
		proxy_pass vs-qb4x1fv1vvxrsw;	# backend : vsID
	}


}

tcp_client.py:

import socket
import sys
import time

ip = sys.argv[1]
port = int(sys.argv[2])

socket_l = []

for i in range(1000):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((ip, port))
    socket_l.append(s)
    s.send(b"data")
    print(i, s.recv(1024).decode('utf-8'))
    time.sleep(1)

for s in socket_l:
    s.close()

the number of connections cannot be counted in the beginning of a period of time：

connections can be seen in 60 seconds

This is because I set proxy_ timeout 60s;

To get the graphical result, do I need to use third party monitoring tool?

Is there any way to output only specific columns?

stream {
geoip_country /usr/share/GeoIP/GeoIP.dat;
server_traffic_status_zone;
server_traffic_status_filter_by_set_key $geoip_country_code country::*;
}

Please help me, why I got that error?

@vozlt please make a release version for this project.

Description

If one is using the nginx-stream-upsync-module to dynamically update a upstream's backend servers the nginx-module-sts will never recognize the new backend to pull stats from. Stats are properly calculated for the initially defined backend server in servers.conf however any update to the file from the upsync module will not be registered without a nginx restart.

Config File

nginx.conf

events {
    worker_connections  1024;
    multi_accept on;
    use epoll;
}

http {
    include       /etc/nginx/mime.types;

    default_type  application/octet-stream;
    stream_server_traffic_status_zone;

    server {
        server_name status;
        listen 80;
        location /status {
            stream_server_traffic_status_display;
            stream_server_traffic_status_display_format html;
        }
    }
}

stream {
    server_traffic_status_zone;
    upstream backend {
        upsync etcd:2379/v2/keys/upstreams/backend upsync_timeout=6m upsync_interval=500ms upsync_type=etcd strong_dependency=off;
        upsync_dump_path /etc/nginx/servers.conf;
        include /etc/nginx/servers.conf;
    }
    server {
        listen 443;
        proxy_connect_timeout 1s;
        proxy_timeout 3s;
        proxy_pass backend;
    }
}

servers.conf

server 192.168.1.10:443 max_fails=3 fail_timeout=10s;

Just for fun I try to compile nginx with you module in VS2017 RTM (want to debug some piece of code), but cl say me C1091 :-(

Problem in NGX_HTTP_STREAM_SERVER_TRAFFIC_STATUS_HTML_DATA - defined string too long.
I think you can review string embedding mechanism (if you want).

Generator script (without perl deps):

#!/bin/bash

if [[ $# -ne 2 ]]; then
    echo "Usage: $0 FILENAME VARNAME"
    exit 1
fi
file=$1
cname=$2

if [[ ! -f "$file" ]]; then
    echo "File not found: $file"
    exit 1
fi

echo "static unsigned char $cname[] = {"
hexdump -v -e '" " 16/1 "  0x%02x, " "\n"' $file | \
   sed -e '$s/0x  ,//g'
echo "};"

Dont forget double-check sizeof(xxx) macros in sources if apply...

Looking on v0.1.1...master I think that it would be good to flush those commits and release new version.
Doable? 🤔