Comments (1)
This is as close as I got. Turns out web log collector already allows for wildcards.
Nginx ingress helm values:
USER-SUPPLIED VALUES:
controller:
config:
log-format-upstream: "$host $remote_addr - \"$request\" $status $bytes_sent \"$http_referer\" \"$http_user_agent\" $request_length $request_time $upstream_response_time $upstream_addr"
Netdata helm values
child:
extraVolumes:
- name: containerlogs
hostPath:
path: /var/log/pods
type: Directory
extraVolumeMounts:
- name: containerlogs
mountPath: /var/log/pods
readOnly: true
configs:
go.d-web_log:
enabled: true
path: /etc/netdata/go.d/web_log.conf
data: |
jobs:
- name: nginx-ingress
path: /var/log/pods/*ingress-nginx-controller*/controller/*.log
log_type: regexp
regexp_config:
pattern: (?P<time>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{9}-\d{2}:\d{2}) stdout F (?P<host>[^ ]+) (?P<remote_addr>[^ ]+) - "(?P<request>[^"]+)" (?P<status>\d+) (?P<bytes_sent>\d+) "(?P<http_referer>[^"]+)" "(?P<http_user_agent>[^"]+)" (?P<request_length>\d+) (?P<request_time>[\d\.]+) (?P<upstream_response_time>[\d\.]+) (?P<upstream_addr>[^ ]+)
I tried using the CSV log_type but I couldn't get it to match. The error message just said that it couldn't match the line.
For reference, this is a line
'2024-03-24T13:17:51.265395971-03:00 stdout F netdata.test 10.42.0.76 - "GET /api/v2/ HTTP/1.1" 200 4039 "http://netdata.test/spaces/netdata-parent/rooms/local/nodes/cabc8b98-b969-11ee-9a5d-52540049cf1a" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" 1008 0.004 0.004 10.42.0.40:19999'
And this was my pattern:
$time_local stdout F $host $remote_addr - "$request" $status $bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time $upstream_response_time $upstream_addr
The docs for web_log are pretty confusing. The "jobs" are defined here: https://learn.netdata.cloud/docs/developer-and-contributor-corner/monitor-nginx-or-apache-web-server-log-files-with-netdata , but the configuration here https://learn.netdata.cloud/docs/collecting-metrics/web-servers-and-web-proxies/web-server-log-files#configuration doesn't mention that "jobs" anywhere.
Remaining issue is that I'd like to have bandwidth usage per vhost. Seems this can only be done if I create url_patterns for all my vhosts?
from netdata.
Related Issues (20)
- [Bug]: Timeout while request K3S cluster HOT 1
- Fix MAC Installation and Claiming using the Kickstart script HOT 1
- [Feat]: one of the most important alerts of one system are "silenced" by default!!!! HOT 3
- [Feat]: disable query log in go.d.plugin mysql collector HOT 2
- [Bug]: Netdata agent seems to lose all connection and vanishes from Netdata Cloud HOT 3
- [Bug]: Agent Registration via Key Fails HOT 7
- [Bug]: API v1 format array returns null values HOT 3
- [Bug]: Netdata web_log doesn't automatically match PURGE requests HOT 2
- [Bug]: service.netdata constantly segfault at 7fda6e11d7d8 HOT 10
- [Bug]: CGROUP: cannot read directory HOT 2
- [Bug]: HOT 4
- [Bug]: homebrew build: Mac CMakeLists.txt assumes `brew --prefix` command HOT 3
- [Bug]: freeipmi plugin stuck in defunct state HOT 3
- [Bug]: Building fails on Ubuntu 18.04 since 1.45.0 HOT 3
- [Bug]: Windows Nodes not showing up on dashboard HOT 4
- [Feat]: Connection from an ipv6-only host HOT 1
- [Bug]: Alert configuration host labels simple pattern negative matches don't work.
- [Bug]: unbound module failed to start HOT 2
- [Bug]: Cannot start Netdata when binding to a privileged port (80) HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from netdata.