Comments (4)
I think you need to provide more details on what you did. You're using a container, did you mount the logs into it?
You're also using the debug container so can shell in and check.
from fluent-bit.
Yes i using two containers(demo application, fluentbit), and mount log file path in both containers
how can i using debug container check, Where can I find documents for reference?
from fluent-bit.
https://kubernetes.io/docs/tasks/debug/debug-application/get-shell-running-container/
from fluent-bit.
Thank you for your response, but unfortunately, I am unable to use that method.
I am using gcp Cloud Run deploy, and Cloud run is a fully managed service and does not allow exec into the container.
below is my test config
Sample app setting
Sample app dockerfile
FROM ubuntu:latest
# Install Python 3
RUN apt-get update && \
apt-get install -y python3 && \
rm -rf /var/lib/apt/lists/*
# Expose port 8080
EXPOSE 8080
# Create a directory for the file
RUN mkdir -p /usr/src/app
RUN mkdir -p /logs
# Copy your script into the container
COPY entrypoint.sh /usr/src/app/
# Set the working directory
WORKDIR /usr/src/app
# Make the script executable
RUN chmod +x entrypoint.sh
# Run the entrypoint script
CMD ["./entrypoint.sh"]
Sample app entrypoint.sh
#!/bin/bash
# Start a simple Python HTTP server on port 8080 and log requests to a file
python3 -u -m http.server 8080 2> /logs/access.log
Sample app build and push to gcp project
docker build -t gcr.io/{project}/demo-app:v1
docker push gcr.io/{project}/demo-app:v1
fluentbit setting
fluentbit dockerfile
FROM fluent/fluent-bit:2.2.0-debug
RUN mkdir -p /logs
COPY fluent-bit.conf /fluent-bit/etc/fluent-bit.conf
fluentbit conf
[Service]
log_level trace
flush 10
[INPUT]
Name tail
Tag samplapp
Path /logs/*.log
Refresh_Interval 90
Read_from_Head true
[OUTPUT]
Name stdout
Match samplapp
fluentbit build and push to gcp project
docker build -t gcr.io/{project}/fluentbit-test:v1
docker push gcr.io/{project}/fluentbit-test:v1
gcp cloud run yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: demo-log
labels:
cloud.googleapis.com/location: asia-east1
annotations:
run.googleapis.com/launch-stage: BETA
run.googleapis.com/ingress: internal
run.googleapis.com/ingress-status: internal
run.googleapis.com/minScale: '1'
run.googleapis.com/max-surge: '0'
spec:
template:
spec:
containers:
- name: demo-1
image: gcr.io/{project}/demo-app:v1
ports:
- name: http1
containerPort: 8080
resources:
limits:
cpu: 1000m
memory: 512Mi
volumeMounts:
- name: in-memory-1
mountPath: /logs
- name: demo-2
image: gcr.io/{project}/fluentbit-test:v1
resources:
limits:
cpu: 1000m
memory: 512Mi
volumeMounts:
- name: in-memory-1
mountPath: /logs
volumes:
- name: in-memory-1
emptyDir:
medium: Memory
sizeLimit: 128Mi
gcp cloud run deploy service
gcloud run services replace gcloud-demo.yaml
and log will appear
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [error] [/src/fluent-bit/plugins/in_tail/tail_file.c:1620 errno=2] No such file or directory
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [error] [/src/fluent-bit/plugins/in_tail/tail_file.c:1694 errno=2] No such file or directory
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [error] [/src/fluent-bit/plugins/in_tail/tail_file.c:1009 errno=2] No such file or directory
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [debug] [input:tail:tail.0] scan_blog add(): dismissed: /logs/access.log, inode 2
2024-05-03 11:08:53.922 HKT
[2024/05/03 03:08:53] [debug] [input:tail:tail.0] 0 new files found on path '/logs/*.log'
from fluent-bit.
Related Issues (20)
- Syslog output doesn't send data to Qradar HOT 4
- stackdriver output plugin broken on arm32v7 docker images since v3.0.0 HOT 2
- Crash Output + Forward + TLS + ARM HOT 2
- Multiline is processing lines out of order
- Need Grok parsing support in fluentbit HOT 3
- Amazon S3: Mismatch when reading HTTP header from GCS
- Configuration reload or SIGINT does not interrupt flushing output plugin with retry_limit=False, potential loops forever
- Fluent Bit occasionally corrupts/truncates log entries when processing multiple log files. HOT 1
- IPv6 support for the AWS Metadata filter
- Fluentbit OpenTelemetry Plugins drops some fields from metrics
- Provide docker images that can run on asahi-linux HOT 1
- Systemd_filter not working HOT 3
- Stackdriver Output When not on GCP/GCE
- [out_loki] A lot of warning 'Tenant ID is overwritten A -> B' if tenant_id_key is used HOT 2
- Binary data over tcp input
- tail input stops reading data and gets stuck forever unless deleting SQLite DB
- Connection metadata annotating logs.
- Fluent-bit version 1.9.6 [ Error] timerfd: Too many open files, errno=24 at /__w/1/s/external/fluent-bit/lib/monkey/mk_core/mk_event_epoll.c:208 HOT 3
- Error Encountered During Hot Reload in Fluent Bit 3.0.2 HOT 3
- Fluent Bit silently ignores wrong parameter names in the service section
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-bit.