Comments (12)
I am seeing the error below: is enableMetricData deprecated?
Yes that field has been removed long time ago. CustomNamespace jobs are also deprecated, that's why I was suggesting to swithc to a Static job.
from yet-another-cloudwatch-exporter.
@bc-ibiks what version of YACE are you running? Usage of custom namespaces is deprecated and might be removed or changed in future versions.
I think the error might be due to the period and length you're using, which are shorter than the scrape interval. If you definitely need to keep the value of 60
, I'd suggest you run yace with -scraping-interval=60
.
Please replace your custom namespace job with a static job as well.
from yet-another-cloudwatch-exporter.
@cristiangreco thanks for the update
v0.43.0-alpha
I will make changes as you suggested. will let you know how this goes
cheers
from yet-another-cloudwatch-exporter.
@bc-ibiks great, please let me know 👍
from yet-another-cloudwatch-exporter.
@cristiangreco I have move my config to static job
The errors have gone but no metrics captured. I can see these data in cloudwatch
- name: ani
namespace: ANI
regions:- us-west-2
- us-east-2
roles: - roleArn: arn:aws:iam::<>:role/prometheus
enableMetricData: true
metrics: - name: PROCESSED_BATCHES
statistics:- Sum
- Average
period: 300
length: 300
nilToZero: true
- name: PROCESSED_BATCHES_ITEMS
statistics:- Sum
- Average
period: 300
length: 300
nilToZero: true
from yet-another-cloudwatch-exporter.
@bc-ibiks what version of YACE are you using? Can you make sure it's the latest please?
Also, can you run with -debug
and paste the relevant output here? (just make sure it contains no auth details)
from yet-another-cloudwatch-exporter.
I have updated the latest version of YACE - v0.51.0
I am seeing the error below: is enableMetricData deprecated?
{"level":"info","msg":"Parsing config","time":"2023-05-08T13:59:27Z"}
{"err":"line 32: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 41: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 56: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 118: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 214: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 320: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 418: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 452: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 476: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
I removed enableMetricData = true from my config
I am looking through the logs with -debug switched on
saw the below for the metric in the custom namespace
-----------------------------------------------------
2023/05/08 13:41:09 <GetMetricStatisticsResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">
<GetMetricStatisticsResult>
<Datapoints/>
<Label>PROCESSED_BATCHES</Label>
</GetMetricStatisticsResult>
<ResponseMetadata>
<RequestId>2761db6e-474c-46cc-b5d7-55382768590d</RequestId>
</ResponseMetadata>
</GetMetricStatisticsResponse>
{"account":"244731367290","arn":"arn:aws:iam::1234567890:role/prometheus","level":"debug","msg":"GetMetricStatistics","output":"{\n Label: \"PROCESSED_BATCHES\"\n}","region":"us-east-2","static_job_name":"ani","time":"2023-05-08T13:41:09Z"}
2023/05/08 13:41:09 DEBUG: Request monitoring/GetMetricStatistics Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: monitoring.us-east-2.amazonaws.com
User-Agent: aws-sdk-go/1.44.249 (go1.20.3; linux; amd64)
Content-Length: 268
Authorization: [redacted]
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20230508T134109Z
X-Amz-Security-Token: [redacted]
Accept-Encoding: gzip
Action=GetMetricStatistics&Dimensions.member.1.Name=Protocol&Dimensions.member.1.Value=MQTT&EndTime=2023-05-08T13%3A41%3A09.004Z&MetricName=Ping.Success&Namespace=AWS%2FIoT&Period=60&StartTime=2023-05-08T13%3A31%3A09.004Z&Statistics.member.1=Average&Version=2010-08-01
-----------------------------------------------------
2023/05/08 13:41:09 DEBUG: Response monitoring/GetMetricStatistics Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 338
Content-Type: text/xml
Date: Mon, 08 May 2023 13:41:09 GMT
X-Amzn-Requestid: eb33b780-2d38-4c53-b5ca-48ef25958258
-----------------------------------------------------
2023/05/08 13:41:09 <GetMetricStatisticsResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">
<GetMetricStatisticsResult>
<Datapoints/>
<Label>TimeToFirstByte</Label>
</GetMetricStatisticsResult>
<ResponseMetadata>
<RequestId>eb33b780-2d38-4c53-b5ca-48ef25958258</RequestId>
</ResponseMetadata>
</GetMetricStatisticsResponse>
from yet-another-cloudwatch-exporter.
@bc-ibiks it seems the response from AWS GetMetricStatistics does not contain any datapoint.
You mentioned that you can see metrics in CloudWatch. What happens if you run the AWS cli aws cloudwatch get-metric-statistics
? (you can find an example in YACE debug logs: YACE emits a log line like CLI helper - ...
which tells you how to reproduce with aws cli tool what YACE is requesting).
from yet-another-cloudwatch-exporter.
I will let you know what I find out - thanks
from yet-another-cloudwatch-exporter.
I have YACE running in >60 accounts and ran into this exact same issue. What solved it for me was adding the delay
attribute.
period: 60
length: 600
delay: 60
I also found that the addCloudwatchTimestamp
attribute helped with this (but did not completely fix), but it did also align the metrics with native prometheus metrics on the Grafana dashboard. Without this attribute the metrics coming from CW/YACE were misaligned.
Also, I ran into another issue with "duplicate" metrics when using nilToZero
, and I didn't find any real benefit to using that attribute. I would suggest playing with removing that attribute and seeing if you still get what you expect/want.
from yet-another-cloudwatch-exporter.
thanks! @mhester-nutrien. I will give this a test and let you know
from yet-another-cloudwatch-exporter.
@cristiangreco and @mhester-nutrien I was able to retrieve the custom metric using prometheus cloudwatch exporter. (which is a tad annoying to use)
I still wondering why I can pull this with YACE. I am able to pull the metrics using aws cli
Seems I am missing some settings in my config
from yet-another-cloudwatch-exporter.
Related Issues (20)
- [BUG] metrics name depends on job `type` configuration
- [BUG] Invalid timestamps on metrics with addCloudwatchTimestamp after upgrading from 0.53.0 to 0.54.1 HOT 2
- [BUG] ECS ContainerInsights metrics not pulling for `ClusterName,ServiceName` HOT 3
- [BUG] Route53 Health checks missing tag labels
- Promote `max-dimensions-associator` feature flag to default
- Promote `list-metrics-callback` feature flag to default
- [BUG] Resource tagging API does not support `mwaa` service
- [QUESTION] Cloudwatch alb metrics with timestamps in the future? HOT 1
- [BUG] inconsistent mix of namespace and alias in config files breaks `exportedTagsOnMetrics`
- [BUG] 4x duplicated metrics when using discovery of exportedTagsOnMetrics HOT 1
- [BUG] SageMaker skipped metrics as resource not matched HOT 1
- [FEATURE] AWS IPAM support HOT 1
- [BUG] Resources without tags not getting picked up HOT 3
- Unable to scrape metrics from cloudwatch
- [FEATURE] missing assets (rpm) for 0.55 HOT 1
- [BUG] <title> -enable-feature=aws-sdk-v2, have error "Missing Region"
- [BUG] S3 Replication Cross Account Cross-Region Replication (CRR) Metrics ignored by discovery HOT 1
- KafkaConnect Metrics Streaming
- [FEATURE] Missing all assets except the source code for version 0.55.0 HOT 2
- [BUG] VPN metrics missing dimensions
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yet-another-cloudwatch-exporter.