Giter Club home page Giter Club logo

Comments (12)

cristiangreco avatar cristiangreco commented on June 27, 2024 1

I am seeing the error below: is enableMetricData deprecated?

Yes that field has been removed long time ago. CustomNamespace jobs are also deprecated, that's why I was suggesting to swithc to a Static job.

from yet-another-cloudwatch-exporter.

cristiangreco avatar cristiangreco commented on June 27, 2024

@bc-ibiks what version of YACE are you running? Usage of custom namespaces is deprecated and might be removed or changed in future versions.

I think the error might be due to the period and length you're using, which are shorter than the scrape interval. If you definitely need to keep the value of 60, I'd suggest you run yace with -scraping-interval=60.

Please replace your custom namespace job with a static job as well.

from yet-another-cloudwatch-exporter.

bc-ibiks avatar bc-ibiks commented on June 27, 2024

@cristiangreco thanks for the update
v0.43.0-alpha
I will make changes as you suggested. will let you know how this goes
cheers

from yet-another-cloudwatch-exporter.

cristiangreco avatar cristiangreco commented on June 27, 2024

@bc-ibiks great, please let me know 👍

from yet-another-cloudwatch-exporter.

bc-ibiks avatar bc-ibiks commented on June 27, 2024

@cristiangreco I have move my config to static job
The errors have gone but no metrics captured. I can see these data in cloudwatch

  • name: ani
    namespace: ANI
    regions:
    • us-west-2
    • us-east-2
      roles:
    • roleArn: arn:aws:iam::<>:role/prometheus
      enableMetricData: true
      metrics:
    • name: PROCESSED_BATCHES
      statistics:
      • Sum
      • Average
        period: 300
        length: 300
        nilToZero: true
    • name: PROCESSED_BATCHES_ITEMS
      statistics:
      • Sum
      • Average
        period: 300
        length: 300
        nilToZero: true

from yet-another-cloudwatch-exporter.

cristiangreco avatar cristiangreco commented on June 27, 2024

@bc-ibiks what version of YACE are you using? Can you make sure it's the latest please?

Also, can you run with -debug and paste the relevant output here? (just make sure it contains no auth details)

from yet-another-cloudwatch-exporter.

bc-ibiks avatar bc-ibiks commented on June 27, 2024

I have updated the latest version of YACE - v0.51.0

I am seeing the error below: is enableMetricData deprecated?

{"level":"info","msg":"Parsing config","time":"2023-05-08T13:59:27Z"}
{"err":"line 32: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 41: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 56: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 118: field enableMetricData not found in type config.Job","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 214: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 320: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 418: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 452: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}
{"err":"line 476: field enableMetricData not found in type config.Static","level":"warning","msg":"config file syntax error","time":"2023-05-08T13:59:27Z"}

I removed enableMetricData = true from my config

I am looking through the logs with -debug switched on

saw the below for the metric in the custom namespace

-----------------------------------------------------
2023/05/08 13:41:09 <GetMetricStatisticsResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">
  <GetMetricStatisticsResult>
    <Datapoints/>
    <Label>PROCESSED_BATCHES</Label>
  </GetMetricStatisticsResult>
  <ResponseMetadata>
    <RequestId>2761db6e-474c-46cc-b5d7-55382768590d</RequestId>
  </ResponseMetadata>
</GetMetricStatisticsResponse>

{"account":"244731367290","arn":"arn:aws:iam::1234567890:role/prometheus","level":"debug","msg":"GetMetricStatistics","output":"{\n  Label: \"PROCESSED_BATCHES\"\n}","region":"us-east-2","static_job_name":"ani","time":"2023-05-08T13:41:09Z"}
2023/05/08 13:41:09 DEBUG: Request monitoring/GetMetricStatistics Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: monitoring.us-east-2.amazonaws.com
User-Agent: aws-sdk-go/1.44.249 (go1.20.3; linux; amd64)
Content-Length: 268
Authorization: [redacted]
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20230508T134109Z
X-Amz-Security-Token: [redacted]
Accept-Encoding: gzip

Action=GetMetricStatistics&Dimensions.member.1.Name=Protocol&Dimensions.member.1.Value=MQTT&EndTime=2023-05-08T13%3A41%3A09.004Z&MetricName=Ping.Success&Namespace=AWS%2FIoT&Period=60&StartTime=2023-05-08T13%3A31%3A09.004Z&Statistics.member.1=Average&Version=2010-08-01
-----------------------------------------------------
2023/05/08 13:41:09 DEBUG: Response monitoring/GetMetricStatistics Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 338
Content-Type: text/xml
Date: Mon, 08 May 2023 13:41:09 GMT
X-Amzn-Requestid: eb33b780-2d38-4c53-b5ca-48ef25958258


-----------------------------------------------------
2023/05/08 13:41:09 <GetMetricStatisticsResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">
  <GetMetricStatisticsResult>
    <Datapoints/>
    <Label>TimeToFirstByte</Label>
  </GetMetricStatisticsResult>
  <ResponseMetadata>
    <RequestId>eb33b780-2d38-4c53-b5ca-48ef25958258</RequestId>
  </ResponseMetadata>
</GetMetricStatisticsResponse>

from yet-another-cloudwatch-exporter.

cristiangreco avatar cristiangreco commented on June 27, 2024

@bc-ibiks it seems the response from AWS GetMetricStatistics does not contain any datapoint.

You mentioned that you can see metrics in CloudWatch. What happens if you run the AWS cli aws cloudwatch get-metric-statistics? (you can find an example in YACE debug logs: YACE emits a log line like CLI helper - ... which tells you how to reproduce with aws cli tool what YACE is requesting).

from yet-another-cloudwatch-exporter.

bc-ibiks avatar bc-ibiks commented on June 27, 2024

I will let you know what I find out - thanks

from yet-another-cloudwatch-exporter.

mhester-nutrien avatar mhester-nutrien commented on June 27, 2024

I have YACE running in >60 accounts and ran into this exact same issue. What solved it for me was adding the delay attribute.

        period: 60
        length: 600
        delay: 60

I also found that the addCloudwatchTimestamp attribute helped with this (but did not completely fix), but it did also align the metrics with native prometheus metrics on the Grafana dashboard. Without this attribute the metrics coming from CW/YACE were misaligned.

Also, I ran into another issue with "duplicate" metrics when using nilToZero, and I didn't find any real benefit to using that attribute. I would suggest playing with removing that attribute and seeing if you still get what you expect/want.

from yet-another-cloudwatch-exporter.

bc-ibiks avatar bc-ibiks commented on June 27, 2024

thanks! @mhester-nutrien. I will give this a test and let you know

from yet-another-cloudwatch-exporter.

bc-ibiks avatar bc-ibiks commented on June 27, 2024

@cristiangreco and @mhester-nutrien I was able to retrieve the custom metric using prometheus cloudwatch exporter. (which is a tad annoying to use)
I still wondering why I can pull this with YACE. I am able to pull the metrics using aws cli

Seems I am missing some settings in my config

from yet-another-cloudwatch-exporter.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.