Giter Club home page Giter Club logo

Comments (16)

pdreker avatar pdreker commented on September 13, 2024 2

It wasn't the next weekend, but here we are, with (maybe) a little present for the holidays :-)

I have just pushed some changes to the develop version, which read information from devices via the HTTP AHA API. Currently it only checks every device reported by the box for "battery" and "battery_low" values and converts them into metrics if found.

As I myself do not have any of the HA devices I cannot test those changes. Could you have a got with the current "develop" image and check, if you can see fritz_ha_battery_level_percentand fritz_ha_battery_low metrics?

If this works I can extend the metrics reported relatively easily.

from fritz_exporter.

pdreker avatar pdreker commented on September 13, 2024 1

Holy cow, Bat-Man!

Hadn't checked for quite some time, but this is some really nice information, especially if this is usable through fritzconnection.

I'll have a look towards the weekend, but obviously this will be re-opened :-D

Additionally this might solve the "No Cable Data" issues.

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

I've dug around in the code and the AVM docs for a while now and I can't find corresponding data for the sensor states. I tried the DECT and Homeauto services but to no avail.

from fritz_exporter.

pdreker avatar pdreker commented on September 13, 2024

As discussed in the issue of the original feature request #224 Fritz!OS does not seem to expose some information through the TR-064 API. The windows sensors are simply not provided by AVM (according to the documentation and my quick tests). Battery state of battery powered devices is also not reported, which to me seems like the most inconvenient omission.

I'm sorry, but the device simply does not provide the metrics via the API this exporter uses :( . In #224 there is a mention of another exporter which uses the http API, which provides a different set of data points. Maybe a combination of these two exporters can capture everything.

I will have to make a note of this in the documentation.

from fritz_exporter.

pdreker avatar pdreker commented on September 13, 2024

I have made the docs a little more explicit that window sensors and battery state are missing due to an omission from the API.

I'll close this issue, as currently there is nothing I can do about this other than waiting for AVM to add these metrics or actually implementing HTTP access, which is an undocumented API and quite a lot of work.

from fritz_exporter.

pdreker avatar pdreker commented on September 13, 2024

closed

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

which is an undocumented API and quite a lot of work.

I've written to AVM support and they helpfully pointed me to this doc which was published only recently (18th September 23):

https://avm.de/fileadmin/user_upload/Global/Service/Schnittstellen/AHA-HTTP-Interface.pdf

It contains the full HTTP API spec (for home automation) and after some testing I can indeed get all the missing stuff from there by using the fritzconnection library that you're already employing.

I'd be happy to collaborate on implementing that stuff into the exporter if you're up for it.

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

Initial findings:

Sensors can be accessed through AHA-HTTP Interface

HAN-FUN Sensors hide behind the AIN of their device and the "-1" duplicate (that's where the state data is)
For the telekom window sensors there are two states (open and closed)

You can access this data with the getdeviceinfos action and the AIN of the device.

The important bits here is the <alert> tag. This contains the state and the timestamp (since epoch) of the last state change.

The states are:

0 - Closed
1 - Open

.Sensor state "open"

...
<alert><state>1</state><lastalertchgtimestamp>1699253655</lastalertchgtimestamp></alert>
...

{'content': '<device identifier="11934 0323032" id="408" functionbitmask="1" '
            'fwversion="31.35" manufacturer="0x0feb" '
            'productname="HAN-FUN"><present>1</present><txbusy>0</txbusy><name>Fenster '
            'Toilette</name></device>\n',
 'content-type': 'text/xml',
 'encoding': 'utf-8'}

{'content': '<device identifier="11934 0323032-1" id="2001" '
            'functionbitmask="8208" fwversion="0.0" manufacturer="0x0feb" '
            'productname="HAN-FUN"><present>1</present><txbusy>0</txbusy><name>Fenster '
            'Toilette</name><etsiunitinfo><etsideviceid>408</etsideviceid><unittype>514</unittype><interfaces>256</interfaces></etsiunitinfo><alert><state>1</state><lastalertchgtimestamp>1699253655</lastalertchgtimestamp></alert></device>\n',
 'content-type': 'text/xml',
 'encoding': 'utf-8'}

.Sensor state "closed"

...
<alert><state>0</state><lastalertchgtimestamp>1699259643</lastalertchgtimestamp></alert>
...

{'content': '<device identifier="11934 0351393" id="411" functionbitmask="1" '
            'fwversion="31.35" manufacturer="0x0feb" '
            'productname="HAN-FUN"><present>1</present><txbusy>0</txbusy><name>Tür '
            'Kinderzimmer </name></device>\n',
 'content-type': 'text/xml',
 'encoding': 'utf-8'}

{'content': '<device identifier="11934 0351393-1" id="2003" '
            'functionbitmask="8208" fwversion="0.0" manufacturer="0x0feb" '
            'productname="HAN-FUN"><present>1</present><txbusy>0</txbusy><name>Tür '
            'Kinderzimmer '
            '</name><etsiunitinfo><etsideviceid>411</etsideviceid><unittype>514</unittype><interfaces>256</interfaces></etsiunitinfo><alert><state>0</state><lastalertchgtimestamp>1699259643</lastalertchgtimestamp></alert></device>\n',
 'content-type': 'text/xml',
 'encoding': 'utf-8'}

Thermostats

The Thermostats (at least Fritz ones) can also be accessed via their AIN via AHA-HTTP.
This exposes all kinds of data including battery.

tsoll, tist, komfort are the temperature values that are set with the programmed schedule. NOTE: They are for some reason multiplied by 2 over what is set in the UI (probably to avoid having floats and the UI only allows half degree steps).

Weirdly the <temperature> (measured temperature) is multiplied by 10.

lock and devicelock the operation locks. Need to determine further which is which (one is button lock and the other is overall operations lock).

windowopenactive shows if the window detection is triggered (probably also by associated sensor, need to test)

Basically all the info from the FritzUI is exposed here.

.Fritz Dect 301 device (formatted)

{'content': '<device identifier="09995 0390185" id="16" functionbitmask="320" '
            ' fwversion="05.08" manufacturer="AVM" productname="FRITZ!DECT '
            '301">
  <present>1</present>
  <txbusy>0</txbusy>
  <name>Wohnzimmer '
    'Süd</name>
  <battery>80</battery>
  <batterylow>0</batterylow>
  <temperature>
    <celsius>195</celsius>
    <offset>0</offset>
  </temperature>
  <hkr>
    <tist>39</tist>
    <tsoll>30</tsoll>
    <absenk>30</absenk>
    <komfort>42</komfort>
    <lock>0</lock>
    <devicelock>1</devicelock>
    <errorcode>0</errorcode>
    <windowopenactiv>0</windowopenactiv>
    <windowopenactiveendtime>0</windowopenactiveendtime>
    <boostactive>0</boostactive>
    <boostactiveendtime>0</boostactiveendtime>
    <batterylow>0</batterylow>
    <battery>80</battery>
    <nextchange>
      <endperiod>1699290000</endperiod>
      <tchange>42</tchange>
    </nextchange>
    <summeractive>0</summeractive>
    <holidayactive>0</holidayactive>
    <adaptiveHeatingActive>1</adaptiveHeatingActive>
    <adaptiveHeatingRunning>0</adaptiveHeatingRunning>
  </hkr>
</device>\n',
'content-type': 'text/xml',
'encoding': 'utf-8'}

TODO

. Implement device discovery from HTTP API
. Find list of sensors
. Write function to generate metrics from the sensor data
. Write logic that treats this data like all the other data
. Modify the access helper to work in HTTP mode with parameters and allows looking at singular devices

Nice to have:
. Decipher how the energy data stats for the DECT 200/210 are used
+
.getbasicdevicestats for a DECT 210

{'content': '<devicestats><temperature><stats count="96" grid="900" '
            'datatime="1699286551">110,110,110,115,115,115,115,115,115,120,120,125,130,130,125,125,125,130,125,120,120,120,115,115,115,110,110,110,115,110,110,110,110,110,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,110,105,105,105,105,110,110,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,110,100,105,100,105,105,105,105,105,105,105,105,105,110,110,110,110,115</stats></temperature><voltage><stats '
            'count="360" grid="10" '
            'datatime="1699286551">235333,235333,235333,235333,235333,235333,235333,235333,235333,235333,235333,235333,234841,234841,234841,234841,234841,234841,234841,234841,234841,234841,234841,234841,235530,235530,235530,235530,235530,235530,235530,235530,235530,235530,235530,235530,235606,235606,235606,235606,235606,235606,235606,235606,235606,235606,235606,235606,235606,235754,235754,235754,235754,235754,235754,235754,235754,235754,235754,235754,235754,235645,235645,235645,235645,235645,235645,235645,235645,235645,235645,235645,235645,235462,235462,235462,235462,235462,235462,235462,235462,235462,235462,235462,235462,235494,235494,235494,235494,235494,235494,235494,235494,235494,235494,235494,235494,235750,235750,235750,235750,235750,235750,235750,235750,235750,235750,235750,235750,235736,235736,235736,235736,235736,235736,235736,235736,235736,235736,235736,235736,235695,235695,235695,235695,235695,235695,235695,235695,235695,235695,235695,235841,235841,235841,235841,235841,235841,235841,235841,235841,235841,235841,235841,235255,235255,235255,235255,235255,235255,235255,235255,235255,235255,235255,235255,235419,235419,235419,235419,235419,235419,235419,235419,235419,235419,235419,235419,235240,235240,235240,235240,235240,235240,235240,235240,235240,235240,235240,235240,235209,235209,235209,235209,235209,235209,235209,235209,235209,235209,235209,235209,235492,235492,235492,235492,235492,235492,235492,235492,235492,235492,235492,235492,235646,235646,235646,235646,235646,235646,235646,235646,235646,235646,235646,235646,235456,235456,235456,235456,235456,235456,235456,235456,235456,235456,235456,235456,235665,235665,235665,235665,235665,235665,235665,235665,235665,235665,235665,235665,235397,235397,235397,235397,235397,235397,235397,235397,235397,235397,235397,235397,235275,235275,235275,235275,235275,235275,235275,235275,235275,235275,235275,235275,235771,235771,235771,235771,235771,235771,235771,235771,235771,235771,235771,235771,235877,235877,235877,235877,235877,235877,235877,235877,235877,235877,235877,235877,235882,235882,235882,235882,235882,235882,235882,235882,235882,235882,235882,235882,236150,236150,236150,236150,236150,236150,236150,236150,236150,236150,236150,236150,236143,236143,236143,236143,236143,236143,236143,236143,236143,236143,236143,236143,236183,236183,236183,236183,236183,236183,236183,236183,236183,236183,236183,236183,235572,235572,235572,235572,235572,235572,235572,235572,235572,235572,235572,235572,236118,236118,236118,236118,236118,236118,236118,236118,236118,236118,236118,236118</stats></voltage><power><stats '
            'count="360" grid="10" '
            'datatime="1699286551">14,14,14,14,14,14,14,14,14,14,14,14,13,13,13,13,13,13,13,13,13,13,13,13,14,14,14,14,14,14,14,14,14,14,14,14,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,14,14,14,14,14,14,14,14,14,14,14,14,15,15,15,15,15,15,15,15,15,15,15,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,181,181,181,181,181,181,181,181,181,181,181,181,345,345,345,345,345,345,345,345,345,345,345,345,578,578,578,578,578,578,578,578,578,578,578,578,746,746,746,746,746,746,746,746,746,746,746,746,529,529,529,529,529,529,529,529,529,529,529,529,523,523,523,523,523,523,523,523,523,523,523,523,476,476,476,476,476,476,476,476,476,476,476,476,444,444,444,444,444,444,444,444,444,444,444,444,384,384,384,384,384,384,384,384,384,384,384,384,439,439,439,439,439,439,439,439,439,439,439,439,472,472,472,472,472,472,472,472,472,472,472,472,488,488,488,488,488,488,488,488,488,488,488,488,537,537,537,537,537,537,537,537,537,537,537,537,577,577,577,577,577,577,577,577,577,577,577,577,597,597,597,597,597,597,597,597,597,597,597,597,693,693,693,693,693,693,693,693,693,693,693,693,621,621,621,621,621,621,621,621,621,621,621,621,624,624,624,624,624,624,624,624,624,624,624,624,625,625,625,625,625,625,625,625,625,625,625,625</stats></power><energy><stats '
            'count="12" grid="2678400" '
            'datatime="1699232408">3119,9126,0,0,0,0,0,0,0,0,0,0</stats><stats '
            'count="31" grid="86400" '
            'datatime="1699232407">838,267,438,609,189,778,533,419,136,186,680,435,137,132,1022,505,1536,309,264,1712,1120,0,0,0,0,0,0,0,0,0,0</stats></energy></devicestats>\n',
 'content-type': 'text/xml',
 'encoding': 'utf-8'}

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

I did all the testing by doing some very basic modifications to the fritz_export_helper.py. All it needs is using call_http instead of call_action to the authenticated connection and then providing the correct action (and optional AIN) from the API doc. The sid is handled automatically by fritzconnection.

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

Additionally this might solve the "No Cable Data" issues.

From a first glance the API does not provide anything related to the network or internet connections. AHA stands for "AVM Home Automation" and the data is entirely related to smart home devices and their control (which is awesome anyway).

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

https://fritzconnection.readthedocs.io/en/latest/sources/getting_started.html#combining-the-apis

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

Oh look, open source Santa dancing on my roof. 😄

I'll try have a look at this in the coming days.

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

So I finally managed to make some time to test and the develop container crashes immediately after start for me:

2024-01-03 20:11:44,508     INFO fritzexporter.fritzdevice | Connection to 192.168.178.1 successful, reading capabilities
2024-01-03 20:11:47,111     INFO fritzexporter.fritzdevice | Reading capabilities for 192.168.178.1, got serial <REDACTED> , model name FRITZ!Box 7530 completed
2024-01-03 20:11:47,111     INFO fritzexporter | registering 192.168.178.1 to collector
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/xml/etree/ElementTree.py", line 1701, in close
    self.parser.Parse(b"", True) # end of data
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
xml.parsers.expat.ExpatError: no element found: line 1, column 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/local/lib/python3.12/site-packages/fritzexporter/__main__.py", line 126, in <module>
    main()
  File "/usr/local/lib/python3.12/site-packages/fritzexporter/__main__.py", line 109, in main
    REGISTRY.register(fritzcollector)
  File "/usr/local/lib/python3.12/site-packages/prometheus_client/registry.py", line 40, in register
    names = self._get_names(collector)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/prometheus_client/registry.py", line 80, in _get_names
    for metric in desc_func():
  File "/usr/local/lib/python3.12/site-packages/fritzexporter/fritzdevice.py", line 105, in collect
    yield from capa.get_metrics(self.devices, name)
  File "/usr/local/lib/python3.12/site-packages/fritzexporter/fritzcapabilities.py", line 87, in get_metrics
    self._generate_metric_values(device)
  File "/usr/local/lib/python3.12/site-packages/fritzexporter/fritzcapabilities.py", line 1413, in _generate_metric_values
    http_data = parse_aha_device_xml(http_result["content"])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/fritzexporter/fritz_aha.py", line 5, in parse_aha_device_xml
    device: ElementTree = ElementTree.fromstring(deviceinfo)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/defusedxml/common.py", line 127, in fromstring
    return parser.close()
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/xml/etree/ElementTree.py", line 1703, in close
    self._raiseerror(v)
  File "/usr/local/lib/python3.12/xml/etree/ElementTree.py", line 1603, in _raiseerror
    raise err
xml.etree.ElementTree.ParseError: no element found: line 1, column 0

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

Btw. there is super useful but undocumented overview for all the DECT Smart Home devices: http://fritz.box/#uleoverview

from fritz_exporter.

r0ckarong avatar r0ckarong commented on September 13, 2024

I did some more testing and I think the problem is that I have non AVM devices in my setup that do not report battery stats and therefore there is no "data" in the result for this check on such a device.

HAN-FUN device

{'content': '<device identifier="11934 0324621-1" id="2000" '
            'functionbitmask="8208" fwversion="0.0" manufacturer="0x0feb" '
            'productname="HAN-FUN"><present>1</present><txbusy>0</txbusy><name>Fenster '
            'Badezimmer</name><etsiunitinfo><etsideviceid>407</etsideviceid><unittype>514</unittype><interfaces>256</interfaces></etsiunitinfo><alert><state>0</state><lastalertchgtimestamp>1704277584</lastalertchgtimestamp></alert></device>\n',

AVM device

{'content': '<device identifier="09995 0387424" id="20" functionbitmask="320" '
            'fwversion="05.08" manufacturer="AVM" productname="FRITZ!DECT '
            '301"><present>1</present><txbusy>0</txbusy><name>Schlafzimmer '
            </name><battery>70</battery><batterylow>0</batterylow><temperature><celsius>180</celsius><offset>0</offset></temperature><hkr><tist>36</tist><tsoll>38</tsoll><absenk>32</absenk><komfort>38</komfort><lock>0</lock><devicelock>0</devicelock><errorcode>0</errorcode><windowopenactiv>0</windowopenactiv><windowopenactiveendtime>0</windowopenactiveendtime><boostactive>0</boostactive><boostactiveendtime>0</boostactiveendtime><batterylow>0</batterylow><battery>70</battery><nextchange><endperiod>1704320100</endperiod><tchange>32</tchange></nextchange><summeractive>0</summeractive><holidayactive>0</holidayactive><adaptiveHeatingActive>1</adaptiveHeatingActive><adaptiveHeatingRunning>0</adaptiveHeatingRunning></hkr></device>\n',

I guess the easiest fix is to, for now, limit the check to manufacturer="AVM" for the battery.

There probably are plenty of devices out there that report interesting metrics. My HAN-FUN sensors for example give me their state and when they last triggered. That could be interesting to visualize with the temperature changes etc.

I am still thinking about how to make this easier to integrate than you having to hardcode any metrics and filter for all types of device manufacturers. My first idea was to somehow make the metrics and sources configurable outside the code but this takes some more thinking.

My suggestion would be to stick to the "known" AVM devices and their reported stuff and integrate those metrics.

from fritz_exporter.

pdreker avatar pdreker commented on September 13, 2024

In the past I quite literally stuck to the Python motto of "It is better to ask foregiveness than permission." In this context this means that I would go for a variant, which just tries to "go for it" and if for example the XML Parsing ocde throws an exception like above, I just catch that exception, log a (DEBUG level?) message and just ignore the problem and not add a metric there.

This avoids the whole hardcoding IDs problem and still works whenever a device returns a battery state. There is an example in the capabilities check, where it seems that some device just report endpoint they are actually not supporting. I just catch the FritzServiceError (and similar) and remove the metric. In this case it would be more prudent to do something like this when actually fetching the metrics from the device, instead when chacking capabilities.

from fritz_exporter.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.