In dms_parser/clients/serial.py it says:
import serial

This cannot work, 'serial.py' imports itself...

WARNING (MainThread) [hassio.addons.data] Can't read repository information from /data/addons/git/4852bf21/repository.json

So I can't install the addon on my installation.

Feature request

Would it be possible to add a configuration item to reduce the maximum generated amount of data per minute (or hour)? My home-assistant database is flooded, and there seems no good way to prevent that (hence, the recorder purging doesn't seem to do much / enough)

It seems sometimes when the value in the gasreading returns "2C" the parser can't handle this.

Is this something i've wrongly setup in Home assistant?

Note: I am running the DMSR add-on on Home Assistant with a Landys & Gyr E350, however with dsmr 2.2.

2020-10-22 11:00:09 DEBUG (MainThread) [dsmr_parser.clients.protocol] got telegram: /XMX5XMXABCE000069683

0-0:96.1.1(31333637343139322020202020202020)
1-0:1.8.1(15237.388*kWh)
1-0:1.8.2(18759.401*kWh)
1-0:2.8.1(00000.000*kWh)
1-0:2.8.2(00000.000*kWh)
0-0:96.14.0(0002)
1-0:1.7.0(0002.94*kW)
1-0:2.7.0(0000.00*kW)
0-0:96.13.1()
0-0:96.13.0()
0-1:96.1.0(3238303131303038333032323537303133)
0-1:24.1.0(03)
0-1:24.3.0(201022100000)(2C)(60)(1)(0-1:24.2.0)(m3)
(03575.334)
!

2020-10-22 11:00:09 ERROR (MainThread) [homeassistant] Error doing job: Fatal error: protocol.data_received() call failed.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 860, in _read_ready__data_received
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 92, in data_received
    self.handle_telegram(telegram)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 107, in handle_telegram
    parsed_telegram = self.telegram_parser.parse(telegram)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 57, in parse
    telegram[signature] = parser.parse(match.group(0))
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 159, in parse
    return MBusObject(self._parse(line))
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 137, in _parse
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 137, in <listcomp>
    return [self.value_formats[i].parse(value)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 235, in parse
    value = self.coerce_type(value) if value is not None else value
ValueError: invalid literal for int() with base 10: '2C'
2020-10-22 11:00:09 ERROR (MainThread) [dsmr_parser.clients.protocol] disconnected due to exception
NoneType: None

I see many lines in my home assistant (v. 0.45.1) log file, showing this:
2017-05-29 19:41:35 ERROR (MainThread) [dsmr_parser.clients.protocol] failed to parse telegram Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/dsmr_parser/clients/protocol.py", line 100, in handle_telegram parsed_telegram = self.telegram_parser.parse(telegram) File "/usr/local/lib/python3.6/site-packages/dsmr_parser/parsers.py", line 47, in parse self.validate_checksum(telegram_data) File "/usr/local/lib/python3.6/site-packages/dsmr_parser/parsers.py", line 88, in validate_checksum expected_crc dsmr_parser.exceptions.InvalidChecksumError: Invalid telegram. The CRC checksum '56532' does not match the expected '64335' 201

I don't want the log to show these exceptions. How can I get rid of them?

The README could describe the protocol client and async serial client. Also using the parsing logic without the clients (the libraries main goal) is not documented.

hello,

it is possible to filter this out of the data sent by the meter
0-0: 96.13.0.255,
it is not relevant data and its processing is problematic, because the service provider, 0xff, fills in the location of the message.

Thanks, Tomas

A small refactoring of the way DSMRProtocol and SerialReader feed data into TelegramParser.parse. This currently consists of a list of strings.

The preferred solution is that TelegramParser.parse will accept a single bytestring as outputted by the meter without any alterations. This removes any form of logic in the clients themselves and will also sort issues with line endings (see pull request #15)

Hi,

When I useFileReader, the telegrams are successfully parsed and checksummed but the loop doesn't stop after reading the whole file. Is there something wrong with my data file? Is this the intended behaviour? (I have Python 3.9.10 and dsmr-parser==0.32)

If it isn't, I think the lines should be probably changed to something

 while True:
    data = file_handle.readline()
    if not data:
        break
    s = data.decode()
    self.telegram_buffer.append(s)

I just got a new meter, as I upgraded my 1x40A to 3x25A.

But when I connect it to my Home-Assistent, I no longer receive any data.

When trying to figure out why, I noticed the logs are getting filled with the same message every time the p1 message is received.

2021-01-08 19:31:14 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SerialTransport._read_ready() Traceback (most recent call last): File "/usr/local/lib/python3.8/asyncio/events.py", line 81, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.8/site-packages/serial_asyncio/__init__.py", line 119, in _read_ready self._protocol.data_received(data) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 96, in data_received self.handle_telegram(telegram) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 111, in handle_telegram parsed_telegram = self.telegram_parser.parse(telegram) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 59, in parse telegram[signature] = parser.parse(match.group(0)) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 174, in parse return MBusObject(self._parse(line)) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 153, in _parse return self._parse_values(values) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 138, in _parse_values return [self.value_formats[i].parse(value) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 138, in return [self.value_formats[i].parse(value) File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 284, in parse value = self.coerce_type(value) if value is not None else value File "/usr/local/lib/python3.8/site-packages/dsmr_parser/value_types.py", line 7, in timestamp naive_datetime = datetime.datetime.strptime(value[:-1], '%y%m%d%H%M%S') File "/usr/local/lib/python3.8/_strptime.py", line 568, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) File "/usr/local/lib/python3.8/_strptime.py", line 352, in _strptime raise ValueError("unconverted data remains: %s" % ValueError: unconverted data remains: 525

Hopefully you have an idea what caused this, and where I can look further.

Thanks in advance!

This makes it much easier to know what is contained inside "version 0.3" for instance. It is also easier for packaging things up.

Thanks!

pytz is now pinned to 2016.3 this causes conflicts with requirements of other packages. Better change to 'pytz>=2016' to avoid conflicts.

Hi,

I am using the homeassistant integration with your dsmr_parser module.
My smartmeter is a version 5 smartmeter with standard settings (115200 8N1).

The readout with the parsed MBusObject always has value '0' for the HOURLY_GAS_METER_READING.
This readout is also used in the homeassistant integration.

When I look at the raw telegram_data of the TelegramObject I see that the gas meter reading is not 0 (truncated telegram with header and relevant line):

/ISK5\2M550T-1012

1-3:0.2.8(50)
[...]
0-2:24.2.1(200426223001S)(00246.138*m3)

So there seems to be a problem with the parsing :(

I believe the parser does not correctly deal with a smart meter installed in Belgium (Fluvius netbeheerder). I am using the parser through the Home Assistant sensor.
For gas the telegram should be 0-1:24.2.3, but maybe there are other differences. I can help to provide test telegrams but need instructions on how to do so (totally new to home assistant and just learning to set it up).
Ideally a config switch for NL/BE would be great.

I get the foloowing message on my SSh:

Collecting dsmr-parser
Using cached https://files.pythonhosted.org/packages/9e/d0/eb5c275d7b293f3fb6a754e017a79e3e929bf32c14e920e2a7278edfd04a/dsmr-parser-0.14.tar.gz
Collecting pyserial<4,>=3 (from dsmr-parser)
Downloading https://files.pythonhosted.org/packages/0d/e4/2a744dd9e3be04a0c0907414e2a01a7c88bb3915cbe3c8cc06e209f59c30/pyserial-3.4-py2.py3-none-any.whl (193kB)
100% |████████████████████████████████| 194kB 1.5MB/s
Collecting pyserial-asyncio<1 (from dsmr-parser)
Using cached https://files.pythonhosted.org/packages/41/3f/e26f71269cbc0890a527a736d9afc5c0d5838a2c188be680558d635b7dc2/pyserial-asyncio-0.4.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-6hhyQU/pyserial-asyncio/setup.py", line 15, in
raise RuntimeError("pyserial-asyncio requires at least Python 3.4")
RuntimeError: pyserial-asyncio requires at least Python 3.4

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-6hhyQU/pyserial-asyncio/

What do I need to do to resolve this? (yes I'm a newbee ... :-) )

Telegram values include a CRC16 checksum which can be used that the telegram is valid. This check prevents invalid data due to corrupted values, as the serial interface doesn't have any built-in verification.

It will raise a subtype of the existing ParseError from the serial client.

The CRC16 check currently verifies DSMR 4.2 telegrams. Telegrams of other versions are appreciated. I will anonymize them of equipment identifiers before adding them to the repository. If you decide to supply one, please leave them intact so i have valid data to test with before anonimizing them.

Hi,

Trying to find out if the Luxembourg smart meter which has a P1 port, can be integrated into Home Assistant.

In their specifications , they say the following:

P1 protocol is a plain text protocol: it is required to secure the link to ensure end user data confidentiality. In deviation to DSMR P1, an encryption layer is added over the P1 protocol. This encryption layer is based on DLMS security suite 0 algorithm: AES128-GCM

Does your library support the decryption ?

Hey,

Nice job. I tried to add ACE3000 with same 2.2-protocol, only you have to do two things:

• send wake up string ("\x2f\x3f\x21\x0d\x0a" > /dev/ttyUSB0)
• use 300 baud

Baud setting seem easy, but sending the string before read - I don't really know.

Could you give a hint or help?

I start dsmr_console -v

on an othe rbash I send this:
echo -n -e '\x2F\x3F\x21\x0D\x0A' > /dev/ttyUSB0

I got this via dsmr_console then:

DEBUG:dsmr_parser.clients.protocol:got telegram: /ACE0\3k260V01.19
F.F(00)
C.1(1126130053505702)
C.5.0(01)
1.8.0(020974.0*kWh)
2.8.0(014699.9*kWh)

Would great to get this running.
Thanks in advanced.

JayTee

It seems you are supporting the Benelux region now (since commit 3bfb555) and not only the dutch smart meter protocol, shouldn't the naming of the project reflect that?

To keep the D I would suggest to change it to Digital Smart Meter Requirements instead of Dutch Smart Meter Requirements, what do you think?

I've tried to use the parser example with my ME382 meter, but something goes wrong in the parsing of the telegram. I've used the example code and changed 4 to 2_2 for my meter.

The following error is logged on my screen:

pi@raspberrypi:~ $ sudo python3 test.py
No line class found for: '/ISk5\2ME382-1003'
No line class found for: ''
No line class found for: '0-0:96.1.EC   13Y**'
No line class found for: '"'
No line class found for: 'kL(6K0:1.8.1(066EsCS[Wh)'
No line class found for: 'EkRsBs'
Traceback (most recent call last):
  File "test.py", line 11, in <module>
    for telegram in serial_reader.read():
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/serial.py", line 71, in read
    yield self.telegram_parser.parse(telegram)
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/parsers.py", line 67, in parse
    return super().parse(join_lines(line_values))
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/parsers.py", line 32, in parse
    obis_reference, dsmr_object = self.parse_line(line_value.strip())
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/parsers.py", line 47, in parse_line
    return obis_reference, parser.parse(line_value)
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/parsers.py", line 135, in parse
    return CosemObject(self._parse(line))
  File "/usr/local/lib/python3.4/dist-packages/dsmr_parser/parsers.py", line 84, in _parse
    raise ParseError("Invalid '%s' line for '%s'", line, self)
dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", 'C\x02\x02\x02\x02\x02r\x02\x02\x12S[o\x1d\x7fjc\x0f1-0:2.8.2(00000.00ES[o\x1d\x7fj', <dsmr_parser.parsers.CosemParser object at 0x76868210>)

When I read out the serial port without the parser, the telegrams look like this:

pi@raspberrypi:~ $ cu -l /dev/ttyUSB0 -s 9600 --parity=even
Connected.
/ISk5\2ME382-1003

0-0:96.1.1(4B413650303035313238363432312332)
1-0:1.8.1(06612.282*kWh)
1-0:1.8.2(04532.419*kWh)
1-0:2.8.1(00000.002*kWh)
1-0:2.8.2(00000.001*kWh)
0-0:96.14.0(0002)
1-0:1.7.0(0000.16*kW)
1-0:2.7.0(0000.00*kW)
0-0:17.0.0(0999.00*kW)
0-0:96.3.10(1)
0-0:96.13.1()
0-0:96.13.0()
0-1:24.1.0(3)
0-1:96.1.0(4730303135353631312333343534363133)
0-1:24.3.0(161115140000)(00)(60)(1)(0-1:24.2.1)(m3)
(07857.901)
0-1:24.4.0(1)
!

Edit:

I just managed to solve this by changing this:

SERIAL_SETTINGS_V2_2 = {
    'baudrate': 9600,
    'bytesize': serial.SEVENBITS,
    'parity': serial.PARITY_NONE,
    'stopbits': serial.STOPBITS_ONE,
    'xonxoff': 0,
    'rtscts': 0,
    'timeout': 20
}

to this:

SERIAL_SETTINGS_V2_2 = {
    'baudrate': 9600,
    'bytesize': serial.SEVENBITS,
    'parity': serial.PARITY_EVEN,
    'stopbits': serial.STOPBITS_ONE,
    'xonxoff': 0,
    'rtscts': 0,
    'timeout': 20
}

in file:
dsmr_parser/serial.py

Is it possible that is is depended on the P1 cable hardware you use?

Hi @ndokter and @lowdef

I'm from Hungary and my power meter: Sanxing SX631 (S34U18)

I'm trying to use DSMR slimme meter integration in HA but I received an error message:

Traceback (most recent call last):
File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 862, in _read_ready__data_received
self._protocol.data_received(data)
File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 91, in data_received
data = data.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 140: ordinal not in range(128)

Here is an example from my telegram data:

/AUX59902730044

0-0:1.0.0(210511102330S)
0-0:42.0.0(AUX1030302730044)
0-0:96.1.0(9902730044)
0-0:96.14.0(0001)
0-0:96.50.68(ON)
0-0:17.0.0(90.000kW)
1-0:1.8.0(000101.821kWh)
1-0:1.8.1(000040.496kWh)
1-0:1.8.2(000061.325kWh)
1-0:1.8.3(000000.000kWh)
1-0:1.8.4(000000.000kWh)
1-0:2.8.0(000653.426kWh)
1-0:2.8.1(000398.181kWh)
1-0:2.8.2(000255.245kWh)
1-0:2.8.3(000000.000kWh)
1-0:2.8.4(000000.000kWh)
1-0:3.8.0(000002.626kvarh)
1-0:4.8.0(000143.011kvarh)
1-0:5.8.0(000002.247kvarh)
1-0:6.8.0(000000.379kvarh)
1-0:7.8.0(000075.324kvarh)
1-0:8.8.0(000067.687kvarh)
1-0:15.8.0(000755.250kWh)
1-0:32.7.0(237.2V)
1-0:52.7.0(235.5V)
1-0:72.7.0(237.1V)
1-0:31.7.0(008A)
1-0:51.7.0(008A)
1-0:71.7.0(008A)
1-0:13.7.0(0.996)
1-0:33.7.0(0.996)
1-0:53.7.0(0.995)
1-0:73.7.0(0.997)
1-0:14.7.0(50.00Hz)
1-0:1.7.0(00.000kW)
1-0:2.7.0(05.868kW)
1-0:5.7.0(00.000kvar)
1-0:6.7.0(00.000kvar)
1-0:7.7.0(00.473kvar)
1-0:8.7.0(00.000kvar)
0-0:98.1.0(210501000000S)(000041.191kWh)(000015.425kWh)(000025.766kWh)(000274.705kWh)(000189.336kWh)(000085.369kWh)(000001.034kvarh)(000059.658kvarh)(000000.866kvarh)(000000.168kvarh)(000031.678kvarh)(000027.980kvarh)(000315.899kWh)(03.644kW)(03.188kW)(03.644kW)(06.872kW)(06.872kW)(06.544kW)
0-0:96.13.0(????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????)
!6D64

Could you help me pls?

Thank you in advance!

Requesting support for Kamstrup Omnipower HAN-module and Aidon AMI system smart meters with "HAN-NVE" RJ45 interface.

"HAN-NVE" has a physical RJ45 port which in Sweden and Scandinavia is referred to as the "Norwegian RJ45 HAN-port standard".

https://hanporten.se/norska/porten/

https://hanporten.se/norska/protokollet/

https://hanporten.se/norska/kretsschema/

Google search says that they at least sell an Kamstrup HAN-module with "HAN-NVE" RJ45 HAN interface with part number 6840004 which according to its specifications listed in the data sheet it looks like the reader needs to comply with MBUS EN-13757-2 standard as per recommendation from NVE (Norwegian Water Resources and Energy Directorate) for Norway, a.k.a. AMS+HAN version 2.0 or IEC 62056-7-5, annex D ”MBUS” slave, e.g. an IEC 62056-21 protocol mode B client.

https://www.nek.no/wp-content/uploads/2018/10/Kamstrup-HAN-NVE-interface-description_rev_3_1.pdf

https://www.nek.no/wp-content/uploads/2017/10/AMS-HAN-personvernnotat-h%C3%B8ringsversjon.pdf

https://byggebolig.no/imageoriginals/88b3d1774ecb41e6a3fe067ae9e6a893.pdf

https://www.energiforetagen.se/globalassets/energiforetagen/det-erbjuder-vi/publikationer/branschrekommendation-lokalt-granssnitt-v1-2-2018.pdf

https://www.utomhusliv.se/wp-content/uploads/2020/10/Specifikation-f%C3%B6r-HAN-modulen-f%C3%B6r-elm%C3%A4tare-engelska.pdf

Google also says that Aidon AMI system smart meters use the same type of HAN-NVE HAN with RJ45 port with M-Bus interface:

https://www.skekraft.se/wp-content/uploads/2021/03/Aidon_Feature_description_RJ45_HAN_Interface_EN.pdf

https://www.tekniskaverken.se/siteassets/tekniska-verken/elnat/aidonfd-rj45-han-interface-se-v13a.pdf

There looks to be some guides for parsing the protocol and getting the data through some kind of M-Bus converter here:

https://www.kode24.no/guider/smart-meter-part-1-getting-the-meter-data/71287300

https://xipher.dk/posts/2020-05-17-using-esp8266-to-monitor-kamstrup-omnipower/

https://github.com/Claustn/esp8266-kamstrup-mqtt

https://github.com/roarfred/AmsToMqttBridge/blob/master/Samples/Kaifa/readme.md

https://github.com/roarfred/AmsToMqttBridge/blob/master/Samples/Kamstrup/obisdata.md

https://hanporten.se/norska/protokollet/

"The meter acts as Mbus master pushing data. To read the information, a circuit for Mbus' slave is needed."

"They are following the DLMS (Device Language Message Specification) protocol and are sent inside HDLC frames and contains OBIS (Object Identification System) codes that describe the electricity usage. Everything is part of IEC 62056 which is a set of standards for electricity metering data exchange."

It will probably be clear from my question that I'm no Python expert, but I'm struggling to get the hourly gas timestamp value from the telegram parser. Is there a possibility to get this timestamp in my application?

This is a snippet of my current application, around which I would like to get the timestamp:

    for telegram in serial_reader.read():
        # Get the raw gas and electricity readings
        gas_reading = telegram[obis_references.HOURLY_GAS_METER_READING]
        elec_reading = telegram[obis_references.CURRENT_ELECTRICITY_USAGE]

I'm getting the following issues when trying to install dsmr_parser:

$ pip3 install dsmr_parser
[...]
ERROR: Could not find a version that satisfies the requirement PyCRC<2,>=1.2 (from dsmr_parser) (from versions: 0.9.2)
ERROR: No matching distribution found for PyCRC<2,>=1.2 (from dsmr_parser)

Looking on PyPi I only see a lowercase version of this package, at version 0.9.2:
https://pypi.org/project/pycrc/
but I think this is a different package?

piwheels does seem to have both versions:
https://www.piwheels.org/project/pycrc/
https://www.piwheels.org/project/PyCRC/

Did the PyCRC package get removed?

Misc info, tried this on both a Pi:

 $ pip3 --version
pip 20.0.2 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)

And an Ubuntu machine:

$ pip3 --version
pip 19.3.1 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)

Edit: found a workaround by download the whl file directly from piwheels and installing that:

$ wget https://www.piwheels.org/simple/PyCRC/PyCRC-1.21-py2.py3-none-any.whl#sha256=61f07008dd7813888d4793dca630a6b02bc9f9b9f6f700af81cb54722dfe91d6
$ pip install PyCRC-1.21-py2.py3-none-any.whl

After parsing a telegram in the form of a dict is returned. The values can be retrieved using defined constants.

It would probably be nicer to have a specific Telegram object that has some handy dynamic properties to retrieve the values with. The raw data will still be available.

Will try to maintain backwards compatibility (for now) by implementing getitem so most usages will remain intact.

Edit: repurposed this ticket to make it more concrete

I believe the regexes should include \d-\d: just like the others, but there are missing for the following objects:

LONG_POWER_FAILURE_COUNT = r'\d-\d:96\.7\.9.+?\r\n'
SHORT_POWER_FAILURE_COUNT = r'\d-\d:96\.7\.21.+?\r\n'
POWER_EVENT_FAILURE_LOG = r'\d-\d:99\.97\.0.+?\r\n'

I just got my warmtelink, it has a P1 port, but only gives GJ information from the "stadsverwarming"

/AUR2NWA-MYRSKY

1-3:0.2.8(50)
0-0:1.0.0(220107195809W)
0-0:96.1.1(0)
0-1:24.1.0(004)
0-1:96.1.0(123456789A1B234C)
0-1:24.2.1(220107195809W)(0.477*GJ)
!571

(the checksum doesn't work, I changed the serial number, but you get the point I assume)

It would be nice if dsmr_parser could understand this message and provide a usable interface for it. The device behaves like a normal DSMR 5 meter, but there's of course no electricity information.

I wouldn't be surprised if some setups combine this info via the DSMR electricity meter, just like with the gas meter info via the P4 port.
So at some point, this type of info might show up in a "normal" DSMR reading as well.

DSMR wont read under Home Assistant and gives always the same error.

Logger: dsmr_parser.clients.protocol
Source: /usr/local/lib/python3.9/site-packages/dsmr_parser/clients/protocol.py:121
First occurred: 02:07:00 (1308 occurrences)
Last logged: 12:34:42

disconnected due to exception
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/serial_asyncio/init.py", line 114, in _read_ready
data = self._serial.read(self._max_read_size)
File "/usr/local/lib/python3.9/site-packages/serial/serialposix.py", line 595, in read
raise SerialException(
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

This problem seems to be a while now with no clear solution:
https://community.home-assistant.io/t/dsmr-slimme-meter-problem/15956/4
https://community.home-assistant.io/t/dsmr-broken/255149

I'm using HomeAssistant since today (Migrated from Domoticz) In domoticz the values from the shown correctly.
Now i'm using HA and connected with ser2net to HA.
If i telnet to my host, the data is shown as expected. However when i configure it within HA i'm getting an error:
File "/usr/local/lib/python3.7/asyncio/selector_events.py", line 816, in _read_ready__data_received
self._protocol.data_received(data)
File "/usr/local/lib/python3.7/site-packages/dsmr_parser/clients/protocol.py", line 83, in data_received
data = data.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

I used this topic to configure: https://www.home-assistant.io/components/dsmr/ Configured it as v5.
The meter specs are ISKRA AM550 (2016)

The settings in ser2net:
2001:telnet:0:/dev/ttyUSB0:115200 8DATABITS NONE 1STOPBIT XONXOFF LOCAL -RTSCTS

I'm hoping you can point me in the right direction.

Cheers,

Maarten

Hello, I have a MT382 smart meter with 8 tariffs and it uses DSMR (plain text).

Here is an example of the output via HTERM on Windows:
/ISK5\2MT382-1008<\r>
0-0:96.1.1(314xxxxxxxxxxxxxxxxxxxxx37)<\r><\n>
0-0:128.20.0(016)<\r><\n>
1-0:0.9.2(210528)<\r><\n>
1-0:0.9.1(215658)<\r><\n>
1-0:1.7.0(001.322kW)<\r><\n>
1-0:2.7.0(000.000kW)<\r><\n>
1-0:1.8.0(010192.57kWh)<\r><\n>
1-0:1.8.1(006094.31kWh)<\r><\n>
1-0:1.8.2(001410.19kWh)<\r><\n>
1-0:1.8.3(000981.30kWh)<\r><\n>
1-0:1.8.4(001706.76kWh)<\r><\n>
1-0:1.8.5(000000.00kWh)<\r><\n>
1-0:1.8.6(000000.00kWh)<\r><\n>
1-0:1.8.7(000000.00kWh)<\r><\n>
1-0:1.8.8(000000.00kWh)<\r><\n>
1-0:1.2.0(106.732kW)<\r><\n>
1-0:1.6.0(007.072kW)<\r><\n>
1-0:2.8.0(006715.38kWh)<\r><\n>
1-0:3.8.0(000122.04kvarh)<\r><\n>
1-0:4.8.0(006255.99kvarh)<\r><\n>
1-0:0.2.2(Smart001)<\r><\n>
1-0:0.3.0(00500)<\r><\n>
1-0:0.3.1(00500)<\r><\n>
1-0:0.3.3(250)<\r><\n>
0-0:97.97.0(00000000)<\r>

My problem is, that in my country the meter requires initialization via the optical interface:
HEX 2F 3F 21 0D 0A ("/?!") at 300 baud/7E1
(afterwards it is possible to switch-over to 9600 baud: hex 06 30 35 30 0D 0A ("050") )
see: https://github.com/lvzon/dsmr-p1-parser/blob/master/doc/IEC-62056-21-notes.md#sign-on-sequence

The P1 interface is not accessible in my country, the readout is only possible optically.

I tried to use the dsmr integration in home assistant (https://www.home-assistant.io/integrations/dsmr/) , but optical readout (initialisation) seems to be not supported via the combination of the integration and your library.

I opened an issue here and asked about it:
home-assistant/core#51271

RobBie1221 pointed me to your library as per Home Assistant rules, communication with devices should only be done through 3rd party libraries. He said that the dsmr library currently officially complies to specs from The Netherlands, Belgium and Luxembourgh. Other countries may work by accident.

So I kindly ask you if it would be possible to add the sign-on-sequence (that seems not to be needed in your country as you can access the P1-Interface) in your library. If you add that, then the library can be used in other countries, that use the same meters, but only support the optical read-out.

Thx for an opinion and your great work ( that I wish I could use :-),

Sebastian

Just a reminder of something discussed earlier:

Please add release notes to the tags in your Github repo as well. You could probably just copy the contents of each release listed in the CHANGELOG.rst file to each tag.

I have a Kaifa MA304C which is using dsmr 4.4.2 but that doesn't seem parsed. I'm using it in combination with https://www.home-assistant.io/integrations/dsmr/

Are you gonna support this in a near feature?

HI,

I'm using this in Home Assistant. I get quite some CRC errors (2020-12-11 21:01:13 WARNING (MainThread) [dsmr_parser.clients.protocol] Invalid telegram. The CRC checksum '45204' does not match the expected '29681') and the error below. I connect to a Landis & Gyr E360 via DSMR 5 directly connected. It looks like I get a value in the CRC field? most of the error are CRC, sometimes I get this:

2020-12-12 14:11:47 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback SerialTransport._read_ready()
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.8/site-packages/serial_asyncio/__init__.py", line 106, in _read_ready
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 92, in data_received
    self.handle_telegram(telegram)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 107, in handle_telegram
    parsed_telegram = self.telegram_parser.parse(telegram)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 47, in parse
    self.validate_checksum(telegram_data)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 83, in validate_checksum
    expected_crc = int(checksum_hex.group(0), base=16)
ValueError: invalid literal for int() with base 16: '140W'

Let me know if you need more information (and some pointers where to get it) if needed.

Hi,
To make a serial connection to my USB device, I need to set the state of the RTS line specifically to FALSE.
Is this possible with dsmr_parser and if yes where can I assign this value?

Even though loop is optional, line 54 expects loop to be there

I would propose a fix like this:

def create_tcp_dsmr_reader(host, port, dsmr_version,
                           telegram_callback, loop=None):
    """Creates a DSMR asyncio protocol coroutine using TCP connection."""
    if not loop:
        loop = asyncio.get_event_loop()
    protocol, _ = create_dsmr_protocol(
        dsmr_version, telegram_callback, loop=loop)
    conn = loop.create_connection(protocol, host, port)
    return conn

I use ser2net to send de P1 telegram to home-assistent, and it gives the following error.

File "/usr/local/lib/python3.6/site-packages/dsmr_parser/clients/protocol.py", line 83, in data_received
data = data.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

In protocol.py line 83 I changed the line from data = data.decode('ascii') to data = data.decode('ascii', 'ignore'), and that solves the problem.

Hi,

I am running into the following error. For some reason the telegram information for OBIS 99.97.0 is empty for my ISKRA-AM550. I assume this has always been the case, I am not sure why that would change suddenly.

I upgraded to the latest version of HA (2020.12.0 and OS 5.8) and suddenly this because an issue.

Logger: dsmr_parser.clients.protocol
Source: /usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py:112
First occurred: 10:56:35 AM (394 occurrences)
Last logged: 11:03:39 AM
failed to parse telegram

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 108, in handle_telegram
parsed_telegram = self.telegram_parser.parse(telegram)
File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 59, in parse
telegram[signature] = parser.parse(match.group(0))
File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 249, in parse
return ProfileGenericObject(self._parse(line))
File "/usr/local/lib/python3.8/site-packages/dsmr_parser/parsers.py", line 148, in _parse
raise ParseError("Invalid '%s' line for '%s'", line, self)
dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", '99.97.0()\r\n', <dsmr_parser.parsers.ProfileGenericParser object at 0x7f13411fe2e0>)

I removed all entries for 99.97 in the python code (power event log or something along those lines) and that helped.

Here is an example of the actual telegram:

1-3:0.2.8(50)
0-0:1.0.0(201214112909W)
0-0:96.1.1(4530303533303037353234343134343139)
1-0:1.8.1(000177.571kWh)
1-0:1.8.2(000130.312kWh)
1-0:2.8.1(000072.503kWh)
1-0:2.8.2(000031.774kWh)
0-0:96.14.0(0002)
1-0:1.7.0(00.000kW)
1-0:2.7.0(00.139kW)
0-0:96.7.21(00010)
0-0:96.7.9(00002)
1-0:99.97.0()
1-0:32.32.0(00000)
1-0:32.36.0(00002)
0-0:96.13.0()
1-0:32.7.0(241.2V)
1-0:31.7.0(001A)
1-0:21.7.0(00.000kW)
1-0:22.7.0(00.134kW)
!72C0
~/ISK5\2M550E-1013

Python 3.5 (released 2015) introduced new async and await keywords. They replace the @asyncio.coroutine decorator and yield from.

The @asyncio.coroutine decorator has been deprecated in Python 3.8 and will be removed in Python 3.10.

It would be great if this package could migrate to using async and await keywords.

# Old
@asyncio.coroutine
def something():
    yield from asyncio.sleep(1)

# New
async def something():
    await asyncio.sleep(1)

Documentation

Seems 5S is the only one, which detects the Power Meter

Hello,
Got a problem with my DSMR, about 3 weeks ago (maybe earlier) I noticed in the llog file that I receive thousands messages a day regarding CRC checksum warnings. Although my energy dashboards works (power delivered and consumed and gas consumed).

All was ok when USB port was configured in YAML, dont know when it 'broke'.

although nothing has changed, I rearranged the P1 Cable, swapped USB ports, reinstalled the integration, but to no success.

Can somebody help me to explain how to debug the usb port and catch the data received. The error below would suggests
that the RPI 4 cant cope with the speeds of my DSRM V5 anymore.

Thanks in Advance
Frank

There is an error in my log:
Logger: dsmr_parser.clients.protocol
Source: /usr/local/lib/python3.9/site-packages/dsmr_parser/clients/protocol.py:138
First occurred: 15:38:58 (2 occurrences)
Last logged: 15:54:57

failed to parse telegram
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/dsmr_parser/clients/protocol.py", line 134, in handle_telegram
parsed_telegram = self.telegram_parser.parse(telegram)
File "/usr/local/lib/python3.9/site-packages/dsmr_parser/parsers.py", line 48, in parse
self.validate_checksum(telegram_data)
File "/usr/local/lib/python3.9/site-packages/dsmr_parser/parsers.py", line 82, in validate_checksum
raise ParseError(
dsmr_parser.exceptions.ParseError: Failed to perform CRC validation because the telegram is incomplete. The checksum and/or content values are missing.

And a warning with thousands occurrences at the end of the day.
Logger: dsmr_parser.clients.protocol
Source: /usr/local/lib/python3.9/site-packages/dsmr_parser/clients/protocol.py:136
First occurred: 15:18:59 (181 occurrences)
Last logged: 16:03:45

Invalid telegram. The CRC checksum '48812' does not match the expected '3484'
Invalid telegram. The CRC checksum '6181' does not match the expected '50447'
Invalid telegram. The CRC checksum '55319' does not match the expected '39915'
Invalid telegram. The CRC checksum '47198' does not match the expected '48688'
Invalid telegram. The CRC checksum '33152' does not match the expected '57677'

My configuration:

System Health

Hoi Nigel,

Ik wil de parser in docker installeren maar krijg de melding:

Wat te doen?

The DSMR version can automatically detected using the value in OBIS reference '0.2.8'. This allows the parser to automatically pick the correct specification.

Also update create_dsmr_protocol to have a better way in selecting serial settings and the specification/

I have been using the DSMR Slime Meter integration in Home Assistant for some time, with a Belgian electricity meter. Everything has been working fine until today. Today I get spammed by the errors:

Logger: dsmr_parser.parsers
Source: /usr/local/lib/python3.9/site-packages/dsmr_parser/parsers.py:61
First occurred: 15:27:52 (905 occurrences)
Last logged: 15:42:54

ignore line with signature \d-\d:31\.4\.0.+?\r\n, because parsing failed.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/dsmr_parser/parsers.py", line 59, in parse
    telegram[signature] = parser.parse(match.group(0))
  File "/usr/local/lib/python3.9/site-packages/dsmr_parser/parsers.py", line 201, in parse
    return CosemObject(self._parse(line))
  File "/usr/local/lib/python3.9/site-packages/dsmr_parser/parsers.py", line 151, in _parse
    raise ParseError("Invalid '%s' line for '%s'", line, self)
dsmr_parser.exceptions.ParseError: ("Invalid '%s' line for '%s'", '1-1:31.4.0(100.0*A,-100.0*A)\r\n', <dsmr_parser.parsers.CosemParser object at 0xaf84ac58>)

Any ideas?

Hi,

I have the following in my configuration.yaml

sensor:

• platform: dsmr
  port: /dev/ttyUSB1
  dsmr_version: 4.0

group:

!include groups.yaml

meter_readings:
name: Meter Waardes
entities:
- sensor.power_consumption_low
- sensor.power_consumption_normal
- sensor.power_production_low
- sensor.power_production_normal
- sensor.gas_consumption

But when i add something to my gui is see:
unknown

The EN dictionary in obis_name_mappings.py contains duplicated keys. The following keys are duplicated:

• ELECTRICITY_IMPORTED_TOTAL
• LUXEMBOURG_ELECTRICITY_USED_TARIFF_GLOBAL
• SWEDEN_ELECTRICITY_USED_TARIFF_GLOBAL

All have the same value: r'\d-\d:1\.8\.0.+?\r\n' .

• ELECTRICITY_EXPORTED_TOTAL
• LUXEMBOURG_ELECTRICITY_DELIVERED_TARIFF_GLOBAL
• SWEDEN_ELECTRICITY_DELIVERED_TARIFF_GLOBAL

All have the same value: r'\d-\d:2\.8\.0.+?\r\n'

It seems that these three different names indicate the same values? If that's the case I guess it makes sense to skip the Luxembourg and Sweden names from the obis_name_mappings. Let me know if I can be of any help, I'm willing to open a PR if that seems useful to you?

Using a Wemos D1 Mini module to connect with a SmartMeter, running Tasmota firmware forwarding data in through Serial to TCP Bridge.

Although in Home Assistant it is possible to connect with serial 2 network proxies, this fails for me with the following error:

021-02-18 17:56:16 ERROR (MainThread) [homeassistant] Error doing job: Fatal error: protocol.data_received() call failed.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 862, in _read_ready__data_received
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 91, in data_received
    data = data.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 2: ordinal not in range(128)
2021-02-18 17:56:16 ERROR (MainThread) [dsmr_parser.clients.protocol] disconnected due to exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 862, in _read_ready__data_received
    self._protocol.data_received(data)
  File "/usr/local/lib/python3.8/site-packages/dsmr_parser/clients/protocol.py", line 91, in data_received
    data = data.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 2: ordinal not in range(128)

Could this pull request be a solution for me? #70

TelegramParserV2_2 currently has logic specific to gas meter readings of DSMR 2. See if these can be moved into the specification configuration somehow.

This is in preparation of refactoring the separate parsing classes.

Recently I purchased a RFXtrxLAN with both a RFXtrx443XL and a DSMR P1 cable installed.

This device reads the telegram data from the P1 port and wraps it line-by-line into packets that fit the RF packet protocol that is normally pushed onto its USB or network connection.
One such RF packet looks like this:

• A length byte (the size of the rest of the RF packet)
• A packettype byte, this is 0x62 for "async transmit/receive", used also for the DSMR telegram lines
• A subtype byte (01 for "P1", 02 for Teleinfo, 03 for "raw" receive (not yet used)
• A sequence number byte (wraps when it gets to 255)
• The rest are data-bytes, in the case of a P1 packet a single line of telegram data.
  All of this is properly documented in the RFXcom SDK documentation.

Over the last week(s) I've been trying to get this data into Home Assistant by feeding it through the RFXtrx integration. See e.g. Danielhiversen/pyRFXtrx#125.
However, the RFXtrx integration expects one RF packet to equal a complete sensor update, including identifier. Fitting in the case where multiple "RF" packets form a single telegram that includes the serial identifiers is turning out to get complicated.

I considered starting a new Home Assistant integration that handles this particular combination of RFXtrx and DSMR and started looking at the DSMR integration to see how things are done there. I realised that it would be trivial if we could simply add a variation where the RF packets are interpreted, skipping RF packets that aren't DMSR data and collecting the telegram data for ones that are.

If I would implement this variation, would this make any chance of getting accepted into this library?