fabian-lauer / dbus-shelly-3em-smartmeter Goto Github PK

Hi,
as in forums discussed the 3EM builds no SUM over L1, L2 and L3.
so values for total returned are wrong if your PV inverter is only on one phase.
please compare the values with your meter from the distributor.
my suggustion:
at midnight set all values to zero and build your own sum over the day.
i made this for an ESP32 that reads my data from the 3EM and sends it to the solax X1 over RS485.
it works and the values in the solax portal looks rigth for me.
here is a bit of my code:

_//saldierung shelly
unsigned long time_now = millis();
unsigned long time_diff = time_now - last_time_Returned_energy_saldo;
last_time_Returned_energy_saldo = time_now;
if (power_gesamt_shelly_EM3 < 0) {
energy_returned_saldiert_Wattmillisekunde = energy_returned_saldiert_Wattmillisekunde + ((power_gesamt_shelly_EM3 * -1) * time_diff);
}

and dont forget:

  day_of_month = timeinfo.tm_mday;
  if (day_of_month != last_day){
  Serial.println("neuer tag: " + String(timeinfo.tm_mday));
  last_day = timeinfo.tm_mday;
 
  energy_returned_saldiert_Wattmillisekunde = 0; //returned energy um mitternacht zuruecksetzen,_

Hi Fabian,

thanks a lot for the script.
On my Raspi with Venus OS it's working fine.

My cousin has a Gerbo connected to a fritzbox.
in this case the Gerbo has an IP V4 and an IP V6 Address!

From a SSH Terminal on the Gerbo the Shelly with the IP v4 address is not reachable by ping (also not with your script)

The shelly seems not to able to deal with a IP v6

Do you know if I can disable IP v6 on the Gerbo?
Or any other idea?

Thx
Michael

Is there any way to implement multiple shellys?

What do you think about adding support for the 1EM shelly?

(I have a micro-inverter on the AC out on my Multiplus 2, connected via an 1EM shelly, and it would be quite cool to integrate it into my Venus stuff. - It looks straight forward to do it in a separate project, but maybe you would be interested in supporting both the 1 and the 3 channel shelly)

Due to structural conditions, I had to mount the sensors of the Shellys in the opposite direction.

It is not possible to invert the values of the sensors.

Short story: I think the script should set current power meter values to 0, whenever the meter is not reachable.

Long story:
I'm running the latest version with my Multiplus II and Venus on a raspberry pi.

Recently I've discovered my inverter pushing max power into L1, while the actual load only was around 300-500W.
As a root cause I could find a broken Wifi connection to the shelly device, the log did show a long list of exceptions.
Multiplus would still see a usage of 500 W and would push up power for compensating that ..... the meter value would
not get back to Multiplus accordingly, and Multiplus increased even more.
If a missing meter fetch attempt would have set values to 0, this would at least have prevented Multiplus from trying to compensating and emptying the battery.

Hi,

I have a Shelly Em installed on my house now.

I've recently installed a ESS with a cerbo gx and installed the driver for this.

It runs fine for an hour or so then freezes and the only way to kick it back into.action is to reboot the gx.

The shelly is still connected to network and update fine.

Any idea on what this could be or how I could fault find on it?

Cheers

Gareth

Hi,

I have a CCGX (slow CPU) and using dbus-shelly-3em-smartmeter is noticable. Even more powerful Cerbo-s will benefit from the following optimizations:

• _getConfig() gets called 2 times per update, meaning config file is open/read/parsed/closed 2 times every ~500ms.
  Consider caching configuration, and configurable update period (FR).
• for each update a new TCP connection is created
  Consider caching connection, i.e. using requests.Session object. See https://stackoverflow.com/questions/24873927/python-requests-module-and-connection-reuse

Great project!!!

Hi, I really love the integration, but somehow I am missing the total values for the AC Input in the vrm portal.
In the venusOS dashboard everything shows up, but not in VRM portal dashboard:

Is there a way to fix that?

Hi, the current.log tends to gain all the free space on partition. Deleting from time to time solves that, but if you would clear it on boot, it will be better for all others. I can do it by cron, of course.

Hi, and great work! that saves a lot of users the em24 enery meter.

it is possible to integrate every data over http api in JSON format? without high costs in my meter housing i cant integrate a em24. a shelly e3m is possible, but i have all energy data on iobroker (generation, grid relation and grid feeding). iobroker can provide with a plugin all data over http GET in JSON format. all values of my energy meters phases shown in total.

#5

Hi, erstmal top Arbeit, das erspart vielen den em24.

Mich würde interessieren ob es möglich ist, auch auf Daten von iobroker (http api) zuzugreifen? Ohne größere Umbauarbeiten am Zählerschrank (ist voll bei mir) bekomme ich kein em24 rein. Jedoch bekomme ich die Werte von Bezug, Einspeisung und Erzeugung von meinem Wechselrichter und smartmeter direkt. Diese könnten per iobroker simpleapi per http JSON zu Verfügung gestellt werden. Abtastraten ab 1 Sekunde sind dort möglich.

https://github.com/ioBroker/ioBroker.simple-api

sieht dann z.b. folgendermaßen aus:

http://192.168.2.8:8090/getPlainValue/javascript.0.Haus_Strom_Verbrauch_gesamt?json 

1158

Es wird der Bezug, Einspeisung und Eigenverbrauch saldierend anzeigt.

Vielleicht kann ich unterstützend mitwirken oder bei der Machbarkeit helfen…

Although there is not a persistent problem with the network nor with shelly 3em, the service stops after some minutes with the following message on current.log "CRITICAL Error getting data from Shelly - check network or Shelly status. Setting power values to 0".
After I restart the service manually and it starts working again for more some minutes/hour.

There is a way to make this access more resilient not being necessary o keep restarting it?

Thanks

Hallo,

habe den Shelly zum messen eines PV Inverters auf AC In eingesetzt. Allerdings ignoriert Venus OS die Werte völlig und nutzt die Leistung nicht zum Laden der Batterie.

Muss ich im ESS Menü externe Zähler auswählen damit das funktioniert? Das Problem ist bloß, dass die Victrons dann auf Durchleiten schalten.

Grüße

Hi There,
this Project sounds quite interesting!
In my setup iobroker works as middleware between Smartmeter and Multiplus 2 (VenusOS running on Raspberry Pi).
I would like to try to dockerize this application, but I don't have any experiences doing it.
https://www.geeksforgeeks.org/how-to-run-a-python-script-using-docker/

Has someone else tried this way?

eventuell ein Problem mit VenusOS 3.0 ?

aber Energy Meters ist leer:

und ESS steuert deswegen nicht.

hi, ive adopted your script to read json data from a Tasmota 12.4 with hichi ir read head.

ive no python skills and changed only the parts to get the JSON data.

it works, but I get after 4 hours or sometimes after 2 days no updates, the last value hangs and I had to restart the script to work again.

maybe you can look at it and find a error in my changing?

thank you very much

`#!/usr/bin/env python
import platform
import logging
import sys
import os
if sys.version_info.major == 2:
import gobject
else:
from gi.repository import GLib as gobject
import sys
import time
import requests # for http GET
sys.path.insert(1, os.path.join(os.path.dirname(file), '/opt/victronenergy/dbus-systemcalc-py/ext/velib_python'))
from vedbus import VeDbusService
class DbusMT681Service:
def init(self, servicename, paths, productname='MT681', connection='MT681 JSON service'):
deviceinstance = 42
customname = 'MT681-Netz'
self.dbusservice = VeDbusService("{}.http{:02d}".format(servicename, deviceinstance))
self._paths = paths
logging.debug("%s /DeviceInstance = %d" % (servicename, deviceinstance))
# Create the management objects, as specified in the ccgx dbus-api document
self._dbusservice.add_path('/Mgmt/ProcessName', file)
self._dbusservice.add_path('/Mgmt/ProcessVersion', 'Unkown version, and running on Python ' + platform.python_version())
self._dbusservice.add_path('/Mgmt/Connection', connection)
self._dbusservice.add_path('/DeviceInstance', deviceinstance)
self._dbusservice.add_path('/ProductId', 0xFFFF) # id assigned by Victron Support from SDM630v2.py
self._dbusservice.add_path('/ProductName', productname)
self._dbusservice.add_path('/CustomName', customname)
self._dbusservice.add_path('/Connected', 1)
self._dbusservice.add_path('/Role', 'grid')
self._dbusservice.add_path('/Latency', None)
self._dbusservice.add_path('/FirmwareVersion', 0.1)
self._dbusservice.add_path('/HardwareVersion', 0)
self._dbusservice.add_path('/Position', 0) # normaly only needed for pvinverter
self._dbusservice.add_path('/Serial', self._getSerial())
self._dbusservice.add_path('/UpdateIndex', 0)
self._dbusservice.add_path('/StatusCode', 0) # Dummy path so VRM detects us as a PV-inverter.
# add path values to dbus
for path, settings in self._paths.items():
self._dbusservice.add_path(
path, settings['initial'], gettextcallback=settings['textformat'], writeable=True, onchangecallback=self._handlechangedvalue)
# last update
self._lastUpdate = 0
# add _update function 'timer'
gobject.timeout_add(250, self._update) # pause 250ms before the next request
# add _signOfLife 'timer' to get feedback in log every 5minutes
gobject.timeout_add(self._getSignOfLifeInterval()601000, self._signOfLife)
def _getSerial(self):
meter_data = self._getData()
if not meter_data['StatusSNS']['mt681']['Meter_id']:
raise ValueError("Response does not contain 'mac' attribute")
serial = meter_data['StatusSNS']['mt681']['Meter_id']
return serial
def _getSignOfLifeInterval(self):
value = 1
if not value:
value = 0
return int(value)
def _getData(self):
URL = "http://192.168.100.23/cm?cmnd=status%208"
meter_r = requests.get(url = URL)
# check for response
if not meter_r:
raise ConnectionError("No response from mt681 - %s" % (URL))
meter_data = meter_r.json()
# check for Json
if not meter_data:
raise ValueError("Converting response to JSON failed")
return meter_data
def _signOfLife(self):
logging.info("--- Start: sign of life ---")
logging.info("Last _update() call: %s" % (self._lastUpdate))
logging.info("Last '/Ac/Power': %s" % (self._dbusservice['/Ac/Power']))
logging.info("--- End: sign of life ---")
return True
def _update(self):
try:
#get data from Device
meter_data = self._getData()
self._dbusservice['/Ac/Power'] = meter_data['StatusSNS']['mt681']['Power_curr']
self._dbusservice['/Ac/L1/Voltage'] = 230
self._dbusservice['/Ac/L2/Voltage'] = 230
self._dbusservice['/Ac/L3/Voltage'] = 230
self._dbusservice['/Ac/L1/Power'] = meter_data['StatusSNS']['mt681']['Power_p1']
self._dbusservice['/Ac/L2/Power'] = meter_data['StatusSNS']['mt681']['Power_p2']
self._dbusservice['/Ac/L3/Power'] = meter_data['StatusSNS']['mt681']['Power_p3']
self._dbusservice['/Ac/L1/Current'] = meter_data['StatusSNS']['mt682']['Power_p1'] /230
self._dbusservice['/Ac/L2/Current'] = meter_data['StatusSNS']['mt682']['Power_p2'] /230
self._dbusservice['/Ac/L3/Current'] = meter_data['StatusSNS']['mt682']['Power_p3'] /230
self._dbusservice['/Ac/Energy/Forward'] = meter_data['StatusSNS']['mt681']['Total_out']
self._dbusservice['/Ac/Energy/Reverse'] = meter_data['StatusSNS']['mt681']['Total_in']
#logging
logging.debug("House Consumption (/Ac/Power): %s" % (self._dbusservice['/Ac/Power']))
logging.debug("---");
# increment UpdateIndex - to show that new data is available
index = self._dbusservice['/UpdateIndex'] + 1 # increment index
if index > 255: # maximum value of the index
index = 0 # overflow from 255 to 0
self._dbusservice['/UpdateIndex'] = index
#update lastupdate vars
self._lastUpdate = time.time()
except Exception as e:
logging.critical('Error at %s', '_update', exc_info=e)
# return true, otherwise add_timeout will be removed from GObject -
# see docs http://library.isr.ist.utl.pt/docs/pygtk2reference/gobject-functions.html#function-gobject--timeout-add
return True
def _handlechangedvalue(self, path, value):
logging.debug("someone else updated %s to %s" % (path, value))
return True # accept the change
def main():
try:
logging.info("Start");
from dbus.mainloop.glib import DBusGMainLoop
DBusGMainLoop(set_as_default=True)
#formatting
_kwh = lambda p, v: (str(round(v, 2)) + 'KWh')
_a = lambda p, v: (str(round(v, 1)) + 'A')
_w = lambda p, v: (str(round(v, 1)) + 'W')
_v = lambda p, v: (str(round(v, 1)) + 'V')
#start our main-service
pvac_output = DbusMT681Service(
servicename='com.victronenergy.grid',
paths={
'/Ac/Energy/Forward': {'initial': 0, 'textformat': _kwh}, # energy bought from the grid
'/Ac/Energy/Reverse': {'initial': 0, 'textformat': _kwh}, # energy sold to the grid
'/Ac/Power': {'initial': 0, 'textformat': _w},
'/Ac/Current': {'initial': 0, 'textformat': _a},
'/Ac/Voltage': {'initial': 0, 'textformat': _v},
'/Ac/L1/Voltage': {'initial': 0, 'textformat': _v},
'/Ac/L2/Voltage': {'initial': 0, 'textformat': _v},
'/Ac/L3/Voltage': {'initial': 0, 'textformat': _v},
'/Ac/L1/Current': {'initial': 0, 'textformat': _a},
'/Ac/L2/Current': {'initial': 0, 'textformat': _a},
'/Ac/L3/Current': {'initial': 0, 'textformat': _a},
'/Ac/L1/Power': {'initial': 0, 'textformat': _w},
'/Ac/L2/Power': {'initial': 0, 'textformat': _w},
'/Ac/L3/Power': {'initial': 0, 'textformat': _w},
})

  #logging.info('Connected to dbus, and switching over to gobject.MainLoop() (= event based)')
  mainloop = gobject.MainLoop()
  mainloop.run()            

except Exception as e:
logging.critical('Error at %s', 'main', exc_info=e)
if name == "main":
main()
`

I have connected all load and PV at AC out. My Growatt inverter is measured by the 3EM, so I configured it at the AC OUT.
What happens is that VRM subtracts the PV power from the AC Load. So the AC Load does not shows the real power.

When I connect the PV to the AC IN and the load to AC OUT and I configure the Shelly as connected to AC IN, the grid power is not shown correctly.

The Grid Metering in the ESS menu is set to Inverter/Charger. I don't have a grid meter

What am I doing wrong?

there is a problem with the service when the raspberry reboot
Then there are no values ​​from the dbus-shelly-3em-smartmeter at "AC input" and device list shows no 3EM service.

Hi Fabian,

i was not able to create a branch to fix it on my own. Here the fix:

Changing to value from "power" to "total" for Forward like:
self._dbusservice['/Ac/L1/Energy/Forward'] = meter_data['emeters'][0]['total']/1000 if meter_data['emeters'][0]['total'] > 0 else 0
Changing to value from "power" to "total_returned" for Reverse like:
self._dbusservice['/Ac/L1/Energy/Reverse'] = (meter_data['emeters'][0]['total_returned']/1000) if meter_data['emeters'][0]['total_returned'] < 0 else 0

This change works on my setup like expected.

BR,
Viktor

Hallo,
ich habe den Shelly Pro 3EM nun unter Venus Os zum laufen bekommen,
Jedoch wird er unter Energy Meter nicht angezeigt.

Hier wird er aber nicht angezeigt

Hi!

as dbus-shelly is now, I expect every user using it will run into trouble (full data partition, a really bad situation) at some point.

see see henne49/dbus-opendtu#173 for how I recommend to fix it.

Matthijs / Victron

Hello,

is it possible to integrate the Shelly 1pm plus also as smartmeter with this script?

Hallo,
wäre es möglich den pro auch zu implementieren?

lg Heinz

After reboot shelly3em shows up again.

Hello,

it is not working with a shelly 3em pro, because the status page is not available: Message "Not Found".
Seems to be that a shelly 3em pro don't have a status page.

Thanks
Best Regards

Marco Eberhardt

Hi!
Hope you have time to change the script for a Shelly EM.
Here is the output for it:
{"power":2724.16,"reactive":-1050.38,"pf":-0.93,"voltage":221.42,"is_valid":true,"total":12716863.2,"total_returned":1742722.0}

It seems when using the Venus OS Large Image v2.93 there is a problem with the service when the Raspberry was shutted down and then started a few hours later.
Then there are no values from the dbus-shelly-3em-smartmeter at "AC-Eingang" and "AC-Lasten" and dbus-spy shows no 3EM service.