utilitechas / amsreader-firmware Goto Github PK

When saving MQTT configuration, the webserver authentication settings is deleted.

Include in the MQTT JSON. Also show in web gui

Ends up using the default ESP8266 profile. Change this to a custom ESP32 profile

Add a firmware tab where the user can upload a new firmware. Look into how to handle firmware upgrade from uploaded file. Ideally, disconnect wifi and go to full modem sleep while upgrading to preserve power. Maybe then it is possible to upgrade HAN powered devices as well.

Store and show the cumulative values from the hourly meter updates.
Also show active export.

If the user configures a incorrect MQTT config or the MQTT server goes offline, the web gui is unavailable.

Add configuration option to calculate the missing value

The name of the second meter ("Production") is misleading.
This is not the production but the exported power, which is (Production - SelfConsumption).

I am not sure which is the best / most intuitive term to use here.

• Exported power ?
• Delivery to grid (and change the first meter to "Consumption from grid") ?
• Delivery ?
• (Any other ideas ?)

Implement RemoteDebugger into the code, but default to off. Add a configuration panel for debugging. Should be possible to choose if debugging should be sent to serial, telnet or both. Option to choose debugging level.

When clicking the tick mark box, the entry of parameters is enabled, but the "Enable" tick mark or button does not change.

If you have set a configuration and then trigger AP mode, the web server will retain the authentication settings. Make web server aware of AP mode and disable authentication

I am using NodeRed to link the data into Home Assistant, and noted that you changed the string format recently, but do not know exactly when:

String provided in February 2020:
""temp":18.5,"lv":"KFM_001","id":"6970631402281XXX","type":"MA304H3E","P":6947,"Q":81,"I1":20.558,"I2":13.881,"I3":25.14,"U1":210.6,"U2":0,"U3":202.6,"tPI":103184047,"tPO":0,"tQI":4831895,"tQO":255935}}"
Please note that the beginning of the string is truncated.

String Provided in version1.1.1:
"{"id":"EC:FA:BC:4C:XX:XX","up":274830607,"t":1585533610,"vcc":3.437,"rssi":-47,"temp":22.87,"data":{"rtc":1585533610,"tPI":1.134553e8,"tPO":0,"tQI":5017038,"tQO":303751,"lv":"KFM_001","id":"6970631402281XXX","type":"MA304H3E","Q":0,"PO":0,"QO":327,"I1":20.93,"I2":21.829,"I3":29.34,"U1":215.5,"U2":0,"U3":213.3,"P":8834}}"

I have re-structured most of the data, as the "temp" reading is now handled differently. But this is not a problem.

However the data for "tPI" is a problem, I use this data to calculate aggregated power consumption and link this to price from nor-pool to calculate price per hour.

In the earlier versions the full string was given (103184047), and now only (1.134553e8).

Can you please consider giving the full figure with all digits ?
Please note that capturing this figure is a bit of a hassle, since the reader only prints it once every hour.

I am using the original Roar Fred HW. So I have used the HW-01 version of the image.
Meter is Kaifa.

When my solar panel kicks in and I have net power export the GUI signals HAN error (red indicator), and no export data is indicated:

MQTT messages are still flowing, and my Node-Red implementation shows export power correctly.
As soon as net import is back, the red HAN light goes green, and the GUI behaves correctly, showing imported power.

The serial output that shows IP address for GUI is sent with NO parity, while (in my setup) debug data on serial output is sent with EVEN parity.
Should be same parity.

When production and exporting energy the current value is displayed incorrectly.

Now that most of the data is presented in UI, lets organize it all better and add some explaining text etc.

Allow the user to select an available IP address for the web GUI as part of the configuration.
Should them be set as static.

Hi. I tried to find some instructions on how to set this up. After flashing the image to an ESP8266 and rebooting it I detect the ESP wifi for a short period but I'm unable to connect to it. I tried to look for more instructions here and on @roarfred's repo but I was not able to find any more info.

The unit for the figure returned by EDP.getVcc() is 1/1024 V
So to get the 'correct' voltage in V you need to divide by 1024.
Like:
float vccVolt = ((float)ESP.getVcc())/1024;

The configuration page is now only available when in AP mode. Make this page available in default mode so that configuration can be changed more easily.

Also, add option to password protect the configuration page.

feather esp32:
The latest version fails on boot if complied with -D DEBUG_MODE=1.
-Tested both with clean flash and existing configuration.
Version 1.1.2 is ok.

Trace:

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5828
entry 0x400806ac
/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:1442 (xQueueGenericReceive)- assert failed!
abort() was called at PC 0x40088d55 on core 1

Backtrace: 0x4008c920:0x3ffb1ca0 0x4008cb51:0x3ffb1cc0 0x40088d55:0x3ffb1ce0 0x4015480e:0x3ffb1d20 0x40154aea:0x3ffb1d40 0x40143228:0x3ffb1d60 0x40143291:0x3ffb1d80 0x4013ecea:0x3ffb1da0 0x4013d714:0x3ffb1dc0 0x4014218c:0x3ffb1e00 0x4015efab:0x3ffb1e20 0x400e4dc5:0x3ffb1e60 0x400e4fab:0x3ffb1e80 0x400d71c2:0x3ffb1eb0 0x400f00cb:0x3ffb1fb0 0x40089069:0x3ffb1fd0

Rebooting...
ets Jun 8 2016 00:22:57

Remove the AP mode trigger from boot and instead make it so that it can be triggered at any time. This is mainly to save power for self powered devices, as the boot consumes a lot more power.

At the same time, add option to do full factory reset from web gui. Require re-enter of password for this.

Image and link to device is wrong. Also add some information on how to wire it

On bootup, the voltage is checked and sends device into deep sleep if it is too low. This is to make sure BUS powered devices has enough power to boot. This does not work well with all devices, so maybe this should be configurable.

I have tried your code, and somtimes it workes fine for a few hours and then it stops and goes into a loop, see picture. Sometimes it goes directly into this loop.

Do you have any idea of what is wrong, or what to check?

I have other traffic going to the same broker, but it is stable.

I also have been using the orginal roarfred code with the PubSubClient.h and it worked fine, just that i was not able to get kwh readings out.

In web ui:

• Show RSSI (and other details?) from WiFi
• Show MQTT connected or not as well as last error
• Show VCC (and other details?) from ESP
• Show last error from HAN

Set hostname and also start a mDNS service for easier access to device

Hi, thanks for keeping this project alive, and I really like your new WEB GUI.

I had version 0.2.0 working in January and February, using Kaifa meter.
Utility company is BKK in Bergen (where I live).

Everythink worked until resently, when I noted at the data as corrupt.
I used the AMS bridge to MQTT and use a NodeRed flow to extract the data into Home Assistant.

Now I do not have any valid data, I have checked Mosquitto and it seems the data from the meter have changed.

Tried to upgrade to your 1.1.0 version, but here I have no data at all. All "buttons" are green except HAN bus (red), but I have access to the WEB GUI.
There are no data sent to MQTT either.

I have two questions:

• Can you include a "dubug" enable in the WEB GUI ? it would make it easier to enable?
• Or possibly a "transparent" mode where the raw HAN data is simply passed to MQTT?
  In version 0.2.0 I have MQTT data, but the format looks different.
• Are you aware of any protocol changes executed by any of the power companies lately?

Thanks in advance.
Kjell

Change from turning ON/OFF before/after reading to just flashing 50ms.

Kamstrup HAN-NVE interface description_rev_3_1.pdf
According to NVE documentation (attached), the Kamstrup sends two lists: List 1 with 10 sec interval, List 2 with 1 hour interval. List 2 contains cumulative meter data.
The current code (downloaded 12feb20) does not read List 2.
The kamstrup.h file indicates List1 and List3 (not List2), and both seems to collect cumulative data?
This kamstrup.h file has previously worked correctly on my setup:

// Kamstrup.h

#ifndef _KAMSTRUP_h
#define _KAMSTRUP_h


enum class Kamstrup
{
	List1 = 0x19,
	List2 = 0x23
};

enum class Kamstrup_List1
{
	ListSize,
	ListVersionIdentifier,
	MeterID_OBIS,
	MeterID,
	MeterType_OBIS,
	MeterType,
	ActiveImportPower_OBIS,
	ActiveImportPower,
	ActiveExportPower_OBIS,
	ActiveExportPower,
	ReactiveImportPower_OBIS,
	ReactiveImportPower,
	ReactiveExportPower_OBIS,
	ReactiveExportPower,
	CurrentL1_OBIS,
	CurrentL1,
	CurrentL2_OBIS,
	CurrentL2,
	CurrentL3_OBIS,
	CurrentL3,
	VoltageL1_OBIS,
	VoltageL1,
	VoltageL2_OBIS,
	VoltageL2,
	VoltageL3_OBIS,
	VoltageL3
};


enum class Kamstrup_List2
{
	ListSize,
	ListVersionIdentifier,
	MeterID_OBIS,
	MeterID,
	MeterType_OBIS,
	MeterType,
	ActiveImportPower_OBIS,
	ActiveImportPower,
	ActiveExportPower_OBIS,
	ActiveExportPower,
	ReactiveImportPower_OBIS,
	ReactiveImportPower,
	ReactiveExportPower_OBIS,
	ReactiveExportPower,
	CurrentL1_OBIS,
	CurrentL1,
	CurrentL2_OBIS,
	CurrentL2,
	CurrentL3_OBIS,
	CurrentL3,
	VoltageL1_OBIS,
	VoltageL1,
	VoltageL2_OBIS,
	VoltageL2,
	VoltageL3_OBIS,
	VoltageL3,
	MeterClock_OBIS,
	MeterClock,
	CumulativeActiveImportEnergy_OBIS,
	CumulativeActiveImportEnergy,
	CumulativeActiveExportEnergy_OBIS,
	CumulativeActiveExportEnergy,
	CumulativeReactiveImportEnergy_OBIS,
	CumulativeReactiveImportEnergy,
	CumulativeReactiveExportEnergy_OBIS,
	CumulativeReactiveExportEnergy
};


#endif

Seems to be a bug in the raw values.
The "vcc" has the value from the "temp" and "temp" header is missing. I think you see this more clearly in the attached screenshot containing both json and raw.

I'm running ams2mqtt-d1mini-1.2.1.bin, by the way. Had the same bug in 1.2.0. Haven't tested any other releases.

Is it possible for the web GUI to push an update to the browser for each new update from the AMS meter?

Add option to either send the data to MQTT as JSON (as current version) or raw values

......
data["QO"] = hanReader.getInt( (int)Aidon_List3PhaseIT::ReactiveExportPower);
data["I1"] = ((double) hanReader.getInt( (int)Aidon_List3PhaseIT::CurrentL1)) / 10;
data["I1"] = 0;
........
Last line shall be
data["I2"] = 0;

Whenever Import Power is zero and Export Power positive (e.g. while solar panels produce more than the self-consumption), the GUI meter shows nothing:

Improvement options:

1. In the GUI Configuration, make an option to provide local production installation kWp (kW peak) power, and let the meter scale go from minus [Installed_kWp] to plus [main fuse power].
   (And if possible green meter dot when delivering to grid, red dot when consuming from grid.)
2. Beside the "Consumption" meter, a separate meter "Production" that is non-zero when delivering to grid.

ESP uptime is direct readout of millis(), which has data type unsigned long, ranging from 0 to 4,294,967,295.
So it will roll over to zero after approximately 50 days.

Support configuration of Meter config, MQTT and web config without restarting ESP.

Add a new root page showing readings for all meter values. Refresh every 2.5/10s.

Move config page to different path. Add option on webserver to require authentication for all or only config.

Add some code to only send if the values have changed.
Motivation: Preserve power. Not everyone have a low traffic, high DTIM network.
Maybe it doesn't matter much?
Maybe optional thresholds makes this a valid enhancement?

Needs testing

Using incorrect system variable in GitHub Actions script.

A great feature to those who want to be able to look at live data from the meter, but does not have a MQTT to publish to.

Default to UART0 for ESP8266 and UART2 for ESP32, but make it configurable with software serial to a predefined set of GPIO pins. This will reduce the number of firmwares from six to two.

Hello,

Thanks for keeping this project alive!
I have Aidon 3Phase IT meter (6525) and i have this running on a Wemos D1 mini with MQTT to my main server.

I get all of the values defined in case 1 and 2 in the HanToJson.cpp but the values from case 3 is not transfered. Have you encountered this?

This is the values that are not beeing sent on the MQTT:

case 3:
jd["rtc"] = data.getMeterTimestamp();
jd["tPI"] = data.getActiveImportCounter();
jd["tPO"] = data.getActiveExportCounter();
jd["tQI"] = data.getReactiveImportCounter();
jd["tQO"] = data.getReactiveExportCounter();

All of the values show up in the WebUI, so they are being transfered from the meter to the Wemos. (Except rtc value)

Easy fix or bug from Aidon?

If configuration is missing, try to read from M-Bus and detect what type of meter is connected. If nothing received in 8E1 within reasonable timeframe, try 8N1 (Kamstrup) for a bit of time. Loop this procedure until meter is either detected or set by user.

Before reading the temperature, check if we have a temperature sensor. Let this be activated for all boards with different pins

From my Kamstrup meter I receive at 11:19 the timestamp 1582715950 in MQTT, which according to https://time.is/Unix_time_converter translates to:
Wed Feb 26 2020 12:19:10 UTC+0100 (Central European Standard Time)

Which seems to be 1 hour off.

According to https://github.com/gskjold/AmsToMqttBridge/blob/master/doc/NVE_Info_kunder_HANgrensesnitt.pdf
the timestamp shall be local time in Norway, taking into account daylight saving time (DST).
But the timestamp from my meter seems to be at DST already?

• Is the MQTT timestamp raw data from the meter?
• Do users of other meters / meter brands see the same?

Feature request

Add another gaugemeter with temperature display for @roarfred based boards.