Sems Portal Nightly Midnight Zero is double dipping upsetting HA Recorder about goodwe-sems-home-assistant HOT 34 OPEN

Ok went back to my original suggestion and worked perfectly last night.

So for those looking for a workaround, the YAML code below works well, create a new automation and use this, make sure you change the sensors in the entity_id list to match your sensors. You also need to disable polling on the 'GoodWe SEMS PV API' integration (Open integration, click 3 dot menu, system options, turn off 'Enable polling for updates')

alias: GoodWe SEMS Portal Polling
description: ""
trigger:
  - platform: time_pattern
    seconds: "00"
condition:
  - condition: time
    after: "00:20:00"
    before: "23:59:00"
action:
  - service: homeassistant.update_entity
    data: {}
    target:
      entity_id:
        - sensor.homekit_xyz
        - sensor.homekit_xyz_export
        - sensor.homekit_xyz_import
        - sensor.inverter_xyz
        - sensor.inverter_xyz_energy
mode: single

from goodwe-sems-home-assistant.

Thanks for the update @davidsonimagerygmailcom

I've come up with a similar workaround for now and will update tomorrow if it has helped.
Essentially I've disabled update polling on the integration itself and setup an automation to update the entities every minute between 00:15:00 through to 00:00:30 which hopefully gets the 0kWh at midnight but skips everything between 12:01AM to 12:15AM

YAML for the automation below, you must have disabled polling on the integration first (Open integration, click 3 dot menu, system options, turn off 'Enable polling for updates') Also untested so don't do this if you're unsure, I'll provide an update tomorrow advising if I had any luck with this or not.

Automation is triggered every minute to update the entities in the list. You'll need to change these to match your entity ids, I just added all entities associated with the integration.

alias: GoodWe SEMS Portal Polling
description: ""
trigger:
  - platform: time_pattern
    seconds: "00"
condition:
  - condition: time
    after: "00:15:00"
    before: "00:00:30"
action:
  - service: homeassistant.update_entity
    data: {}
    target:
      entity_id:
        - sensor.homekit_xyz
        - sensor.homekit_xyz_export
        - sensor.homekit_xyz_import
        - sensor.inverter_xyz
        - sensor.inverter_xyz_energy
mode: single

from goodwe-sems-home-assistant.

Create a new automation as normal, then press three dots in top right and select Edit in YAML.
Paste the code replacing anything already visible and then change it back to Edit in visual editor from that top menu again.

from goodwe-sems-home-assistant.

Ok, so based on the fact that we don't really know when the data will spike after zeroing the first time and that the first zeroing may not be exactly at midnight I've modified things slightly and added a second automation. The first automation is same as above however I've removed the condition so it'll run every minute no matter what. The second automation will trigger when the entity hits 0, it will then disable the first automation wait fifteen minutes and then enable it again. This way we don't have to worry about exact timing from the SEMS side.

Automation 1

alias: GoodWe SEMS Portal Polling
description: ""
trigger:
  - platform: time_pattern
    seconds: "00"
condition: []
action:
  - service: homeassistant.update_entity
    data: {}
    target:
      entity_id:
        - sensor.homekit_xyz
        - sensor.homekit_xyz_export
        - sensor.homekit_xyz_import
        - sensor.inverter_xyz
        - sensor.inverter_xyz_energy
mode: single

Automation 2

alias: GoodWe SEMS Portal Polling Controller
description: ""
trigger:
  - platform: state
    entity_id:
      - sensor.homekit_xyz_export
    to: "0"
condition: []
action:
  - service: automation.turn_off
    target:
      entity_id: automation.goodwe_sems_portal_polling
    data:
      stop_actions: true
  - delay:
      hours: 0
      minutes: 15
      seconds: 0
      milliseconds: 0
  - service: automation.turn_on
    target:
      entity_id: automation.goodwe_sems_portal_polling
    data: {}
mode: single

from goodwe-sems-home-assistant.

Re 30 second polling, you would just need a second action, one covers 00 seconds the other covers 30 seconds.

trigger:
  - platform: time_pattern
    seconds: "00"
  - platform: time_pattern
    seconds: "30"

from goodwe-sems-home-assistant.

Another option:

trigger:
  - platform: time_pattern
    seconds: /30

from goodwe-sems-home-assistant.

Hi all, great that you've found a workaround. I'm currently limited in time and energy to work on this, but I'm happy to merge a fix if someone provides it. :)

from goodwe-sems-home-assistant.

Thanks for the automation to resolve the midnight double dip issue @skreza. I have also noticed that at around the same time this issue started, i am now seeing a gap in my house consumption between 10pm and midnight, is anyone else seeing this too? it's happening on my main SEMS homekit integration sensor sensor.homekit_91000hkuxxxxxxx. Any thoughts?

It was happening prior to disabling to my polling and moving to the automation for updating entity. I should note that my grid import stats are unaffected between 10pm and midnight which makes this ever stranger. House Consumption

Grid Import

Im also seeing this, but it is for the 'current consumption' value, not the running daily total which the energy dashboard uses. So its not impacting the $ calc on the enegy dashboard, but is annoying that the other dashboard I setup has a gap in data

from goodwe-sems-home-assistant.

Right I'm seeing weird things were house load is reading the same value for hours on end. THought it was HA, but sems portal shows the same thing. Yet, the graph is in sems portal shows the correct house load changing. Am I missing something here? Only seemed to break down today. Not a polling issue because the consumtion counters are increasing (and in HA too)

from goodwe-sems-home-assistant.

So sorry for the edits, git hub "post" button wasn't functonal (known issue with github) and I was trying to get post to work with less text (which worked).
Screenies ;

from goodwe-sems-home-assistant.

I'm also seeing this exact same issue since 28th Dec.
Agree with @davidsonimagerygmailcom that it's likely an issue on the SEMS portal side, maybe they need to configure NTP on their servers?

Unfortunately I can't see a way to use an automation to disable the SEMS service for a period of time, as that would have been a quick workaround within HA. Only other thing I can thing of as a workaround for now is to create a new template sensor in HA that copies the value of the original sensor from this integration and stop this sensor from updating between 00:00 - 00:15, you would then have to add this sensor to the energy dashboard for it to reflect correctly. Hardly ideal but would achieve the desired outcome!

Any help from @TimSoethout would be greatly appreciated!

from goodwe-sems-home-assistant.

As a workaround I've done a few things, as yet I am not 100% sure if it's sorted it (and its only a bandaid).

So the issue as ew know is the double dip ... the import kwh and export kwh all go to zero around midnight, then dip back to some form of the previous days numbers but not exactly (all of which is data coming from Sems Portal, yet doens't show up in the reports you can run from sems portal CSV export) . SemsPortal accounting bug of some sort which came in as soon as they fixed the bigger bug which was KWH metrics weren't resetting at all (for days, even on semsportal).

So the issue is HA seeing that zero'ing, followed by data - and thinks its new usage (hooray for generating power from the moon right??).

I did initially try using an automation that does a Utlility meter reset (as pictured below) ... but even so the "consumed" power would still show (and util meter calibrate doesnt apply to that).

So I went a step further and actually turned off the recorder from 11.59pm to 1230am, my idea being not recording any energy data at all therefore it disregards what it sees. In theory turning off the recorder should actually do the whole trick but i added in the zero'ing automations as a bit of a scatter-gun approach.

If this doesnt work, another idea I had was I might just disallow HA from being able to talk to the internet between midniht ad 1230 ... so it never gets exposed to the data from the double dip (it's back to zero by 1215 usually).

I've not yet confirmed if this actually works (before anyone goes following down this line).

from goodwe-sems-home-assistant.

You could even apply the logic of missing the first zero'ing entirely (as you'll get it after 1215 anyway) ... kinda why I went for 1159pm (bypass the whole shemozzle)

from goodwe-sems-home-assistant.

I wasn't sure if it needed to reset to zero at midnight for the energy dashboard to calculate it correctly.
Assuming it doesn't, then absolutely the condition could be changed to 12:15am through to 11:59pm

condition:
  - condition: time
    after: "00:15:00"
    before: "23:59:00"

from goodwe-sems-home-assistant.

Just to add spanner to the works, I notice the first zero'ing was about 00:01:22 ... then 00:02:24 back up to 10.5kwh ... 00:06:02 back to 0kwh

The kwh readings aren't the same either so not sure what's going on back end.

from goodwe-sems-home-assistant.

I wasn't sure if it needed to reset to zero at midnight for the energy dashboard to calculate it correctly. Assuming it doesn't, then absolutely the condition could be changed to 12:15am through to 11:59pm

condition:
  - condition: time
    after: "00:15:00"
    before: "23:59:00"

Good point. Not sure!
Might depend if it's total_increasing or total ... I've never deeply explored/understood

from goodwe-sems-home-assistant.

alias: GoodWe SEMS Portal Polling
description: ""
trigger:

• platform: time_pattern
  seconds: "00"
  condition:
• condition: time
  after: "00:15:00"
  before: "00:00:30"
  action:
• service: homeassistant.update_entity
  data: {}
  target:
  entity_id:
  - sensor.homekit_xyz
  - sensor.homekit_xyz_export
  - sensor.homekit_xyz_import
  - sensor.inverter_xyz
  - sensor.inverter_xyz_energy
  mode: single

Where / how do you define your YAMl code? is it on the Automation itself, or into a file and then calling it somehow?

from goodwe-sems-home-assistant.

How would we define every 30 seconds for polling instead of minute?

I wonder if ;

trigger:

• platform: time_pattern
  seconds: "00"
  seconds: "30"

is permissible

(please excuse my amateurism!)

from goodwe-sems-home-assistant.

Done and working well, i might just to go straight to using your method and see what happens tonight (it's 3pm here).

from goodwe-sems-home-assistant.

Skreza's workaround worked perfectly

Stopped polling a bit before midnight... oblivious to the double dipping being done at Goodwe's Cloud , and then at 1220 polls again to observe a zero and move on

from goodwe-sems-home-assistant.

Glad to hear it worked @davidsonimagerygmailcom Now I need to figure out why it didn't work for me! Hahaha.
Did you end up making it time based entirely or did you use the last method I posted which looked for 0 kWh and then disabled the polling automation for 15 minutes?
I tried this way and it did hit 0 however it didn't seem to trigger that automation for some reason, last triggered should be around 12 hours ago not 19! Hmmmm

from goodwe-sems-home-assistant.

Hey Mate, I just went with your first solution ... the simple one.
It just polls all day long (every 30 seconds), except betweeen 23:59 and 00:20 ... worked a treat.

from goodwe-sems-home-assistant.

Thanks for that, I might have made it more complex than it needed to be haha!
Reverted it back to time based with the condition, hopefully all good tonight!

from goodwe-sems-home-assistant.

Fingers Crossed!
All crusing along on my side. I guess ew won't actually know if the double dip gets fixed until we go back to polling during 2359 > 0020hrs

from goodwe-sems-home-assistant.

Just found this one, seems similar issue (possibly the same?)

home-assistant/core#106677

from goodwe-sems-home-assistant.

Just found this one, seems similar issue (possibly the same?)

home-assistant/core#106677

Yeeepp that's what first caught my eye ... I was perplexed how I'd made so much power from the Moon, and who was running around pulling huge loads at midnight :)

from goodwe-sems-home-assistant.

Thanks for the automation to resolve the midnight double dip issue @skreza. I have also noticed that at around the same time this issue started, i am now seeing a gap in my house consumption between 10pm and midnight, is anyone else seeing this too? it's happening on my main SEMS homekit integration sensor sensor.homekit_91000hkuxxxxxxx. Any thoughts?

It was happening prior to disabling to my polling and moving to the automation for updating entity. I should note that my grid import stats are unaffected between 10pm and midnight which makes this ever stranger.

House Consumption

Grid Import

from goodwe-sems-home-assistant.

Confirming same issue for me also, however my data is blank from 9pm - midnight, I wonder if this is time zone based? I'm in Melb so currently +11 UTC.

I hadn't noticed this as I don't look at house consumption all that much, I have another integration that pulls data from a Powerpal on my meter and tend to look more at grid import power rather than ongoing house consumption. As @justinmaiuto mentioned, the total consuming is still correct though so data in energy dashboard should still be pretty accurate.

Data in SEMS portal looks fine, pretty weird issues but unfortunately I'm no dev, so trying to fix the source code is above my expertise!

from goodwe-sems-home-assistant.

Confirming same issue for me also, however my data is blank from 9pm - midnight, I wonder if this is time zone based? I'm in Melb so currently +11 UTC.

I hadn't noticed this as I don't look at house consumption all that much, I have another integration that pulls data from a Powerpal on my meter and tend to look more at grid import power rather than ongoing house consumption. As @justinmaiuto mentioned, the total consuming is still correct though so data in energy dashboard should still be pretty accurate.

Data in SEMS portal looks fine, pretty weird issues but unfortunately I'm no dev, so trying to fix the source code is above my expertise!

so you lose 3 hours of data? weird! Im brisbane so im +10 UTC

from goodwe-sems-home-assistant.

No issue here on that 9pm to midnight bit
Perth GMT+8

The double dip still occurs though (for humour last night I turned the polling back on between 2359 > 0015, double dipping still there).

from goodwe-sems-home-assistant.

Right I'm seeing weird things were house load is reading the same value for hours on end. THought it was HA, but sems portal shows the same thing. Yet, the graph is in sems portal shows the correct house load changing. Am I missing something here? Only seemed to break down today. Not a polling issue because the consumtion counters are increasing (and in HA too)

Yep, I noticed the same in SEMS portal. The graph is ok, but the text value gets stuck. This coincides with what happens in HA.

from goodwe-sems-home-assistant.

The double dip issue is still there, tested last night.

Goodwe support (when they choose to respond to repeated followups) simply say "I can see from my end that the inverter is back online and the readings are showing.
And regarding the double dip issue is it still presisting ? because from my end i am seeing that it doesn't do that and that it restarts to zero at midnight with no problem"

Without any regard to the extensive outlining of my information prior to them. The issue doesn't appear to SemsPortal because whatever this "zero'ing" procedure is that they're doing, it's not being ingested by the sems graph / portal data.

Have repeatedly asked to have the issue escalated to the team whom actually look after the back end databases but falls on deaf ears.

back to leaving the polling off between 2359 and 0020

from goodwe-sems-home-assistant.

Deleted the last comment about 1130pm uptick, as realised internet had been down and homekit wasn't reporting usage for several hours. Disregard for anyone that read it before I deleted.

from goodwe-sems-home-assistant.

Thanks everyone for your input into this. @skreza I've implemented your automation and data looks spot on today, so thanks for sharing.

from goodwe-sems-home-assistant.