Netatmo integration: API error: 500 (Internal Server Error) about core HOT 61 OPEN

https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

It won't last very long. It is due to the API changes of Netatmo (see https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370). The integration need to be fixed.

from core.

Same here. I have had the issue since last night

from core.

I wonder if this is related:

https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

from core.

Same problem here. For me it coincided with switching from sd to ssd, but I don't think it's linked.

from core.

same problem

from core.

I sorted out my issue by deleting my hub and re-adding it again. I noticed that some of the entities also hand slight name changes.

from core.

Bonjour, même problème pour moi également lors d'une maj de HA.

from core.

I have the same Error

from core.

I have the same Problem since installation of

Core
2024.6.3
Supervisor
2024.06.0
Operating System
12.4
Frontend
20240610.1

Logger: homeassistant.components.netatmo
Quelle: components/netatmo/init.py:90
Integration: Netatmo (Dokumentation, Probleme)
Erstmals aufgetreten: 10:53:16 (7 Vorkommnisse)
Zuletzt protokolliert: 10:57:19

API error: 500 (Internal Server Error)

from core.

Sorry, should've read the logger docs. I'll try to configure the logger for the Netatmo integration and see if I can get more details – at least more than HTTP 500 (which actually by RFC definition is a server, and not a client error… but I don't trust it in this case ^^).

from core.

Hey there @cgtobi, mind taking a look at this issue as it has been labeled with an integration (netatmo) you are listed as a code owner for? Thanks!

_{^{(message by CodeOwnersMention)}}

netatmo documentation
netatmo source
_{^{(message by IssueLinks)}}

from core.

Same problen for me. It seems to be connectef with the last Home Assistant Core update to 2024.6.1

from core.

Same here, upgraded from 2024.3.3 to 2024.6.1 and the integration did not want to work afterwards.
I'm using the docker version of HA.

I also tried downgrading to 2024.6.0 but that did not help unfortunately.

from core.

I found a Solution (?)

—>

Authentication problem solved as follows:

I called my own account at https://home.netatmo.com/ 3
Here is the menu My Account/Account Management/Third Party Apps
There is a list of applications connected to the account
Click into the “Home Assistant Cloud” app, select “Remove App”
In Home assistant I brutally deleted the Netatmo integration
I restarted HA
I called wizzard “Add Integration” again, selected Netatmo
On the Netatmo web page called by HA, I enabled the third party application

from core.

I found a Solution (?)

—>

Authentication problem solved as follows:

I called my own account at https://home.netatmo.com/ 3 Here is the menu My Account/Account Management/Third Party Apps There is a list of applications connected to the account Click into the “Home Assistant Cloud” app, select “Remove App” In Home assistant I brutally deleted the Netatmo integration I restarted HA I called wizzard “Add Integration” again, selected Netatmo On the Netatmo web page called by HA, I enabled the third party application

This worked for me too but since it involved losing all historical data it's a less than ideal solution

from core.

I'm unsure if this is related to newest HA version to be honest. It started suddenly for me, and I haven't updated to 2024.6.x yet, and run 2024.5.5.

If I delete the integration and re-add it, it will work again for anything between 2 to 12 hours.

from core.

API error: 500 (Internal Server Error)
12:31:58 – (WARNING) Netatmo - message first occurred at June 9, 2024 at 09:31:50 and shows up 1203 times

Same issue on my side. Yes, did update to the latest version recently, but also hear that others have no issue having updated. Seems to support @Daniel-Joergensen’s comment.

from core.

Problem here also. I get this:
Failed setup, will retry: 500, message='Internal Server Error', url=URL('https://account-link.nabucasa.com/refresh_token/netatmo')

from core.

I wonder if this is related:

https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

Hmm. If so, somebody here who knows how to deal with this?
@cgtobi is that an issue which you need to solve?

from core.

Same problem here. Still in HA version 2024.5.3

from core.

I sorted out my issue by deleting my hub and re-adding it again. I noticed that some of the entities also hand slight name changes.

That has only helped for me, for a few hours, then it will stop working again

from core.

Same issue here.

from core.

Update 6.1 caused other, even bigger, problems on my Green in addition to Netatmo.
I went back to 6.0, deleted Netatmo and then set it up again.
Now everything has been running stable for over 72 hours, now also with 6.3.

from core.

Same error here

from core.

I have removed my integration and-readded it again. All works now, gotta wait for how long. Annoyingly the values are now in Dutch (which I am), but all the old ones I had renamed. Have renamed them now back to what Netatmo returns and lets see how it goes. Does mean all history is gone.

from core.

I have removed my integration and-readded it again. ... Does mean all history is gone.

Same in my installation. Sadly, history values are gone now. But hey, it's just weather...

from core.

Same issue here.

from core.

Same issue here

from core.

Same issue, worked one day, woke up to it not working at all.

I'd really like to avoid losing my historical data.

from core.

Hi, same hier since yesterday.

from core.

I found a Solution (?)

I could (temporarily) resolve the issue by deleting and re-adding the integration. However, today the integration fails again with the same error. Deleting and Re-adding the integration is NOT a permanent solution!

from core.

Bonjour, pour ma part, en supprimant l'extension et en la réinstallant, j'ai seulement perdu les historiques mais l'intégration fonctionne à nouveau parfaitement depuis 2 semaines maintenant.

from core.

Same error here, and I'm on 2024.6.4 :/

Please note I've been on 2023.12.xx before, and it already happened there. The update to 2024.6.4 didn't help.

from core.

Are we sure this isn't related to this change?

When you refreshed an access_token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers responded with a couple of tokens : an access_token and a refresh_token. If the previous access_token was still valid, the refresh_token value was never renewed

Starting from now, this behavior changed to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users

When refreshing tokens, access_token and refresh_token values will be automatically renewed and former tokens invalidated

So, if you do not update and use the new refresh_token value when refreshing your access_token, your users will be disconnected after 3 hours and you will retrieve an “invalid_grant” error

To fix it, you need to update the tokens value as soon as you get the newly generated ones

from core.

Is there any solution already how to fix this? Can soneone explain in detail what to do?
Thanks a lot :)

Are we sure this isn't related to this change?

When you refreshed an access_token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers responded with a couple of tokens : an access_token and a refresh_token. If the previous access_token was still valid, the refresh_token value was never renewed
Starting from now, this behavior changed to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users
When refreshing tokens, access_token and refresh_token values will be automatically renewed and former tokens invalidated
So, if you do not update and use the new refresh_token value when refreshing your access_token, your users will be disconnected after 3 hours and you will retrieve an “invalid_grant” error
To fix it, you need to update the tokens value as soon as you get the newly generated ones

from core.

I have removed my integration and-readded it again. All works now, gotta wait for how long. Annoyingly the values are now in Dutch (which I am), but all the old ones I had renamed. Have renamed them now back to what Netatmo returns and lets see how it goes. Does mean all history is gone.

Update from my side. After re-adding it it has worked stable since then. Running on all latest versions.

from core.

@RemBrandNL, does it still work after restarting the HA hardware?

from core.

@Fotokroth Yes. My HA restarts every das and it runs without Problems since 2 weeks

from core.

@RemBrandNL, do you mean the system, i.e. the hardware or just the HA?

from core.

@RemBrandNL, do you mean the system, i.e. the hardware or just the HA?

Just did a full reboot of the hardware for you, still available.

from core.

I have removed my integration and-readded it again. All works now, gotta wait for how long. Annoyingly the values are now in Dutch (which I am), but all the old ones I had renamed. Have renamed them now back to what Netatmo returns and lets see how it goes. Does mean all history is gone.

Update from my side. After re-adding it it has worked stable since then. Running on all latest versions.

means you lost all your history, right?

from core.

If you don't restart HA in between your history should not be cleared.

from core.

Hey,

is this the recommended way? Or is there a possible solution with an update coming?

I‘m asking because I have seen entity ids change in the past …

thanks

from core.

@RemBrandNL, Thank you. I just updated to 2024.7.1 and after the system restart, the Netatmo devices were still active on my device.

from core.

@maehmann, recommended, probably not. Took this route based on the posts of @cyhyryiys and @Ph0B1uS earlier in this thread. Main reason being that I don’t see ‘official’ acknowledgement of the issue and no fixes on the way if needed.

With re-adding my entities were messed up and couldn’t bother aligning them to keep my history. It’s “just” weather data and I don’t run automations that need historical data.

2 cents: I’m no dev, but there’s a chance that HA implementation is correct and the changes netatmo side just cause a one time fuckup. Seems to have to do with renewing the tokens to keep access to netatmo data. Integration misses a re-authentication on an existing connection for incidents like this it seems. If the cycle of getting a token with a proper renewal token the process will continue again as it’s supposed to do.

Cheers,
Remco

from core.

I removed the integration and readded it. That worked but when I chose Netatmo directly the authentication fails. Only the implementation through NabuCase worked.
Does that mean that the communication now works like Netatmo <=> NabuCasa <=> My HA? If that is the case I wouldn't really like that, to be honest. I would rather want Netatmo <=> My HA.

from core.

No data is processed by NabuCasa. It's just that the cloud authentication is used.

from core.

@cgtobi do you plan to fix this issue?
My integration is still not working and to be honest I don’t have a clue how to fix this on my own 😕

from core.

Of course, if only I'd have a clue what the issue would be. I cannot reproduce and the integration is utilising the authentication mechanism of HA.

from core.

Of course, if only I'd have a clue what the issue would be.

Isn’t it about this?
https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

from core.

If that would be the case wouldn't everyone have to have the same issue?

from core.

Of course, if only I'd have a clue what the issue would be. I cannot reproduce and the integration is utilising the authentication mechanism of HA.

I checked the Netatmo HA integration for a few minutes, but couldn't figure out how the whole oauth thingy works, resp. saw that it's using HA's internals to do it. Can we check the HTTP 500 response, resp. the payload of it? I'm a Python developer, but didn't do much with HA :/

Of course, if only I'd have a clue what the issue would be.

Isn’t it about this? https://helpcenter.netatmo.com/hc/en-us/community/posts/19320250276626/comments/19329138660370

I already mentioned that above, seems it can be related… 

If that would be the case wouldn't everyone have to have the same issue?

tbh, I didn't had the problem until I restarted HA. Other people seem to have the same issue (after restarting). Some said it works after re-adding the integration, some claimed it failed again after a few hours or days.

from core.

So, this is going to be hard to debug, because:

1. Activating debug for the Netatmo integration didn't help
2. The HTTP 500 came from NabuCasa, and not Netatmo directly

Debugging the error with those 2 limitations (esp. the last one) is quite hard. I suspect (or hope?) the Netatmo API will give a more useful error, but NabuCasa just throws away that «origin & hopefully useful error message» and shows us a HTTP 500 instead (similar to this issue – not the cause, but the error handling) 😄. This is bad error handling practice par excellence (from the HA core / NabuCasa cloud, not the Netatmo integration), but there might be a reason for the obscurity :/

At least when there's an error on the NabaCasa integration, then a HTTP 500 might be «correct» in that context. Still annoying to debug. Or does anyone know if I can access «my» NabuCasa logs?

from core.

But, is it really related to authenticating via NabuCasa? I understood that also people with individual Netatmo dev accounts had that issue. I just tried both setups on my dev environment without issues which is not helping either. Nor has my production env (dev account) any issues of that sort. So It is really difficult to find a relation here.

from core.

But, is it really related to authenticating via NabuCasa? I understood that also people with individual Netatmo dev accounts had that issue. I just tried both setups on my dev environment without issues which is not helping either. Nor has my production env (dev account) any issues of that sort. So It is really difficult to find a relation here.

I agree, it's a strange case, and quite hard to debug, or even to find any relations. I think it's related to the token renewal, at least that's the endpoint which is throwing an HTTP 500:

Failed setup, will retry: 500, message='Internal Server Error', url=URL('https://account-link.nabucasa.com/refresh_token/netatmo')

Of course it can be a side-effect of something else, but right now everything is pointing in that direction, and we don't have anything else to start with :/

Since I'm using HA core with venv's, I can easily clone my live venv to my test venv, then remove & re-add the Netatmo integration to see if it works (again). However, if there's something fucked up with the tokens (or other side effects) on NabuCasa, this can probably «solve the issue once and for all». Which means, the chances I can't reproduce the error in the live environment again are quite high. It's a bit like Pandora's box :/

However, at least we then know it's a NabuCasa (or cloud or whatever) issue and not a local one. If I can reproduce the error after that in live again, then it might be a local one.

I'd love to keep my broken integration in place, so I can reproduce the error and can help out – esp. because you're currently trapped in «works-on-my-machine trap» and you can't do anything about it :)

Do you have any tips on how to debug this issue, for example logging HTTP payloads of requests & responses of my local HA, as well as the ones NabuCasa is sending?

from core.

I understood that also people with individual Netatmo dev accounts had that issue.

Q: Is the oauth flow, resp. the «token handling», implemented the same way in both cases?

As mentioned above, Netatmo did change their oauth implementation around that time, to match the official specs. Which means, they changed how tokens are invalidated. It's a pretty wild guess, but probably its an unfortunate «timing issue» or «racing condition» for some users, which probably led to existing tokens to be invalidated (i.e. invalid_grant), and /authorize won't be executed yet / again.

I know, it's just a wild guess. But guessing is the only option without seeing the origin error or alike :/

from core.

Since the error is at the nabucasa domain it's obviously something that's wrong on the home assistant (nabucasa) end. Netatmo might have changed something that broke it, but an internal server error should never happen :)

So I'm not sure who has the logs for the nabucasa side of things but it would be good to know what's happening over there.

from core.

As I unterstand from last messages, The required update token was not given in time, so will still failing while is not update first?
¿Is there a way from the console in HA to push this refresh_token process? I don't know if there is a line to put in the console one time to push manually this refresh_token and the Netatmo integration works again?

from core.

I've personally just removed the integration and added it again. If you keep the entity names the same you will keep the history

from core.

well.. after a crash updating to 2024.7.3 from 2024.7.1... I reinstall NETATMO, deleting before that the 3th party permisions.
And the names changed before that, the attribute's names were like *.temperaturE (french) and now are *.temperaturA (spanish).

I've lost all records, I think. I search by the french names, but HA doesn't purpose any coincidences.

from core.

well.. after a crash updating to 2024.7.3 from 2024.7.1... I reinstall NETATMO, deleting before that the 3th party permisions.
And the names changed before that, the attribute's names were like *.temperaturE (french) and now are *.temperaturA (spanish).

I've lost all records, I think. I search by the french names, but HA doesn't purpose any coincidences.

If you can track down the exact entity ids you were using before you'll have your history back. Hass doesn't delete history at all from what I've seen.

I used the dashboards i was previously using to find the original names.

from core.