2FA refresh token expiring too quickly about ring-mqtt HOT 43 CLOSED

yes my refresh token is getting reset daily at this point

from ring-mqtt.

Yep, all dependencies were bumped up to latest versions. For the service install no additional files, only config.json being updated, which of course means whatever account the service is running under needs permissions to write to this file/directory.

from ring-mqtt.

Hi all, so I have published v3.2.1 and pushed new images to Docker hub which include support for storing updated refresh tokens via all install methods (standard, docker and add-on).

For standard installation the config.json is updated directly each time a new refresh token is issued so the only thing required is that the account running ring-mqtt.js must have permissions to write this file.

For both Docker and the add-on the most recent refresh token is stored in /data/ring-state.json. For this to be persistent in Docker you must mount a volume to the container /data directory. For the add-on this is automatic.

For Docker the RINGTOKEN environment option is only used as a fallback in case authentication with the token in the saved file does not work. Otherwise it can be left out.

For the add-on the ring_token parameter is no longer required and can be left blank. Instead the built in web UI has been enhanced to automatically save the token and start the connection once the wizard is finished. Just configure the MQTT user/password, start the add-on, generate the token via the web UI and off it should go. If for some reason you still prefer to manually generate a token, the ring_token parameter can be set, but it is totally optional and only used as a fallback.

This should address all of the "token expiring too quick;y" and "token expiring on reboot" issues so I'm going to close this for now. Please feel free to open a specific issue if you still have problems and I will look into it.

from ring-mqtt.

Refresh tokens should be long lived. I'm still using the same token I created when I first started working on 2FA back in Jan and it still works (I just verified). There are of course certain things that cause refresh tokens to expire, for example, changing the password on the account immediately expires all refresh tokens, and, if you don't have an active login with a token then it will expire (I'm not sure of the exact time but it seems like a few hours). You shouldn't have to change the refresh token any more often than you have to login to the Ring app as it's the exact same method the app uses to authenticate once you enter your 2FA code.

I'd really don't want to make the web service run all the time, the web service was just an ugly hack to make it easier to get the initial token. I'm not sure how to help with your issue of the token expiring other than to say that it shouldn't happen that often. Are you also having to re-authenticate in the Ring app?

from ring-mqtt.

I have a dedicated account set up for HA that I never use to log in to the app. Previously it would work across restarts/reboots, but this seems like it just started happening a few days ago. Does HA calling the API count as an active login?

For the notification flow I was thinking something like this (rather than keep the web service running):

1. Add-on starts - detects no saved key
2. Start web service
3. Notify the user using HA notifications
4. Prompt for username/password
5. Prompt for 2FA temp code
6. Store auth key
7. Stop web service
8. Add-on continues to start as normal

Just a suggestion - I realize it's probably a non-trivial effort

from ring-mqtt.

@tsightler while I'm not seeing it every reboot. I have had to refresh my token a couple of times since you introduced 2FA. Do you think maybe, we should be scheduling restarts of the service. To ensure Ring sees logins happening and not expiring the token?

from ring-mqtt.

I don't think you need to login over and over as it's not the authentication itself that keeps the token alive, rather the session stays alive and uses the refresh token to acquire short lived access tokens. As long as this process is occurring the refresh token should remain valid, otherwise you'd be having to login with the app over and over as well.

If you've been experiencing this issue over the last couple of days it seems that Ring itself is having a number of issues over the last 24 hours that impact their apps as well so I think it's really unrelated:

https://status.ring.com/

And other using of the Ring API have also reported this issue in the last 24 hours:

dgreif/ring#279

I don't really know what else I could do about it anyway as it's outside of my control. The suggestion to implement notifications in Home Assistant is way outside of the scope of this project as ring-mqtt is simple a script to integrate ring with MQTT and is not specific to Home Assistant, even if it happens to implement some discovery options to make integration with Home Assistant easier.

The suggestions above might be a valid request for the HassIO addon, but it would require a lot of work and make the plugin far more complex. I could maybe enhance the script to just automatically start the web server if it can't login to Ring, that would be easy, but I'm not even sure that's worth it. If the refresh token expires so often it's probably best to just kill this project because it's not really worth the trouble at that point IMO.

from ring-mqtt.

I've decided to reopen this issue. Some investigation indicates that the API provides an observable for monitoring for new refresh tokens. Perhaps it's possible for me to use this to update the refresh token in the config file. I'm not sure how I'd deal with that for the Docker users though, I guess I'd have to require a mapped location to store the configuration, or at least the token, rather than using just the environment variables.

I'll have to make a choice if it's worth continuing this project given these issues, but I'll at least try to look into it over the next few weeks and see if it can be resolved.

from ring-mqtt.

I thought I'd update here as I've managed to get all of my ring-api integrations working again... it seems ring have decided to update the refresh token after every authentication now making them effectively single use. As you say the API provides a way to easily detect when this happens and provides you with the new token. I have several integrations so I've moved to storing the latest refresh token in a single location (in my case an OpenHAB item), and all of my scripts now retrieve and update it with the latest token as necessary.

from ring-mqtt.

Yeah, I'm just not sure how to deal with storing the updated token. For the config file case it's pretty easy I guess, I can just update the config file directly, but for the Docker case, where the token is typically passed in during startup via command line, there's nowhere to store it. I guess I could store it in MQTT itself, as long as the MQTT broker supported persistence storage I could store it as a MQTT retained message and read it on startup from there, but one of the goals at the start of this project was specifically NOT to require an MQTT broker with persistent storage since it seemed like so many people ran brokers without persistence.

The other thing that is weird is that I'm not seeing this behavior at all. I'm still using the refresh token that I acquired back in January and I can restart the script with no issues. Also, I acquired a new token two days ago that also continues to work just fine.

Plus, I can't deny that I'm just getting frustrated with the whole Ring thing and all the changes. I don't have time to tinker with this stuff all the time, I built this script primarily to fill a gap in functionality for arming/disarming that my prior alarm had for 10 years. Now I'm spending time maintaining this just to keep those features working. It's quickly becoming not worth it.

from ring-mqtt.

Plus, I can't deny that I'm just getting frustrated with the whole Ring thing and all the changes. I don't have time to tinker with this stuff all the time, I built this script primarily to feel a gap in functionality for arming/disarming that my prior alarm had for 10 years. Now I'm spending time maintaining this just to keep those features working. It's quickly becoming not worth it.

I agree - as heavily invested as I am in Ring hardware right now, this constant fire-fighting with scripts due to the lack of an official API from Ring is becoming really frustrating. Where as some companies either encourage or turn a blind eye to hobbyists making use of undocumented features, unfortunately I think it's only a matter of time before Ring close out any ability to have custom integration completely - especially now that they're focused on privacy/security issues after recent headlines. Once that day comes I'll be waving goodbye to all of my devices.

from ring-mqtt.

Hello, having the same issues here.
Before using 2FA I never had issues but since month I had to regenerate the key twice already.

from ring-mqtt.

Just wanted to chime in too, I am having the same issues. However, I noticed that it coincided with subscribing to Ring's monitoring service. Perhaps a coincidence, but might be worth investigating.

from ring-mqtt.

So the last few days my tokens have stayed valid through a couple of reboots. I'm wondering how we can help you diagnose the cause?

from ring-mqtt.

Since I posted my daily reset, it stopped happening. Conspiracy? But in all seriousness, I am now monitoring daily to see if I can find a pattern. It was everyday for 3 days and now its fine.

from ring-mqtt.

I have a weird question, why do you guys reboot so often? :)

I haven't rebooted my production HA setup one time since this issue has been opened, and probably only once or twice so far in 2020, and one of those was to replace my RPi3b+ with an RPi4. Before that it had been months.

Anyway, as I had mentioned above, it seemed that Ring servers had some serious issues back about a week ago when this problem started, there was downtime on the Ring status page for the app and then the problems all started after that. Perhaps Ring has just gone back to previous behavior after implementing some type of temporary workaround.

That being said, I do know that I need to update this code to write the refresh token in the config when it changes, it looks like it will change every 15 days or so in most cases. I'm not really sure how to deal with this for the Docker use case, other than require some persistent storage from the host, but for those running it native or using the Hass.io addon it shouldn't be too bad to code up some type of "fix". Unfortunately I just don't have a lot of time to work on this right now. If I get super lucky perhaps I'll sneak some time over the weekend, but I make no guarantees.

from ring-mqtt.

from ring-mqtt.

For me, most of reboots are due to configuration file change, like add a new sensor or binary_sensor.

from ring-mqtt.

@tsightler I'm not rebooting daily. In fact, I've restarted the script more times since this was reported than I did when I was figuring out the fan code in the old V1 codebase.
Mostly I was trying to find a pattern to give you more insight.
Also, I run native and not even on my HA host. So reboots only happen when I want them to.

That being said if I never reboot/restart the script I never see the issue. But if I restart after more than an hour or so of re-issuing the token then I need a new one.

from ring-mqtt.

For me, most of reboots are due to configuration file change, like add a new sensor or binary_sensor.

Adding new devices to Ring? If so, I guess that means I need to increase the priority of dynamically adding devices (something that's on the to-do list). If it's other devices, the script shouldn't need to be rebooted as long as birth/last will messages are configured in HA.

Thanks for the feedback guys. I reboot my production HA setup very rarely because my home is too dependent on it. I think my wife and kids may have forgotten how to manually turn things on/off if the automation doesn't do it for them. I'm like "you see those switches on the wall over there? Yeah, they still work!".

Interestingly, my production instance continues to work with the token from January, perhaps because I don't reboot it very often, who knows. Even if I copy the token from that instance and use it on my dev instance, it still authenticates.

from ring-mqtt.

Adding new devices to Ring? If so, I guess that means I need to increase the priority of dynamically adding devices (something that's on the to-do list). If it's other devices, the script shouldn't need to be rebooted as long as birth/last will messages are configured in HA.

Sorry I may mistaken, I meant reboot HA instance to add new sensors in the configuration file. You are right, there is no need to restart this script/container. In fact I survived this round because I haven't reboot this container since 5 weeks ago.

from ring-mqtt.

I'm not really sure how to deal with this for the Docker use case, other than require some persistent storage from the host

I run this on Docker and am totally cool with giving it some persistent storage. Honestly, a lot of things require this. While it's nice to have this completely deployed in a docker-compose file, adding a single config file would be no big deal.

from ring-mqtt.

@formerghostlyform Thanks for the feedback. I've been considering a few options for the Docker use case (well, really all use cases):

1. Just use the same config file and require persistent storage. This is probably the easiest from my perspective, especially since I don't use the Docker version at all myself. It allows me to completely remove the code that parses environment variables and handle all cases basically the same. Disadvantage is that it requires all current Docker users to completely change their current setup.

2. Store only updated tokens in a separate file on persistent storage. This is an approach that I also need to do with the Hassio addon because, as far as I can tell, there's no method to programatically change the addon config, but I can save the token to a new file in /data and it persist.

3. Provide an option to store the updated token as a persistent value in an MQTT topic. This way, if you have an MQTT broker with persistent storage configured, you can keep the updated refresh token there and still no need for persistent storage. This method could work for all cases, but I'm a little worried about security of this as anyone with access to the MQTT broker could potentially get the token as I've seen a lot of insecure MQTT setups. I could potentially encrypt the token before storing it (perhaps by using 32 characters from the token in the config or environment variable). This would at least work for all cases, but there's just something about it I don't like.

I suppose there's the option of some type of "hybrid" setup, i.e. still support the environment variables, but automatically store a config file in the persistent location to use if environment options aren't set or if the token is newer.

This weekend I had a few minutes and hacked up the dev branch with some changes to update the config.json file with the latest token. I'm running this on my production instance now and it seems to work OK so if I can steal a few minutes during the week I'll try to clean it up and see about pushing it into the main branch.

from ring-mqtt.

Do we know why the token is expiring when the official Ring phone app doesn't ever seem to need to request re-auth? I just went through this for the 2nd time. And, for some reason, the first two refresh tokens didn't work (first was for my main Ring account by accident but that still should have worked). The successful attempt used the local web server to generate the token while the first two attempts used the local script so maybe the local script doesn't work properly?

Avoiding reboots isn't really a solution. I have to reboot my Pi fairly regularly - often to get homeassistant to load updated config files but somewhat commonly because my Pi eventually becomes unresponsive if I don't.

from ring-mqtt.

Yes, the reasons are stated in this thread, here's a quote:

That being said, I do know that I need to update this code to write the refresh token in the config when it changes, it looks like it will change every 15 days or so in most cases. I'm not really sure how to deal with this for the Docker use case, other than require some persistent storage from the host, but for those running it native or using the Hass.io addon it shouldn't be too bad to code up some type of "fix". Unfortunately I just don't have a lot of time to work on this right now. If I get super lucky perhaps I'll sneak some time over the weekend, but I make no guarantees.

I clearly didn't get "super lucky" and just haven't really had time to work on this project at all over the last months. Basically the script need to be modified to monitor api.onRefreshTokenUpdated and automatically update the config.json. I do have a crude fix for the non-Hass.io/non-Docker case in my local version, because I don't use all of the HASS.io/Docker stuff in my own setup, but, before I can publish it I need to create fixes and test those use cases as well.

I'll happily entertain pull requests if somebody wants to take a stab at fixing it, but I don't know when I'll have time to dig into this project again. I keep thinking I'll get to it "in the coming weeks" but then those weeks pass and no time appears.

from ring-mqtt.

Ah, I hadn't understood how the tokens worked until your response just now. So it's supposed to be self-refreshing but there's currently no way to persist the updated token in the current ring mqtt implementation.

I know the feeling well. I used to do a lot of outside-work stuff in my spare time. Then I bought a house, had kids and moved up in my company and now I'm usually too tired/already busy to do coding stuff outside of work!

from ring-mqtt.

Yep, that's exactly it. For the "normal" case, i.e. no Docker/Hass.io, I just wrote a function that rewrites the config file on refresh, but for Hass.io I need to persist the data somewhere. You can write the config because that's built from the YAML for that addon so I probably need to write it to /data/ring.token or something like that, still pretty easy, just need to code it up and test it.

However, it's the Docker case that's the most annoying because right now it's started with just environment variables. Now I'll need to modify to accept a path for persistent storage to store the token, but then I can't just get rid of the environment variables without breaking existing setups when they upgrade. Or store the token in a different file, etc.

I have a plan that I think will work, but I just need to find an hour or so to sit down, code it up, and test it well enough to be willing to push it out there. I swore I was going to get that done by the end of July, but I'm starting to run out of days.

from ring-mqtt.

I'm the former case. Is there an ETA on a version that works for that or can that not go out without the equivalent docker fix?

from ring-mqtt.

I'll try to get something up with this, at least in the dev branch, by the end of this week. I had just a little time to work on the code this past weekend and I fixed a few other bugs that have been hanging around and started work on finalizing the code for storing the persistent token. Maybe the month of August will finally give me time to address a few of the backlog of feature requests as well.

from ring-mqtt.

I've pushed an update which has fixes for storing the updated refresh tokens for both the service based install and the HASS.io addon case (the HASS.io case still needs some work but it should be "good enough"). I'm still working on the Docker case but hope to have something there in a few days. If people could test and report any issues that would be great.

from ring-mqtt.

Sweet thanks for that. For the service install, I had to do npm install again. So presumably some dependency changed,
Also, should I be seeing a new file created for the stored token? Or config.json being updated?

from ring-mqtt.

I tried to upgrade but now the service won't start. Running the script locally with /usr/bin/node /home/XXX/ring-mqtt/ring-mqtt.js just runs for a little while then exits. I can't find any logs to indicate what's going wrong.

from ring-mqtt.

Did you do an npm install - to update the dependencies?

from ring-mqtt.

Also, if you happened to have a "camera only" setup, i.e. no Ring alarm, then there was a bug that could cause this. I've pushed another update to address this. But indeed, you will definitely need to update dependencies with "npm install" in the directory where the script is installed (basically, follow install steps again.

from ring-mqtt.

Yes I did the npm install. The first time I did though my pi ended up locking up and I had to reset it (after I left if for a couple of hours). A subsequent re-run didn't indicate any issues. Found log content in /var/log/syslog for the failed launch as a service:

Aug 6 10:12:37 raspberrypi systemd[11046]: ring-mqtt.service: Failed at step CHROOT spawning /home/pi/ring-mqtt/ring-mqtt.js: No such file or directory
Aug 6 10:12:37 raspberrypi systemd[1]: ring-mqtt.service: Main process exited, code=exited, status=210/CHROOT

My old service file had:
ExecStart=/home/pi/ring-mqtt/ring-mqtt.js
The latest one in the repo has:
ExecStart=/usr/bin/node /opt/ring-mqtt/ring-mqtt.js

I also tried adding the /usr/bin/node to match this but that just changed the text of the error in the log file to reference that instead:
Aug 6 09:39:15 raspberrypi systemd[10232]: ring-mqtt.service: Failed at step CHROOT spawning /usr/bin/node: No such file or directory

from ring-mqtt.

Ah, progress. I had messed up updating the new WorkingDirectory config variable so it was a non-existant path. Correcting that showed me I had a config.json error - I forgot to add the comma at the end of the "ring_token" line after I refreshed the 2FA token earlier.

from ring-mqtt.

Now I just need to fix permissions/config to allow for updating config.json. I was previously running the service as the 'homeassistant' user but the repo is located under the pi user so I was expecting this to fail. I see you switched it to run as root in the latest change - but I'd prefer not to do that.

from ring-mqtt.

Moved everything to /opt to match your default config and running as root is now working. Initially I tried moving it to /home/homeassistant and keeping the user as homeassistant but that didn't work.

from ring-mqtt.

Yeah, I've debated over what to do there. The service file is just intended to be an example that can be modified to suite the specific config, but too many people just copy it and try to use it and then open an issue when it doesn't work. Now that I need to save a file I was afraid too many issues would be caused by users not understanding that the service would need permissions to the file/directory so running as root was a way around it. I wasn't super crazy about it but it also didn't seem very high risk since the script doesn't listen on any ports by default so it should (hopefully) be pretty difficult to exploit.

Besides, the people using ring-mqtt are just downloading some code written by a random guy on the Internet and letting it have complete control over their alarm system, and in some cases cameras, including authentication tokens that bypass 2FA, how security conscious can they be? :)

from ring-mqtt.

Besides, the people using ring-mqtt are just downloading some code written by a random guy on the Internet and letting it have complete control over their alarm system, and in some cases cameras, including authentication tokens that bypass 2FA, how security conscious can they be? :)

Hah! Indeed :).

from ring-mqtt.

from ring-mqtt.

Amazing work. I really appreciate all the effort 🙌

from ring-mqtt.

@tsightler Thank you so much for doing this. I'm back in business with my ring sensors and hands off so I can use them for other automations. It's working perfectly so far! Thanks again.

from ring-mqtt.