Integration with Google assistant about wled HOT 73 OPEN

This is now possible as there is a local API exposed in the New SDK i think this should be reopened. https://developers.google.com/assistant/smarthome/concepts/local

from wled.

Watching this open issue eagerly!

from wled.

Just testing out the integration a bit tonight. Figure I'd share the progress in a video.

https://www.youtube.com/watch?v=s2G-iIrtfhc

The Code isn't ready for prime time… Lots of embedded secrets and passwords… But I'll keep working on it to get it up to a server for other people to try in the near future.

In its current state... it is insecure.

from wled.

This is not a bug, and there is no way for me to add native Assistant support in the near future sadly.
Therefore I close this issue now.
Feel free to reopen it if you have any further ideas :)

from wled.

Looking forward to an official integration!

from wled.

I HATE the idea of opening ports and using IFTTT. Very bad idea imho! Is there anyone working on a direct GA integration? I am keen to work on one just checking to make sure I am not re-inventing the wheel. :)

from wled.

Relevant to this issue: Google announced that their Local Home SDK has emerged from developer preview and is now GA:

https://developers.googleblog.com/2020/04/local-home-sdk-ready-for-actions.html

from wled.

Just reading the updates, this is exciting! Can't wait to see this feature land! :-) Keep up the great work!

from wled.

Following this one as well. My kids will appreciate the ability to control their desk LEDs with the Google home speakers in their rooms!

from wled.

I was able to set this up. First using this guide for having a static FQDN with a cert https://www.youtube.com/watch?v=hqRT2VKhxJ0 i had no router loopback problems. Then i mainly used the https://www.home-assistant.io/integrations/google_assistant/ guide to do the rest. I think i also watched https://www.youtube.com/watch?v=MMtCzX6US9M and https://www.youtube.com/watch?v=mDqllkmxiOI the only gotcha was google no longer accepts the apikey and will only accept a json fil. so i had upload my json file i got from google to home assist and make my config file look have this.
google_assistant:
project_id: myproj-845100
service_account: !include myproj-43756435645.json

The part i dont have working and would be happy to do a write up if i figured out how would be how to change things like fx via google.

from wled.

Done! Thanks Aircookie for making such a flexible platform!
https://www.instructables.com/WLED-on-ESP8266-IFTTT-Google-Assistant/

from wled.

@axl12ap yes, they even had a preview of it two years ago. Unfortunately, it does not seem like the new SDK would be useful for WLED's use case as the local option is merely there to reduce latency, device discovery is to my understanding still not possible without a "manufacturer cloud"...

from wled.

@balucanb sorry, it does not work with ESP32 either. Almost all WLED features are the same on both platforms.

from wled.

Not sure this is 100% relevant to this thread issue thread, but if you have Samsung Smartthings, WLED can be integrated with that, and Google assistant has some ability to control them. I used this to do it:
https://community.smartthings.com/t/wled-with-smarthings/175628/8

from wled.

from wled.

Matter as a replacement for current Google Home infrastructure looks promising as a robust solution to this as it doesn't require cloud access for smart home devices, similar to HomeKit. But no idea on licensing, security or implementation on esp32.

from wled.

hola, estoy usando esp32 con rainmake

Hi, unfortunately most people here (including myself) do not speak Portuguese, could you translate your comment into English? Thanks.

from wled.

Has anyone investigated Google's local fulfillment sdk? https://developers.home.google.com/local-home/overview

from wled.

Hi,

sorry that is not possible because the Google Assistant doesn't have a local API. I'd need to have a complicated cloud infrastructure to support Google home which I can't do as a small open source project.

You can do it with the free IFTTT website though, which is compatible with google Assistant. It requires you to do a router port forwarding so the ESP8266 is accessible from the internet. Then you need two IFTTT Webhooks applets that trigger <YourPublicIP>:<YourESPNetworkPort>/win&T=1 for on and <YourPublicIP>:<YourESPNetworkPort>/win&T=0 for off, respectively.

from wled.

Könnte man nicht eine von Google unterstütze api integrieren? Die api von tuya smart (smart life) ist offen und soll leicht verständlich sein. Viele sub-hersteller (smart life oder auch hama) nutzen die.
Vielleicht hilft ja der link?
https://docs.tuya.com/en/cloudapi/cloud_access.html#access-mode

from wled.

Könnte man überlegen. Würde sicherlich funktionieren, aber im Moment möchte ich keine cloudabhängige API integrieren. Es ist aber möglich, das Ganze mithilfe von IFTTT und Blynk zu bauen: https://www.instructables.com/id/GOOGLE-ASSISTANT-CONTROLLED-SWITCH-USING-NODEMCU/ dazu könnte ich mal ein an WLED angepasstes Tutorial erstellen, weil Blynk ja schon implementiert ist.

from wled.

Hey, ja mit ifttt habe ich schon einiges umgesetzt; aber die weiblichen Mitglieder der Familie bombardieren die Google home Geräte mit Phrasen die ich vorher nicht angelegt habe und sind dann enttäuscht, wenn auf "hey Google, lightbox Farbe Lavendel" die Farbe dann nicht umgesetzt wird.
In cloud APIs geht's dann aber.

Vielleicht könnte man die Farben ja sprachbasiert mit rgb Werten hinterlegen? /win&color=red/rot

from wled.

This is now possible as there is a local API exposed in the New SDK i think this should be reopened. https://developers.google.com/assistant/smarthome/concepts/local

right on!

from wled.

Not yet looked into it in detail, but I agree that this is a promising change!

from wled.

Any update as to having google automation? I've tried using WebCoRE with Smartthings, no luck. Couldn't do Blynk as I'm new to programming. I already set up my ESP8266 with my WS2812b lights and WLED but I want at least Google home to turn them off or on. Any other ideas? I tried adafruit but i'll have to see as I followed a tutorial and I got a bit lost

from wled.

Are there any updates for this feature? I'm looking forward to being able to control my lights with my google home natively!

from wled.

I spent a little time getting the Blynk webhook working for one of my projects. It takes a bit of leg work, but here's a write up of what I went through to use Google Assistant via IFTTT and Blynk: http://www.robopenguins.com/web-based-door-control-update/

from wled.

This would make WLED even more user friendly and ready for the mass

from wled.

Is there a tutorial how can i integrate wled with google assistant?

from wled.

+1 here for Google Integration in the near future too.
I'm a Google fan more than I am an Alexa fan.
Although, besides IFTTT, I could control it with Home Assistant but I would rather see direct native support. 🙃

from wled.

Is there a tutorial how can i integrate wled with google assistant?

Found this. Any help? @Aircoookie ?

from wled.

after briefly talking to @Aircoookie and "e_town" on discord i was able to setup saying things like "OK google turn on Bedroom popcorn effect"
Since there are currently 103 effects and 52 palettes i decided to whip up a quick PowerShell the generates the data for the configuration files.
No real time was spent on this but should work for the basics.
https://gist.github.com/justusiv/177c0750a1936486b735a2a2d503e424

EDIT
I should also note that there are several words that google assist doesn't really like. Things like "twinklecat" should be twinkle cat otherwise google doesn't know what to do. also things like "eyes" and "ice" get it easily confused. maybe adding triggers for both would be the best handle of that. and the last thing i added was a random one as well.

bedroom_random:
    name: Bedroom Random

set_random: 'curl "http://192.168.0.123/win&FP=$(( $RANDOM % 51 ))&FX=$(( $RANDOM % 102))"'

- id: '9999999950051'
  alias: random
  description: ''
  trigger:
  - entity_id: input_boolean.bedroom_random
    from: 'off'
    platform: state
    to: 'on'
  condition: []
  action:
  - data: {}
    service: shell_command.set_random
  - delay: 00:00:10
  - data: {}
    entity_id: input_boolean.bedroom_random
    service: input_boolean.turn_off

from wled.

Hi, just wanted to ask if there are any news on the integration?

from wled.

maybe it can have more future support once Connected Home over IP becomes more developed and supported

https://github.com/project-chip/connectedhomeip#connected-home-over-ip
https://www.connectedhomeip.com/

from wled.

CHIP seems awesome! Perhaps this is finally the standard we've been waiting for that will ideally work with all major voice assistants without needing a "manufacturer server". I just hope that the overhead for implementing security won't be too much for an ESP8266, they tend to be quite unusable in most things crypto.

from wled.

maybe it can have more future support once Connected Home over IP becomes more developed and supported

https://github.com/project-chip/connectedhomeip#connected-home-over-ip
https://www.connectedhomeip.com/

isnt this what zigbee was suppoed to be until it got butcherd by the very people involved in CHIP ?

Ahh so its not a radio its piggybacking over other radio stacks, ooh i wonder if this could be used to run connected lights/switches/sockets on ethernet over power (powerline) well when they support ethernet that is

from wled.

The part i dont have working and would be happy to do a write up if i figured out how would be how to change things like fx via google.

I haven't had the chance to go through those links yet, but is it controlling it via http request on the local network? If so I think I can help you with controlling the fx.

from wled.

Ok got it working using Google assistant and IFTTT. Took all day of experimentation. Will make an instructable soon

from wled.

from wled.

I am looking forward to it! Does it allow for the effect to be changed?
…
On Fri, Nov 27, 2020, 19:14 tronicsmasta @.***> wrote: Ok got it working using Google assistant and IFTTT. Took all day of experimentation. Will make an instructable soon — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#31 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARX5XD35GF5SXIWVAINFDCTSSA6IFANCNFSM4FH2MTMQ .

Yes!!

from wled.

from wled.

Would it be possible to use ingredients in the "if" side of ifttt to control certain aspects of the lights?
…
On Sat, Nov 28, 2020, 00:09 tronicsmasta @.***> wrote: Done! Thanks Aircookie for making such a flexible platform! https://www.instructables.com/WLED-on-ESP8266-IFTTT-Google-Assistant/ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#31 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARX5XDYB6PV3ZPFIDHNSIP3SSCAZHANCNFSM4FH2MTMQ .

I haven't tried that yet. I was excited I got it to work with http requests. Will try with dynamic parameters today and maybe I can get json working with it.

from wled.

from wled.

Send me an email, my username @ gmail dot com

from wled.

Done! Thanks Aircookie for making such a flexible platform!
https://www.instructables.com/WLED-on-ESP8266-IFTTT-Google-Assistant/

Thank you for the instructions.

from wled.

from wled.

google launches something called google actions local SDk. it says all the right things.

it allows thr google.home mini to make requests to other devices.

https://developers.googleblog.com/2019/07/developer-preview-of-local-home-sdk.html?m=1

i haven't read it completely yet

from wled.

google launches something called google actions local SDk. it says all the right things.

it allows thr google.home mini to make requests to other devices.

https://developers.googleblog.com/2019/07/developer-preview-of-local-home-sdk.html?m=1

i haven't read it completely yet

Good starting point, have you any idea on how to do it?
@Aircoookie did you read the article?

from wled.

ok screw it. it can't be done stand alone.
but then wled is DIY.

I have no problem writing an express app on a raspberry pi and have it send GET requests to WLED.

or something

I'm intrigued with getting a hubitat.
I've seen i believe, the hookup guy mention it.

on 'google, light up my bed', send GET
to wled

i've seen no comments or projects or tutorials on something like this.

any ideas?
a mere link would do.

we could get a old android to act as a wled bridge in liu of a rpi.

from wled.

Just read through the whole issue, I noticed only ESP8266 was mentioned, will a ESP32 work with this also?

from wled.

I might be able to build a service that does this.

IMO, the hard part would be building a way for IT to talk to our WLED instances without port forwarding.

One method I thought of would be to have a setting in WLED where it would poll an HTTP server every few seconds until it got a response that told it to change its current settings.

It's not the push like notifications we've come to love, but it's an idea. I'll let everyone know if I actually make an attempt at setting something up like this.

from wled.

@Ryanb58
i just realized my comment was imcomplete.
yes, the port forwarding thing.
i was talking about installing the Google smarthings sdk on a rpi and make the rpi masquerade as a lamp or whatever.

after it recieves the command from the google cloud. then it forwards it to your local network to wled. it it posible that I'm simply describing what node-red does(doh!) or home assistant.
i still have some learning to do.

surely all the google enabled things don't require port forwarding.

from wled.

@Ryanb58 that would be absolutely amazing!

One method I thought of would be to have a setting in WLED where it would poll an HTTP server every few seconds until it got a response that told it to change its current settings.

It's not the push like notifications we've come to love, but it's an idea. I'll let everyone know if I actually make an attempt at setting something up like this.

How about MQTT? The publish/subscribe type messages mean no polling and thus almost instant change. I've thought about the architecture [Google] <> [Integration] <> [MQTT-Broker] <> [WLED-Device] before, but I honestly have no idea how to build a service that integrates with Assistant, authenticates the user, and so on.

from wled.

Little bit of progress:
Today, I was able to successfully send commands from my Google Assistant to an AWS Ec2 instance I set up(after the oauth 2.0 authentication was established). No progress with sending commands to the actual WLED devices yet. Still need to figure out a way to get it to connect to my public server that is exposed to the internet(required by Google Assistant).

@Aircoookie The ways to send change requests(JSON, MQTT, etc…) are only available on the local network(inbound/outbound traffic issue). To pursue the MQTT approach, I think I would need a Samsung SmartThings device or something similar. I'd like to go at this with an approach that doesn't require me to have to buy any more hardware. Depending on how I can establish the connection.

Side Note/Discovery:
With Google's requirement to have a public OAuth 2.0 server running for authentication... it is going to(already is) costing me $$ to operate. IDK how I am going to deal with that, but one thing at a time.

from wled.

@Aircoookie I may not understand MQTT. I'm going to do more research into it. My last comments about it may not apply.

from wled.

@Ryanb58 wow, that is awesome! If you manage to implement something that works, I'd definitely be happy to cover the hosting $ if it's not outrageous.

About MQTT, indeed, it works outside the local network, which I believe is a key requirement for this sort of thing to work.
Both the WLED device and your AWS service connect to the public MQTT server (also called broker) via TCP. We can use a general purpose free public MQTT server like broker.hivemq.com for the time being, if this gets big, we can consider hosting our own MQTT broker later.

How it works is also quite simple, the WLED subscribes to the /wled/abcxyz topic (where abcxyz is some unique identifier, currently using part of the MAC address) on the MQTT server and when the AWS service sends a control command to the MQTT broker (called a publish), the MQTT server forwards that to the subscribed client (the WLED device). The most tricky part would be giving the service the correct identifier for the WLED device the user would like to control, so that it can publish to the correct MQTT topic. Maybe for a start, the user would need to fill in the identifier and device name manually in the same place that they authenticate (I'm very sorry, OAuth2 goes a little over my head, got no idea how it works)

from wled.

@Aircoookie Are there any plans on supporting TLS 1.2 and x.509 certs during the MQTT rewrite?

from wled.

@Ryanb58 not currently, on ESP8266 we definitely do not have enought memory for crypto. On 32, it is less of an issue, but it is still quite hard. The only way that worked when I tried it was hard-coding a certificate, which of course breaks all communication if it expires, and only allows interfacing with one specific pre-set server.
That said, the connection between MQTT broker and AWS could be configured as TLS/x.509 with relative ease, while still using unsecured TCP between broker and WLED device.
It boils down to whether we'd need to send confidential data (e.g. OAuth tokens) over MQTT, I would be very uncomfortable without TLS, but for simple device name/announce/identifier/brightness or color change command types of traffic. It also only applies if we had a dedicated MQTT broker, with a public one, theoretically anyone could mass-subscribe to topics and intercept traffic, even if all transport paths were secured.

from wled.

@Aircoookie I played around with MQTT a bit more over this last weekend, and learned a lot. I went through the steps of setting up a local Mosquito MQTT server and controlling my WLED lights from it; from a python script. That gave me a way better understanding of how it operates.

I didn't have to send any OAuth Tokens over the wire to MQTT broker.

If I get some more time this next weekend, I will attempt to push it all onto an EC2 instance and test out my Oauth App's code alongside a dedicated MQTT broker instance. It will be over an insecure transport, but should give me a good feel for what this will look like running together. I'll keep ya updated.

from wled.

I was looking at this and try to find some Google assistant to mqtt integration and i came across this three projects that can be use to control wled over Google assistant for two of them need to have full SSL, setting up google actions port forwarding so you can run them
https://github.com/i8beef/HomeAutio.Mqtt.GoogleHome
https://github.com/mikejac/node-red-contrib-google-smarthome

but one of them is much easier it just need node-red no need for port opening and SSL
https://github.com/andrei-tatar/node-red-contrib-smartnora

from wled.

yesterday I tried a theory.
here's another breadcrumb .

I used a shelly 1 and added an action to have it turn on Wled on switch.

it only works for turning on and off but it is very close to what I wanted.
this feels like something that could be done in hubitat or home assistant.

from wled.

After a few weeks of messing with this stuff, there isn't really anything we can add to this repo to make this work. It involves a cloud provider setup with a connection to your IoT devices. Thus, maybe this PR should be closed until one of these two items are completed.

• More mature local fulfillment by Google.
• MQTT connections over TLS on the module.

My original setup involved an insecure MQTT broker on the web. Which was abused. Thus, wouldn't provide a good user experience and is a security risk.

from wled.

Is there any update on getting wled working natively with google home? I'm looking forward to this as i have two nest hub 2 and a sonos one with google Assistant.
The lights can only be cobtrolled by the one echo spot i own.

from wled.

Any updates here? Or atleast a guide with the best DIY solution?

from wled.

Hi all. For starters, I´m a total noob. I just made my first LED strip with a Wemos D1 mini for the back of my tv with Wled+Hyperion. This works great. Wife asked if they could be turned on/off with Google assistant? So I am looking for this feature to. I did some digging around and found something called ESP Rainmaker which can be set up using a local MQTT broker. It can be used for ESP32 and ESP8266. The ESP Rainmaker app can then be added to Google Home without issue. I am no coder and only understand half of it. My thought was if ESP Rainmaker some how could be implemented in Wled? https://www.youtube.com/watch?v=Ta9dwDD9GyA

Regards

from wled.

from wled.

Thats what I mean. This should work without a HA instance. Full control using Google assistant. Watch the video ;)

from wled.

from wled.

Yes yes agreed. The way I understand it is there has to be some sort of code on the ESP board to be able to add it to the Rainmaker app as an IOT device. The Rainmaker app is then added as a "skill" to Home Assistant there by adding voice control. I dont care if its local or cloud or both. As long as I can use voice control.

from wled.

I've setup some IFTTT webhook, is working "okayish"

from wled.

Based on my experience with AWS IoT Core and on what I found on the web about connecting an ESP32 to AWS, I'd like to share an idea how this might work without an MQTT broker in the middle.

The solution consists of two parts:

• WifiClientSecure to handle the certificates required by AWS to secure the connection
• Arduino Client for MQTT for the MQTT part - this is probably not even necessary, but this example uses it. Would be worth testing with WLED's own MQTT setup

The part I can't say much about is how much effort this is to integrate, since I'm rather new to programming microcontrollers. My motivation is to get feedback on this aspect and on whether I'm missing something...

Cheers

from wled.

Sorry, forgot to add the link to the tutorial...

from wled.

hola, estoy usando esp32 con rainmaker y te deja configurar varios dispositivos, ya que tengo varias luces con wled esp8266 y no es compatible con casi nada el esp8266, se me ocurrio usar el esp32 como master y crear varias luces virtuales en el y que cada luz virtual le envie los comandos desde rainmaker a wled por GET o por MQTT, aun no lo termino de probar pero creo que va a ser una buena solucion para los viejos esp8266 wled que tengo ya en casa y funcionan muy bien. voy a ir comentando aca mis avances y si funciona entonces publico mi codigo para que lo trabajemos en conjunto y tengamos una solucion estable y sin tanto problemas de configurar servidores en la nube.
yo estuve probando esp32 rainmaker y funciona muy bien con google home y alexa.
@Aircoookie

from wled.