Logitech Harmony Companion remote not working with 2.04.40 about deconz-rest-plugin HOT 27 CLOSED

@manup I didn't see the body in the logs, running with --dbg-http=1 and --dbg-info=2 . I might just have missed it. Should I expect to see it with those flags?

Didn't find anything obvious in the code, so I'll have a go at a tcpdump in a couple of hours when I have access to my raspberry pi and harmony.

from deconz-rest-plugin.

I can also confirm that the Harmony hub cannot see DeCONZ when searching for a Hue Hub.
My DeCONZ is on version 2.04.82

So for now, i have to use the Philips hub, for the lights that i need to control from the Harmony remote.
Would be nice to get rid of the Philips hub.

from deconz-rest-plugin.

Glad to hear that I'm not the only one.
I have done some analytics.

Using 2.04.84 I get a "body contains invalid JSON" error

PUT /api/5A6DAEFCD7/lights/32/state HTTP/1.1
connection: close, TE
content-length: 19
user-agent: LuaSocket 2.0.2
te: trailers
content-type: application/json
host: 192.168.207.27

{"on":true,"bri":0}

HTTP/1.1 400 Bad Request
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=utf-8
Content-Length:94

[{"error":{"address":"/lights/32/state","description":"body contains invalid JSON","type":2}}]

Looking deeper I found this is because of "bri":0. Makes not much sense to turn on a light with a brightness of null.
The reason is that the reported status of the light is different to version 2.04.35

On 2.04.35 a GET /api/5A6DAEFCD7/lights gives
"state":{"alert":"none","bri":54,"colormode":"ct","ct":370,"effect":"none","hue":0,"on":false,"reachable":true,"sat":0,"xy":[0.382037,0.378391]}

On 2.04.84 a GET /api/5A6DAEFCD7/lights gives
"state":{"alert":"none","bri":0,"colormode":"ct","ct":370,"on":false,"reachable":true}

The difference to 2.04.35 is "bri":0.
I think the value should be the brightness of the lamp the last time it was switched on.

from deconz-rest-plugin.

Can confirm that I'm seeing this issue as well on 2.05.01 with my IKEA Trådfri lights and my Logitech Harmony. Worked fine in 2.04.35 but hasn't worked on 2.04.99 or 2.05.01.

Harmony triggers this error in the logs:
HTTP API PUT /api/D4E0709D44/lights/2/state
[{"error":{"address":"/lights/2/state","description":"body contains invalid JSON","type":2}}]

The state GET responses look like this:
HTTP API GET /api/D4E0709D44/lights
{"alert":"none","bri":210,"colormode":"ct","ct":417,"on":false,"reachable":true}
{"alert":"none","bri":254,"colormode":"ct","ct":392,"on":false,"reachable":true}

Any fixes in sight? Can I help debug in any way?

from deconz-rest-plugin.

Can you provide the body of the harmony PUT request?

It's needed to check what deCONZ might misinterpret as invalid JSON.

from deconz-rest-plugin.

So after poking around with tcpdump a while and getting various result, here's some news.
Today the problems seemed to happen 50/50 of the times. Just turning a single light on and off repeatedly I was managing to get it to give me that 400 - invalid json regularly. Both on turning on and turning off. Bear with me below as I've tried to grab the text out of the tcpdumps, could be that I missed some control-char in there.

First, here's when it worked:
Harmony request:
PUT /api/D4E0709D44/lights/1/state HTTP/1.1
Headers:

connection: close, TE
content-length: 12
user-agent: LuaSocket 2.0.2
te: trailers
content-type: application/json
host: <<masked by me>>

Body:
{"on":false}

Secondly, when it didn't work. About 1m 15sec and a couple of tries later
Harmony Request:
PUT /api/D4E0709D44/lights/1/state HTTP/1.1
Headers:

connection: close, TE
content-length: 12
user-agent: LuaSocket 2.0.2
te: trailers
content-type: application/json
host: <<masked by me>>

Body:
{"on":false}

Deconz Response:
[{"error":{"address":"/lights/1/state","description":"body contains invalid JSON","type":2}}]

What's really interesting here is that Harmony seems to retry all failed requests 3 times. Usually I've seen it get 400's all 3 times. But this time it actually succeeded on it's first retry just over 1 sec later
Harmony request:
PUT /api/D4E0709D44/lights/1/state HTTP/1.1

Headers:

connection: close, TE
content-length: 12
user-agent: LuaSocket 2.0.2
te: trailers
content-type: application/json
host: <<masked by me>>

Body:
{"on":false}

Deconz Response:

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=utf-8
Content-Length:42
ETag:"1cec795f68d8bc942e3cf4b26971057c"

[{"success":{"/lights/1/state/on":false}}]

All the examples above happened to be on a turning off. But here's another body that had the same response:
{"on":true,"bri":134}

It seems like perfectly valid JSON to me, and the fact that the Content-Length of the requests seem to match up with the body contents as well makes it feel like an intermittent problem on the parsing side. And that the same request seems to go through eventually seems odd as well.

from deconz-rest-plugin.

Noticed a very weird thing right now...
Doing the exact same thing as I was doing before (turning the same light on and off repeatedly using the Harmony Android App).. when I don't have my tcpdump running it seems to fail consistently. Turning on my tcpdump makes it a lot more stable (still failing a lot tho).

Maybe I'm superstitious, but it seemed very repeatable.

from deconz-rest-plugin.

Interesting logs the commands do look alright. You may also try to debug with --dbg-http=1 it should print some more info about the request

from deconz-rest-plugin.

I did so as well, but I didn't get more info out of that dbg flag...

from deconz-rest-plugin.

Can you post the requests as captured, not broken up in request, headers, and body. I’ve a hunch something could be wrong with the empty line between headers and body, causing deCONZ to position the body a character too early or too late. Does the Harmony end lines with \n or with \r\n?

from deconz-rest-plugin.

Here's the output from tcpdump -s 1500 -XXvvv:

17:56:21.463488 IP (tos 0x0, ttl 64, id 43999, offset 0, flags [DF], proto TCP (6), length 97)
    HarmonyHub.59692 > raspberrypi.http: Flags [P.], cksum 0x3607 (correct), seq 1:46, ack 1, win 2920, options [nop,nop,TS val 922263392 ecr 27682328], length 45: HTTP, length: 45
	PUT /api/D4E0709D44/lights/1/state HTTP/1.1
	0x0000:  b827 eb7a a42f 0004 20f5 df60 0800 4500  .'.z./.....`..E.
	0x0010:  0061 abdf 4000 4006 0c96 c0a8 0064 c0a8  .a..@[email protected]..
	0x0020:  006d e92c 0050 f0e4 3483 e28b 421a 8018  .m.,.P..4...B...
	0x0030:  0b68 3607 0000 0101 080a 36f8 9f60 01a6  .h6.......6..`..
	0x0040:  6618 5055 5420 2f61 7069 2f44 3445 3037  f.PUT./api/D4E07
	0x0050:  3039 4434 342f 6c69 6768 7473 2f31 2f73  09D44/lights/1/s
	0x0060:  7461 7465 2048 5454 502f 312e 310d 0a    tate.HTTP/1.1..
17:56:21.463656 IP (tos 0x0, ttl 64, id 9219, offset 0, flags [DF], proto TCP (6), length 52)
    raspberrypi.http > HarmonyHub.59692: Flags [.], cksum 0x8248 (incorrect -> 0x8106), seq 1, ack 46, win 453, options [nop,nop,TS val 27682329 ecr 922263392], length 0
	0x0000:  0004 20f5 df60 b827 eb7a a42f 0800 4500  .....`.'.z./..E.
	0x0010:  0034 2403 4000 4006 949f c0a8 006d c0a8  .4$.@[email protected]..
	0x0020:  0064 0050 e92c e28b 421a f0e4 34b0 8010  .d.P.,..B...4...
	0x0030:  01c5 8248 0000 0101 080a 01a6 6619 36f8  ...H........f.6.
	0x0040:  9f60                                     .`
17:56:21.466412 IP (tos 0x0, ttl 64, id 44000, offset 0, flags [DF], proto TCP (6), length 193)
    HarmonyHub.59692 > raspberrypi.http: Flags [P.], cksum 0xe4b4 (correct), seq 46:187, ack 1, win 2920, options [nop,nop,TS val 922263393 ecr 27682329], length 141: HTTP
	0x0000:  b827 eb7a a42f 0004 20f5 df60 0800 4500  .'.z./.....`..E.
	0x0010:  00c1 abe0 4000 4006 0c35 c0a8 0064 c0a8  ....@[email protected]..
	0x0020:  006d e92c 0050 f0e4 34b0 e28b 421a 8018  .m.,.P..4...B...
	0x0030:  0b68 e4b4 0000 0101 080a 36f8 9f61 01a6  .h........6..a..
	0x0040:  6619 636f 6e6e 6563 7469 6f6e 3a20 636c  f.connection:.cl
	0x0050:  6f73 652c 2054 450d 0a63 6f6e 7465 6e74  ose,.TE..content
	0x0060:  2d6c 656e 6774 683a 2031 320d 0a75 7365  -length:.12..use
	0x0070:  722d 6167 656e 743a 204c 7561 536f 636b  r-agent:.LuaSock
	0x0080:  6574 2032 2e30 2e32 0d0a 7465 3a20 7472  et.2.0.2..te:.tr
	0x0090:  6169 6c65 7273 0d0a 636f 6e74 656e 742d  ailers..content-
	0x00a0:  7479 7065 3a20 6170 706c 6963 6174 696f  type:.applicatio
	0x00b0:  6e2f 6a73 6f6e 0d0a 686f 7374 3a20 3139  n/json..host:.19
	0x00c0:  322e 3136 382e 302e 3130 390d 0a0d 0a    2.168.0.109....
17:56:21.466533 IP (tos 0x0, ttl 64, id 9220, offset 0, flags [DF], proto TCP (6), length 52)
    raspberrypi.http > HarmonyHub.59692: Flags [.], cksum 0x8248 (incorrect -> 0x8067), seq 1, ack 187, win 470, options [nop,nop,TS val 27682329 ecr 922263393], length 0
	0x0000:  0004 20f5 df60 b827 eb7a a42f 0800 4500  .....`.'.z./..E.
	0x0010:  0034 2404 4000 4006 949e c0a8 006d c0a8  .4$.@[email protected]..
	0x0020:  0064 0050 e92c e28b 421a f0e4 353d 8010  .d.P.,..B...5=..
	0x0030:  01d6 8248 0000 0101 080a 01a6 6619 36f8  ...H........f.6.
	0x0040:  9f61                                     .a
17:56:21.469338 IP (tos 0x0, ttl 64, id 44001, offset 0, flags [DF], proto TCP (6), length 64)
    HarmonyHub.59692 > raspberrypi.http: Flags [P.], cksum 0x31a3 (correct), seq 187:199, ack 1, win 2920, options [nop,nop,TS val 922263394 ecr 27682329], length 12: HTTP
	0x0000:  b827 eb7a a42f 0004 20f5 df60 0800 4500  .'.z./.....`..E.
	0x0010:  0040 abe1 4000 4006 0cb5 c0a8 0064 c0a8  .@..@[email protected]..
	0x0020:  006d e92c 0050 f0e4 353d e28b 421a 8018  .m.,.P..5=..B...
	0x0030:  0b68 31a3 0000 0101 080a 36f8 9f62 01a6  .h1.......6..b..
	0x0040:  6619 7b22 6f6e 223a 6661 6c73 657d       f.{"on":false}
17:56:21.469486 IP (tos 0x0, ttl 64, id 9221, offset 0, flags [DF], proto TCP (6), length 52)
    raspberrypi.http > HarmonyHub.59692: Flags [.], cksum 0x8248 (incorrect -> 0x8059), seq 1, ack 199, win 470, options [nop,nop,TS val 27682330 ecr 922263394], length 0
	0x0000:  0004 20f5 df60 b827 eb7a a42f 0800 4500  .....`.'.z./..E.
	0x0010:  0034 2405 4000 4006 949d c0a8 006d c0a8  .4$.@[email protected]..
	0x0020:  0064 0050 e92c e28b 421a f0e4 3549 8010  .d.P.,..B...5I..
	0x0030:  01d6 8248 0000 0101 080a 01a6 661a 36f8  ...H........f.6.
	0x0040:  9f62                                     .b
17:56:21.481207 IP (tos 0x0, ttl 64, id 9222, offset 0, flags [DF], proto TCP (6), length 271)
    raspberrypi.http > HarmonyHub.59692: Flags [P.], cksum 0x8323 (incorrect -> 0x3594), seq 1:220, ack 199, win 470, options [nop,nop,TS val 27682331 ecr 922263394], length 219: HTTP, length: 219
	HTTP/1.1 400 Bad Request
	Access-Control-Allow-Origin: *
	Content-Type: application/json; charset=utf-8
	Content-Length:93

	[{"error":{"address":"/lights/1/state","description":"body contains invalid JSON","type":2}}][!http]
	0x0000:  0004 20f5 df60 b827 eb7a a42f 0800 4500  .....`.'.z./..E.
	0x0010:  010f 2406 4000 4006 93c1 c0a8 006d c0a8  ..$.@[email protected]..
	0x0020:  0064 0050 e92c e28b 421a f0e4 3549 8018  .d.P.,..B...5I..
	0x0030:  01d6 8323 0000 0101 080a 01a6 661b 36f8  ...#........f.6.
	0x0040:  9f62 4854 5450 2f31 2e31 2034 3030 2042  .bHTTP/1.1.400.B
	0x0050:  6164 2052 6571 7565 7374 0d0a 4163 6365  ad.Request..Acce
	0x0060:  7373 2d43 6f6e 7472 6f6c 2d41 6c6c 6f77  ss-Control-Allow
	0x0070:  2d4f 7269 6769 6e3a 202a 0d0a 436f 6e74  -Origin:.*..Cont
	0x0080:  656e 742d 5479 7065 3a20 6170 706c 6963  ent-Type:.applic
	0x0090:  6174 696f 6e2f 6a73 6f6e 3b20 6368 6172  ation/json;.char
	0x00a0:  7365 743d 7574 662d 380d 0a43 6f6e 7465  set=utf-8..Conte
	0x00b0:  6e74 2d4c 656e 6774 683a 3933 0d0a 0d0a  nt-Length:93....
	0x00c0:  5b7b 2265 7272 6f72 223a 7b22 6164 6472  [{"error":{"addr
	0x00d0:  6573 7322 3a22 2f6c 6967 6874 732f 312f  ess":"/lights/1/
	0x00e0:  7374 6174 6522 2c22 6465 7363 7269 7074  state","descript
	0x00f0:  696f 6e22 3a22 626f 6479 2063 6f6e 7461  ion":"body.conta
	0x0100:  696e 7320 696e 7661 6c69 6420 4a53 4f4e  ins.invalid.JSON
	0x0110:  222c 2274 7970 6522 3a32 7d7d 5d         ","type":2}}]

from deconz-rest-plugin.

That’s interesting, the Harmony breaks the request up into three network packages, where it could easily fit a single package. The raspsberry does send an ACK for each package, so that shouldn’t be a problem. Looking at the payloads, the request seems correct, but it would be prudent to enable some logging in deCONZ to see if the re-assembled request is correct. Alternatively save the tcpdump to file and check how e.g. Wireshark re-assembles the request.

from deconz-rest-plugin.

Is there an option in deCONZ to enable logging of request bodies?

from deconz-rest-plugin.

Here's the re-assembled request from wireshark, pretty-exported (wireshark seems to properly parse the body-contents as JSON):

Frame 34: 78 bytes on wire (624 bits), 78 bytes captured (624 bits)
Ethernet II, Src: , Dst: 
Internet Protocol Version 4, Src: , Dst: 
Transmission Control Protocol, Src Port: 45659, Dst Port: 80, Seq: 187, Ack: 1, Len: 12
[3 Reassembled TCP Segments (198 bytes): #29(45), #31(141), #34(12)]
Hypertext Transfer Protocol
    PUT /api/D4E0709D44/lights/1/state HTTP/1.1\r\n
    connection: close, TE\r\n
    content-length: 12\r\n
    user-agent: LuaSocket 2.0.2\r\n
    te: trailers\r\n
    content-type: application/json\r\n
    host: 192.168.0.109\r\n
    \r\n
    [Full request URI: http://192.168.0.109/api/D4E0709D44/lights/1/state]
    [HTTP request 2/2]
    File Data: 12 bytes
JavaScript Object Notation: application/json
    Object
        Member Key: on
            False value
            Key: on

And the raw hex-ASCII of the same request:

0000   50 55 54 20 2f 61 70 69 2f 44 34 45 30 37 30 39  PUT /api/D4E0709
0010   44 34 34 2f 6c 69 67 68 74 73 2f 31 2f 73 74 61  D44/lights/1/sta
0020   74 65 20 48 54 54 50 2f 31 2e 31 0d 0a 63 6f 6e  te HTTP/1.1..con
0030   6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73 65 2c 20  nection: close, 
0040   54 45 0d 0a 63 6f 6e 74 65 6e 74 2d 6c 65 6e 67  TE..content-leng
0050   74 68 3a 20 31 32 0d 0a 75 73 65 72 2d 61 67 65  th: 12..user-age
0060   6e 74 3a 20 4c 75 61 53 6f 63 6b 65 74 20 32 2e  nt: LuaSocket 2.
0070   30 2e 32 0d 0a 74 65 3a 20 74 72 61 69 6c 65 72  0.2..te: trailer
0080   73 0d 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3a  s..content-type:
0090   20 61 70 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f   application/jso
00a0   6e 0d 0a 68 6f 73 74 3a 20 31 39 32 2e 31 36 38  n..host: 192.168
00b0   2e 30 2e 31 30 39 0d 0a 0d 0a 7b 22 6f 6e 22 3a  .0.109....{"on":
00c0   66 61 6c 73 65 7d                                false}

from deconz-rest-plugin.

Can the Hamony Hub discover DeCONZ as a Philips bridge or does there need to be done some other stuff before that could work?

from deconz-rest-plugin.

@ldplusse For me it discovered it fine. It will then ask you to press a button on the Philips Hue bridge, and when it did I just did a couple of "unlock gateway" from within the deCONZ web ui. It can take a couple of tries before getting past that "press the center button" screen on the harmony app.

from deconz-rest-plugin.

It might be a case sensitive problem of HTTP headers content-length instead of Content-Length, that expectation is a bug in deCONZ, will be fixed in 2.05.03.

from deconz-rest-plugin.

With the 2.05.03 version the control of individual lights via Harmony works perfect - thanks a lot. The control of groups (created in Harmony) still has some bugs - not able to turn them on/off. Once I managed it but not able to reproduce.

from deconz-rest-plugin.

Glad to hear we make progress :)

To investigate the group issue we need more data: snippets of Wireshark sniffer logs and deCONZ debug output would be helpful.

$ deCONZ --http-port=80 --dbg-info=2 --dbg-http=1

from deconz-rest-plugin.

I will be on a business trip next week - as soon as I will be back - I will try to get the info.

from deconz-rest-plugin.

It seems the Harmony for some reason creates a new group on the first request to turn on/off the group. I got a new group hgrp-00000000006.

In the debug-log I got this the first time I turned the group on:

08:21:03:183 HTTP API GET /api/79EAAF6642 - 192.168.0.104
... "groups":{"2":{"action":{"bri":127,"colormode":"hs","ct":0,"effect":"none","hue":0,"on":false,"sat":127,"xy":[0,0]} ...

which the harmony then follows up with the following request:

POST /api/0181805ACC/groups HTTP/1.1
connection: close, TE
content-length: 45
user-agent: LuaSocket 2.0.2
te: trailers
content-type: application/json
host: <<ip>>

08:21:04:245 HTTP API POST /api/0181805ACC/groups - <<ip>>
08:21:04:247 Text Data: 	{"lights":["2","1"],"name":"hgrp-0000000006"}

The group as shown by the web UI is empty (no lights). If I remove that group in the web UI it starts to work again (albeit a bit slow, but I don't have time to debug further right now)

I have more logs available if needed :)

from deconz-rest-plugin.

When i try to add the DeCONZ gateway to my Harmony Companion it says it found a Philips Hue bridge and ask to press the button. When i pres 'Authenticate App' in Phoscon, nothing happens in Harmony. Have also tryed 'Unlock Gateway' in the DeCONZ Web GUI.

I have turned off my Philips bridge, so it cannot be that.
How to add the Deconz to the Harmony Companion as a Philips bridge?

I am on DeCONZ 2.05.04 on windows 10.

from deconz-rest-plugin.

With the latest version of DeCONZ (2.05.06) it seems to work with groups as well. I do not know if there have been some changes regarding this issue - but I re-configured my groups directly with Harmony App after installing 2.05.06 and now I can switch them on/off and change brightness directly with the remote. Great experience - thank you!

from deconz-rest-plugin.

I have now been able to reproduce the "problem". When you create a group of lights in Harmony app and than first "use" the group with remote it will create a new group in DeCONZ. But they will not work on remote. If you delete this group in DeCONZ it will work properly with your remot :-)

In addition to that I found out, that the new version of DeCONZ sets a static IP on my Respberry....strange.

from deconz-rest-plugin.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from deconz-rest-plugin.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from deconz-rest-plugin.

I tryed to add DeCONZ to my Harmony Companion.
I select Philips HUE and it scans and finds nothing.
No different if i click on 'Authenticate App' in Phoscon before i start the scan.
I am on DeCONZ 2.05.74 on Windows 10 with the FW turned off.
I use ConBee and is on firmware 26330500.

Has it been confirmed that this also works when using Conbee. I see alot of entrys in this blog with the RaspBee.

from deconz-rest-plugin.