Use camera's motion detection about neolink HOT 75 OPEN

Thanks very much, I am going to finish of my PR motion based streaming then I'll try this one

from neolink.

Agree, it's good to have a tracking issue.

The industry standard here is ONVIF, and if Neolink were an ONVIF bridge you could connect any client to it and it would "just work" with all these more advanced features. ONVIF is also somewhat long in the tooth, and I don't see many open source libraries for being an ONVIF server (except one in Javascript of all things).

Ideas for other standard APIs are welcome, but I'd like to avoid rolling something custom for Neolink - it'll just make work for anyone who wants to consume it.

from neolink.

Yes I actually wanted ONVIF but as you say I too couldn't find a server version.

from neolink.

Could this be helpful to you Guys?

EliasKotlyar/Xiaomi-Dafang-Hacks#78

from neolink.

I had a look at their PR that added it but it seems to be a binary commit so I can't read the source and the changes.
Some of the links in that issue may be useful though.

from neolink.

The actual code is in Dafang-Hacks/Main#41. Their overall approach is to use Gsoap to generate bindings to the ONVIF spec.

from neolink.

Thanks I've looked at that now my c++ is rusty (no pun intended) but it seems somewhat doable but quite some work.

It would be best if we could find a soap implementation in rust ive found this but it seems quite minimal. But perhaps a good starting point. It will take soap scheme and generate code. It's designed for a client though not a server.

We could alternatively generate c code with gsoap then either bind or port it.

from neolink.

This is promising here it's a soap schema->rust built with ONVIF in mind

from neolink.

This may also be of use in generating from soap

from neolink.

This is a simple server and client written in C++ but will probably be helpful

from neolink.

It would be pretty slick if it would output motion events via MQTT.

from neolink.

MQTT might be easier to implement than ONVIF too should be able to connect to home assistant as well.

I plan to add ONVIF when I have time but will also look into MQTT too

from neolink.

Great... I've been bugging manufacturers over and over about some simple MQTT publishes and subscribes. Would be awesome for doing integrations with NodeRed, HomeAssistant, or whatever.

from neolink.

I'm thinking to make the MQTT messages complimant enough to work with this HA plugin and use this rust library for the server side code

from neolink.

As I have understand the plugin we set the topic to the cameraname the subtopic to motion and the message as either motion_start motion_stop would that work with the design in nodered you had?

from neolink.

I don't have the plugin. Pretty wide open to however it would be implemented. A good idea might be to do a prefix configuration. Maybe something like //state/motion then a payload of ON or OFF.

neolink/reardoor/state/motion with a payload on or off

Thinking ahead where other stuff could be output as states, like LWT, etc. Then if it is possible, the ability to send commands to the camera like to turn on the white LEDs on the Lumus (not sure if that is possible)

send an on payload to neolink/reardoor/command/light

Might a bit much but you get the idea.

from neolink.

I don't mind but this would be going down the route of rolling our own interface, it might be better if there were a standard but it seems from most mqtt projects I have seen that it is more transport than protocol and they often roll there own messages

At least on the plus side the mqtt clients like nodered and HA are configurable in which message to listen for

from neolink.

I do eventually plan to get controls set up too but thats a longer project

from neolink.

@thirtythreeforty Could I get some recommendations on where to implement the subscribe for msg_id 33 (motion)

Currently we subscribe to id 3 in the start_video function. Then we loop polling the rx with a timeout. Do you think I should add a second subscription here in the loop polling id 33 as well?

from neolink.

@digiblur I think I have a working implementation in #78 would you be willing to test it?

from neolink.

@digiblur I think I have a working implementation in #78 would you be willing to test it?

Sure!

from neolink.

Cool, could you checkout the last version from my github assets here

Then add this to your neolink.toml

[[mqtt.servers]]
port = 8883
next_connection_delay_ms = 10
connection_timeout_ms = 100
max_client_id_len = 256
max_connections = 1
throttle_delay_ms = 0
disk_persistence = false
disk_retention_size = 100
disk_retention_time_sec = 1000
auto_save_interval_sec = 1000
max_payload_size = 2048
max_inflight_count = 100
max_inflight_size = 1024
instant_ack = false
# this enables tls connection
#cert_path = "tlsfiles/server.cert.pem"
#key_path = "tlsfiles/server.key.pem"
# provide ca_path to enable client authentication
#ca_path = "tlsfiles/ca-chain.cert.pem"

[mqtt.router]
id = 0
dir = "/tmp/rumqttd"
# A commitlog read will pull full segment. Make sure that a segment isn't
# too big as async tcp writes readiness of one connection might affect tail
# latencies of other connection. Not a problem with prempting runtimes (tokio)
max_segment_size = 10240
max_segment_count = 10
max_connections = 10001

Probably don't need all of those settings they are just the ones that I copy pasted from here

from neolink.

If you put neolink into debug mode with

export RUST_LOG=error,neolink=debug

Then you will see this for the debug messages

   Compiling neolink v0.3.0 (/Users/awk21/Projects/Software/neolink)
    Finished dev [unoptimized + debuginfo] target(s) in 16.53s
     Running `target/debug/neolink --config target/test.toml`
[2020-10-03T08:56:01Z INFO  neolink] Neolink v0.3.0-140-g0f83c7d debug
[2020-10-03T08:56:01Z DEBUG neolink] Create mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Start mqtt server
[2020-10-03T08:56:01Z DEBUG neolink] Start polling mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Send mqtt test
[2020-10-03T08:56:01Z INFO  neolink::mqtt::live] Starting MQTT Server
[2020-10-03T08:56:01Z DEBUG neolink] mqtt ready
[2020-10-03T08:56:01Z DEBUG neolink::mqtt::live] Incoming. Topic = /neolink, Payload = [b"start"]
[2020-10-03T08:56:01Z DEBUG neolink::gst] Permitting anonymous to access /Cammy02, /Cammy02/mainStream
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Connecting to camera at 192.168.1.101:9000
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Connected and logged in
[2020-10-03T08:56:01Z DEBUG neolink::bc_protocol::connection] Ignoring uninteresting message ID 78
[2020-10-03T08:56:01Z DEBUG neolink::bc_protocol::connection] Ignoring uninteresting message ID 79
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Camera time is already set: 2020-10-03 15:56:05 +7
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Starting video stream mainStream

Look out for these mqtt messages in the log to check its working

[2020-10-03T08:56:01Z DEBUG neolink] Create mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Start mqtt server
[2020-10-03T08:56:01Z DEBUG neolink] Start polling mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Send mqtt test
[2020-10-03T08:56:01Z INFO  neolink::mqtt::live] Starting MQTT Server
[2020-10-03T08:56:01Z DEBUG neolink] mqtt ready
[2020-10-03T08:56:01Z DEBUG neolink::mqtt::live] Incoming. Topic = /neolink, Payload = [b"start"]

You should be able to connect on port 8883 with your mqtt client and listen to messages

Topic = /neolink/Cammy02/status/motion, Payload = "on"
Topic = /neolink/Cammy02/status/motion, Payload = "off"

Where Cammy02 is your camera name

from neolink.

Where do you set the mqtt server at? Mine is just a simple auth one as it stays local.

from neolink.

Neolink IS the MQTT server.

from neolink.

Hmmm would you rather run it as a client?

from neolink.

Typically that is the case as I'd imagine most people doing home automation stuff with MQTT needs would already have a MQTT server running. I have one kicking with about 80 or 90 clients off of it already.

from neolink.

I see I see...

Oh well It was fun to make the server at least and learned lots doing it.

Give me a day or two and I'll swap it over a client. I'll call you again for a retest?

from neolink.

@digiblur I have a client based version up and running

Please download the latest version from same place as before.

Your config toml should look like this

[[cameras]]
name = "Cammy02"
username = "UserForCamera"
password = "passForCam"
address = "127.0.0.3:9000"
[cameras.mqtt]
server = "127.0.0.1"
port = 1883


[[cameras]]
name = "Cammy03"
username = "AnotherUserForCamera"
password = "passForAnotherCam"
address = "127.0.0.2:9000"
[cameras.mqtt]
server = "127.0.0.1"
port = 1883
credentials = ["User4MQTT", "Pass4MQTT"]

Each [[cameras]] must have a [cameras.mqtt] to post a message. Each camera runs its own client with the cilient id Neolink-CameraName.

The [cameras.mqtt] table supports the following options

[cameras.mqtt]
# The address of the mqtt server
server = "127.0.0.1" 

# The port of the mqtt server
port = 1883 

# Optional: User name and password for mqtt
credentials = ["User4MQTT", "Pass4MQTT"] 

# Optional: The servers encryption certificate
# Enabling this WILL turn on encryption
# You cannot use both ca and client_auth
ca = "/path/to/cert" 

# Optional: The clients encryption certificate and key
# Enabling this WILL turn on encryption
# You cannot use both ca and client_auth
client_auth = ["/Path/to/client/cert", "/Path/to/client/key"] 

from neolink.

It also send the message

topic: /neolink/CameraName/start
message: start

When neolink starts the client

If there any other message you think we should send please let me know.

from neolink.

Cool, I'll check it out. Is the start message configured as a LWT message to MQTT?

from neolink.

No idea what LWT is could you explain?

from neolink.

The start message is not a command just a report as a normal publish at the start.

from neolink.

P.s. it's getting late where I am so if you have issues I wont be able too help until morning. Hope it goes well :)

from neolink.

No worries. Basically when the client logs in it posts a message as last will and testament with the retained flag set. If the MQTT server detects the client is gone for whatever reason it will post the unavailable payload of the LWT. It's great for knowing if a service, device, etc fell off the network or crashed etc. https://www.hivemq.com/blog/mqtt-essentials-part-9-last-will-and-testament/

from neolink.

Oh so that is what the retain flag does! Thanks :)

from neolink.

I think I can maintain two messages, one for neolink online/offline and one for camera connected/disconnected this should be a handy features thanks :)

from neolink.

Perhaps a single message with

• Startup
  • Disconnected
• Camera connected
  • Connected
• Camera disconnected
  • Disconnected
• Neolink shutdown/terminate
  • Offline

from neolink.

It does need to be specified as the LWT though so the broker can flip it to offline if the device/service no longer responds.

from neolink.

Yes I think I get it. Needs LWT and a retain

from neolink.

Is it working for you otherwise?

from neolink.

Not sure. Is there a docker container of the change?

from neolink.

Umm do you need a docker?

from neolink.

Can't you just use the binary?

from neolink.

Can find the assets here

from neolink.

Will see if I can swap it out.

from neolink.

Should be a matter of

• download
• extract
• chmod 755 ./neolink
• ./neolink --config config_file_name

from neolink.

You could try my docker

docker pull quantumentangledandy/neolink:mqtt

But I have not tested it

Be aware that setting the mqtt server to 127.0.0.1 or localhost will not work from inside a docker as it has its own seperate network. You have to use host.docker.internal instead

from neolink.

I'll give it a shot today. Thanks!

from neolink.

Cool I see the MQTT client logging in... drop the initial / of the topic as it creates a null initial entry on MQTT explorer and others. Motion events seem to send across just fine. Very cool! I'm not seeing the LWT topic working though. It should come across as a retain and LWT option. I brought the container down and it didn't post the offline message after getting a MQTT timeout.

I'd also like to see a motion event retain message as an option. For myself I'd combine with the use of LWT to determine if the device was online or not then use the retained state to see the last event even during a home automation system reboot.

Definitely cool to see MQTT messages off a motion event! Nice! Is there a way to send messages to Reolink cams? Like the Lumus to turn on the white LEDs?

from neolink.

Yes I haven't had time to setup the retain or LWT yet been busy with other things. But I agree that we should retain the on off status of the motion event since they do represent continuous states.

Thanks for the update on the leading / will change when I can.

With regards to control messages: We are currently reverse engineering more of the protocol in #80, I think we now know enough to control aspects such as the LED. I shall add this functionality into the MQTT for you to test if you'd like, but will have to wait until I have the time.

from neolink.

Excellent and no rush! Looking good.

from neolink.

I have a build with the LWT and the new connection status setup, will take about an hour for it to publish to docker but you can test after that :)

from neolink.

@digiblur Ok so I have a working LED status light change over MQTT.

Please try the latest docker

docker pull quantumentangledandy/neolink:mqtt

The control done via is:

• topic: neolink/CAMERANAME/control/led
• message: on or off

This is not perfect yet but it seems to be working... p.s. so many changes to our protocol were required!

from neolink.

Been a few days since I've been able to circle back due to storm issues.... So the LED control kind of works. That's the status LED, the little one. There's another flood LED on the Lumus. Maybe a LED2? But it does indeed turn the little blue LED on and off.

I'm not sure if you got the LWT going yet, I don't see that one working yet.

from neolink.

Do you mean the red circle ones? Those are the IR lights, I have only ever seen the official client set thoses to "auto" or "close" so auto or off (close and off are the same things in some Asian languages). I can send it an "open" but since I have never seen the offical client send that it may not work

from neolink.

Also LWT should be up and working

from neolink.

Do you mean the red circle ones? Those are the IR lights, I have only ever seen the official client set thoses to "auto" or "close" so auto or off (close and off are the same things in some Asian languages). I can send it an "open" but since I have never seen the offical client send that it may not work

Nope. Not the IR. I guess technically the Lumus has 3 LED types. The IR LED, the Blue status light and the 2 white flood lights. Guess it would be hard to snag the commands it sends without the camera itself in hand.

I will check the LWT maybe I am missing the topic.

from neolink.

Well if your willing to use wireshark and a desktop reolink client you can snag the packets for me and send me the dump

from neolink.

Will do. Question on LWT, I was looking at the code quickly. Is the LWT topic and payload getting configured and sent at connect?

from neolink.

Yes its done on line 86 of mqtt.rs

        // On unclean disconnect send this
        mqttoptions.set_last_will(LastWill::new(
            format!("neolink/{}/status", name),
            "offline",
            QoS::AtLeastOnce,
            true,
        ));

Unless I have misread the docs on this

from neolink.

I am not familiar with that library. I will have to read up on it. I know on the Arduino code side I had to set the will options. Set the topic and death message so the MQTT broker knows what to publish if the client falls off. On connect I had to send the birth message to the same topic.

from neolink.

Yep yep thats how I am doing it, birth message is sent on line 204 after the client receives conack from the broker

from neolink.

Birth message starts off as disconnected but swaps to connected right after reolink connects to the camera (which dosen't take long) it will swap to disconnected if the camera goes offline but neolink stays online.

from neolink.

Ok at initial connect with the camera disconnected. I see two retains...the disconnected and motion off. That's good to see both of those agreeing.

Plug in the camera and wait for it to reconnect.

I can see the motion going on/off but the status is still disconnected.

from neolink.

@QuantumEntangledAndy Just wanted to check in on this, since I'd like to use the PR as a base for an ftp event. I have a Lumus I'll be using for this test. Does it look like this will be merged to main? Or needs more testing?

from neolink.

@brgavino Do you mean my #78 PR? I think it will need some more work really, this is why it is a draft. I am also not sure how thirtythreeforty feels about binding neolink with mqtt. I also think that #83 or something like it that restructures the message BC structures should be completed first aswell. Perhaps we should ping @thirtythreeforty for his comments.

from neolink.

@QuantumEntangledAndy Yeah #78 was what I was referencing. What dependencies do you see with #83 that are outstanding to close? Adding a general event forwarder interface could decouple mqtt, but the ability to send motion events is a big value add to the features here. If we have a plan for these PRs, I can pull in the right code to test ftp. Otherwise, I'll just take your last commit as a start and see what I can do.

from neolink.

@QuantumEntangledAndy Yeah #78 was what I was referencing. What dependencies do you see with #83 that are outstanding to close? Adding a general event forwarder interface could decouple mqtt, but the ability to send motion events is a big value add to the features here. If we have a plan for these PRs, I can pull in the right code to test ftp. Otherwise, I'll just take your last commit as a start and see what I can do.

That would be great as not everyone wants MQTT, but for the ones that do, it's a game changer.

from neolink.

If you have problems with compiling this branch/PR, see #78 (comment)

from neolink.

I realise there are some commands implemented but not sure of the limits so far. Can the light of e.g. Reolink Lumus be controlled using a command from Neolink? I.e. turn on motion activation of LEDs, or turn off LEDs completely, without using Reolink app.

from neolink.

@sumade123 I want to add this kind of LED as it was requested some time ago. However the camera I use only has a status LED and IR LEDs not this flood light LEDs. It would be really good if you (or someone else) could send me some wireshark dumps that capture the packets that the official client sends to activate the lights.

from neolink.

You can find a dump from my Lumus attached @QuantumEntangledAndy.
Frame 99 and 435 is turning the LED on and frame 255 and 572 turned the LED off.

Looks like the messageId is 288.

I hope this helps. Let me know if you have something that needs testing. Really looking forward to you getting some of your great PRs merged.

Lumus_led.pcapng.zip

from neolink.

@petergp thank you for the dump unfortunately this is using the newer AES firmware. It is AES encrypted and not easy for me to decrypt.

• Assuming it is encryption mode 02: To decrypt it I would need the NONCE that is send in the login msg_id 1 and the password for the camera.
• If it is encryption mode 03: To decrypt it.... we don't know at the moment that's on the todo list.

The easiest way to make it readable is to use an old reolink client that was released last year and record those packets. This should force it to use encryption mode 01 which is trivially breakable without any passwords.

from neolink.

Right, that explains why I wasn't able to make anything from the body.

I've included the dump from running an older version of the Reolink Android app in an Emulator. That version works a little different than the Reolink iOS app since the light will turn off after 180 seconds however this is configurable in the body as seen below. There must be a value for the duration that will turn it on permanently.

Turn on:

<?xml version="1.0" encoding="UTF-8" ?>
<body>
<FloodlightManual version="1.1">
<channelId>0</channelId>
<status>1</status>
<duration>180</duration>
</FloodlightManual>
</body>

Turn off:

<?xml version="1.0" encoding="UTF-8" ?>
<body>
<FloodlightManual version="1.1">
<channelId>0</channelId>
<status>0</status>
<duration>180</duration>
</FloodlightManual>
</body>

The relevant frames are 319 and 500.

Lumus_led_unencrypted.pcapng.zip

from neolink.