It will be desirable to operate Dexters remotely via the Internet either to monitor, operate, and/or repair remote devices, or to make Dexters available for general purpose work.
Dexter supports socket connections natively, but those are typically limited to the local internet and do not pass through firewalls without additional (and often not allowed) holes being opened. WebSockets can operate freely between most networks, but despite the name, are NOT the same as native socket connections. WebSocket connections can be supported on the Dexter via a simple NodeJS proxy server, but getting out to the internet requires "chat" server accessible from the internet in general. The local Dexters NodeJS server would connect to the DexChat server, registering the robot as active and opening a WebSocket connection. Humans would log into the DexChat server via browser which would be served a web page which would then open WebSocket connections. The DexChat server would list available robots, perhaps filtering the list according to access rights.
The human could initiate a connection to the robot through the DexChat server. Messages would go to the server, which would then relay it to the open WebSocket connection to that Dexter. The NodeJS server on the Dexter would receive the message, relay it as a true socket message to the firmware on localhost, pick up the response, and relay it back to the chat server via websocket. The chat server would then relay that back to the human via the websocket connection to the browser. This is basically, like a Private Message on a standard chat server.
Latency
The major concern here is the speed of the connection. The NodeJS Proxy server takes about 1.6ms between Dexter and a Chrome web browser. Because NodeJS is known for very rapid response times and we are a JavaScript heavy house, we will try that first.
Latency compensation
But as we move out into the internet, latency of hundreds of ms are not unknown. As a result, Dexters haptic feedback system can get into oscillations or just lag, causing touch feedback to become unusable. The solution is probably to work towards predicting the next movement when sending data from the robot with the human, and predicting forces when sending data back. Obviously, the former is easier than the latter, but 3D scanning the work area and proximity detection before actual touch can help.
Forward Latency compensation
We've already experimented with predicting where the operator is moving the arm via spline curve fitting and it seems like that will work well, but we are concerned about noise in the system and it's needs more testing.
Back Latency compensation
Understanding what forces the remote arm will encounter /in the future/, which is required for latency compensation, is very difficult. Understanding the 3D environment and simulating when the arm will contact an object is one possible approach. Point cloud data is accesable via existing 3D scanners, and can be localized to the end effector to reduce data size. Having to transmit all the points back to the controller is a non-starter; it will be critical to difference the current scan and the prior scan and only transmit errors, or to convert point cloud to mesh, or some combination of that and other tactics.
The video game industry has done important work on this and we hope game developers will be interested in adding haptic feedback via the robot arm.