billmania / roboquest_ui Goto Github PK

After the addWidget feature is used to add a widget to the UI or to change the definition of a widget, what happens next? How does the widget become fully enabled? How does its configuration become persistent?

1. Automatically save the configuration
2. Request the user to explicitly save
3. Manually restart the software via a UI element
4. Manually reboot the robot
5. Saving the configuration automatically causes the container to restart and the browser to reload

With v14 of rq_ui, adding a Value, Slider, Button, or Indicator widget which uses a topic or service already in use by some other widget doesn't require any further activation. As soon as the widget is defined, it can immediately interact with the ROS graph.

This Issue refers then specifically to the scenario where a new topic or service is used by the new widget.

When the motors are OFF, consider disabling the joystick. The backend complains about receiving messages on the cmd_vel topic when the motors are OFF, but otherwise ignores the messages. The cmd_vel emissions from the browser on the socket.io channel are extra traffic, too, during this condition.
The same condition occurs when the servos are OFF.
Before making changes for this Issue, evaluate the added complexity of tying the state of one widget to the state of another widget.

The addWidget feature in Configuration settings doesn't completely configure a Joystick widget, at least when another Joystick widget is already configured. It creates the RQ portion of the widget but doesn't include the joystick knob inside that widget.

From the perspective of configuration.json, the widget is correctly configured. However, that second joystick widget can't be deleted using the UI. Neither can the original, fully functional, joystick be deleted.
If the original joystick is manually deleted from the configuration.json file and then added via "add widget" it's fully functional, but still can't be deleted.

With the browser connected to the socket and the motors ON, moving the joystick to a non-zero position and holding it there causes a flurry of TwistStamped messages onto the /cmd_vel topic. For example, between the start at 1693505815.984 and the last one at 1693505817.842, ie. 1.858 seconds, five TwistStamped messages were published.
The control pipeline from browser joystick widget to hardware motor control needs a tunable flow rate with a few parameters:

• maximum period at which to send the current joystick values, ie. at least every N seconds
• minimum period to send anything, ie. no more frequent than X
• maximum delay before sending the (0, 0) values

with never more than one joystick value queued for sending.

On the backend, the base node was queuing 5 cmd_vel messages and 18 servos messages.

Related to Issue #30 . See also roboquest_core Issue 29

1. debounce to rqparams.emit_frequency_limit = 0.5
2. debounce includes both repeater and interactive values
3. if value == 0 bypass debounce

For widgets, such as joystick and slider, which specify the topicPeriodS configuration attribute to continually emit events from the browser to the NodeJS server on the socket.io connection, only emit the events when the socket is connected with the NodeJS server. Don't start emitting until the socket is connected. Stop when the connection is broken, Resume when the connection is restored.

To reproduce, start the complete backend, connect the browser, verify the events are emitted, then shutdown the backend.

Add the option to add keyboard events to produce the same effect as joystick options. This will allow a student to drive the robot using the keyboard instead of the joystick widget.

• forward
• reverse
• turn left
• turn right

For each key, the keydown event will start the motion and the keyup event will stop the motion. The speed of motion is fixed by the value in the configuration file.

Add a keyboard driving mode, enabled by button.

This Issue includes the Slider support described originally in Issue #65

show widget min, max, current somewhere int he slider widget

Moving the slider doesn't cause the current slider value to be published until the mouseUp event. The current slider value is emitted once and then the interval resumes emitting what looks like the default slider value, even though the slider button is not at that value.

To reproduce, start the backend, start the frontend, connect the browser, use ros2 topic echo /servos to display the angle values. Move the slider without a mouseUP and note the published value doesn't change. Cause the mouseUp event and note a single published message at the current value after which the default value is continually published.

In web_server.js when handling a POST to save the configuration file, a check is made for the presence of a "config" section in the new configuration file. That section isn't provided by the browser and also isn't used anywhere.
Remove that check.
To reproduce the condition, start everything and click Save Config in the configuration settings dialog at the browser UI.
PR #41

use some other corners as anchors in the default file.
the processing to use them is all there.

Enable a slider to control the max speed of the drive motors. This will entail changes in rq_core. See billmania/roboquest_core#43

The slider will publish rq_msgs/msg/MaxSpeed to the topic named /motor_speed. The max_speed attribute will be set to the 8 bit positive integer 1 to 100 inclusive. 100 will be full speed and 1 will be the slowest possible speed greater than 0 (which will likely not actually spin the motors).

Clicking the Save Button in the menu of the browser UI sends a POST request to the backend but it doesn't include a configuration file in the body.

Issue #38 depends on this.

In order to update the application software on the robot, implement an UPDATE button in the browser UI which will cause the 'update' event to be emitted. The payload for this event is

{"timestamp": 0, "version": 1, "action": "UPDATE", "args": "UI"}

where the timestamp property is assigned the browser's current timestamp in seconds since the epoch.

The browser should expect the socket connection to be broken when the UPDATE process begins. When the UPDATE button is clicked, the user must be shown a new page. For guidance, see roboquest_ui/README_updating.md. For testing, see roboquest_core/scripts/update_cli.bash.

See Issue #45 for the corresponding backend change.

When displaying an active network interface, remove the network prefix size. For example, when the IP address is 192.168.0.14/24, display only 192.168.0.14. Including the CIDR "/24" is confusing to students.

To assign key events for Joystick, Slider, and Button widgets, one must manually edit the configuration.json file.
Add functionality to do this via Configuration Settings in the UI.

Clicking on the joystick knob doesn't move the center of the knob to track the mouse pointer. This occurs using Firefox 116.0.3 64 bit on ubuntu 20.04.

remove top left dropzone until it can edit
trash can in top right drop zone.
Using the trashcan area in the upper right currently throws an exception.

call the new positioning function on init and resize

The position of the widgets on the browser page must behave reasonably when the page is resized. After the page layout is changed, the new configuration is saved, and the robot is rebooted, the new page layout must be rendered

Don't allow the MENU button to be hidden behind a widget.

Consider combining with Issue #36

Rename the MENU button to CONFIGURE.

Consider combining with Issue #29

Update NodeJS code to support updated format of testing_config.json. Update docs/README_data_flow.md. See public/config/README_configuration.md for details of configuration attributes.

The format of the configuration file has been updated to better suit the UI implementation, to better organize the configuration attributes, and to remove obsolete attributes. This Issue applies to roboquest_core v12 and must be coordinated with roboquest_ui.

Use the CONFIGURE button and then the "add widget" button. Define a new widget of type "value". Set the topic, direction, and attribute. Set the format details. Click the "create" button and verify the widget appears.
Does it show any data?
Click the "save config" button.
Was anything recorded in the "format" or "data" sections of the widget's config?

Also, doesn't assign a unique widget id number.

Related to Issue #73

The blank square anchored to the top left corner of the page was intended as an "add widget" function. Remove it in favor of using the CONFIGURE button, add widget button.

The set socket button in the Configuration settings is no longer needed. Remove it.

add joystick formatting options easy to put in for the library used

Add the ability to assign a keyboard event to the button widget. Define one keyboard event which will have the same effect as clicking the button.

Related to Issue #64 and Issue #68

In web_server.js on the NodeJS side, the payload for the update event name is not being written to the FIFO. Instead, a dummy payload is written. Replace the dummy with the real payload from the browser.

See Issue #40 for the corresponding UI change.

put a debug toggle in the menu to turn console logs on and off, could tie to other features later

The position of the widgets on the browser page must behave reasonably when the page is resized. After the page layout is changed, the new configuration is saved, and the robot is rebooted, the new page layout must be rendered.

Dragging the slider can cause a high frequency of value emissions, which in turn causes increased latency in the response of the servos. To reduce the latency when dragging the slider, increase the "step" value and don't emit another event unless the value has changed by at least "step" OR a configurable amount of time has elapsed since the last emission.

Do not define an interval for continually emitting the same slider value.

Related to Issue #65

There are many TODO comments sprinkled throughout the JavaScript code in the public/js files. Review and resolve them, noting any which would require changes to any code outside of public/js.

Moving the joystick off the 0 position or depressing a joystick key and the start of drive motor motion sometimes has about a second of delay. Identify the source of the latency and reduce it.

when widget is dropped into top right it's supposed to show the form with values to edit

Not getting camera frames currently.

requested to have a quick switch shortcut between cameras.

can we support multiple cameras as video widgets?

or will we only support one at a time and switch which is feeding it.

Up to 50 robots can be connected to the same network at the same time. Therefore a unique ROS_DOMAIN_ID is required to keep each robot's ROS graph separate.
Implement a mechanism where at each boot, a value is assigned to ROS_DOMAIN_ID which is unique to that robot and can be used by both rq_core and rq_ui.

Depends on billmania/roboquest_core#41.

The payload emitted onto the socket is malformed.The malformed payload is

["angle,name":"178"]

but must instead be

{ "angle": 178, "name": "camera_pan"}

when creating a new widget, then dragging to a new position the offset isnt getting saved.

the drag-> stop needs to call the offset update function, had it in there got lost

To reproduce, render the page, drag a widget, save config, examine /opt/persist/configuration.json on the RaspPi file system. The position attribute will not have changed.

The likely cause of this is not updating the widget's position in jQuery(element).data('widget').position.

the background image is stretching to fit the screen, but should maintain its aspect ratio

When assigning a key for a Button, where the downValues and upValues are supposed to be empty, the downValues attribute must be set to an empty object, but this isn't happening.

To reproduce, choose a button without any keys and add a key, leaving the downValues and upValues untouched. Enable keys and press the newly assigned key. Nothing will happen.

Related to PR #82

Moving the slider doesn't cause the current slider value to be published until the mouseUp event. The current slider value is emitted once and then the interval resumes emitting what looks like the default slider value, even though the slider button is not at that value.

To reproduce, start the backend, start the frontend, connect the browser, use ros2 topic echo /servos to display the angle values. Move the slider without a mouseUP and note the published value doesn't change. Cause the mouseUp event and note a single published message at the current value after which the default value is continually published.

Reported by Sean on one robot kit. Not yet reproduced.

Without requiring the direct manual editing of the servo configuration file, provide a means via the browser UI to change the configuration of servos. This includes all of:

1. delete a configured servo
2. add a servo to an unused channel
3. change the configuration of an already defined servo.

The backend NodeJS server will support HTTP GET and POST for:

1. retrieve current servo configuration
2. save new servo configuration

The current configuration parameters for servos are:

channel - integer channel number from 0 to SERVO_QTY-1
servo_make_model - a string describing the servo
servo_angle_min_deg - integer minimum possible angle for the servo, usually 0
servo_angle_max_deg - integer maximum possible angle for the servo, usually 180
pulse_min_us - pulse duration in microseconds to command servo_angle_min_deg
pulse_max_us - pulse duration in microseconds to command servo_angle_max_deg
joint_name - a string name for the joint
joint_init_wait_ms - integer quantity of milliseconds to wait after
                     initializing the servo. the more time required
                     for the servo to move half its range, the greater
                     this value must be.
joint_angle_init_deg - integer initialization angle in degrees. since
                       there's no feedback from the servo, initialization
                       will often cause high acceleration.
joint_angle_min_deg - integer minimum possible angle for the joint
joint_angle_max_deg - integer maximum possible angle for the joint

This depends upon rq_core Issue 31 and rq_core PR 47
Include Issue #68

create a widget with a sparkline plot for values

Add a configuration.json file which defines all of:

• wall power state
• charger state
• motors state
• servos state
• charger ON and OFF
• motors ON and OFF
• servos ON and OFF
• slider for camera_pan
• slider for camera_tilt
• joystick for motor control
  Included with PR #51

The NodeJS server periodically emits an event with statistics about the traffic on the socket between the NodeJS server and the browser. The event name is "_counters" and the payload is a JSON string.

Every emission of event counters will have the timestamp_ms, _counters, sent, received, string, and object properties. Depending upon which events were emitted there will be more properties included. Each event name property will reference an object which has a sent property, a received property, or both.

A sample JSON string is

{ 
 "timestamp_ms": 1693075363114,
  "sent": 6911,
  "received": 1924,
  "string": 4039,
  "object": 4796,
  "telemetry": {
    "sent": 2088},
  "mainImage": { 
    "sent": 4796},
  "cmd_vel": {
    "received": 1082},
  "servos": {
    "received": 838},
  "_counters": {
    "sent":27 },
  "control_hat": {
    "received":4}
}

Implement an option in the browser UI which will display the payload from the _counters emission.

The current URL for the RoboQuest application is http://:3456/index.htm

Replace it with http://IP/

Resizing the browser page doesn't reasonably adjust the joystick position.
The joystick widget isn't repositionined correctly based on its anchors and offsets.

Other widgets are correctly adjusted.

NodeJS went from 18.17.1 to 18.18.0.

Change generate_messages.sh to expect both an architecture and a version.

The intended way to remove widgets from the UI page is to drag them to the blank square anchored to the top-right corner. Finish implementing that functionality. Verify the configuration is correct when using the "save config" button in Configuration settings.
It is conceivable that a more intuitive, easier implemented "remove a widget" functionality is discovered, while investigating a design for this functionality. See PR #75 and #81 for context and inspiration.