Description

When displaying cloth tracking in RVIZ, the edge markers are constructed using the rope edge function. Instead, the markers should be constructed from the tracked configuration's edges like in 953cebf. This was fixed in another branch but the changes were never pulled into the main branch.

This issue was brought to my attention by Chen Li of Tsinghua University.

To Do:

• Determine if I should pull in the TrackingMap changes
  • If I do, create a PR from 953cebf
  • Add other commits in tree to fix RVIZ visualization.
• If not, pull only relevant RVIZ changes from the commit above
  • Pull any tests that are necessary.
  • Adapt edge visualization function to only work with one template on master

I installed the git-lfs tool via sudo apt install git-lfs. This is the tool that actually does the large file storage.

The tool that's used to configure git-lfs to use dropbox, lfs-folderstore, is downloaded at: https://github.com/sinbad/lfs-folderstore/releases/download/v1.0.1/lfs-folderstore-linux-amd64-v1.0.1.zip
I then extracted the binary and put that in /home/dcolli23/.local/bin

I then have to configure lfs-folderstore

To download dropbox on linux, go to https://www.dropbox.com/install-linux and download the 64 bit deb. Sign into SSO UMich dropbox account. Execute the deb, install it. Execute dropbox from the ubuntu menu. finish installation.

• I then had to remove the shared folder exclusion via dropbox exclude remove Dropbox\ \(University\ of\ Michigan\)/ENGIN-ARMLab. This allowed the shared folder to sync with my local machine.

To get the bounding box to work in rviz I had to clone and catkin build the jsk_visualization repository. This should probably be added to the package.xml file dependency list

Running catkin_make returns following errors:

`CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "smmap_models" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "smmap_infrastructure" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "smmap_controllers" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "smmap_transitions" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "smmap_planning" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "smmap_utilities" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "kinematics_toolbox" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "kinematics_toolbox_ros_types" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "sdf_tools" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "deformable_manipulation_experiment_params" that is not in any export set.
CMake Error: install(EXPORT "cdcpd-targets" ...) includes target "cdcpd" which requires target "arc_utilities" that is not in any export set.
CMake Error at cdcpd/cdcpd_ros/CMakeLists.txt:116 (target_link_libraries):
Error evaluating generator expression:

$<TARGET_PROPERTY:torch_cpu,INTERFACE_COMPILE_OPTIONS>

Target "torch_cpu" not found.

CMake Error at cdcpd/cdcpd_ros/CMakeLists.txt:116 (target_link_libraries):
Error evaluating generator expression:

$<TARGET_PROPERTY:torch,INTERFACE_COMPILE_OPTIONS>

Target "torch" not found.

CMake Error at cdcpd/cdcpd_ros/CMakeLists.txt:116 (target_link_libraries):
Error evaluating generator expression:

$<TARGET_PROPERTY:torch_cpu,INTERFACE_COMPILE_DEFINITIONS>

Target "torch_cpu" not found.
`

Have you encountered something similar?

Besides, as I want to compare the results of cpd_output and gurobi_output in Rviz directly, I find the TY is output by TY = cpd(X, Y, TY_pred, Y_emit_prior);, but the Y is already updated by the newest gurobi_output (deformable_object_configuration_ ->tracked_.points_) in upper codes.

Thus, it also causes that the published message cdcpd/template cannot show the the pure CPD result, and it would cover the gurobi_output completely.

This issue is related to the issue #46.

I left it running for a while and my compute ran out of memory and killed the process, we need to fix this

I downloaded the "rosbags_compressed" zip folder and decompressed it. It has 2 demos. The first one 'demo_1_static_rope.bag' does work well, but the second 'demo_2_dynamic_rope_with_obstacle.bag' can't play the demo for dynamic rope. Could you please fix it? Thanks.

• use ~/.local instead of ~/local
• can we install libsvm, and cgal-qt5 from apt?
• NOMAD needs to be built/installed not just downloaded. Running install.sh worked for me, but we should add these instructions in the readme

The RGB, depth, and point cloud topics should not automatically subscribe to the topic if no topic is given. This leads to having to specify rgb, depth, and point cloud topics every time we run cdcpd even if we only want to use point clouds.

I imagine defaulting the topic names to an empty string and checking if they're empty before subscribing would be a sufficient implementation.

Right now, the starting orientation of the cloth matters; there are multiple local minima depending on rotations by 90 degrees.

While the install scripts are helpful, they're fairly outdated and install unnecessary packages as well as failing to install some necessary packages for CDCPD. These need to be updated.

Currently the cuda-samples repo is required to be cloned and maked to get freenect2 up and running. Should we document this here or should this be handled elsewhere? Maybe make an install script for freenect2?

This is the issue that I followed that helped me to make at least some progress: OpenKinect/libfreenect2#777

Hi,
Thanks very much for open source this amazing work.
I am trying to run this code with other examples using your virtual machine env. But somehow I cannot get the result I want. Below are some hints.
The code I use:

rosrun cdcpd_ros bagfile \
_bagfile:=/home/yixuanwang/catkin_ws/src/cdcpd/cdcpd_ros/dataset/rope_by_hand_6_comp \
_zeta:=2.0 \
_is_sim:=false \
_is_rope:=true \
_is_pred1:=false \
_is_pred2:=false \
_is_no_pred:=true \
_is_record:=true \
_is_gripper_info:=false \
_is_interaction:=false \
_translation_dir_deformablity:=5.0 \
_translation_dis_deformablity:=5.0 \
_rotation_deformablity:=10.0 \
_rope_points:=50 \
_init_pt_0:=-0.55 \
_init_pt_1:=-0.16 \
_init_pt_2:=1.5 \
_init_pt_3:=0.4 \
_init_pt_4:=-0.25 \
_init_pt_5:=1.5 \
_init_pt_6:=-0.20 \
_init_pt_7:=0.08 \
_init_pt_8:=1.3 \

Some output:

-------------------- frame 0 --------------------

Waiting for input, enter 'r' to run without stopping, anything else to step once ... 1
prediction choice: 0
Points in filtered: (1 x 1429)
Points in cloud before leaf: 1429
Points in fully filtered: 93
[ INFO] [1641613464.269561229]: Retrieving zeta                                               as 2
[ INFO] [1641613464.271275539]: Retrieving zeta                                               as 2
[ INFO] [1641613464.272059540]: Retrieving zeta                                               as 2
[ INFO] [1641613464.272719143]: Retrieving zeta                                               as 2
[ INFO] [1641613464.273252389]: Retrieving zeta                                               as 2
Error code = 26
License expired 2021-01-15
no prediction used
Points in filtered: (1 x 1429)
Points in cloud before leaf: 1429
Points in fully filtered: 93
real cheng is running...
Error code = 26
License expired 2021-01-15
before drawing
after drawing
after writing

Seems like something is wrong with the gurobi optimizer?

Thanks
Shijie Lin

CDCPD offline should either be updated to work again (and it's geometric_shapes package dependency documented in the README) or removed (along with the dependency on the geometric_shapes package).

It seems like a really helpful tool for resimulating bag files offline, so I'd like to keep it around and update it but if it's just going to sit there and decay it should be removed along with the specification of CDCPD's build dependency on the geometric_shapes package.

cdcpd prints using cout, which isn't great. I want to use ros logging, but we need a way to disable it if someone doesn't want ros.

Also, I'm seeing print statements like this:

original cheng is used
real cheng is running...

which are confusing, we ought to clean those up

Description

The rviz commands present in the "Demo" section are prefaced with roslaunch which to my understanding is not necessary. These should be removed once I confirm this is true.

Additionally, we're missing an additional cdcpd/ path in the Demo section since there is a nested cdcpd/cdcdp/ directory structure

arc_utilities is noted as a build dependency but it's currently ambiguous in the README as to where arc_utilities should be built. The README should be updated to include this information as well as information about any other dependencies for the workspace src folder specifically.

Poking around the code I noticed what might be a bug with the CPD dynamics prediction regularization. Currently, the prediction regularization added to the B matrix (when solving for W) is given by:

zeta * (Y_pred.transpose() - Y.transpose());

in the CPD inner loop. The problem is that:

1. I don't believe Y is updated in the inner loop, meaning that the prediction regularization doesn't change with our updated CPD TY.
2. If Y is updated in the inner loop (due to TY being a reference to Y, for example), then this is very confusing syntax and should be changed to reflect the paper more closely.

I propose that the prediction regularization term should instead be:

zeta * (Y_pred.transpose() - TY.transpose());

Need to incorporate the stretch parameter (lambda in the paper) into the cloth max segment length calculation.

Additionally, I'm not sure that taking the minimum of the grid lengths is appropriate. I think it logically should be the maximum length.

Code in question:

ClothConfiguration::ClothConfiguration(float const length_initial, float const width_initial,
    float const grid_size_initial_guess)
    ...
{
    ...
    max_segment_length_ = std::min({grid_size_initial_actual_length,
        grid_size_initial_actual_width});
}

where ... indicates omitted lines.

Examples are the parameters that control tracking rope vs cloth, and the size of the item, etc.

The constructor for CDCPD is messy and should be refactored. I've got most of the way there with https://github.com/UM-ARM-Lab/cdcpd/tree/cdcpd_constructor_refactor but there was a problem with the pybind11 bindings that I didn't have time to fix.

Hey, I get the following error message when trying to run the install_dep.sh script:

-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:4 (find_package):
  By not providing "Findcatkin.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "catkin", but
  CMake did not find one.

  Could not find a package configuration file provided by "catkin" with any
  of the following names:

    catkinConfig.cmake
    catkin-config.cmake

  Add the installation prefix of "catkin" to CMAKE_PREFIX_PATH or set
  "catkin_DIR" to a directory containing one of the above files.  If "catkin"
  provides a separate development package or SDK, be sure it has been
  installed.


-- Configuring incomplete, errors occurred!
See also "/home/josef/.local/share/pybind11_catkin/build/CMakeFiles/CMakeOutput.log".
make: *** No rule to make target 'install'.  Stop.

Have you encountered anything similar? Do you have any suggested solutions for this issue?

we could dramatically increase our use of const, which would improve grokking and debugging.

Currently max_segment_length = max_rope_length /

But this is incorrect as the denominator should be the number of segments in the rope, not the number of points.
We have:
num_rope_segments = num_points - 1

So the max segment length should be:
max_segment_length = max_rope_length / (num_points - 1)

hi,I followed your instructions to run the .bag file on your virtual machine, but I can only see the data processing process, and did not get the results like yours.Like the below picture.

Please help me how to get the running effect like yours, thank you very much.

we have other open source dependencies we are now including in this repo directly, so we should include their licenses

When poking around, I discovered that my reimplimentation of the routine was more numerically stable and lead to less vertex "scrunching" when inputting many many points or points in the point cloud that are quite far away from the tracked template. See image for visual description of "scrunching".

I checked for differences in CPD routines and parameters and I'm fairly sure the reason is due to the use of the less numerically stable, but faster, Householder QR decomposition in the CPD inner loop. Eigen docs say that this is less numerically stable than FullPivHouseholderQr or ColPivHouseholderQR. Seems that these methods check for matrix rank in the decomposition.

FAISS is misspelled in the CMake file for CDCPD. This leads to an unnecessary warning message even if FAISS is installed.

Offending code:

catkin_package(
        INCLUDE_DIRS include
        LIBRARIES cdcpd
        CATKIN_DEPENDS ${CATKIN_PACKAGES}
        DEPENDS OpenCV Eigen3 GUROBI FIASS Qt5 PCL # NOTE: adding CGAL errors, but it's working so whatever
)

Look at 0676ae0 to see the fix in another branch.

If there are gripper constraints, then when a "reset" is trigger, we should use the current positions of the grippers not the initial ones

Replace with ROS lookup.

Right now the tracker just outputs an error and then loops again; in general we will want some sort of behaviour that is reasonably robust to this sort of issue.

I also want to use the CPD+Physics method for comparison, where can I find the code?

CDCPD depends on pyrosmsg to build, specifically, it depends on the UM ARM lab's fork of pyrosmsg and I believe that cloning and attempting to build with the official up-to-date main branch of pyrosmsg does not work.

For this reason, should we add the UM ARM Lab's fork of pyrosmsg to CDCPD like victor_hardware_interface, sdf_tools, etc.?

The README's dependency list is somewhat ambiguous right now and has no indication that you need some UM ARM lab-specific forks of e.g. pyrosmsg.

It would help to explicitly list what versions of packages we're using as well as where we actually cloned them from.

Additionally, the install scripts need to be updated (#38).