scazlab / baxter_tictactoe Goto Github PK

@alecive I tried the baxter tictactoe package on Baxter and encountered an issue during calibration stage. The tictactoeBrain node randomly gets stuck in its the constructor and the process doesn't exit gracefully after Ctrl-c is pressed.

I think there is a problem that the "setBrainState()" function tries to lock "_mutex_brain", which is a pthread mutex, in the class constructor before it's properly initialized.

Quote from http://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_mutex_lock.html

"If mutex does not refer to an initialized mutex object, the behavior of pthread_mutex_lock(), pthread_mutex_trylock(), and pthread_mutex_unlock() is undefined."

After doing a catkin_make clean to my workspace, the repo does not compile any more. Here is the output:

...
/home/alecive/code/catkin_ws/src/baxter_tictactoe/ttt_board_sensor/src/tictactoe_board_display/board_scheme.cpp:9:40: fatal error: ttt_board_sensor/ttt_board.h: No such file or directory
 #include "ttt_board_sensor/ttt_board.h"
                                        ^
compilation terminated.
In file included from /home/alecive/code/catkin_ws/src/baxter_tictactoe/tictactoe/src/vacuum_gripper/vacuum_gripper.cpp:1:0:
/home/alecive/code/catkin_ws/src/baxter_tictactoe/tictactoe/src/vacuum_gripper/vacuum_gripper.h:8:47: fatal error: baxter_core_msgs/EndEffectorState.h: No such file or directory
 #include <baxter_core_msgs/EndEffectorState.h>
                                               ^
compilation terminated.
In file included from /home/alecive/code/catkin_ws/src/baxter_tictactoe/ttt_board_sensor/src/ttt_cells.h:21:0,
                 from /home/alecive/code/catkin_ws/src/baxter_tictactoe/ttt_board_sensor/src/ttt_cells.cpp:1:
/home/alecive/code/catkin_ws/src/baxter_tictactoe/ttt_board_sensor/include/ttt/ttt_definitions.h:7:40: fatal error: ttt_board_sensor/ttt_board.h: No such file or directory
 #include "ttt_board_sensor/ttt_board.h"
                                        ^
compilation terminated.
In file included from /home/alecive/code/catkin_ws/src/baxter_tictactoe/tictactoe/src/robot_state/robot_state.cpp:1:0:
/home/alecive/code/catkin_ws/src/baxter_tictactoe/tictactoe/src/robot_state/robot_state.h:5:44: fatal error: baxter_core_msgs/AssemblyState.h: No such file or directory
 #include <baxter_core_msgs/AssemblyState.h>
                                            ^
compilation terminated.
...

I was going to debug it, and I didn't find the ttt_board.h file anywhere. @omangin I remember that you moved some files here and there. Is it possible that this is one of them? Maybe we did something wrong during the pull request and we lose some commits.

Valgrind message:

7,373,824 (704 direct, 7,373,120 indirect) bytes in 8 blocks are definitely lost in loss record 1,861 of 1,861
    at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
    by 0x6EA9216: cv::StdMatAllocator::allocate(int, int const*, int, void*, unsigned long*, int, cv::UMatUsageFlags) const (in /usr/local/lib/libopencv_core.so.3.1.0)
    by 0x6EAC6ED: cv::Mat::create(int, int const*, int) (in /usr/local/lib/libopencv_core.so.3.1.0)
    by 0x6ED519F: cv::_OutputArray::create(cv::Size_<int>, int, int, bool, int) const (in /usr/local/lib/libopencv_core.so.3.1.0)
    by 0x52D8C36: cv::ipp_cvtColor(cv::Mat&, cv::_OutputArray const&, int, int) (in /usr/local/lib/libopencv_imgproc.so.3.1.0)
    by 0x52E33B3: cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int) (in /usr/local/lib/libopencv_imgproc.so.3.1.0)
    by 0x79FEBBF: cv_bridge::toCvCopyImpl(cv::Mat const&, std_msgs::Header_<std::allocator<void> > const&, std::string const&, std::string const&) (in /opt/ros/indigo/lib/libcv_bridge.so)
    by 0x7A01253: cv_bridge::toCvCopy(sensor_msgs::Image_<std::allocator<void> > const&, std::string const&) (in /opt/ros/indigo/lib/libcv_bridge.so)
    by 0x7A0133F: cv_bridge::toCvShare(sensor_msgs::Image_<std::allocator<void> > const&, boost::shared_ptr<void const> const&, std::string const&) (in /opt/ros/indigo/lib/libcv_bridge.so)
    by 0x7A016B5: cv_bridge::toCvShare(boost::shared_ptr<sensor_msgs::Image_<std::allocator<void> > const> const&, std::string const&) (in /opt/ros/indigo/lib/libcv_bridge.so)
    by 0x4170E8: ttt::BoardState::image_callback(boost::shared_ptr<sensor_msgs::Image_<std::allocator<void> > const> const&) (in /home/baxter/ros_devel_ws/devel/lib/baxter_tictactoe/board_state_sensing)
    by 0x41C8CF: boost::_mfi::mf1<void, ttt::BoardState, boost::shared_ptr<sensor_msgs::Image_<std::allocator<void> > const> const&>::operator()(ttt::BoardState*, boost::shared_ptr<sensor_msgs::Image_<std::allocator<void> > const> const&) const (in /home/baxter/ros_devel_ws/devel/lib/baxter_tictactoe/board_state_sensing)

Note: to run valgrind when executing roslaunch, add 'launch-prefix="valgrind --log-file=/tmp/log --leak-check=full"' as an attribute of the node in the .launch file

• processImage should use more robust estimation of the board position in the 3D world. In particular, setOffsets needs to be improved.

This issue is done to track the code cleanup that is necessary after #5 and #7 will be closed.

This issue is to keep track of the things we should do at some point in the future:

• Create a single-package repo (see #5 )
• Automatic perception of board cell/board boundaries see #10
• Move Baxter's right arm away from camera's POV upon launching tictactoe.launch (currently done manually)
• Move from a set of predefined movements to a proper Inverse Kinematics algorithm (for the left arm)
• Feedback in event that Baxter does not successfully pick up tile
• Disable perception when Baxter is making move (screen shows 'ghost' red tiles due to camera interpreting Baxter's left arm as red tiles)
• Move vacuum gripper to tile position (currently tiles' position must be manually calibrated to Baxter's arm movements)
• Detect which tile Baxter is using (currently Baxter assumes use of blue tile by default)
• Adapt the vacuum gripper software to the new vacuum gripper (see ScazLab/human_robot_collaboration#5 )
• Fix img_screen node Error message: "/home/baxter/ros_ws/src/baxter_examples/scripts/xdisplay_image.py", line 36, in import cv2 ImportError: libopencv_core.so.3.1: cannot open shared object file: No such file or directory
• Move from festival to svox_tts (from here https://github.com/ScazLab/svox_tts )

Right now, we have a semi-working situation with the vacuum gripper (i.e. it works, but we didn't have time to code it properly). It would be nice to fully port the sw to a robust scenario.

@alecive If possible, will you bring back and test the fine calibration of arm based on hand camera? It would help in a dynamic environment. For example the table moves slightly over time or the tokens are needed to be cleaned from the chessboard by Baxter. Or do you have any special reason not to use the fine calibration? I've had my algorithm tested against static camera image but not yet merged into my code. Your experience helps.

Also, I have problem with the current way to sink end effector into tokens. The arm works fine with default position of token pool. However in my case some tokens are located closer to Baxter on the table. The current code has no problem to hover end effector over the tokens, but arm slides slightly towards the body of Baxter while reaching down onto the tokens, even if the code tell Baxter to lower end effector straight down.

My idea is that the computeIK() always allows a small amount of positional error in each invocation and always chooses the trajectory with least effort in joint rotation. The current gripToken() calls computeIK many times to lower the end effector, so the error accumulates and don't cancel each other. The arm drift would be noticeable if the lowering step is short enough.

I haven't found a perfect solution to work with the IR sensor.

Hey there,
I am trying use you vacuum_gripper package. I want to make a simple pick and place scheme for baxter using a vacuum gripper.I am using ROS Groovy.This is what I did so far :-
1.) Added the 2 .cpp files into the src folder and added vacuum_gripper.h file into my include folder.
2.) In the CMakeLists file, I added executable and target_link_library lines and I ran catkin_make from the root of the package.
I get the following error :
In file included from /home/baxter/Kshitij/baxter_workspace/src/baxter_inspace/vacuum_gripper/src/vacuum_gripper.cpp:1:0:
/home/baxter/Kshitij/baxter_workspace/src/baxter_inspace/vacuum_gripper/include/vacuum_gripper/vacuum_gripper.h:10:38: fatal error: baxter_msgs/GripperState.h: No such file or directory
compilation terminated.
make[2]: ** [baxter_inspace/vacuum_gripper/CMakeFiles/vacuum_gripper.dir/src/vacuum_gripper.cpp.o] Error 1
make[1]: *** [baxter_inspace/vacuum_gripper/CMakeFiles/vacuum_gripper.dir/all] Error 2
make: *** [all] Error 2
Invoking "make" failed

It seems like it doesnt find the gripperState.h file.
Any suggestions on how to fix this ?
Please advise.

@alecive The current arm service provided in baxter_collaboration::ArmCtrl class is blocking, and in turn the ArmCtrl::serviceCb() is blocking as well. It gave me hardship when I was trying to move both arms at the same time or make Baxter doing other social actions at the same time. I made an non-block arm server to do the job in worker threads and join them later in user code. However I feel an action server could be a better solution.

Maybe baxter_collaboration is the better place for this issue. Feel free to move it.

We were planning to move the two packages together into a single one, with further code cleanup and a more streamlined codebase.

Note:

• Detection of board & cells was partially incorporated into the master branch after merging the merge-package branch

To-do:

• Send board data as service
• Determine what to do w/ old way of using board data (saving it in ttt_board.xml and loading it)
• Update documentation to reflect changes
• FIx bug (currently cells_definition_auto node crashes board is not in camera's view upon launch
• (currently program doesn't work if pieces already on board) maybe have some kind of error message to user to remove pieces?

@omangin I thought you set up a tag that links to a working state of this repository with the old codebase. Isn't that true?

It would be incredibly beneficial to add some 2D to 3D projection of objects in the camera frame. We can assume the following information available a priori:

• end effector pose
• camera pose
• extrinsic and intrinsic camera calibration parameters
• some information about the size of the object we would like to track in 3D