ros-industrial / ros_canopen Goto Github PK

Error description:

• sensorring cannot be initialized (init service returns success, but errors on terminal output)

Occurance(s) (when? Which hardware? How often?):

• always

How to reproduce the error:

• bringup
• init --> returns success, but throws errors (abort6060#0, reason: General error
  ) on terminal (see log below)

Log:

[ INFO] [1416558665.002256471]: Initializing XXX
[ INFO] [1416558665.002868878]: Current state: 1 device error: system:0 internal_error: 0 (OK)
[ INFO] [1416558665.003242759]: Current state: 2 device error: system:0 internal_error: 0 (OK)
RPDOs: 2
TPDOs: 3
Homing not started
Homing succesful
Propertly initialized module
[ INFO] [1416558665.332125875]: waitForService: Service [/arm_left/controller_manager/load_controller] has not been advertised, waiting...
[ INFO] [1416558665.442685945]: waitForService: Service [/sensorring/controller_manager/load_controller] is now available.
[ INFO] [1416558665.895025586]: joint_trajectory_controller loaded
[cob4-2-t3-0]: process[torso_cam3d_right/depth_metric_rect-15]: started with pid [3476]
[ INFO] [1416558666.516731790]: joint_group_position_controller loaded
[ INFO] [1416558666.854974967]: joint_group_velocity_controller loaded
[ INFO] [1416558666.874867110]: Switched Controllers
abort6060#0, reason: General error
[ERROR] [1416558666.885974748]: Exception thrown while processing service call: std::exception
[ERROR] [1416558666.886219071]: Service call failed: service [/sensorring/controller_manager/switch_controller] responded with an error: std::exception
[ERROR] [1416558666.886303233]: ServiceCall failed: switch_controller
[FATAL] [1416558666.889944173]: The switch controller stop and start list are not empty that the beginning of the swithcontroller call. This should not happen.
[ WARN] [1416558666.890120971]: Resource conflict on [sensorring_joint]. Controllers = [joint_trajectory_controller, joint_group_velocity_controller, ]
[ERROR] [1416558666.890235522]: Could not switch controllers, due to resource conflict
[ERROR] [1416558666.890466935]: Could not switch controllers

When it happens? When initializing some modules, or when the EM stop is pressed, for all of them.

This information was collected during the tests with the base.

Affected HW:

• It happens for the chain of 6 Elmos, configured with 1Mbit.
• For the individual Elmo modules

The console output is:

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<canopen::TimeoutException> >'
  what():  std::exception
[base/base_driver-1] process has died [pid 4985, exit code -6, cmd /home/thiago/indigo/devel/lib/canopen_motor_node/canopen_motor_node joint_states:=/joint_states __name:=base_driver __log:=/home/thiago/.ros/log/76fa58e8-a62f-11e4-bd4f-f01faf6235e5/base-base_driver-1.log].
log file: /home/thiago/.ros/log/76fa58e8-a62f-11e4-bd4f-f01faf6235e5/base-base_driver-1*.log 

After a small investigation, the Exception comes from the sdo.cpp file.

I could not solve the problem using the solution from ipa320/cob_robots@9131ed8 .

As some additional info, the following generated candump can be used.

cleanup regarding

• author(s), maintainer(s)
• URLs (source code, bug tracker)
• License --> LGPL
• description
• version number (same for all packages)

Error description:

• If the Joint Trajectory Controller Action is aborted, then stop

Occurance(s) (when? Which hardware? How often?):

• Always
• Elmo, Schunk, cob4-2

How to reproduce the error:

• bringup
• init
• move(within a big range in between a short period of time, what implies V>Vmax)

Prio A:

• timeout for homing --> done
• reflect motor state machine (switch mode, set velocities, ...) --> done
• proper checks for mode switching
• base class for modes (for mode layer)

Prio B:

• add 402 diagnostics (see #43)
• improve 402 threading
• move enums into namespace (402 namespace) or into SM classes
• encapsulate in base class (for motor layer)

[ INFO] [1427907991.122397041]: Recovering XXX
abort1003#1, reason: No data available
terminate called after throwing an instance of 'boost::exception_detail::clone_implboost::exception_detail::error_info_injector<canopen::TimeoutException >'
what(): std::exception
[arm_right/arm_right_driver-12] process has died [pid 16278, exit code -6, cmd /u/tdf/git/devel/lib/canopen_motor_node/canopen_motor_node __name:=arm_right_driver __log:=/u/fmw/.ros/log/49647b7e-d88f-11e4-a10f-eca86bff61c1/arm_right-arm_right_driver-12.log].
log file: /u/fmw/.ros/log/49647b7e-d88f-11e4-a10f-eca86bff61c1/arm_right-arm_right_driver-12*.log

a) this one works:
roslaunch cob_bringup robot.launch
init
rosservice call /arm_left/controller_manager/switch_controllers (stop_controllers = joint_trajectory_controller, start_controllers = joint_group_velocity_controller)

b) this one crashes (sometimes, but very often):
roslaunch cob_bringup robot.launch
init
rostopic pub /arm_left/joint_group_velocity_controller/command std_msgs/Float64MultiArray ...

The difference between a) and b) is that in a) we're only switching controllers and in b) we're switching controllers while publishing velocities. There are two subscribers (cob_control_mode_adapter and canopen driver) for the velocity topic. So the crash might be caused by a race condition when a callback in the driver is executed before the callback and switch in the cob_control_mode_adapter has finished successfully.
@thiagodefreitas: can you please investigate that on monday?
If that's true, the driver should ignore all velocity inputs as long as not all modules are switched successfully to velocity mode (initiated by the cob_control_mode_adapter).

@ipa-mdl: FYI

• use generic 'command' instead of pos, vel and eff
• enable object access in * to_device conversions

Error description:

• If the init service is not called, the Controller Manager re-spawns every minute

Occurance(s) (when? Which hardware? How often?):

• Always
• Elmo, Schunk, cob4-2

How to reproduce the error:

• bringup
• do not call init
• wait

Error description:

• If the arm is stopped during the movement, the arm twitches after recover

Occurance(s) (when? Which hardware? How often?):

• Always
• Elmo + Schunk

How to reproduce the error:

• bringup
• init
• move + EM stop
• release EM Stop
• recover

Implement checks for heartbeat producer, that checks if nodes are configured for a heartbeat.
Ensure single producer per ID.

Some vendors use hex numbers for "Unsigned#" data types in eds files. It would greatly enhance the compatibliity if the parser would accept these values.
And according to CanOpen 306 chapter 4.3 this should be supported.

Example EDS:
[MandatoryObjects]
SupportedObjects=0x03
1=0x1000
2=0x1001
3=0x1018

For the whole file see: http://en.nanotec.com/products/1604-pd4-c-high-pole-plug-drive-dc-servo-motor/

Error description:

• After init, the arm moves(mostly only one joint)

Occurance(s) (when? Which hardware? How often?):

• Only sometimes
• Schunk LWA4d, cob4-2 arms

How to reproduce the error:

• bringup
• init

When sending a joint_group_velocity/command with joint velocities of 0.01, the arm moves very jerky, i.e. stop-and-go.
When joint velocities of 0.1 are sent, the motion is much smoother and continuous.

Solution: Check error tolerances/accuracy of Schunk motor controller

Error description:
Switching from JTC to JGVC works, I can move sending velocities. Swithcing back to JTC does not work. controller manager says switching ok (JGVC stoppend and JTC running) but no movement is possible.
The state topic reports that the actual pos is static while the JTC tried to move setting the desired pos.

Occurance(s) (when? Which hardware? How often?):
always
cob4-2 arms

How to reproduce the error:

• bringup
• init
• move JTC (e.g. through dashboard) --> works
• move JGVC (e.g. through rqt or rostopic pub std_msgs/Float64MultiArray /arm_left/joint_group_velocity_controller/command) --> works for Schunk, does not work for Elmo (see log below)
• move JTC (e.g. through dashboard) --> does not work

multiple calls to recover solve the error --> can move JTC again

Error description:

• arm cannot be initialized (init service is blocking forever)

Occurance(s) (when? Which hardware? How often?):

• sometimes (often multiple restarts of bringup helps)

How to reproduce the error:

• bringup
• init --> blocks sometimes and throws "boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::PointerInvalid >"

Log:
[ INFO] [1416557367.643892202]: Initializing XXX
[ INFO] [1416557367.644528496]: Current state: 1 device error: system:0 internal_error: 0 (OK)
[ INFO] [1416557367.644858421]: Current state: 2 device error: system:0 internal_error: 0 (OK)
RPDOs: 3
TPDOs: 2
RPDOs: 3
TPDOs: 2
[ INFO] [1416557367.837664433]: waitForService: Service [/sensorring/controller_manager/load_controller] has not been advertised, waiting...
[ INFO] [1416557367.912405622]: waitForService: Service [/arm_left/controller_manager/load_controller] has not been advertised, waiting...
RPDOs: 3
TPDOs: 2
RPDOs: 3
TPDOs: 2
RPDOs: 3
TPDOs: 2
RPDOs: 3
TPDOs: 2
RPDOs: 3
TPDOs: 2
Propertly initialized module
Propertly initialized module
Propertly initialized module
Propertly initialized module
Propertly initialized module
Propertly initialized module
Propertly initialized module
[ INFO] [1416557369.322104614]: waitForService: Service [/arm_left/controller_manager/load_controller] is now available.
[ INFO] [1416557369.876763953]: joint_trajectory_controller loaded
[ INFO] [1416557370.349831145]: waitForService: Service [/arm_right/controller_manager/load_controller] has not been advertised, waiting...
[ INFO] [1416557370.526872957]: joint_group_position_controller loaded
[ INFO] [1416557370.866774447]: joint_group_velocity_controller loaded
[ INFO] [1416557370.886587375]: Switched Controllers
[ERROR] [1416557370.896748454]: /u/fmw/git/care-o-bot/src/ros_canopen/ipa_canopen_master/include/ipa_canopen_master/objdict.h(342): Throw in function void ipa_canopen::ObjectStorage::Entry::set(const T&) [with T = int]
Dynamic exception type: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::PointerInvalid >
std::exception::what: std::exception

[ INFO] [1416557370.907292199]: Switched Controllers
[ERROR] [1416557371.897297268]: /u/fmw/git/care-o-bot/src/ros_canopen/ipa_canopen_master/src/pdo.cpp(270): Throw in function void ipa_canopen::PDOMapper::RPDO::sync()
Dynamic exception type: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::TimeoutException >
std::exception::what: std::exception

[ INFO] [1416557372.860173446]: waitForService: Service [/sensorring/controller_manager/load_controller] has not been advertised, waiting...
[ERROR] [1416557372.897410983]: /u/fmw/git/care-o-bot/src/ros_canopen/ipa_canopen_master/src/pdo.cpp(270): Throw in function void ipa_canopen::PDOMapper::RPDO::sync()
Dynamic exception type: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::TimeoutException >
std::exception::what: std::exception

[ERROR] [1416557373.898152641]: /u/fmw/git/care-o-bot/src/ros_canopen/ipa_canopen_master/src/pdo.cpp(270): Throw in function void ipa_canopen::PDOMapper::RPDO::sync()
Dynamic exception type: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::TimeoutException >
std::exception::what: std::exception

[ERROR] [1416557374.898411403]: /u/fmw/git/care-o-bot/src/ros_canopen/ipa_canopen_master/src/pdo.cpp(270): Throw in function void ipa_canopen::PDOMapper::RPDO::sync()
Dynamic exception type: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::TimeoutException >
std::exception::what: std::exception

@ipa-mdl and @thiagodefreitas: the arm stopped during normal operation, recovering was not successfull (error state 41)

Full Name: /Actuators/Arm Right/arm_right arm_right_driver: arm_right_1_joint
Component: arm_right arm_right_driver: arm_right_1_joint
Hardware ID: none
Level: ERROR
Message: Node has emergency error

error_register: 41
errors: 

after some attempts to recover I saw the node crashing

[ERROR] [1428438184.127710041]: /u/hmi/git/care-o-bot/src/ros_canopen/canopen_master/include/canopen_master/objdict.h(371): Throw in function void canopen::ObjectStorage::Entry<T>::set(const T&) [with T = int]
Dynamic exception type: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<canopen::PointerInvalid> >
std::exception::what: std::exception

[ERROR] [1428438186.824844120]: /u/hmi/git/care-o-bot/src/ros_canopen/canopen_master/include/canopen_master/objdict.h(371): Throw in function void canopen::ObjectStorage::Entry<T>::set(const T&) [with T = int]
Dynamic exception type: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<canopen::PointerInvalid> >
std::exception::what: std::exception

• Move all generic utils into a canopen_utils packages
• move rig setups to ipa320/cob_test_robots
• clean-up

Move declarations and templates into dedicated headers.

@thiagodefreitas
After homing (if necessary) the drive mode is switched back to the default mode (before homing), but the init method does not wait until the mode is entered.
This leads to an inconsistent state, that introduces problems at other layers.

std_srvs/Trigger is in the release pipeline already, see ros/rosdistro#7964.

we'll need to change the dependencies to use std_srvs/Trigger instead of cob_srvs/Trigger

Line 86 in https://github.com/ipa320/ros_canopen/blob/indigo_dev/socketcan_interface/include/socketcan_interface/asio_base.h starts an additional thread to call io_service::run() even though it gets called in the current thread at line 91. The second thread does not seem to provide much in the way of performance enhancements, and it causes CAN frames to get dispatched out of order occasionally when the data rate is very high.

This was seen running a CAN device that outputs ~80 messages at 25 Hz each (total of 2000Hz). Simply commenting out line 86 fixed the reordering issue. Without that line, boost/thread/thread.hpp no longer needs to be included.

Error description:

• The sensorring twitches after init+recover, even though no move command has been given

Occurance(s) (when? Which hardware? How often?):

• Always
• Elmo

How to reproduce the error:

• bringup
• init
• recover

Error description:

• The node dies after sometime in standby. The error listed is controller problems on the 301.

Occurance(s) (when? Which hardware? How often?):

• After recovering the node, everything appeared to properly work, then after sometime in Standby without sending commands, the node died. Time not precisely recorded, but no more than 10 minutes.
• Schunk LWA4d with Peak-USB adapter

How to reproduce the error:

• Possible hint: Start the node, move the arm, then let it for 10+ minutes in standby mode.

Error description:

• if one module has an error it is still possible to move other modules of the same chain --> this is not good. If that happens, we should not allow to move any module and report an error

Occurance(s) (when? Which hardware? How often?):

• Schunk LWA4D

How to reproduce the error:

• bringup
• init
• move
• send velocity commands for one joint (e.g. using rqt)
  --> after that is is not possible to move the joint where velocities were sent to (see #34) , but I can move all other joints

The current implementation might not work properly with multiple chains.

Issues:

• shared memory clean-up
• permission settings between multiple users
• crash recovery of locks (#53)

To be refactored:

• Book keeping of processes
• Book keeping of node IDs
• Master mutex should not be locked all the time
• Crashed masters should be detected
• Recovery
• Diagnostics for missed intervals etc.
• plugin interface for easy exchange of sync implementation

@ipa-mdl @thiagodefreitas: here are two candumps from the cob4-6 base which crashes very reproducible after a few seconds of operation.

crash directly after init and loading of twist_controller: https://gist.github.com/ipa-fmw/55cc9892ee2c5dc80b5b

crash after driving for some seconds: https://gist.github.com/ipa-fmw/9b8bc16f32f32cc282a3

After starting the driver with the following command:
roslaunch cob_bringup canopen_402.launch can_module:=can3 component_name:=arm_left

core service [/rosout] found
process[arm_left/arm_left_driver-1]: started with pid [11549]
terminate called after throwing an instance of 'boost::interprocess::interprocess_exception'
what(): Permission denied
process[arm_left/arm_left_joint_state_controller_spawner-2]: started with pid [11564]
[arm_left/arm_left_driver-1] process has died [pid 11549, exit code -6, cmd /u/robot/git/care-o-bot/devel/lib/canopen_motor_node/canopen_motor_node joint_states:=/joint_states __name:=arm_left_driver __log:=/u/robot/.ros/log/94872932-9d5f-11e4-a05f-eca86bff61c1/arm_left-arm_left_driver-1.log].
log file: /u/robot/.ros/log/94872932-9d5f-11e4-a05f-eca86bff61c1/arm_left-arm_left_driver-1*.log
process[arm_left/control_mode_adapter-3]: started with pid [11565]
[ INFO] [1421402705.666247316]: waitForService: Service [/arm_left/controller_manager/load_controller] has not been advertised, waiting...
[ INFO] [1421402710.684724028]: waitForService: Service [/arm_left/controller_manager/load_controller] has not been advertised, waiting...
^C[arm_left/control_mode_adapter-3] killing on exit
[arm_left/arm_left_joint_state_controller_spawner-2] killing on exit
[WARN] [WallTime: 1421402712.897394] Controller Spawner couldn't find the expected controller_manager ROS interface.
[arm_left/control_mode_adapter-3] escalating to SIGTERM
shutting down processing monitor...

Possible reasons ?
Shared memories are not released properly?
SocketCAN interface not functional

Error description:

• If recovering while EMStop is still active the node freezes and recover does not return

Occurance(s) (when? Which hardware? How often?):

• always
• cob4-2 arms

How to reproduce the error:
bringup
init
move
EMStop
recover (without releasing EMStop)

Error description:

• After init, only a part of the modules can be moved, although the service returns OK

Occurance(s) (when? Which hardware? How often?):

• sometimes
• Schunk LWA4D, cob4-2 arms

How to reproduce the error:

• bringup
• init
• move

We're breaking the OSRF buildfarm at the moment. A new release will fix that.

Code is already pushed to the indigo_release_candidate branch. Next step is to release with bloom and push to the release repository.

@shaun-edwards: please add the release repository (https://github.com/ros-industrial/ros_canopen-release) to the admin team (https://github.com/orgs/ros-industrial/teams/canopen-admin/repositories) so that we can continue with the release. Thanks.

There are no entries in the velocity section of the joint_states topic

---
header: 
  seq: 8573
  stamp: 
    secs: 1421858827
    nsecs: 323728640
  frame_id: ''
name: ['arm_left_1_joint', 'arm_left_2_joint', 'arm_left_3_joint', 'arm_left_4_joint', 'arm_left_5_joint', 'arm_left_6_joint', 'arm_left_7_joint']
position: [0.6421589916862737, 1.1260340735091816, -0.343812409350363, -0.00757472895365539, 0.7335095247356569, 0.9614320717535964, -0.3070208687183225]
velocity: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
effort: [0.0, 2.89e-321, 2.895e-321, 1.040420863149002e+224, 0.0, 1.8146065985459775e+237, 0.0]

---

not even in velocity mode

When the joint limit is reached, e.g. arm_7_joint, and the control_mode_adapter switches back from velocity to trajectory mode, switching fails and the switching service returns false, and by false the controller_manager stays in velocity control mode but the joints that were not in the joint limits, have already switched to trajectory mode while arm_7_joint is in error state(joint limits) and thus cannot switch.

The desired behavior would be to do not switch any joint, in case one has an error state.

The socketcan txqueue might run full on fast updates, socketcan_interface does not handle this properly.

A temporary fix is to increase the queue len:

sudo ip link set <NETWORK> txqueuelen 15 # or even higher, default: 10

socketcan_interface should detect the buffer overrun and handle it properly.
Problem: In most case the error is caught in the following read and not in the actual write.
Use async write instead?

After setting new limits with the Schunk software, it is not possible to initialize with ROS again. This can be solved with multiple (>3x) restarts of bringup+init

Exceptions should list the objects they are related to.
In addition exception should only used where necessary.

After init and command a Twist msg come the error:

CAN not ready; RPDO timeout;

and the base is not moving.

after an EMStop it was not possible to recover. Error message:

rosservice call /arm_right/driver/recover
success:
data: False
error_message:
data: /u/fmw/git/care-o-bot/src/ros_canopen/ipa_canopen_402/src/ipa_canopen_402/ipa_canopen_402.cpp(374): Throw in function virtual void ipa_canopen::Node_402::recover(ipa_canopen::LayerStatus&)
Dynamic exception type: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::TimeoutException >
std::exception::what: std::exception

It was possible to shutdown, but not possible to init again. Error message:
rosservice call /arm_right/driver/init
success:
data: False
error_message:
data: /u/fmw/git/care-o-bot/src/ros_canopen/ipa_canopen_402/src/ipa_canopen_402/ipa_canopen_402.cpp(760): Throw in function virtual void ipa_canopen::Node_402::init(ipa_canopen::LayerStatus&)
Dynamic exception type: boost::exception_detail::clone_implboost::exception_detail::error_info_injector<ipa_canopen::TimeoutException >
std::exception::what: std::exception

candump for init see https://gist.github.com/ipa-fmw/ae1a15b3c3ec952dcdc6

Notes:

• During EMStop a new JTC command was sent. (Don't know if this is important to know).

The gripper driver crashes at startup because of the pos and vel_unit_factor in the yaml-file...

• should give some human readable information about the status of each module and each chain.
• should at least report overall, status, bus error, under voltage error (e.g. EMStop active)

should include:

• setting modul ids
• setting limits (table for earch joint)
• set motor direction (for arm_1_joint)

in canopen_master/src/node.cpp we currently need

for SCHUNK:

        reset();

and for ELMO (if more than one module is on the bus):

        reset_com();
        emcy_.recover();

@thiagodefreitas, @ipa-mdl: please think about a solution which works for both

Error description:

• After initializing there are 0x to 10x recover calls necessary to be able to move

Occurance(s) (when? Which hardware? How often?):

• Always if not recovered before. Amount of recover calls varies.
• Schunk LWA4d, cob4-2 arms

How to reproduce the error:

• bringup
• init
• 0x to 10x recover ==> this should always be 0x

Error description:

• When switching from Joint Trajectory Controller to Joint Position Controller, the 402 has an "invalid pointer".

Occurance(s) (when? Which hardware? How often?):

• Always during the tests on 11/12/2014
• Schunk LWA4d

How to reproduce the error:

• roslaunch schunk_lwa4d lwa4d.launch
• init
• rqt "stop Joint Trajectory Controller"
• rqt "start Group Position Controller"

needs to be done in preparation to release the repository under the ROS-I label

Error description:
Segmentation fault when publishing a velocity message before moving the arm with Trajectory Controller

Occurance(s) (when? Which hardware? How often?):
most times
cob4-2 arms

How to reproduce the error:
bringup
init
do not use(move the arm with) trajectory controller
publish velocity message

Propertly initialized module
[ INFO] [1416591307.171674507]: waitForService: Service [/arm_left/controller_manager/load_controller] is now available.
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[New Thread 0x7fffc8996700 (LWP 9766)]
[ INFO] [1416591307.552940521]: joint_trajectory_controller loaded
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[New Thread 0x7fffa7de7700 (LWP 9769)]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[ INFO] [1416591307.862866977]: joint_group_position_controller loaded
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[tcsetpgrp failed in terminal_inferior: Inappropriate ioctl for device]
[ INFO] [1416591308.042937821]: joint_group_velocity_controller loaded
Enter mode7
Enter mode7
Enter mode7
Enter mode7
Enter mode7
Enter mode7
Enter mode7
[ INFO] [1416591308.192893655]: Switched Controllers. From no_stop_controller_defined to joint_trajectory_controller
---Type to continue, or q to quit---

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe4ff9700 (LWP 9752)]
0x00007fffa71b8d55 in forward_command_controller::ForwardJointGroupCommandController<hardware_interface::VelocityJointInterface>::commandCB (this=0x7fffd0076c80, msg=...)
at /u/fmw/git/care-o-bot/src/ros_controllers/forward_command_controller/include/forward_command_controller/forward_joint_group_command_controller.h:128
128 { commands_[i] = msg->data[i]; }
(gdb) bt
#0 0x00007fffa71b8d55 in forward_command_controller::ForwardJointGroupCommandController<hardware_interface::VelocityJointInterface>::commandCB (this=0x7fffd0076c80, msg=...)

at /u/fmw/git/care-o-bot/src/ros_controllers/forward_command_controller/include/forward_command_controller/forward_joint_group_command_controller.h:128

#1 0x00007fffa73de12c in operator() (a0=..., this=)

at /usr/include/boost/function/function_template.hpp:767

#2 boost::detail::function::void_function_obj_invoker1<boost::function<void (boost::shared_ptr<std_msgs::Float64MultiArray_<std::allocator > const> const&)>, void, boost::shared_ptr<std_msgs::Float64MultiArray_<std::allocator > const> >::invoke(boost::detail::function::function_buffer&, boost::shared_ptr<std_msgs::Float64MultiArray_<std::allocator > const>)

(function_obj_ptr=..., a0=...) at /usr/include/boost/function/function_template.hpp:153

#3 0x00007fffa73de5d1 in operator() (a0=..., this=0x7fffd00773a8)

at /usr/include/boost/function/function_template.hpp:767

#4 ros::SubscriptionCallbackHelperTboost::shared_ptr<std_msgs::Float64MultiArray_<std::allocator const> const&, void>::call (this=0x7fffd00773a0, params=...)

at /opt/ros/indigo/include/ros/subscription_callback_helper.h:144

#5 0x00007ffff69d5498 in ros::SubscriptionQueue::call() ()

should be documented in github README. we should document

• internal architecture and related packages (can, canopen, 301, 402, ros_control plugin, ...)
• Code API (non-ROS)
• how to extend?
• what needs to be done to setup a device?

I think the calibration rising values from the urdf are ignored at the moment...

we should document

• architecture (linking to ros_control)
• list controller which are tested with ros_canopen
• list namespaces
• list services (init, recover, halt, shutdown)
• list topic and action interfaces (per controller)
• add example yaml and launch files (maybe link to schunk_robots)
• show the user how to move the components (prepare can, roslaunch, init, move with rostopic pub and rqt plugin)

please document in http://wiki.ros.org/ros_canopen

currently canopen_test_utils depends on gazebo_ros and gazebo_ros_control. As this package is only used for testing it is not worth to depend on gazebo with the whole repository.