#79 copies CTest Test.xml files over to the directory specified by --test-result-base, but not the *.xunit.xml result files. Since I am looking for my xunit "test results", the following documentation is misleading to me:

The base path for all test results. The default value is the --build-base argument. Each package uses a subdirectory in that base path as its package specific test result directory.

https://colcon.readthedocs.io/en/released/reference/verb/test.html

It seems like *.xunit.xml files should be copied over as well, or the documentation could be clarified.

A package not integrating any artifacts, makes the build system think there's a bug and it throws out a warning. It's usually ignored for packages not intended to be ever distributed.

But in some cases, like for a package whose entire purpose is to run tests and is only integrated when -DBUILD_TESTING=ON, the above assumption doesn't hold true.

If the warning is bothersome, brute force fixes (just for reference):

• Have the test package install some innocuous empty hello_world.txt file.
• Set "cmake-target": "all" in colcon.pkg will ensure when a specific target is specified, the install target is no longer added implicitly.

The short-term clean fix is to include the required artifacts will added when BUILD_TESTING is true and allowing the ament_package() call to happen even when its false, resulting in the package.xml getting installed. See test_rclcpp example.

if(BUILD_TESTING)
  ### code block
endif()

# TODO should not install anything
ament_package()

A more elegant but time consuming fix might be to add support for a field in colcon.pkg "ignore-no-install-target": true which does nothing but suppresses that message when there is no install target. Then give these packages a colcon.pkg file that's simply

{
    "ignore-no-install-target": true
}

This is an under-serviced scenario for both ament and colcon. The "no install target" warning message should not be disabled entirely. Possibly a new package type for test-only CMake projects could be introduced, but needs further discussion.

I think the fact that release mode can only be specified with -DCMAKE_BUILD_TYPE=Release should be documented somewhere besides this one github issue. If you try googling colcon build release mode you'll see what I mean. Perhaps put this information on the ros2 tutorial page? https://docs.ros.org/en/humble/Tutorials.html

Originally posted by @Nate711 in #96 (comment)

When I'm installing python3-colcon-cmake package it changes CMake to 3.10.2. There are any opportunities to install it with CMake 3.13 or higher?

Hi,

I'd like to propose an extension to colcon-cmake build arguments to support selecting the build configuration for a multi-configuration-generator, such as Visual Studio. I have a relatively simple change proposed below, nI'm looking for comments on the wider implications of this change within the colcon ecosystem.

Currently, when colcon-cmake build.py invokes CMake, it appends --config CONFIG based on the current CMAKE_BUILD_TYPE, defaulting to "Release" if none is specified. The change I propose is to allow a command line selection of this value.

In CmakeBuildTask.add_arguments(), add a definition for a --config CONFIG argument:

        parser.add_argument(
            '--config',
            help='Select the CMake build configuration: [Debug, MinSizeRel, Release, RelWithDebInfo]. Only for multi configuration generators (e.g., Visual Studio)')

Modify CmakeBuildTask._get_configuration() as follows. Changes are highighted between # @@@ Change begin and # @@@ Change end comments.

    def _get_configuration(self, args):
        # check for CMake build type in the command line arguments
        arg_prefix = '-DCMAKE_BUILD_TYPE='
        build_type = None
        for cmake_arg in (args.cmake_args or []):
            if cmake_arg.startswith(arg_prefix):
                build_type = cmake_arg[len(arg_prefix):]
        if build_type is None:
            # get the CMake build type from the CMake cache
            build_type = get_variable_from_cmake_cache(
                args.build_base, 'CMAKE_BUILD_TYPE')
        if build_type in ('Debug', 'MinSizeRel', 'RelWithDebInfo'):
            return build_type

        # @@@ Change begin
        # Select build configuration for multi-configuration generators if the
        # --config option is presend on the command line
        multi_configuration_generator = is_multi_configuration_generator(
            args.build_base, args.cmake_args)
        if multi_configuration_generator and args.config:
            return args.config
        # @@@ Change end
        return 'Release'

This works in my isolated, CMake only build tree, but I suspect I'm missing something similar for running in test mode. Also, this may not play nice with other build tasks, such as ament.

Comments welcome.

Is there a reason why colcon will only accept project names that have alphanumerics, dashes and underscores?

https://github.com/colcon/colcon-cmake/blob/master/colcon_cmake/package_identification/cmake.py#L165

I agree that in an ideal world, that would be a nice naming convention, but in reality some projects do not conform to that because CMake doesn't make them. Examples:

• https://github.com/eclipse/paho.mqtt.c/blob/master/CMakeLists.txt#L21
• https://github.com/sewenew/redis-plus-plus/blob/master/CMakeLists.txt#L6

Can we make the regex more lax, without breaking other things in colcon? The fallback behavior to the directory name is good, but it shouldn't happen when there is a valid CMake project name specified.

When generating a minimal testing container, I noticed the debian package version of this module doesn't pull in cmake, perhaps it should?

Hi,

I'm getting the following deprecation warning from flake8 which is failing some PRs I've put up

DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
[snip]
FAILED test/test_flake8.py::test_flake8 - DeprecationWarning: SelectableGroup...

As far as I can tell, this is not to do with my changes. I'd infer that something has changed in flake8 and the flake8 validation script needs to be updated.

I've successfully run the test_flake8 script locally with Python 3.8 and flake8 3.9.2.

Please advise. Thanks.

Hello,
This is a proposal for supporting a --minify option (or similar) in colcon build.

Problem Statement

• Resource-constrained environments are limited in the amount of physical memory they can spare for running ROS applications. A significant portion of the memory consumption is due to the size of the binaries: the application being run, along with all of the dynamically linked libraries it depends on, which are being loaded into the RAM.
• While the binary installation of ROS2 for Tier1 Debian/Ubuntu platforms is already optimized for size through the use of debian packaging utilities (-Os compilation flag and dh-strip), developers in non-Tier1 architectures would have to build from source and figure out size optimizations for themselves.

Proposal

Running colcon build --minify would have the following effect:

1. Implies --cmake-args ' -DCMAKE_BUILD_TYPE=MinSizeRel' (and in turn, -Os)
2. On Linux/Unix-based systems, seamlessly run strip --strip-unneeded on all resulting binaries

Implementation

I've implemented a proof of concept in the following manner. If the general idea makes sense, I'll be happy to hear your thoughts about a better way of implementing this.

1. Export build targets from ament_cmake_core/ament_package.cmake with the following addition to the ament_package macro:

get_directory_property(BUILDSYSTEM_TARGETS_VAR BUILDSYSTEM_TARGETS)
set(EXPORTED_BUILD_TARGETS "${BUILDSYSTEM_TARGETS_VAR}" CACHE STRING)

2. colcon-cmake/build.py - scans the build directory for any of the names exported from ament and runs strip after build. Roughly:

# _reconfigure
if args.minify:
    cmake_args += ['-DCMAKE_BUILD_TYPE=MinSizeRel']

# _build
build_targets = get_variable_from_cmake_cache(args.build_base, 'EXPORTED_BUILD_TARGETS')
files_in_build_dir = glob.glob(os.path.join(args.build_base, '*'))
for each name in files_in_build_dir, if it matches a build target in build_targets, invoke strip

Deliberately Omitted

This proposal does not cover everything. Specifically, non-ament packages or the dependency on strip were not dealt with. If we have a green light about the general idea we'll iron out those issues too.

Example

• Default: colcon build --packages-select rclcpp
  Size of librclcpp.so: 7.6MB
  Size of the entire build directory ./build/rclcpp: 97MB
• Minified: colcon build --cmake-clean-cache --packages-select rclcpp --minify
  Size of librclcpp.so: 966K
  Size of the entire build directory ./build/rclcpp: 32M
  For reference, a MinSizeRel build without strip produces a 1.4MB librclcpp.so.

Colcon does not respect using --ctest-args --build-type <config> (where --build-type is equivalent to -C). Instead it reads the CMakeCache and extracts the last value for CMAKE_BUILD_TYPE. This does not work well with multi-configuration generators as you can only test the last build run.

Colcon's test functionality should prefer an explicityl --build-type specification on the command line over the current behaviour.

Hi there,

Is there a way to generate a compile_commands.json file for the ROS workspace that colcon is used for? I've tried passing --cmake-args=-DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON but that seems to only generate compile_commands for very few of the packages in my workspace as well as lots of warnings like the following:

Starting >>> gtest_vendor
--- stderr: gtest_vendor
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_COMPILE_COMMANDS


---

I've also tried generating the compile_commands.json using Bear but that resulted in an empty compile_commands file.

Any pointers?

When executing colcon build, the cmake configuration phase starts from launch to launch (the Visual Studio generator is used). This significantly increases the compilation time of individual projects under Windows.

Apparently, the problem lies in the following lines of the async def _reconfigure(self, args, env) function code:

 if not run_configure:
            buildfile = get_buildfile(cmake_cache)
            run_configure = not buildfile.exists()

In this case, the get_buildfile(qmake_cache) function returns a deliberately non-existent file for the Visual Studio generator (build.ninja or Makefile).

Hi,

I'm coming here from https://answers.ros.org/question/324217/ros2-for-win32-x86-how-to-make-colcon-call-cmake-without-win64/

I need to build ROS2 for Win32 and I am missing the option to select the cmake target platform "Visual Studio 15 2017". Is it possible to pass this as an argument? Or could I just remove the "Win64" here:

Thanks
marosso

Hello ROS Developers,

I'm encountering the same error while attempting to build the RSL package on Windows. I'm struggling to identify the solution to this problem. Currently, I'm using ROS Humble, Windows 10, and Visual Studio 16 2019.

Here is the command I'm using and the resulting error:

C:\ros2\humble>colcon build --packages-select rsl --merge-install
Starting >>> rsl
--- stderr: rsl
Traceback (most recent call last):
  File "c:\python38\lib\site-packages\colcon_core\executor\__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "c:\python38\lib\site-packages\colcon_core\task\__init__.py", line 93, in __call__
    return await task_method(*args, **kwargs)
  File "c:\python38\lib\site-packages\colcon_ros\task\catkin\build.py", line 74, in build
    rc = await extension.build(
  File "c:\python38\lib\site-packages\colcon_cmake\task\cmake\build.py", line 103, in build
    rc = await self._build(
  File "c:\python38\lib\site-packages\colcon_cmake\task\cmake\build.py", line 236, in _build
    if not await has_target(args.build_base, target):
  File "c:\python38\lib\site-packages\colcon_cmake\task\cmake\__init__.py", line 61, in has_target
    assert target == 'install'
AssertionError

What steps should I take to successfully build this package?

When using Ninja as CMake's build system generator, colcon does not show error information:

colcon build --cmake-args " -GNinja" --packages-select rmw_cyclonedds_cpp
Starting >>> rmw_cyclonedds_cpp
Failed   <<< rmw_cyclonedds_cpp	[ Exited with code 1 ] 

Summary: 0 packages finished [3.15s]
  1 package failed: rmw_cyclonedds_cpp

These errors do show up in build logs:

build_2020-03-13_15-32-21.zip

Hey there,
I was surprised to notice that in a plain colcon build --packages-select foo, it defaults to unix makefiles and doesn't set the build type. That is, in the build directory,

grep CMAKE_BUILD_TYPE CMakeCache.txt
CMAKE_BUILD_TYPE:STRING=

and the compiler invocation has no optimization flags in it; e.g.

/usr/bin/c++ -o foo.o -c foo.cpp

I'm aware that the build type is ignored when generating multiconfigurations for IDEs and the potential confusion about that; see https://stackoverflow.com/a/24470998/14409577

I also know how to opt in to build in release mode with

colcon build --packages-select foo --cmake-args -DCMAKE_BUILD_TYPE=Release

But I would expect that a C++ robotics package by default is built in release mode because performance matters. After all, why do we use C++?

Was this is discussed before and there are arguments that I'm missing?

fyi @nnmm

Now colcon build compilation should be used
cmake --build . --target install

I want to implement cmake --build . --target package
Generate the content I need to install as deb

All install content is generated in deb

How can I modify core to do this

Requirement: Multiple function packages in a workspace are compiled to generate a deb file

The below describes a reduction exhibiting the symptom:

• ROS2-Humble-Desktop docker image from OSRF (osrf/ros:humble-desktop):

$ printenv | grep ROS
ROS_VERSION=2
ROS_PYTHON_VERSION=3
ROS_LOCALHOST_ONLY=0
ROS_DISTRO=humble

• created a ROS2 package via ros2 pkg create --build-type ament_cmake <package_name>
• Export, for example, these variables within the environment. Crucially in my case, CMAKE_C_FLAGS and CMAKE_CXX_FLAGS are exported by a pre-colcon-build step involving conan.

export CMAKE_C_FLAGS="foo"
export CMAKE_CXX_FLAGS="bar; baz"

• In the CMakeLists.txt, print out CMAKE_C_FLAGS and CMAKE_CXX_FLAGS:

message("CMAKE_C_FLAGS are:   ${CMAKE_C_FLAGS}")
message("CMAKE_CXX_FLAGS are: ${CMAKE_CXX_FLAGS}")

• During a clean colcon build, those variables are empty:

$ colcon build
...
CMAKE_C_FLAGS are:    
CMAKE_CXX_FLAGS are: 

The solution is to force the set in CMakeLists.txt. But why?

set(CMAKE_C_FLAGS "$ENV{CMAKE_C_FLAGS} $ENV{CFLAGS}" CACHE STRING "" FORCE)
set(CMAKE_CXX_FLAGS "$ENV{CMAKE_CXX_FLAGS} $ENV{CXXFLAGS}" CACHE STRING "" FORCE)

$ colcon build
...
CMAKE_C_FLAGS are:   foo 
CMAKE_CXX_FLAGS are: foo; bar; baz

In build.py, _install(), the command invoked to run installation uses either cmake --build <dir> --target install or invoked MSBUILD to achieve a similar result. This works out OK for make platforms, but inccurs unnecessary overheads with Visual Studio as Visual Studio ensures all the dependencies of the INSTALL target are up to date.

Installation is normally invoked immediately after a build, so we essentially build twice. Windows slower file IO and the additional work Visual Studio does here makes for a significantly slower build process than with Linux/make.

I suggest using cmake --install <dir> or cmake --install <dir> --config <config> instead. This is significantly faster than the MSBUILD approach. I have one simple project where this is ~75 ms with cmake install vs. 1.6s with MSBUILD. The MSBUILD overhead scales for larger projects.

This would also unify the code executed by _install() no longer needing to do the generator check, just check where it needs the --config argument.

Note: this suggestion is also immune to bug #54

I'm attempting to generate a compile_commands.json file for use with clangd, however the file is not generated (neither on a package level nor the workspace level). Attempting to build the turtlesim package from ros_tutorials with

PS D:\Documents\dev_ws> colcon build --merge-install --event-handlers console_cohesion+ --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON

gives the following warning:

CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_COMPILE_COMMANDS

My platform is Windows 10 with Ros2 Foxy installed from the aka.ms/ros Chocolatey package.

I build colcon from source following the instructions in the docs, and colcon version-check gives:

colcon-cd 0.1.1: up-to-date
colcon-cmake 0.2.26: up-to-date
colcon-core 0.7.1: up-to-date
colcon-defaults 0.2.6: up-to-date
colcon-devtools 0.2.3: up-to-date
colcon-library-path 0.2.1: up-to-date
colcon-metadata 0.2.5: up-to-date
colcon-mixin 0.2.1: up-to-date
colcon-notification 0.2.13: up-to-date
colcon-output 0.2.12: up-to-date
colcon-package-information 0.3.3: up-to-date
colcon-package-selection 0.2.10: up-to-date
colcon-parallel-executor 0.2.4: up-to-date
colcon-pkg-config 0.1.0: up-to-date
colcon-powershell 0.3.7: up-to-date
colcon-python-setup-py 0.2.7: up-to-date
colcon-recursive-crawl 0.2.1: up-to-date
colcon-ros 0.3.22: up-to-date
colcon-test-result 0.3.8: up-to-date
colcon-zsh 0.4.0: up-to-date

Attempting the same in an Ubuntu 20.04 with identical colcon versions correctly generates compile_commands.json on a package and workspace level.

I also tested on windows with the change proposed in #114, however this did not result in any changed behavior. Additionally, I tested ommitting the --merge-install option, which produced no change for either platform.

Hi,

ISSUE: Neither colcon build --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -G Ninja nor colcon build --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON produce any ros2_ws/build/compile_commands.json.

on Ubuntu 20.04 LTS using ROS2 Foxy, colcon version-check:

colcon-argcomplete 0.3.3: up-to-date
colcon-bash 0.4.2: up-to-date
colcon-cd 0.1.1: up-to-date
colcon-cmake 0.2.27: up-to-date
colcon-core 0.12.1: up-to-date
colcon-defaults 0.2.8: up-to-date
colcon-devtools 0.2.3: up-to-date
colcon-installed-package-information 0.1.0: up-to-date
colcon-library-path 0.2.1: up-to-date
colcon-metadata 0.2.5: up-to-date
colcon-notification 0.2.15: up-to-date
colcon-output 0.2.13: up-to-date
colcon-override-check 0.0.1: up-to-date
colcon-package-information 0.3.3: up-to-date
colcon-package-selection 0.2.10: up-to-date
colcon-parallel-executor 0.2.4: up-to-date
colcon-pkg-config 0.1.0: up-to-date
colcon-powershell 0.3.7: up-to-date
colcon-python-setup-py 0.2.8: up-to-date
colcon-recursive-crawl 0.2.1: up-to-date
colcon-ros 0.3.23: up-to-date
colcon-test-result 0.3.8: up-to-date
colcon-zsh 0.4.0: up-to-date

especially colcon-cmake 0.2.27: up-to-date with

ros2_ws
  └── src
      └── cpp_pubsub
          ├── CMakeLists.txt
          ├── include
          │   └── cpp_pubsub
          ├── package.xml
          └── src

Neither colcon build --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -G Ninja nor colcon build --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON produce any ros2_ws/build/compile_commands.json.

Output of colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -G Ninja:

Starting >>> cpp_pubsub
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done      
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ament_cmake: 0.9.11 (/opt/ros/foxy/share/ament_cmake/cmake)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.10", minimum required is "3") 
-- Using PYTHON_EXECUTABLE: /usr/bin/python3
-- Found ament_lint_auto: 0.9.8 (/opt/ros/foxy/share/ament_lint_auto/cmake)
-- Added test 'copyright' to check source files copyright and LICENSE
-- Added test 'lint_cmake' to check CMake code style
-- Added test 'xmllint' to check XML markup files
-- Configuring done
-- Generating done
-- Build files have been written to: $mypath/ros_ws/build/cpp_pubsub
ninja: no work to do.
-- Install configuration: ""
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/ament_index/resource_index/package_run_dependencies/cpp_pubsub
-- Installing: /home/eric/Software/ros_ws/install/cpp_pubsub/share/ament_index/resource_index/parent_prefix_path/cpp_pubsub
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/environment/ament_prefix_path.sh
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/environment/ament_prefix_path.dsv
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/environment/path.sh
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/environment/path.dsv
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/local_setup.bash
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/local_setup.sh
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/local_setup.zsh
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/local_setup.dsv
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/package.dsv
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/ament_index/resource_index/packages/cpp_pubsub
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/cmake/cpp_pubsubConfig.cmake
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/cmake/cpp_pubsubConfig-version.cmake
-- Installing: $mypath/ros_ws/install/cpp_pubsub/share/cpp_pubsub/package.xml
Finished <<< cpp_pubsub [0.50s]                  

Summary: 1 package finished [0.60s]

Edit: #76 (comment) i.e. colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -G Ninja --no-warn-unused-cli produces the same result.

Any suggestions?

Originally posted by @Scoeerg in #61 (comment)

I recently started needing to repeatedly run a specific test in a gtest framework for the ros2_control/ros2_controllers repository so I implemented gtest_discover_tests in the CMakeLists.txt file so that I could properly pass ctest-args to colcon test and filter according to that test; however, when I did so, I noticed that tests started taking a lot longer to run. For example:

• on ctest branch (with gtest_discover_tests):
  • colcon test --packages-select force_toque_sensor_broadcaster
    • takes ~3.5 seconds
• on master (no gtest discovery):
  • colcon test --packages-select force_torque_sensor_broadcaster
    • takes ~0.8 seconds

When digging into the logs of the test's stdout_stderror.log, it looks like the speed difference is coming from colcon executing each test as a separate executing run when gtest_discover_tests is enabled versus running the whole executable once when discovery is not enabled. For example:

When I run the test executable directly, I get the following output:

Running main() from gmock_main.cc
[==========] Running 14 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 14 tests from ForceTorqueSensorBroadcasterTest
[ RUN      ] ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet
[       OK ] ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet (15 ms)
[ RUN      ] ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet
[       OK ] ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet (12 ms)
[ RUN      ] ForceTorqueSensorBroadcasterTest.InterfaceNames_FrameId_NotSet
[       OK ] ForceTorqueSensorBroadcasterTest.InterfaceNames_FrameId_NotSet (10 ms)
[ RUN      ] ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_Set
[       OK ] ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_Set (8 ms)

... etc for brevity...

[----------] 14 tests from ForceTorqueSensorBroadcasterTest (162 ms total)

[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (172 ms total)
[  PASSED  ] 14 tests.

However when running from colcon test, I get the following output:

UpdateCTestConfiguration  from :/mnt/lund/Code/ros2_control_ws/build/force_torque_sensor_broadcaster/CTestConfiguration.ini
Parse Config file:/mnt/lund/Code/ros2_control_ws/build/force_torque_sensor_broadcaster/CTestConfiguration.ini
   Site: 1f10b170ae5d
   Build name: (empty)
 Add coverage exclude regular expressions.
Create new tag: 20220419-1251 - Experimental
UpdateCTestConfiguration  from :/mnt/lund/Code/ros2_control_ws/build/force_torque_sensor_broadcaster/CTestConfiguration.ini
Parse Config file:/mnt/lund/Code/ros2_control_ws/build/force_torque_sensor_broadcaster/CTestConfiguration.ini
Test project /mnt/lund/Code/ros2_control_ws/build/force_torque_sensor_broadcaster
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
      Start  1: ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet

1: Test command: /mnt/lund/Code/ros2_control_ws/build/force_torque_sensor_broadcaster/test_force_torque_sensor_broadcaster "--gtest_filter=ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet" "--gtest_also_run_disabled_tests"
1: Test timeout computed to be: 10000000
1: Running main() from gmock_main.cc
1: Note: Google Test filter = ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet
1: [==========] Running 1 test from 1 test suite.
1: [----------] Global test environment set-up.
1: [----------] 1 test from ForceTorqueSensorBroadcasterTest
1: [ RUN      ] ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet
1: [ERROR] [1650372703.599663675] [test_force_torque_sensor_broadcaster]: 'sensor_name' or at least one 'interface_names.[force|torque].[x|y|z]' parameter has to be specified.
1: [       OK ] ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet (18 ms)
1: [----------] 1 test from ForceTorqueSensorBroadcasterTest (18 ms total)
1: 
1: [----------] Global test environment tear-down
1: [==========] 1 test from 1 test suite ran. (35 ms total)
1: [  PASSED  ] 1 test.
 1/16 Test  #1: ForceTorqueSensorBroadcasterTest.SensorName_InterfaceNames_NotSet ...........   Passed    0.15 sec
test 2
      Start  2: ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet

2: Test command: /mnt/lund/Code/ros2_control_ws/build/force_torque_sensor_broadcaster/test_force_torque_sensor_broadcaster "--gtest_filter=ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet" "--gtest_also_run_disabled_tests"
2: Test timeout computed to be: 10000000
2: Running main() from gmock_main.cc
2: Note: Google Test filter = ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet
2: [==========] Running 1 test from 1 test suite.
2: [----------] Global test environment set-up.
2: [----------] 1 test from ForceTorqueSensorBroadcasterTest
2: [ RUN      ] ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet
2: [ERROR] [1650372703.744065436] [test_force_torque_sensor_broadcaster]: 'frame_id' parameter has to be provided.
2: [       OK ] ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet (14 ms)
2: [----------] 1 test from ForceTorqueSensorBroadcasterTest (14 ms total)
2: 
2: [----------] Global test environment tear-down
2: [==========] 1 test from 1 test suite ran. (25 ms total)
2: [  PASSED  ] 1 test.
 2/16 Test  #2: ForceTorqueSensorBroadcasterTest.SensorName_FrameId_NotSet ..................   Passed    0.14 sec
test 3
      Start  3: ForceTorqueSensorBroadcasterTest.InterfaceNames_FrameId_NotSet

etc for brevity...

which I believe from the lines reading Note: Google Test filter = Force.... on every single test that it's running each test against the executable with the given filter individually when it could run the executable once to run them all.

Does colcon-cmake support a way to batch test runs together when discovery shows that there are tests part of the same executable?

Hi, does anyone know how can I use the compile_commands.json in CLion when using ROS2?

It looks like the new CompileCommandsEventHandler (introduced by #69) is being invoked by colcon test.

For example, if you do

colcon build --event-handlers compile_commands- --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
colcon test  # don't specify any --event-handlers options

there will still end up being a workspace-level compile_commands.json under build/.

Is this intended behavior? I think it's more intuitive if the workspace-levelcompile_commands.json is only generated during colcon build.

The "help" target is used to check if a particular target exists:

But when using the Ninja generator, the "tests" target does not seem to appear in the output of that command, and therefore is not built by a colcon build on a ROS package containing cmake such as:

if(CATKIN_ENABLE_TESTING)
  find_package(rostest REQUIRED)
  add_rostest_gtest(dummy_node_test test/dummy_node.test test/DummyNodeTest.cpp)
  target_include_directories(dummy_node_test SYSTEM PRIVATE ${catkin_INCLUDE_DIRS})
  target_link_libraries(dummy_node_test ${catkin_LIBRARIES})
endif()

This causes colcon test failures if rostest tries to launch the unbuilt test binary.

I saw there's a --cmake-force-configure option, but I didn't see an option that would allow me to run just the configure step for a package, and skip building. This would be helpful while iterating on the configuration.

My use case is for building a single package, and I imagine this option may be problematic when building multiple packages.

When building an ament_cmake package ld_library_path_lib.xx files are generated in the install tree setting up LD_LIBRARY_PATH (on Linux). This is not the case for build_type 'cmake'. Executables depending on these libraries will fail with xxx.so not found.

Note: Running with --symlink-install this apparently works, as the dependent executables will find the libraries through the runpath variable in the build executable.

Support reading build/<pkgname>/Testing/<timestamp>/Test.xml files and showing them in the test-result verb. This will require the test-result verb to provide an extension point for external result parsers.

I have a default.yaml where I specify some default cmake options to be passed to my cmake projects.

{
    "build":
    {
        "cmake-args": ["-G", "Ninja", "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON", "-DCMAKE_CXX_COMPILER_LAUNCHER=ccache", "-DCMAKE_CXX_FLAGS=\"-fdebug-prefix-map=${CMAKE_BINARY_DIR}=\""]
    }
}

Also I add some default cmake options in some project using colcon.meta.

{
    "names":
    {
        "fastrtps" :
        {
            "cmake-args": ["-DEPROSIMA_BUILD=ON"]
        }
    }
}

If I use colcon to build using a command line like colcon build --paths ../.. --metas ../.., the cmake projects are configured with the default cmake options. But if I add a cmake option using --cmake-args like colcon build --paths ../.. --metas ../.. --cmake-args -DCMAKE_BUILD_TYPE=Debug, the CMake projects are configured only with the cmake option specified with --cmake-args. colcon is not using default values.

Platform: windows 10
Shell: cmd
Installed via pip (freeze below)

The documentation for --cmake-args says arugments beginning with - must have an escaped space in front. It looks like escaping a space does not stop colcon's argument parser from trying to parse the cmake argument using the cmd shell on windows.

D:\overlay-ros2>colcon build --cmake-args \ -DCMAKE_BUILD_TYPE=Debug
usage: colcon [-h] [--log-level LOG_LEVEL]
              {build,extension-points,extensions,info,list,metadata,test,test-result}
              ...
colcon: error: unrecognized arguments: -DCMAKE_BUILD_TYPE=Debug

Quoting a space in front of the cmake argument works fine.

D:\overlay-ros2>colcon build --cmake-args " -DCMAKE_BUILD_TYPE=Debug"

Installed versions

C:\Users\osrf>pip freeze
attrs==17.4.0
colcon-cmake==0.1.1
colcon-common-extensions==0.1.1
colcon-core==0.1.0
colcon-defaults==0.1.0
colcon-devtools==0.1.0
colcon-library-path==0.1.0
colcon-metadata==0.1.0
colcon-notification==0.1.1
colcon-output==0.1.1
colcon-package-information==0.1.0
colcon-package-selection==0.1.0
colcon-parallel-executor==0.1.0
colcon-powershell==0.1.0
colcon-python-setup-py==0.1.0
colcon-recursive-crawl==0.1.0
colcon-ros==0.1.0
colcon-test-result==0.1.0
colorama==0.3.9
coloredlogs==9.0
coverage==4.5.1
cycler==0.10.0
empy==3.3.2
enum34==1.1.6
flake8==3.5.0
flake8-blind-except==0.1.1
flake8-builtins==1.0.post0
flake8-class-newline==1.6.0
flake8-comprehensions==1.4.1
flake8-deprecated==1.3
flake8-docstrings==1.3.0
flake8-import-order==0.17
flake8-polyfill==1.0.2
flake8-quotes==0.14.0
humanfriendly==4.10
matplotlib==2.0.2
mccabe==0.6.1
mock==2.0.0
nose==1.3.7
numpy==1.13.1
pbr==3.1.1
pep8==1.7.1
pluggy==0.6.0
py==1.5.2
pycodestyle==2.3.1
pydocstyle==2.1.1
pyflakes==1.6.0
pyparsing==2.2.0
pypiwin32==223
pyreadline==2.1
pytest==3.4.1
pytest-cov==2.5.1
pytest-runner==4.0
python-dateutil==2.6.1
pytz==2017.2
pywin32==223
PyYAML==3.12
six==1.10.0
snowballstemmer==1.2.1
trollius==2.1
vcstool==0.1.32

By testing the latest setuptools version on Fedora (67.5.1) colcon-cmake fails to build due to the pkg_resources deprecation.

==================================== ERRORS ====================================
______________ ERROR collecting test/test_parse_cmake_version.py _______________
test/test_parse_cmake_version.py:4: in <module>
    from colcon_cmake.task.cmake import _parse_cmake_version_string
colcon_cmake/task/cmake/__init__.py:12: in <module>
    from pkg_resources import parse_version
/usr/lib/python3.11/site-packages/pkg_resources/__init__.py:121: in <module>
    warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
E   DeprecationWarning: pkg_resources is deprecated as an API
=========================== short test summary info ============================
ERROR test/test_parse_cmake_version.py - DeprecationWarning: pkg_resources is...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.13s ===============================

While ament_uninstall is supposedly provided for uninstalling packages, often doing so results in a crash as colcon processes the packages in topological dependency order, removing build-time dependencies that later packages need:

➜ colcon build --packages-select-regex ".*yaml.*" --cmake-target uninstall
Starting >>> libyaml_vendor
Starting >>> launch_yaml                      
Starting >>> yaml_cpp_vendor                                                   
Finished <<< yaml_cpp_vendor [7.19s]                                                                                                                      
Finished <<< libyaml_vendor [7.91s]                                                                     
Starting >>> rcl_yaml_param_parser
Finished <<< launch_yaml [8.78s]                                                      
--- stderr: rcl_yaml_param_parser                                             
CMake Error at CMakeLists.txt:7 (find_package):
  By not providing "Findlibyaml_vendor.cmake" in CMAKE_MODULE_PATH this
  project has asked CMake to find a package configuration file provided by
  "libyaml_vendor", but CMake did not find one.

  Could not find a package configuration file provided by "libyaml_vendor"
  with any of the following names:

    libyaml_vendorConfig.cmake
    libyaml_vendor-config.cmake

  Add the installation prefix of "libyaml_vendor" to CMAKE_PREFIX_PATH or set
  "libyaml_vendor_DIR" to a directory containing one of the above files.  If
  "libyaml_vendor" provides a separate development package or SDK, be sure it
  has been installed.


make: *** [cmake_check_build_system] Error 1
---
Failed   <<< rcl_yaml_param_parser	[ Exited with code 2 ]

Summary: 3 packages finished [22.6s]
  1 package failed: rcl_yaml_param_parser
  1 package had stderr output: rcl_yaml_param_parser

The patch for this has already slipped in as part of this PR: https://github.com/colcon/colcon-cmake/pull/5/files#diff-180360612c6b8c4ed830919bbb4dd459R4

CMake supports specifying the generator as either one argument or two arguments; '-G' or '-G ' repectively. Colcon does not recognise the former, one argument syntax.

PR to fix this is here: #85

colcon test --packages-select rosbag2_transport --ctest-args " --help" is expected to show help for ctest arguments but does not. Output is similar to if tests were run and succeeded.

colcon test --packages-select rosbag2_transport --ctest-args " --help"
Starting >>> rosbag2_transport
Finished <<< rosbag2_transport [0.19s]          

Summary: 1 package finished [1.06s]

The output from ctest --help can be found in log/latest_test/rosbag2_transport/stdout.log.

Perhaps the help text should come out in stdout or the help for the --ctest-args option should use an argument other than " --help" as in:

$ colcon test --help
usage: colcon test [-h] [--build-base BUILD_BASE] [--install-base INSTALL_BASE] [--merge-install]
...
Arguments for 'cmake' packages:
  --ctest-args [* [* ...]]
                        Pass arguments to CTest projects. Arguments matching other options must be prefixed by a space,
                        e.g. --ctest-args " --help"

Example:

colcon build --test-result-base test_results
colcon test --test-result-base test_results
colcon test-result --test-result-base test_results --all --result-files-only

Expected behavior would be that the CTest Test.xml results from any ctest invocations would be printed. However, those files ended up in the --build-base. This seems to contradict the explicit purpose of --test-result-base, "the base path for all test results".

From the example above, if you run colcon test-result --all --result-files-only so that --test-result-base defaults to be the same as --build-base, the paths to the CTest Test.xml results are printed as expected.

Request

This is a feature request to add an option to append cmake arguments to the cmake-args defined in the defaults.yaml: e.g.

$ colcon build --cmake-add-args -DCMAKE_EXPORT_COMPILE_COMMANDS

• Naturally, if no cmake-args are set in the defaults, cmake-add-args will behave exactly like cmake-args, so there may be a better way to address the use-case

Use-case details

• We use a colcon defaults.yml file to standardize the colcon build options for developers; for the scope of this ticket, we are setting a few CMake variables with cmake-args
• Occasionally a developer will need to adjust the cmake configuration from the command line: e.g.
  • Build type
  • CMake verbosity
  • CMAKE_EXPORT_COMPILE_COMMANDS
• The obvious way to do this is by using --cmake-args; however, this overwrites the defaults.yaml (which is intended behavior #29); unless the developer copies all the options out of the defaults.yaml
  • So a command like colcon build --cmake-args -DCMAKE_BUILD_TYPE=Releases becomes colcon build --cmake-args -DCMAKE_FLAG_1... -DCMAKE_FLAG2... -DCMAKE_FLAG3... -DCMAKE_BUILD_TYPE=Releases
• This feature would allow developers to use the shorter command

Known alternatives

1. Add the options in the defaults.yaml
   • The main disadvantage is that it requires editing a file to toggle between two options, so, e.g., to toggle between building verbose and not verbose, one has to open, edit, save, build, open, edit, save, build
2. Use a mixin
   • This is already possible for build types, and we could create one for verbosity, etc.; however, does it really make sense to create a mixin for every CMake option that one may want to toggle?

When we run colcon build with the following cmake-args option, we can got compile_commands.json for each package.

colcon build --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=ON

It's very useful to use auto complete of programming ROS / ROS 2 by the language server protocol of clangd.
It'll be even more useful to develop multiple packages at a time if the multiple compile_commands.json to unify one compile_commands.json.

I'm thinking about adding a --cmake-concat-compile-commands option. How do you think about it? I try to implement it soon.

Thanks to the colcon extensibility, my development experience is better than before!

$ colcon  build --symlink-install --event-handlers console_direct+ --cmake-args -C /mnt/mervin/workspaces/devel/camera/initial_cache.cmake
loading initial cache file /mnt/mervin/workspaces/devel/camera/initial_cache.cmake
Re-run cmake no build system arguments
CMake Error: The source directory "/mnt/mervin/workspaces/devel/camera/initial_cache.cmake" is a file, not a directory.

So, loads fine, but fails because it confuses the initial cache path with the source directory. Digging into an example of the actual command that colcon uses for cmake (use --log-level=debug with colcon):

[0.214s] DEBUG:colcon.colcon_core.event_handler.log_command:Invoked command in '/mnt/mervin/workspaces/devel/camera/build/ueye_cam' returned '1':  CMAKE_PREFIX_PATH=/opt/ros/dashing /usr/bin/cmake /mnt/mervin/workspaces/devel/camera/src/ueye_cam -C /mnt/mervin/workspaces/devel/camera/initial_cache.cmake -DAMENT_CMAKE_SYMLINK_INSTALL=1 -DCMAKE_INSTALL_PREFIX=/mnt/mervin/workspaces/devel/camera/install/ueye_cam

Here the source directory is specified prior to the -C argument (as well as the other arguments). While this works for other cmake args, it does not work for the -C argument. In addition, this would appear not to be a cmake problem - cmake does in fact suggest options should come first:

cmake --help
Usage

  cmake [options] <path-to-source>
  cmake [options] <path-to-existing-build>

Q: Are there any ramifications to moving the specified source directory to the end of this list of arguments?

colcon-cmake globally expects a CMakeLists.txt. This prevents building with other build systems, e.g. meson.

To reproduce, install colcon-meson and create a workspace with a meson package:

pip install colcon-common-extensions
pip install colcon-meson
mkdir -p ~/cam_ws/src
cd ~/cam_ws/src
git clone https://git.libcamera.org/libcamera/libcamera.git
cd ~/cam_ws
colcon build

This will fail with:

CMake Error: The source directory "[workspace]/src/libcamera" does not appear to contain CMakeLists.txt.

For now, Colcon automatically pass -j{os.cpu_count()} -l{os.cpu_count} to the cmake --build command, unless the MAKEFLAGS environment variable contains them. There is no other way to control what arguments the builder receives. This works fine with make, but is useless with Ninja, which doesn't use an environment variable.

As a workaround, I use taskset to change the CPU affinity and limit the number of parallel build, but it only limit the number of parallelism and thus doesn't work with tools such as distcc which allow to build with more parallelism.

It would be great to at least be able to control the parallelism via a colcon flag, or perhaps better, be able to directly pass arguments to the cmake --build invocation.

We recently ran into issues with packages that leverage the CMake package registry feature. This feature breaks workspace isolation, should colcon automatically be passing the CMake disable flags?

  -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON
  -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON

I'm testing on a clean Windows 10 install of ROS2 with Visual Studio 2017 15.9.7 and python 3.7.2, and building the demo_nodes_cpp fails with the following error:

Traceback (most recent call last):
  File "c:\python37\lib\site-packages\colcon_core\executor\__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "c:\python37\lib\site-packages\colcon_core\task\__init__.py", line 92, in __call__
    return await task_method(*args, **kwargs)
  File "c:\python37\lib\site-packages\colcon_ros\task\ament_cmake\build.py", line 72, in build
    additional_hooks=additional_hooks)
  File "c:\python37\lib\site-packages\colcon_cmake\task\cmake\build.py", line 103, in build
    if rc and rc.returncode:
AttributeError: 'int' object has no attribute 'returncode'

Actually the _build function is a little strange for me, since it may have an unclear return value (the only return is inside an if, inside a loop), and the receiving function (build) makes no type check.

Do you have any idea why build is receiving an int instead of a subprocess.CompletedProcess?

I'm trying to run colcon build with MSBuild command line tools (free) rather than Visual Studio (licenced). I'm trying to compile ROS2/Humble.

I could force this compiler to be used by adding --cmake-args "-GNMake Makefiles" -DCMAKE_BUILD_TYPE=Release, however, I encounter issues mainly because colcon code checks if 'Visual Studio' in generator: and if false code considers we are using MAkefile generator, while here it's NMake, which should be treated as 'Visual Studio'.

Is it intended that MSBuild is not supported? Why do we need the whole Visual Studio IDE to compile a project?

This is a downstream bug caused by colcon/colcon-core#248

Building a package using Visual Studio generator can fail during installation due to the afforementioned bug. The failure message looks something like what follows:

Build started 18/10/2019 2:21:14 PM.
Project "<project-path>\INSTALL.vcxproj" on node 1 (default targets).
                                                          
C:\Program Files (x86)\Windows Kits\10\DesignTime\CommonConfiguration\Neutral\UAP\10.0.18362.0\UAP.props(51,18): error MSB4086: A numeric comparison was attempted on "$(VisualStudioVersion)" that evaluates to "16 [<project-path>\INSTALL.vcxproj]
C:\Program Files (x86)\Windows Kits\10\DesignTime\CommonConfiguration\Neutral\UAP\10.0.18362.0\UAP.props(51,18): error MSB4086: ProgramFiles(x86)=C:\Program Files (x86)" instead of a number, in condition "'$(VisualStudioVersion)' != '' and '$(VisualStudioVersion)' <= '14.0'". [<project-path>\INSTALL.vcxproj]
Done Building Project "<project-path>\INSTALL.vcxproj" (default targets) -- FAILED.

Build FAILED.

"<project-path>\INSTALL.vcxproj" (default target) (1) ->
  C:\Program Files (x86)\Windows Kits\10\DesignTime\CommonConfiguration\Neutral\UAP\10.0.18362.0\UAP.props(51,18): error MSB4086: A numeric comparison was attempted on "$(VisualStudioVersion)" that evaluates to "16 [<project-path>\INSTALL.vcxproj]
C:\Program Files (x86)\Windows Kits\10\DesignTime\CommonConfiguration\Neutral\UAP\10.0.18362.0\UAP.props(51,18): error MSB4086: ProgramFiles(x86)=C:\Program Files (x86)" instead of a number, in condition "'$(VisualStudioVersion)' != '' and '$(VisualStudioVersion)' <= '14.0'". [<project-path>\INSTALL.vcxproj]

    0 Warning(s)
    1 Error(s)

Note the message "$(VisualStudioVersion)" that evaluates to.... The value should be simply 16 (for Visual Studio 2019). The message about is a little difficult to decode, because VisualStudioVersion contains a newline character, but the value is actually 16\nProgramFiles(x86)=C:\Program Files (x86). This has come about as described in colcon/colcon-core#248 when decoding the environment and VisualStudioVersion is followed by ProgramFiles(x86).

It's highy dependent on the order environment variables, and can present itself somewhat sporadically by restarting the environment from which colcon is run.