rwth-hpc / cmake-codecov Goto Github PK

View Code? Open in Web Editor NEW

89.0 89.0 35.0 142 KB

CMake module for code coverage

License: BSD 3-Clause "New" or "Revised" License

CMake 90.44% C 5.35% Shell 3.20% Fortran 1.00%

cmake code-coverage codecov gcov lcov

cmake-codecov's People

Contributors

Stargazers

Watchers

cmake-codecov's Issues

Test if compiler is compatible to gcov

Clang and gcov may support different interfaces. This can cause warnings and even crashes. FindGcov.cmake should implement a check to check if all compilers are compatible to gcov.

Another way could be to check if the compiler has an own gcov implementation, e.g. clang ships a binary named llvm-gcov which may be used as alternative to gcov.

header-only libraries

I couldn't find a way to use the macros for a header-only library - I wanted to have a few executables exercising the coverage only in the header - I tried this:

add_library(dummy STATIC foo2.h)
set_target_properties(dummy PROPERTIES LINKER_LANGUAGE CXX)
add_coverage(dummy)

and had a few tests added with add_test but their targets had not add_coverage() called for them (intentionally)

gcov generation fails with Ninja generator

Steps to reproduce:

CMakeLists.txt:

cmake_minimum_required(VERSION 3.4)
project(test CXX)
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake/codecov/cmake" ${CMAKE_MODULE_PATH})
find_package(codecov)
add_executable(foo test.cc)
add_coverage(foo)

test.cc:

#include <iostream>
using namespace std;
int main(int argc, char **argv)
{
    cout << "Hello World!\n";
    return 0;
}

Setup:

$ git submodule add https://github.com/RWTH-ELP/CMake-codecov.git cmake/codecov
$ mkdir build
$ cd build
$ CXX=clang++ CC=clang cmake -G Ninja .. -DENABLE_COVERAGE=On
$ ninja-build -v foo
$ ./foo
$ ninja-build -v foo-gcov

Actual result:

$ ninja-build -v foo-gcov
[1/1] cd /home/juser/cmake-codecov-test/build/CMakeFiles/foo.dir && LLVM_COV_BIN=/usr/bin/llvm-cov /home/juser/cmake-codecov-test/cmake/codecov/cmake/llvm-cov-wrapper /home/juser/cmake-codecov-test/build/CMakeFiles/foo.dir/test.cc.gcno > /dev/null
../test.cc: No such file or directory
$ ninja-build -v foo-gcov
ninja: no work to do.

Expected result:

A .gcov file under in the build directory, i.e. CMakeFiles/foo.dir/test.cc.gcov.

I tested it on Fedora 23.

When compiling with the CMake makefile generator the gcov report is generated as expected.

Travis won't build without errors

Hi, i just forked this project in order to add some support for windows platforms. When i added the project into travis i saw that the build fails for the "clang-builds" (on branch master).

Would you please take a look into this and maybe rerun your builds as well?

https://travis-ci.org/farrrb/CMake-codecov/builds/335606460

Thank you,
Fabian

GCov not detected on OS X travis image

I am trying to use CMake-codecov to capture code coverage for Catch and after some bumbling around I got it to work properly under Linux for both GCC and Clang.

However, I ran into trouble when I was trying to integrate it with OS X builds. The scripts properly detect that AppleClang supports coverage, finds lcov, but then is unable to find gcov, with this output

CMake Warning at CMake/FindGcov.cmake:129 (message):
  No coverage evaluation binary found for AppleClang.

Call Stack (most recent call first):
  CMake/Findcodecov.cmake:248 (add_gcov_target)
  CMake/Findcodecov.cmake:38 (add_coverage_target)
  CMakeLists.txt:308 (add_coverage)


CMake Warning at CMake/FindLcov.cmake:154 (message):
  No coverage evaluation binary found for AppleClang.

Call Stack (most recent call first):
  CMake/FindLcov.cmake:39 (lcov_capture_initial_tgt)
  CMake/Findcodecov.cmake:249 (add_lcov_target)
  CMake/Findcodecov.cmake:38 (add_coverage_target)
  CMakeLists.txt:308 (add_coverage)


CMake Warning at CMake/FindLcov.cmake:245 (message):
  No coverage evaluation binary found for AppleClang.

Call Stack (most recent call first):
  CMake/FindLcov.cmake:42 (lcov_capture_tgt)
  CMake/Findcodecov.cmake:249 (add_lcov_target)
  CMake/Findcodecov.cmake:38 (add_coverage_target)
  CMakeLists.txt:308 (add_coverage)

The full travis output can be found here.

Btw, thanks for the script. After I stopped being dumb about using it on Linux, it worked nicely.

Automatic tests and zerocounter

I just try to use CMake-codecov (I'm new to lcov, just learning, so sorry if my question is out of scope)

I want to automatically call check target before calling lcov.
after calling coverage_evaluate() at the end of main CMakeLists.txt, I added add_dependencies(lcov check)
But I have one issue, I want to "zerocounter" lcov data before tests, to have fresh results in case of rebuild.
I have done

add_custom_command(TARGET lcov-zerocounter COMMAND ${LCOV_BIN} --quiet --zerocounter --directory ${CMAKE_BINARY_DIR})
add_dependencies(check lcov-zerocounter)

But I'm asking myself should this be present directly in CMake-codecov ? or is there a better way to handle lcov data cleanup before capture.
Thanks.

Lcov aborts on files with no functions

Gcov prints a warning, if a gcno file contains no functions. Lcov aborts in this situation.

A possible solution to solve this issue may be to delete the gcno file after compilation, if it contains no functions and lcov will ignore this (non existing) file.

ENABLE_COVERAGE_ALL

HI,
Where is the ENABLE_COVERAGE_ALL option applied?
I can't see a reference to it in the source.

Check the required minimum CMake version.

This module should be checked for compatibility with older CMake versions and a badge for the compatibility added.

How to enable branch coverage?

Are there variables exposed to generate branch coverage as it appears to be disabled by default?

lcov targets fail with Ninja

The same example provided in issue #6 shows that also the support for lcov is failing when using the Ninja generator.

$ cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DENABLE_COVERAGE=ON ..
$ ninja -v foo
[1/2] /usr/bin/c++    -g    -O0 -g -fprofile-arcs -ftest-coverage -MD -MT CMakeFiles/foo.dir/test.cc.o -MF CMakeFiles/foo.dir/test.cc.o.d -o CMakeFiles/foo.dir/test.cc.o -c ../test.cc
[2/2] : && /usr/bin/c++  -g  -O0 -g -fprofile-arcs -ftest-coverage CMakeFiles/foo.dir/test.cc.o  -o foo   && :
$ ./foo
Hello World!
$ ninja -v foo-genhtml
[1/8] cd /home/giaco/tmp/lcov-cmake/build && /usr/bin/geninfo --quiet --base-directory /home/giaco/tmp/lcov-cmake --initial --gcov-tool /usr/bin/gcov-7 --output-filename /home/giaco/tmp/lcov-cmake/build/CMakeFiles/foo.dir/test.cc.info.init --no-external /home/giaco/tmp/lcov-cmake/build/CMakeFiles/foo.dir/test.cc.gcno
geninfo: WARNING: could not open /home/giaco/tmp/test.cc
geninfo: WARNING: some exclusion markers may be ignored
[2/8] cd /home/giaco/tmp/lcov-cmake/build && cat /home/giaco/tmp/lcov-cmake/build/CMakeFiles/foo.dir/test.cc.info.init > /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info.raw
[3/8] cd /home/giaco/tmp/lcov-cmake/build && /usr/bin/lcov --quiet -a /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info.raw --output-file /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info --base-directory /home/giaco/tmp/lcov-cmake --initial && /usr/bin/lcov --quiet -r /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info --output-file /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info --initial
FAILED: lcov/data/init/foo.info 
cd /home/giaco/tmp/lcov-cmake/build && /usr/bin/lcov --quiet -a /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info.raw --output-file /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info --base-directory /home/giaco/tmp/lcov-cmake --initial && /usr/bin/lcov --quiet -r /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info --output-file /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info --initial
lcov: ERROR: no valid records found in tracefile /home/giaco/tmp/lcov-cmake/build/lcov/data/init/foo.info.raw
ninja: build stopped: subcommand failed.

At least the first error (geninfo: WARNING: could not open /home/giaco/tmp/test.cc) can be fixed passing ${CMAKE_BINARY_DIR} instead of ${CMAKE_SOURCE_DIR} to the --base-directory option of geninfo. This because Ninja works with paths relative to the binary directory.

Everything works fine with the Makefile generator, which works with absolute paths.

Use find_package_message for printing messages

Instead of printing the same message over and over again (e.g. if the compiler doesn't support code coverage, but FindCodecov.cmake is called more than once), the find_package_message() function should be used to print these messages only once.

get_filename_component doesn't cover .t.cpp

Some source code layout guidelines suggest to name test source files with the combined extension .t.cpp. Unfortunately the use of get_filename_component at

CMake-codecov/cmake/Findcodecov.cmake

Line 136 in 9a24e83

get_filename_component(FILE_EXT "${FILE}" EXT)

is not adequate in such a situation, because it extracts the maximal extension .t.cpp instead of just .cpp, which makes the source file not recognizable.

An alternative would be the use of a regular expression, i.e. instead of:

get_filename_component(FILE_EXT "${FILE}" EXT)
string(TOLOWER "${FILE_EXT}" FILE_EXT)
string(SUBSTRING "${FILE_EXT}" 1 -1 FILE_EXT)

one could use

string(REGEX REPLACE "^.*\\.([^\\.]+)$" "\\1" FILE_EXT "${FILE}")
string(TOLOWER "${FILE_EXT}" FILE_EXT)

Case mismatch in passing lcov to find_package_handle_standard_args

The latest version of cmake warns at

CMake-codecov/cmake/FindLcov.cmake

Line 56 in 2682f89

find_package_handle_standard_args(lcov

[cmake] CMake Warning (dev) at /usr/local/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:272 (message):
[cmake]   The package name passed to `find_package_handle_standard_args` (lcov) does
[cmake]   not match the name of the calling package (Lcov).  This can lead to
[cmake]   problems in calling code that expects `find_package` result variables
[cmake]   (e.g., `_FOUND`) to follow a certain pattern.
[cmake] Call Stack (most recent call first):
[cmake]   cmake/FindLcov.cmake:56 (find_package_handle_standard_args)
[cmake]   cmake/Findcodecov.cmake:271 (find_package)
[cmake]   CMakeLists.txt:49 (find_package)
[cmake] This warning is for project developers.  Use -Wno-dev to suppress it.

The fix is simply to pass Lcov instead of lcov.

Add .gcda, .gcno, .gcov and lcov-output to CMakes make clean

The .gcno file generated at compile time should be deleted when the object file will be deleted.

The .gcda file generated at execution time should be deleted when the object file will be deleted. If one object of a target gets recompiled all .gcda files of the target should be deleted. (I think this could be done with add_custom_command(... PRE_BUILD ...) and find -name *.gcda). This should also be done with dependent targets.

.gcov files and lcov output should be handled in the same way as .gcda files.

All of these files should be deleted when make clean is called.

Findcodecov.cmake looks for RC compiler with MinGW-w64

I am using CMake-covedoc on Windows 10 with MinGW-w64. It works great but the script includes some logs that made me think there were issues at first and which might be silenced to avoid confusion:

-- Try GNU code coverage flag = [-O0 -g -fprofile-arcs -ftest-coverage]
-- Performing Test COVERAGE_FLAG_DETECTED
-- Performing Test COVERAGE_FLAG_DETECTED - Success
-- Try  code coverage flag = [-O0 -g -fprofile-arcs -ftest-coverage]
CMake Warning at cmake/Findcodecov.cmake:121 (message):
  Code coverage is not available for compiler.  Targets using this compiler
  will be compiled without it.
Call Stack (most recent call first):
  testsuite/CMakeLists.txt:42 (find_package)


-- Try  code coverage flag = [-O0 -g --coverage]
CMake Warning at cmake/Findcodecov.cmake:121 (message):
  Code coverage is not available for compiler.  Targets using this compiler
  will be compiled without it.
Call Stack (most recent call first):
  testsuite/CMakeLists.txt:42 (find_package)

What is not obvious from these logs is that after successfully finding MinGW-w64 g++, Findcodecov.cmake looks for a compiler for RC (which is among the languages in ENABLED_LANGUAGES), fails to find a compiler (there is no CMAKE_RC_COMPILER_ID) and as a result doesn't detect any flag.

To avoid those logs, I see two solutions:

Specifically ignoring RC when found in ENABLED_LANGUAGES.
Skipping the flags lookup and displaying a warning when ${COMPILER} is empty.

Both solutions could probably live side by side in order to avoid the proposed warning anyway when RC isn't found.

Thanks again for the project, it's really nice!

llvm-cov support

LLVM clang also understands the gcc options for enabling coverage collection (e.g. --coverage and -fprofile-arcs -ftest-coverage). But - in that compatibility mode - llvm targets a certain gcc/gcov version. For example one that isn't supported anymore by current gcov/lcov (yielding inaccuracies and lcov errors).

Thus, to increase the portability of the cmake codecov module, it would be useful to also add support for llvm-cov.

kind license change request

The project looks quite good. However, I can't use it in most of my github projects due to the restriction to GPL only license. Would it be considerable for you to change the license to something very liberal, so it becomes a no brainer for most project maintainers to add your cmake code to their projects?

Using CMake-codecov with $<TARGET_OBJECTS:xxxxxxxx>

I am facing a Coverage disabled warning while trying to use CMake-codecov.

I am building executable with TARGET_OBJECTS option, that prevent CMake-codecov to discover the language used. The internal variable NUM_COMPILERS is equal to zero in Findcodecov.cmake. Would it be possible to investigate this development?

ADD_EXECUTABLE(my_test_prog $<TARGET_OBJECTS:my_test_prog_obj>)
add_coverage(my_test_prog)

Below is a CMakeLists.txt, that repoduces the problem (one needs to have codecov available to CMake)

CMAKE_MINIMUM_REQUIRED(VERSION 2.8.8)
PROJECT(hello_world)
IF(NOT DEFINED CMAKE_BUILD_TYPE)
    SET(CMAKE_BUILD_TYPE Coverage)
ENDIF()
STRING(TOUPPER "${CMAKE_BUILD_TYPE}" CMAKE_BUILD_TYPE_UPPER)
IF(${CMAKE_BUILD_TYPE_UPPER} STREQUAL "COVERAGE")
    MESSAGE(STATUS "Adding coverage")
    SET(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake) # ;${CMAKE_MODULE_PATH}
    FIND_PACKAGE(codecov)
ENDIF()
FILE(WRITE "${CMAKE_CURRENT_BINARY_DIR}/my_test_prog.cpp"
"
#include <iostream>
using namespace std;
int main()
{
    cout << \"Hello World!\" << endl;
    return 0;
}
")

ADD_LIBRARY(my_test_prog_obj OBJECT ${CMAKE_CURRENT_BINARY_DIR}/my_test_prog.cpp)
ADD_EXECUTABLE(my_test_prog $<TARGET_OBJECTS:my_test_prog_obj>)
add_coverage(my_test_prog)

The log is here

-- Adding coverage
-- Try  code coverage flag = [-O0 -g -fprofile-arcs -ftest-coverage]
-- Try  code coverage flag = [-O0 -g --coverage]
CMake Warning (dev) at cmake/Findcodecov.cmake:174 (message):
  Coverage disabled for target my_test_prog because there is no sanitizer
  available for target sources.
Call Stack (most recent call first):
  cmake/Findcodecov.cmake:34 (add_coverage_target)
  CMakeLists.txt:25 (add_coverage)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: buildCoverage

#make
Scanning dependencies of target my_test_prog_obj
[ 50%] Building CXX object CMakeFiles/my_test_prog_obj.dir/my_test_prog.cpp.obj
[ 50%] Built target my_test_prog_obj
Scanning dependencies of target my_test_prog
[100%] Linking CXX executable my_test_prog.exe
[100%] Built target my_test_prog

Guillaume

rwth-hpc / cmake-codecov Goto Github PK

cmake-codecov's People

Contributors

Stargazers

Watchers

Forkers

cmake-codecov's Issues

Recommend Projects

Recommend Topics

Recommend Org