jasmcaus / tau Goto Github PK

Currently, Muon's internal tests sit at ~69 tests. We would like to have more tests.
This issue is open to anyone who'd like to contribute to this project.

Good resources:

• Doctest
• Catch2

The floor is open! Good luck!
-J

See here:

Missing things:

1. What guarantees provides tau on those types?
2. Minimal comment on why are those defined
3. Where to look for compiler instrumentation to check more stuff.

First of all I'd like to thank @jasmcaus for this useful library.

So, I found out a little problem with custom failure message.
According to documentation it's enough to add a further parameter. But If I add a parameter, the check never fails.

Below an example.

TEST(TAU, ItShouldFailWithMessage)
{
    CHECK(0);                           // this check fails correctly.
    CHECK(0, "It should FAIL");         // this check doesn't fail.
}

Am i missing something?

It's not possible to use entire test name (TESTSUITE.TESTNAME) to run a single test by mean of "--filter" option.
Anyway if I remove just last char from the test name, it works.

Hello @jasmcaus ,
I decided to contribute to this unit test framework because I am really going to use it in practice for all my projects. I found it here and I chose it among the others due to its simplicity and effectiveness.
So, I like it very much, but in my opinion there is one aspect that absolutely it is worth to improve. If we consider CHECK_EQ and CHECK_NE as the opposite statement, as well as CHECK_BUF_EQ and CHECK_BUF_NE, I would expect that for strings comparison CHECK_STREQ would be coupled with CHECK_STRNE. Unfortunately, CHECK_STRNE is already used to compare two substrings that are supposed to be equal. In other words, the pattern "_EQ/_NE" is broken.
As such, in order to avoid confusions and keep the whole structure intuitive, I would propose to rename the macros related to strings and substrings as indicated below:

Whole-string Checks:

CHECK_STREQ --> CHECK_STR_EQ
CHECK_STRNEQ --> CHECK_STR_NE
REQUIRE_STREQ --> REQUIRE_STR_EQ
REQUIRE_STRNEQ --> REQUIRE_STR_NE

Substring Checks:

CHECK_STRNE --> CHECK_STRN_EQ
CHECK_STRNNE --> CHECK_STRN_NE
REQUIRE_STRNE --> REQUIRE_STRN_EQ
REQUIRE_STRNNE --> REQUIRE_STRN_NE

As an alternative solution for the substring checks (but this has still to be evaluated) we could allow an optional third parameter in the string check macro. Something like this: CHECK_STR_EQ(actual, expected, n) is like calling CHECK_STRN_EQ(actual, expected, n)

Just to make this transition smooth I would also keep the old names as "deprecated" defines at the bottom of tau.h, so that they can be removed safely at some point in the future.
What do you think?
If you agree, I would be more than happy to take care of this in my repository and then submit a pull request.

Stefano

Hi, this is the follow up of adorad/adorad#6 here is the error message that showed up. (CMakeOutput.log)

-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
-- The C compiler identification is MSVC 19.26.28806.0
-- The CXX compiler identification is MSVC 19.26.28806.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:206 (set_target_properties):
  INTERFACE_LIBRARY targets may only have whitelisted properties.  The
  property "VERSION" is not allowed.


-- Configuring incomplete, errors occurred!
See also "C:/lang/Muon/build/CMakeFiles/CMakeOutput.log".

And I've did some googling and found the problem that caused. which is I'm using MSVC 19.26, MSVS 2019 and cmake 3.15. (Note that I've no prior knowledge in cmake). Your cmake script require 3.5+ cmake_minimum_required(VERSION 3.5 FATAL_ERROR) and my version is lower, however the FATAL_ERROR wasn't also working because

The FATAL_ERROR option is accepted but ignored by CMake 2.6 and higher. It should be specified so CMake versions 2.4 and lower fail with an error instead of just a warning.

Sorry if this issue is is pointless but I just want to point out the problem that caused, maybe you could print an error and terminate the script if the version is not compatible yourself, apparently cmake won't.

I am wondering, if
#include <string.h>
#include <stdarg.h>
are strictly necessary, as we only need a few of the functions and argument handling doesnt sound super complicated either.

Can you add a comment on the other headers, what they are used for?

    #include <errno.h> // Kernel error codes
    #include <libgen.h> // regex for filter [but only contains dirname and __xpg_basename, so kinda useless???
    #include <unistd.h> // file seek
    #include <sys/types.h> // system types, but where are those used?!
    #include <sys/wait.h> // also includes features.h, pid exit status etc as process info
    #include <signal.h> // signal handler
    #include <time.h> // time

Do you have any advice for such user cases?

int input[] = {1, 2, 3, 4};
int output[] = {1, 2, 3, 4};
for (int i = 0; i < 4, i++) {
  // ...
  CHECK_EQ(result, output[i]);
}

I prefer developing in a CI and thus I think disabling warnings as in TAU_DISABLE_DEBUG_WARNINGS is very uncool.

Aside, what warnings and errors are ignored should be the one of the first things in the README and documentation.

Ideally, a testing tool also allows simple integration of valgrind or better document how the user should invoke tests.
You dont mention in the README, how tests should be run (as convention).

From a little toy project https://github.com/matu3ba/pbcd/blob/4ffd0dfe6006c5342302d4099c02d537facf4ffe/runTest.sh

^
test.c:28:3: error: void function '_TAU_TEST_FUNC_foo_bar1' should not return a value [-Wreturn-type]
  return status;

Alternatively (but probably more cumbersome): support custom return values.

As of now, there is no clang-format or other formatting policy. clang-format would be one option.

Without following fix --list option causes segfault.

diff --git a/tau/tau.h b/tau/tau.h
index d446912..361ae6a 100644
--- a/tau/tau.h
+++ b/tau/tau.h
@@ -416,10 +416,9 @@ tauColouredPrintf(int colour, const char* fmt, ...) {
 }
 
 #ifndef TAU_NO_TESTING
-    #define tauPrintf(...)                                  \
-        if(tauTestContext.foutput)                          \
-            fprintf(tauTestContext.foutput, __VA_ARGS__);   \
-        printf(__VA_ARGS__)
+    #define tauPrintf(...) (                                  \
+        tauTestContext.foutput ? fprintf(tauTestContext.foutput, __VA_ARGS__) : 0,  \
+        printf(__VA_ARGS__) )
 #else
     #define tauPrintf(...)   \
         printf(__VA_ARGS__)

My working env is msys2 on Windows, and gcc is of version 12.1.0.

When I compile the example code, an error occurs, which tells:

tau/types:h:22:58: error: size of array 'static_assertion_at_line__LINE__' is negative
22 | #define TAU_STAT_ASSERT1__(cond, line)        typedef char static_assertion_at_line_##line[[(!!(cond)*2-1]
   |                                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

However, gcc of version 11.2.0 on windows can compile the example sucessfully.

I have no idea whether this error is related to the gcc of specific version or not.

As of now, char, which is de facto used in string.h has no abi tests for the user.

Macros for usage

// MACROS
#define IS_SIGNED(Type) (((Type)-1) < 0)

#include <assert.h>
static_assert(sizeof(U8) == 1, "U8 must have 1 byte size"); // compile-time error

// http://scaryreasoner.wordpress.com/2009/02/28/checking-sizeof-at-compile-time/
#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
usage: BUILD_BUG_ON( sizeof(someThing) != PAGE_SIZE );

The macros themself are also useful for a quick abi tests and providing one header+main.c example (ie under 0BSD) would be nice.

In the primer doc there is a mention of REQUIRE_STRNNEQ and CHECK_STRNNEQ
but I think, in the code they're actually REQUIRE_STRNE / REQUIRE_STRNNE and CHECK_STRNE / CHECK_STRNNE

I've never used tau before, but decided to give it a shot today. In order to get it to work, I'm just including it in a test file and compiling, but I'm getting a warning:

In file included from tests/test_lists.c:2:
vendor/tau/tau/tau.h: In function ‘tauClock’:
vendor/tau/tau/tau.h:272:1: warning: no return statement in function returning non-void [-Wreturn-type]
  272 | }
      | ^

Despite the warning, when I run the test, it does run successfully, and if I introduce an error, it does fail successfully. However, the timing functions aren't working. Even when I introduce a 2 second delay, the times are being reported as 0:

SUCCESS: 2 test suites passed in -0.00s

I took a look at the source code of tauClock, and it looks like there are a number of checks trying to figure out what to return based on which architecture I'm on, but I suppose it isn't exhaustive. FWIW, I'm compiling with gcc version 11.4.0 (GCC) on Cygwin on Windows 10. I think it's Cygwin that's giving tau the problem. I tried everything under WSL on the same machine and it works fine with no warning, and it provides correct timings.

Hello, do you plan to include support for Mocks? What do you think is still needed or has to be improved?

Is there even a simple way to macro overload arithmetic expressions to auto-insert overflow checks ie via
SEI Standard https://wiki.sei.cmu.edu/confluence/pages/viewpage.action?pageId=87152052 ?

Alternatively documentation how to emit the instructions for testing with clang/gcc would be great.

Hi and thank you for this project.

I'm trying to use cmake's install() macro in my project which is using Tau.
I followed the instructions from cmake-quickstart.md.
But 'make install' fails with:

CMake Error at _deps/tau-build/cmake_install.cmake:46 (file):
  file INSTALL cannot find  "my/build/_deps/tau-src/include/tau":  No such file or directory.
Call Stack (most recent call first):
  cmake_install.cmake:47 (include)

Appreciate any support.