How to write tests for a parallel code about cgreen HOT 10 CLOSED

I'm not sure what you are asking here. If your code has units (I'm not familiar with "MPI parallel code") then you should be able to write unit tests, right? Parallel code might be trickier to debug, but to test drive, no, I see no reason why not.

If you are asking, can I use Cgreen for "my environment", then the answer is "if it's linux-like with a C or C++ library that supports compiling Cgreen, sure!"

Is there a specific problem you see?

from cgreen.

Ok, let me try to explain my problem so you can point where I'm going wrong here. Assume I want to write a parallel code to sum all the integers between a until b using MPI:
First the sum_parallel files:

# sum_parallel.h
#ifndef SUM_PARALLEL_H
#define SUM_PARALLEL_H

typedef struct _mympi {
    int rank, size;
} mympi;

int sum_parallel(int, int, mympi*);

#endif

# sum_parallel.c
#include "sum_parallel.h"

int sum_parallel(int a, int b, mympi *mpi)
{
    int end, start = mpi->rank * ((b - a) / mpi->size) + a;

    if(mpi->rank == mpi->size - 1)
        end = b;
    else
        end = start + ((b - a) / mpi->size);

    int i, sum = 0;
    for(i = start; i < end; i++)
        sum += i;

    int total = 0;
    MPI_Reduced(&sum, &total, 1, MPI_INT, ...);

    return total;
}

and here is my main function:

# main.c
#include "sum_parallel.h"

int main(int argc, char **argv)
{
    MPI_Init(&argc, &argv);

    mympi *mpi = malloc(sizeof(*mpi));
    MPI_Comm_size(MPI_COMM_WORLD, &mpi->size); // Number of threads
    MPI_Comm_rank(MPI_COMM_WORLD, &mpi->rank); // Thread id

    int total = sum_parallel(1, 1000, mpi);

    MPI_Finalize();
}

and for compiling, we need mpicc.

$ mpicc -o sum sum_parallel.c main.c
$ mpiexec -np 16 ./sum

Well, now I want to write some tests for sum_parallel module (actually, I'm writing my tests before writing the program itself but anyway):

Ensure(sum_parallel, works_properly)
{
    // Initialize mpi here, like the first 4 lines of main.c
    int total = sum_parallel(0, 10, &mpi);
    assert_that(total, is_equal_to(50));
    // Finalize mpi
}

Usually, I'm using cgreen-runner, so in this case, we should have something like this:

$ mpicc -fPIC -c sum_parallel.c sum_parallel_test.c
$ mpicc -shared -o tests.so sum_parallel.o sum_parallel_test.o
$ cgreen-runner tests.so

as you can see, in this way, I'm not able to simply use cgreen-runner, so I'm wondering what is the right way to implement tests for a parallel code in the framework of cgreen. Here is some errors I'm getting after running my tests:

[name:26040] *** Process received signal ***
[name:26040] Signal: Segmentation fault (11)
[name:26040] Signal code: Address not mapped (1)
[name:26040] Failing at address: 0xcf102b18
[name:26040] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x2b0952640330]
[name:26040] [ 1] /lib/x86_64-linux-gnu/libc.so.6(strlen+0x2a)[0x2b09528d89da]
[name:26040] [ 2] /users/username/.local/lib/libopen-pal.so.20(opal_argv_join+0x3c)[0x2b09536ef43c]
[name:26040] [ 3] /users/username/.local/lib/libmpi.so.20(ompi_mpi_init+0x60d)[0x2b095316edbd]
[name:26040] [ 4] /users/username/.local/lib/libmpi.so.20(MPI_Init+0x83)[0x2b095318c663]
[name:26040] [ 5] sum_parallel_tests.so(MYMPI_Init+0x22)[0x2b0952f23887]
[name:26040] [ 6] sum_parallel_tests.so(+0x27f4)[0x2b0952f227f4]
[name:26040] [ 7] /users/username/.local/lib/libcgreen.so.1(run_the_test_code+0x5b)[0x2b0952219b6b]
[name:26040] [ 8] /users/username/.local/lib/libcgreen.so.1(run_test_in_its_own_process+0x156)[0x2b095221ab36]
[name:26040] [ 9] /users/username/.local/lib/libcgreen.so.1(+0x98f7)[0x2b09522198f7]
[name:26040] [10] /users/username/.local/lib/libcgreen.so.1(+0x9926)[0x2b0952219926]
[name:26040] [11] /users/username/.local/lib/libcgreen.so.1(run_test_suite+0x35)[0x2b0952219ae5]
[name:26040] [12] cgreen-runner(runner+0x2d5)[0x403885]
[name:26040] [13] cgreen-runner(main+0x19b)[0x401afb]
[name:26040] [14] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x2b0952871f45]
[name:26040] [15] cgreen-runner[0x401c74]
[name:26040] *** End of error message ***
sum_parallel_test.c: Exception: sum_parallel -> works_properly 
    Test terminated with signal: Segmentation fault

from cgreen.

Sorry, I'm not much help here (I don't have a clue as to what all those MPI-functions are doing), but isn't yoursum_parallel.cdependent on the setup of the parallelisation that you do in yourmain()`?

If so you need to do that for your tests too. Possibly in a setup/before-function.

Cgreen has no knowledge of parallel or non-parallel code. It will run setup/before code and then the test. cgreen-runner don't do anything in it's main program, if you need something to be executed before your test, you need to ensure that it is called.

from cgreen.

Thanks a lot for your quick response.

About setupping the parallelisation, I put a comment on the first line of my test which says like the first 4 lines of the main function, I should initialize the parallelization inside my test too. I was too lazy to copy paste the code there.

Well, maybe I can narrow down my problem into the fact that MPI codes should be run by mpiexec or mpirun commands, however cgreen-runner use a different method for running the tests.

Anyway, I guess we cannot go further from here, so feel free if you want to close this issue.

Again, thanks for this great framework, I really enjoy working with cgreen :)

from cgreen.

Yes, if there are requirements like the one you mention, then the Cgreen runner can't do that. However, you could try writing your own runner that does that, as described in the docs. That runner would be a main program and could do all the things that you do in your main program. You would lack the auto-discovery, though.

Thank you for using Cgreen!

from cgreen.

@saeedSarpas, tests using MPI are a bit tricky because cgreen forks new processes for each test case and MPI requires that processes are created with MPI_Comm_spawn. I believe there is a way to turn off forking via an environment variable but I forgot how that works. @thoni56 do you remember?

If it helps, I have a project that uses some MPI and cgeen for unit testing (https://github.com/d-meiser/BeamLaser). I ended up encapsulating the parts that need MPI in separate functions which I tested without cgreen.

from cgreen.

@d-meiser, thanks for the response. Encapsulating parallel parts in separate functions was also my original idea, but I've thought it would be nice if there is a standard way to also test parallel parts using cgreen. Also thanks for the link to your repository.

from cgreen.

There is an open issue to add a mechanism to launch tests in-process instead of forking (#5), and there was some hope that a patch could be provided. But nothing has happened with that so far.

from cgreen.

@thoni56 thanks for clarifying.

from cgreen.

I've created PR #97 that addresses the fork()-ing. It will probably be merged in a few days. I'd be happy to hear of any feedback on if this helped you with this problem.

from cgreen.