How to invoke boost::tests? about cloudabi-ports HOT 6 CLOSED

Hi Wladimir,

Today I didn't have as much time as I had hoped, but I did find some time to at least take a look at how Boost's unit testing framework works. I think this work can be divided up in three steps:

1. Actually get the Boost unit testing framework to run.
2. Figure out a way to inject argdata_t objects into tests in a structured way, so that you can run tests that use the filesystem, network, etc.
3. Make it all feel 'CloudABI native'. As in, having proper bindings from YAML to command line arguments, having a sane default entry point, etc.

I think I just succeeded at the first part. Here's the code that I used:

#include <program.h>
#include <stdio.h>

#include <argdata.hpp>
#include <cstdlib>
#include <string_view>
#include <vector>

#define BOOST_TEST_MODULE example
#include <boost/test/unit_test.hpp>

BOOST_AUTO_TEST_CASE(hello) {}                                                   

void program_main(const argdata_t *ad) {
  std::vector<char *> arguments{const_cast<char *>("argv0")};
  for (auto[key, value] : ad->as_map()) {
    std::string_view keystr = key->as_str();
    if (keystr == "terminal") {
      // boost::test prints output to stderr. Let stderr use to the file
      // descriptor provided.
      FILE *fp = fdopen(value->as_fd(), "w");
      if (fp != nullptr) {
        setvbuf(fp, NULL, _IONBF, 0);
        fswap(fp, stderr);
        fclose(fp);
      }
    } else if (keystr == "arguments") {
      // Command line arguments that need to get forwarded to the unit
      // testing framework.
      for (auto arg : value->as_seq()) {
        const char *argstr;
        if (argdata_get_str_c(arg, &argstr) == 0)
          arguments.push_back(const_cast<char *>(argstr));
      }
    }
  }
  arguments.push_back(nullptr);

  // Invoke Boost's unit testing entry point.
  std::exit(boost::unit_test::unit_test_main(
      &init_unit_test_suite, arguments.size() - 1, arguments.data()));
}

You can then build it as follows:

x86_64-unknown-cloudabi-c++ -std=c++1z -o boost_test boost_test.cc  -lboost_unit_test_framework

If you then run it with this configuration file:

%TAG ! tag:nuxi.nl,2015:cloudabi/
---
terminal: !fd stdout
arguments: [-p, --color_output]

It gives you this output (with fancy colors):

cloudabi-run boost_test < boost_test.yaml
Running 1 test case...

0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
***************************************************

*** No errors detected

Be sure to let me know whether that's sufficient for you to get started!

from cloudabi-ports.

Hi Wladimir!

What's a bit nasty is that you can build both CloudABI applications that use main() and program_main(). The former can be executed directly (./foo) and the latter through cloudabi-run. The former has the downside that you can't pass in file descriptors in a structured way; they just get inherited from the previous process as usual. Ideally we should only support the latter, but that breaks a lot of stuff (e.g., Autoconf that has int main() {} build tests).

When I ported over Boost, I applied some quick fixes to at least make boost::test build, but right now it is pretty much in a broken state. I should spend some time to just rip out the main()-based test runner that's shipped with Boost and develop a replacement that uses program_main(), so that it can receive all of the relevant command line options through argdata, etc.

Wednesdays are the days on which I can hack on CloudABI full-time. As I think getting stuff like this to work has a high priority, I think I'll get started on this this week.

from cloudabi-ports.

When I ported over Boost, I applied some quick fixes to at least make boost::test build

Right, that makes sense.

I'd be okay with the answer "you have to roll your own program_main() entry point". Also because it may depend on what is to be tested, what needs to be passed in. I think for bitcoin we need just a log output (for keeping track of test results) and a temporary directory, to create temporary leveldb databases and such.

A way to pass the arguments normally passed to boost::test runner would also be nice (e.g. to run only specific tests).

from cloudabi-ports.

Great! Yes that should do the job, I'll give it a try tomorrow.

from cloudabi-ports.

It works. My only suggestion would be to disable buffering on stderr:

setvbuf(stderr, NULL, _IONBF, 0);

This makes sure that everything is printed when a crash happens, which is kind of useful for tests.

from cloudabi-ports.

Ah, yes. That's a good idea. I've just edited the code above to include it, in case anyone else were to copy-paste it.

Thanks for publishing your blog post about CloudABI earlier today, by the way. It's nice when people spread the word!

from cloudabi-ports.