Unit Testing — Dawn of Giants

There is no standard unit testing framework in C++ yet, but many options are available. At our stage of development, we want a simple internal framework that allows us to run unit tests in isolation, nothing fancy, really just a way to declare tests, and run over them all at once. As we are using CMake for our build system, we also want the framework to integrate with CTest.

We started by defining a polymorphic context for our unit tests using the runtime concept idiom (see this post ):

class UnitTest {
public:
  template<typename T>
  UnitTest(T&& impl) : _self(std::make_shared<model<T>>(std::forward<T>(impl)) {};
  UnitTest(UnitTest const&) = default;
  UnitTest(UnitTest&&) noexcept = default;
  UnitTest& operator=(UnitTest const&) = default;
  UnitTest& operator=(UnitTest&&) noexcept = default;

  friend void test(UnitTest const& unitTest) {
    unitTest._self->_test();
  }

private:  
  struct concept_t {
    virtual ~concept_t() = default;
    virtual void _test() const = 0;
  };

  template<typename T>
  struct model final : public concept_t {
    model_t(T&& data) : _data(std::forward<T>(data)) {}
    void _test() const override { test(_data); }
    T _data;
  };

  std::shared_ptr<concept_t const> _self;
};

template<typename T>
void test(T const& x) {
  x();
}

where the model implements the test concept by calling a non-member non-friend test function that executes the test by calling its call operator. We then created a test register that holds all our tests and iterates over them:

class UnitTestRegister {
public:
  UnitTestRegister() = default;
  ~UnitTestRegister() = default;
  UnitTestRegister(UnitTestRegister const&) = delete;
  UnitTestRegister& operator=(UnitTestRegister const&) = delete;

  void push(UnitTest test) { _unitTests.emplace_back(std::move(test)); }
  void run() {
    for (auto const& unitTest : _unitTests) test(unitTest);
  }

private:
  std::vector<UnitTest> _unitTests;
  sempervirens::logging::ConsoleLogger _logger;
};

The register also declares a logger, so tests do not have to declare their own to log their output.

In Sempervirens, we do not separate the source code from the testing code (just a preference) for the classes we want to test. Those provide a static Test method that is only parsed in Debug mode. We also want to be able to build any unit test by providing a callable object. We are using two macros and a helper function to cover these two cases:

#define SEMPERVIRENS_TESTTYPE(type) []() { type::Test(); }
#define SEMPERVIRENS_TESTCALL(callable, ...) \                                       
  sempervirens::unittesting::createUnitTest(callable, ##__VA_ARGS__)  

template<typename F, typename... Args>
UnitTest createUnitTest(F&& func, Args&&... args) {
  return [_func = std::forward<F>(func), 
          _args = std::make_tuple(std::forward<Args>(args)...)]() {
    std::apply(_func, _args);
  };
}

For Sempervirens classes, a simple lambda calling the static Test method is used. For other types, the createUnitTest helper is meant to be generic enough to cover most cases such as free functions, closures, or member functions, all possibly taking any number of arguments. Note that, lacking perfect capture of parameter packs in lambdas before C++20, we capture the parameters inside a tuple which is then passed to std::apply, available in C++17.

With that set up, we can either write different .cpp files for individual tests (each declaring its own unit test register). Or we can group many tests in a single register in one file. In any case, integrating that test framework with CTest is easy: when run in Debug mode, CMake will create one executable per file using a simple helper function. Ctest will then iterate over the tests.

Here is a made up example that illustrates testing a Sempervirens class providing its own Test method, another type we want to test member functions of, and other possible callables. The code is followed by the ctest -V command output:

// A Sempervirens type with embedded Test method
class SempervirensType {  
public:
  void print() const { SEMPERVIRENS_MSG("%d\n", _val); }
  void add(int n) { SEMPERVIRENS_MSG("adding %d\n", n); _val += n; }
private:
  int _val{0};

#if SEMPERVIRENS_BUILD(UNITTESTING)
public:
  void static Test()
  {
    SempervirensType t;
    t.print();
    t.add(2);
    t.print();
  }
#endif
};

// Another type we want to test member functions of
class OtherType {  
public:
  void print() const { SEMPERVIRENS_MSG("%d\n", _val); }
  void add(int n) { SEMPERVIRENS_MSG("adding %d\n", n); _val += n; }
private:
  int _val{0};
};

// Other callables
auto closure = [](int val){ SEMPERVIRENS_MSG("closure with arg: %d\n", val); };
void freeFunctionNoArg() { SEMPERVIRENS_MSG("free function with no arg"); }
void freeFunctionOneArg(int val) { SEMPERVIRENS_MSG("free function with one arg: %d\n", val); }

// A test wrapper for additional control (e.g. code injection)
class TestWrapper
{
public:
  TestWrapper(std::string filename) : _file{std::make_unique<std::ofstream>(std::move(filename))} {}
  void operator()() const
  {
    SEMPERVIRENS_MSG("Logging test result to file\n");
    *_file << "test result\n";
  }
private:
  std::unique_ptr<std::ofstream> _file;
};

int main()
{
  auto reg = UnitTestRegister();
  reg.push(SEMPERVIRENS_TESTTYPE(SempervirensType));
  OtherType t;
  reg.push(SEMPERVIRENS_TESTCALL(&OtherType::add, &t, 10));
  reg.push(SEMPERVIRENS_TESTCALL(&OtherType::print, &t));
  reg.push(SEMPERVIRENS_TESTCALL(closure, 2));
  reg.push(SEMPERVIRENS_TESTCALL(freeFunctionNoArg));
  reg.push(SEMPERVIRENS_TESTCALL(freeFunctionOneArg, 10));
  reg.push(TestWrapper{"TestLog.txt"});
  reg.run();
  return 0;
}

Output:
2: Test command: /home/nico/DawnOfGiants/build-Sempervirens-Debug-Linux/tests/test_unittesting
2: Test timeout computed to be: 10000000
2: [MSG] UnitTestingTest.cpp, void SempervirensType::print() const, 11: 0
2: [MSG] UnitTestingTest.cpp, void SempervirensType::add(int), 12: adding 2
2: [MSG] UnitTestingTest.cpp, void SempervirensType::print() const, 11: 2
2: [MSG] UnitTestingTest.cpp, void OtherType::add(int), 32: adding 10
2: [MSG] UnitTestingTest.cpp, void OtherType::print() const, 31: 10
2: [MSG] UnitTestingTest.cpp, , 37: closure with arg: 2
2: [MSG] UnitTestingTest.cpp, void freeFunctionNoArg(), 38: free function with no arg
2: [MSG] UnitTestingTest.cpp, void freeFunctionOneArg(int), 39: free function with one arg: 10
2: [MSG] UnitTestingTest.cpp, void MyTest::operator()() const, 47: Logging test result to file
2/4 Test #2: test-unittesting .................   Passed    0.00 sec