, dev
Rexo: A Unit Testing Framework in C (Part 2)
I revamped my C89/C++ unit testing framework around the end of 2019 but I never really shared what’s cool about it... let’s finally address this!
When I implemented Rexo’s first iteration in 2018, which I already wrote about here, I mostly set out to understand why the majority of the unit testing frameworks for C required so much boilerplate in comparison to the ones offered in C++.
The short answer is that the C standard doesn’t offer any portable way to implement automatic registration of tests.
I ended up going ahead with the usual approach but, for someone who prides himself with always aiming at providing the best user experience possible, I wasn’t happy at all with my own implementation and wanted to address it somehow.
Automatic Test Registration
The C frameworks NovaProva and Criterion workaround C’s limitation by respectively reading the DWARF debug information generated during a debug compilation, and by parsing the ELF (Unix), Mach-O (macOS), and PE (Windows) executable file formats to read their data sections.
This might just be my lack of knowledge but I found both implementations to be so involved that I couldn’t fully make sense of their fine details. I was hoping for a much simpler solution.
The data section approach used by Criterion is pretty neat because it allows storing some data initialized at the file scope and then retrieve it later on, which is exactly what we were missing.
Although data sections are not part of the C/C++ standards, it turns our that they are so important to low-level system programming that each compiler expose them in a way or another.
So I looked into this direction and... tada! I got a fairly straightforward implementation working with support for the GNU-like compilers (GCC, clang, ICC) and MSVC[1], which is pretty much the same coverage as Criterion’s, and can also be extended as needed.
#include <stdio.h>
/* Example of automatic test registration.
This approach uses a data section ‘bar’ to store multiple static instances
of type ‘foo’. These objects are first statically instantiated at the file
scope and a pointer is then stored in the data section, between some given
‘start’ and ‘end’ locations. Objects stored this way can then be retrieved
at runtime by iterating over all the pointers found between these ‘start’
and ‘end’ locations.
*/
struct foo {
int value;
};
/* Implementation Details */
/* -------------------------------------------------------------------------- */
#if defined(_MSC_VER)
__pragma(section("bar$a", read))
__pragma(section("bar$b", read))
__pragma(section("bar$c", read))
__declspec(allocate("bar$a"))
extern const struct foo * const bar_begin = NULL;
__declspec(allocate("bar$c"))
extern const struct foo * const bar_end = NULL;
#define DEFINE_SECTION \
__declspec(allocate("bar$b"))
#elif defined(__APPLE__)
extern const struct foo * const
__start_bar __asm("section$start$__DATA$bar");
extern const struct foo * const
__stop_bar __asm("section$end$__DATA$bar");
#define DEFINE_SECTION \
__attribute__((used,section("__DATA,bar")))
DEFINE_SECTION
extern const struct foo * const dummy = NULL;
#elif defined(__unix__)
extern const struct foo * const __start_bar;
extern const struct foo * const __stop_bar;
#define DEFINE_SECTION \
__attribute__((used,section("bar")))
DEFINE_SECTION
extern const struct foo * const dummy = NULL;
#endif
/* Public API */
/* -------------------------------------------------------------------------- */
#if defined(_MSC_VER)
#define SECTION_BEGIN \
(&bar_begin + 1)
#define SECTION_END \
(&bar_end)
#elif defined(__unix__) || defined(__APPLE__)
#define SECTION_BEGIN \
(&__start_bar)
#define SECTION_END \
(&__stop_bar)
#endif
#define REGISTER_FOO(id, value) \
static const struct foo id = { value }; \
DEFINE_SECTION \
extern const struct foo * const id##_ptr = &id;
/* Usage */
/* -------------------------------------------------------------------------- */
REGISTER_FOO(b, 234);
REGISTER_FOO(a, 123);
REGISTER_FOO(c, 345);
int
main(
void
)
{
const struct foo * const *it;
for (it = SECTION_BEGIN; it < SECTION_END; ++it)
{
if (*it != NULL)
{
printf("%d\n", (*it)->value);
}
}
return 0;
}
And... that’s it! I think that’s fairly sweet and short, at least in comparison to what I’ve seen in NovaProva and Criterion.
Rexo makes use of this automatic registration feature through the macros RX_TEST_CASE and RX_TEST_SUITE, as shown in this minimal example from the documentation.
Designated Initializer-like Syntax With Cascading Configuration
C99, and later C++20, introduced designated initializers which is a neat feature to default initialize a struct to zero while letting users set the value of hand-picked fields, similarly to default named function parameters for other languages.
struct my_struct {
const char *foo;
int bar;
double baz;
};
/* Only setting the bar field leaves foo and baz to 0. */
struct my_struct inst = {.bar = 123};
This kind of syntax is something that I’ve been looking forward to implement in Rexo to configure the test cases, suites, and fixtures. Not only because it’s a nice syntactic sugar but also because it would allow me to extend the configuration options as needed without breaking the API and requiring users to update their code.
It turns out that here again Criterion got this one sorted out. Almost.
From my understanding, Criterion allows to configure all the tests in a suite at once by setting configuration values on the suite itself. That’s great! On top of that, it’s still possible to override these values for each individual test case. However, this overriding operation works by replacing the whole set of configuration values instead of inheriting the ones from the parent suite and only updating the values passed to the test cases.
TEST_SUITE(my_suite, .foo = "hello", .baz = 1.23);
TEST_CASE(my_suite, my_case, .foo = "world")
{
// Expectation : {foo = "hello", bar = 0, baz = 1.23}
// Criterion : {foo = "world", bar = 0, baz = 0.0}
}
Not happy with that, I decided to try taking Criterion’s approach to the next level while still supporting C89 and all C++ versions.
It took me some time to figure it out but the solution is quite simple in the end! We just need to close our eyes on all these redundant macros required to support C89 and its lack of variadic arguments.
#include <stdio.h>
#include <string.h>
/* Macros */
/* -------------------------------------------------------------------------- */
#define UNUSED(x) (void)(x)
#define EXPAND(x) x
#define CONCAT_(a, b) a##b
#define CONCAT(a, b) CONCAT_(a, b)
#define STRUCT_SET_MEMBER(x) (*obj) x;
#define STRUCT_UPDATE_0()
#define STRUCT_UPDATE_1(_0) \
STRUCT_SET_MEMBER(_0)
#define STRUCT_UPDATE_2(_0, _1) \
STRUCT_SET_MEMBER(_0) \
STRUCT_UPDATE_1(_1)
#define STRUCT_UPDATE_3(_0, _1, _2) \
STRUCT_SET_MEMBER(_0) \
STRUCT_UPDATE_2(_1, _2)
#define STRUCT_UPDATE_4(_0, _1, _2, _3) \
STRUCT_SET_MEMBER(_0) \
STRUCT_UPDATE_3(_1, _2, _3)
#define STRUCT_UPDATE_5(_0, _1, _2, _3, _4) \
STRUCT_SET_MEMBER(_0) \
STRUCT_UPDATE_4(_1, _2, _3, _4)
#define STRUCT_UPDATE_6(_0, _1, _2, _3, _4, _5) \
STRUCT_SET_MEMBER(_0) \
STRUCT_UPDATE_5(_1, _2, _3, _4, _5)
#define STRUCT_UPDATE_7(_0, _1, _2, _3, _4, _5, _6) \
STRUCT_SET_MEMBER(_0) \
STRUCT_UPDATE_6(_1, _2, _3, _4, _5, _6)
#define STRUCT_UPDATE_8(_0, _1, _2, _3, _4, _5, _6, _7) \
STRUCT_SET_MEMBER(_0) \
STRUCT_UPDATE_7(_1, _2, _3, _4, _5, _6, _7)
#define STRUCT_DEFINE_UPDATE_FN(id, type, arg_count, args) \
static void \
id(type *obj) \
{ \
UNUSED(obj); \
EXPAND(CONCAT(STRUCT_UPDATE_, arg_count) args) \
}
/* Config */
/* -------------------------------------------------------------------------- */
struct config {
const char *foo;
int bar;
double baz;
};
typedef void (*config_update_fn)(
struct config *
);
struct config_desc {
const config_update_fn update;
};
#define CONFIG(id, arg_count, args) \
STRUCT_DEFINE_UPDATE_FN( \
id##_config_update_fn, \
struct config, \
arg_count, \
args) \
\
static const struct config_desc \
id##_config_desc = {id##_config_update_fn}
/* Test Suite */
/* -------------------------------------------------------------------------- */
struct test_suite_desc {
const char *name;
const struct config_desc *config_desc;
};
#define TEST_SUITE_(id, arg_count, args) \
CONFIG(id, arg_count, args) \
/* Test suite registration and declaration goes here. */
#define TEST_SUITE_1(id, _0) \
TEST_SUITE_(id, 1, (_0))
#define TEST_SUITE_2(id, _0, _1) \
TEST_SUITE_(id, 2, (_0, _1))
/* Test Case */
/* -------------------------------------------------------------------------- */
struct test_case_desc {
const char *suite_name;
const char *name;
const struct config_desc *config_desc;
};
#define TEST_CASE_(suite_id, id, arg_count, args) \
CONFIG(suite_id##_##id, arg_count, args) \
/* Test case registration and declaration goes here. */
#define TEST_CASE_1(suite_id, id, _0) \
TEST_CASE_(suite_id, id, 1, (_0))
#define TEST_CASE_2(suite_id, id, _0, _1) \
TEST_CASE_(suite_id, id, 2, (_0, _1))
/* Usage */
/* -------------------------------------------------------------------------- */
TEST_SUITE_2(my_suite, .foo = "hello", .baz = 1.23);
TEST_CASE_1(my_suite, my_case, .foo = "world");
int
main(
void
)
{
struct config config;
memset(&config, 0, sizeof config);
my_suite_config_desc.update(&config);
my_suite_my_case_config_desc.update(&config);
printf("foo: %s\n", config.foo);
printf("bar: %d\n", config.bar);
printf("baz: %f\n", config.baz);
return 0;
}
The trick basically consists in defining a function which contains the necessary logic to update an existing configuration instance with the given field values.
Maybe it’ll be clearer after expanding the two macros calls from the usage section above.
/* TEST_SUITE_2(my_suite, .foo="hello", .baz=1.23) */
/* -------------------------------------------------------------------------- */
static void
my_suite_config_update_fn(
struct config *obj
)
{
(void)(obj);
(*obj).foo = "hello";
(*obj).baz = 1.23;
}
static const struct config_desc
my_suite_config_desc = {my_suite_config_update_fn};
/* TEST_CASE_1(my_suite, my_case, .foo="world") */
/* -------------------------------------------------------------------------- */
static void
my_suite_my_case_config_update_fn(
struct config *obj
)
{
(void)(obj);
(*obj).foo = "world";
}
static const struct config_desc
my_suite_my_case_config_desc = {my_suite_my_case_config_update_fn};
/* -------------------------------------------------------------------------- */
int
main(
void
)
{
struct config config;
memset(&config, 0, sizeof config);
my_suite_config_desc.update(&config);
my_suite_my_case_config_desc.update(&config);
printf("foo: %s\n", config.foo);
printf("bar: %d\n", config.bar);
printf("baz: %f\n", config.baz);
return 0;
}
Straighforward, right?
If you’re curious to see how it looks like in action in Rexo, check out this concrete example from the documentation.
Closing Notes
I’m fairly happy with the current state of Rexo although there would still be some more work needed to be done in order to implement more assertion macros, to output the report summaries in different standard formats (e.g.: jUnit XML), and to provide more visual diagnostic messages, amongst other things.
But it’s still plenty enough for me to rely on it for my other projects, and maybe it would be satisfactory for yours as well? If it’s something that you’d be interested in exploring, maybe a good place to start would be to have a peek at its comprehensive documentation. Or you can always head straight to its GitHub repository otherwise.
If you end up using it, please let me know how it goes!
