Ahhh. Nothing like a beautiful user interface.
CxxTest is a JUnit/CppUnit/xUnit-like framework for C++.
Its advantages over existing alternatives are that it:
In addition, CxxTest is slightly easier to use than the C++ alternatives, since you don't need to "register" your tests.
CxxTest is available under the GNU Lesser General Public License.
This guide is not intended as an introduction to Extreme Progamming and/or unit testing. It describes the design and usage of CxxTest.
The homepage for CxxTest is http://cxxtest.sourceforge.net. You can always get the latest release from the SourceForge downloads page. The latest version of this guide is available online at http://cxxtest.sourceforge.net/guide.html. A PDF version is also available at http://cxxtest.sourceforge.net/guide.pdf.
Here's a simple step-by-step guide:
A test suite is a class that inherits from CxxTest::TestSuite.
A test is a public void (void) member function of that class whose name starts with test,
e.g. testDirectoryScanner(), test_cool_feature() and even TestImportantBugFix().
// MyTestSuite.h
#include <cxxtest/TestSuite.h>
class MyTestSuite : public CxxTest::TestSuite
{
public:
void testAddition( void )
{
TS_ASSERT( 1 + 1 > 1 );
TS_ASSERT_EQUALS( 1 + 1, 2 );
}
};
# cxxtestgen.pl --error-printer -o runner.cpp MyTestSuite.h
or, for those less fortunate:
C:\tmp> perl -w cxxtestgen.pl --error-printer -o runner.cpp MyTestSuite.h
# g++ -o runner runner.cpp
or perhaps
C:\tmp> cl -GX -o runner.exe runner.cpp
or maybe even
C:\tmp> bcc32 -erunner.exe runner.cpp
# ./runner
Running 1 test(s).OK!
Now let's see what failed tests look like. We will add a failing test to the previous example:
// MyTestSuite.h
#include <cxxtest/TestSuite.h>
class MyTestSuite : public CxxTest::TestSuite
{
public:
void testAddition( void )
{
TS_ASSERT( 1 + 1 > 1 );
TS_ASSERT_EQUALS( 1 + 1, 2 );
}
void testMultiplication( void )
{
TS_ASSERT_EQUALS( 2 * 2, 5 );
}
};
Generate, compile and run the test runner, and you will get this:
# ./runner
Running 2 test(s).
MyTestSuite.h:15: Expected (2 * 2 == 5), found (4 != 5)
Failed 1 of 2 test(s)
Success rate: 50%
Fixing the bug is left as an excercise to the reader.
CxxTest can also display a simple GUI. The way to do this is depends on your compile, OS and environment, but you might try this:
perl cxxtestgen.pl -o
runner.cpp --gui=Win32Gui MyTestSuite.h. Note that to link the
executable you may need to add the library comctl32.lib.
./cxxtest.pl --gui=X11Gui -o runner.cpp MyTestSuite. You may
need to tell the compiler where to find X, usually something like
g++ -o runner -L/usr/X11R6/lib runner.cpp -lX11.
cxxtestgen.pl with the option --gui=QtGui. As
always, compile and link the the Qt headers and libraries.
See Graphical user interface and Running the samples for more information.
There is much more to CxxTest than seeing if two times two is four. You should probably take a look at the samples in the CxxTest distribution. Other than that, here are some more in-depth explanations.
Here are the different "assertions" you can use in your tests:
| Macro | Description | Example
|
TS_FAIL()
| Fail unconditionally | TS_FAIL("Test not implemented");
|
TS_ASSERT(expr)
| Verify | TS_ASSERT(messageReceived());
|
TS_ASSERT_EQUALS(x, y)
| Verify | TS_ASSERT_EQUALS(nodeCount(), 14);
|
TS_ASSERT_DELTA(x, y, d)
| Verify | TS_ASSERT_DELTA(sqrt(4.0), 2.0, 0.0001);
|
TS_ASSERT_DIFFERS(x, y)
| Verify | TS_ASSERT_DIFFERS(exam.numTook(), exam.numPassed());
|
TS_ASSERT_LESS_THAN(x, y)
| Verify | TS_ASSERT_LESS_THAN(ship.speed(), SPEED_OF_LIGHT);
|
TS_ASSERT_THROWS(expr, type)
| Verify that | TS_ASSERT_THROWS(parse(file), Parser::ReadError);
|
TS_ASSERT_THROWS_ANYTHING(expr)
| Verify that | TS_ASSERT_THROWS_ANYTHING(buggy());
|
TS_ASSERT_THROWS_NOTHING(expr)
| Verify that | TS_ASSERT_THROWS_NOTHING(robust());
|
TS_WARN(message)
| Print | TS_WARN("TODO: Check invalid parameters");
|
TS_FAIL
TS_FAIL just fails the test.
It is like an assert(false) with an error message.
For example:
class SillySuite : public CxxTest::TestSuite {
public:
void testSomething( void )
{
TS_FAIL( "I don't know how to test this!" );
}
};
TS_ASSERT
TS_ASSERT is the basic all-around tester.
It works just like the assert() macro (which I sincerely
hope you know and use!)
An example:
class TestFileLibrary : public CxxTest::TestSuite
{
public:
void testSquare( void )
{
MyFileLibrary::createEmptyFile("test.bin");
TS_ASSERT( access( "test.bin", 0 ) == 0 );
}
};
TS_ASSERT_EQUALS
This is the second most useful tester. As the name hints, it is used to test if two values are equal.
class TestIntegerMathLibrary : public CxxTest::TestSuite
{
public:
void testSquare( void )
{
TS_ASSERT_EQUALS( square(-5), 25 );
}
};
TS_ASSERT_DELTA
Similar to TS_ASSERT_EQUALS, this macro
verifies two values are equal up to a delta.
This is basically used for floating-point values.
class TestFloatingPointMathLibrary : public CxxTest::TestSuite
{
public:
void testSquareRoot( void )
{
TS_ASSERT_DELTA( squareRoot(4.0), 2.0, 0.00001 );
}
};
TS_ASSERT_DIFFERS
The opposite of TS_ASSERT_EQUALS, this macro is used to assert
that two values are not equal.
class TestNumberGenerator : public CxxTest::TestSuite
{
public:
void testNumberGenerator( void )
{
int first = generateNumber();
int second = generateNumber();
TS_ASSERT_DIFFERS( first, second );
}
};
TS_ASSERT_LESS_THAN
This macro asserts that the first operand is less than the second.
class TestNumberGenerator : public CxxTest::TestSuite
{
public:
void testFindLargerNumber( void )
{
TS_ASSERT_LESS_THAN( 23, findLargerNumber(23) );
}
};
TS_ASSERT_THROWS and friends
These assertions are used to test whether an expression throws an exception.
TS_ASSERT_THROWS is used when you want to verify the type of exception
thrown, and TS_ASSERT_THROWS_ANYTHING is used to just make sure something
is thrown. As you might have guessed, TS_ASSERT_THROWS_NOTHING asserts
that nothing is thrown.
class TestTouchyFunctions : public CxxTest::TestSuite
{
public:
void testStandardThrower( void )
{
TS_ASSERT_THROWS_NOTHING( checkInput(1) );
TS_ASSERT_THROWS( checkInput(-11), std::runtime_error );
}
void testOtherThrower( void )
{
TS_ASSERT_THROWS_ANYTHING( otherThrower() );
}
};
TS_WARN
TS_WARN just prints out a message, like the #warning
preprocessor directive.
I find it very useful for "TODO" items.
For example:
class ToDoList : public CxxTest::TestSuite {
public:
void testToDoList( void )
{
TS_WARN( "TODO: Write some tests!" );
TS_WARN( "TODO: Make $$$ fast!" );
}
};
In the GUI, TS_WARN sets the bar color to yellow (unless it was
already red).
ETS_ macros
The TS_ macros mentioned above will catch exceptions thrown from tested code
and fail the test, as if you called TS_FAIL().
Sometimes, however, you may want to catch the exception yourself; when you do, you can
use the ETS_ versions of the macros.
class TestInterestingThrower : public CxxTest::TestSuite
{
public:
void testInterestingThrower()
{
// Normal way: if an exception is caught we can't examine it
TS_ASSERT_EQUALS( foo(2), 4 );
// More elaborate way:
try {
ETS_ASSERT_EQUALS( foo(2), 4 );
} catch( const BadFoo &e ) {
TS_FAIL( e.bar() );
}
}
};
TSM_ macros
Sometimes the default output generated by the ErrorPrinter doesn't give you enough
information. This often happens when you move common test functionality to helper functions
inside the test suite; when an assertion fails, you do not know its origin.
In the example below (which is the file sample/MessageTest.h from the CxxTest distribution),
we need the message feature to know which invocation of checkValue() failed:
class MessageTest : public CxxTest::TestSuite
{
public:
void testValues()
{
checkValue( 0, "My hovercraft" );
checkValue( 1, "is full" );
checkValue( 2, "of eels" );
}
void checkValue( unsigned value, const char *message )
{
TSM_ASSERT( message, value );
TSM_ASSERT_EQUALS( message, value, value * value );
}
};
Note: As with normal asserts, all TSM_ macros have their
non-exception-safe counterparts, the ETSM_ macros.
CxxTest comes with some samples in the sample/ subdirectory of
the distribution. If you look in that directory, you will see three
Makefiles: Makefile.unix, Makefile.msvc and
Makefile.bcc32 which are for Linux/Unix, MS Visual C++ and
Borland C++, repectively. These files are provided as a starting point,
and some options may need to be tweaked in them for your system.
If you are running under Windows, a good guess would be to run
nmake -fMakefile.msvc run_win32 (you may need to run
VCVARS32.BAT first). Under Linux, make
-fMakefile.unix run_x11 should probably work.
When you have several test cases for the same module, you often find that all of them start with more or less the same code--creating objects, files, inputs, etc. They may all have a common ending, too--cleaning up the mess you left.
You can put all this code in a common place by overriding
the virtual functions TestSuite::setUp() and
TestSuite::tearDown(). setUp() will
then be called before each test, and tearDown()
after each test.
class TestFileOps : public CxxTest::TestSuite
{
public:
void setUp() { mkdir("playground"); }
void tearDown() { system( "rm -Rf playground"); }
void testCreateFile()
{
FileCreator fc( "playground" );
fc.createFile( "test.bin" );
TS_ASSERT_EQUALS( access( "playground/test.bin", 0 ), 0 );
}
};
It's very hard to maintain your tests if you have to generate, compile and run the test runner manually all the time. Fortunately, that's why we have build tools!
Let's assume you're developing an application. What I usually do is the following:
Unfortunately, there are way too many different build tools and IDE's for me to give ways to use CxxTest with all of them.
I will try to outline the usage for some cases.
Generating the tests with a makefile is pretty straightforward. Simply add rules to generate, compile and run the test runner.
# -*- Makefile -*-
all: lib run_tests app
# Rules to build your library
lib: ...
# Rules to build your application
app: ...
# A rule that runs the unit tests
run_tests: runner
./runner
# How to build the test runner
runner: runner.cpp
g++ -o runner runner.cpp
# How to generate the test runner
runner.cpp: SimpleTest.h ComplicatedTest.h
cxxtestgen.pl -o $@ --error-printer $^
See sample/Construct in the CxxTest distribution for an example of building CxxTest test runners
with Cons.
There are several ways to integrate CxxTest with visual studio, none of which is perfect. Here is one:
.cpp file generated by CxxTest,
and write the cxxtestgen.pl command as the custom build step.
This works, but is a little hacky and quite tedious. If anoyone has a better idea, or wants to implement this as an addin/macro, please do!
Unit testing for device drivers?! Why not?
And besides, the build utility can also be used to build
user-mode application.
To use CxxTest with the build utility,
you add the generated tests file as an extra dependency
using the NTBUILDTARGET0 macro and the Makefile.inc
file.
You can see an example of how to do this in the CxxTest distribution
under sample/winddk.
There are currently three GUIs implemented: native Win32, native X11 and
Qt. To use this feature, just specify --gui=X11Gui,
--gui=Win32Gui or --gui=QtGui as a parameter for
cxxtestgen (instead of e.g. --error-printer). A
progress bar is displayed, but the results are still written to standard
output, where they can be processed by your IDE (e.g. Emacs or Visual
Studio).
Note that whatevr GUI you use, you can combine it with the
--runner option to control the formatting of the text output,
e.g. Visual Studio likes it better if you use
--runner=ParenPrinter.
If you run the generated Win32 or Qt GUIs with the command line
-minimized, the test window will start minimized (iconified)
and only pop up if there is an error (the bar turns red). This is useful
if you find the progress bar distracting and only want to check it if
something happens.
As with any self-respecting GUI application, here are some screenshots for you to enjoy:
Ahhh. Nothing like a beautiful user interface.
Topics in this section are more technical, and you probably won't find them interesting unless you need them.
You may have noticed that TS_ASSERT_EQUALS only works for built-in
types.
This is because CxxTest needs a way to compare object and to convert them to strings,
in order to print them should the test fail.
If you do want to use TS_ASSERT_EQUALS on your own data types,
this is how you do it.
First of all, don't forget to implement the equality operator (operator==())
on your data types!
Since CxxTest tries not to rely on any external library (including the standard library, which is not always available), conversion from arbitrary data types to strings is done using value traits.
For example, to convert an integer to a string, CxxTest does the following actions:
int i = value to convert;
CxxTest::ValueTraits<int> converter(i);
string = temp.asString();
CxxTest comes with predefined ValueTraits for int, char,
dobule etc. in cxxtest/ValueTraits.h.
Defining value traits for new types is easy. This example shows how:
class MyClass {
public:
//...
bool operator== ( const MyClass & ) const;
const char *converToString() const;
};
#ifdef CXXTEST_RUNNING // This is defined by the test runner
#include <cxxtest/ValueTraits.h>
namespace CxxTest {
class ValueTraits<MyClass> {
const MyClass &_m;
public:
ValueTraits( const MyClass &m ) : _m(m) {}
const char *asString() { return _m.convertToString(); }
};
};
#endif // CXXTEST_RUNNING
If you don't like the way CxxTest defines the default ValueTraits,
you can override them by #define-ing CXXTEST_USER_VALUE_TRAITS;
this causes CxxTest to omit the default definitions, and from there on you are
free to implement them as you like.
You can see a sample of this technique in test/UserTraits.tpl.
Usually, when a TS_ASSERT_* macro fails, CxxTest moves on to
the next one. In many cases, however, this is not the desired way.
Consider the following code:
class DangerousTest : public CxxTest::TestSuite
{
public:
void testUtilities()
{
char *buffer = new char[1024];
TS_ASSERT( buffer );
memset( buffer, 0, 1024 ); // But what if buffer == 0?
}
};
If you have exception handling enabled, you can make CxxTest exit each
test as soon as a failure occurs. To do this, you need to define
CXXTEST_ABORT_TEST_ON_FAIL before including the CxxTest
headers. This can be done using the --abort-on-fail
command-line option or in a template file; see
sample/aborter.tpl in the distribution. Note that if CxxTest
doesn't find evidence of exception handling when scanning your files,
this feature will not work. To overcome this, use the
--have-eh command-line option.
A TestListener is a class that receives notifications about
the testing process, notably which assertions failed. CxxTest defines
a standard test listener class, ErrorPrinter, which is
responsible for printing the dots and messages seen above. When the
test runners generated in the examples run, they create an
ErrorPrinter and pass it to
TestRunner::runAllTests(). As you might have guessed, this
functions runs all the test you've defined and reports to the
TestListener it was passed.
If you don't like or can't use the ErrorPrinter, you can use
any other test listener.
To do this you have to omit the --error-printer switch when
generating the tests file.
It is then up to you to write the main() function, using the
test listener of your fancy.
stdio printer
If the ErrorPrinter's usage of std::cout clashes
with your environment or is unsupported by your compiler, don't dispair!
You may still be able to use the StdioPrinter, which does the
exact same thing but uses good old printf().
To use it, invoke cxxtestgen.pl with the --runner=StdioPrinter option.
As an example, CxxTest also provides the simplest possible test listener,
one that just reports if there were any failures.
You can see an example of using this listener in sample/yes_no_runner.cpp.
To use you own test runner, or to use the supplied ones in different ways, you can use
CxxTest template files. These are ordinary source files with the embedded "command"
<CxxTest world> which tells cxxtestgen.pl to insert the world definition
at that point. You then specify the template file using the --template option.
See samples/file_printer.tpl for an example.
Most of the time you write test suites to actually test some of your own code.
For example, for any class X you might write a test suite TestX.
Now, it is generally considered a good thing for test code to reside close to the code
being tested. For this reason, CxxTest allows you to embed a test suite inside the "real" class
being tested.
You do this by including the file cxxtest/SelfTest.h and using the macros CXXTEST_SUITE
and CXXTEST_CODE.
When CxxTest scans your file, it collects CXXTEST_CODE blocks and writes them in a new class
inside the generated source file.
NOTE: What I usually do is use these embedded test suites where I would write comments and leave the "heavier" tests to seperate test suites. This way you get "living" comments in the code and avoid clutter.
Note that you must use double curly braces in CXXTEST_CODE blocks.
Here is an example:
#include <cxxtest/SelfTest.h>
class SillyMath
{
public:
CXXTEST_SUITE(TestSillyMath);
CXXTEST_CODE({{
void testGetLargerNumber() {
SillyMath s;
TS_ASSERT_LESS_THAN( 3, s.getLargerNumber(3) );
}
}});
int getLargerNumber( int i ) {
return i + 1;
}
};
Usually, your test suites are instantiated statically in the tests file, i.e. say you
defined class MyTest : public CxxTest::TestSuite, the generated file will
contain something like static MyTest g_MyTest;.
If, however, your test suite must be created dynamically (it may need a constructor,
for instance), CxxTest doesn't know how to create it unless you tell it how.
You do this by writing two static functions, createSuite() and destroySuite().
See sample/CreatedTest.h for a demonstration.
CxxTest does a very simple analysis of the input files, which is sufficient in most cases. This means, for example, that you can't indent you test code in "weird" ways.
A slight inconvenience arises, however, when you want to comment out tests. Simply commenting out the tests would not work:
class MyTest : public CxxTest::TestSuite
{
public:
// void testCommentedOutStillGetsCalled()
// {
// }
#if 0
void testMarkedOutStillGetsCalled()
{
}
#endif
};
The "correct" way to comment out tests (until proper parsing is written for CxxTest) is
to temporarily change the name of the test function, e.g. by prefixing it with x:
class MyTest : public CxxTest::TestSuite
{
public:
void xtestFutureStuff()
{
}
};