IMPORTANT: You should read this document even if you are not writing tests for your code — although the real question is why are you not? — but still. This page nicely explains, with examples, how to write modular, testable and therefore good code.

1. What are unit tests?

From Wikipedia:

Unit testing is a software testing method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine whether they are fit for use.

...

To do that, we will "mock" the dependencies of the unit that is subject to testing. Hence, the unit will think it interacts with real units, while we will only provide it facades. That way we can give the unit all possible inputs that could happen during normal operation, and then verify that, for each of the inputs, the unit responds correctly. We can do that because we mocked the dependencies, and the mocks we are using allow us to specify behavior when queried by our unit, and we can check that the right outputs were given to them. We always treat the unit under test as a black box and never test implementation details.

2. How to write code that can be unit tested

In order to be able to unit test code, it has to comply with the following criteria:

• It has to consist of many separate units. Every unit must have clearly defined interaction with other units.
• Every unit needs to have a clear interface it exposes to other units, so that it can be easily mocked
• Every unit should hold as little state as possible — see below.
• If a unit is hard to test, it probably means it should be split in multiple components. This usually happens when a component starts holding a lot of state and/or has many possible actions and different outcomes — see below.
• Every public interface method declaration should be preceded by the TEST_VIRTUAL macro — see this subsection.

2.1. A side note on state

It is much easier to test pure functions because their output only depends on the input, while with impure functions, the output depends on the state. As this state is internal, it is harder to manipulate it - that usually involves hacks and relies on implementation details. The other possibility is to have helper methods to set up state, but if that state is not meant to be manipulated from the outside, this pollutes the code, creates a lot of ambiguity, and leads to less isolation and more bugs. We will come back to this in the 3rd paragraph of the next section.

2.2. A side note on splitting components

As a rule of thumb, most people usually create too little components, so you shouldn't be afraid of splitting code into more components. That being said, you should always think thoroughly about what a single unit is capable of doing.

...

If you have a component that does a lot of processing and holds a lot of state, split it into one that does processing and one that holds state. That way, you will be able to test the processing one because you will give it inputs and verify outputs, without having to set up the state manually (because state will be an input from the other component). It will also be easier to test the one that holds state, because it will be less complicated to achieve every particular state.

2.3. The TEST_VIRTUAL macro

Every method on a unit that is a dependency for another unit (so, literally every unit) needs to have a TEST_VIRTUAL declaration in front.

...

What this does is declares a method virtual in testing and HOOTL but makes it non-virtual in production. Virtual functions have a bit of overhead because of the way C++ function resolving works - it involves a function pointer table (called vtable) lookup. That is why we do not want them in production code. In testing, however, the mock must be able to extend from the class and override the methods other units use, so the methods have to be virtual.

3. An example of a component and tests for it

Let's consider an example, the Armed state. It is a great example because it is derived from another component and interacts with several others.

...

• StateMachine (for state transitions and to access components not owned by Armed)
• Estimator (for altitude and velocity change detection)
• Buzzer
• Logger
• ComHandler (sending data over radio)
• DecayingCondition (to eliminate noise during threshold detection)

3.1 Constructors

The only component that is directly owned by Armed is DecayingCondition - all the others will be accesses through the StateMachine. Therefore, the two constructors for Armed look like this:

...

NOTE: In the header file, we only use a forward declaration of DecayingCondition. We only #include "DecayingCondition.h" in the cpp file. Otherwise, a change in DecayingCondition.h would require a recompilation of "Armed.h", which would require a recompilation of all the .cpp files that include Armed - there is many - and that is unnecessary because those files do not care about DecayingCondition (if they do, they include it themselves and they will be recompiled anyways).

3.1.1 Sensor dependencies

Sensor dependencies are a different story. Sensor classes are not available at all in testing, so the default constructor won't be able to construct the sensor class unless we provide it one. Therefore, we need to pay special attention when creating a mock for the sensor - more on this here.

We also need to conditionally include this mock class in the .cpp implementation, as so:

...

Here, a forward declaration in the .h file of the component using the sensor is even more important, as the header file won't yet know which implementation it is getting.

3.2 Main methods made mockable

Now, let's have a look at how to make Armed mockable - it has to be mocked because it is used by other components (ComHandler in particular which calls disarm() on Armed when it receives the disarm command over radio). Therefore, we need to mark disarm() with TEST_VIRTUAL.

...

In the second case, make sure you mark it with override and not final. We do this because the mock of this class must be able to override those methods. If you wanted to not allow subclasses to override this method, use the TEST_FINAL macro in place of final. It is defined together with the TEST_VIRTUAL macro and will mark the function final in production but override in testing. Therefore, mock will be able to override this method while child classes (in production) will not. To be precise, derived classes will also be able to override the method during testing, but not during production, so you will catch any bastards trying to override your untouchable method when compiling code for production.

3.2.1 Clarification on virtual functions and destructors.

Skip this section only if you know everything about virtual functions in C++ and destructors.

...

Virtual destructors do not behave exactly like other virtual functions. If a destructor is not virtual, the child will automatically call the parent implementation. In parent's context, the parent will not call the child's implementation as is the case with other non-virtual functions. However, with destructors, this problem is even worse, because it means that one of the destructors might not be called. This leads to major memory leaks, especially if we create and destroy many object during the lifecycle of our program.

3.3 Destructors

Now, we can finally write the destructors. The procedure here is a bit different than with other functions. The destructor has to be virtual for components from which other components are derived. Therefore, it is also virtual for the child classes. This means that the destructor will not be virtual only if the component is not derived from anything and that no component extends it. The only case (afaik at least) in our code where this happens is the StateMachine.

So, if the destructor is virtual, it will be virtual in production and the mock class can easily override it - no macros needed. However, in the rare case that it does not need to be virtual (from the standpoint of production code), you can still make it virtual, or use the TEST_VIRTUAL macro. The first option is preferred because there is very little overhead of a virtual destructor for the StateMachine - it only happens after the code finishes executing = never. It is only worth it if you are really writing a component with no inheritance that is created and destroyed a lot. Additionally, if a destructor is not virtual, when it should be, this causes memory leaks because if a component is destroyed in the parent context, child's destructor is not called unless it is virtual, as we've seen in this example.

state.h

virtual ~State(); // base class destructor, important to be virtual 

...

3.4 The mock of the component

Now that we have made the component mockable, we can write a mock for it - yay!

...

And that's it! We have successfully completed the mock and the class, now let's use them in a test!

3.4.1 Sensor mocks

A sensor mock serves as a full substitute for a sensor class (as described here), so its name must be identical to the original class name. The name of the file containing the mock should still start with Mock and be located under test/. It also does not extend from the original class, so double-check that their signatures match, as compiling tests won't verify that as is the case with other components.

#include <gmock/gmock.h>
// we don't include the sensor class header

class Sensor { // same name as the original class 
 
public:
    Sensor(Wire x, uint8_t){} // whatever the constructor signature for the sensor is.
    
    MOCK_METHOD0(bar, void());
    MOCK_METHOD1(foo, int(double));
    
    // destructor testing
    MOCK_METHOD0(__die__, void());
    ~Sensor() {
        __die__();
    }
}

3.5 The unit tests for the component

If you forgot, this section describes the expected behavior of Armed in detail. We will write separate tests for every feature of the component.

We will write our tests in a kebab-case named file - test-armed.cpp. In general, tests for LongComponentName will be in test-long-component-name.cpp.

3.5.1 Testing setup code

Of course, there is some code we need to write before we actually start writing the tests.

...

Normally, when writing tests with GoogleTest, your fixture will extend ::testing::Test, but here, you should extend StateMachineFixture, located under pyxida/test. This class has some helper methods and sets up the mock for arduino hardware functions and the StateMachine mock, which will be required for virtually every test. This is only to remove boilerplate and make your life easier.

Which fixture class should I extend

Ordered by hierarchy:

• Extend ::testing::Test if you don't require hardware functions or the StateMachine. (if you are testing a utility class for example)
• Extend ArduinoTestFixture if you require hardware functions (yes, millis() is one of them) but not the StateMachine
• Extend StateMachineFixture if you require the StateMachine
• Extend another fixture that is derived from StateMachineFixture if you are testing a derived class, for example TestState)

Additionally, we have other base fixtures that are subclasses of StateMachineFixture. TestState implements some common testing behavior for testing all states. TestState serves both as a base class test fixture (base class State -> fixture TestState) and a base for fixtures of derived classes (derived class Armed -> fixture TestArmed, extends TestState);

Code

#include "TestState.h"

class TestArmed : public TestState {
protected:
    // must be protected because every single test is actually a subclass
    // behind the scenes - that's just how GoogleTest implements it -
    // and we need to make those available to the subclass.
    
    MockDecayingCondition *mockDC; // all dependencies as mocks (except StateMachine)
    
    Armed * armed; // instance under testing
    
    TestArmed() {
        mockDC = new MockDecayingCondition;
        armed = new Armed(mockSM, mockDC); // Using the general constuctor of Armed to inject mocks
    }
    ~TestArmed() override {
        // Make sure armed deletes its direct dependencies but not indirect (like StateMachine)
        EXPECT_CALL(*mockDC, __die__());
        delete armed;
    }
} 

In every test, the TestArmed fixture will be constructed and destructed at the end. Hence, we will start with a fresh Armed instance and fresh instances of all mocks.

Test that the destructor was called

EXPECT_CALL is a GoogleTest macro that helps us verify that the method __die__() was called on mockDC. This makes sure mockDC's destructor is called. __die__() does not actually refer to the destructor, but we defined the destructor of MockDecayingCondition to call __die__ as shown in this section. For more information on EXPECT_CALL or other GoogleTest and GoogleMock macros, see this subsection or the useful links section. Usually, googling things will also work.

3.5.2 Test cases

So, let us begin writing the tests. The first feature of Armed:

...

This is a very simple example of what a test should be doing. It sets some expectations on the mocks and then executes our action. We don't even need to verify the expectations manually; it's done by google test in the background when mocks are destructed. That is another reason why we check for mock destruction in every test (destructor of TestArmed is called after every test).

...

Read more about the EXPECT_CALL macro and all of its features below.

4. In-depth explanation of some testing features and special cases for testing

4.1 The EXPECT_CALL macro

The second argument of EXPECT_CALL is the function name, along with parameter matchers. You can read more on parameter matchers in GoogleMock documentation. In our case, we are not even using them - rather, we are using values directly, which means that our expectations will only be satisfied if the function is called with exactly the specified parameters.

...

EXPECT_CALL(obj, method(_)).Times(0);
component->update();
Mock::VerifyAndClearExpectations(); // verify expectations and clear them
...

4.2 Testing abstract classes

Sometimes, we need to test a functionality of an abstract class - how do we do that, if we cannot even have an instance of it?

...

5. How to run unit tests for pyxida-firmware

Actually, our great team lead Zack put together a shell script, that executes the whole below procedure (still see the error section). It is located at bin/runUnitTests.sh (bin is a sibling of pyxida) You can run it from bin or from the top-level folder, but not from anywhere else (for example: pyxida; bash code for that would require some dark magic not even Zack is certified for).

But, if you are feeling adventurous/want to know how it works, read on:

The full procedure

We are using cmake to build code for our tests, as it also pulls in GoogleTest and the Arduino mock. To build code with unit tests, starting in the pyxida folder:

mkdir build
cd build
cmake ..
make runUnitTests.o
./runUnitTests.o

5.1 Errors and fixes

Cannot find googletest

A common error when compiling unit tests will be something in terms of cannot find googletest. That probably means you haven't initialized git submodules, so run git submodule update --init

Error: leaked mock object

As this error tells you, you have a leaked mock object, which was not deleted. There are multiple possible reasons for this:

1. The class you are mocking does not have a virtual destructor

If the object you are mocking does not have a virtual destructor and it is deleted in the parent scope, the mock's destructor won't be called, which is very bad because no expectations on that mock can be verified.

...

• Make the base class destructor virtual
• Mark the mock's destructor with override

2. You forgot to delete the dependency in the code

If the object that is being mocked is a direct dependency, the component using it (under test) should delete in its destructor. Make sure:

• that destructor is called (you are deleting the component under test in the test)
• that destructor is virtual (you might be deleting the component but it's destructor isn't virtual so the child one isn't being called)
• you are deleting the dependency in that destructor.

3. You forgot to delete the dependency in the test

If the mocked object is not a direct dependency of the component under test, it is not responsible for deleting it, so you should delete the mock yourself.

SIGSEGV: Segmentation fault

Oh, aren't they our favorite? The full beauty of C++ is only expressed when we encounter segmentation faults. Nevertheless, let's see some potential reasons for it:

...

In general, if you debug the program, you should find the exact spot where the SEGFAULT occurs.

Not on this list

If you have an error that is not on this list, let us know and the fix will be added to the list.

6. Useful references and further reading

GoogleTest

• GoogleTest docs
• GoogleTest primer

GoogleMock

• GoogleMock docs
• GoogleMock cook book
• GoogleMock cheat sheet (a much shorter version than CookBook)

...