Using Unit Tests To Write Better Embedded Software - LEKULE

Breaking

10 Feb 2016

Using Unit Tests To Write Better Embedded Software

Unit tests can help you write better embedded software. Here’s how.

Unit tests are additional software functions that you write to test the software "units" of your application. These tests help you ensure that your embedded software is working correctly -- now and as it changes over time.

In an embedded C application, a "unit" is typically a single source file (and corresponding header file). This source "module" is usually an abstraction for some part of the system and implements a group of related functions, e.g. a ring buffer or a protocol parser.

The unit tests for that module are a group of functions which exercise the "module under test." The unit test functions call functions of the module under test in a specific order and with specific arguments -- and verify that the module returns the correct results.
These test functions are not included in your release build, but are run during development (or after any changes) to test your application.
Typically, each source module to be tested has a corresponding unit test file, where all the unit tests for that module go.

For example, if you had your own implementation of some data structure -- like a stack -- then the unit tests might call the push and pop functions to make sure that the stack behaves as expected in a variety of conditions.
Here's an example of just one of those functions:

                    #include "some_test_framework.h"
#include "my_stack.h"

// A test for my_stack.
void test_WhenIPushAnItem_ThenTheCountIncreases(void)
{ 
    // Do something.
    stack_push('a');
    
    // Make sure it had the right effect.
    ASSERT(stack_get_count() == 1);
}
                  

This particular test is just the tip of the iceberg. We could quickly and easily add more tests for other conditions -- even conditions unlikely to be encountered when your software is in operation.
For example, how does the stack behave when it fills up? Sure I think I know what's going to happen based on how I wrote it, but when will that code ever actually be called?
Hint: I really hope it's not 10 years from now, when the device is a mile under the ocean, and you're nowhere to be found!

If you create a unit test for this case, you can run that code right now and be sure of what it actually does.

                    // Another test for my_stack.
void test_GivenTheStackIsFull_WhenIPushAnotherItem_ThenItIsRejected(void)
{
    // Fill the stack.
    for (int i = 0; i < 100; i++)
    {
        stack_push('a');
    }
 
    // Try to push another.
    bool success = stack_push('a');
    
    // Make sure it was rejected.
    ASSERT(success == false);
    ASSERT(stack_get_count() == 100);
}
                  

This is especially relevant for embedded software, since it has to deal with real hardware. With hardware, you can't usually exercise all of its behavior and so it's difficult to know with certainty that your software is going to handle all of it okay.
For example, how can I test my temperature conversion logic across all ranges of temperature, when my temp sensor is reading a comfortable 72 degrees -- the temperature of my office?
I suppose I could stick my hardware in a freezer or thermal chamber, but that is going to 1) take some physical effort to set up and 2) not be very repeatable.
A better option, as you might have guessed by now, would be to put all of my temperature conversion logic in its own source module and write a bunch of unit tests for it. I could feed in any raw sensor value that I want (including errors) and check that each is handled correctly.
The goal of a unit test is to test your software "unit" in isolation from the rest of the system. You treat the unit as a black box, call functions in a specific order and with specific arguments, and verify that you get the correct results. The reason to test in isolation is that when something goes wrong, you know exactly where the problem is -- in the module under test.
Most source modules have dependencies though. To test a module in isolation, you can not include other modules that it might depend on. So what do you need to do? Ah, the answer is that you need to "mock" those dependencies.
A mock is a fake implementation of a module that allows you to simulate and inspect the interactions to it. You can control how a mock behaves, so that you can fully exercise the module under test.
In the temperature sensor example, the temp sensor driver (with the conversion logic) might need to use an I2C driver to talk to the sensor. To test the temp sensor driver in isolation, you would need to mock the I2C driver.

The I2C driver mock allows you to return whatever test data you want to the temp sensor driver when it makes calls into the I2C driver. When reading the current temperature register, instead of actually going out the hardware, you just tell it to return 0xFF (or whatever value you want) instead.
The other great thing about mocking the I2C driver is that it removes any hardware dependencies from the tests. This means you don't actually need the real hardware to test the application. You can compile the tests and run them on the host PC.
Sounds great so far, right? Good. So how do you actually do this? Okay, okay, I'm getting to that.
There are two main components to any unit test setup: the unit test framework itself, and the mocking framework. The unit test framework is what allows you to define and execute tests, and gives you some "assertion" functions to assert that a particular test has passed or failed. The mocking framework is what you use to mock your dependencies and test each module isolation.
If you're developing a .NET application in Visual Studio or a Java app in Eclipse, the unit test support is built right in to the IDE. You just set up your tests and click the "run tests" button. This is automatic test discovery, and is super convenient. When you set up your test files correctly, the test framework can automatically run all your tests in a single step.
If you're writing an embedded application in C, the best option right now is Ceedling. It's a unit test system built around Rake (like make but for the Ruby language). To use it you'll need to install Ruby, but you don't actually have to know anything about Ruby.
Ceedling uses Unity as its unit test framework and CMock as its mocking framework. The reason it's so great is that it provides automatic test discovery and execution. This makes it easy to get up and running quickly. And it also will automatically generate mock modules if you ask it correctly.
Ceedling is designed to work by running tests on a host PC -- not on target hardware. The tests are compiled using a native compiler (gcc by default). This means that the tests run quickly -- no waiting to flash your hardware -- and can be run continuously during development without slowing you down.
Since the tests are running on your host PC, all your hardware dependencies need to be mocked -- like the I2C driver in the temperature sensor above. Since the tests are running on a PC, the tests can't access the target processor's I2C registers because they don't exist.
This encourages a well-designed, layered architecture where the hardware interfaces are decoupled from the rest of the application logic.
Have you ever worked on a project where the hardware wasn't ready yet? Or there wasn't enough to go around? Or it was changing in the next board rev?  Being able to develop and test some, or maybe even most, of your application without the hardware can help in each of these cases.

You're still going to need to test on real hardware at some point, but you can get pretty far without it.

Since the application is built from a bunch of individually unit tested modules, when you do test on real hardware there will be a lot less to test. You're only testing the integration of those modules. And... the best part is that there will be fewer bugs to find and fix.

No comments: