Test Driven Development with CppUTest, now in Debian

I have recently read Test Driven Development with Embedded C by James W. Grenning and published by Pragmatic Programmers.

I really enjoyed the book: while I was aware of the huge benefits of having a comprehensive test suite, I never studied seriously the principles behind Test Driven Development (TDD) and this book makes a good introduction to the topic. At the same time it focuses on the C language and contains lots of examples on how you can create tests even for projects which have to interact with hardware or other unpredictable components (the key is to create many abstractions) using all the possibilities that C offers.

The author convincingly argues that developing code with TDD forces you to create a modular design that is easier to evolve when the underlying requirements change. He also highlights how the tests serve as reference documentation of the API.

James W. Grenning recommends CppUTest as his xUnit test framework of choice. When I wanted to try this test framework, I discovered that it was not available in Debian. I decided to package it because it has some interesting features not offered by the contenders (at least not to my knowledge). It’s now available in Debian and in Ubuntu.

First, it doesn’t require any explicit registration of tests and has a very lightweight syntax. The small downside is that CppUTest requires the usage of C++ for the tests. But C++ is compatible with C so it doesn’t matter much if you have a C++ compiler for your target. On the contrary, usage of variables and methods scoped to the test group makes it easy to write clear tests. Here’s a short sample of test code:

extern "C" {
#include "timer.h"
#include "timefn.h"
}
 
#include "CppUTest/TestHarness.h"
 
static Time the_time;
static const int start_sec = 123;
static const int start_nsec = 456789000;
static const int delay_sec = 8;
static const int delay_nsec = 111111000; // start_nsec + delay_nsec < 10^9
 
TEST_GROUP(Timer)
{
    /* Class variables available to all tests in the group */
    Timer timer;
    Delay remaining;
 
    /* Standard setup/teardown methods of xUnit tests */
    void setup() {
        timer = timer_new();
        time_set(&the_time, start_sec, start_nsec);
        /* [...] */
    }
 
    void teardown() {
        timer_free(timer);
        /* [...] */
    }
 
    /* Helper functions specific to the test group */
    void start_timer_with_delay(long sec, long nsec)
    {
        timer_set_real_delay(timer, sec, nsec);
        timer_start(timer);
    }
 
    void ensure_remaining_is(long sec, long nsec)
    {
        CHECK_EQUAL(sec, delay_get_seconds(remaining));
        CHECK_EQUAL(nsec, delay_get_nanoseconds(remaining));
    }
};
 
TEST(Timer, NewIsNotStarted)
{
    CHECK(!timer->started);
}
/* [...] */
TEST(Timer, GetRemainingTimeWithNanosecondPrecision_ShiftOfSeconds)
{
    start_timer_with_delay(delay_sec, delay_nsec);
    time_set(&the_time, start_sec + delay_sec - 5, start_nsec + delay_nsec + 1000);
 
    remaining = timer_get_remaining_time(timer);
 
    ensure_remaining_is(4, 999999000);
}

To run those tests, you just need this boilerplate code in a main.cpp:

#include "CppUTest/CommandLineTestRunner.h"
 
int main(int argc, char** argv)
{
   return CommandLineTestRunner::RunAllTests(argc, argv);
}

Another interesting feature is its integrated memory leak detection system. Any test that hasn’t released allocated memory at the end of the “teardown” process will be marked as failed.

The upstream developers have made some unusual choices (static library only, installation in a private directory) but this will likely change with the switch to an automake and autoconf-based build system. I have reported the oddities that I found and I requested them to provide a pkg-config file to make it easier to compile and link unit tests exploiting CppUTest.

I already used CppUTest to develop a small application running on an embedded Linux. At some point, I might try to use CppUTest for dpkg development. I believe that it makes for a good fit. dpkg is already C++ ready since dselect is written in C++ and reuses a good part of dpkg’s code.

In any case, if you like Test Driven Development and are writing C or C++ based applications, I invite you to try CppUTest.

People Behind Debian: Mark Shuttleworth, Ubuntu’s founder

I probably don’t have to present Mark Shuttleworth… he was already a Debian developer when he became millionaire after having sold Thawte to Verisign in 1999. Then in 2002 he became the first African (and first Debian developer) in space. 2 years later, he found another grandiose project to pursue: bring the Microsoft monopoly to an end with a new alternative operating system named Ubuntu (see bug #1).

I have met Mark during Debconf 6 in Oaxtepec (Mexico), we were both trying to find ways to enhance the collaboration between Debian and Ubuntu. The least I can say is that Mark is opinionated but any leader usually is, and in particular the self-appointed ones! 🙂

Read on to discover his view on the Ubuntu-Debian relationship and much more.

Raphael: Who are you?

Mark: At heart I’m an explorer, inventor and strategist. Change in technology, society and business is what fascinates me, and I devote almost all of my time and wealth to the catalysis of change in a direction that I hope improves society and the environment.

I’m 38, studied information systems and finance at the University of Cape Town. My ‘hearts home’ is Cape Town, and I’ve lived there and in Star City and in London, now I live in the Isle of Man with my girlfriend Claire and 14 precocious ducks. I joined Debian in around 1995 because I was helping to setup web servers for as many groups as possible, and I thought Debian’s approach to packaging was very sensible but there was no package for Apache. In those days, the NM process was a little easier 😉

Raphael: What was your initial motivation when you decided to create Ubuntu 7 years ago?

Mark: Ubuntu is designed to fulfill a dream of change; a belief that the potential of free software was to have a profound impact on the economics of software as well as its technology. It’s obvious that the technology world is enormously influenced by Linux, GNU and the free software ecosystem, but the economics of software are still essentially unchanged.

Before Ubuntu, we have a two-tier world of Linux: there’s the community world (Debian, Fedora, Arch, Gentoo) where you support yourself, and the restricted, commercial world of RHEL and SLES/SLED. While the community distributions are wonderful in many regards, they don’t and can’t meet the needs of the whole of society; one can’t find them pre-installed, one can’t get certified and build a career around them, one can’t expect a school to deploy at scale a platform which is not blessed by a wide range of institutions. And the community distributions cannot create the institutions that would fix that.

Ubuntu brings those two worlds together, into one whole, with a commercial-grade release (inheriting the goodness of Debian) that is freely available but also backed by an institution.

The key to that dream is economics, and as always, a change in economics; it was clear to me that the flow of money around personal software would change from licensing (“buying Windows”) to services (“paying for your Ubuntu ONE storage”). If that change was coming, then there might be room for a truly free, free software distribution, with an institution that could make all the commitments needed to match the commercial Linux world. And that would be the achievement of a lifetime. So I decided to dedicate a chunk of my lifetime to the attempt, and found a number of wonderful people who shared that vision to help with the attempt.

It made sense to me to include Debian in that vision; I knew it well as both a user and insider, and believed that it would always be the most rigorous of the community distributions. I share Debian’s values and those values are compatible with those we set for Ubuntu.

“Debian would always be the most rigorous of the community distributions.”

Debian on its own, as an institution, could not be a partner for industry or enterprise. The bits are brilliant, but the design of an institution for independence implies making it difficult to be decisive counterparty, or contractual provider. It would be essentially impossible to achieve the goals of pre-installation, certification and support for third-party hardware and software inside an institution that is designed for neutrality, impartiality and independence.

However, two complementary institutions could cover both sides of this coin.

So Ubuntu is the second half of a complete Debian-Ubuntu ecosystem. Debian’s strengths complement Ubuntu’s, Ubuntu can achieve things that Debian cannot (not because its members are not capable, but because the institution has chosen other priorities) and conversely, Debian delivers things which Ubuntu cannot, not because its members are not capable, but because it chooses other priorities as an institution.

Many people are starting to understand this: Ubuntu is Debian’s arrow, Debian is Ubuntu’s bow. Neither instrument is particularly useful on its own, except in a museum of anthropology 😉

“Ubuntu is Debian’s arrow, Debian is Ubuntu’s bow.”

So the worst and most frustrating attitude comes from those who think Debian and Ubuntu compete. If you care about Debian, and want it to compete on every level with Ubuntu, you are going to be rather miserable; you will want Debian to lose some of its best qualities and change some of its most important practices. However, if you see the Ubuntu-Debian ecosystem as a coherent whole, you will celebrate the strengths and accomplishments of both, and more importantly, work to make Debian a better Debian and Ubuntu a better Ubuntu, as opposed to wishing Ubuntu was more like Debian and vice versa.

Raphael: The Ubuntu-Debian relationship was rather hectic at the start, it took several years to “mature”. If you had to start over, would you do some things differently?

Mark: Yes, there are lessons learned, but none of them are fundamental. Some of the tension was based on human factors that cannot really be altered: some of the harshest DD critics of Canonical and Ubuntu are folk who applied for but were not selected for positions at Canonical. I can’t change that, and wouldn’t change that, and would understand the consequences are, emotionally, what they are.

Nevertheless, it would have been good to be wiser about the way people would react to some approaches. We famously went to DebConf 5 in Porto Allegre and hacked in a room at the conference. It had an open door, and many people popped a head in, but I think the not-a-cabal collection of people in there was intimidating and the story became one of exclusion. If we’d wanted to be exclusive, we would have gone somewhere else! So I would have worked harder to make that clear at the time if I’d known how many times that story would be used to paint Canonical in a bad light.

As for engagement with Debian, I think the situation is one of highs and lows. As a high, it is generally possible to collaborate with any given maintainer in Debian on a problem in which there is mutual interest. There are exceptions, but those exceptions are as problematic within Debian as between Debian and outsiders. As a low, it is impossible to collaborate with Debian as an institution, because of the design of the institution.

“It is generally possible to collaborate with any given maintainer […] [but] it is impossible to collaborate with Debian as an institution.”

In order to collaborate, two parties must make and keep commitments. So while one Debian developer and one Ubuntu developer can make personal commitments to each other, Debian cannot make commitments to Ubuntu, because there is no person or body that can make such commitments on behalf of the institution, on any sort of agile basis. A GR is not agile ;-). I don’t say this as a critique of Debian; remember, I think Debian has made some very important choices, one of those is the complete independence of its developers, which means they are under no obligation to follow a decision made by anyone else.

It’s also important to understand the difference between collaboration and teamwork. When two people have exactly the same goal and produce the same output, that’s just teamwork. When two people have different goals and produce different product, but still find ways to improve one anothers product, that’s collaboration.

So in order to have great collaboration between Ubuntu and Debian, we need to start with mutual recognition of the value and importance of the differences in our approach. When someone criticises Ubuntu because it exists, or because it does not do things the same way as Debian, or because it does not structure every process with the primary goal of improving Debian, it’s sad. The differences between us are valuable: Ubuntu can take Debian places Debian cannot go, and Debian’s debianness brings a whole raft of goodness for Ubuntu.

Raphael: What’s the biggest problem of Debian?

Mark: Internal tension about the vision and goals of Debian make it difficult to create a harmonious environment, which is compounded by an unwillingness to censure destructive behaviour.

Does Debian measure its success by the number of installs? The number of maintainers? The number of flamewars? The number of packages? The number of messages to mailing lists? The quality of Debian Policy? The quality of packages? The “freshness” of packages? The length and quality of maintenance of releases? The frequency or infrequency of releases? The breadth of derivatives?

Many of these metrics are in direct tension with one another; as a consequence, the fact that different DD’s prioritise all of these (and other goals) differently makes for… interesting debate. The sort of debate that goes on and on because there is no way to choose between the goals when everyone has different ones. You know the sort of debate I mean 🙂

Raphael: Do you think that the Debian community improved in the last 7 years? If yes, do you think that the coopetition with Ubuntu partly explains it?

Mark: Yes, I think some of the areas that concern me have improved. Much of this is to do with time giving people the opportunity to consider a thought from different perspectives, perhaps with the benefit of maturity. Time also allows ideas to flow and and of course introduces new people into the mix. There are plenty of DD’s now who became DD’s after Ubuntu existed, so it’s not as if this new supernova has suddenly gone off in their galactic neighbourhood. And many of them became DD’s because of Ubuntu. So at least from the perspective of the Ubuntu-Debian relationship, things are much healthier.

We could do much better. Now that we are on track for four consecutive Ubuntu LTS releases, on a two-year cadence, it’s clear we could collaborate beautifully if we shared a freeze date. Canonical offered to help with Squeeze on that basis, but institutional commitment phobia reared its head and scotched it. And with the proposal to put Debian’s first planned freeze exactly in the middle of Ubuntu’s LTS cycle, our alignment in interests will be at a minimum, not a maximum. Pure <facepalm />.

Raphael: What would you suggest to people (like me) who do not feel like joining Canonical and would like to be paid to work on improving Debian?

Mark: We share the problem; I would like to be paid to work on improving Ubuntu, but that’s also a long term dream 😉

Raphael: What about using the earnings of the dormant Ubuntu Foundation to fund some Debian projects?

Mark: The Foundation is there in the event of Canonical’s failure to ensure that commitments, like LTS maintenance, are met. It will hopefully be dormant for good 😉

Raphael: The crowdfunding campaign for the Debian Administrator’s Handbook is still going on and I briefly envisioned the possibility to create the Ubuntu Administrator’s Handbook. What do you think of this project?

Mark: Crowdfunding is a great match for free software and open content, so I hope this works out very well for you. I also think you’d find a bigger market for an Ubuntu book, not because Ubuntu is any more important than Debian but because it is likely to appeal to people who are more inclined to buy or download a book than to dive into the source.

Again, this is about understanding the difference in audiences, not judging the projects or the products.

Raphael: Is there someone in Debian that you admire for their contributions?

Mark: Zack is the best DPL since 1995; it’s an impossible job which he handles with grace and distinction. I hope praise from me doesn’t tarnish his reputation in the project!


Thank you to Mark for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook

.

20 Things to Learn About APT With the Free Chapter of the Debian Administrator’s Handbook

We just released a sample chapter of the Debian Administrator’s Handbook. It covers the APT family of tools: apt-get, aptitude, synaptic, update-manager, etc.


Click here to get your free sample chapter

I’m sure you will enjoy it. There are many interesting things to learn:

  • How to customize the sources.list file
  • The various APT repositories that Debian offers (Security Updates, Stable Updates, Proposed Updates, Backports, Experimental, etc.)
  • How to select the best Debian mirror for you
  • How to find old package versions
  • How to install the same selection of packages on multiple computers
  • How to install and remove a package on a single command-line
  • How to reinstall packages and how to install a specific version of a package
  • How to pass options to dpkg via APT
  • How to configure a proxy for APT
  • How to set priorities to various package sources (APT pinning)
  • How to safely mix packages from several distributions on a single system
  • How to use aptitude’s text-mode graphical interface
  • How to use the tracking of automatically installed packages to keep a clean system
  • How APT checks the authenticity of packages that it downloads
  • How to add supplementary GnuPG keys to APT’s trusted keyring
  • How to upgrade from one stable distribution to the next
  • How to handles problems after an upgrade
  • How to keep a system up-to-date
  • How to automate upgrades
  • How to find the package that you’re looking for

If you liked this chapter, click here to contribute a few euros towards the liberation of the whole book. That way you’ll get a copy of the ebook as soon as it’s available. Thank you!

I also invite you to share this sample chapter as widely as possible. We’re only at 40% of the liberation fund and there’s less than 2 weeks left. I hope this book extract will convince enough people that the book is going to be great, and that it really deserves to be liberated and bundled with Debian!

What about creating The Ubuntu Administrator’s Handbook?

I am currently running a crowdfunding campaign whose ultimate goal is to liberate the English translation of a French book that I have written. This book will be named The Debian Administrator’s Handbook because it has primarily been written for Debian.

Creating a new Ubuntu book based on The Debian Administrator’s Handbook

But since Ubuntu is based on Debian, a large part of its content applies equally well to Ubuntu. While discussing with Mark Shuttleworth, he suggested me to reuse those parts and to create a new book dedicated to Ubuntu. It would also cover the latest cloud technologies that Ubuntu has been delivering (since this is a topic that the current book does not cover).

This is something that I have been envisioning for a while and something that I would be ready to try if we manage to complete the liberation of the current book. This project would then bring a truly free book to the Ubuntu ecosystem.

Why? The official Ubuntu books are not really free

There’s a policy in place that ensures that official Ubuntu books use a free software/culture license and they are effectively available under the terms of a Creative Commons Share Alike license. But try to create a derivative book… you won’t find the “sources” (LaTeX or DocBook usually with most big books). You can only find a few PDF copies if you google for it. But this is really not the preferred form of modification for such a book.

Those books are also not packaged. Ubuntu much like Debian deserves to have a good book embodying the values of free software that can be shipped together with its product.

When I speak of liberation of the book, I really mean it in the way that free software hackers are used to: a public Git repository containing the DocBook sources, the pictures and the .dia files for the various schemas.

Help Ubuntu by spreading the word

I understand that at this point this proposed Ubuntu book is really hypothetical (“vaporware” one could say) but we need to go step by step to make it a reality. And the first step is to ensure that we manage to liberate the Debian Administrator’s Handbook.

For this I am seeking the support of the Ubuntu community to promote the current fundraising campaign. If the perspective of the Ubuntu book is not enough to convince you, you’ll be glad to learn that I also commit to give back to Ubuntu 15% of the money raised via the link below (once VAT has been subtracted).

Click here to go to the crowdfunding campaign page and pledge a few euros. Then share this article (or the link https://debian-handbook.info/go/ulule-ubuntu/) and convince others to participate.

At this point, the liberation target is entirely reachable with your help and the help of the community: the remaining 18 K€ needed in the liberation fund represent 720 persons giving 25 EUR each or 1800 persons giving 10 EUR each.

Thank you very much for your support and your help in this project!