My Free Software Activities in October 2012

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (120.46 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

At the start of the month, I reconfigured dpkg’s git repository to use KGB instead of the discontinued CIA to send out commit notices to IRC (on #debian-dpkg on OFTC, aka irc.debian.org).

I didn’t do anything else that affects dpkg and I must say that Guillem does not make it easy for others to get involved. He keeps all his work hidden in his private “for 1.17.x” branch and refuses to open an official “jessie” branch as can be seen from the lack of answer to this mail.

On the bright side, he deals with almost all incoming bugs even before I have a chance to take care of them. But it’s a pity that I can never review any of his fixes because they are usually pushed shortly before an upload.

Misc packaging

I helped to get #689336 fixed so that the initrd properly setups the keymap before asking for a passphrase for an encrypted partition. Related to this I filed #689722 so that cryptsetup gains a dependency ensuring that the required tools for keymap setup are available.

I packaged a new upstream version of zim (0.57) and also a security update for python-django that affected both Squeeze and Wheezy. I uploaded an NMU of revelation (0.4.13-1.2) so that it doesn’t get dropped from Wheezy (it was on the release team list of leaf packages that would be removed if unfixed) since my wife is using it to store her passwords.

I sponsored a new upstream version of ledgersmb.

Debian France

We managed to elect new officers for Debian France. I’m taking over the role of president, Sylveste Ledru is the new treasurer and Julien Danjou is the new secretary. Thank you very much to the former officers: Carl Chenet, Aurélien Jarno and Julien Cristau.

We’re in the process of managing this transition which will be completed during the next mini-Debconf in Paris so that we can exchange some papers and the like.

In the first tasks that I have set myself, there’s recruiting two new members for the boards of directors since we’re only 7 and there are 9 seats. I made a call for volunteers and we have two volunteers. If you want to get involved and help Debian France, please candidate by answering that message as soon as possible.

The Debian Handbook

I merged the translations contributed on debian.weblate.org (which led me to file this wishlist bug on Weblate itself) and I fixed a number of small issues that had been reported. I made an upload to Debian to incorporate all those fixes…

But this is still the book covering Squeeze so I started to plan the work to update it for Wheezy and with Roland we have decided who is going to take care of updating each chapter.

Librement

Progress is annoyingly slow on this project. Handling money for others is highly regulated, at least in the EU apparently. I only wanted an escrow account to secure the money of users of the service but opening this account requires either to be certified as a “payment institution” by the Autorité de contrôle prudentiel or to get an exemption from the same authority (covering only some special cases) or to sign a partnership with an established payment institution.

Being certified is out of scope for now since it requires a minimum of 125000 EUR in capital (which I don’t have). My bank can’t sign the kind of partnership that I would need. So I have to investigate whether I can make it fit in the limited cases of exemption or I need to find another “payment institution” that is willing to work with me.

Gittip uses Balanced a payment service specialized in market places but unfortunately it’s US-only if you want to withdraw money from the system. I would love a similar service in Europe…

If I can’t position Librement as a market place for the free software world (and save each contributor the hassle to open a merchant account), then I shall fallback to the solution where Librement only provides the infrastructure but no account, and developers who want to collect donations will have to use either Paypal or any other supported merchant account to collect funds.

That’s why my latest spec updates concerning the donation service and the payment service mentions Paypal and the possibility of choosing your payment service for your donation form.

Thanks

See you next month for a new summary of my activities.

My Debian Activities in September 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (1086.48 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

I am subscribed to Launchpad’s dpkg bug tracker and I was getting annoyed with the amount of noise I got under the form of bug reports that look like “package foo failed to install/upgrade: package foo is already installed and configured”. Those reports are a combination of a bug in APT and of random other failures (often hardware related like corrupted .deb files, or I/O errors, but sometimes also real problems in other packages) but they always end up assigned on dpkg (because dpkg is outputting an error message complaining about APT’s decision to configure something that doesn’t have to be configured).

I simply don’t have the time required to manually process and inspect all those reports, so I decided to filter them at the apport level with a new “Ubuntu bug pattern” that indicates that those reports are a duplicate of LP#541595. Thanks to this, the dpkg bug count quickly went down from 130 to about 80.

Packaging

I sponsored a new upstream version of ledgersmb. I quickly updated WordPress to version 3.4.2 since it contains security relevant fixes.

I also pushed a small update of nautilus-dropbox fixing #686863 because upstream renamed the binary package that they hand out on their website from nautilus-dropbox to dropbox. Their dropbox package only conflicts with old versions of nautilus-dropbox and not with the version that Debian is shipping and thus I had to add a Conflicts on our side to forbid co-installation of both packages.

Testing wheezy’s installation

I bought a new laptop (Lenovo Thinkpad X230) and used this as an excuse to test Wheezy’s installation process. It worked mostly fine except for two things:

  1. First I noticed that it would not accept my passphrase for my encrypted partition during early boot… this turned out to be already reported as #619711 but was no longer getting any attention from the package maintainer. After some IRC discussion with Julien Cristau, we prodded Michael Prokop who had apparently already offered to take care of this issue. I tested his updated package and the result got quickly uploaded.
  2. I had weird networking problems that turned out to be related to the lack of the loopback network (i.e. on localhost). This was the result of a broken /etc/network/interfaces: it had been incorrectly modified by NetworkManager. I reported this in #688355. This issue affects people with IPv6 enabled networks.

Debian France

There’s a resurgence of activity in Debian France. Sylvestre Ledru is leading the organization of a mini-debconf in Paris on November 24-25th. And Tanguy Ortolo is now taking care of some merchandising (Polo shirts, to change from the usual T-Shirt).

I might give a talk during this mini-debconf, possibly about multi-arch.

Misc

It’s been a few months that I noticed a 2 second lag of gnome-shell everytime that smuxi (my IRC client) sent a notification. It’s very annoying, you have the impression that the entire machine freezes.

So I contacted Mirco Bauer on #smuxi and we investigated a bit. It turns out that smuxi is using an old version of the notification protocol where the picture is sent as a bytestream leading to huge dbus messages. This is clearly sub-optimal so smuxi will be fixed to be able to send the path of the picture instead of the picture itself. On the other hand, it’s really a bug of gnome-shell that it freezes during the time it takes to handle the bigger-than-usual dbus message. So I also filed a bug on GNOME Shell (Bugzilla #683829) to get this fixed.

Librement: funding free software work

I started a new project with the goal of helping free software developers to fund their free software work. It’s still mostly vaporware for now but I have a public code repository, a nice logo and lots of ideas.

If the topic is of interest to you, and you’d like to be involved, feel free to get in touch. Otherwise stay tuned.

Thanks

See you next month for a new summary of my activities.

My Debian Activities in August 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (88.41 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

This month has again been a short one since I have mostly been in vacation during the last 2 weeks.

Dpkg

Things are relatively quiet during the freeze. I only took care of fixing 3 bugs: a regression of “3.0 (quilt)” (#683547), a segfault of “dpkg-query -W -f ”” (commit) and a bad auto-completion for French users (#685863).

Testing the upgrade to wheezy

We got several reports of wheezy upgrade that failed because dpkg ran the trigger while the dependencies of the package with pending triggers are not satisfied. Unfortunately fixing this in dpkg is not without problems (see #671711 for details)… so Guillem decided to defer this fix for Jessie. My suggestion of an intermediary solution has fallen in limbo. Instead we now have to find solutions for each case where this can fail (example of failure: 680626).

Another way to avoid those errors is to ensure that triggers are run as late as possible. We can improve this in multiple ways.

The first way is to modify most triggers so that they use the “interest-noawait” directive. In that case, the packages activating the trigger will be immediately marked as configured (instead of “triggers-awaited”) and the trigger will thus not need to be run as part of further dependency solving logic. But as of today, there’s no package using this new feature yet despite a nudge on debian-devel-announce. :-(

The second way is to modify APT to use dpkg –no-triggers, and to let the trigger processing for the end (with a last “dpkg –configure -a” call). I requested this early in the wheezy timeframe but for various reasons, the APT maintainers did not act on it. I pinged them again in #626599 but it’s now too late for wheezy. I find this a bit sad because I have been using those options for the entire wheezy cycle and it worked fine for me (and I used them for a dist-upgrade on my wife’s laptop too).

It would have been good to have all this in place for wheezy so that we don’t have to suffer from the same problems during the jessie upgrade, but unless someone steps up to steer those changes, it seems unlikely to happen.

Instead, we’re back to finding klumsy work-arounds in individual packages.

Packaging

I prepared security updates for python-django (1.4.1 for unstable,
1.2.3-3+squeeze3 for stable). I packaged a new upstream version for cpputest (3.2-1). I reviewed ledgersmb 1.3.21-1 prepared by Robert James Clay and asked him to prepare another version with further fixes.

I released nautilus-dropbox 1.4.0-2 with supplementary changes of my own to support https_proxy and to display better diagnostic information when the download fails.

With the help of Paul van der Vlis and Michael Ziegler, we did what was required to be able to migrate python-django-registration 0.8 to Wheezy even though it’s a new upstream version with backwards incompatible changes. Thanks to Adam D. Barratt who unblocked the package, we now have the right version in Wheezy despite the fact that I missed the freeze deadline.

Debian France

Julien Cristau reminded the board of Debian France that we have to elect officers (President, Secretary, Treasurer) as the current officers have withdrawn. I was somewhat afraid that nobody would take over so I pinged each member to try to get new volunteers. We now have volunteers (me, Julien Danjou and Sylvestre Ledru) and we’re waiting until Julien finds some time to run the election.

Misc

With the help of DSA, I setup antispam rules for the owner@packages.qa.debian.org alias because I was getting tired by the amount of spam. In the process, they asked me to write a wiki page for dsa.debian.org to document everything so that they can refer to it for future queries. I did it but it looks like that they did not apply my patch yet.

I also tested an upstream patch for gnome-keyring (see bugzilla #681081) that reintroduces the support of forgetting GPG passphrases after a specified amount of time.

Thanks

See you next month for a new summary of my activities.

Looking back at 16 years of dpkg history with some figures

With Debian’s 19th anniversary approaching, I thought it would be nice to look back at dpkg’s history. After all, it’s one of the key components of any Debian system.

The figures in this article are all based on dpkg’s git repository (as of today, commit 9a06920). While the git repository doesn’t have all the history, we tried to integrate as much as possible when we created it in 2007. We have data going back to April 1996…

In this period between April 1996 and August 2012:

  • 146 persons contributed to dpkg (result of git log --pretty='%aN'|sort -u|wc -l)
  • 6948 commits have been made (result of git log --oneline | wc -l)
  • 3133612 lines have been written/modified (result of git log --stat|perl -ne 'END { print $c } $c += $1 if /(\d+) insertions/;')

Currently the dpkg source tree contains 28303 lines of C, 14956 lines of Perl and 6984 lines of shell (figures generated by David A. Wheeler’s ‘SLOCCount’) and is translated in 40 languages (but very few languages managed to translate everything, with all the manual pages there are 3997 strings to translate).

The top 5 contributors of all times (in number of commits) is the following (result of git log --pretty='%aN'|sort| uniq -c|sort -k1 -n -r|head -n 5):

  1. Guillem Jover with 2663 commits
  2. Raphaël Hertzog with 993 commits
  3. Wichert Akkerman with 682 commits
  4. Christian Perrier with 368 commits
  5. Adam Heath with 342 commits

I would like to point out that those statistics are not entirely representative as people like Ian Jackson (the original author of dpkg’s C reimplementation) or Scott James Remnant were important contributors in parts of the history that were recreated by importing tarballs. Each tarball counts for a single commit but usually bundles much more than one change. Also each contributor has its own habits in terms of crafting a work in multiple commits.

Last but not least, I have generated this 3 minutes gource visualization of dpkg git’s history (I used Planet’s head pictures for dpkg maintainers where I could find it).

Watching this video made me realize that I have been contributing to dpkg for 5 years already. I’m looking forward to the next 5 years :-)

And what about you? You could be the 147th contributor… see this wiki page to learn more about the team and to start contributing.

My Debian Activities in July 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (72.65 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

This month has been a short one since I have been away for 2 weeks of vacation.

Dpkg

My dpkg work encompasses a bunch of small tasks:

  • I uploaded dpkg 1.16.7 with an important bugfix for a regression.
  • I initiated a new round of discussions about how we were going to solve the problem that source packages with “Multi-Arch: same” binary packages can’t be individually bin-nmued.
  • Following that discussion, I opened a bunch of bugs to plan/discuss the transition of changelog/copyright files within the package metadata (#681289 on debian-policy, #681293 on apt-listchanges, #681295 on www.debian.org for packages.debian.org).
  • I also filed #681292 on sbuild to get it to use dpkg’s new syntax for bin-nmus. It will allow us to do binary-only rebuilds with arbitrary versions (instead of only “+b1″ suffixes). Ubuntu could use this for their +rebuild1, we could use this to build backports which do not require source changes (and share the common source package instead of duplicating it). It can also be useful if we ever get to the situation where transitions are prepared in external repositories and where we want bin-nmus in those repositories to have unique versions (even though the same package might be bin-nmued in multiple repositories in case of concurrent transitions).
  • I filed an unblock request for dpkg once it was almost 10 days old.
  • I did reconsider the bug #316521 where dpkg looses track of some shared directories with manually created files and proposed an updated patch. No words from Guillem on the patch yet. Fixing this would help to fix a bunch of piuparts issues.
  • And just before my vacation, I filed many bugs against dpkg itself, effectively moving some of the items that accumulated in my TODO in a public place where others can help (I’ll be happy to mentor anyone who wants to tackle one of these):
    • #681443: dpkg-source –commit should be able to merge changes in an existing patch
    • #681470: dpkg-shlibdeps: should also scan Build-Depends-Arch for minimal versions
    • #681474: Dpkg::Vendor: should support /etc/os-release and /etc/os-release.d/*
    • #681477: dpkg-vendor: implement –select-closest command
    • #681480: base-files: Provide HOME_URL, SUPPORT_URL and BUG_REPORT_URL in /etc/os-release
    • #681489: base-files: Add /etc/os-release.d/debian and make it easy to provide supplementary /etc/os-release.d/* files
  • In #595112 we discussed the specifics of a new dpkg-mainstscript-helper feature to move a conffile from one package to another.

Packaging

I updated nautilus-dropbox to version 1.4.0 and python-django-registration to version 0.8. Both have been uploaded to unstable and I initially wanted to request an unblock for the latter, but it turns out it has gained reverse dependencies and version 0.8 introduces API changes so it’s not an option at this point of the freeze.

QA work

I investigated and fixed #678356 where it had been reported that the PTS static news were no longer working as expected.

At the start of the month, I also unblocked the mostly-unknown but important “mole” service… it was out of date of several weeks and several people were annoyed that the information about new upstream versions was no longer up-to-date.

Vacation

Almost no Debian work during my vacation but the lack of Wifi nearby made me look for solutions to connect my computer through my Nokia N900 3G/GPRS connection. I discovered the “Mobile Hotspot” application (homepage) and it worked like a charm (although it required Maemo’s non-default devel repository to be able to install the alternative kernel “for power users”).

The Debian Handbook

Michal Čihař proposed us to host a Weblate instance to help translate the book with a web interface. He kindly agreed to implement some improvements to better suit my requirements. Those have been completed and the weblate instance is now live at debian.weblate.org.

There’s no requirement to use Weblate for translations teams but for those that do, it sure makes it easier to recruit volunteers who have no prior knowledge of Git and PO files. If you want to help, please checkout this page first though, you should not start using Weblate without getting in touch with the respective translations teams.

Apart from translations, I also had the pleasure to merge some patches from Philipp Kern who improved the section covering IPv6 and a few other parts. We can make the book even better if more people share their expertise in the part of the book where they know better than me and Roland. :-)

Thanks

See you next month for a new summary of my activities.

My Debian Activities in June 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (168.12 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

This month, I resumed my work on dpkg. I concentrated my efforts on some “polishing” of the “3.0 (quilt)” format. With the latest version (1.16.6 — which was uploaded to unstable shortly before the freeze), dpkg-source restores the source tree in a clean state after a failed patch application (#652970), doesn’t overwrite the patch header from the pre-existing automatic patch, updates automatically debian/source/include-binaries during dpkg-source –commit, and supports a new –no-unapply-patches option for those who dislike the auto-unapplication at the end of the process when the patches were not applied at the start.

I wanted to go further and offer a new feature that could insert the automatic patch at the bottom of the quilt series but I have been short on time to complete this feature. I just managed to factorize all the quilt handling in a dedicated Perl module (Dpkg::Source::Quilt) to have cleaner code in the module handling the source format (Dpkg::Source::Package::V3::quilt).

For those who wonder, this feature is meant primarily for the X Strike Force team which maintains packages in Git and are doings lots of upstream cherry-picks (to fix regressions, etc.). But they also use quilt on top of that tree to keep some lasting Debian specific changes. With the 1.0 format, the “automatic diff” is a bit messy but at least it gets smaller automatically when a new upstream release gets out, there’s nothing to clean out. I’d like them to be able to use “3.0 (quilt)” while keeping their workflow. I’m leaning towards allowing “--auto-commit=first:cherry-picks” that would name the automatic patch “cherry-picks” and put it in the first position in the quilt series. (Opinions welcome on that feature, BTW)

Packaging

There’s been quite some packaging in this last month before the freeze:

  • I packaged CppUTest (a test framework for C/C++), and I wrote an article about it.
  • I prepared a stable update of Publican to fix a missing dependency. I also updated the unstable version to include a backport of a fix that some user requested me to include.
  • I updated dh-linktree to improve its documentation (following a discussion that happened on debian-devel) and to deal properly with trailing slashes in its input (#673408).
  • I sponsored dblatex 0.3.4-1 and ledgersmb 1.3.18-1.
  • I updated gnome-shell-timer to a new upstream snapshot that was tagged as compatible with GNOME 3.4 (#6776516).
  • I packaged wordpress 3.4 and spent a whole day triaging the old bugs that accumulated. A few days later I developed a new infrastructure to properly manage plugins/themes/language files. The canonical directory where the user is expected to drop his custom plugins/themes is now in /var/lib/wordpress/wp-content/ and the official plugins/themes are “installed” there with symlinks pointing back to /usr/share/wordpress/wp-content/ where they actually reside.
  • I wanted to commit 2 patches for the developers-reference but then I noticed that some translations were complete and were waiting for an upload. So I cleaned the packaging (switch to dh) and I uploaded version 3.4.8 before committing the patches for #678710 and #678712.

While doing all this packaging work, I found 2 possible improvements that I filed as bug reports:

  • #676606: debcommit should be able to identify alone that a new release is prepared (when the distribution field of the changelog changes from UNRELEASED to something else).
  • #679132: lintian outputs false positives for the tag package-uses-local-diversion when neither –local nor –package is given on the dpkg-divert command line.

Debian France Booth at Solutions Linux

From June 19th to June 21th, I manned the Debian France booth at Solutions Linux together with Carl Chenet, Tanguy Ortolo and other members of the association. We answered lots of questions, sold all t-shirts and umbrellas that Carl imported from Germany and Switzerland (we really need to get our own merchandising stuff produced in France!), got people to join the association. We also presented a printed copy of the Debian Administrator’s Handbook and of the corresponding French book.

You can see Carl, me and Tanguy on this picture (click on it to see a bigger picture, thanks to Sébastien Dubois of Evolix for this one!):

I know lots of people are preparing for Debconf but I decided to not attend this year, the price of the air plane ticket was a bit too hefty for me and it was also in partial conflict with our family vacations. I thought about attending the Libre Software Meeting instead but alas I won’t go there either (but Roland Mas will be there!), I have too much work to complete before my own vacation in 2 weeks.

Thanks

See you next month for a new summary of my activities.

Test Driven Development with CppUTest, now in Debian

I have recently read Test Driven Development with Embedded C by James W. Grenning and published by Pragmatic Programmers.

I really enjoyed the book: while I was aware of the huge benefits of having a comprehensive test suite, I never studied seriously the principles behind Test Driven Development (TDD) and this book makes a good introduction to the topic. At the same time it focuses on the C language and contains lots of examples on how you can create tests even for projects which have to interact with hardware or other unpredictable components (the key is to create many abstractions) using all the possibilities that C offers.

The author convincingly argues that developing code with TDD forces you to create a modular design that is easier to evolve when the underlying requirements change. He also highlights how the tests serve as reference documentation of the API.

James W. Grenning recommends CppUTest as his xUnit test framework of choice. When I wanted to try this test framework, I discovered that it was not available in Debian. I decided to package it because it has some interesting features not offered by the contenders (at least not to my knowledge). It’s now available in Debian and in Ubuntu.

First, it doesn’t require any explicit registration of tests and has a very lightweight syntax. The small downside is that CppUTest requires the usage of C++ for the tests. But C++ is compatible with C so it doesn’t matter much if you have a C++ compiler for your target. On the contrary, usage of variables and methods scoped to the test group makes it easy to write clear tests. Here’s a short sample of test code:

extern "C" {
#include "timer.h"
#include "timefn.h"
}
 
#include "CppUTest/TestHarness.h"
 
static Time the_time;
static const int start_sec = 123;
static const int start_nsec = 456789000;
static const int delay_sec = 8;
static const int delay_nsec = 111111000; // start_nsec + delay_nsec < 10^9
 
TEST_GROUP(Timer)
{
    /* Class variables available to all tests in the group */
    Timer timer;
    Delay remaining;
 
    /* Standard setup/teardown methods of xUnit tests */
    void setup() {
        timer = timer_new();
        time_set(&the_time, start_sec, start_nsec);
        /* [...] */
    }
 
    void teardown() {
        timer_free(timer);
        /* [...] */
    }
 
    /* Helper functions specific to the test group */
    void start_timer_with_delay(long sec, long nsec)
    {
        timer_set_real_delay(timer, sec, nsec);
        timer_start(timer);
    }
 
    void ensure_remaining_is(long sec, long nsec)
    {
        CHECK_EQUAL(sec, delay_get_seconds(remaining));
        CHECK_EQUAL(nsec, delay_get_nanoseconds(remaining));
    }
};
 
TEST(Timer, NewIsNotStarted)
{
    CHECK(!timer->started);
}
/* [...] */
TEST(Timer, GetRemainingTimeWithNanosecondPrecision_ShiftOfSeconds)
{
    start_timer_with_delay(delay_sec, delay_nsec);
    time_set(&the_time, start_sec + delay_sec - 5, start_nsec + delay_nsec + 1000);
 
    remaining = timer_get_remaining_time(timer);
 
    ensure_remaining_is(4, 999999000);
}

To run those tests, you just need this boilerplate code in a main.cpp:

#include "CppUTest/CommandLineTestRunner.h"
 
int main(int argc, char** argv)
{
   return CommandLineTestRunner::RunAllTests(argc, argv);
}

Another interesting feature is its integrated memory leak detection system. Any test that hasn’t released allocated memory at the end of the “teardown” process will be marked as failed.

The upstream developers have made some unusual choices (static library only, installation in a private directory) but this will likely change with the switch to an automake and autoconf-based build system. I have reported the oddities that I found and I requested them to provide a pkg-config file to make it easier to compile and link unit tests exploiting CppUTest.

I already used CppUTest to develop a small application running on an embedded Linux. At some point, I might try to use CppUTest for dpkg development. I believe that it makes for a good fit. dpkg is already C++ ready since dselect is written in C++ and reuses a good part of dpkg’s code.

In any case, if you like Test Driven Development and are writing C or C++ based applications, I invite you to try CppUTest.

My Debian Activities in May 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (338.26 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

Like last month, I did almost nothing concerning dpkg. This will probably change in June now that the book is out…

The only thing worth noting is that I have helped Carey Underwood who was trying to diagnose why btrfs was performing so badly when unpacking Debian packages (compared to ext4). Apparently this already resulted in some btrfs improvements.

But not as much as what could be hoped. The sync_file_range() calls that dpkg are doing only force the writeback of the underlying data and not of the meta-data. So the numerous fsync() that follow still create many journal transactions that would be better handled as one big transaction. As a proof of this, replacing the fsync() with a sync() brings the performance on par with ext4.

(Beware this is my own recollection of the discussion, while it should be close to the truth, it’s probably not 100% accurate when speaking of the brtfs behaviour)

Packaging

I uploaded new versions of smarty-gettext and smarty-validate because they were uninstallable after the removal of smarty. The whole history of smarty in Debian/Ubuntu has been a big FAIL since the start.

Once upon a time, there was a smarty package and some plugins. Everything was great except that the files were installed in a way that differs from the upstream recommendations. So Ubuntu changed the path in their version of the package and did not check whether it broke anything else (and it did break all the plugins). Despite the brokenness of the plugins, this divergence survived for years. So several packages that were using Smarty were modified to use dpkg-vendor to use the correct path depending on whether it was built on Debian or Ubuntu.

In 2010, Smarty 3.0 has been released and instead of upgrading the smarty package to this version, one of the smarty co-maintainers introduced a smarty3 package that used yet another path (despite the fact that smarty 3 had a mode to be compatible with smarty 2).
At some point, I informed him that he had to handle the migration of users of smarty to smarty3… he acknowledged and then lost interest in smarty (“I’m no longer using it”) and did nothing.

After some more bitrot, smarty has been forcefully orphaned in August 2011 by a member of the security team. And in March this year, it has been removed from unstable despite the fact that it still had reverse dependencies (usually removals only happen when they impact no other packages, I don’t know why this wasn’t the case here).

At least the brokenness attracted some attention to the situation and Mike Gabriel contacted me about it. I offered him to take over the various packages since they all needed a real maintainer and he accepted. I sponsored his uploads of all smarty related packages (bringing in the latest upstream versions at the same time).

In the end, the situation is looking better now, except that there’s no migration path from users who rely on smarty in Squeeze. They will discover that they need smarty3 in Wheezy and that the various paths have to be adjusted. It’s probably acceptable since the new upstream versions are no longer backwards compatible with smarty 2…

The Debian Administrator’s Handbook

At the start of the month, I was busy preparing the release of the book. I introduced the publican-debian package to unstable, it’s a Publican brand (aka a set of CSS and XSL stylesheets to tailor the output of Publican) using the Debian colors and using the Debian logo. This brand is used by the book.

I also created the debian-handbook package and setup the public Git repository on alioth.debian.org.

I was ready or so I thought. A few hours after the announce, the website became unusable because the numerous visitors were exhausting the maximum number of client connections. And I could not increase the limit due to Apache’s memory usage (with PHP and WordPress). We quickly off-loaded most of the static files traffic to another machine and we setup bittorrent. The problem was solved for the short term. Thousands of persons downloaded the ebook and to this date, 135 copies of the paperback have been sold.

Then I took a one-week vacation. Even though I had no Internet at the place I was, I wandered in the street to find a “Freewifi” wifi network (customers of the Free ISP can use those freely) to stay on top of incoming email. We quickly received some bug reports and I dealt with the easy ones (typos and the like) on the fly.

When I came back at home, I manually placed 54 lulu orders for the people who opted for the paperback as reward during the fundraising campaign. A bit tedious but it had to be done (if only Lulu supported a way to batch many orders at once…).

I also wanted a long term solution to avoid the use of an external host to serve static files (should a new traffic spike arrive…). So I installed nginx as a front-end. It serves static files directly, as well as WordPress pages which have been cached by wp-super-cache. Apache is still here listening on a local port and responding to the remaining queries forwarded by nginx. Once I’ll migrate to wheezy, I might completely ditch apache in favor of php5-fpm to handle the PHP pages.

Last but not least, I wanted to bootstrap the various translations that people offered to contribute. I wrote some documentation for interested translators and blogged about it. It’s shaping up nicely… check it out if you’re interested to help!

Thanks

See you next month for a new summary of my activities.

The Debian Administrator’s Handbook is available

The Debian Administrator's Handbook CoverI am so glad that we managed to complete this project. Roland and I have spent countless hours on this book since December, both for the translation itself and also for all the things that we tend to forget: a nice book cover, a great book layout for the print version, coordinating the work of reviewers, registering as an editor to get an ISBN, etc. I think I will come back to this in a future article because some parts of the story are interesting.

In the mean time, enjoy the DFSG-free Debian Administrator’s Handbook:

  • get it from unstable with apt-get install debian-handbook;
  • browse the online version;
  • get the paperback or the ebook (available as PDF, EPUB, MOBI);
  • grab the sources with git clone git://anonscm.debian.org/debian-handbook/debian-handbook.git and contribute a translation :-)

Check out the official announce (there’s a discount for early buyers of the paperback).

My Debian Activities in April 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (186.38 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg News

For the first time since several years, there has been a dpkg release (1.16.3) where the changelog doesn’t contain any entry of my own. The 3-4 commits I did were not really worthy of a changelog entry. I must admit that it was easy to put dpkg aside given Guillem’s message and given how busy I have been with my other projects.

Keeping a low-profile for a while certainly doesn’t hurt. But I don’t intend to stop contributing to dpkg. Quite on the contrary in fact, it’s something that I (usually) enjoy doing.

Packaging News

I packaged a new upstream release of SQL-Ledger (3.0.0) and later in the month I sponsored the upload of LedgerSMB, a fork of SQL-Ledger which — unlike the former — is maintained like a typical free software project.

I also uploaded version 0.56 of Zim and updated WordPress to version 3.3.2 with its slew of security fixes.

The Debian Administrator’s Handbook

Like several months now, most of my time has been directed towards the Debian Administrator’s Handbook. The big news is that the liberation fund has been completed… this means that the book will be published under DFSG-free licenses from the start (GPL-2+ / CC-BY-SA 3.0).

We hope to publish the book next week (i.e. between May 7th and May 11th). The PDF output for the printed book is almost ready. There’s some work left for the HTML/EPUB output and we have to prepare for the release of the sources as well…

Hopefully everything will work out like planned. Stay tuned!

Thanks

See you next month for a new summary of my activities.