300 EUR for Debian

It’s the amount of the wire transfer that I made to ffis e.V., the German association that works together with Software in the Public Interest to hold Debian’s monetary assets.

This amount includes the 10% of the benefits of my Debian DVD sales (for a total of 47 USD =~ 36 EUR) and the 15% affiliate revenue that was granted to Debian by some supporters of the Debian Administrator’s Handbook crowdfunding campaign (for a total of 259.49 EUR).

Thank you everybody, this donation is really yours and not mine!

If you want to complement this donation, and make a nice gift to Debian for Christmas, go to this page and follow the instructions.

“3.0 (quilt)” is the most widely used Debian source package format

My goal with the “3.0 (quilt)” source format has always been to standardize the patch management in Debian source packages. This message seems to have been well understood. dbs and dpatch have been deprecated by their respective maintainers.

I made numerous efforts to make this source format useful in as many use cases as possible (but some improvements are still possible) and I have added hints to encourage maintainers to switch. Thanks to this, the adoption rate of this new source format has been very good and it’s now the most widely used source package format in Debian—only two years after its introduction in Debian unstable.

With 9829 source package using “3.0 (quilt)”, it surpassed the number of source package still using “1.0” (7368). (Those numbers have been taken from http://upsilon.cc/~zack/stuff/dpkg-v3/ on december 13th 2011.) The number of source packages using “3.0 (quilt)” doubled this year.

(Click on the picture to see it full size)

Of the 7368 packages using the old format, 6816 packages trigger the missing-debian-source-format lintian tag. This means that only 552 source packages have explicitly opted to keep using the old format and that the bulk of the remaining packages are rarely updated packages that have not been switched yet.

People behind Debian: Ben Hutchings, member of the kernel team

Ben Hutchings, photo by Andrew Mc Millan, license CC-BY-2.0
Ben Hutchings is a rather unassuming guy… but hiding behind his hat, there’s a real kernel hacker who backports new drivers for the kernel in Debian stable so that our flagship release supports very recent hardware.

Read on to learn more about Ben and the kernel team’s projects for Debian Wheezy!

Raphael: Who are you?

Ben: I’m a professional programmer, living in Cambridge, England with my long-suffering wife Nattie. In Debian, I mostly work on the Linux kernel and related packages.

Raphael: How did you start contributing to Debian?

Ben: I started using Debian in 1998 and at some point I subscribed to Debian Weekly News. So in 2003 I heard about the planned Debian 10th birthday party in Cambridge, and thought I would like to go to that. Somehow I persuaded Nattie that we should go, even though it was on the day of our wedding anniversary! We both enjoyed it; we made new friends and met some old ones (small world). From then on we have
both been socially involved in Debian UK.

In 2004 there was a bug-squashing party in Cambridge, and we attended that as well. That’s where I really started contributing – fixing bugs and learning about Debian packaging. Then in 2005 I made my first package (sgt-puzzles), attended DebConf, and was persuaded to enter the New Maintainer process.

NM involved a lot of waiting, but by the time I was given questions and tasks to do I had learned enough to get through quite quickly. In April 2006 I was approved as a Debian Developer.

Meanwhile, I looked at the videos from DebConf 5 and thought that it would be useful to distribute them on a DVD. That led me to start writing video software and to get involved in the video team for the next year’s DebConf.

Raphael: You have been one the main driver behind the removal of non-free firmwares from the kernel. Explain us what you did and what’s the status nowadays?

Ben: That’s giving me a bit more credit than I deserve.

For a long time the easy way for drivers to load ‘firmware’ programs was to include them as a ‘blob’ in their static data, but more recently the kernel has included a simple method for drivers to request a named blob at run-time. These requests are normally handled by udev by reading from files on disk, although there is a build-time option to include blobs in the kernel. Several upstream and distribution developers worked to convert the older drivers to use this method. I converted the last few of these drivers that Debian included in its binary packages.

In the upstream Linux source, those blobs have not actually been removed; they have been moved to a ‘firmware’ subdirectory. The long-term plan is to remove this while still allowing the inclusion of blobs at build-time from the separate ‘linux-firmware’ repository. For now, the Debian source package excludes this subdirectory from the upstream tarball, so it is all free software.

There are still a few drivers that have not been converted, and in Debian we just exclude the firmware from them (so they cannot be built). And from time to time a driver will be added to the ‘staging’ section of Linux that includes firmware in the old way. But it’s understood in the kernel community that it’s one of the bugs that will have to be fixed before the driver can move out of ‘staging’.

Raphael: Do you believe that Debian has done enough to make it easy for users to install the non-free firmwares that they need?

Ben: The installer, the Linux binary packages and initramfs-tools will warn about specific files that may be needed but are missing. Users who have enabled the non-free section should then be able to find the necessary package with apt-cache search, because each of the
binaries built from the firmware-nonfree source package includes driver and file names within its description. For the installer, there is a single tarball that provides everything.

We could make this easier, but I think we have gone about as far as we can while following the Debian Social Contract and Debian policy.

Raphael: At some point in the past, the Debian kernel team was not working very well. Did the situation improve?

Ben: Back in 2008 when I started working on the Linux kernel package to sort out the firmware issues, I think there were some problems of communication and coordination, and quite possibly some members were burned-out.

Since then, many of the most active kernel team members have been able to meet face-to-face to discuss future plans at LPC 2009 in Portland and the 2010 mini-DebConf in Paris. We generally seem to have productive discussions on the debian-kernel mailing list and elsewhere, and I think the team is working quite well. Several new contributors have joined after me.

I would say our biggest problem today is that we just don’t have enough time to do all we want to. Certainly, almost all my Debian time is now taken up with integrating upstream kernel releases and handling some fraction of the incoming bug reports. Occasionally I can take the time to work on actual features or the other packages I’m neglecting!

“Our biggest problem today is that we just don’t have enough time to do all we want to.”

Raphael: It is widely known that Linux is maintained in a git repository. But the Debian kernel team is using Subversion. I believe a switch is planned. Why was not git used from the start?

Ben: The linux-2.6 source package dates from the time when Linus made his first release using git. I wasn’t part of the team back then so I don’t know for sure why it was imported to Subversion. However, at that time hardly anyone knew how to use git, no-one had experience hosting public git repositories, and Alioth certainly didn’t offer that option.

Today there are no real blockers: everyone on the kernel team is familiar with using git; Alioth is ready to host it; we don’t have per-architecture patches that would require large numbers of branches. But it still takes time to plan such a conversion for what is a relatively complex source package (actually a small set of related source packages).

Raphael: What are your plans for Debian Wheezy?

Ben: Something I’ve already done, in conjunction with the installer team, is to start generating udebs from the linux-2.6 source package. The kernel and modules have to be repacked into lots of little udebs to avoid using too much memory during installation. The configuration for this used to be in a bunch of separate source packages; these could get out of step with the kernel build configuration and this would only be noticed some time later. Now we can update them both at the same time, they are effectively cross-checked on every upload, and the installer can always be built from the latest kernel version in testing or unstable.

I think that we should be encouraging PC users to install the 64-bit build (amd64), but many users will still use 32-bit (i386) for backward compatibility or out of habit. On i386, we’ve slightly reduced the variety of kernel flavours by getting rid of ‘686’ and making ‘686-pae’ the default (previously this was called ‘686-bigmem’). This means that the NX security feature will be used on all systems that support it. It should also mean that the first i386 CD can have suitable kernel packages for all systems.

I have been trying to work on providing a full choice of Linux Security Modules (LSMs). Despite their name, they cannot be built as kernel modules, so every enabled LSM is a waste of memory on the systems that don’t use it. This is a significant concern for smaller Debian systems. My intent is to allow all unused LSMs to be freed at boot time so that we can happily enable all of them.

I recently proposed to drop support for older x86 systems, starting with 486-class processors in wheezy. In general, this would allow the use of more compiler optimisations throughout userland and the kernel. However it seems that there isn’t that much to be gained unless we also drop 586-class processors, and there are still quite a few of those in use. So I think this will have to wait.

Uwe Kleine-König has been working to include real-time support (also known as PREEMPT_RT). This can provide low and very predictable I/O latency, which is useful for live audio synthesis, for example. It still requires a number of patches and a build configuration change, resulting in a separate binary package. We’re currently only building that for 64-bit PCs. (You may notice this is missing for Linux 3.1, because the real-time developers skipped this release.)

Raphael: What’s the biggest problem of Debian?

Ben: I think we try too hard to accommodate every possible option, without regard for the cost to developers and users in general. As an example, we now have sysvinit, file-rc, upstart and systemd all in testing. Daemon maintainers can’t rely on any advanced features of upstart or systemd because we refuse to choose between them. And the decision to support the FreeBSD kernel means that we cannot choose upstart or systemd as the only option. So all daemon maintainers will have to maintain those baroque init scripts for the indefinite future. We really should be able to decide as a distribution that when one option is technically good and popular then it can be made the only option. But no-one really has the authority to do that, so we muddle along with the pretence that all the options are equally valid and functional, while none of them is supported as well as they should be.

“We try too hard to accommodate every possible option, without regard for the cost to developers and users in general.”

We also try to build every package on every architecture, in general. I’m quite sure there are many (package, architecture) combinations that have no users, ever. But if at some point that combination FTBFS, developers will waste time investigating and fixing that – time that could have been spent working on bugs and features that users actually care about. Yes, sure, portability is good but you can’t prove portability just by making a package compile on every architecture. This also applies to the selection of drivers for the kernel, by the way.

Raphael: Is there someone in Debian that you admire for their contributions?

Well, there are many people, but I will pick out just a few:

Steve McIntyre, for his work as DPL to improve communication with the various Debian derivatives and to bring fresh blood into various core teams. Also for being a generous host for countless Debian social and bug-squashing events.

Stefano Zacchiroli, for improving further on communications with both downstream and upstream projects, and for regularly exercising his power to lead discussions to the benefit of the project.

Julien Cristau, for maintaining good humour while not only fighting against the tide of graphics driver regressions in X and the Linux kernel but also working on release management.

Jonathan Nieder, for taking on the unglamorous and frustrating task of kernel bug triage as a non-maintainer and developing it to a fine art.

Thank you to Ben for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook


My Debian Activities in November 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (310.73 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg: Multi-Arch Saga

I know lots of people are waiting the landing of multiarch in Debian unstable, and so am I. Things are progressing, though not as quickly as I hoped. Guillem merged about half of the branch between the 24th October and the 6th of November. After that most of the work happened on his personal repository in his pu/multiarch/master branch.

I verify this repository from time to time because Guillem does not inform me when he has made progress. I noticed changes on his repository on the 10th, 19th, 23th, 28th of November and on the 1th of December.

He announced a long time ago that he had some “interface changes” and up to now only wrote about the switch from the command-line option --foreign-architecture (to put in /etc/dpkg/dpkg.cfg) to the explicit command dpkg --add-architecture that only needs to be called once (see mail here). As of today (December 2th), the promised email for the other interface changes is still not here.

On November 23th, I reviewed Guillem’s work and tried to run the code in his branch. I spent the whole day chasing up regressions and submitted lots of fixes to Guillem. Thanks to the extensive test-suite I wrote when I developed my branch, it has been fairly easy to track them all down.

All the issues I reported have been fixed in the latest version of Guillem’s branch although the fixes are often slightly different from those that I submitted.

Dpkg: Squeeze Backport

At the start of the month, I uploaded what I expected to be a fairly uncontroversial backport of dpkg It turns out I was wrong.

After some discussion, I think we came to an agreement that it was acceptable to backport dpkg-dev and libdpkg-perl only. My goal was not to bring the latest dpkg to users but to make it easier for package maintainers to backport packages using new features provided by dpkg-dev >= 1.16 (such as hardening build flags, the makefile snippets provided in /usr/share/dpkg/, or the improved dpkg-buildflags interface).

Thus I modified the source package uploaded to squeeze-backports to build only dpkg-dev and libdpkg-perl. It has been uploaded on November 23th and it’s waiting in the NEW queue for a backports admin to process it.

Misc Dpkg Work

I merged a patch of Colin Watson to be able to verify build-dependencies for a foreign architecture (taking into account the Multi-Arch status of each package listed).

I released dpkg with two minor fixes that were sitting in the sid branch. I wanted to get rid of this so that the path is clear for a 1.16.2 upload with multiarch. The package just migrated to testing so we’re fine.

I spent another day doing dpkg bug triaging on Launchpad, we’re now down to 77 bugs with many of them tagged as incomplete and likely to expire in 2 months.

The Debian Administrator’s Handbook

eBookWe released a sample chapter so that it’s easier to have an idea of the quality of the book. The chapter covers the APT tools quite extensively. I bet that even you could learn something about apt-get/aptitude…

The crowfunding campaign on Ulule ended on November 28th.
With 673 supporters, we raised 24345 EUR. Of those, 14935 EUR have been put in the liberation fund and the rest corresponds to the various pre-orders and rewards offered.

This means that the translation will happen (we just started) but that the book is currently not going to be released under a free license. Don’t despair… As planned, the liberation campaign is carried on until the 25 K€ target is reached!

Instead of being hosted on Ulule, this permanent campaign is on the project website at debian-handbook.info/liberation/. Note that any contribution of 10 EUR or more means that you get a copy of the ebook as soon as it’s available (even if the liberation target is not reached).

Package Tracking System

At the start of the month, I filed two ideas of improvements for the PTS in the bug tracking system: #647258 is about showing outstanding bugs that relate to a release goal and #647901 is about warning maintainers that the package is affected by a current transition. If you’re a coder and want to start contributing to Debian and its QA team, those bugs could be interesting targets for a start. 🙂 In both cases, I have been in contact with members of the release team because those ideas require some structured data from the release team as input. Thanks to Meddi Dohguy and Niels Thykier for their help.

Later in the month, the topic of relocating the PTS once again came up. For historical reasons, the PTS was hosted on master.debian.org together with the BTS. Nowadays the BTS has its own host and it made no sense anymore to have the PTS separate from the rest of the QA services hosted on qa.debian.org (currently quantz.debian.org). So together with Martin Zobel Helas we took care to plan the migration and on November 19th we executed the plan. It worked like a charm and almost nobody noticed (only one undocumented dependency was missed, which broke the SOAP interface).

Misc Packaging Work

WordPress was broken in Ubuntu and it was also not properly synchronized with Debian due to an almost useless change on their side. Thus I requested a sync so that the working version from Debian gets imported in Ubuntu.

I sponsored the docbook-xsl 1.76.1 upload that I needed for Publican. Then I updated Publican just to discover that the test-suite triggers a new bug in fop (filed as #649476). I disabled the test-suite temporarily and uploaded Publican 2.8 to unstable. BTW, I also filed 2 upstream bugs with patches for issues I discovered while trying to generate the sample chapter of my book (see here and here).

I uploaded a version 0.7.1 of nautilus-dropbox and fixed #648215 at the same time. I made an NMU of bison to fix a long-standing release critical bug that hit me once more during an upgrade (see #645038).

I uploaded to experimental a new version of gnome-shell-timer compatible with GNOME 3.2. I took the opportunity to install from experimental the few GNOME 3.2 packages which are not yet in unstable…


See you next month for a new summary of my activities.