Ben Hutchings is a rather unassuming guy… but hiding behind his hat, there’s a real kernel hacker who backports new drivers for the kernel in Debian stable so that our flagship release supports very recent hardware.

Read on to learn more about Ben and the kernel team’s projects for Debian Wheezy!

Raphael: Who are you?

Ben: I’m a professional programmer, living in Cambridge, England with my long-suffering wife Nattie. In Debian, I mostly work on the Linux kernel and related packages.

Raphael: How did you start contributing to Debian?

Ben: I started using Debian in 1998 and at some point I subscribed to Debian Weekly News. So in 2003 I heard about the planned Debian 10th birthday party in Cambridge, and thought I would like to go to that. Somehow I persuaded Nattie that we should go, even though it was on the day of our wedding anniversary! We both enjoyed it; we made new friends and met some old ones (small world). From then on we have
both been socially involved in Debian UK.

In 2004 there was a bug-squashing party in Cambridge, and we attended that as well. That’s where I really started contributing – fixing bugs and learning about Debian packaging. Then in 2005 I made my first package (sgt-puzzles), attended DebConf, and was persuaded to enter the New Maintainer process.

NM involved a lot of waiting, but by the time I was given questions and tasks to do I had learned enough to get through quite quickly. In April 2006 I was approved as a Debian Developer.

Meanwhile, I looked at the videos from DebConf 5 and thought that it would be useful to distribute them on a DVD. That led me to start writing video software and to get involved in the video team for the next year’s DebConf.

Raphael: You have been one the main driver behind the removal of non-free firmwares from the kernel. Explain us what you did and what’s the status nowadays?

Ben: That’s giving me a bit more credit than I deserve.

For a long time the easy way for drivers to load ‘firmware’ programs was to include them as a ‘blob’ in their static data, but more recently the kernel has included a simple method for drivers to request a named blob at run-time. These requests are normally handled by udev by reading from files on disk, although there is a build-time option to include blobs in the kernel. Several upstream and distribution developers worked to convert the older drivers to use this method. I converted the last few of these drivers that Debian included in its binary packages.

In the upstream Linux source, those blobs have not actually been removed; they have been moved to a ‘firmware’ subdirectory. The long-term plan is to remove this while still allowing the inclusion of blobs at build-time from the separate ‘linux-firmware’ repository. For now, the Debian source package excludes this subdirectory from the upstream tarball, so it is all free software.

There are still a few drivers that have not been converted, and in Debian we just exclude the firmware from them (so they cannot be built). And from time to time a driver will be added to the ‘staging’ section of Linux that includes firmware in the old way. But it’s understood in the kernel community that it’s one of the bugs that will have to be fixed before the driver can move out of ‘staging’.

Raphael: Do you believe that Debian has done enough to make it easy for users to install the non-free firmwares that they need?

Ben: The installer, the Linux binary packages and initramfs-tools will warn about specific files that may be needed but are missing. Users who have enabled the non-free section should then be able to find the necessary package with apt-cache search, because each of the
binaries built from the firmware-nonfree source package includes driver and file names within its description. For the installer, there is a single tarball that provides everything.

We could make this easier, but I think we have gone about as far as we can while following the Debian Social Contract and Debian policy.

Raphael: At some point in the past, the Debian kernel team was not working very well. Did the situation improve?

Ben: Back in 2008 when I started working on the Linux kernel package to sort out the firmware issues, I think there were some problems of communication and coordination, and quite possibly some members were burned-out.

Since then, many of the most active kernel team members have been able to meet face-to-face to discuss future plans at LPC 2009 in Portland and the 2010 mini-DebConf in Paris. We generally seem to have productive discussions on the debian-kernel mailing list and elsewhere, and I think the team is working quite well. Several new contributors have joined after me.

I would say our biggest problem today is that we just don’t have enough time to do all we want to. Certainly, almost all my Debian time is now taken up with integrating upstream kernel releases and handling some fraction of the incoming bug reports. Occasionally I can take the time to work on actual features or the other packages I’m neglecting!

“Our biggest problem today is that we just don’t have enough time to do all we want to.”

Raphael: It is widely known that Linux is maintained in a git repository. But the Debian kernel team is using Subversion. I believe a switch is planned. Why was not git used from the start?

Ben: The linux-2.6 source package dates from the time when Linus made his first release using git. I wasn’t part of the team back then so I don’t know for sure why it was imported to Subversion. However, at that time hardly anyone knew how to use git, no-one had experience hosting public git repositories, and Alioth certainly didn’t offer that option.

Today there are no real blockers: everyone on the kernel team is familiar with using git; Alioth is ready to host it; we don’t have per-architecture patches that would require large numbers of branches. But it still takes time to plan such a conversion for what is a relatively complex source package (actually a small set of related source packages).

Raphael: What are your plans for Debian Wheezy?

Ben: Something I’ve already done, in conjunction with the installer team, is to start generating udebs from the linux-2.6 source package. The kernel and modules have to be repacked into lots of little udebs to avoid using too much memory during installation. The configuration for this used to be in a bunch of separate source packages; these could get out of step with the kernel build configuration and this would only be noticed some time later. Now we can update them both at the same time, they are effectively cross-checked on every upload, and the installer can always be built from the latest kernel version in testing or unstable.

I think that we should be encouraging PC users to install the 64-bit build (amd64), but many users will still use 32-bit (i386) for backward compatibility or out of habit. On i386, we’ve slightly reduced the variety of kernel flavours by getting rid of ‘686’ and making ‘686-pae’ the default (previously this was called ‘686-bigmem’). This means that the NX security feature will be used on all systems that support it. It should also mean that the first i386 CD can have suitable kernel packages for all systems.

I have been trying to work on providing a full choice of Linux Security Modules (LSMs). Despite their name, they cannot be built as kernel modules, so every enabled LSM is a waste of memory on the systems that don’t use it. This is a significant concern for smaller Debian systems. My intent is to allow all unused LSMs to be freed at boot time so that we can happily enable all of them.

I recently proposed to drop support for older x86 systems, starting with 486-class processors in wheezy. In general, this would allow the use of more compiler optimisations throughout userland and the kernel. However it seems that there isn’t that much to be gained unless we also drop 586-class processors, and there are still quite a few of those in use. So I think this will have to wait.

Uwe Kleine-König has been working to include real-time support (also known as PREEMPT_RT). This can provide low and very predictable I/O latency, which is useful for live audio synthesis, for example. It still requires a number of patches and a build configuration change, resulting in a separate binary package. We’re currently only building that for 64-bit PCs. (You may notice this is missing for Linux 3.1, because the real-time developers skipped this release.)

Raphael: What’s the biggest problem of Debian?

Ben: I think we try too hard to accommodate every possible option, without regard for the cost to developers and users in general. As an example, we now have sysvinit, file-rc, upstart and systemd all in testing. Daemon maintainers can’t rely on any advanced features of upstart or systemd because we refuse to choose between them. And the decision to support the FreeBSD kernel means that we cannot choose upstart or systemd as the only option. So all daemon maintainers will have to maintain those baroque init scripts for the indefinite future. We really should be able to decide as a distribution that when one option is technically good and popular then it can be made the only option. But no-one really has the authority to do that, so we muddle along with the pretence that all the options are equally valid and functional, while none of them is supported as well as they should be.

“We try too hard to accommodate every possible option, without regard for the cost to developers and users in general.”

We also try to build every package on every architecture, in general. I’m quite sure there are many (package, architecture) combinations that have no users, ever. But if at some point that combination FTBFS, developers will waste time investigating and fixing that – time that could have been spent working on bugs and features that users actually care about. Yes, sure, portability is good but you can’t prove portability just by making a package compile on every architecture. This also applies to the selection of drivers for the kernel, by the way.

Raphael: Is there someone in Debian that you admire for their contributions?

Well, there are many people, but I will pick out just a few:

Steve McIntyre, for his work as DPL to improve communication with the various Debian derivatives and to bring fresh blood into various core teams. Also for being a generous host for countless Debian social and bug-squashing events.

Stefano Zacchiroli, for improving further on communications with both downstream and upstream projects, and for regularly exercising his power to lead discussions to the benefit of the project.

Julien Cristau, for maintaining good humour while not only fighting against the tide of graphics driver regressions in X and the Linux kernel but also working on release management.

Jonathan Nieder, for taking on the unglamorous and frustrating task of kernel bug triage as a non-maintainer and developing it to a fine art.

---

Thank you to Ben for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook

.