My Debian Activities in January 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (213.68 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

The “biggest change” I made is a small patch that brings to an end years and years of recurring discussions about the build-arch and build-indep targets of debian/rules (see #229357). Last year the technical committee took this issue in its hands (see #629385) but it failed to take any resolution. Fortunately thanks to this we got some concrete numbers on the colateral damages inflicted on the archive for each possible approach. In the end, Guillem and I managed to agree on the way forward.

The remaining of what I did as dpkg maintainer has not much to do with coding. I reviewed the work of Gianluca Ciccarelli on dpkg-maintscript-helper who is trying to provide helper functions to handle migration between directories and symlinks. I also reviewed a 2000-lines patch from Patrick Schoenfeld who’s trying to provide a perl API to parse dpkg log files and extract meaningful data out of them.

I updated the dpkg-architecture manual page to document the Makefile snippet /usr/share/dpkg/architecture.mk and to drop information that’s no longer releveant nowadays.

I reviewed a huge patch prepared by Russ Allbery to update the Debian policy and document the usage of symbols files for libraries. As the author of dpkg-gensymbols, I was keen to see it properly documented at the policy level.

I brought up for discussion a detail that was annoying me for quite some time: some copyright notices were embedded in translatable strings and updating them resulted in useless work for translators. In the end we decided to drop those notices and to keep them only at the source level.

I updated my multiarch branch on top of Guillem’s branch several times, all the fixes that were in my branch have been integrated (often in a modified form).

Unfortunately even if the code works quite well, Guillem doesn’t want to release anything to Debian until he has finished to review everything… and many people are annoyed by the unreasonable delay that it imposes. Cyril Brulebois tried to release a snapshot of the current multiarch branch to experimental but Guillem has been prompt to revert this upload.

I’m somewhat at a loss in this situation. I offered my help to Guillem multiple times but he keeps doing his work in private, he doesn’t share many details of his review except some comments in commit logs or when it affects the public interface. I complained once more of this sad situation.

Debian Package Maintenance Hub

That’s the codename I use for a new infrastructure that I would like to develop to replace the Package Tracking System and the DDPO and several other services. I started to draft a Debian Enhancement Proposal (DEP), see DEP-2, and requested some comments within the QA team.

For now, it looks like that nobody had major objections on the driving idea behind this project. Those who commented were rather enthusiastic. I will continue to improve this DEP within the QA team and at some point I will bring the discussion to a larger audience like debian-devel@lists.debian.org.

Package Tracking System

Even if I started to design its replacement, the PTS will still be used for quite some time so I implemented two new features that I deemed important: displaying a TODO notice when there is (at least) one open bug related to a release goal, displaying a notice when the package is involved in an ongoing or upcoming transition.

Misc packaging tasks

I created and uploaded the dh-linktree package which is a debhelper addon to create symlink trees (useful to replace embedded copies of PHP/JavaScript libraries by symlinks to packaged copies of those files).

I packaged quilt 0.50. I helped the upstream authors to merge a Debian patch that had been forwarded by Martin Quinson (a quilt’s co-maintainer). I packaged a security release of WordPress (3.3.1) and a new upstream release of feed2omb and gnome-shell-timer.

I prepared a new Debian release of python-django with a patch cherry-picked from the upstream SVN repository to fix the RC bug #655666.

Book update

We’re again making decent progress in the translation of the Debian Administrator’s Handbook, about 12 chapters are already translated.

The liberation campaign is also (slowly) going forward. We’re at 72% now (thanks to 63 new supporters!) while we were only at 67% at the start of January.

Thanks

See you next month for a new summary of my activities.

Contributing to the translation of Debian

If you’re not into packaging and if you asked how you could help Debian, someone probably suggested that you help to translate it.

It’s true that translating Debian is essential if we want to make Debian available to everybody on the world. There are many persons who are stuck as soon as they get a message in English, so it’s important to aim for 100% coverage in terms of localization.

Some vocabulary: localization vs internationalization

Internationalization (i18n) is the work that makes it possible to translate messages in a given application.

Localization (l10n) is the work of translating messages of said application. So as a translator, you’ll be doing “localization” but some knowledge of “internationalization” is still useful… because it will define how you’re supposed to provide the translations. We’ll come back to that later.

Join your localization team

Usually the translation work is shared among multiple translators within a localization team. Check out the Debian International page on www.debian.org to find out instructions for translators for each language.

Many teams have a debian-l10n-*@lists.debian.org mailing list used for coordination, feel free to ask questions on those lists when you start (but make sure that you have read the relevant documentation before).

Each team has its own workflow, so observe for a while to get used to what’s happening before asking your first questions.

What is there to translate?

The translation of most of the software provided by Debian is not handled by Debian. The Debian translation teams “only” handle the translation of:

Now before contributing to your first translation, I have to come back to internationalization to teach you a few things. In the above list, the projects marked with “(*)” do use PO files for their translation and the next sections will explain you how to work with those files.

Introduction to Gettext

The free software community has mostly standardized on a single internationalization infrastructure known as Gettext. With this tool, you’re provided a “POT file” which contains all the translatable strings. It looks like this:

# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR Software in the Public Interest, Inc.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR , YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: dpkg 1.16.1\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME \n"
"Language-Team: LANGUAGE \n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"

#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr ""

#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr ""

[…]

The lines starting with “#:” are comments that indicate the source files where the (English) string is used. This can be useful if you want check the source to have more information about how the string is used.

The lines starting with “#,” contain flags that can be important. If the “fuzzy” flag is set, the translated string is not used because it must be updated (or at least verified) since the original string evolved. The “c-format” flags indicates that the string must be a C format string, this has some implications in what’s allowed in the string (in particular when it embeds conversion specifier for arguments submitted to printf-like functions).

Another thing to note is that the translation of the empty string is used to store some meta-information about the translation itself.

Contributing a translation as a PO file

When you start a new translation, you copy that POT file to create a “PO file” for your own language (eg. fr.po for the French language). You replace some template values (identified with the upper case words in the POT file) and you replace all the empty strings on “msgstr” lines with the translation of the string that appears in the previous “msgid” line.

The result could be something like this:

# translation of fr.po to French
# Messages français pour dpkg (Linux-GNU Debian).
msgid ""
msgstr ""
"Project-Id-Version: fr\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: 2012-01-16 07:57+0100\n"
"Last-Translator: Christian Perrier \n"
"Language-Team: French \n"
"Language: fr\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: Plural-Forms: nplurals=2; plural=n>1;\n"
"X-Generator: Lokalize 1.2\n"

#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr "caractère invalide « %1$c » dans la taille du membre « %3$.16s » de l'archive « %2$.250s »"

#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr "impossible d'écrire le fichier « %s »"

[…]

If there’s already a “PO file” for your language, there might still work to do: there might be strings that have not yet been translated and there might be “fuzzy” strings which have to be updated — strings which were already translated but where the original string has been modified.

There are software that can assist you to edit PO files: poedit, virtaal, lokalize, gtranslator. There are also special extensions for vim (packaged in vim-scripts) and for Emacs.

Submit the translation for inclusion

Once you have a complete PO file, you should submit it for inclusion. Sometimes you will have been granted commit rights to the source code repository so that you can include your translation by yourself. In the other cases, you should submit your translation with a bug report tagged “l10n” and someone else will include your work in the next release.

Depending on the team, the workflow might require a review before the submission. In that case, you usually have to send a call for review on the coordination mailing list.

Go ahead!

Hopefully those explanations will be enough to get you started. There are many other things to learn¹ but it’s good to learn while practicing…

¹ For example, can you find out why the French translation above changed “%c” in “%1$c”?

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

People Behind Debian: Josselin Mouette, founder of the Debian GNOME team

Josselin Mouette is one the leaders of the pkg-gnome team, he takes sound technical decisions and doesn’t fear writing code to work-around upstream issues. He deserves kudos for the work he has put into packaging GNOME over the years. He can also be very sarcastic (sometimes he even enjoys participating to flamewars on debian lists), and there are quite a few topics where we have long agreed to disagree. But this kind of diversity is also what makes Debian a so interesting place…

Read on to learn more about the pkg-gnome team, its plans for Wheezy, Josselin’s opinion on the GNOME 3 switch, and much more.

Raphael: Who are you?

Josselin: I am a 31 years old Linux systems engineer. I started in life with physics, which I studied at the ENS Lyon. I started a thesis on experimental and numerical models for optoelectronics, but when it became clear that research was not for me, I abandoned it and accepted a job at the CEA, which holds the largest computing center in Europe. Working on these machines has been the most awesome job ever (except for it being near Paris). After that I worked a bit on system monitoring technologies.

I am married, currently living in Lyon, and working for EDF (the French historical electricity company) on scientific workstations using Debian. EDF is using Debian on more than a thousand workstations and holds the fastest Debian supercomputer in the world (200 Tflops), which makes it another obvious place for Debian developers.

Raphael: How did you start contributing to Debian?

Josselin: I discovered Debian in 1999 while studying at the ENS, which is one of the biggest nests of Debian developers – while being a small place, it is producing almost one Debian developer per year on average. After wondering for a while what it could be useful for, hacking on a slink snapshot made me think that it was for, well, everything except for gaming. Later, in 2002, when I was working on optoelectronics computing codes, I started to package them for Debian in order to make them easier to install, for us as well as other labs over the world. I started the NM process, and it was going smoothly but also going to take time. However, at that moment, the frozen-bubble game went out and made quite some buzz. Since I knew a guy who knew the game’s developer, he asked me to package it. The package found 3 sponsors in a very short time and was fast-tracked into the archive at a speed that was unseen before. After which the NM process was completed very quickly.

At that time, I was a heavy WindowMaker user, but I didn’t like the direction the project was taking (actually, I wonder if there was one). GNOME was starting to become attractive, but its packaging in Debian was very ineffective, with many inconsistent packages maintained by people who didn’t ever talk to each other – some of them didn’t speak English, and some of them didn’t talk at all. Together with awesome people, among which Jordi Mallach, Gustavo Noronha Silva, JHM Dassen, Ross Burton and Sébastien Bacher, we started the GNOME team in 2003, introducing consistent packaging practices, and initiating synchronized uploads. Releasing a completely integrated GNOME 2.8 in sarge was a considerable achievement; proving (together with the Perl team) that a team was the best way to maintain large package sets changed the way people work on Debian.

“Proving […] that a team was the best way to maintain large package sets changed the way people work on Debian.”

Raphael: You’re one of the most active contributors of the team which is packaging GNOME for Debian. What would you suggest to a new contributor who would like to help the team?

Josselin: There are several ways to contact the team, but the recommended one has always been IRC. We hang on #debian-gnome on the OFTC network, so just come around and ask for us.¹ The real question is what you want to do in the team. Of course, most new volunteers want to help packaging the latest and greatest version of GNOME into unstable as soon as possible, but unless they already have Debian background, this is not the easiest task. Since there are already people working on this, the “big” packages are usually waiting on dependencies.

I used to direct newcomers towards bug triage, but it is a tedious task and I’m now convinced that our huge bug backlog will never be dealt with. The most useful thing to do for newcomers now is probably to find a GNOME or GNOME-related package that needs improvement or is lagging behind, and simply try to work on it. You can also come and fix the bugs you find annoying. Find a patch on the GNOME bugzilla, or cook it yourself, propose it, and if it’s worthy enough you’ll soon get commit access.

“Our huge bug backlog will never be dealt with.”

¹ At this point I feel worth mentioning that if no one answers in 10 minutes, it doesn’t mean that no one will answer in 2 hours, so please stay on the channel after asking.

Raphael: There’s been some controversy about GNOME 3 and the direction that the project is taking. What’s your personal stance on GNOME 3? And what’s the position of the pkg-gnome team?

Josselin: The controversy is not new to GNOME 3, but the large-scale changes made with it have put it more prominently. The criticism usually boils down to a few categories:

  1. General lack of configurability
  2. Strange design decisions
  3. Red Hat centric development
  4. Hardware requirements
  5. Change resistance

The lack of configuration options has been an ongoing criticism since GNOME 2.0 has decided to rip off most of them. Of course, when the control center was redesigned again for 3.0, there was a surge of horrified exclamations from people who missed their favorite buttons. On this topic, I fully concur with GNOME developers. The configuration option that is useful for you is not necessarily useful for someone else. Of course, sometimes developers go a bit too far, but the general direction is right. At work, we found that only a minority of users actually configure anything on their desktops: they just want something that works to launch their applications. Apple and Google have sold millions of devices by making them the simplest possible and without any configuration.

Design decisions are, on the contrary, individual decisions, and each of them, while having reasons behind it, can be questioned. I remember seeing a lot of complaints when the OK and Cancel buttons were reversed in dialog boxes, something that nobody questions anymore. GNOME Shell is full of such changes; some are easy to get accustomed with, some others just make eyebrows raise. The most obvious example is the user menu in GNOME 3.2, which contains an entry to configure your Google account, but no entry to shutdown the computer. Both decisions were taken independently, each of them with (good or bad) reasons, but the result is simply ridiculous. The default configuration in Debian will contain an extension to make it a bit better, but on the whole we don’t intend to diverge from the upstream design, on which a lot of good work has been done.

“On the whole we don’t intend to diverge from the upstream design, on which a lot of good work has been done.”

Point 3 is more complex. Red Hat being the company spending the most on GNOME, it is obvious that their employees work on making things work for their distribution. An example is the recurring discussions about relying on system services that are currently only implemented by systemd. Since there is a lot of (mostly unjustified) resistance against systemd in Debian, and since it won’t work on kFreeBSD anyway, someone needs to develop an alternative implementation of these services for upstart and sysvinit. Everything is in place for someone else to do the job but it has to be done, and this can be frustrating. Especially since it can also be hard to integrate changes needed for other distributions¹.

Hardware requirements are mostly a consequence of the previous criticism: there’s hardware that most distributions just don’t want to bother supporting. We’ve seen it in squeeze with the introduction of a hard dependency on PulseAudio. The Debian GNOME team (together with the Gentoo maintainers) made this dependency optional, carrying heavy patches, in order to cover the cases where it does not work. Now that it has gained more maturity, making this effort obsolete, the new tendency is to require 3D acceleration. For various reasons, it is not available to everyone². On this matter, the position of the Debian GNOME team has always been to support as much different configurations as possible with reasonable effort. Thanks to efforts from the incredible Vincent Untz, upstream supports a so-called “fallback mode”, which is the GNOME panel from 2.x with a lot of its bugs fixed. We intend to support this mode for as long as reasonably possible in Debian, possibly even after upstream ends up dropping it. However, other applications are going to require 3D because GStreamer is moving to clutter too, affecting video playback performance on non-accelerated systems³. For epiphany this is not a problem; only embedded video will be affected. But for totem, this is a major issue; because of that we will probably keep totem 3.0 in wheezy.

Finally, there is a natural human tendency to dislike change (I have it too), and it applies a lot to desktop users’ habits. Needless to say a change of such a scale as introducing GNOME Shell can trigger reactions. However, I don’t think it is reasonable, because of this resistance, to keep gnome-panel 2.x in Debian. This would be a lot of work on obsolete technology, and would prevent the upcoming removal of a lot of deprecated libraries. This time is much better spent improving gnome-panel 3.x in Debian and keeping the “fallback mode” great. One of the change that was made in Debian was to make it easier to find, being available as “GNOME Classic” directly from the login manager, instead of having to find it in an obscure configuration panel. In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it. I had never been accustomed to a new environment as quickly ever before.

“In all cases, I would recommend to actually try GNOME Shell for a few hours before ditching it.”

¹ Having seen several of my GDM patches reverted without a warning, I know we are not finished with carrying patches in Debian packages.
² Scientific workstations are a non-trivial example, since there is a measurable effect of using 3D in the window manager on heavy 3D applications.
³ On the other hand, on accelerated systems, this feature should end up improving performance a lot.

Raphael: What are your plans for Debian Wheezy?

Josselin: The first goal of the GNOME team is, of course, to provide again a great desktop environment to work on. For wheezy it will probably be based on GNOME 3.4. There also needs to be some work on package management interfaces. Upstream bases everything on PackageKit, but it is not as featureful as the aptdaemon Ubuntu technology. If I have time, I would also like to improve HTTP proxy support, since currently it is based on a stack of terrible hacks.

Raphael: If you could spend all your time on Debian, what would you work on?

Josselin: Obviously I would like to make GNOME in Debian even better. That would imply working on underneath dependencies (what we now like to call plumbing) to make sure everything is working great. This would also imply working more as GNOME upstream to make it more suitable for our needs.

I would also work on large-scale improvements on the distribution, like conditional recommends which I’d love to see implemented¹, or automatic build-dependency generation. I would also work on the installer to make it better for desktops machines.

¹ The idea is to automatically install language packs, or glues between two packages when both packages are installed.

Raphael: What’s the biggest problem of Debian?

Josselin: The obvious answer is the same as the one most people you interviewed before gave: not enough members in core teams. A lot of developers join Debian to work on a small number of pet packages, and don’t necessarily want to be involved with existing teams. It is probably still not obvious enough that the primary way to start contributing to Debian is to join an existing team.

But if there is one thing that is preventing Debian from gaining more momentum now, it is a completely different one: the too short support timeframe. 3 years is really not enough for corporate users. One year to migrate from one version to another is too short, and it is not possible to skip a release. It is definitely possible to change that with reasonable effort: the long-term support after 3 years doesn’t have to cover the same perimeter as the short-term one. For example, we could upgrade the kernel to the version in the current stable release, and stop fixing all non-remote security holes. The important thing is to cover the most basic needs: companies are ready to take the risk of having less support if it allows skipping a version, but not the risk of having no support at all. And even more important is to say that you do something. Red Hat says they support a release for 10 years, but of course after 5 years the supported perimeter is extremely small.

“3 years [of support] is really not enough for corporate users.”

Long-term support will not magically fix all problems in Debian, but it will bring more corporate users into the picture. And with corporate users come paid Debian developers, who can work on critical pieces of the system. Debian was built on the synergy between individuals and companies, and in recent years – perhaps as a reaction against what happened with Ubuntu – we’ve kind of forgot the latter. A lot of individuals have joined the project, and they are actively working, for example, on shortening the release cycle, which goes against the interest of professionals. We should embrace again such users and developers, and that means adapting to the current needs of larger entities.

Raphael: You’re the maintainer of python-support, a packaging helper that was competing with python-central. Both helpers are now deprecated in favor of dh_python2. Does this mean that the situation of Python in Debian is now sane? Or are there remaining problems?

Josselin: dh_python2 (and the Python3 version, dh_python3) has a sane enough design. It fixes a lot of issues in python-central and also python-support, at the expense of somehow reduced functionality for developers. However, just like the previous tools, it merely works around design mistakes in the Python interpreter. For example it is not possible to split binary modules, pure-Python modules and byte-compiled modules in different directory trees, like Perl does – although PEP 3147 introduces a way to do so. There is still no sane and standardized way to deal with module versions. There is no difference made between the module (which is a part of language semantics) and the file containing it (an information which depends on the implementation). Developers heavily rely on introspection features and make assumptions based on the implementation, that make it impossible to work around problems with module files.

Such problems are not restricted to Python. Those who fought against Ruby gems could tell even worse stories. While introducing GObject introspection packages in Debian (they can be used in JavaScript and Python to provide modules based on GObject libraries), I was pleased to see a clear distinction between file and module, but I was again struck by the fact you are not forced to declare API versions in your Python/JS code. In all cases, there is no reliable way to detect runtime dependencies in a given Python or JavaScript file, which leaves the maintainer to declare them by hand, and of course, often be wrong about them. Add to that the fact that most errors cannot be detected before runtime. For all these reasons, and while still being fond of Python for scripts and prototyping, I’ve become really skeptical of using purely interpreted languages to write real applications. Some GNOME developers are moving away from Python and JavaScript, mostly towards Vala; I can only approve of that move and hope the same happens to other projects.

Raphael: Is there someone in Debian that you admire for their contributions?

Of course there is the never-sleeping, never-stopping, Michael Biebl who can upload a whole GNOME release in a single week-end. But there are a lot of awesome people who make Debian something that simply works. I could talk about Cyril Brulebois from the X strike force, Julien Cristau from the release team, Sjoerd Simons for his sound advice and work on plumbing, Luca Falavigna who is so fast at processing NEW, to quote only a few of those I work with frequently. And of course, Jordi and Sam for their humor.


Thank you to Josselin for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that you can find older interviews on http://wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook

.

Answering questions of Debian users on various support channels

When you start your journey with Debian, you tend to have lots of questions. You’ll find some answers in various documentations but there always are remaining questions. Those can be asked on various support channels:

Those are the places where you can also start your journey as a Debian contributor… instead of asking questions, you just have to answer questions of other users! Let me share some advice if you want to do some user support.

User support is difficult…

It’s not always an easy task. Some users are more skilled than others and there might be difficulties related to the language, English is not always the native language of a user who asks a question in English.

Be respectful and courteous when you answer user questions, even if they made mistakes. You’re effectively representing Debian and you should give out a good image of the project. If you don’t have the patience or the time needed to do a good answer, don’t reply and let someone else take care of this user. I invite you to read (and follow!) the Debian Community Guidelines.

Avoid RTFM answers, instead you should show the users how they could have found (alone) the solution to their problem. We don’t want to scare people away, we want to grow our community.

But it’s also rewarding

In some cases, the problem reported by the user will be a real problem and you’ll have an opportunity to file a good bug report, thus helping to improve Debian for everybody.

Often, you don’t even have the answer to the user’s question. But you’re more skilled than him/her to do researches on the web, or you know of a good documentation that might contain the relevant bits of information, in any case you’re doing further research to help this user. In this process, you also grow your own skills since you’re learning stuff that you didn’t know yet.

At least that’s how I learned many things during my first year in the Debian community… there’s no reason why you couldn’t learn lots of stuff that way, in particular if you also read the answers of other skilled people on those channels (it takes a bit of training to learn who are the skilled people though).

I still believe that doing user support is one of the best ways to join the Debian community and to start contributing. It helps you to grow your skills, and to slowly progress from “average user” to “advanced user”.

If you want to start contributing to Debian, click here to subscribe to my newsletter and get future updates for new contributors. You can also follow me on Identi.ca, Google+, Twitter and Facebook.

Debian related goals for 2012

Like last year, here’s a list of Debian related goals that I’d like to achieve this year. I might not have the time to implement all the projects, but I like to write them down to keep me motivated. And maybe it can inspire other people to implement some of them (or to help me).

  1. Finish the translation of the Debian Administrator’s Handbook

    The target is to have the book available in April. It would be nice to complete the liberation fund until then so that the book is immediately made available under a DFSG-free license.

  2. Update the Debian Administrator’s Handbook for Wheezy
  3. Translating the book in English is only the start of the journey. The real challenge is to keep the book up-to-date with each subsequent release of Debian. And Wheezy should hopefully be released in 2012 since the freeze is in June.

  4. Design and implement the Debian Package Maintenance Hub

    It’s an ambitious project that aims to merge and replace the PTS, the DDPO and their respective mail variants. It should also standardize the flow of information directed towards package maintainers. I’m going to use the DEP process to drive this project.

    This could easily take most of the year, but hopefully I’ll motivate other people to chime in and help.

  5. Implement dpkg --check-db and dpkg --repair-db

    While dpkg is fairly reliable, it’s not exempt of bugs and more annoyingly, harddrives/filesystems are not 100% reliable either, thus it happens that some internal database files get corrupted. Given that most files are text based, advanced users can manually fix them but many less skilled users are just left with a broken system that they tend to reinstall.

    To avoid this, we could provide a command that would try to automatically bring back the internal database to a sane state by looking for a working backup to restore (while at the same time marking some packages as requiring re-installation since we have some indications that they were present).

  6. Implement storage of dpkg’s internal files in Git

    This would be an extension of the former idea. Installing a package dpkg-db-history (any idea for a better name?) would setup dpkg hooks that would record every database change in a git repository. This repository could then be used to restore the last working version of the database.

Besides those concrete projects, I want to do better than last year on the topic of funding my Debian work. I will thus reiterate some objectives:

  1. Write useful articles for Debian users and Debian contributors.

    They should complete the pages Mastering Debian, Contributing to Debian 101, Debian Packaging Tutorials, and help me increase the audience of this blog.

  2. Write at least one Debian-related ebook (different from the Debian Admin Handbook) and sell it.

    It could be an ebook targetting testing users since I believe that many more users could benefit from it if they had some better knowledge of the limitations and of the way to mitigate the problems that arise from time to time.

    Or maybe it could be an ebook for people who want to start contributing to Debian, it could even be bundled with a few hours of mentoring.

  3. By the end of the year, have at least 1/3 of my time funded by donations and/or earnings of my information products.

    This means doing 3,5 times better than in 2011. It should be doable given that the sales of the Debian Administrator’s Handbook will contribute to this goal (once the translation is over).

That makes up lots of challenges for this year. Feel free subscribe to my newsletter to stay up-to-date with my progress and to get my monthly summary of the Debian/Ubuntu news. It’s also a good way to help me reach those goals since you will be informed of all my new projects.

Review of my Debian related goals for 2011

Last year I shared my “Debian related goals for 2011”. I tend to put more goals than what I can reasonably complete alone and this year was no exception. Let’s have a look.

  1. Translate my Debian book into English: PARTLY DONE
    It took more time than expected to prepare and to run the fundraising campaign but it has been successful and the translation is happening right now.
  2. Finish multiarch support in dpkg: DONE BUT NOT ENTIRELY MERGED YET
    Yes, multiarch support was already in the pipe last year in January. I completed the development between January and April (it was sponsored by Linaro) and since then it has mostly been waiting on Guillem to review it, tweak it, and integrate it.
  3. Make deb files use XZ compression by default: TRIED BUT ABANDONED
    After discussing the issue with Colin Watson and Joey Hess during debconf, I came to the conclusion that it was not really desirable at this point. The objections were that debian-installer was not ready for it and that it adds a new dependency on xz for debootstrap to work on non-Debian systems. I believe that the debian-installer side is no longer a problem since “unxz” is built in busybox-udeb (since version 1:1.19.3-2). For the other side, there’s not much to do except ensuring that xz is portable to all the other OS we care about. DAK has been updated too (see #556407).
  4. Be more reactive to review/merge dpkg patches: PARTLY DONE
    I don’t think we had any patch that received zero input. We still have a backlog of patches, and the situation is far from ideal but the situation improved.
  5. Implement the rolling distribution proposed as part of the CUT project and try to improve the release process: NOT DONE

    We had a BoF during debconf, we discussed it at length on debian-devel, but in the end we did nothing out of it. Except Josselin Mouette who wrote a proof of concept for his idea.

    For me testing is already what people are expecting from a rolling distribution. It’s just a matter of documenting how to effectively use testing, and of some marketing by defining rolling as alias to testing.

  6. Work more regularly on the developers-reference: PARTLY DONE
    I did contribute some new material to the document but not as much as I could have hoped. On the other hand, I have been rather reactive to ensure that sane patches got merged. We need more people writing new parts and updating the existing content.
  7. Write a 10-lesson course called “Smart Package Management”: NOT DONE
  8. Create an information product (most likely an ebook or an online training) and sell it on my blog: NOT DONE
    This was supposed to happen after the translation of the Debian Administrator’s Handbook. Since the translation is not yet over, I did not start to work on this yet.
  9. By the end of the year, have at least 1/3 of my time funded by donations and/or earnings of my information products: NOT REACHED
    My target was rather aggressive with 700 € each month, and given that I did not manage to complete any information product, I’m already very pleased to have achieved a mean amount of 204 € of donations each month (min: 91 €, max: 364 €). It’s more than two times better than in 2010. Thank you! Note that those figures do not take into account the revenues of the fundraising of the Debian Administrator’s Handbook since they will be used for its translation.

That makes quite a lot of red (for things that I did not achieve)… on the other hand I completed projects that I did not foresee and did not plan. For instance improving dpkg-buildflags and then merging Kees Cook work on hardened build flags was an important step for Debian. This was waiting for so long already…

People Behind Debian: Steve McIntyre, debian-cd maintainer, former Debian Project Leader

Steve McIntyre has been contributing to Debian since 1996, 2 years before I joined! But I quickly stumbled upon Steve: in 1999, he was struggling with getting his debian-cd script to produce 2 ISO images (it was the first time that Debian did no longer fit on a single CD), I helped him by rewriting debian-cd with a robust system to split packages on as many ISO images as required.

I remember those times very well because Steve was very supportive of my efforts and it was a real pleasure to get this done. His friendly nature probably also explains why he got elected Debian Project Leader twice!

Anyway, enough history, check out his interview to learn more about the great work he’s doing nowadays. My questions are in bold, the rest is by Steve.

Raphael: Who are you?

Steve: I’m a professional software engineer, 37, living in Cambridge (England) with my new wife Jo. I studied for the EIST degree at the University of Cambridge, then (like many people here, it seems) I just forgot to go home again afterwards and settled here. I spent more of my “study” time playing with Linux than working on my degree, so I guess I’m lucky that it worked and I found a career in that area!

Raphael: How did you start contributing to Debian?

Steve: During my time in college, I started hacking on software in my free time, using Slackware as my first Linux distribution from the middle of 1994. After encountering more and more problems with Slackware, I was encouraged by a number of friends to make the jump over to Debian and in October 1996 I did. The installation process back then was much harder than anything people see today, but after a long weekend I finally had my Debian system up and running.

I was already one of the main upstream developers for the Mikmod music player at that time, so that very same weekend I applied to be a DD so I could maintain it in Debian too. Back then, the NM process was much simpler: I just mailed a key to Bruce and he set me up with an account almost immediately!

I then found that Joey Hess had beaten me to it and already packaged Mikmod. Grrr! :-)

Raphael: What’s your biggest achievement within Debian?

Steve: Without a doubt, my proudest achievement within Debian is being elected Project Leader for 2 years by the other developers. It’s a great feeling to have earned the trust of your friends and peers, and also a great responsibility to go and help Debian where needed: talking to the press about Debian, assisting wherever problems crop up, etc. The DPL job is certainly a lot of hard work, and I have nothing but respect for anybody who volunteers for it.

“It’s a great feeling to have earned the trust of your friends and peers.”

Elsewhere, I’ve been leading the Debian CD team for years too, both doing most of the maintenance of the debian-cd package and producing and testing the regular installation CDs and DVDs that we ship to the world. Again, this is a time-consuming job but it needs doing and it’s worthwhile.

Raphael: You’re currently employed by ARM. What are you working on and are they supportive of your Debian involvement?

Steve: The situation within ARM is very interesting; I’m employed in PDSW (Processor Division, SoftWare), a new group founded just a couple of years back to help improve the state of software on ARM. Most of the people in the group are working on Free Software at this stage (e.g. toolchains, browsers, Linux kernel), which is lovely. Some of the engineers have also been seconded into a new non-profit company Linaro, which is a collaboration between ARM and a number of other companies investing in core Linux software and tools for ARM-based CPUs. I’m one of the ARM engineers in Linaro, and I’m a Technical Architect in the Office of the CTO. My role includes looking at future projects for Linaro to help with (e.g. ARM servers), but for the last few months I’ve been concentrating on the new armhf “architecture” in Debian, Ubuntu and elsewhere.

armhf is a new “architecture” in Debian and Ubuntu terms, but it’s not strictly a new type of hardware. Instead, it’s a new ABI. We have two reasons for doing this work:

  1. It targets the latest version of 32-bit ARM CPUs (v7) and makes better use of the hardware, for better performance. Compare targetting i686 instead of i386, for example. We’ll still support the older “armel” port for the foreseeable future for users with older hardware that can’t run armhf.
  2. More importantly: we are standardising on the ABI / compiler options / hardware support for future users.

In the past, there has been a huge amount of specialisation (aka fragmentation) in the ARM Linux environment, and that worked OK for specialised devices that only ever ran the software shipped with them. ARM CPUs are now becoming more and more mainstream, so people will expect to be able to install generic software on their machines. That gives a requirement for a standard base platform, and armhf (arm-linux-gnueabihf in GNU triplet terms) is that standard that we are pushing in the community. Debian, Ubuntu, Fedora, Suse and others are all going to use this, making compatibility possible.

I’ve been working with a small team of people to make armhf happen, helping where needed: putting together build machines; patching Debian packages directly; discussing and fixing toolchain issues with Ubuntu folks; agreeing ABI specifications with people from Fedora; advising people from other distros bootstrapping their new ARM ports.

ARM and Linaro are very supportive of this work, and it’s been lovely being sponsored to work directly on Free Software like this. It’s work that will directly benefit ARM and its partners (of course!), but it’s also helping out more generally too: Debian QA work, cross-build support, bootstrapping efforts, multi-arch. More and more of the ARM market is driven by Free Software, and companies are acknowledging that. I should probably also mention that we’re hiring…! :-)

Raphael: What are your plans for Debian Wheezy?

Steve: There are three main tracks here.

Obviously, I’m interested in seeing armhf release with Wheezy. We’ve just been added to Testing last weekend, so that’s going well. We’ve got over 90% of the archive built now, and we’re mopping up the remaining issues.

I’m the primary maintainer of cdrkit at this point, but I’d prefer to have it go away. Xorriso and the associated software in libisoburn is almost capable of replacing all the aging cdrtools-derived software that we have in Debian, The only missing feature that I’m aware of is creating the HFS hybrid filesystems that we use for installations on Mac systems. I’ve been talking with the upstream folks about this for some time already, and I’m hoping we can finish this soon enough that we can get it into Wheezy.

Finally, I’ve got the ever-growing wishlist of things for debian-cd. We’ve got the beginnings of an automated test suite that Martín Ferrari has written, but it needs integrating and improving. I want to help get regular weekly/daily/release debian-live builds running on the main CD build machine. There’s work needed if we want to make good installation media for the new multi-arch world, too. The Emdebian people are asking for help making CD images… The list goes on :-)

Raphael: The ARM community seems to be very interested in multi-arch. Can you explain why?

Steve: There are a number of reasons for ARM people to be interested in multi-arch; two really stand out for me:

  • With the historical issues around the plethora of ARM ABIs in the wild, multi-arch will allow us to potentially support multiple ABIs cleanly on one system. That allows users to have (for example) an up-to-date system that makes the most of their current hardware, yet also run legacy programs that might use an older ABI. There’s also a new 64-bit architecture coming (ARMv8) which will run older 32-bit software; again, multi-arch makes mixed installation of old and new software reasonable.
  • ARM has traditionally been a common target for cross-compilation, and I’d expect that to remain the case for a long time to come yet. For a lot of embedded developers, using a big fast i386/amd64 machine to compile is much faster than using a limited-power small ARM CPU. However, setting up sane cross-compilation environments has long been a bugbear for developers. Getting the toolchain and all the cross-architecture libraries to work together correctly can be like black magic. This is potentially the “killer app” for multi-arch: simply install the libraries for the target architecture directly on your development machine. Install a simple cross-gcc package and (maybe) qemu, and you’re all set.

“This is potentially the “killer app” for multi-arch: simply install the libraries for the target architecture […], install a simple cross-gcc package […] and you’re all set.”

Raphael: What’s the biggest problem of Debian?

Steve: For me, Debian’s biggest problem has been the same for a long time: we are forever short of enough people to do the work that we’re trying to do. That might sound like a weird thing to claim when Debian is one of the largest Free Software projects on the planet, but it’s more a statement of just how huge our goals are. Many of the largest things in Debian are developed or controlled by very small teams working very hard, and there’s always a risk of losing people due to burnout in those situations.

“We are forever short of enough people to do the work that we’re trying to do.”

Some of the tasks that should be easy given our large membership (e.g. large-scale packaging transitions) can often instead take a very long time. We are fortunate to have more people wanting to join in Debian’s work all the time, but we also need to be careful to keep on promoting what we’re doing and recruiting new contributors, encouraging them to get more and more involved in core work. Debian gets ever bigger in terms of the size and the number of packages we distribute; we’re not currently matching that growth rate elsewhere.

Raphael: What motivates you to continue to contribute year after year?

Steve: This one is much easier to answer! The thing that first attracted me to Debian was the fact that I could help to develop it, help to decide how things could and should be done within it. Instead of being forced to accept what some corporation decided I could do with my computer, I could change the software to suit my needs and preferences. Alongside that, I could get involved with a strong community of similar people all over the world, all with their own strong opinions about how software should work.

I joined in and found it was great fun and very rewarding. That hasn’t changed for me in the intervening years, and that’s why I’m still around. I work on Debian because it helps me to get the OS that I want to use. It seems that lots of people around the world find it useful too, and that’s awesome. :-)

Raphael: Do you believe that Stefano Zacchiroli will be the first DPL who managed to stay 3 consecutive years on the seat? Would you like him to candidate again?

Steve: To be honest, I would be very surprised if Zack stood again for DPL this year. He told me himself that he wasn’t planning on it, and I can understand that decision. He’s been an awesome DPL in my opinion, and I’m glad that he took the job. But: it is also a very difficult and time-consuming task that would be enough to wear down anybody. If Zack does decide to stand again, I would support him 100%. But I know that we also have lots of other good people in Debian who would be ready to take up the challenge next.

Raphael: Is there someone in Debian that you admire for their contributions?

Steve: There are lots of people I admire in Debian, so many so that I almost don’t want to list individuals here for fear of missing people out. But… :-)

Bdale Garbee has been an inspiration to many of us, for many years. He’s technically excellent, a great friend to many of us, an endless source of sage advice and (last but not least) he has some wonderful stories to tell about his experiences over the years. On top of that, he’s just cool. :-)

Christian Perrier is another exceptional developer, in my eyes – he’s great at co-ordinating people in translations, working tirelessly to make this very important part of Debian work better and better with every release. He’s also a really nice guy and we all love him.

I also have to mention Joey Hess here, whether he likes it or not. *grin* He’s been responsible for so many good things in Debian over the years, even if he did steal my first package…

Finally, the teams of people who make sure that Debian is always working: the security team and DSA. The rest of us can choose to take time off from Debian to go and do other things, but these people need to cover things every day. That’s a major responsibility, and I salute them for taking on that challenge.


Thank you to Steve for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that you can find older interviews on http://wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook

.

My Debian Activities in December 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (364.18 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg and Multiarch

I had some hope to have a multiarch-enabled dpkg in sid for Christmas as Guillem told me that it was realistic. Alas Guillem got sick. We’re in January and we’re still not there.

While some of Guillem’s commits in December were related to multi-arch, the size of his pu/multiarch/master branch did not really shrink. We still have 36 commits to merge… most of the work he did was refactoring some parts of the code that were already merged. And he initiated some discussion on interface changes. I participated to those discussions hoping to bring them to a quick resolution.

I’m still maintaining my own pu/multiarch/full branch, it is based on Guillem’s branch but with further fixes that I developed and that he has not yet merged and with a change reverted (Guillem’s branch allows crossgrading packages between different architectures while dpkg does not manage this correctly yet).

I can only hope that January will be the last month of this never-ending saga. It’s been one year since I started working on this project. :-|

Misc dpkg work

I reviewed (and later merged) a patch of Kees Cook to enhance dpkg-buildflags so that it can report which hardening features are enabled. This feature might then be used by tools like lintian to detect missing hardening features.

I mentored Gianluca Ciccarelli who is trying to enhance dpkg-maintscript-helper to take care of replacing a directory by a symlink and vice-versa.

I took care of #651993 so that dpkg-mergechangelogs doesn’t fail when it encounters an invalid version in the changelog, and of #652414 so that dpkg-source --commit accepts a relative filename when a patch file is explicitly given.

Guillem also merged a fix I developed for LP#369898.

Packaging work

WordPress 3.3 came out so I immediately packaged it. Despite my upstream bug report, they did not update their GPL compliance page which offers the corresponding sources for what’s bundled in the tarball. So I hunted for the required sources myself, and bundled them in the debian.tar.xz of the Debian source package. It’s a rather crude solution but this allowed me to close the release critical bug #646729 and to reintroduce the Flash files that were dropped in the past… which is great since the Flash-based file uploader is nicer than the one using the browser’s file field.

Quilt 0.50 came out after 2 years of (slow) development. The Debian package has many patches and several of them had to be updated to cope with the new upstream release. Fortunately some of them were also merged upstream. It still took an entire morning to complete this update. I also converted the packaging from CDBS to dh with a short rules file.

Zim 0.54 came out and I immediately updated the package since it fixed a bug that was annoying me.

Review of the ledgersmb packaging

As the sql-ledger maintainer (and a user of this software for my accounting), I have been hoping to get ledgersmb packaged as a possible replacement for it. I have been following the various efforts initiated over time but none of them resulted in a real package in Debian.

This is a real pity so I tried to fix this by offering to sponsor package uploads. That’s why I did a first review of the packaging. It took several hours because you have to explain everything that’s not good enough.

I also filed a wishlist bug against lintian (#652963) to suggest that lintian should detect improper usage of dpkg-statoverride (this is a mistake that was present in the package that I reviewed).

nautilus-dropbox work

I wanted to polish the package in time for the Ubuntu LTS release and since Debian Import Freeze is in January, I implemented some of the important fixes that I wanted.

The Debian package diverges from upstream in that the non-free binaries are installed in /var/lib/dropbox/ instead of $HOME.
Due to a bug, the files were not properly root-owned so I first fixed this (unpacking the tarball as root lead to reuse of the embedded user & group information, and those information changed recently on the Dropbox side apparently).

Then we recently identified other problems related to proxy handling (see #651065). I fixed this too because it’s relatively frequent that the initial download triggered during the package configuration fails… and in that case it’s the user that will re-trigger a package download after having given the appropriate credentials through PackageKit. Without my fix, usage of pkexec would imply the loss of the http_proxy environment variable and thus it would not be possible for a user to download through a proxy.

Last but not least I reorganized the Debian specific patches to better separate what can and should be merged upstream, from the changes that upstream doesn’t want. Unfortunately Dropbox insists on being able to auto-update their non-free binaries, they are, thus, against the installation under /var/lib/dropbox and the corresponding changes.

Book update

We’re making decent progress in the translation of the Debian Administrator’s Handbook, about 6 chapters are already translated (not yet reviewed though).

The liberation campaign is also (slowly) going forward. We’re at 67% now (thanks to 90 new supporters!) while we were only at 60% at the start of December.

Thanks

See you next month for a new summary of my activities.

300 EUR for Debian

It’s the amount of the wire transfer that I made to ffis e.V., the German association that works together with Software in the Public Interest to hold Debian’s monetary assets.

This amount includes the 10% of the benefits of my Debian DVD sales (for a total of 47 USD =~ 36 EUR) and the 15% affiliate revenue that was granted to Debian by some supporters of the Debian Administrator’s Handbook crowdfunding campaign (for a total of 259.49 EUR).

Thank you everybody, this donation is really yours and not mine!

If you want to complement this donation, and make a nice gift to Debian for Christmas, go to this page and follow the instructions.

“3.0 (quilt)” is the most widely used Debian source package format

My goal with the “3.0 (quilt)” source format has always been to standardize the patch management in Debian source packages. This message seems to have been well understood. dbs and dpatch have been deprecated by their respective maintainers.

I made numerous efforts to make this source format useful in as many use cases as possible (but some improvements are still possible) and I have added hints to encourage maintainers to switch. Thanks to this, the adoption rate of this new source format has been very good and it’s now the most widely used source package format in Debian—only two years after its introduction in Debian unstable.

With 9829 source package using “3.0 (quilt)”, it surpassed the number of source package still using “1.0″ (7368). (Those numbers have been taken from http://upsilon.cc/~zack/stuff/dpkg-v3/ on december 13th 2011.) The number of source packages using “3.0 (quilt)” doubled this year.

(Click on the picture to see it full size)

Of the 7368 packages using the old format, 6816 packages trigger the missing-debian-source-format lintian tag. This means that only 552 source packages have explicitly opted to keep using the old format and that the bulk of the remaining packages are rarely updated packages that have not been switched yet.