People behind Debian: Mehdi Dogguy, release assistant

Mehdi Dogguy

Picture of Mehdi taken by Antoine Madet

Mehdi is a Debian developer for a bit more than a year, and he’s already part of the Debian Release Team. His story is quite typical in that he started there by trying to help while observing the team do its work. That’s a recurrent pattern for people who get co-opted in free software teams.

Read on for more info about the release team, and Mehdi’s opinion on many topics. My questions are in bold, the rest is by Mehdi (except for the additional information that I inserted in italics).

Who are you?

I’m 27 years old. I grew up in Ariana in northern Tunisia, but have been living in Paris, France, since 2002.

I’m a PhD Student at the PPS laboratory where I study synchronous concurrent process calculi.

I became interested in Debian when I saw one of my colleagues, Samuel Mimram (first sponsor and advocate) trying to resolve #440469, which is a bug reported against a program I wrote. We have never been able to resolve it but my intent to contribute was born there. Since then, I started to maintain some packages and help where I can.

What’s your biggest achievement within Debian?

I don’t think I had time to accomplish a lot yet :) I’ve been mostly active in the OCaml team where we designed a tool to compute automatically the dependencies between OCaml packages, called dh-ocaml. This was a joint work with Stéphane Glondu, Sylvain Le Gall and Stefano Zacchiroli. I really appreciated the time spent with them while developing dh-ocaml. Some of the bits included in dh-ocaml have been included upstream in their latest release.

I’ve also tried to give a second life to the Buildd Status Pages because they were (kind of) abandoned. I intend to keep them alive and add new features to them.

If you had a wand and could change one thing in Debian, what would that be?

Make OCaml part of a default Debian installation :D

But, since I’m not a magician yet, I’d stick to more realistic plans:

  1. A lot of desktop users fear Debian. I think that the Desktop installation offered by Debian today is very user-friendly and we should be able to attract more and more desktop users. Still, there is some work to be done in various places to make it even more attractive. The idea is trying to enhance the usability and integration of various tools together. Each fix could be easy or trivial but the final result would be an improved Desktop experience for our users. Our packaged software run well. So, each person can participate since the most difficult part is to find the broken scenarios. Fixes could be found together with maintainers, upstream or other interested people.

    I’ll try to come up with a plan, a list of things that need polishing or fixes and gather a group of people to work on it. I’d definitely be interested in participating in such a project and I hope that I’ll find other people to help. If the plan is clear enough and has well described objectives and criteria, it could be proposed to the Release Team to consider it as a Release Goal for Wheezy.

  2. NMUs are a great way to make things move forward. But, sometimes, an NMU could break things or have some undesirable effects. For now, NMUers have to manually track the package’s status for some time to be sure that everything is alright. It could be a good idea to be auto-subscribed to the bugs notifications of NMUed packages for some period of time (let’s say for a month) to be aware of any new issues and try to fix them. NMUing a package is not just applying a patch and hitting enter after dput. It’s also about making sure that the changes are correct and that no regressions have been introduced, etc…

  3. Orphaned packages: It could be considered as too strict and not desired, but what about not keeping orphaned and buggy packages in Testing? What about removing them from the archive if they are buggy and still unmaintained for some period? Our ftp archive is growing. It could make sense to do some (more strict) housekeeping. I believe that this question can be raised during the next QA meeting. We should think about what we want to do with those packages before they rot in the archive.

[Raphael Hertzog: I would like to point out that pts-subscribe provided by devscripts makes it easy to temporarily subscribe to bug notifications after an Non-Maintainer Upload (NMU).]

You’re a Debian developer since August 2009 and you’re already an assistant within the Release Management team. How did that happen and what is this about?

In the OCaml team, we have to start a transition each time we upload a new version of the OCaml compiler (actually, for each package). So, some coordination with the Release Team is needed to make the transition happen.

When we are ready to upload a new version of the compiler, we ask the Release Team for permission and wait for their ack. Sometimes, their reply is fast (e.g. if their is no conflicting transition running), but it’s not always the case. While waiting for an ack, I used to check what was happening on debian-release@l.d.o. It made me more and more interested in the activities of the Release Team.

Then (before getting my Debian account), I had the chance to participate in DebConf9 where I met Luk and Phil. It was a good occasion to see more about the tools used by the Release Team. During April 2010, I had some spare time and was able to implement a little tool called Jamie to inspect the relations between transitions. It helps us to quickly see which transitions can run in parallel, or what should wait. And one day (in May 2010, IIRC), I got offered by Adam to join the team.

As members of the Release Team, we have multiple areas to work on:

  1. Taking care of transitions during the development cycle, which means making sure that some set of packages are correctly (re-)built or fixed against a specific (to each transition) set of packages, and finding a way to tell Britney that those packages can migrate and it would be great if she also shared the same opinion. [Raphael Hertzog: britney is the name of the software that controls the content of the Testing distribution.]
  2. Paying attention to what is happening in the archive (uploads, reported RC bugs, etc…). The idea is to try to detect unexpected transitions, blocked packages, make sure that RC bug fixes reach Testing in a reasonable period of time, etc…
  3. During a freeze, making sure that unblock requests and freeze exceptions are not forgotten and try to make the RC bug count decrease.

There are other tasks that I’ll let you discover by joining the game.

Deciding what goes (or not) in the next stable release is a big responsibility and can be incredibly difficult at times. You have to make judgement calls all the time. What are your own criteria?

That’s a very hard to answer question (at least, for me). It really depends on the “case”. I try to follow the criteria that we publish in each release update. Sometimes, an unblock request doesn’t match those criteria and we have to decide what to accept from the set of proposed changes. Generally, new features and non-fixes (read new upstream versions) changes are not the kind of changes that we would accept during the freeze. Some of them could be accepted if they are not intrusive, easy and well defended. When, I’m not sure I try to ask other members of the Release Team to see if they share my opinion or if I missed something important during the review. The key point is to have a clear idea on what’s the benefit of the proposed update, and compare it to the current situation. For example, accepting a new upstream release (even if it fixes some critical bugs) is taking a risk to break other features and that’s why we (usually) ask for a backported fix.

It’s also worth noticing that (most of the time) we don’t decide what goes in, but (more specifically) what version of a given package goes in and try to give to the contributors an idea on what kind of changes are acceptable during the freeze. There are some exceptions though. Most of them are to fix a critical package or feature.

Do you have plans to improve the release process for Debian Wheezy?

We do have plans to improve every bit in Debian. Wheezy will be the best release ever. We just don’t know the details yet :)

During our last meeting in Paris last October, the Release Team agreed to organize a meeting after Squeeze’s release to discuss (among other questions) Wheezy’s cycle. But the details of the meeting are not fixed yet (we still have plenty of time to organize it… and other more important tasks to care about). We would like to be able to announce a clear roadmap for Wheezy and enhance our communication with the rest of the project. We certainly want to avoid what happened for Squeeze. Making things a bit more predictable for developers is one of our goals.

Do you think the Constantly Usable Testing project will help?

The original idea by Joey Hess is great because it allows d-i developers to work with a “stable” version of the archive. It allows them to focus on the new features they want to implement or the parts they want to fix (AIUI). It also allows to have constantly available and working installation images.

Then, there is the idea of having a constantly usable Testing for users. The idea seems nice. People tend to like the idea behind CUT because they miss some software disappearing from Testing and because of the long delays for security fixes to reach Testing.

If the Release Team has decided to remove a package from Testing, I think that there must be a reason for that. It either means that the software is broken, has unfixed security holes or was asked for the removal by its maintainer. I think that we should better try to spend some time to fix those packages, instead of throwing a broken version in a new suite. It could be argued that one could add experimental’s version in CUT (or sid’s) but every user is free to cherry-pick packages from the relevant suite when needed while still following Testing as a default branch.

Besides, it’s quite easy to see what was removed recently by checking the archive of debian-testing-changes or by querying UDD. IMO, It would be more useful to provide a better interface of that archive for our users. We could even imagine a program that alerts the user about installed software that got recently removed from Testing, to keep the user constantly aware any issue that could affect his machine. About the security or important updates, one has to recall the existence of Testing-security and testing-proposed-updates that are used specifically to let fixes reach Testing as soon as possible when it’s not possible to go through Unstable. I’m sure that the security team would appreciate some help to deal with security updates for Testing. We also have ways to speed migrate packages from Unstable to Testing.

I have to admit that I’m not convinced yet by the benefits brought by CUT for our users.


Thank you to Mehdi for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

5 reasons why Debian Unstable does not deserve its name

Debian Unstable (also known as sid) is one of the 3 distributions that Debian provides (along with Stable and Testing).

It’s not conceived as a product for end-users, instead it’s the place where contributors are uploading newer packages. Daily. Yes that means that Unstable is a quickly moving target and it’s not for everybody. But you can use it and your computer won’t explode.

1. It contains mainly stable versions of the software

Yes, you read it right. Unstable is not full of development versions of the various software. It happens on some software but then it’s usually a conscious decision of the maintainer who believes that this specific version is already better than the previous one.

The packages in sid are supposed to migrate to testing, the place where the next Debian stable release is prepared. So maintainers are advised to only upload stuff that is of release quality, the rest should be uploaded to experimental instead.

2. It doesn’t break badly every other day

Breakages happen but they are not a big deal usually. It has been long time since I could not reboot my computer after an upgrade or since the graphical interface was no longer working. The kind of breakages that you have is that one software stops working, or triggers an annoying bug, or that a few packages are uninstallable.

In most cases, you can save yourself by downgrading to the version available in Testing. Or by finding a work-around in the bug tracking system. Or by not upgrading because you have apt-listbugs installed and you have been warned about the problem.

3. It’s the basis of other distributions

If Debian Unstable was really so bad, it would not be a good basis to build a derivative distribution, isn’t it? But Ubuntu and SiduxAptosid (to name only two) are based on Debian Sid.

4. It’s not inherently less secure than Stable or Testing

High impact security vulnerabilities will usually be quickly fixed in Stable and Unstable. The stable upload is done by the security team while the unstable one is made by the maintainer. Testing will usually get the fix through the package uploaded to Unstable, so testing users get security updates with a delay.

For less serious vulnerabilities, it’s entirely possible that stable does not get any update at all. In that case, unstable/testing users are better served since they will get the fix with the next upstream version anyway.

Of course, it happens that maintainers are busy or that something falls through the cracks, but there are other people watching RC bugs who will fix this if the maintainer doesn’t react at all.

5. I use it on my main computer

And many other people do the same. And you can do the same if you meet the criteria below:

  • you can work on the command-line (enough to downgrade a problematic package, to edit configuration files, etc.);
  • you know how to work with APT and multiple distributions in /etc/apt/sources.list;
  • you are able to read/write English so that you can read/file bug reports when needed;
  • you have another computer connected to the Internet that you can use to lookup documentation (or the bug tracking system, or the support mailing lists) when your usual computer is off-line for a reason that you don’t understand.

If you feel you are not ready for the jump, click here to subscribe to this blog (or here via the RSS feed), I’ll surely teach some of the required skills in future articles.

PS: All that said, if you have a working sid installation, do not upgrade it just before an important presentation, or before a trip. It will always break at the most annoying time. Unless you like to live dangerously, of course.

Avoid a newbie packager mistake: don’t build your Debian packages with dpkg -b

In the last years, I have seen many people try to use dpkg --build to create Debian packages. Indeed, if you look up dpkg’s and dpkg-deb‘s manual pages, this option seems to be what you have to use:

-b, --build directory [archive|directory]

Creates a debian archive from the filesystem tree stored in directory. directory must have a DEBIAN subdirectory, which contains the control information files such as the control file itself. This directory will not appear in the binary package’s filesystem archive, but instead the files in it will be put in the binary package’s control information area.

And indeed, dpkg-deb is what ultimately creates the .deb files (aka binary packages). But it’s a low-level tool that you should not call yourself. If you want to properly package a new software, you should rather create a Debian source package that will transform upstream source code into policy-compliant binary packages.

Creating a source package also involves preparing a directory tree (but with a “debian” sub-directory), it’s probably more complicated than calling dpkg -b on a manually crafted directory. But the result is much more versatile: the tools used bring value by dynamically analyzing/modifying the files within your package (for example, the dependencies on C libraries that your package needs are automatically inserted).

If this is news to you, you might want to check out the New Maintainers’ Guide and the Debian Policy.

Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.

Howto to rebuild Debian packages

Being able to rebuild an existing Debian package is a very useful skill. It’s a prerequisite for many tasks that an admin might want to perform at some point: enable a feature that is disabled in the official Debian package, rebuild a source package for another suite (for example build a Debian Testing package for use on Debian Stable, we call that backporting), include a bug fix that upstream developers prepared, etc. Discover the 4 steps to rebuild a Debian package.

1. Download the source package

The preferred way to download source packages is to use APT. It can download them from the source repositories that you have configured in /etc/apt/sources.list, for example:

deb-src http://ftp.debian.org/debian unstable main contrib non-free
deb-src http://ftp.debian.org/debian testing main contrib non-free
deb-src http://ftp.debian.org/debian stable main contrib non-free

Note that the lines start with “deb-src” instead of the usual “deb”. This tells APT that we are interested in the source packages and not in the binary packages.

After an apt-get update you can use apt-get source publican to retrieve the latest version of the source package “publican”. You can also indicate the distribution where the source package must be fetched with the syntax “package/distribution“. apt-get source publican/testing will grab the source package publican in the testing distribution and extract it in the current directory (with dpkg-source -x, thus you need to have installed the dpkg-dev package).

$ apt-get source publican/testing
Reading package lists... Done
Building dependency tree       
Reading state information... Done
NOTICE: 'publican' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/collab-maint/publican.git
Need to get 727 kB of source archives.
Get:1 http://nas/debian/ squeeze/main publican 2.1-2 (dsc) [2253 B]
Get:2 http://nas/debian/ squeeze/main publican 2.1-2 (tar) [720 kB]
Get:3 http://nas/debian/ squeeze/main publican 2.1-2 (diff) [4728 B]
Fetched 727 kB in 0s (2970 kB/s)  
dpkg-source: info: extracting publican in publican-2.1
dpkg-source: info: unpacking publican_2.1.orig.tar.gz
dpkg-source: info: unpacking publican_2.1-2.debian.tar.gz
$ ls -dF publican*
publican-2.1/                 publican_2.1-2.dsc
publican_2.1-2.debian.tar.gz  publican_2.1.orig.tar.gz

If you don’t want to use APT, or if the source package is not hosted in an APT source repository, you can download a complete source package with dget -u dsc-url where dsc-url is the URL of the .dsc file representing the source package. dget is provided by the devscripts package. Note that the -u option means that the origin of the source package is not verified before extraction.

2. Install the build-dependencies

Again APT can do the grunt work for you, you just have to use apt-get build-dep foo to install the build-dependencies for the last version of the source package foo. It supports the same syntactic sugar than apt-get source so that you can run apt-get build-dep publican/testing to install the build-dependencies required to build the testing version of the publican source package.

If you can’t use APT for this, enter the directory where the source package has been unpacked and run dpkg-checkbuilddeps. It will spit out a list of unmet build dependencies (if there are any, otherwise it will print nothing and you can go ahead safely). With a bit of copy and paste and a “apt-get install” invocation, you’ll install the required packages in a few seconds.

3. Do whatever changes you need

I won’t detail this step since it depends on your specific goal with the rebuild. You might have to edit debian/rules, or to apply a patch.

But one thing is sure, if you have made any change or have recompiled the package in a different environment, you should really change its version number. You can do this with “dch --local foo” (again from the devscripts package), replace “foo” by a short name identifying you as the supplier of the updated version. It will update debian/changelog and invite you to write a small entry documenting your change.

4. Build the package

The last step is also the simplest one now that everything is in place. You must be in the directory of the unpacked source package.
Now run either “debuild -us -uc” (recommended, requires the devscripts package) or directly “dpkg-buildpackage -us -uc”. The “-us -uc” options avoid the signature step in the build process that would generate a (harmless) failure at the end if you have no GPG key matching the name entered in the top entry of the Debian changelog.

$ cd publican-2.1
$ debuild -us -uc
 dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): 
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor): 
dpkg-buildpackage: source package publican
dpkg-buildpackage: source version 2.1-2rh1
dpkg-buildpackage: source changed by Raphaël Hertzog 
 dpkg-source --before-build publican-2.1
dpkg-buildpackage: host architecture i386
[...]
dpkg-deb: building package `publican' in `../publican_2.1-2rh1_all.deb'.
 dpkg-genchanges  >../publican_2.1-2rh1_i386.changes
dpkg-genchanges: not including original source code in upload
 dpkg-source --after-build publican-2.1
dpkg-buildpackage: binary and diff upload (original source NOT included)
Now running lintian...
Finished running lintian.

The build is over, the updated source and binary packages have been generated in the parent directory.

$ cd ..
$ ls -dF publican*
publican-2.1/                    publican_2.1-2rh1.dsc
publican_2.1-2.debian.tar.gz     publican_2.1-2rh1_i386.changes
publican_2.1-2.dsc               publican_2.1-2rh1_source.changes
publican_2.1-2rh1_all.deb        publican_2.1.orig.tar.gz
publican_2.1-2rh1.debian.tar.gz

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

People behind Debian: David Kalnischkies, an APT developer

The two first interviews were dedicated to long-time Debian developers. This time I took the opposite approach, I interviewed David Kalnischkies who is not (yet) a Debian developer. But he’s contributing to one of the most important software within Debian—the APT package manager—since 2009. You can already see him in many places in Debian sharing his APT knowledge when needed.

English is not his native language and he’s a bit shy, but he accepted the interview nevertheless. I would like to thank him for the efforts involved and I hope his story can inspire some people to take the leap and just start helping… My questions are in bold, the rest is by David.

Who are you?

I am David Kalnischkies, 22 years old, living in the small town Erbach near Wiesbaden in Germany and I’m studying computer science at the TU Darmstadt. Furthermore I am—for more than half a decade now—young group leader of my hometown.

I never intended to get into this position, but it has similarities with my “career” in this internet-thingy here. I don’t remember why, but in April 2009 I was at a stage that some simple bugs in APT annoyed me so much that I grabbed the source, and most importantly—I don’t know why I did it— but I published my changes in Mai with #433007, a few more bugs and even a branch on launchpad. And this public branch got me into all this trouble in June: I got a mail from “Mr. package managment” Michael Vogt regarding this branch…

A few days later I joined an IRC session with him and closely after that my name appeared for the first time in a changelog entry. It’s a strange but also addicting feeling to read your own name in an unfamiliar place. And even now after many IRC discussions, bugfixes and features, three Ubuntu Developer Summits and a Google Summer of Code in Debian, my name still appear in places I have never even thought about—e.g. in an interview.

What’s your biggest achievement within Debian?

I would like to answer “MultiArch in APT” as it was my Google Summer of Code project, but as it has (not much) use for the normal user at this point—will hopefully change for wheezy—I chose three smaller things in squeeze’s APT that many people don’t even know yet:

  • “MMap run out of room”, the most famous error related to the APT::Cache-Limit option is gone. The DynamicMMap in APT now deserves its name :)
  • Download and usage of multiple Translations of Descriptions
  • crazy stuff like apt-cache show ^apt-.*$/experimental works

If your impression is now that I only do APT stuff: that’s completely right, but that’s already more than enough for me for now as the toolchain behind the short name APT contains so many tools and use cases that you always have something different.

You’re an active member of the APT development team. Are there plans for APT in Debian Wheezy? What features can we expect?

That’s very hard to answer, as the team is too small to be able to really plan something. I mean, you can have fancy plans and everything and half a second later someone arrives on the mailing list with a “small” question which eats days of development time just for debugging…

But right now the TODO list contains (in no particular order):

  • MultiArch – managing packages built for different architectures like i386 on amd64 to ease the usage of packages which only work on specific architectures or cross-compiling.
  • rewriting the subsystem responsible for downloading files to e.g. enable tools like apt-file or debtags to reuse the infrastructure by plugging into it.
  • splitting long descriptions out of the Packages file to possibly decrease the size to download on an “apt-get update”. Includes mostly pulling the button only to enable it by the archive masters
  • backporting popular features of some frontends to enable all frontends to use them, e.g. “apt-get changelog” or “apt-get download”
  • support research on resolver improvements by implementing support for a distribution independent upgrade problem format (CUDF).
  • an additional single binary for users wrapping apt-get, apt-cache and co.
  • getting in contact (again) with various forks like apt-rpm and derivatives like maemo or telesphoreo to join forces instead of splitting them by working on different (and possibily very old) heads of the dragon

We will see what will get real for wheezy and what is postponed, but one thing is sure: more will be done for wheezy if you help!

If you could spend all your time on Debian, what would you work on?

I would spend it on APT’s debbugs count… zero would be cool to look at! We make progress in this regard, but with the current velocity we will reach it in ten years or so.

Reading more mailing lists would be interesting, as I am kind of an information junky. Maintaining a package could be interesting to share the annoyance of a maintainer with handcrafted dependencies just to notice that APT doesn’t get it in the way I intended it to be. Through, to make it feel real I need to train a few new APT contributors before so they can point my mistake out, but this unfortunately doesn’t depend so much on time but on victims… Maybe I could even be working on getting an official status.

Beside that, I would love to be able to “apt-get dist-upgrade” the increasing mass of systems I and many others carry around in their pockets. In regards to my phone, this is already fixed, but there is much room for improvements.

What’s the biggest problem of Debian?

You need to be lucky. You need to talk at the right time to the right person. That’s not really a debian-only problem as such, but in a global project full of volunteers you can see it clearly as there are plenty of opportunities to be unlucky.

For example, it’s unlikely that an interview would be made with me now if Michael had not contacted me in June 2009. In a big project like Debian, you are basically completely lost without a mentor guiding you, so things like the debian-mentors list are good projects, but I am pretty certain they could benefit from some more helping hands.

The other thing which I consider a problem is that—and I read from time to time—some people don’t care for translations. That’s bad. Yes, a developer is able to read English, otherwise s/he couldn’t write code or participate on the mailinglists.

Still, I personally prefer to use a translated application if I have the chance as it’s simply easier for me to read in my mother tongue, not only because I am dyslexic, but because my mind still thinks in German and not in English. Yes, I could personally fix that by thinking in English only from now on, but its a quite big problem to convince my family—which is not really familiar with tech-stuff—to use something if they can’t understand what is written on screen.

It was hard enough to tell my mother how to write an SMS in a German interface. My phone with English words all over the place would be completely unusable for her—despite the fact that my phone is powered by Debian and better for the task from a technical point of view.

You are not yet an official Debian developer/maintainer, but you’re already perceived in the community as one the most knowledgeable person about APT. It’s a great start! What’s your advice to other people who want to start contributing to Debian in general, and to APT in particular?

It was never a goal in my life to “start contributing”. My goal was and still is to make my life easier by letting the computer work for me. At some point APT hindered the success of this goal, so it needed to be fixed. I didn’t expect to open pandora’s box.

So, my advice is simple: Just start. Ignore the warning signs telling you that this is not easy. They are just telling you that you do something useful. Only artificial problems are easy. Further more, contribution to APT, dpkg or any other existing package is in no way harder than opening an ITP and working on your own, and it’s cooler as you have a similar minded team around you to talk to. :)

“APT didn’t accept release codenames as target release” was one of the first things I fixed. If I had asked someone if that would be a good starting point the answer would have been a clear “no”, but I didn’t search for a good starting point…

As a kid I can start playing football by just walking on the field and play or I can sit near the field, watching the others play, while analyzing which position would be the best for me to start ruling out one by one as the technical requirements seem too high… “Oh – bicycle kick – that sounds complicated… I can’t do that”

Julian Andreas Klode is working on a APT replacement, there’s also Cupt by Eugene V. Lyubimkin. Both projects started because their authors are not satisfied with APT, they find APT’s code difficult to hack partly due to the usage of C++. Do you share their concerns and what’s your opinion on those projects?

I don’t think C++ is a concern in this regard, after all cupt is currently rewritten to C++0x and APT2 started in vala and is now C + glib—last time I checked at least. I personally think that something is wrong if we need to advertise an application by saying in which language it is written…

The major “problem” for APT is probably that the code is “old”: APT does its job for more than 12 years now, under different maintainers with an always changing environment around it: so there are lines in APT which date from a time when nobody knew what a “Breaks” dependency is, that packages can have long descriptions which can be translated or even that package archives can be signed with a gpg key! And yet we take all those for granted today. APT has proven to adapt to these changes in the environment and became in this process very popular. So I don’t think the point is near (if it will come at all) that APT can go into retirement as it is completely replaced by something else.

The competitors one the other hand have their first 12 years still to go. And it will be interesting to see how they will evolve and what will be the state of the art in 2022…

But you asked what I think about the competitors: I prefer the “revolution from inside” simply because I can see effects faster as more users will profit from it now. Cupt and co. obviously prefer the “normal” revolution. The goal is the same, creating the best package manager tool, but the chosen way to the goal is different. aptitude and cupt have an interactive resolver for example: that’s something I dislike personally, for others that is the ultimate killer feature. cupt reading the same preference file as APT will have a different pinning result, which we should consider each time someone mentions the word “drop-in replacement”.

APT2 isn’t much more than the name—which I completely dislike—currently from a user point of view, so I can’t really comment on that. All of them make me sad as each line invested in boilerplate code like configuration file parsing would be in my eyes better be spent in a bugfix or new feature instead, but I am not here to tell anyone what they should do in their free time…

But frankly, I don’t see them really as competitors: I use the tools I use, if other do that too that’s good, if not that’s their problem. :) The thing that annoys me really are claims like “plan is to remove APT by 2014″ as this generates a “vi vs. emacs” like atmosphere we don’t need. If some people really think emacs is a good editor… who cares?

I really hope we all can drink a beer in 2022 in Milliways, the restaurant at the end of the package universe, remembering the good old 2010… ;)

Is there someone in Debian that you admire for his contributions?

No, not one, many!

Michael Vogt who has nearly the monopole of “package manager maintainer” by being upstream of APT, synaptics and software center to name only the biggest and still has the time to answer even the dumbest of my questions. :) Jason Gunthorpe for being one of the initial developers behind “deity” who I will probably never meet in person beside in old comments and commit logs. Christian Perrier for caring so much about translations. Obey Arthur Liu as a great admin for Debian’s participation in Google’s Summer of Code. Paul Wise for doing countless reviews on debian-mentors which are a good source of information—not only for the maintainer of the package under review.

I guess I need to stop here because you asked for just one. So let’s end with some big words instead: I am just a little cog in the big debian wheel…


Thank you to David Kalnischkies for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

What’s annoying with Flattr buttons on Planet Debian?

Dear Planet Debian readers, a significant number of people have expressed concerns over the presence of Flattr buttons on Planet Debian. The concerns were expressed in a thread on the debian-project mailing list and they were quite diverse.

While the discussion brought some of the issues into light, it’s not really possible to find out whether a specific concern is widely shared or not. That’s why I set up a poll on selectricity.org.

Please take a few minutes and rank the various options: put the most important concern first in the list, and sort the others by decreasing importance. If there are concerns that you do not agree with, put them below the entry named “Special: I don’t share the concerns quoted below”. And to prove that you have read through all the options before voting, put the other special entry last (it’s named “Special: I know how condorcet voting works and thus I put this item last”).

Click here to go to the poll. You need only one minute to vote and no login is required.

PS: The discussion on debian-project is the reason why I changed the footer in my RSS feed to contain only plain text (in a smaller font). I also improved the wording to be more neutral.

State of the Debian-Ubuntu relationship

Debian welcoming contributions from derivatives

The relationship between Debian and Ubuntu has been the subject of many vigorous debates over the years, ever since Ubuntu’s launch in 2004. Six years later, the situation has improved and both projects are communicating better. The Natty Narwhal Ubuntu Developer Summit (UDS) featured—like all UDS for more than 2 years—a Debian Health Check session where current cooperation issues and projects are discussed. A few days after that session, Lucas Nussbaum gave a talk during the mini-Debconf Paris detailing the relationship between both projects, both at the technical and social level. He also shared some concerns for Debian’s future and gave his point of view on how Debian should address them. Both events give valuable insights on the current state of the relationship.

Lucas Nussbaum’s Debian-Ubuntu talk

Lucas started by introducing himself. He’s an Ubuntu developer since 2006 and a Debian developer since 2007. He has worked to improve the collaboration between both projects, notably by extending the Debian infrastructure to show Ubuntu-related information. He attended conferences for both projects (Debconf, UDS) and has friends in both communities. For all of these reasons, he believes himself to be qualified to speak on this topic.

Collaboration at the technical level

He then quickly explained the task of a distribution: taking upstream software, integrating it in standardized ways, doing quality assurance on the whole, delivering the result to users, and assuring some support afterward. He pointed out that in the case of Ubuntu, the distribution has one special upstream: Debian.

Indeed Ubuntu gets most of its software from Debian (89%), and only 7% are new packages coming from other upstream projects (the remaining 4% are unknown, they are newer upstream releases of software available in Debian but he was not able to find out whether the Debian packaging had been reused or not). From all the packages imported from Debian, 17% have Ubuntu-specific changes. The reasons for those changes are varied: bugfixes, integration with Launchpad/Ubuntu One/etc., or toolchain changes. The above figures are based on Ubuntu Lucid (10.04) while excluding many Ubuntu-specific packages (language-pack-*, language-support-*, kde-l10n-*, *ubuntu*, *launchpad*).

The different agendas and the differences in philosophy (Debian often seeking perfect solutions to problems; Ubuntu accepting temporary suboptimal workarounds) also explain why so many packages are modified on the Ubuntu side. It’s simply not possible to always do the work in Debian first. But keeping changes in Ubuntu requires a lot of work since they merge with Debian unstable every 6 months. That’s why they have a strong incentive to push changes to upstream and/or to Debian.

There are 3 channels that Ubuntu uses to push changes to Debian: they file bug reports (between 250 to 400 during each Ubuntu release cycle), they interact directly with Debian maintainers (often the case when there’s a maintenance team), or they do nothing and hope that the Debian maintainer will pick up the patch directly from the Debian Package Tracking System (it relays information provided by patches.ubuntu.com).

Lucas pointed out that those changes are not the only thing that Debian should take back. Ubuntu has a huge user base resulting in lots of bug reports sitting in Launchpad, often without anyone taking care of them. Debian maintainers who already have enough bugs on their packages are obviously not interested in even more bugs, but those who are maintaining niche packages, with few reports, might be interested by the user feedback available in Launchpad. Even if some of the reports are Ubuntu-specific, many of them are advance warnings of problems that will affect Debian later on, when the toolchain catches up with Ubuntu’s aggressive updates. To make this easier for Debian maintainers, Lucas improved the Debian Package Tracking System so that they can easily get Ubuntu bug reports for their packages even without interacting with Launchpad.

Human feelings on both sides

Lucas witnessed a big evolution in the perception of Ubuntu on the Debian side. The initial climate was rather negative: there were feelings of its work being stolen, claims of giving back that did not match the observations of the Debian maintainers, and problems with specific Canonical employees that reflected badly on Ubuntu as a whole. These days most Debian developers find something positive in Ubuntu: it brings a lot of new users to Linux, it provides something that works for their friends and family, it brings new developers to Debian, and it serves as a technological playground for Debian.

On the Ubuntu side, the culture has changed as well. Debian is no longer so scary for Ubuntu contributors and contributing to Debian is The Right Thing to do. More and more Ubuntu developers are getting involved in Debian as well. But at the package level there’s not always much to contribute, as many bugfixes are only temporary workarounds. And while Ubuntu’s community follows this philosophy, Canonical is a for-profit company that contributes back mainly when it has compelling reasons to do so.

Consequences for Debian

In Lucas’s eyes, the success of Ubuntu creates new problems. For many new users Linux is a synonym for Ubuntu, and since much innovation happens in Ubuntu first, Debian is overshadowed by its most popular derivative. He goes as far as saying that because of that “Debian becomes less relevant”.

He went on to say that Debian needs to be relevant because the project defends important values that Ubuntu does not. And it needs to stay as an independent partner that filters what comes out of Ubuntu, ensuring that quality prevails in the long term.

Fixing this problem is difficult, and the answer should not be to undermine Ubuntu. On the contrary, more cooperation is needed. If Debian developers are involved sooner in Ubuntu’s projects, Debian will automatically get more credit. And if Ubuntu does more work in Debian, their work can be showcased sooner in the Debian context as well.

The other solution that Lucas proposed is that Debian needs to communicate on why it’s better than Ubuntu. Debian might not be better for everybody but there are many reasons why one could prefer Debian over Ubuntu. He listed some of them: “Debian has better values” since it’s a volunteer-based project where decisions are made publicly and it has advocated the free software philosophy since 1993. On the other hand, Ubuntu is under control of Canonical where some decisions are imposed, it advocates some proprietary web services (Ubuntu One), the installer recommends adding proprietary software, and copyright assignments are required to contribute to Canonical projects.

Debian is also better in terms of quality because every package has a maintainer who is often an expert in the field of the package. As a derivative, Ubuntu does not have the resources to do the same and instead most packages are maintained on a best effort basis by a limited set of developers who can’t know everything about all packages.

In conclusion, Lucas explained that Debian can neither ignore Ubuntu nor fight it. Instead it should consider Ubuntu as “a chance” and should “leverage it to get back in the center of the FLOSS ecosystem”.

The Debian health check UDS session

While this session has existed for some time, it’s only the second time that a Debian Project Leader was present at UDS to discuss collaboration issues. During UDS-M (the previous summit), this increased involvement from Debian was a nice surprise to many. Stefano Zacchiroli—the Debian leader—collected and shared the feedback of Debian developers and the session ended up being very productive. Six months later is a good time to look back and verify if decisions made during UDS-M (see blueprint) have been followed through.

Progress has been made

On the Debian side, Stefano set up a Derivatives Front Desk so that derivative distributions (not just Ubuntu) have a clear point of contact when they are trying to cooperate but don’t know where to start. It’s also a good place to share experiences among the various derivatives. In parallel, a #debian-ubuntu channel has been started on OFTC (the IRC network used by Debian). With more than 50 regulars coming from both distributions, it’s a good place for quick queries when you need advice on how to interact with the distribution that you’re not familiar with.

Ubuntu has updated its documentation to prominently feature how to cooperate with Debian. For example, the sponsorship process documentation explains how to forward patches both to the upstream developers and to Debian. It also recommends ensuring that the patch is not Ubuntu-specific and gives some explanation on how to do it (which includes checking against a list of common packaging changes made by Ubuntu). The Debian Derivative Front Desk is mentioned as a fallback when the Debian maintainer is unresponsive.

While organizing Ubuntu Developer Week, Ubuntu now reaches out to Debian developers and tries to have sessions on “working with Debian”. Launchpad has also been extended to provide a list of bugs with attached patches and that information has been integrated in the Debian Package Tracking system by Lucas Nussbaum.

Still some work to do

Some of the work items have not been completed yet: many Debian maintainers would like a simpler way to issue a sync request (a process used to inject a package from Debian into Ubuntu). There’s a requestsync command line tool provided by the ubuntu-dev-tools package (which is available in Debian) but it’s not yet usable because Launchpad doesn’t know the GPG keys of Debian maintainers.

Another issue concerns packages which are first introduced in Ubuntu. Most of them have no reason to be Ubuntu-specific and should also end up in Debian. It has thus been suggested that people packaging new software for Ubuntu also upload them to Debian. They could however immediately file a request for adoption (RFA) to find another Debian maintainer if they don’t plan to maintain it in the long term. If Ubuntu doesn’t make this effort, it can take a long time until someone decides to reintegrate the Ubuntu package into Debian just because nobody knows about it. This represents an important shift in the Ubuntu process and it’s not certain that it’s going to work out. As with any important policy change, it can take several years until people are used to it.

Both issues have been rescheduled for this release cycle, so they’re still on the agenda.

This time the UDS session was probably less interesting than the previous one. Stefano explained once more what Debian considers good collaboration practices: teams with members from both distributions, and forwarding of bugs if they have been well triaged and are known to apply to Debian. He also invited Ubuntu to discuss big changes with Debian before implementing them.

An interesting suggestion that came up was that some Ubuntu developers could participate in Debcamp (one week hack-together before Debconf) to work with some Debian developers, go through Ubuntu patches, and merge the interesting bits. This would nicely complement Ubuntu’s increased presence at Debconf: for the first time, community management team member Jorge Castro was at DebConf 10 giving a talk on collaboration between Debian and Ubuntu.

There was also some brainstorming on how to identify packages where the collaboration is failing. A growing number of Ubuntu revisions (identified for example by a version like 1.0-1ubuntu62) could indicate that no synchronization was made with Debian, but it would also identify packages which are badly maintained on the Debian side. If Ubuntu consistently has a newer upstream version compared to Debian, it can also indicate a problem: maybe the person maintaining the package for Ubuntu would be better off doing the same work in Debian directly since the maintainer is lagging or not doing their work. Unfortunately this doesn’t hold true for all packages since many Gnome packages are newer in Ubuntu but are actively maintained on both sides.

Few of those discussions led to concrete decisions. It seems most proponents are reasonably satisfied with the current situation. Of course, one can always do better and Jono Bacon is going to ensure that all Canonical teams working on Ubuntu are aware of how to properly cooperate with Debian. The goal is to avoid heavy package modifications without coordination.

Conclusion

The Debian-Ubuntu relationships used to be a hot topic, but that’s no longer the case thanks to regular efforts made on both sides. Conflicts between individuals still happen, but there are multiple places where they can be reported and discussed (#debian-ubuntu channel, Derivatives Front Desk at derivatives@debian.org on the Debian side or debian@ubuntu.com on the Ubuntu side). Documentation and infrastructure are in place to make it easier for volunteers to do the right thing.

Despite all those process improvements, the best results still come out when people build personal relationships by discussing what they are doing. It often leads to tight cooperation, up to commit rights to the source repositories. Regular contacts help build a real sense of cooperation that no automated process can ever hope to achieve.

This article was first published in Linux Weekly News. You can get my monthly summary of the Debian/Ubuntu news, all you have to do is to click here to subscribe to my free newsletter.

Go2Linux interviewed me: the biggest problem of Debian

Guillermo Garron of Go2Linux enjoys a lot the “People behind Debian” interviews that I make. That’s why he interviewed me (with somewhat similar questions) and published the result on his blog.

Click here to read the full interview. I speak of my Debian projects, of the Ubuntu-Debian relationship, and more.

The question that I would like to highlight is “What’s the biggest problem of Debian?”. I answered this:

Our project identity is somewhat minimalistic. It evolves around the social contract and the Debian Free Software Guidelines. Both documents answer the question of what we’re doing but we lack a clear answer to the question of how we’re supposed to work towards our goals. It would be great if Debian could agree on some principles concerning topics like goal setting, collaboration, team work, politeness, respect. We could then advertise those and build on them while recruiting volunteers.

PS: The interview also hit LinuxToday.

Librement: a new way to help people who want to contribute to free software

Find your way in the free software worldI have this project in my head, I want to work on it but I always lack the time. In order to go forward, I thought I could write about it, at least it would let me clarify my ideas and the core goals. So here I am, I will present you Librement (I have registered the alioth project but it’s empty).

The core goal is to make it easy for every user to contribute to free software in some way. I will now present the main features that I envision.

Defining skills and interests

In order to propose tasks that the user can do, we must have an idea of his skills. So on the first run (and later through a preferences menu) the user will be invited to define his skills:

  • his native languages (multiple allowed)
  • other languages he can understand
  • programming languages he knows
  • version control systems he can use
  • markup language he knows (HTML, DocBook, Wiki-like formats, etc.)
  • etc.

Maybe we can also ask which skills he would like to learn. Because contributing to free software is a nice opportunity to learn new skills!

We should also find out what the user is interested in. What are his favorite free software projects? What kind of contributions would he like to do (documentation, translation, coding, bug fixing, bug triaging, creating artwork, donations, etc.)?

Choose activities and pick concrete tasks

Based on the user’s skills and his interests, the software shows a list of possible activities. The user can then sort that list, from the most interesting one to those that he doesn’t want to do.

Each activity can generate concrete tasks. For example, the activity “Do translation for Debian” could generate a task “Translate strings in debconf/fr.po” or “Review translations in partman/fr.po”.

Work on tasks

When the user decides to work on a task, a step-by-step assistant helps him/her. It can automate some steps and provide explanations for the remaining ones, for example in the case of a translation for Debian:

  • grab the PO file (from a VCS, from an HTTP URL, from a translation server, etc.);
  • select and install a software to work with PO file (if not already done);
  • edit the PO file with the preferred program;
  • check the PO file (is it complete? is there no mistakes like missing substitutions?);
  • send back the completed PO file in a mail to the Debian bugtracking system.

If the tasks is not completed in one go, the user can resume it the next time.

Each free software project must provide some meta-information describing the various workflows involved for contributing to the different parts of the project. If necessary the project can also provide new plugins to support new operations that are not available in the default library.

Setting goals

In order to keep the user motivated, the software could track how much time he spent contributing to free software and it could verify if the user reached the goals he picked up for himself. Maybe it can also hook into the OMG Trophy Awarding System.

The sky is the limit

I hope that you now have a clearer idea of what this desktop application is supposed to be. There are literally hundreds of ways to contribute to free software and I like the idea that we can streamline the process for most users.

All the plugins implementing activities can use local information (list of packages installed with their versions, configuration settings, etc.) to propose tasks targetted to the user and highly beneficial for the corresponding free software projects. For example, a bug tagged unreproducible might benefit from a few more users trying to reproduce it. The software could direct the user to this bug report if it detects that he/she runs the same version on the same architecture and that this software is regularly run on the system.

Many projects have created “operations” or “events” to encourage people to contribute, they could all be implemented as dedicated activities in Librement. I’m thinking of stuff like Gnome Love, Ubuntu’s 5-a-day, Ubuntu’s 100 papercuts, etc.

Even for people who have no time to contribute, the application can still be useful by referencing the various ways to donate money (or material) to projects that they are using.

Feedback welcome

I’m excited by the potential of such an application, but it’s normal since it’s my idea. Do you believe it can be useful and popular? Do you have ideas of exciting activities that such a framework can offer?

PS: If you wonder how I came up with the name “Librement”, here’s the explanation. It’s a French word which means “freely”. And users who want to give back are trying to live up to the principles of free software, which I sum up by “they are trying to live freely”.

Follow me on Identi.ca, Twitter and Facebook. Or subscribe to this blog by RSS or by email.

Support 5 free software with Flattr

Flattr FOSS LogoA new month means new free software projects to support with Flattr FOSS. I’m happy to see that it’s gaining traction outside of the Debian world as well. I saw quite a few new entries for free software projects so that I don’t have to fear running out of suggestions in the next few months. :-)
Let’s go over the 5 projects that I recommend you for December:

  1. Getting Things GNOME (flattr link) is a GNOME task manager. Its name is a play on David Allen’s Getting Things Done (GTD) but the software doesn’t enforce the GTD methodology. You can implement GTD however since the software is very flexible and lets you organize the tasks with arbitrary tags. It’s a promising software and it can be extended with many plugins.
  2. The W3C validator (flattr link) is used by thousands of web developers to verify that their HTML pages are well formed. But did you know that it was free software? Yes you can contribute code, or you can help them with a flattr.
  3. Bitlbee (flattr link) is a gateway between IRC and many other instant messaging protocols. Geeks are known to be IRC addicted (at least I am using it for Debian development) and with bitlbee it’s one reason less to watch something else than the IRC client. :-)
  4. Arch Hurd (flattr link) is a port of the ArchLinux distribution to the GNU Hurd kernel.
  5. Paste.debian.net (flattr link) is a simple website where you can share some textual content for a limited amount of time (you usually paste the content from some other applications, hence its name). Very handy when you want to quickly show something to others on IRC. This service is run by Debian developer Alexander Wirt.

That’s it for this month. A quick question to finish this issue: I count at least 15 Debian contributors using Flattr currently, would you be interested by a small directory listing them?