My Free Software Activities in January 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (84.25 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Debian Packaging

In one of my customer projects, I had to use libwebsockets and since it was not packaged for Debian, I filed a “Request For Package” (RFP #697671). I discovered a fork of this library on github and decided to mail the original author and the author of the fork to learn a bit more about the reason of the fork. It turns out that they miscommunicated and that the original author was interested by most of the improvements. The fork still exists but the important fixes and most of the improvements have been merged (and he released a version 1.0 after that!). Furthermore the original author setup a bug tracker to better organize the project and so that the author of the fork can submit patches and be sure that they won’t be forgotten (as it happened in the past). I spend quite some time discussing with both parties but at the end I’m pleased to see that good progress has been made (although nobody stepped up to maintain this package in Debian).

I packaged zim 0.59 (an important bugfix release) and wordpress 3.5.1 (with several security fixes). I updated the dpkg-dev squeeze backports to version 1.16.9~bpo60+1 on request of Daniel Schepler. This backport led me to file #698133 on kgb-client because the bot literally spammed the #debian-dpkg IRC channel for multiple hours by resending old commit notices that got merged in the squeeze-backports branch. BTW, they need help to get this issue fixed.

I updated python-django-registration to fix a compatibility issue with python3-sphinx (see #697721 for details).

Misc Debian Stuff

Serious bug with salt. I filed a grave bug on salt (#697747—) and prepared the upload to fix the issue on request of the maintainer. In the mean time, the maintainer orphaned the package. Franklin G. Mendoza already announced its willingness to take over but this package deserves multiple maintainers since this is a good piece of software that is getting more and more popular.

net-retriever and alternate keyrings. I filed a wishlist bug (#698618) on net-retriever to request a way for derivatives to use another keyring package (i.e. not debian-archive-keyring-udeb) without having to fork net-retriever.

Linux 3.7 on armel/armhf. I helped the kernel maintainers to fix the 3.7 kernel on armel/armhf by reporting on IRC the results of successive failing kernel rebuilds on those architectures (this kernel version is only in experimental).

Carl9170 firmware. I also pinged the kernels maintainers about a missing firmware for the carl9170 driver (already reported in #635840) and Ben Hutchings took care of re-activating its inclusion in upstream’s linux-firmware.git and then uploaded firmware-free 3.2 to Debian. Thanks Ben!

New QA team member. And to finish with the miscellaneous stuff, I helped Holger Levsen to be added to the “qa” group so that he could integrate his awesome work on automated QA checks with Jenkins.

Debian France

Preparation for Solutions Linux. The people organizing the “village of associations” in the Solutions Linux conference have asked all organizations to apply for a booth if they wanted one. Last year Carl Chenet took care of organizing this and this time we had to find someone else. I made multiple call for volunteers (on the mailing list, on my blog) without much success but I finally managed to convince Tanguy Ortolo to take care of this. Thank you Tanguy!

Get in touch with treasurer who disappeared. During the transition with the former Debian France officers, it has been said that Aurélien Gérôme — another former treasurer of Debian France — had entirely disappeared together with some papers that he never gave to his successor. I didn’t want to give up on this without at least trying to get in touch by myself… so after multiple tries (over IRC, phone, and snail mail), and some weeks without answers, he got back to me, explaining that he’s currently in a foreign country and that he will take care of that next time that he comes in France. \o/

New website in preparation. Replacing the single-page website webpage with a more comprehensive website is an important goal. Alexandre Delanoë provided a basic ikiwiki setup inspired by dsa.debian.org. I cleaned it and integrated it in a git repository on our machine. There’s thus a new test website on http://france.debian.net/test/. Tanguy Ortolo and Fernando Lagrange immeditaly made some small improvements but since then nobody stepped up to further complete the website. I’ll try to do this in February and put the new website in production.

Paypal and handling of members. We installed a paypal plugin in galette so that members can renew their membership online. I asked Christian Bayle to try it out and we found some issues that I reported upstream and that got fixed. But this is only the first step, we want to go much further and automate all the membership handling, from membership renewal mail reminders up to integration in the accounting system. To this end, I filed some new tickets in the Galette tracker and completed some that were already opened: #490, #368 and #394. We requested a quote for those tickets and Debian France is going to fund the work on those tickets so that we have a 100% free software solution for our needs.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in December 2012

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (836.78 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Debian Packaging

I uploaded Zim 0.58, WordPress 3.5 (and I had to file a ticket about the availability of sources of minified javascript files — again) and another security update for python-django (696535).

Speaking of python-django, I forwarded one bug report of Anders Kaseorg concerning Django’s bash completion (#695811).

I also sponsored the upload of ledgersmb 1.3.25-1.

Other Debian work

I contributed some patches to improve debian-installer support in live-build. I also added a work-around for bug #652946 in live-installer and committed a fix for this same bug in a jessie branch of partman-target.

I also prepared a bugfix for a counter-productive behavior of choose-mirror (#695261, it was not possible to override the codename of the release to install via preseed if you install from a CD with a full base system).

I discovered an oddity in the Packages.diff/Index file for the architectures armhf and s390x, I reported it to ftpmasters in #696792.

(All those issues were discovered while working for a customer)

Debian France

I spent quite some time on Debian France this month again. I started by setting up an internal gitolite to manage our accounting/administrative documents.

Then I updated galette, the web application that we are using to manage our database of members. In the process, I filed two bugs that we discovered. I immediately tested Galette by registering 4 members that joined during the former mini-Debconf in Paris.

We have plans to automate the membership renewal process so we have opened a Paypal account. This month we also cleared the last steps so that I and Sylvestre Ledru have full control on the Debian France bank account.

I also registered the new officers at the “Tribunal d’instance de Sarreguemines”.

Salt bug reports

During the last mini-debconf, I discovered Salt (thanks to Julien Cristau!) and since I had to switch some servers of mine, I took this opportunity to upgrade to wheezy and try out salt at the same time.

It took much more time than expected but the result is pleasant. The configuration of all my servers is now well documented/specified in a central Git repository, and moving services is much easier than before.

In the process, I filed quite some bugs (#2865, #2851, #2866 and #2875), most of them have been fixed in the 0.11.1 release that just happened.

Thanks

I wish you a happy new year and all the best for 2013!

See you next month for a new summary of my activities.

My Free Software Activities in November 2012

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (692.20 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Misc packaging

I updated the publican package (a tool for publishing material authored in DocBook XML) with version 3.0, a major new upstream version. As with any important update, it had its share of problems and I created two patches that I sent upstream. I uploaded the package to experimental since we’re in freeze.

The Debian Administrator’s Handbook

Since the translation teams have been working for a few months, I wanted to put the result of their work online. I did it and I blogged about it on debian-handbook.info. By the way, we have a Polish translation that just started.

This took quite some time because many translators were not well versed with Docbook XML and its structure. So I fixed their mistakes and asked the Weblate developer (Michal Cihar) to implement new checks to avoid those basic XML mistakes.

I also added a couple of build scripts to the git repository to make it easier to rebuild translations in multiple formats. I used this opportunity to file a couple of bugs I encountered with Publican (concerning ePub output mainly, and custom brands).

I also blogged about our plans to update the book for Wheezy. Roland started to work on it but I did not have the time yet.

Debian France

The officers (president, treasurer, secretary) have just changed and we had to organize the transition. As the new president, I got administrator access on our Gandi virtual machine (france.debian.net) as well as access to our bank account. I got also got a bunch of administrative papers retracing the history of the association. Carl Chenet (the former president) gave them to me during the mini-debconf that was organized in Paris.

Indeed, Sylvestre Ledru and Mehdi Dogguy organized our second mini-debconf Paris and they did it very well. It was a great success with over 100 attendants each of the 2 days it lasted (November 24-25th). Carl managed a merchandising booth that was well stuffed (Luca Capello also brought goodies of Debian.ch)

I gave small lightning talk to present the ideas behind my Librement project (it’s about funding free software developers). BTW I have not been very good at it, it was only my second lightning talk and I have been a bit too verbose. The talk did not fit in my 5 minutes time slot ;-)

Back from the mini-debconf, I have been trying to delegate some projects (like get a real website, improve the work-flow of members management, update our server which was still running Lenny).

Julien Cristau was willing to upgrade the server did not exactly knew how to upgrade the kernel (it’s a bit special since Gandi manages the kernel on the Xen hypervisor side). So I took care of this part and also did some cleanup (adding a backup with its associated remote disk, tweaking the email configuration). And Julien completed the upgrade on November 30th.

Alexandre Delanoë volunteered to have a try at the website and Emmanuel Bouthenot has been looking a bit to see if there was something better than Galette to handle our members. It looks like we’ll stay with Galette but have to take care of upgrading it to a newer version.

I also processed the first membership applications and organized a vote to extend the board of administrators (since we have two vacant seats). On Monday, we should be back to 9 administrators.

Librement

Except for the talk during the mini-debconf, I did not do much on this project. That said I got an answer from the “Autorité de Contrôle Prudentiel” saying that I might be eligible for the exemption case (see discussion of last month) and that I should fill out a form to get a confirmation.

I also contacted Tunz.com who might be able to provide the services I need (their E-money manager product in particular). They have the required accreditation as a banking/credit institution and are willing to partner with enterprises who setup platforms where you must manage flows of money between several parties. I’m now waiting for details such as the cost of their various services.

I expect to have much more to show next month… I’m working with two developers to implement the first building blocks of all this.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in October 2012

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (120.46 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

At the start of the month, I reconfigured dpkg’s git repository to use KGB instead of the discontinued CIA to send out commit notices to IRC (on #debian-dpkg on OFTC, aka irc.debian.org).

I didn’t do anything else that affects dpkg and I must say that Guillem does not make it easy for others to get involved. He keeps all his work hidden in his private “for 1.17.x” branch and refuses to open an official “jessie” branch as can be seen from the lack of answer to this mail.

On the bright side, he deals with almost all incoming bugs even before I have a chance to take care of them. But it’s a pity that I can never review any of his fixes because they are usually pushed shortly before an upload.

Misc packaging

I helped to get #689336 fixed so that the initrd properly setups the keymap before asking for a passphrase for an encrypted partition. Related to this I filed #689722 so that cryptsetup gains a dependency ensuring that the required tools for keymap setup are available.

I packaged a new upstream version of zim (0.57) and also a security update for python-django that affected both Squeeze and Wheezy. I uploaded an NMU of revelation (0.4.13-1.2) so that it doesn’t get dropped from Wheezy (it was on the release team list of leaf packages that would be removed if unfixed) since my wife is using it to store her passwords.

I sponsored a new upstream version of ledgersmb.

Debian France

We managed to elect new officers for Debian France. I’m taking over the role of president, Sylveste Ledru is the new treasurer and Julien Danjou is the new secretary. Thank you very much to the former officers: Carl Chenet, Aurélien Jarno and Julien Cristau.

We’re in the process of managing this transition which will be completed during the next mini-Debconf in Paris so that we can exchange some papers and the like.

In the first tasks that I have set myself, there’s recruiting two new members for the boards of directors since we’re only 7 and there are 9 seats. I made a call for volunteers and we have two volunteers. If you want to get involved and help Debian France, please candidate by answering that message as soon as possible.

The Debian Handbook

I merged the translations contributed on debian.weblate.org (which led me to file this wishlist bug on Weblate itself) and I fixed a number of small issues that had been reported. I made an upload to Debian to incorporate all those fixes…

But this is still the book covering Squeeze so I started to plan the work to update it for Wheezy and with Roland we have decided who is going to take care of updating each chapter.

Librement

Progress is annoyingly slow on this project. Handling money for others is highly regulated, at least in the EU apparently. I only wanted an escrow account to secure the money of users of the service but opening this account requires either to be certified as a “payment institution” by the Autorité de contrôle prudentiel or to get an exemption from the same authority (covering only some special cases) or to sign a partnership with an established payment institution.

Being certified is out of scope for now since it requires a minimum of 125000 EUR in capital (which I don’t have). My bank can’t sign the kind of partnership that I would need. So I have to investigate whether I can make it fit in the limited cases of exemption or I need to find another “payment institution” that is willing to work with me.

Gittip uses Balanced a payment service specialized in market places but unfortunately it’s US-only if you want to withdraw money from the system. I would love a similar service in Europe…

If I can’t position Librement as a market place for the free software world (and save each contributor the hassle to open a merchant account), then I shall fallback to the solution where Librement only provides the infrastructure but no account, and developers who want to collect donations will have to use either Paypal or any other supported merchant account to collect funds.

That’s why my latest spec updates concerning the donation service and the payment service mentions Paypal and the possibility of choosing your payment service for your donation form.

Thanks

See you next month for a new summary of my activities.

Auto Mounting Windows Shares in GNOME with Gigolo and gvfs-fuse

The traditional way to mount Windows (or Samba) shares involves hardcoding the credentials in a plain-text file and some /etc/fstab entry to mount it automatically at boot time. If you don’t want to store a plain-text copy of your password, you’re bound to mount your shares interactively.

This is clearly sub-optimal and I thought that there must be a better way to handle this in the context of a GNOME desktop (which already supports connecting to such shares in the file browser). So I looked for a solution and after a bit of googling I found one.

Start by installing a few packages:

$ sudo apt-get install gigolo gvfs-fuse

Launch Gigolo, setup your shares as bookmarks and mark them as “Auto-Connect”.

During (first) connection, you will be prompted for your password and you have the possibility to store it in the GNOME keyring (and here it’s encrypted, not in plain-text!).

You should also configure the Gigolo preferences so that it starts minimized in the system tray (because we’re going to run it on session startup):

The last step is to ensure that Gigolo is executed at the start of each GNOME session. Unfortunately this GNOME feature is no longer accessible from the control center so you have to execute gnome-session-properties manually (from a terminal or the command line accessible via Alt+F2). Click on “Add” to add a new startup program:

You’re done!

The main limitation is that those shares are not real mounts, instead they are available within GNOME’s virtual file system (GVFS). If you use only 100% GNOME application, then it’s not a problem but otherwise it’s pretty annoying. You can’t “cd” in those shares from a terminal for example.

There’s a workaround though, it’s called “gvfs-fuse” and you installed it right at the start of this HOWTO. This service hooks into GVFS and exports all the virtual filesystem(s) in a real fuse-based mount that is automatically setup in ~/.gvfs/. However for this to work, the user must be in the “fuse” group. So you should run something like this:

$ sudo adduser $USER fuse

By the way, I haven’t found a way to use a non-hidden directory so if you want this directory to be more visible, I suggest that you create a symlink pointing to it.

Do you want to read more HOWTO like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

My Debian Activities in September 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (1086.48 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

I am subscribed to Launchpad’s dpkg bug tracker and I was getting annoyed with the amount of noise I got under the form of bug reports that look like “package foo failed to install/upgrade: package foo is already installed and configured”. Those reports are a combination of a bug in APT and of random other failures (often hardware related like corrupted .deb files, or I/O errors, but sometimes also real problems in other packages) but they always end up assigned on dpkg (because dpkg is outputting an error message complaining about APT’s decision to configure something that doesn’t have to be configured).

I simply don’t have the time required to manually process and inspect all those reports, so I decided to filter them at the apport level with a new “Ubuntu bug pattern” that indicates that those reports are a duplicate of LP#541595. Thanks to this, the dpkg bug count quickly went down from 130 to about 80.

Packaging

I sponsored a new upstream version of ledgersmb. I quickly updated WordPress to version 3.4.2 since it contains security relevant fixes.

I also pushed a small update of nautilus-dropbox fixing #686863 because upstream renamed the binary package that they hand out on their website from nautilus-dropbox to dropbox. Their dropbox package only conflicts with old versions of nautilus-dropbox and not with the version that Debian is shipping and thus I had to add a Conflicts on our side to forbid co-installation of both packages.

Testing wheezy’s installation

I bought a new laptop (Lenovo Thinkpad X230) and used this as an excuse to test Wheezy’s installation process. It worked mostly fine except for two things:

  1. First I noticed that it would not accept my passphrase for my encrypted partition during early boot… this turned out to be already reported as #619711 but was no longer getting any attention from the package maintainer. After some IRC discussion with Julien Cristau, we prodded Michael Prokop who had apparently already offered to take care of this issue. I tested his updated package and the result got quickly uploaded.
  2. I had weird networking problems that turned out to be related to the lack of the loopback network (i.e. on localhost). This was the result of a broken /etc/network/interfaces: it had been incorrectly modified by NetworkManager. I reported this in #688355. This issue affects people with IPv6 enabled networks.

Debian France

There’s a resurgence of activity in Debian France. Sylvestre Ledru is leading the organization of a mini-debconf in Paris on November 24-25th. And Tanguy Ortolo is now taking care of some merchandising (Polo shirts, to change from the usual T-Shirt).

I might give a talk during this mini-debconf, possibly about multi-arch.

Misc

It’s been a few months that I noticed a 2 second lag of gnome-shell everytime that smuxi (my IRC client) sent a notification. It’s very annoying, you have the impression that the entire machine freezes.

So I contacted Mirco Bauer on #smuxi and we investigated a bit. It turns out that smuxi is using an old version of the notification protocol where the picture is sent as a bytestream leading to huge dbus messages. This is clearly sub-optimal so smuxi will be fixed to be able to send the path of the picture instead of the picture itself. On the other hand, it’s really a bug of gnome-shell that it freezes during the time it takes to handle the bigger-than-usual dbus message. So I also filed a bug on GNOME Shell (Bugzilla #683829) to get this fixed.

Librement: funding free software work

I started a new project with the goal of helping free software developers to fund their free software work. It’s still mostly vaporware for now but I have a public code repository, a nice logo and lots of ideas.

If the topic is of interest to you, and you’d like to be involved, feel free to get in touch. Otherwise stay tuned.

Thanks

See you next month for a new summary of my activities.

My Debian Activities in August 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (88.41 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

This month has again been a short one since I have mostly been in vacation during the last 2 weeks.

Dpkg

Things are relatively quiet during the freeze. I only took care of fixing 3 bugs: a regression of “3.0 (quilt)” (#683547), a segfault of “dpkg-query -W -f ”” (commit) and a bad auto-completion for French users (#685863).

Testing the upgrade to wheezy

We got several reports of wheezy upgrade that failed because dpkg ran the trigger while the dependencies of the package with pending triggers are not satisfied. Unfortunately fixing this in dpkg is not without problems (see #671711 for details)… so Guillem decided to defer this fix for Jessie. My suggestion of an intermediary solution has fallen in limbo. Instead we now have to find solutions for each case where this can fail (example of failure: 680626).

Another way to avoid those errors is to ensure that triggers are run as late as possible. We can improve this in multiple ways.

The first way is to modify most triggers so that they use the “interest-noawait” directive. In that case, the packages activating the trigger will be immediately marked as configured (instead of “triggers-awaited”) and the trigger will thus not need to be run as part of further dependency solving logic. But as of today, there’s no package using this new feature yet despite a nudge on debian-devel-announce. :-(

The second way is to modify APT to use dpkg –no-triggers, and to let the trigger processing for the end (with a last “dpkg –configure -a” call). I requested this early in the wheezy timeframe but for various reasons, the APT maintainers did not act on it. I pinged them again in #626599 but it’s now too late for wheezy. I find this a bit sad because I have been using those options for the entire wheezy cycle and it worked fine for me (and I used them for a dist-upgrade on my wife’s laptop too).

It would have been good to have all this in place for wheezy so that we don’t have to suffer from the same problems during the jessie upgrade, but unless someone steps up to steer those changes, it seems unlikely to happen.

Instead, we’re back to finding klumsy work-arounds in individual packages.

Packaging

I prepared security updates for python-django (1.4.1 for unstable,
1.2.3-3+squeeze3 for stable). I packaged a new upstream version for cpputest (3.2-1). I reviewed ledgersmb 1.3.21-1 prepared by Robert James Clay and asked him to prepare another version with further fixes.

I released nautilus-dropbox 1.4.0-2 with supplementary changes of my own to support https_proxy and to display better diagnostic information when the download fails.

With the help of Paul van der Vlis and Michael Ziegler, we did what was required to be able to migrate python-django-registration 0.8 to Wheezy even though it’s a new upstream version with backwards incompatible changes. Thanks to Adam D. Barratt who unblocked the package, we now have the right version in Wheezy despite the fact that I missed the freeze deadline.

Debian France

Julien Cristau reminded the board of Debian France that we have to elect officers (President, Secretary, Treasurer) as the current officers have withdrawn. I was somewhat afraid that nobody would take over so I pinged each member to try to get new volunteers. We now have volunteers (me, Julien Danjou and Sylvestre Ledru) and we’re waiting until Julien finds some time to run the election.

Misc

With the help of DSA, I setup antispam rules for the owner@packages.qa.debian.org alias because I was getting tired by the amount of spam. In the process, they asked me to write a wiki page for dsa.debian.org to document everything so that they can refer to it for future queries. I did it but it looks like that they did not apply my patch yet.

I also tested an upstream patch for gnome-keyring (see bugzilla #681081) that reintroduces the support of forgetting GPG passphrases after a specified amount of time.

Thanks

See you next month for a new summary of my activities.

Happy Birthday Debian! And memories of an old-timer…

For Debian’s birthday, Francesca Ciceri of the Debian Publicity team suggested that developers “blog about their first experiences with Debian”. I found this a good idea so I’m going to share my own early experience. It’s quite different from what happens nowadays…

Before speaking of my early Debian experience, I have to set some context. In my youth, I have always been a Windows user and a fan of Bill Gates. That is until I got Internet at home… at that point, I got involved in Usenet and made some friends there. One of those made me discover Perl and it has been somewhat of a revelation for me who had only been programming in Visual Basic, Delphi or ObjectPal. Later the same friend explained me that Perl was working much better on Linux and that Debian Linux installs it by default so I should try this one.

I had no idea of what Linux was, but given how I loved Perl, I was eager to try his advice. So I got myself a Tri-Linux CD with Debian/RedHat/Slackware on it and started the installation process (which involved preparing boot floppies). But I did not manage to get the graphical interface working despite lots of fiddling with Xfree86’s configuration file. So I ended up installing RedHat and used it for a few months. But since many of the smart guys in my Usenet community were Debian users, I persisted and finally managed to get it to work!

After a few months of usage, I was amazed at everything that was available for free and I wanted to give back. I filed my first bug report in July 1998, I created my first Debian packages in August 1998 and I got accepted as an official Debian developer in September 1998 (after a quick chat over the phone with Martin Schulze or James Troup — I never understood the name of my interlocutor on the phone and I was so embarassed to have to use my rusty English over the phone that I never asked). That’s right, it took me less than 3 months to become a Debian developer (I was 19 years old back then).

I learned a lot during those months by reading and interacting with other Debian developers. Many of those went away from Debian in the mean time but some of them are still involved (Joey Hess, Manoj Srivastava, Ian Jackson, Martin Schulze, Steve McIntyre, Bdale Garbee, Adam Heath, John Goerzen, Marco D’Itri, Phil Hands, Lars Wirzenius, Santiago Vila, Matthias Klose, Dan Jacobowitz, Michael Meskes, …).

My initial Debian work was centered around Perl: I adopted dpkg-ftp (the FTP method for dselect) because it was written in Perl and had lots of outstanding bug reports. But I also got involved in more generic Quality Assurance work and tried to organize the nascent QA team. It was all really a lot of fun, I could take initiatives and it was clear to me that my work was appreciated.

I don’t know if you find this story interesting but I had some fun time digging through archives to find out the precise dates… if you want to learn more about what I did over the following years, I maintain a webpage for this purpose.

Looking back at 16 years of dpkg history with some figures

With Debian’s 19th anniversary approaching, I thought it would be nice to look back at dpkg’s history. After all, it’s one of the key components of any Debian system.

The figures in this article are all based on dpkg’s git repository (as of today, commit 9a06920). While the git repository doesn’t have all the history, we tried to integrate as much as possible when we created it in 2007. We have data going back to April 1996…

In this period between April 1996 and August 2012:

  • 146 persons contributed to dpkg (result of git log --pretty='%aN'|sort -u|wc -l)
  • 6948 commits have been made (result of git log --oneline | wc -l)
  • 3133612 lines have been written/modified (result of git log --stat|perl -ne 'END { print $c } $c += $1 if /(\d+) insertions/;')

Currently the dpkg source tree contains 28303 lines of C, 14956 lines of Perl and 6984 lines of shell (figures generated by David A. Wheeler’s ‘SLOCCount’) and is translated in 40 languages (but very few languages managed to translate everything, with all the manual pages there are 3997 strings to translate).

The top 5 contributors of all times (in number of commits) is the following (result of git log --pretty='%aN'|sort| uniq -c|sort -k1 -n -r|head -n 5):

  1. Guillem Jover with 2663 commits
  2. Raphaël Hertzog with 993 commits
  3. Wichert Akkerman with 682 commits
  4. Christian Perrier with 368 commits
  5. Adam Heath with 342 commits

I would like to point out that those statistics are not entirely representative as people like Ian Jackson (the original author of dpkg’s C reimplementation) or Scott James Remnant were important contributors in parts of the history that were recreated by importing tarballs. Each tarball counts for a single commit but usually bundles much more than one change. Also each contributor has its own habits in terms of crafting a work in multiple commits.

Last but not least, I have generated this 3 minutes gource visualization of dpkg git’s history (I used Planet’s head pictures for dpkg maintainers where I could find it).

Watching this video made me realize that I have been contributing to dpkg for 5 years already. I’m looking forward to the next 5 years :-)

And what about you? You could be the 147th contributor… see this wiki page to learn more about the team and to start contributing.

How to use quilt to manage patches in Debian packages

Most Debian source packages are using the “3.0 (quilt)” format. This means that Debian changes to upstream files are managed in a quilt patch series. Knowledge of quilt is thus a must if you want to get involved in some serious packaging work. Don’t worry, this tutorial will teach you how to use quilt in the context of Debian packaging.

Pre-requisites

Install the packaging-dev package to get a decent set of packages to do Debian packaging. It includes the quilt package.

$ sudo apt-get install packaging-dev

What’s quilt?

Read the description of the Debian package:

Quilt manages a series of patches by keeping track of the changes each of them makes. They are logically organized as a stack, and you can apply, un-apply, update them easily by traveling into the stack (push/pop).

Quilt is good for managing additional patches applied to a package received as a tarball or maintained in another version control system. The stacked organization is proven to be efficient for the management of very large patch sets (more than hundred patches). As matter of fact, it was designed by and for Linux kernel hackers (Andrew Morton, from the -mm branch, is the original author), and its main use by the current upstream maintainer is to manage the (hundreds of) patches against the kernel made for the SUSE distribution.

The various files involved

Stack of files PictureTo better understand the various quilt commands, you should have a basic idea of how the tool works. The “stack of patches” is maintained in a dedicated directory (“patches” by default, but in Debian packages we override this value to “debian/patches”). This directory contains the patch files and a “series” file that gives an ordered list of patches to apply. Example:

$ ls debian/patches/
01_use_stdlib_htmlparser_when_possible.diff
02_disable-sources-in-sphinxdoc.diff
03_manpage.diff
06_use_debian_geoip_database_as_default.diff
series
$ cat debian/patches/series
01_use_stdlib_htmlparser_when_possible.diff
02_disable-sources-in-sphinxdoc.diff
03_manpage.diff
06_use_debian_geoip_database_as_default.diff

When quilt is used, it also maintains some internal files in a directory of its own (it’s named “.pc”). This directory is used to know what patches are currently applied (.pc/applied-patches) and to keep backup copies of files modified by the various patches.

Configuring quilt

Before going further, you should put this in your ~/.quiltrc file:

for where in ./ ../ ../../ ../../../ ../../../../ ../../../../../; do
    if [ -e ${where}debian/rules -a -d ${where}debian/patches ]; then
        export QUILT_PATCHES=debian/patches
        break
    fi
done

This ensures that quilt will always use “debian/patches” instead of “patches” when your current directory is within a Debian source package where debian/patches exists. If you only use quilt for debian packaging, then you can be more expedient and put a simple “export QUILT_PATCHES=debian/patches” in that file.

As a matter of personal preferences, I also have those lines in my ~/.quiltrc:

QUILT_PUSH_ARGS="--color=auto"
QUILT_DIFF_ARGS="--no-timestamps --no-index -p ab --color=auto"
QUILT_REFRESH_ARGS="--no-timestamps --no-index -p ab"
QUILT_DIFF_OPTS='-p'

It enables syntax coloring for the output of several commands and customizes the generated patches to get rid of useless information. I recommend you to use those settings too.

Applying and unapplying patches, navigating in the stack of patches

When you already have a patch series, you can navigate in the stack of patches so that any subset of consecutive patches (starting from the bottom) can be applied. quilt series will list all patches known by quilt.

You can apply all patches with quilt push -a or unapply them all with quilt pop -a. You can also verify what patches are applied (quilt applied) or unapplied (quilt unapplied). quilt push applies the next unapplied patch (i.e. the patch returned by quilt next) and quilt pop unapplies the last applied patch (i.e. the patch returned by quilt top). You can give a patch name as parameter to quilt push/pop and it will apply/unapply all the patches required until the given patch is on the top.

Here are some examples of navigation in a quilt patch series

$ quilt series
02_disable-sources-in-sphinxdoc.diff
03_manpage.diff
06_use_debian_geoip_database_as_default.diff
$ quilt applied
No patches applied
$ quilt next
02_disable-sources-in-sphinxdoc.diff
$ quilt push
Applying patch 02_disable-sources-in-sphinxdoc.diff
patching file docs/conf.py

Now at patch 02_disable-sources-in-sphinxdoc.diff
$ quilt push
Applying patch 03_manpage.diff
patching file docs/man/django-admin.1

Now at patch 03_manpage.diff
$ quilt applied
02_disable-sources-in-sphinxdoc.diff
03_manpage.diff
$ quilt unapplied
06_use_debian_geoip_database_as_default.diff
$ quilt pop -a
Removing patch 03_manpage.diff
Restoring docs/man/django-admin.1

Removing patch 02_disable-sources-in-sphinxdoc.diff
Restoring docs/conf.py

No patches applied
$ quilt push 03_manpage.diff
Applying patch 02_disable-sources-in-sphinxdoc.diff
patching file docs/conf.py

Applying patch 03_manpage.diff
patching file docs/man/django-admin.1

Now at patch 03_manpage.diff
$ quilt top
03_manpage.diff

Creating a new patch

If there’s no quilt series yet, you want to create the “debian/patches” directory first.

If you already have one, you need to decide where to insert the new patch. Quilt will always add the new patch just after the patch which is currently on top. So if you want to add the patch at the end of the series, you need to run “quilt push -a” first.

Then you can use quilt new name-of-my-patch.diff to tell quilt to insert a new empty patch after the current topmost patch. In this operation quilt does almost nothing except updating the series file and recording the fact that the new patch is applied (even if still empty at this point!).

Now to add changes in this patch, you’re supposed to modify files but only after having informed quilt of your intent to modify those files. You do this with quilt add file-to-modify. At this point quilt will make a backup copy of that file so that it can generate the final patch when you’re done with your changes. It’s quite common to forget this step and to be unable to generate the patch afterward. That’s why I recommend you to use quilt edit file-to-modify which is a shorthand for doing quilt add and then opening the file in your favorite text editor.

If you want, you can review your work in progress with quilt diff.

When you’re done with the changes, you should call quilt refresh to generate the patch (or to update it if it was already existing). And since you’re a good packager, you call quilt header --dep3 -e to add DEP-3 meta-information to your patch header.

Importing an external patch

If someone else already prepared a patch, you can just import it right away with quilt import /tmp/the-patch. If you want to import it under a better name you can use the option “-P better-patch-name”. Like quilt new, it inserts the patch after the topmost patch.

Updating patches for a new upstream version

With some luck, your patches will still apply with some offsets in line numbers (quilt displays those offsets) and sometimes with some fuzz:

$ quilt push
[...]
Hunk #1 succeeded at 1362 (offset 11 lines).
Hunk #2 succeeded at 1533 with fuzz 1 (offset 4 lines).
[...]

While offsets are nothing to worry about (it means some lines were added and/or removed before the patched part), fuzz means that patch had to ignore some context lines to find the place where to apply the changes. In that case, you need to double check that patch did the right thing because it might have made changes somewhere where it shouldn’t. Also, you’ll have to update those patches because dpkg-source doesn’t accept any fuzz.

If you’re confident that all patches are correctly applied by quilt, you can refresh them to get rid of those warnings:

$ quilt pop -a
[...]
$ while quilt push; do quilt refresh; done

That was for the easy case. Now let’s deal with the case where some of the patches no longer apply. There’s one case that is usually nice to have:

$ quilt push
Applying patch 04_hyphen-manpage.diff
patching file docs/man/django-admin.1
Hunk #1 FAILED at 194.
1 out of 1 hunk FAILED -- rejects in file docs/man/django-admin.1
Patch 04_hyphen-manpage.diff can be reverse-applied

When the patch can be reverse-applied, it means that the upstream authors included the Debian patch (or that they made the same change even though you forgot to forward the patch). You can thus get rid of it:

$ quilt delete -r 04_hyphen-manpage.diff
Removed patch 04_hyphen-manpage.diff

Note that without the -r the patch is only dropped from the series file. With -r the patch file is also removed.

But there’s a less desirable case where the patch is still relevant but it doesn’t apply any longer:

$ quilt push
Applying patch 01_disable_broken_test.diff
patching file tests/regressiontests/test_utils/tests.py
Hunk #1 FAILED at 422.
1 out of 1 hunk FAILED -- rejects in file tests/regressiontests/test_utils/tests.py
Patch 01_disable_broken_test.diff does not apply (enforce with -f)

In that case, you should follow quilt’s advice to force the patch application, manually apply the parts of the patch that were rejected, and then refresh the patch.

$ quilt push -f
Applying patch 01_disable_broken_test.diff
patching file tests/regressiontests/test_utils/tests.py
Hunk #1 FAILED at 422.
1 out of 1 hunk FAILED -- saving rejects to file tests/regressiontests/test_utils/tests.py.rej
Applied patch 01_disable_broken_test.diff (forced; needs refresh)
$ vim tests/regressiontests/test_utils/tests.py    
$ quilt refresh
Refreshed patch 01_disable_broken_test.diff

Other quilt commands

You should probably read quilt’s manual page too to learn about the various other commands and options that exist.

There’s at least quilt rename new-name that you can also find useful to rename the topmost patch (you can use “-P patch-to-rename” to rename a patch which is not currently at the top).

Feedback

Please leave comments if you have suggestions of improvements, or if there are some tips that are good to know. I might incorporate them in this article.

Feel free to share this article with newbie packagers which are struggling with quilt. For your convenience, you can also refer to this article with this URL: http://raphaelhertzog.com/go/quilt

Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.