My Debian activities in October 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (130.30 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg work

The month started with fixing newly reported bugs to prepare the 1.16.1.1 release:

  • #644492: there was a flaw in a change I made to the trigger setup code. This resulted in packages being incorrectly marked as configured while they were only unpacked in a new chroot.
  • #642656: dpkg-source’s refusal to build when it detects unrecorded changes broke the (mostly unused, except by the lintian test suite apparently) “2.0” source format.
  • #644412: the Makefile snippet “buildflags.mk” did not respect the new maintainer specific environment variables (like DEB_CFLAGS_MAINT_APPEND) because make does not forward environment variable through $(shell …). Fixed that by manually exporting the required variables.
  • I also disabled dpkg-buildpackage’s output of the build flags since it was confusing several maintainers. dpkg-buildpackage invokes debian/rules and it has no (clean) way to discover the build flags changes that maintainer request by setting the dedicated environment variables in debian/rules. Maintainers expect to see the build flags with the modifications they have requested and not just the default values set by the distribution.

With the help of Guillem, we decided on a proper fix for a race condition sometimes triggered by parallel builds when 2 concurrent dpkg-gencontrol try to update debian/files (see #642608). This ended up requiring a new package (libfile-fcntllock-perl) that the Debian perl team kindly packaged for us. With all this sorted, it was a rather easy fix.

Multiarch progress

I also spent lots of time on multiarch. I fixed an old bug that requested to support the multi-arch paths in case of cross-building (see #595144), the discussion was not really conclusive on which of the two proposed patches was better so I ended up picking my own patch because it was closer to how we currently deal with cross-building. Then I fixed 2 issues that have been reported on Ubuntu’s dpkg. The first one (LP #863675) was rather severe since an installed package ended being “disappeared” in favor of its foreign counterpart that was removed (but that had some config files left). The second one (LP #853679) only affected dselect users (apparently there are still some!) who had a self-conflicting library (Provides: foo, Conflicts: foo) installed for multiple architectures.

But the bulk of the time spent on multiarch has been spent discussing with various parties on how to go forward with multiarch. The release team commented on the schedule of the merge to ensure it makes it into Wheezy, and the Debian project leader also commented on the problems encountered so far.

While not the best course of action I could have hoped for, it certainly helped since Guillem started pushing some reviewed commits. Out of the 66 commits that were in my pu/multiarch/full branch one week ago, 20 have been merged in the master branch already.

Python-django security update and RC bug

Since python-django’s maintainer did not manage to prepare the required security updates, I stepped in and prepared version 1.2.3-3+squeeze2 for Squeeze and 1.0.2-1+lenny3 for Lenny. Unfortunately this security update is an example of how an inactive maintainer is likely to result in a severe delay for the release of security updates.

Furthermore in this specific case, the security team did not want to release the Squeeze security update until the Lenny one had been investigated (which required some time since upstream no longer supports the version in Lenny) but they did not make this very clear.

Later another release critical bug had been filed against the package (#646634) but after investigation, it turned out to be a local configuration problem so I downgraded it. I still forwarded the test suite failure to upstream authors since the test could be enhanced.

In any case, co-maintainers for python-django are welcome. I really preferred the situation where I can quietly sit down as backup maintainer… 🙂

WordPress packaging

WordPress sounds similar to python-django. I’m also “only a backup maintainer” but Giuseppe has been inactive for many months and I had to step in August because I wanted the new upstream version. I discovered a bit late that I was not subscribed to wordpress’ bugs and thus the release critical bug #639733 (that I introduced with my new upstream version) went unattended for a rather long time. Once aware, though, I quickly fixed it.

I also took the opportunity to start a discussion on debian-devel about how to deal with embedded javascript libraries and proposed a mechanism of “opportunistic replacement with symlinks”. WordPress is my testbed package for this mechanism, you can check out its debian/dh_linktree that implements the replacement logic.

The discussion has not been very interesting but at least I learned that Debian now requires that each source package shipping minified javascript files includes the original files too. It’s somewhat of a pain since it’s not a license requirement in many cases (many of those libraries are not under the GPL), but just a Debian requirement that many upstreams are not complying with. WordPress is affected and Jakub Wilk thus opened #646729 which is going to be a long-standing RC bug. To give good measures, I spent several hours investigating the case of each javascript file in the WordPress source package and I filed a new ticket on the upstream bugtracker.

Dropbox packaging work

A few months after the introduction of nautilus-dropbox to Debian and Ubuntu, I can say that the decision to only support the download of dropbox in the postinst has been a mistake. Because of this decision I had to make the postinst fail if the download failed. Even if the error message is relatively clear, this lead to many (mostly automated) bug reports on the Ubuntu side. Various other problems cropped up on top of this (trying to start dropbox while the package was not configured would result in an error because the user did not have the required rights to install the software, reinstalling the package while dropbox was running would result in a failure too, etc.).

I have fixed all those issues in the version 0.7.0-2 of the package. Now if the user has to install dropbox, it will use PolicyKit to request the root rights. The postinst will no longer fail if the dropbox download fails since it can be run later by the user. And I fixed the download code to remove the replaced file before unpacking a new file (insead of overwriting the existing file). All this work has been forwarded upstream.

The Debian Administrator’s Handbook Update

I’m glad to tell you that the translation will happen because we reached the minimal funding goal on October 22th with the help of 380 supporters.

Now the fundraising continues, but this time the goal is the liberation of the resulting book. For this to happen, we need to reach 25000 EUR in the liberation fund. So far we’re at 37% of this goal with 9400 EUR in the liberation fund (which means that 59% of the money raised has been put in the liberation fund).

Click here if you want to contribute towards the liberation of this book.

With (less than) 27 days left, it’s going to be a challenge to meet the goal, but we do like challenges, don’t we?

Misc work

  • I filed #644486 against dh-make so that new packages have proper support of dpkg-buildflags from the start.
  • I merged lots of patches from Luca Falavigna in the developers-reference.
  • I discussed debtags integration in the PTS with Enrico Zini and Paul Wise.
  • I updated publican’s packaging for the new upstream version 2.8. I had to write a new patch that I forwarded upstream.
  • I filed an upstream bug on hamster-applet because just running hamster-time-tracker no longer brings its window forward.

Thanks

See you next month for a new summary of my activities.

My Debian activities in September 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (144.3 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg work

While taking care of the last details for the hardening feature in dpkg 1.16.1, I have mailed debian-devel to find volunteers to handle a hardening release goal. The objective is to ensure a large number of packages have been converted/rebuilt to actually use the new hardening build flags.

Then I prepared the draft of the announce of the dpkg 1.16.1 upload (aka Bits of dpkg maintainers sent to debian-devel-announce) which got expanded by Guillem to also cover new features since dpkg 1.15.7.

update-alternatives got some refactoring by Guillem which resulted in a regression that has been fortunately discovered by Sven Joachim. I fixed that regression and did some further cleanup inspired by the root cause of this regression (see top 4 commits here).

Note that Sven is one of the few persons who are running the git version of dpkg. Hopefully the number of tester will increase since I recently documented the APT repositories with autobuilt versions of dpkg in the wiki.

At the end of the month, I started working on a bugfix release (what’s going to be 1.16.1.1) by fixing some of the unavoidable problems discovered after an upload that accumulated more than 4 months worth of work (see top 4 commits here).

The Debian Administrator’s Handbook

I spent countless hours finalizing the launch of the crowdfunding campaign for the Debian Administrator’s Handbook and it went live on September 27th.

So far it’s on good track with more than 63% of the base funding already secured. But we still have a long way to go to reach the liberation goal (we’re at 21%). It’s still worth nothing that more than 55% of the money raised has been put in the liberation fund so there are many persons who care about getting the book freed.

More than 250 persons are supporting the project currently with an average contribution of 38 EUR. I would have expected much less for the average contribution but many more supporters. I still hope we can get more people on board with the perspective of a good DFSG-free Debian ebook.

Did you order your copy? If not, click here and fix this! 😉 By the way Paypal used to be required but it’s no longer the case, you can support the project just with your usual credit card.

Misc blog updates

Over time, I have written many useful articles for Debian users and Debian contributors. But scattered in the history, they are somewhat difficult to find. To fix this I have created some index pages listing them. Check them out:

Two new articles joined those pages this month: How to triage bugs in the Debian Bug Tracking System and Understand dpkg and don’t get stuck with a maintainer script failure.

While writing the first article, I noticed we lacked a good page showing the most buggy packages so I quickly created it (with the help of UDD): http://qa.debian.org/cgi-bin/bugs-by-source

Misc packaging work

I did a small update to the developer’s reference. Luca Falavigna submitted a patch to clarify how one is supposed to deal with meta-packages (cf #569219), I improved it and integrated the result in the SVN repository.

I upgraded nautilus-dropbox to version 0.6.9 and while doing this I discovered a bug in mergechanges (filed as #640782). I uploaded a new release of quilt mainly to add the Multi-Arch: foreign field so that it can satisfy dependencies of foreign packages (i.e. packages of a different architecture).

Django released some security advisories (tracked in #641405) and since the maintainer did not deal with the issue, I stepped up to the task (I’m a backup maintainer) and released the fixed version 1.3.1 to unstable. I took the opportunity to switch from python-support to dh_python2, and do some misc improvements to the packaging (see changelog).

I wanted to update publican to a newer version but it turned out to be not possible because Debian doesn’t have the latest version of docbook-xsl yet. I also discovered some bugs in the test suite and forwarded upstream the patch I created (see upstream bug). On top of this, fop was failing due to some java problem related to the introduction of multiarch. After having reported the bug, the java maintainers quickly released a fixed version.

So now publican is ready in the git repository but it’s waiting on the docbook-xsl update. I got in touch with the maintainer who said he would have the time to take care of it by mid-october.

Thanks

See you next month for a new summary of my activities.

Understand dpkg and don’t get stuck with a maintainer script failure

Continuing my series of articles on dpkg’s errors, this time I’ll cover a pretty common one which has several variations:

Setting up acpid (1:2.0.12-1) ...
rm: cannot remove `/etc/rc1.d/K20acpid': No such file or directory
dpkg: error processing acpid (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 acpid

Even if dpkg is failing and outputting the error message, the real problem is not in dpkg but in the installed package (acpid in the example above). As we already learned, a package contains not only files but also “maintainer scripts” that are executed at various points of the installation process (see some useful graphics to understand how they are called, thanks to Margarita Manterola).

Maintainer scripts in a package upgrade

In the introductory example it was acpid’s “post-installation script” that failed, and dpkg is only forwarding that failure back to the caller. The maintainer scripts are stored in /var/lib/dpkg/info/. You can thus inspect them and even modify them if you hit a bug and want to work around it (do this only if you understand what you do!).

One common modification is to add “set -x” at the start of the script and to retry the failing operation. That way you can see what’s executed exactly. Here’s what the output could look like after the addition of “set -x” to /var/lib/dpkg/info/acpid.postinst:

$ sudo dpkg --configure acpid
Setting up acpid (1:2.0.12-1) ...
+ dpkg --compare-versions 1:2.0.11-1 lt-nl 1.0.10-3
+ dpkg --compare-versions 1:2.0.11-1 lt-nl 1.0.6-16
+ dpkg --compare-versions 1:2.0.11-1 lt 1.0.6-6
+ rm /etc/rc1.d/K20acpid
rm: cannot remove `/etc/rc1.d/K20acpid': No such file or directory
dpkg: error processing acpid (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 acpid

This output helps you locate the command that is actually failing. Here’s it’s relatively easy since we have an error message from “rm”. And the fix is trivial too, we replace “rm” with “rm -f” so that it doesn’t fail when the file doesn’t exist (this is a fake bug I made up for this article—I just added a failing rm call—but it’s inspired by real bugs I experienced).

Maintainer scripts are supposed to be idempotent: we should be able to execute them several times in a row without bad consequences. It happens from time to time that the maintainer gets this wrong… on the first try it works, so he uploads his package and we discover the problem only later once someone ended up executing the same code twice for some reason.

Follow me on Identi.ca, Twitter, Facebook and Google+. Or subscribe to this blog by RSS or by email.

My Debian activities in August 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (91.44 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg work

When I came back from Debconf, I merged my implementation of dpkg-source --commit (already presented last month). I continued some work on the hardening build flags but it’s currently stalled waiting on Kees Cook to provide the required documentation to integrate in dpkg-buildflags(1).

Following a discussion held during DebConf, Michael Prokop has been kind enough to setup a git-triggered auto-builder of dpkg (using Jenkins). You can now help us by testing the latest git version. Follow those instructions:

$ wget -O - http://jenkins.grml.org/debian/C525F56752D4A654.asc | sudo apt-key add -
$ sudo sponge /etc/apt/sources.list.d/dpkg-git <<END
deb http://jenkins.grml.org/debian dpkg main
END
$ sudo apt-get update && sudo apt-get upgrade

On the bug fixing side I took care of #640198 (minor man page update), #638291 (a fix to correctly handle hardlinks of conffiles), #637564 (the simplification logic of union dependencies was broken in some cases) and #631494 (interrupting dpkg-source while building a native source package left some temporary files around that should have been cleaned).

WordPress update

I released WordPress 3.2.1 in unstable (after having taken the time to test the updated package on my blog!) and fixed its RC bug (#625773). In the process I discovered a false positive in lintian (I reported it in 637473).

Gnome-shell-timer package

From time to time, I like to use the Pomodoro Technique. That’s why I was an user of timer-applet in GNOME 2. Now with the switch to GNOME 3, I lost this feature. But I recently discovered gnome-shell-timer, a GNOME Shell extension that provides the same features.

I created a Debian package of it and quickly filed some bugs while I was testing it (two usability issues and an encoding problem)

QA Work

During DebConf I met Giovanni Mascellani and he was interested to help the QA team. He started working on the backlog of bugs concerning the Package Tracking System (PTS) and submitted a bunch of patches. I reviewed them and merged them but since they were good, I quickly got lazy and got him added to the QA team so that he can commit his fixes alone. It also helps to build trust when you have had the opportunity to discuss face to face. 🙂

Vacation

That’s not so much compared to usual but to my defense I also took 2 weeks of vacation with my family. But somehow even in vacation I can’t really forget Debian. Here’s my son:

Thanks

See you next month for a new summary of my activities.