My Debian Activities in March 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (227.83 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

Thanks to Guillem, dpkg with multiarch support is now available in Debian sid. The road has been bumpy, and it has again been delayed multiple times even after Guillem announced it on debian-devel-announce. Finally, the upload happened on March 19th.

I did not appreciate his announce because it was not coordinated at all, and had I been involved from the start, we could have drafted it in a way that sounded less scary for people. In the end, I provided a script so that people can verify whether they were affected by one of the potential problems that Guillem pointed out. While real, most of them are rather unlikely for typical multiarch usage.

Bernhard R. Link submitted a patch to add a new –status command to dpkg-buildflags. This command would print all the information required to understand which flags are activated and why. It would typically be called during the build process by debian/rules to keep a trace of the build flags configuration. The goal is to help debugging and also to make it possible to extract that information automatically from build logs. I reviewed his patch and we made several iterations, it’s mostly ready to be merged but there’s one detail where Bernhard and I disagree and I solicited Guillem’s opinion to try to take a decision. Unfortunately neither Guillem nor anyone else chimed in.

On request of Alexander Wirt, I uploaded a new backport of dpkg where I dropped the DEB_HOST_MULTIARCH variable from dpkg-architecture to ensure multi-arch is never accidentally enabled in other backports.

One last thing that I did not mention publicly at all yet, is that I contacted Lennart Poettering to suggest an improvement to the /etc/os-release file that he’s trying to standardize across distributions. It occurred to me that this file could also replace our /etc/dpkg/origins/default file (and not only /etc/debian_version) provided that it could store ancestry information. After some discussions, he documented new official fields for that file (ID_LIKE, HOME_URL, SUPPORT_URL, BUG_REPORT_URL). Next step for me is to improve dpkg-vendor to support this file (as a fallback or as default, I don’t know yet).

Packaging

I packaged quilt 0.60 (we’re now down to 9 Debian-specific patches, from a whopping 26 in version 0.48!) and zim 0.55.

In prevision of the next upstream version of Publican, I asked the Perl team to package a few Perl modules that Publican now requires. Less than two weeks after, all of them were in Debian Unstable. Congrats and many thanks to the Perl team (and Salvatore Bonaccorso in particular, which I happen to know because we were on the same plane during last Debconf!).

On a side note, being the maintainer of nautilus-dropbox became progressively less fun over the last months, in particular because the upstream authors tried to override some of the (IMO correct) packaging decisions that I made and got in touch with Ubuntu community managers to try to have their way. Last but not least, I keep getting duplicates of a bug that is not in my package but in the official package and that Dropbox did not respond to my query.

Book update

The translation is finished and we’re now reviewing the whole book. It takes a bit more time than expected because we’re trying to harmonize the style and because it’s difficult to coordinate the work of several volunteer reviewers.

The book cover is now almost finalized (click on it to view it in higher definitions):

We also made some progress on the interior design for the paperback. Unfortunately, I have nothing to show you yet. But it will be very nice… and made with just a LaTeX stylesheet tailored for use with dblatex.

The liberation fundraising slowed down with only 41 new supporters this month but it made a nice bump anyway thanks to a generous donation of 1000 EUR by Offensive security, the company behind Backtrack Linux. They will soon communicate on this, hopefully it will boost the operation. It would be really nice if we managed to raise the remaining 3000 EUR in the few weeks left until the official release of the book!

The work on my book dominated the month and explains my relative inactivity on other fronts. I worked much more than usual, and my wife keeps telling me that I look tired and that I should go in bed earlier… but I see the end of the tunnel: if everything goes well, the book should be released in a few weeks and I will be able to switch back to a saner lifestyle.

Thanks

See you next month for a new summary of my activities.

My Debian Activities in December 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (364.18 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg and Multiarch

I had some hope to have a multiarch-enabled dpkg in sid for Christmas as Guillem told me that it was realistic. Alas Guillem got sick. We’re in January and we’re still not there.

While some of Guillem’s commits in December were related to multi-arch, the size of his pu/multiarch/master branch did not really shrink. We still have 36 commits to merge… most of the work he did was refactoring some parts of the code that were already merged. And he initiated some discussion on interface changes. I participated to those discussions hoping to bring them to a quick resolution.

I’m still maintaining my own pu/multiarch/full branch, it is based on Guillem’s branch but with further fixes that I developed and that he has not yet merged and with a change reverted (Guillem’s branch allows crossgrading packages between different architectures while dpkg does not manage this correctly yet).

I can only hope that January will be the last month of this never-ending saga. It’s been one year since I started working on this project. :-|

Misc dpkg work

I reviewed (and later merged) a patch of Kees Cook to enhance dpkg-buildflags so that it can report which hardening features are enabled. This feature might then be used by tools like lintian to detect missing hardening features.

I mentored Gianluca Ciccarelli who is trying to enhance dpkg-maintscript-helper to take care of replacing a directory by a symlink and vice-versa.

I took care of #651993 so that dpkg-mergechangelogs doesn’t fail when it encounters an invalid version in the changelog, and of #652414 so that dpkg-source --commit accepts a relative filename when a patch file is explicitly given.

Guillem also merged a fix I developed for LP#369898.

Packaging work

WordPress 3.3 came out so I immediately packaged it. Despite my upstream bug report, they did not update their GPL compliance page which offers the corresponding sources for what’s bundled in the tarball. So I hunted for the required sources myself, and bundled them in the debian.tar.xz of the Debian source package. It’s a rather crude solution but this allowed me to close the release critical bug #646729 and to reintroduce the Flash files that were dropped in the past… which is great since the Flash-based file uploader is nicer than the one using the browser’s file field.

Quilt 0.50 came out after 2 years of (slow) development. The Debian package has many patches and several of them had to be updated to cope with the new upstream release. Fortunately some of them were also merged upstream. It still took an entire morning to complete this update. I also converted the packaging from CDBS to dh with a short rules file.

Zim 0.54 came out and I immediately updated the package since it fixed a bug that was annoying me.

Review of the ledgersmb packaging

As the sql-ledger maintainer (and a user of this software for my accounting), I have been hoping to get ledgersmb packaged as a possible replacement for it. I have been following the various efforts initiated over time but none of them resulted in a real package in Debian.

This is a real pity so I tried to fix this by offering to sponsor package uploads. That’s why I did a first review of the packaging. It took several hours because you have to explain everything that’s not good enough.

I also filed a wishlist bug against lintian (#652963) to suggest that lintian should detect improper usage of dpkg-statoverride (this is a mistake that was present in the package that I reviewed).

nautilus-dropbox work

I wanted to polish the package in time for the Ubuntu LTS release and since Debian Import Freeze is in January, I implemented some of the important fixes that I wanted.

The Debian package diverges from upstream in that the non-free binaries are installed in /var/lib/dropbox/ instead of $HOME.
Due to a bug, the files were not properly root-owned so I first fixed this (unpacking the tarball as root lead to reuse of the embedded user & group information, and those information changed recently on the Dropbox side apparently).

Then we recently identified other problems related to proxy handling (see #651065). I fixed this too because it’s relatively frequent that the initial download triggered during the package configuration fails… and in that case it’s the user that will re-trigger a package download after having given the appropriate credentials through PackageKit. Without my fix, usage of pkexec would imply the loss of the http_proxy environment variable and thus it would not be possible for a user to download through a proxy.

Last but not least I reorganized the Debian specific patches to better separate what can and should be merged upstream, from the changes that upstream doesn’t want. Unfortunately Dropbox insists on being able to auto-update their non-free binaries, they are, thus, against the installation under /var/lib/dropbox and the corresponding changes.

Book update

We’re making decent progress in the translation of the Debian Administrator’s Handbook, about 6 chapters are already translated (not yet reviewed though).

The liberation campaign is also (slowly) going forward. We’re at 67% now (thanks to 90 new supporters!) while we were only at 60% at the start of December.

Thanks

See you next month for a new summary of my activities.

My Debian Activities in November 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (310.73 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg: Multi-Arch Saga

I know lots of people are waiting the landing of multiarch in Debian unstable, and so am I. Things are progressing, though not as quickly as I hoped. Guillem merged about half of the branch between the 24th October and the 6th of November. After that most of the work happened on his personal repository in his pu/multiarch/master branch.

I verify this repository from time to time because Guillem does not inform me when he has made progress. I noticed changes on his repository on the 10th, 19th, 23th, 28th of November and on the 1th of December.

He announced a long time ago that he had some “interface changes” and up to now only wrote about the switch from the command-line option --foreign-architecture (to put in /etc/dpkg/dpkg.cfg) to the explicit command dpkg --add-architecture that only needs to be called once (see mail here). As of today (December 2th), the promised email for the other interface changes is still not here.

On November 23th, I reviewed Guillem’s work and tried to run the code in his branch. I spent the whole day chasing up regressions and submitted lots of fixes to Guillem. Thanks to the extensive test-suite I wrote when I developed my branch, it has been fairly easy to track them all down.

All the issues I reported have been fixed in the latest version of Guillem’s branch although the fixes are often slightly different from those that I submitted.

Dpkg: Squeeze Backport

At the start of the month, I uploaded what I expected to be a fairly uncontroversial backport of dpkg 1.16.1.1. It turns out I was wrong.

After some discussion, I think we came to an agreement that it was acceptable to backport dpkg-dev and libdpkg-perl only. My goal was not to bring the latest dpkg to users but to make it easier for package maintainers to backport packages using new features provided by dpkg-dev >= 1.16 (such as hardening build flags, the makefile snippets provided in /usr/share/dpkg/, or the improved dpkg-buildflags interface).

Thus I modified the source package uploaded to squeeze-backports to build only dpkg-dev and libdpkg-perl. It has been uploaded on November 23th and it’s waiting in the NEW queue for a backports admin to process it.

Misc Dpkg Work

I merged a patch of Colin Watson to be able to verify build-dependencies for a foreign architecture (taking into account the Multi-Arch status of each package listed).

I released dpkg 1.16.1.2 with two minor fixes that were sitting in the sid branch. I wanted to get rid of this so that the path is clear for a 1.16.2 upload with multiarch. The package just migrated to testing so we’re fine.

I spent another day doing dpkg bug triaging on Launchpad, we’re now down to 77 bugs with many of them tagged as incomplete and likely to expire in 2 months.

The Debian Administrator’s Handbook

eBookWe released a sample chapter so that it’s easier to have an idea of the quality of the book. The chapter covers the APT tools quite extensively. I bet that even you could learn something about apt-get/aptitude…

The crowfunding campaign on Ulule ended on November 28th.
With 673 supporters, we raised 24345 EUR. Of those, 14935 EUR have been put in the liberation fund and the rest corresponds to the various pre-orders and rewards offered.

This means that the translation will happen (we just started) but that the book is currently not going to be released under a free license. Don’t despair… As planned, the liberation campaign is carried on until the 25 K€ target is reached!

Instead of being hosted on Ulule, this permanent campaign is on the project website at debian-handbook.info/liberation/. Note that any contribution of 10 EUR or more means that you get a copy of the ebook as soon as it’s available (even if the liberation target is not reached).

Package Tracking System

At the start of the month, I filed two ideas of improvements for the PTS in the bug tracking system: #647258 is about showing outstanding bugs that relate to a release goal and #647901 is about warning maintainers that the package is affected by a current transition. If you’re a coder and want to start contributing to Debian and its QA team, those bugs could be interesting targets for a start. :-) In both cases, I have been in contact with members of the release team because those ideas require some structured data from the release team as input. Thanks to Meddi Dohguy and Niels Thykier for their help.

Later in the month, the topic of relocating the PTS once again came up. For historical reasons, the PTS was hosted on master.debian.org together with the BTS. Nowadays the BTS has its own host and it made no sense anymore to have the PTS separate from the rest of the QA services hosted on qa.debian.org (currently quantz.debian.org). So together with Martin Zobel Helas we took care to plan the migration and on November 19th we executed the plan. It worked like a charm and almost nobody noticed (only one undocumented dependency was missed, which broke the SOAP interface).

Misc Packaging Work

WordPress was broken in Ubuntu and it was also not properly synchronized with Debian due to an almost useless change on their side. Thus I requested a sync so that the working version from Debian gets imported in Ubuntu.

I sponsored the docbook-xsl 1.76.1 upload that I needed for Publican. Then I updated Publican just to discover that the test-suite triggers a new bug in fop (filed as #649476). I disabled the test-suite temporarily and uploaded Publican 2.8 to unstable. BTW, I also filed 2 upstream bugs with patches for issues I discovered while trying to generate the sample chapter of my book (see here and here).

I uploaded a version 0.7.1 of nautilus-dropbox and fixed #648215 at the same time. I made an NMU of bison to fix a long-standing release critical bug that hit me once more during an upgrade (see #645038).

I uploaded to experimental a new version of gnome-shell-timer compatible with GNOME 3.2. I took the opportunity to install from experimental the few GNOME 3.2 packages which are not yet in unstable…

Thanks

See you next month for a new summary of my activities.

My Debian activities in October 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (130.30 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg work

The month started with fixing newly reported bugs to prepare the 1.16.1.1 release:

  • #644492: there was a flaw in a change I made to the trigger setup code. This resulted in packages being incorrectly marked as configured while they were only unpacked in a new chroot.
  • #642656: dpkg-source’s refusal to build when it detects unrecorded changes broke the (mostly unused, except by the lintian test suite apparently) “2.0” source format.
  • #644412: the Makefile snippet “buildflags.mk” did not respect the new maintainer specific environment variables (like DEB_CFLAGS_MAINT_APPEND) because make does not forward environment variable through $(shell …). Fixed that by manually exporting the required variables.
  • I also disabled dpkg-buildpackage’s output of the build flags since it was confusing several maintainers. dpkg-buildpackage invokes debian/rules and it has no (clean) way to discover the build flags changes that maintainer request by setting the dedicated environment variables in debian/rules. Maintainers expect to see the build flags with the modifications they have requested and not just the default values set by the distribution.

With the help of Guillem, we decided on a proper fix for a race condition sometimes triggered by parallel builds when 2 concurrent dpkg-gencontrol try to update debian/files (see #642608). This ended up requiring a new package (libfile-fcntllock-perl) that the Debian perl team kindly packaged for us. With all this sorted, it was a rather easy fix.

Multiarch progress

I also spent lots of time on multiarch. I fixed an old bug that requested to support the multi-arch paths in case of cross-building (see #595144), the discussion was not really conclusive on which of the two proposed patches was better so I ended up picking my own patch because it was closer to how we currently deal with cross-building. Then I fixed 2 issues that have been reported on Ubuntu’s dpkg. The first one (LP #863675) was rather severe since an installed package ended being “disappeared” in favor of its foreign counterpart that was removed (but that had some config files left). The second one (LP #853679) only affected dselect users (apparently there are still some!) who had a self-conflicting library (Provides: foo, Conflicts: foo) installed for multiple architectures.

But the bulk of the time spent on multiarch has been spent discussing with various parties on how to go forward with multiarch. The release team commented on the schedule of the merge to ensure it makes it into Wheezy, and the Debian project leader also commented on the problems encountered so far.

While not the best course of action I could have hoped for, it certainly helped since Guillem started pushing some reviewed commits. Out of the 66 commits that were in my pu/multiarch/full branch one week ago, 20 have been merged in the master branch already.

Python-django security update and RC bug

Since python-django’s maintainer did not manage to prepare the required security updates, I stepped in and prepared version 1.2.3-3+squeeze2 for Squeeze and 1.0.2-1+lenny3 for Lenny. Unfortunately this security update is an example of how an inactive maintainer is likely to result in a severe delay for the release of security updates.

Furthermore in this specific case, the security team did not want to release the Squeeze security update until the Lenny one had been investigated (which required some time since upstream no longer supports the version in Lenny) but they did not make this very clear.

Later another release critical bug had been filed against the package (#646634) but after investigation, it turned out to be a local configuration problem so I downgraded it. I still forwarded the test suite failure to upstream authors since the test could be enhanced.

In any case, co-maintainers for python-django are welcome. I really preferred the situation where I can quietly sit down as backup maintainer… :-)

WordPress packaging

WordPress sounds similar to python-django. I’m also “only a backup maintainer” but Giuseppe has been inactive for many months and I had to step in August because I wanted the new upstream version. I discovered a bit late that I was not subscribed to wordpress’ bugs and thus the release critical bug #639733 (that I introduced with my new upstream version) went unattended for a rather long time. Once aware, though, I quickly fixed it.

I also took the opportunity to start a discussion on debian-devel about how to deal with embedded javascript libraries and proposed a mechanism of “opportunistic replacement with symlinks”. WordPress is my testbed package for this mechanism, you can check out its debian/dh_linktree that implements the replacement logic.

The discussion has not been very interesting but at least I learned that Debian now requires that each source package shipping minified javascript files includes the original files too. It’s somewhat of a pain since it’s not a license requirement in many cases (many of those libraries are not under the GPL), but just a Debian requirement that many upstreams are not complying with. WordPress is affected and Jakub Wilk thus opened #646729 which is going to be a long-standing RC bug. To give good measures, I spent several hours investigating the case of each javascript file in the WordPress source package and I filed a new ticket on the upstream bugtracker.

Dropbox packaging work

A few months after the introduction of nautilus-dropbox to Debian and Ubuntu, I can say that the decision to only support the download of dropbox in the postinst has been a mistake. Because of this decision I had to make the postinst fail if the download failed. Even if the error message is relatively clear, this lead to many (mostly automated) bug reports on the Ubuntu side. Various other problems cropped up on top of this (trying to start dropbox while the package was not configured would result in an error because the user did not have the required rights to install the software, reinstalling the package while dropbox was running would result in a failure too, etc.).

I have fixed all those issues in the version 0.7.0-2 of the package. Now if the user has to install dropbox, it will use PolicyKit to request the root rights. The postinst will no longer fail if the dropbox download fails since it can be run later by the user. And I fixed the download code to remove the replaced file before unpacking a new file (insead of overwriting the existing file). All this work has been forwarded upstream.

The Debian Administrator’s Handbook Update

I’m glad to tell you that the translation will happen because we reached the minimal funding goal on October 22th with the help of 380 supporters.

Now the fundraising continues, but this time the goal is the liberation of the resulting book. For this to happen, we need to reach 25000 EUR in the liberation fund. So far we’re at 37% of this goal with 9400 EUR in the liberation fund (which means that 59% of the money raised has been put in the liberation fund).

Click here if you want to contribute towards the liberation of this book.

With (less than) 27 days left, it’s going to be a challenge to meet the goal, but we do like challenges, don’t we?

Misc work

  • I filed #644486 against dh-make so that new packages have proper support of dpkg-buildflags from the start.
  • I merged lots of patches from Luca Falavigna in the developers-reference.
  • I discussed debtags integration in the PTS with Enrico Zini and Paul Wise.
  • I updated publican’s packaging for the new upstream version 2.8. I had to write a new patch that I forwarded upstream.
  • I filed an upstream bug on hamster-applet because just running hamster-time-tracker no longer brings its window forward.

Thanks

See you next month for a new summary of my activities.

Debian joins Dropbox’s officially supported platforms along with Fedora and Ubuntu

If you checkout Dropbox’s Linux download page, you will see that Debian packages are provided. Up to a few days ago, they only provided packages for Ubuntu and Fedora.

I’m glad to see that packaging nautilus-dropbox for Debian and being in touch with them due to this led to this result.

Another positive outcome is that the version 0.7.0 now ensures the origin of the downloaded binaries with GPG.