Dpkg and Multiarch

I had some hope to have a multiarch-enabled dpkg in sid for Christmas as Guillem told me that it was realistic. Alas Guillem got sick. We’re in January and we’re still not there.

While some of Guillem’s commits in December were related to multi-arch, the size of his pu/multiarch/master branch did not really shrink. We still have 36 commits to merge… most of the work he did was refactoring some parts of the code that were already merged. And he initiated some discussion on interface changes. I participated to those discussions hoping to bring them to a quick resolution.

I’m still maintaining my own pu/multiarch/full branch, it is based on Guillem’s branch but with further fixes that I developed and that he has not yet merged and with a change reverted (Guillem’s branch allows crossgrading packages between different architectures while dpkg does not manage this correctly yet).

I can only hope that January will be the last month of this never-ending saga. It’s been one year since I started working on this project.

Misc dpkg work

I reviewed (and later merged) a patch of Kees Cook to enhance dpkg-buildflags so that it can report which hardening features are enabled. This feature might then be used by tools like lintian to detect missing hardening features.

I mentored Gianluca Ciccarelli who is trying to enhance dpkg-maintscript-helper to take care of replacing a directory by a symlink and vice-versa.

I took care of #651993 so that dpkg-mergechangelogs doesn’t fail when it encounters an invalid version in the changelog, and of #652414 so that dpkg-source --commit accepts a relative filename when a patch file is explicitly given.

Guillem also merged a fix I developed for LP#369898.

Packaging work

WordPress 3.3 came out so I immediately packaged it. Despite my upstream bug report, they did not update their GPL compliance page which offers the corresponding sources for what’s bundled in the tarball. So I hunted for the required sources myself, and bundled them in the debian.tar.xz of the Debian source package. It’s a rather crude solution but this allowed me to close the release critical bug #646729 and to reintroduce the Flash files that were dropped in the past… which is great since the Flash-based file uploader is nicer than the one using the browser’s file field.

Quilt 0.50 came out after 2 years of (slow) development. The Debian package has many patches and several of them had to be updated to cope with the new upstream release. Fortunately some of them were also merged upstream. It still took an entire morning to complete this update. I also converted the packaging from CDBS to dh with a short rules file.

Zim 0.54 came out and I immediately updated the package since it fixed a bug that was annoying me.

Review of the ledgersmb packaging

As the sql-ledger maintainer (and a user of this software for my accounting), I have been hoping to get ledgersmb packaged as a possible replacement for it. I have been following the various efforts initiated over time but none of them resulted in a real package in Debian.

This is a real pity so I tried to fix this by offering to sponsor package uploads. That’s why I did a first review of the packaging. It took several hours because you have to explain everything that’s not good enough.

I also filed a wishlist bug against lintian (#652963) to suggest that lintian should detect improper usage of dpkg-statoverride (this is a mistake that was present in the package that I reviewed).

nautilus-dropbox work

I wanted to polish the package in time for the Ubuntu LTS release and since Debian Import Freeze is in January, I implemented some of the important fixes that I wanted.

The Debian package diverges from upstream in that the non-free binaries are installed in /var/lib/dropbox/ instead of $HOME.
Due to a bug, the files were not properly root-owned so I first fixed this (unpacking the tarball as root lead to reuse of the embedded user & group information, and those information changed recently on the Dropbox side apparently).

Then we recently identified other problems related to proxy handling (see #651065). I fixed this too because it’s relatively frequent that the initial download triggered during the package configuration fails… and in that case it’s the user that will re-trigger a package download after having given the appropriate credentials through PackageKit. Without my fix, usage of pkexec would imply the loss of the http_proxy environment variable and thus it would not be possible for a user to download through a proxy.

Last but not least I reorganized the Debian specific patches to better separate what can and should be merged upstream, from the changes that upstream doesn’t want. Unfortunately Dropbox insists on being able to auto-update their non-free binaries, they are, thus, against the installation under /var/lib/dropbox and the corresponding changes.

Book update

We’re making decent progress in the translation of the Debian Administrator’s Handbook, about 6 chapters are already translated (not yet reviewed though).

The liberation campaign is also (slowly) going forward. We’re at 67% now (thanks to 90 new supporters!) while we were only at 60% at the start of December.

Thanks

See you next month for a new summary of my activities.

2 comments | Liked this article? Click here. | My blog is Flattr-enabled.

Dpkg work

The month started with fixing newly reported bugs to prepare the 1.16.1.1 release:

• #644492: there was a flaw in a change I made to the trigger setup code. This resulted in packages being incorrectly marked as configured while they were only unpacked in a new chroot.
• #642656: dpkg-source’s refusal to build when it detects unrecorded changes broke the (mostly unused, except by the lintian test suite apparently) “2.0” source format.
• #644412: the Makefile snippet “buildflags.mk” did not respect the new maintainer specific environment variables (like DEB_CFLAGS_MAINT_APPEND) because make does not forward environment variable through $(shell …). Fixed that by manually exporting the required variables.
• I also disabled dpkg-buildpackage’s output of the build flags since it was confusing several maintainers. dpkg-buildpackage invokes debian/rules and it has no (clean) way to discover the build flags changes that maintainer request by setting the dedicated environment variables in debian/rules. Maintainers expect to see the build flags with the modifications they have requested and not just the default values set by the distribution.

With the help of Guillem, we decided on a proper fix for a race condition sometimes triggered by parallel builds when 2 concurrent dpkg-gencontrol try to update debian/files (see #642608). This ended up requiring a new package (libfile-fcntllock-perl) that the Debian perl team kindly packaged for us. With all this sorted, it was a rather easy fix.

Multiarch progress

I also spent lots of time on multiarch. I fixed an old bug that requested to support the multi-arch paths in case of cross-building (see #595144), the discussion was not really conclusive on which of the two proposed patches was better so I ended up picking my own patch because it was closer to how we currently deal with cross-building. Then I fixed 2 issues that have been reported on Ubuntu’s dpkg. The first one (LP #863675) was rather severe since an installed package ended being “disappeared” in favor of its foreign counterpart that was removed (but that had some config files left). The second one (LP #853679) only affected dselect users (apparently there are still some!) who had a self-conflicting library (Provides: foo, Conflicts: foo) installed for multiple architectures.

But the bulk of the time spent on multiarch has been spent discussing with various parties on how to go forward with multiarch. The release team commented on the schedule of the merge to ensure it makes it into Wheezy, and the Debian project leader also commented on the problems encountered so far.

While not the best course of action I could have hoped for, it certainly helped since Guillem started pushing some reviewed commits. Out of the 66 commits that were in my pu/multiarch/full branch one week ago, 20 have been merged in the master branch already.

Python-django security update and RC bug

Since python-django’s maintainer did not manage to prepare the required security updates, I stepped in and prepared version 1.2.3-3+squeeze2 for Squeeze and 1.0.2-1+lenny3 for Lenny. Unfortunately this security update is an example of how an inactive maintainer is likely to result in a severe delay for the release of security updates.

Furthermore in this specific case, the security team did not want to release the Squeeze security update until the Lenny one had been investigated (which required some time since upstream no longer supports the version in Lenny) but they did not make this very clear.

Later another release critical bug had been filed against the package (#646634) but after investigation, it turned out to be a local configuration problem so I downgraded it. I still forwarded the test suite failure to upstream authors since the test could be enhanced.

In any case, co-maintainers for python-django are welcome. I really preferred the situation where I can quietly sit down as backup maintainer…

WordPress packaging

WordPress sounds similar to python-django. I’m also “only a backup maintainer” but Giuseppe has been inactive for many months and I had to step in August because I wanted the new upstream version. I discovered a bit late that I was not subscribed to wordpress’ bugs and thus the release critical bug #639733 (that I introduced with my new upstream version) went unattended for a rather long time. Once aware, though, I quickly fixed it.

I also took the opportunity to start a discussion on debian-devel about how to deal with embedded javascript libraries and proposed a mechanism of “opportunistic replacement with symlinks”. WordPress is my testbed package for this mechanism, you can check out its debian/dh_linktree that implements the replacement logic.

The discussion has not been very interesting but at least I learned that Debian now requires that each source package shipping minified javascript files includes the original files too. It’s somewhat of a pain since it’s not a license requirement in many cases (many of those libraries are not under the GPL), but just a Debian requirement that many upstreams are not complying with. WordPress is affected and Jakub Wilk thus opened #646729 which is going to be a long-standing RC bug. To give good measures, I spent several hours investigating the case of each javascript file in the WordPress source package and I filed a new ticket on the upstream bugtracker.

Dropbox packaging work

A few months after the introduction of nautilus-dropbox to Debian and Ubuntu, I can say that the decision to only support the download of dropbox in the postinst has been a mistake. Because of this decision I had to make the postinst fail if the download failed. Even if the error message is relatively clear, this lead to many (mostly automated) bug reports on the Ubuntu side. Various other problems cropped up on top of this (trying to start dropbox while the package was not configured would result in an error because the user did not have the required rights to install the software, reinstalling the package while dropbox was running would result in a failure too, etc.).

I have fixed all those issues in the version 0.7.0-2 of the package. Now if the user has to install dropbox, it will use PolicyKit to request the root rights. The postinst will no longer fail if the dropbox download fails since it can be run later by the user. And I fixed the download code to remove the replaced file before unpacking a new file (insead of overwriting the existing file). All this work has been forwarded upstream.

The Debian Administrator’s Handbook Update

I’m glad to tell you that the translation will happen because we reached the minimal funding goal on October 22th with the help of 380 supporters.

Now the fundraising continues, but this time the goal is the liberation of the resulting book. For this to happen, we need to reach 25000 EUR in the liberation fund. So far we’re at 37% of this goal with 9400 EUR in the liberation fund (which means that 59% of the money raised has been put in the liberation fund).

Click here if you want to contribute towards the liberation of this book.

With (less than) 27 days left, it’s going to be a challenge to meet the goal, but we do like challenges, don’t we?

Misc work

• I filed #644486 against dh-make so that new packages have proper support of dpkg-buildflags from the start.
• I merged lots of patches from Luca Falavigna in the developers-reference.
• I discussed debtags integration in the PTS with Enrico Zini and Paul Wise.
• I updated publican’s packaging for the new upstream version 2.8. I had to write a new patch that I forwarded upstream.
• I filed an upstream bug on hamster-applet because just running hamster-time-tracker no longer brings its window forward.

Thanks

See you next month for a new summary of my activities.

3 comments | Liked this article? Click here. | My blog is Flattr-enabled.

Dpkg work

When I came back from Debconf, I merged my implementation of dpkg-source --commit (already presented last month). I continued some work on the hardening build flags but it’s currently stalled waiting on Kees Cook to provide the required documentation to integrate in dpkg-buildflags(1).

Following a discussion held during DebConf, Michael Prokop has been kind enough to setup a git-triggered auto-builder of dpkg (using Jenkins). You can now help us by testing the latest git version. Follow those instructions:

$ wget -O - http://jenkins.grml.org/debian/C525F56752D4A654.asc | sudo apt-key add -
$ sudo sponge /etc/apt/sources.list.d/dpkg-git <<END
deb http://jenkins.grml.org/debian dpkg main
END
$ sudo apt-get update && sudo apt-get upgrade

On the bug fixing side I took care of #640198 (minor man page update), #638291 (a fix to correctly handle hardlinks of conffiles), #637564 (the simplification logic of union dependencies was broken in some cases) and #631494 (interrupting dpkg-source while building a native source package left some temporary files around that should have been cleaned).

WordPress update

I released WordPress 3.2.1 in unstable (after having taken the time to test the updated package on my blog!) and fixed its RC bug (#625773). In the process I discovered a false positive in lintian (I reported it in 637473).

Gnome-shell-timer package

From time to time, I like to use the Pomodoro Technique. That’s why I was an user of timer-applet in GNOME 2. Now with the switch to GNOME 3, I lost this feature. But I recently discovered gnome-shell-timer, a GNOME Shell extension that provides the same features.

I created a Debian package of it and quickly filed some bugs while I was testing it (two usability issues and an encoding problem)

QA Work

During DebConf I met Giovanni Mascellani and he was interested to help the QA team. He started working on the backlog of bugs concerning the Package Tracking System (PTS) and submitted a bunch of patches. I reviewed them and merged them but since they were good, I quickly got lazy and got him added to the QA team so that he can commit his fixes alone. It also helps to build trust when you have had the opportunity to discuss face to face.

Vacation

That’s not so much compared to usual but to my defense I also took 2 weeks of vacation with my family. But somehow even in vacation I can’t really forget Debian. Here’s my son:

Thanks

See you next month for a new summary of my activities.

3 comments | Liked this article? Click here. | My blog is Flattr-enabled.

WordPress advanced built-in feeds

You can create feeds for “unions” or “intersections” of tags, you just have to use a URL like /tag/foo,bar/feed/ (all articles tagged with foo or bar) or /tag/foo+bar/feed/ (all articles tagged with foo and bar).

You can also have feeds excluding a category, although that requires you to know the category identifier (and hardcode it in the URL like this: /?feed=rss2&cat=-123 where 123 is the category id that you want to exclude).

But there’s no simple way to have a feed that excludes articles with a given tag. The best solution found involves creating a custom feed. I’ll show you a variation of this below.

Creating a custom feed

1. First of, install the Feed Wrangler plugin, it will take care of registering our custom feeds with wordpress.
2. Go to “Settings > Feed Wrangler” in your WordPress administrative interface and create a new feed, let’s call it “myfeed”.
3. You should now create a “feed-myfeed.php” file and put it in your current theme’s directory. The initial content of that file should be this:

   <?php
   include('wp-includes/feed-rss2.php');
   ?>

4. At this point, you already have a new feed that you can access at /feed/myfeed/ (or /?feed=myfeed). It’s a complete feed like the main one.

Now, we’re going to look at ways to customize this feed. We’re going to do this by changing/overriding the default query that feed-rss2.php’s loop will use.

A feed excluding articles with a tag

If you want to create a feed that excludes the tag “foo”, you could use this:

<?php
global $wp_query;
$tag = get_term_by("slug", "foo", "post_tag");
$args = array_merge(
        $wp_query->query,
        array('tag__not_in' => array($tag->term_id))
);
query_posts($args);
include('wp-includes/feed-rss2.php');
?>

That was relatively easy, thanks to the “tag__not_in” query parameter. Now you can further customize the feed by adding supplementary query parameters to the $args array. The documentation of query_posts details the various parameters that you can use.

A feed excluding articles with a custom field (meta-data)

I went further because I did not want to use a tag to exclude some posts: that tag would have been public even if it was only meaningful to me. So I decided to use a custom field to mark the posts to exclude from my new feed. I named the field “no_syndication” and I always give it the value “1″.

This time it’s not so easy because we have no query parameter that can be used to exclude posts based on custom fields. We’re going to use the “post__not_in” parameter that can be used to exclude a list of posts. But we must first generate the list of posts that we want to exclude. Here we go:

<?php
global $wp_query;
$excluded = array();
$args_excluded = array(
    'numberposts'     => -1,
    'meta_key'        => 'no_syndication',
    'meta_value'      => 1,
    'post_type'       => 'post',
    'post_status'     => 'published'
);
foreach (get_posts($args_excluded) as $item) {
        $excluded[] = $item->ID;
}
$args = array_merge(
        $wp_query->query,
        array('post__not_in' => $excluded)
);
query_posts($args);
include('wp-includes/feed-rss2.php');
?>

A feed with modified content

You might want to add a footer to the articles that are syndicated. I use the Ozh’ Better Feed plugin for this but it applies to all your feeds.

You could do that sort of transformation only in your customized feed by using the WordPress filter named the_content_feed.

Here’s a simple example:

<?php
function myfeed_add_footer($content) {
        return $content . "<hr/>My footer here";
}
add_filter('the_content_feed', 'myfeed_add_footer');
include('wp-includes/feed-rss2.php');
?>

I’ll stop here but obviously you have lots of options and many ways to tweak all the snippets above. They have been tested with WordPress 3.0.4.

Note that in all those examples, I took care to not duplicate the code from feed-rss2.php, instead I used include() to execute it. That way my custom feeds will automatically benefit from all the future enhancements and fixes made by the WordPress developers.

But if you have to modify the XML structure of your custom feeds, you can paste the content of feed-rss2.php in your file and change it like you want…

6 comments | Liked this article? Click here. | My blog is Flattr-enabled.

So I decided to fix this once for all, I have created two new blogs in addition to ouaza.com. The latter is now my private blog (in French) with everything that is not free software related. And free software will be the topic of my 2 new blogs :

• RaphaelHertzog.com is the English one (RSS feed) ;
• RaphaelHertzog.fr is the French one (RSS feed).

I have setup some HTTP redirections on various feed URLs but if you were subscribed to my main feed, you’re now redirected to the new English feed and you might want to subscribe to the French one to continue reading my articles in French. In any case, you might want to update the feed URL that you used.

I also profited from this reorganization to switch to WordPress 3.0 and the 3 blogs are hosted on the same installation thanks to the new multisite feature and to the domain mapping plugin. The main regression in those changes is that I’m back to the (new) default theme of WordPress with the standard header image. I would like to personalize them but I have no graphics skills… but if you do and would like to work on this for me, please get in touch. I’ll make sure you get your pony in return.

More changes concerning those blogs are in the pipe but that will be the topic of other posts. Thanks for following !

No comment | Liked this article? Click here. | My blog is Flattr-enabled.