apt-get install debian-wizard

Review of my Debian related goals for 2010

January 17, 2011 by Raphaël Hertzog

Last year I shared my “Debian related goals for 2010”. I announced that I would not be able to complete them all and indeed I have not, but I have still done more than what I expected. Let’s have a look.

Translate my Debian book into English and get it published: NOT DONE

Or rather not yet done. It’s still an important project of mine and I will do it this year. When I wrote this last year, I expected to find a publisher that would take care of everything but we failed to find a suitable one so we’re going to do it ourselves.

Cleanup the dpkg perl API and create libdpkg-perl: DONE

libdpkg-perl has been introduced with dpkg 1.15.6.

Create dpkg-buildflags: DONE

dpkg-buildflags has been introduced with dpkg 1.15.7. It’s not widely used yet and it won’t really be until debhelper 7 supports it (see #544844). It would be nice to see progress on this front this year.

Ensure the new source formats continue to gain acceptance: DONE

The adoption rate has been steady, it clearly slowed down since the freeze though. I have implemented quite a few features to satisfy the needs of users, like the possibility to unapply the patches after the build, or the possibility to fail in case of unwanted upstream changes.

Design a generic vcs-buildpackage infrastructure to be integrated in dpkg-dev: NOT DONE

I believe it’s something important on the long term but it never made it to my short-term TODO list and it’s unlikely to change any time soon.

Continue fixing dpkg bugs faster than they are reported: PARTLY DONE

We have dealt with many bugs over the year, but we still have 20 more bugs than at the start of last year (370 vs 350). We’re not doing bad compared to many other Debian teams but we can still benefit from some help. Start here if you’re interested.

Enhance our infrastructure to ease interaction between contributors and to have a better view of how each package is maintained: NOT DONE

I am convinced that we need something to have a clearer idea of the commitments made by each contributor. I don’t put the same amount of care in maintaining smarty-gettext that I do on dpkg. If we had a database of the stuff that we know we don’t do well enough, it’s easier to point new contributors towards those.

Anyway, this project is still unlikely to come to the top of my priorities any time soon.

Work on the developers-reference: NOT DONE

We have switched the Maintainer field to debian-policy@lists.debian.org to have more review of the changes suggested through the bug tracking system but that has not changed much on the global situation.

I still hope to become more active on it sometimes this year. Maybe by trying to make it more fun and creating the text for some of the wishlist bugs as blog articles first.

Rewrite in C the last Perl scripts provided by the dpkg binary package: DONE

Dpkg 1.15.8 was the first version working without Perl, I announced it in July.

Integrate the 3-way merge tool for Debian changelogs in dpkg-dev: DONE

dpkg-mergechangelogs is part of dpkg-dev since 1.15.7.

I enjoy it regularly. Unfortunately it doesn’t work well for cherry-picks. Would be nice to see this fixed, anyone up to the task? 🙂

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

How to create custom RSS feeds with WordPress

January 7, 2011 by Raphaël Hertzog

WordPress has many alternate built-in feeds: per category, per tag, per author, per search-keyword. But in some cases, you want feeds built with some more advanced logic. Let’s look at the available options.

WordPress advanced built-in feeds

You can create feeds for “unions” or “intersections” of tags, you just have to use a URL like /tag/foo,bar/feed/ (all articles tagged with foo or bar) or /tag/foo+bar/feed/ (all articles tagged with foo and bar).

You can also have feeds excluding a category, although that requires you to know the category identifier (and hardcode it in the URL like this: /?feed=rss2&cat=-123 where 123 is the category id that you want to exclude).

But there’s no simple way to have a feed that excludes articles with a given tag. The best solution found involves creating a custom feed. I’ll show you a variation of this below.

Creating a custom feed

First of, install the Feed Wrangler plugin, it will take care of registering our custom feeds with wordpress.
Go to “Settings > Feed Wrangler” in your WordPress administrative interface and create a new feed, let’s call it “myfeed”.
You should now create a “feed-myfeed.php” file and put it in your current theme’s directory. The initial content of that file should be this:
<?php include('wp-includes/feed-rss2.php'); ?>
<?php include('wp-includes/feed-rss2.php'); ?>
At this point, you already have a new feed that you can access at /feed/myfeed/ (or /?feed=myfeed). It’s a complete feed like the main one.

Now, we’re going to look at ways to customize this feed. We’re going to do this by changing/overriding the default query that feed-rss2.php’s loop will use.

A feed excluding articles with a tag

If you want to create a feed that excludes the tag “foo”, you could use this:

<?php
global $wp_query;
$tag = get_term_by("slug", "foo", "post_tag");
$args = array_merge(
        $wp_query->query,
        array('tag__not_in' => array($tag->term_id))
);
query_posts($args);
include('wp-includes/feed-rss2.php');
?>

That was relatively easy, thanks to the “tag__not_in” query parameter. Now you can further customize the feed by adding supplementary query parameters to the $args array. The documentation of query_posts details the various parameters that you can use.

A feed excluding articles with a custom field (meta-data)

I went further because I did not want to use a tag to exclude some posts: that tag would have been public even if it was only meaningful to me. So I decided to use a custom field to mark the posts to exclude from my new feed. I named the field “no_syndication” and I always give it the value “1”.

This time it’s not so easy because we have no query parameter that can be used to exclude posts based on custom fields. We’re going to use the “post__not_in” parameter that can be used to exclude a list of posts. But we must first generate the list of posts that we want to exclude. Here we go:

<?php
global $wp_query;
$excluded = array();
$args_excluded = array(
    'numberposts'     => -1,
    'meta_key'        => 'no_syndication',
    'meta_value'      => 1,
    'post_type'       => 'post',
    'post_status'     => 'published'
);
foreach (get_posts($args_excluded) as $item) {
        $excluded[] = $item->ID;
}
$args = array_merge(
        $wp_query->query,
        array('post__not_in' => $excluded)
);
query_posts($args);
include('wp-includes/feed-rss2.php');
?>

A feed with modified content

You might want to add a footer to the articles that are syndicated. I use the Ozh’ Better Feed plugin for this but it applies to all your feeds.

You could do that sort of transformation only in your customized feed by using the WordPress filter named the_content_feed.

Here’s a simple example:

<?php
function myfeed_add_footer($content) {
        return $content . "<hr/>My footer here";
}
add_filter('the_content_feed', 'myfeed_add_footer');
include('wp-includes/feed-rss2.php');
?>

I’ll stop here but obviously you have lots of options and many ways to tweak all the snippets above. They have been tested with WordPress 3.0.4.

Note that in all those examples, I took care to not duplicate the code from feed-rss2.php, instead I used include() to execute it. That way my custom feeds will automatically benefit from all the future enhancements and fixes made by the WordPress developers.

But if you have to modify the XML structure of your custom feeds, you can paste the content of feed-rss2.php in your file and change it like you want…

16 Debian contributors that you can thank

January 3, 2011 by Raphaël Hertzog

I put 5 EUR in Flattr each month and I like to spend those among other Debian contributors. That’s why I keep a list of Debian people that I have seen on Flattr (for most of them I noticed through an article on Planet Debian).

Directory of Debian contributors that you can thank

I thought this list could be useful for others so I put it on a web page. Then I realized that limiting this to Flattr was not a good idea, and indeed several developers already propose multiple ways to be thanked.

I went back through my list and looked up each developer’s website to identify a “Thank me” page (it can be “Donate”, “Support me”, “Amazon Wishlist”, etc.). Obviously this means that Debian contributors who are not on Flattr do not appear on the initial list even if they have some “Thank me” page… please help me fix this and send me the missing entries if you know of any.

Click here to view the directory. The initial listing contains 16 developers and 8 of them have an additional (non-Flattr) “Thank me” link.

Please note the warning I put on the page: the inclusion in the directory should not be taken as an endorsement of the amount or quality of the work done for Debian. You are supposed to make up your own judgment when deciding who you want to thank (but the links can help you learn more about what each contributor is doing).

Flattr subscriptions explained

Since this article replaces the traditional Flattr FOSS issue for this month, I wanted to introduce a new Flattr feature I recently discovered.

With Flattr you have to click on some things every month or your monthly fee is given to a random charity. Now you can avoid this pitfall by “subscribing” to some things that you like. A subscription acts like an automatic click during a period of 3/6/12 months.

If you want to subscribe to something, you just have to click a second time on the Flattr button and you will see this:
Screenshot with Flattr subscription choices

Once you clicked on the desired duration, the subscription is recorded and the button will appear like this:
Screenshot with a subscribed flattr button

Easy, isn’t it?

PS: I installed a WordPress plugin to make it super easy to share my articles on the most common social networks.

People behind Debian: Mehdi Dogguy, release assistant

December 23, 2010 by Raphaël Hertzog

Mehdi is a Debian developer for a bit more than a year, and he’s already part of the Debian Release Team. His story is quite typical in that he started there by trying to help while observing the team do its work. That’s a recurrent pattern for people who get co-opted in free software teams.

Read on for more info about the release team, and Mehdi’s opinion on many topics. My questions are in bold, the rest is by Mehdi (except for the additional information that I inserted in italics).

Who are you?

I’m 27 years old. I grew up in Ariana in northern Tunisia, but have been living in Paris, France, since 2002.

I’m a PhD Student at the PPS laboratory where I study synchronous concurrent process calculi.

I became interested in Debian when I saw one of my colleagues, Samuel Mimram (first sponsor and advocate) trying to resolve #440469, which is a bug reported against a program I wrote. We have never been able to resolve it but my intent to contribute was born there. Since then, I started to maintain some packages and help where I can.

What’s your biggest achievement within Debian?

I don’t think I had time to accomplish a lot yet 🙂 I’ve been mostly active in the OCaml team where we designed a tool to compute automatically the dependencies between OCaml packages, called dh-ocaml. This was a joint work with Stéphane Glondu, Sylvain Le Gall and Stefano Zacchiroli. I really appreciated the time spent with them while developing dh-ocaml. Some of the bits included in dh-ocaml have been included upstream in their latest release.

I’ve also tried to give a second life to the Buildd Status Pages because they were (kind of) abandoned. I intend to keep them alive and add new features to them.

If you had a wand and could change one thing in Debian, what would that be?

Make OCaml part of a default Debian installation 😀

But, since I’m not a magician yet, I’d stick to more realistic plans:

A lot of desktop users fear Debian. I think that the Desktop installation offered by Debian today is very user-friendly and we should be able to attract more and more desktop users. Still, there is some work to be done in various places to make it even more attractive. The idea is trying to enhance the usability and integration of various tools together. Each fix could be easy or trivial but the final result would be an improved Desktop experience for our users. Our packaged software run well. So, each person can participate since the most difficult part is to find the broken scenarios. Fixes could be found together with maintainers, upstream or other interested people.
I’ll try to come up with a plan, a list of things that need polishing or fixes and gather a group of people to work on it. I’d definitely be interested in participating in such a project and I hope that I’ll find other people to help. If the plan is clear enough and has well described objectives and criteria, it could be proposed to the Release Team to consider it as a Release Goal for Wheezy.
NMUs are a great way to make things move forward. But, sometimes, an NMU could break things or have some undesirable effects. For now, NMUers have to manually track the package’s status for some time to be sure that everything is alright. It could be a good idea to be auto-subscribed to the bugs notifications of NMUed packages for some period of time (let’s say for a month) to be aware of any new issues and try to fix them. NMUing a package is not just applying a patch and hitting enter after dput. It’s also about making sure that the changes are correct and that no regressions have been introduced, etc…
Orphaned packages: It could be considered as too strict and not desired, but what about not keeping orphaned and buggy packages in Testing? What about removing them from the archive if they are buggy and still unmaintained for some period? Our ftp archive is growing. It could make sense to do some (more strict) housekeeping. I believe that this question can be raised during the next QA meeting. We should think about what we want to do with those packages before they rot in the archive.

[Raphael Hertzog: I would like to point out that pts-subscribe provided by devscripts makes it easy to temporarily subscribe to bug notifications after an Non-Maintainer Upload (NMU).]

You’re a Debian developer since August 2009 and you’re already an assistant within the Release Management team. How did that happen and what is this about?

In the OCaml team, we have to start a transition each time we upload a new version of the OCaml compiler (actually, for each package). So, some coordination with the Release Team is needed to make the transition happen.

When we are ready to upload a new version of the compiler, we ask the Release Team for permission and wait for their ack. Sometimes, their reply is fast (e.g. if their is no conflicting transition running), but it’s not always the case. While waiting for an ack, I used to check what was happening on debian-release@l.d.o. It made me more and more interested in the activities of the Release Team.

Then (before getting my Debian account), I had the chance to participate in DebConf9 where I met Luk and Phil. It was a good occasion to see more about the tools used by the Release Team. During April 2010, I had some spare time and was able to implement a little tool called Jamie to inspect the relations between transitions. It helps us to quickly see which transitions can run in parallel, or what should wait. And one day (in May 2010, IIRC), I got offered by Adam to join the team.

As members of the Release Team, we have multiple areas to work on:

Taking care of transitions during the development cycle, which means making sure that some set of packages are correctly (re-)built or fixed against a specific (to each transition) set of packages, and finding a way to tell Britney that those packages can migrate and it would be great if she also shared the same opinion. [Raphael Hertzog: britney is the name of the software that controls the content of the Testing distribution.]
Paying attention to what is happening in the archive (uploads, reported RC bugs, etc…). The idea is to try to detect unexpected transitions, blocked packages, make sure that RC bug fixes reach Testing in a reasonable period of time, etc…
During a freeze, making sure that unblock requests and freeze exceptions are not forgotten and try to make the RC bug count decrease.

There are other tasks that I’ll let you discover by joining the game.

Deciding what goes (or not) in the next stable release is a big responsibility and can be incredibly difficult at times. You have to make judgement calls all the time. What are your own criteria?

That’s a very hard to answer question (at least, for me). It really depends on the “case”. I try to follow the criteria that we publish in each release update. Sometimes, an unblock request doesn’t match those criteria and we have to decide what to accept from the set of proposed changes. Generally, new features and non-fixes (read new upstream versions) changes are not the kind of changes that we would accept during the freeze. Some of them could be accepted if they are not intrusive, easy and well defended. When, I’m not sure I try to ask other members of the Release Team to see if they share my opinion or if I missed something important during the review. The key point is to have a clear idea on what’s the benefit of the proposed update, and compare it to the current situation. For example, accepting a new upstream release (even if it fixes some critical bugs) is taking a risk to break other features and that’s why we (usually) ask for a backported fix.

It’s also worth noticing that (most of the time) we don’t decide what goes in, but (more specifically) what version of a given package goes in and try to give to the contributors an idea on what kind of changes are acceptable during the freeze. There are some exceptions though. Most of them are to fix a critical package or feature.

Do you have plans to improve the release process for Debian Wheezy?

We do have plans to improve every bit in Debian. Wheezy will be the best release ever. We just don’t know the details yet 🙂

During our last meeting in Paris last October, the Release Team agreed to organize a meeting after Squeeze’s release to discuss (among other questions) Wheezy’s cycle. But the details of the meeting are not fixed yet (we still have plenty of time to organize it… and other more important tasks to care about). We would like to be able to announce a clear roadmap for Wheezy and enhance our communication with the rest of the project. We certainly want to avoid what happened for Squeeze. Making things a bit more predictable for developers is one of our goals.

Do you think the Constantly Usable Testing project will help?

The original idea by Joey Hess is great because it allows d-i developers to work with a “stable” version of the archive. It allows them to focus on the new features they want to implement or the parts they want to fix (AIUI). It also allows to have constantly available and working installation images.

Then, there is the idea of having a constantly usable Testing for users. The idea seems nice. People tend to like the idea behind CUT because they miss some software disappearing from Testing and because of the long delays for security fixes to reach Testing.

If the Release Team has decided to remove a package from Testing, I think that there must be a reason for that. It either means that the software is broken, has unfixed security holes or was asked for the removal by its maintainer. I think that we should better try to spend some time to fix those packages, instead of throwing a broken version in a new suite. It could be argued that one could add experimental’s version in CUT (or sid’s) but every user is free to cherry-pick packages from the relevant suite when needed while still following Testing as a default branch.

Besides, it’s quite easy to see what was removed recently by checking the archive of debian-testing-changes or by querying UDD. IMO, It would be more useful to provide a better interface of that archive for our users. We could even imagine a program that alerts the user about installed software that got recently removed from Testing, to keep the user constantly aware any issue that could affect his machine. About the security or important updates, one has to recall the existence of Testing-security and testing-proposed-updates that are used specifically to let fixes reach Testing as soon as possible when it’s not possible to go through Unstable. I’m sure that the security team would appreciate some help to deal with security updates for Testing. We also have ways to speed migrate packages from Unstable to Testing.

I have to admit that I’m not convinced yet by the benefits brought by CUT for our users.

Thank you to Mehdi for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.