How to squash Debian release critical bugs

Squashing release critical (RC) bugs is one of the most helpful way to contribute to Debian. Because those are the bugs that we need to get rid of in order to release the next stable version. Let’s see how you can help.

The process is really simple:

  1. Find a RC bug that you can fix
  2. Prepare a fixed package
  3. Send the patch to the BTS
  4. Get your fixed package uploaded

I have already covered points 2 and 3 in a former article: How to prepare patches for Debian packages. So I’m not going to repeat myself in this article, instead I’ll cover how to find a release critical bug to fix and how to get your package uploaded.

Find a RC bug with UDD

The Ultimate Debian Database (UDD) provides a neat interface to query the bug tracking system: http://udd.debian.org/bugs.cgi. The form contains many options to customize the query, we’ll try explain some.

First, at the top, you can decide to list bugs based on whether or not they affect testing/stable/unstable. Indeed, thanks to version tracking the BTS knows if a bug applies to the version of the package present in a given distribution. I suggest you pick a choice that includes “sid” because that’s where you’re going to push your fixed package. If sid is not affected by the bug, you can’t do much about it (except if you want to join the release team and help ensure updates are flowing from unstable to testing).

I usually go for “wheezy and sid” because that filters out packages which are not in testing (and are thus not blocking the next release).

Then you have lots of filters to further reduce the list of bugs. It’s important to reduce the noise as you typically have several hundreds of RC bugs at a given time. I suggest you the following settings for a start, but you’re free to try other combinations depending on what you want to achieve (I’ll give some examples later):

  • ignore bugs tagged patch (a fix is already available);
  • ignore bugs tagged pending (a fix has already been committed to the VCS repository);
  • ignore bugs newer than 7 days (that’s the time we should leave for the maintainer to take care of the bug by himself);

I usually sort the results by source package so that I immediately see if several RC bugs are affecting the source package. And I might add “popularity contest” information to get a feel of whether the package is widely used or not.

You can see the results of the suggested query here.

Other interesting queries:

Find a RC bug with rc-alert

The downside of the above method is that you get bugs on packages that you don’t know at all and that you might not care about. Some of them might even be better removed instead of fixed.

With the rc-alert command (provided by the devscripts package), you will get a list of RC bugs on packages that are installed on your computer. It should be much more restrictive. And you have command line options to further filter the list just like with the UDD form. Bonus feature: you can also filter by debtags allowing you to restrict the bugs to packages which are implemented in programming languages that you know.

$ rc-alert --include-dists TU --exclude-tags P+ --debtags implemented-in::perl,implemented-in::python
Package: gwibber
Bug:     608724
Title:   gwibber bypasses certificate checking when providing the login/password for OAuth
Flags:   [     S  ] (security)
Dists:   [STU] (stable, testing, unstable)
Debtags: implemented-in::python, interface::x11, role::program, uitoolkit::gtk, works-with::im, x11::application
[...]

Get your fixed package uploaded

Usually sending the patch to the BTS is more than enough. The maintainer or another Debian developer will pick it up at some point. But if you’re interested in becoming a Debian maintainer/developer, it might be worth to push your changes more quickly and get credited by a proper upload with your name.

Instead of just preparing a patch, you should also prepare a fixed package following the rules of a Non-Maintainer Upload. Put this package somewhere online (for example on mentors.debian.net).

When you send your patch to the BTS, include the link to the updated package and express your wish to get this update sponsored. If you haven’t heard anything from the maintainer after 2-3 days, try to find a sponsor on debian-mentors@lists.debian.org. NMUs are easier to review than completely new packages and many Debian developers like to kill RC bugs, so there’s a good chance that you’ll find someone.

Now happy RC bug-squashing! And if you really enjoy it, you can subscribe to debian-bugs-rc (~3000 mails per month)… :-)

If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

People behind Debian: Philipp Kern, Stable Release Manager

Philipp is a Debian developer since 2005 and a member of the release team since 2008. Since he took the responsibility of Stable Release Manager, the process has evolved for the best. I asked him to explain how the release team decides what’s fit for stable or not.

His work on the buildd infrastructure is also admirable but I’ll let him describe that. My questions are in bold, the rest is by Philipp.

Who are you?

I’m a 24 year old Computer Science student, living in Karlsruhe, Germany. As a student assistant I’m currently in two jobs: one is taking care of getting our university IPv6 network in shape, or rather generally getting fringe technologies into the network. The second is taking care of a s390x machine in the basement which my faculty got sponsored recently. In my spare time I tend my Debian duties and I’m active in the student council (Fachschaft) as a sys-admin, software developer and until recently as treasurer.

I got accepted as a Debian Developer in 2005. I’m only really active since I was invited to join the Release Team in early 2008 after I contributed rewrites of some scripts that got lost in a disk crash of ftp-master. In late 2008 I also took on wanna-build/buildd duties.

What’s your notable achievement within Debian?

For one I worked on the deployment of a single buildd/sbuild combination over all of our buildds and I rebased it on the sbuild in the archive several times already. All buildds basically look the same on all architectures, with only few variations. The chroots are now also created in a mostly predictable manner.

Then we finally got build autosigning after years of constant poking. However the policy decision to allow it was made by the ftp-masters anyway, for which I’m grateful, as it eases the workload of the buildd maintainers, the Security Team and the Release Team quite a bit.

On the release side there’s the integration of volatile into the proper structures of Debian. But there I’m guilty for choosing the bad name squeeze-updates is, in comparision with security’s squeeze/updates. Let’s see if we can improve that for wheezy.

The policy for stable updates have changed over time. Can you summarize what kind of updates are allowed nowadays?

All the changes that were made to the policy were driven by the aim to keep stable (and to some lesser degree oldstable) usable throughout its lifetime.

I have to admit that we got slightly more liberal in what we accept since I took over. The previous stable update policy included the fixes of critical bugs that break the system in interesting ways and security fixes. We also opened up the possibility of including important fixes for annoying bugs on a case-by-case basis.

The whole “don’t update that much” part of the stable release management is rooted in “let’s don’t change the behaviour of a running system” and “let’s have as few regressions as possible”. We currently try to counter that with a review process that only allows self-contained fixes that were tested in unstable first, if applicable.

In the future I’d also like to have a feedback process that includes the users of the packages to report problems more early. In fact that’s also why we now send calls for testing to debian-stable-announce one week in advance (see this mail as an example).

So stable is updated through point releases that happen roughly every two months for stable and roughly every four for oldstable. They are accompanied by announcements and new CD/DVD sets.

To be able to push some updates with less delay to our users and to avoid that they have to pick them from proposed-updates, there is stable-updates. The current policy for this suite is available in this post on debian-devel-announce. It boils down to everything urgent like tzdata, regression fixes and volatile packages. We noticed that some packages in stable just weren’t useful anymore when they got updated through volatile. Hence those are folded into stable, too. We also relaxed the rules a bit so that leaf packages like clive, that rely on external websites not to change their URLs, can be updated too.

If somebody wonders what happened to the “and a half” releases… Those were intended to mark the inclusion of new hardware support (mostly by including a co-installable new kernel version). This kind of flag point release is thankfully no longer needed because our marvellous kernel team now backports certain hardware drivers to the kernel version in stable. They have some trouble soliciting test feedback for them, though. It would be cool if more people using stable could respond to their calls for testing.

What are your plans for Debian Wheezy?

Luckily the wanna-build/buildd side isn’t much dependent on release cycles. The stable release management also just takes care of what gets dumped on them by the testing RMs. ;-)

The near-term goals are certainly:

  • getting d-i to use some development process in the archive that’s compatible with the normal flow of packages into testing and
  • hopefully getting wanna-build to use a sane SQL layout and hence better performance

Apart from that my work is mainly squashing the bugs on buildd.debian.org for the wanna-build part, so if you want something fixed, you need to report it, I’m afraid.

You have several roles within Debian, in particular stable release manager and member of the wanna-build team. Which role do you enjoy more and why?

So the regular duties of the SRM, apart from setting up policies, are reviewing requests for uploads to proposed-updates, accepting them, chasing builds and scheduling point releases. It’s mainly steady work. As for the Release Team it’s working in a fabulous team where everyone’s doing an awesome job.

For me the wanna-build role is holding the things together and fixing up stuff that breaks while the world continues to evolve. And occassionally taking a stab at the ftp-masters. So rather… different.

My heart is probably siding with the buildd side because it’s infrastructure but release work is fun, too.

What’s the biggest problem of Debian?

Mainly that we’re not quick in picking up new technologies and that we got slow on innovation. Distro-wide changes are usually a very big effort to coordinate and to finally complete. I guess people should be able to push fixes for release goals more eagerly into the archive.

Also it seems to me that many DDs are actually disengaging from the release process, especially during the freeze but also before, which makes me a bit sad. However I’m still not sure how we could counter that. As we’re all volunteers you can’t suddenly make others love the idea of releasing something together, I guess.

Do you have wishes for Debian Wheezy?

It’s getting old, but there’s certainly multiarch. Also a usable Python 3 is something I’d really appreciate.

Is there someone in Debian that you admire for their contributions?

Peter Palfrader did an awesome job of getting a nicely working team together for DSA and got many tasks as automated as they are now. It’s a pleasure to work with them and setting up a new buildd is, for example, a real breeze. He also developed a sane archive snapshot service, kudos for that.


Thank you to Philipp for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

Do we need project-wide support for Debian rolling?

The discussion about Debian rolling started sooner than expected on debian-devel (see the thread here). I initially wanted to iron out the biggest problems through discussions on my blog and try to submit a somewhat polished proposal… instead we ended up discussing the same things both on -devel and on my blog.

That said it’s not that bad (except for the time I lost to have similar conversations in both places) because the debian-devel discussions included members of the release team and it looks like they are not fundamentally opposed to the idea.

Despite this, the introduction of a “frozen” suite branched off from “testing/rolling” is not really consensual (yet?). But the idea of officially supporting testing on a best-effort basis appears to have almost no opposition.

While some will undoubtedly believe that this is a useless exercise, I believe it would help if the project stated this in a somewhat official manner. My answer to the question in the title is thus:

YES

I am thus considering to submit a general resolution to that effect. My current draft is below.

Title: Debian endorses usage of testing by end-users, and renames it to rolling

The Debian project recognizes that the Debian testing distribution—initially created to make it easier to prepare and test the next stable release—has become a useful product of its own. It satisfies the needs of users who are looking for the latest stable versions of software and who can cope (or even appreciate) a system that’s constantly evolving.

The Debian project decides to endorse this usage and will strive to provide a good experience to users of “testing”. To better communicate this policy change to our users, “testing” will be renamed “rolling”.

While we believe that this is a good move, we would like to remind our users that Debian is a volunteer project and that our resources are not infinite. Package maintainers are contributing to Debian on a best-effort basis. This means that they might not be able to properly support their package(s) in all distributions. In that case, the project recommends that maintainers apply the following priorities:

  1. Support in stable (security updates, release critical bugs)
  2. Preparation of the next stable release
  3. Support in rolling

Note that this general resolution could have amendments with s/rolling/current/ and that would solve the bikeshedding over the name that started several times already.

I deliberately separated “Preparation of the next stable release” and “Support in rolling” so that priorities are clear even if we decide to not freeze “rolling” and to introduce a “frozen” distribution to finalize the next stable release.

I also did not go into too much details on the implications that it might have, it’s best to leave that up to each contributor/team/etc.

I hesitated to add a paragraph stating that we want to try to gradually improve the usability of testing but in the end I think it would be somewhat redundant. We’re always trying to do our best when we decide to take on something.

All comments welcome (even if you just agree and would be willing to second such a GR).

Update: I will tweak the draft included in this article when I get good suggestions. Thanks to Lucas Nussbaum for the first one.

No freeze of Debian’s development, what does it entail?

The main feature of rolling is that it would never freeze. This is not without consequences.

Possible consequences

It can divert developers from working on the release

No freeze means developers are free to continue their work as usual in unstable. Will it be more difficult to release because some people will spend their time working on a new upstream version instead of fixing RC bugs in the version that is frozen? Would we lose the work of the people who do lots of NMU to help with the release?

It makes it more difficult to cherry-pick updates from unstable

No freeze also means that unstable is going to diverge sooner from testing and it will be more difficult to cherry-pick updates from unstable into testing. And the release team likes to cherry-pick updates that have been tested in unstable because updates that comes through testing-proposed-updates have often not been tested and need thus a more careful review.

Frozen earth

My responses to the objections

Those are the two major objections that we’ll have to respond to. Let’s try to analyze them a bit more.

It’s not testing vs rolling

On the first objection I would like to respond that we must not put “testing” and “rolling/unstable” in opposition. The fact that a contributor can’t do its work as usual in unstable does not mean that he will instead choose to work on fixing RC bugs in testing. Probably that some do, but in my experience we simply spend our time differently, either working more on non-Debian stuff or doing mostly hidden work that is then released in big batches at the start of the next cycle (which tends to create problems of its own).

I would also like to argue that by giving more exposure to rolling and encouraging developers to properly support their packages in rolling, it probably means that the overall state of rolling should become gradually better compared to what we’re currently used to with testing.

The objection that rolling would divert resources from getting testing in a releasable shape is difficult to prove and/or disprove. The best way to have some objective data would be to setup a questionnaire and to ask all maintainers. Any volunteer for that?

Unstable as a test-bed for RC bugfixes?

It’s true that unstable will quickly diverge from testing and that it will be more difficult to cherry-pick updates from unstable into testing. This cannot be refuted, it’s a downside given the current workflow of the release team.

But I wonder if the importance of this workflow is not overdone. The reason why they like to cherry-pick from unstable is because it gives them some confidence that the update has not caused other regressions and ensures that testing is improving.

But if they’re considering to cherry-pick an update, it’s because the current package in testing is plagued by an RC bug. Supposing that the updated package has introduced a regression, is it really better to keep the current RC bug compared to trading it for a new regression? It sure depends on the precise bugs involved so that’s why they prefer to know up-front about the regression instead of making a blind bet.

Given this, I think we should use testing-proposed-updates (tpu) as a test-bed for RC bug fixes. We should ask beta-testers to activate this repository and to file RC bugs for any regression. And instead of requiring a full review by a release manager for all uploads to testing-proposed-updates, uploads should be auto-accepted provided that they do not change the upstream version and that they do not add/remove binary packages. Other uploads would still need manual approval by the release managers.

On top of this, we can also add an infrastructure to encourage peer-reviews of t-p-u uploads so that reviews become more opportunistic instead of systematic. Positive reviews would help reduce the aging required in t-p-u before being accepted into testing.

This changes the balance by giving a bit more freedom to maintainers but still keeps the safety net that release managers need to have. It should also reduce the overall amount of work that the release team has to do.

Comments welcome

Do you see other important objections beside the two that I mentioned?

Do you have other ideas to overcome those objections?

What do you think of my responses? Does your experience infirm or confirm my point of view?

Towards Debian rolling: my own Debian CUT manifesto

As you might know, I’m of one the people who promote the idea of having a Constantly Usable Testing (CUT). I will post a set of articles on this topic to help me clarify my ideas and to get some early feedback of other Debian contributors. I plan to use the result of this process to kickstart a broader discussion on debian-devel (where I hope to get the release team involved).

Let’s start by defining my own objectives with Debian CUT. I won’t speak of the part of Debian CUT that plans to make regular snapshots of testing with installable media because that’s not where I will invest my time. I do hope other people will do it though. Instead I will concentrate on changes that would improve Debian testing for end-users. I consider that testing (in its current form) is largely usable already but there are two main obstacles to overcome.

Testing without freezes = rolling

The first one is that testing is no longer testing during freezes. The regular flow of new upstream versions—that makes testing so interesting for many users—is stalled because we’re using testing to finalize the next stable version. That’s why I’d like to introduce a new suite called “rolling” that would work like the current testing except that it never freezes. Testing would no longer be a permanent suite, it would only exist during the freeze and it would be branched off from rolling.

Rolling should be a supported distribution

The second obstacle is not really technical. We must advertise rolling as a distribution that ordinary people can use but we should make it clear that it’s never going to have the same level of polish that a stable release can have. And to back up this assertion, we must empower Debian developers to be able to provide good support of their software in rolling, this probably means using “rolling-proposed-updates” more often to push fixes and security updates when the natural flow of updates is blocked by transitions or other problems. Some maintainers won’t have the time required to provide the same level of support as they currently do for a stable release and that’s okay, it’s not worse than the current testing. The goal is to treat it on a best-effort basis and to try to gradually improve the situation.

My goal for wheezy

This is the the minimal implementation of CUT that I would like to target in the wheezy timeframe. Having testing as a temporary branch of rolling is not a strict requirement, although I’d argue that it’s important to not waste resources towards maintaining two separate yet very similar distributions.

Should Debian embrace those goals?

Ignoring all possible problems that will surface while trying to implement those goals, can we agree that Debian should embrace those goals because it means providing a better service to a class of users that is not satisfied by the current stable release?

I will come back to the expected problems in a subsequent post and we will have the opportunity to discuss them. Here I just want to see whether we can have broad buy-in on the principles behind Debian rolling and CUT.

People behind Debian: Adam D. Barratt, release manager

Adam D. Barratt is a Debian developer since 2008, in just a few years he got heavily involved to the point of being now “Release manager”, a high responsibility role within the community. He worked hard with the other members of the release team to make Squeeze happen.

You could expect the release managers to have some rest after a big release, but it’s not really the case. With the long freeze, loads of “transitions” have accumulated and they are now busy to get all those updated packages in the new testing (wheezy). Despite this Adam took some time to answer my questions.

He shares with us his impression on the Squeeze release, his opinion on time-based freezes (regular/predictable freeze) and much more. Read on. My questions are in bold, the rest is by Adam.

Who are you?

I’m a 31 year old software developer and part-time sysadmin for a software and IT services company based in the south of England. I have no children, no pets and a long-suffering partner who puts up with me spending far too much time tinkering with things and people making fun of her Macbook during Debconf.

As well as being on the release team, I’m a member of the maintainer teams for devscripts and lintian.

Can you describe your journey in Debian and in the release team?

I was introduced to Debian as part of an infrastructure upgrade at work, moving from a set of Red Hat and Solaris-based systems. As part of that, we submitted some bugs for issues we found during the upgrade and for small patches we included in some software to add extra functionality we wanted. From that starting point I became more interested in Debian in general and began following some of the mailing lists and IRC channels.

When Julian Gilbey asked for help with the maintenance of devscripts, I submitted some patches for some of the outstanding bug reports and was invited to join the team which was being created to handle maintenance for the package. One of the then Release Managers was also on the team and asked if I’d be interested in working on a couple of updates they wanted to the scripts which generate the proposed-updates overview pages. I added the new functionality which was merged in to the live scripts and a little while later I was invited to join the team, shortly before Debconf 9.

As most readers will be aware, we unfortunately reached a point during last year where we didn’t have anyone filling the Release Manager role. During that period, I became more active in handling transitions and requests for updates to stable and as time went on more people started to suggest that I should put myself forward for the position, or refer to me as already being RM. I procrastinated over the decision for some time but after discussions during Debconf 10 I came round to the idea that we should have the RM role filled again and agreed to take it on, together with Neil. The rest, as they say…

How much time do you usually spend working for the release team ?

I’ve been trying to work out how to usefully answer this question. My initial answer was “approximately two hours each day”, but the longer I thought about it the more I started debating exactly what I should include under the umbrella of release work; after some to-and-fro I’ve decided to stick with my initial answer.

During periods when Debian is frozen and particularly in the lead up to the release that time commitment increases significantly, particularly over weekends. I’m reliably informed that at that point the correct answer to the question is “too much time”. :-)

What’s your own retrospective of the Squeeze release? What went well and what needs to be improved?

Overall, I believe the release went well and that we should all be proud of the Squeeze release. The parts of the release cycle which highlighted the need for improvement all share, imo, a single root cause – communication, particularly around freeze-related plans. We worked hard during the freeze itself to improve our communication with the rest of the project and want to continue in that vein during the Wheezy cycle.

One thing that I personally found quite difficult at times before the freeze was keeping track of the transitions which were still waiting for a place in the queue; it’s also something that we could improve on at this early stage of the Wheezy cycle. In order to help us keep a clear overview of requests for transitions, stable updates and binNMUs, it would be helpful if they could be filed as appropriately user-tagged bugs. This not only allows us to easily get an overview of the status of requests from the BTS but also aids transparency by allowing anyone else to do so; as a useful additional feature, it means that we can use the BTS’s blocking functionality to indicate reasons why a request cannot be fulfilled right now.

Are you in favor of time based freeze?

I think there’s merit in having a time frame that we can work towards in order to achieve the goals which we set ourselves for the release, as individual maintainers, maintenance teams and a project. I do have concerns that even with such a time frame in place there will still be uploads made very close to the proposed freeze point and transitions which may be unfinished, for example because of an unforeseen entanglement with or reliance on the transition of another package.

One thing I’m interested in is how exact and specific that time frame should be and the balance between predictability and being able to achieve everything we want for a great release; this is something we can cover in the debate on this subject which I know many people have strong opinions about.

What are your plans for Debian Wheezy?

The Wheezy to-do list I started before the final Squeeze release begins “multiarch, multiarch, multiarch”. It looks like we’re finally going to get that achieved during this release cycle, thanks to a great deal of hard work from various people. I’m also interested in seeing the C.UTF-8 locale standardised throughout Debian and continuing to work on our tools and processes to make tracking of transitions and stable updates simpler (or at least appearing to be so :-) and more transparent.

With my package maintenance hats on, I’d like to help ensure that both devscripts and lintian are able to keep pace with changes in the development landscape in Debian (e.g. more useful package diffing for source format v3 packages) and continue to be tools that are an integral part of package development in Debian.

Some people (including me) would like a rolling distribution constantly usable by end-users. Do you think that the release process currently geared towards producing “stable” can be accommodated to support this?

I’m not yet convinced that the concept of a rolling, “constantly usable” distribution can be easily integrated in to the workflow that exists around preparing stable releases in Debian. The “testing” distribution was created as, and continues to be used as, a tool to enable the release team to create the next stable release – that it happens to be something that people can use every day for much of the time is mostly a happy side-effect of the fact that we don’t gratuitously break it, but is by no means guaranteed to be the case early in the release cycle or during large, disruptive, transitions.

It’s been suggested that “testing” and “rolling” could be basically the same for most of the cycle, with “rolling” then continuing to be updated when testing is frozen. This would essentially mean an extra suite which is only used for a few months every couple of years or so, which is one of the things that “testing” was intended to avoid (i.e. the old “frozen” suite) and seems like a lot of overhead to introduce in order to reduce disruption to some users during the freeze. The early part of the release cycle also tends to include a number of larger transitions which often require packages to either be removed from testing or broken as part of migrating the transition, if they are not able to be successfully updated in time.

What’s the biggest problem of Debian?

The thing that I’ve been noticing myself becoming frustrated by recently is a tendency to debate the minor details of proposals, rather than concentrating on getting the key points right to begin with. Clearly for some projects such as multiarch the details may be as important as the big picture, but in most cases the people working on a development should be allowed to look after the smaller details themselves.

That’s not meant to imply that feedback from other parts of the project should not be welcomed, simply that if we consider Debian to be a “do-ocracy” then we need to permit people the freedom to “do”.

Is there someone in Debian that you admire for their contributions?

All previous release managers, for making the job look much easier than it seems when you’re in the “hot seat”. :-)

Outside of the release team, Joey Hess for his contributions to various parts of the Debian development environment over the years, such as debhelper and debian-installer, and Colin Watson for his enviable willingness to tackle a wide variety of different projects within Debian.


Thank you to Adam for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

Debian Cleanup Tip #3: get rid of third-party packages

Last week, we learned how to get rid of obsolete packages. This time, we’re going to learn how to bring back your computer to a state close to a “pure” Ubuntu/Debian installation.

Thanks to the power of APT, it’s easy to add new external repositories and install supplementary software. Unfortunately some of those are not very well maintained. They might contain crappy packages or they might simply not be updated. An external package which was initially working well, can become a burden on system maintenance because it will be interfering with regular updates (for example by requiring a package that should be removed in newer versions of the system).

So my goal for today is to teach you how to identify the packages on your system that are not coming from Debian or Ubuntu. So that you can go through them from time to time and keep only those that you really need. Obsolete packages are a subset of those, but I’ll leave them alone. We took care of them last week.

Each (well-formed) APT repository comes with a “Release” file describing it (example). They provide some values that can be used by APT to identify packages contained in the repository. All official Debian repositories are documented with Origin=Debian (and Origin=Ubuntu for Ubuntu). You can verify the origin value associated to each repository (if any) in the output of apt-cache policy:

[...]
 500 http://ftp.debian.org/debian/ lenny/main i386 Packages
     release v=5.0.8,o=Debian,a=stable,n=lenny,l=Debian,c=main
     origin ftp.debian.org
[...]

From there on, we can simply ask aptitude to compute a list of packages which are both installed and not available in an official Debian repository:

$ aptitude search '?narrow(?installed, !?origin(Debian))!?obsolete'
or
$ aptitude search '~S ~i !~ODebian !~o'

You can replace “search” with “purge” or “remove” if you want to get rid of all the packages listed. But you’re more likely to want to remove only a subset of carefully chosen packages… you’re probably still using some of the software that you installed from external repositories.

With synaptic, you can also browse the content of each repository. Click on the “Origin” button and you have a list of repositories. You can go through the non-Debian repositories and look which packages are installed and up-to-date.

But you can do better, you can create a custom view. Click on the menu entry “Settings > Filter”. Click on “New” to create a new filter and name it “External packages”. Unselect everything in the “Status” tab and keep only “Installed”.

Go in the “Properties” tab and here add a new entry “Origin” “Excludes” “ftp.debian.org”. In fact you must replace “ftp.debian.org” with the hostname of your Debian/Ubuntu mirror. The one that appears on the “origin” line in the output of apt-cache policy (see the excerpt quoted above in this article).

Note that the term “Origin” is used to refer to two different things, a field in the release file but also the name of the host for an APT repository. It’s a bit confusing if you don’t pay attention.

Close the filters window with OK. You now have a new listing of “External packages” under the “Custom Filters” screen. You can see which packages are installed and up-to-date and decide whether you really want to keep it. If the package is also provided by Debian/Ubuntu and you want to go back to the version provided by your distribution, you can use the “Package > Force version…” menu entry.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

Debian 6.0 is out, Wheezy kicks off

As you probably already know, Debian released Squeeze aka Debian 6.0 this week-end. It was a really great week-end.

I saw quite a few release already, but none with so much online social activity. Alexander and Meike Schmehl live commented the release on the Debian identi.ca account and Joey Hess held his Debian Party Line.

On top of this, the team in charge of the website rolled out a new design on quite a few online services during the week-end including www.debian.org, wiki.debian.org, planet.debian.org and more.

Congratulations to all the people who made this happen (and the release team in particular). It’s great to see Debian achieve all of this.

Do you know that it’s the third time in a row that we manage to release in the 18-24 months timeframe? 3.1: June 2005 → 4.0: April 2007 → 5.0: February 2009 → 6.0: February 2011.

Yes, despite our size and the fact that we are all volunteers, we have managed to stick to a reasonable schedule for a stable distribution that is deployed on a large scale.

Wheezy kicks off

The best part of the release for us — the developers — is that wheezy is now open for development and we can work on new features for the next release. ;-)

And it started quickly: according to UDD, wheezy already features 488 new source packages that are not in squeeze, 1713 updated source packages and among those 1246 are new upstream versions.

I really look forward to the upcoming projects and related discussions.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.


For the curious, here are the UDD queries I used:

# Updated packages in wheezy
select count(source) from sources_uniq as su where (select version from sources_uniq where release='squeeze' and distribution='debian' and source = su.source) < su.version and release='wheezy' and distribution='debian';
# Updated with a new upstream version
select count(source) from sources_uniq as su where debversion(regexp_replace((select version from sources_uniq where release='squeeze' and distribution='debian' and source = su.source), '-.*', '')) < debversion(regexp_replace(su.version, '-.*', '')) and release='wheezy' and distribution='debian';
# New package in wheezy
select count(source) from sources_uniq as su where source not in (select source from sources_uniq where release='squeeze' and distribution='debian' and source = su.source) and release='wheezy' and distribution='debian';

People behind Debian: Mehdi Dogguy, release assistant

Mehdi Dogguy

Picture of Mehdi taken by Antoine Madet

Mehdi is a Debian developer for a bit more than a year, and he’s already part of the Debian Release Team. His story is quite typical in that he started there by trying to help while observing the team do its work. That’s a recurrent pattern for people who get co-opted in free software teams.

Read on for more info about the release team, and Mehdi’s opinion on many topics. My questions are in bold, the rest is by Mehdi (except for the additional information that I inserted in italics).

Who are you?

I’m 27 years old. I grew up in Ariana in northern Tunisia, but have been living in Paris, France, since 2002.

I’m a PhD Student at the PPS laboratory where I study synchronous concurrent process calculi.

I became interested in Debian when I saw one of my colleagues, Samuel Mimram (first sponsor and advocate) trying to resolve #440469, which is a bug reported against a program I wrote. We have never been able to resolve it but my intent to contribute was born there. Since then, I started to maintain some packages and help where I can.

What’s your biggest achievement within Debian?

I don’t think I had time to accomplish a lot yet :) I’ve been mostly active in the OCaml team where we designed a tool to compute automatically the dependencies between OCaml packages, called dh-ocaml. This was a joint work with Stéphane Glondu, Sylvain Le Gall and Stefano Zacchiroli. I really appreciated the time spent with them while developing dh-ocaml. Some of the bits included in dh-ocaml have been included upstream in their latest release.

I’ve also tried to give a second life to the Buildd Status Pages because they were (kind of) abandoned. I intend to keep them alive and add new features to them.

If you had a wand and could change one thing in Debian, what would that be?

Make OCaml part of a default Debian installation :D

But, since I’m not a magician yet, I’d stick to more realistic plans:

  1. A lot of desktop users fear Debian. I think that the Desktop installation offered by Debian today is very user-friendly and we should be able to attract more and more desktop users. Still, there is some work to be done in various places to make it even more attractive. The idea is trying to enhance the usability and integration of various tools together. Each fix could be easy or trivial but the final result would be an improved Desktop experience for our users. Our packaged software run well. So, each person can participate since the most difficult part is to find the broken scenarios. Fixes could be found together with maintainers, upstream or other interested people.

    I’ll try to come up with a plan, a list of things that need polishing or fixes and gather a group of people to work on it. I’d definitely be interested in participating in such a project and I hope that I’ll find other people to help. If the plan is clear enough and has well described objectives and criteria, it could be proposed to the Release Team to consider it as a Release Goal for Wheezy.

  2. NMUs are a great way to make things move forward. But, sometimes, an NMU could break things or have some undesirable effects. For now, NMUers have to manually track the package’s status for some time to be sure that everything is alright. It could be a good idea to be auto-subscribed to the bugs notifications of NMUed packages for some period of time (let’s say for a month) to be aware of any new issues and try to fix them. NMUing a package is not just applying a patch and hitting enter after dput. It’s also about making sure that the changes are correct and that no regressions have been introduced, etc…

  3. Orphaned packages: It could be considered as too strict and not desired, but what about not keeping orphaned and buggy packages in Testing? What about removing them from the archive if they are buggy and still unmaintained for some period? Our ftp archive is growing. It could make sense to do some (more strict) housekeeping. I believe that this question can be raised during the next QA meeting. We should think about what we want to do with those packages before they rot in the archive.

[Raphael Hertzog: I would like to point out that pts-subscribe provided by devscripts makes it easy to temporarily subscribe to bug notifications after an Non-Maintainer Upload (NMU).]

You’re a Debian developer since August 2009 and you’re already an assistant within the Release Management team. How did that happen and what is this about?

In the OCaml team, we have to start a transition each time we upload a new version of the OCaml compiler (actually, for each package). So, some coordination with the Release Team is needed to make the transition happen.

When we are ready to upload a new version of the compiler, we ask the Release Team for permission and wait for their ack. Sometimes, their reply is fast (e.g. if their is no conflicting transition running), but it’s not always the case. While waiting for an ack, I used to check what was happening on debian-release@l.d.o. It made me more and more interested in the activities of the Release Team.

Then (before getting my Debian account), I had the chance to participate in DebConf9 where I met Luk and Phil. It was a good occasion to see more about the tools used by the Release Team. During April 2010, I had some spare time and was able to implement a little tool called Jamie to inspect the relations between transitions. It helps us to quickly see which transitions can run in parallel, or what should wait. And one day (in May 2010, IIRC), I got offered by Adam to join the team.

As members of the Release Team, we have multiple areas to work on:

  1. Taking care of transitions during the development cycle, which means making sure that some set of packages are correctly (re-)built or fixed against a specific (to each transition) set of packages, and finding a way to tell Britney that those packages can migrate and it would be great if she also shared the same opinion. [Raphael Hertzog: britney is the name of the software that controls the content of the Testing distribution.]
  2. Paying attention to what is happening in the archive (uploads, reported RC bugs, etc…). The idea is to try to detect unexpected transitions, blocked packages, make sure that RC bug fixes reach Testing in a reasonable period of time, etc…
  3. During a freeze, making sure that unblock requests and freeze exceptions are not forgotten and try to make the RC bug count decrease.

There are other tasks that I’ll let you discover by joining the game.

Deciding what goes (or not) in the next stable release is a big responsibility and can be incredibly difficult at times. You have to make judgement calls all the time. What are your own criteria?

That’s a very hard to answer question (at least, for me). It really depends on the “case”. I try to follow the criteria that we publish in each release update. Sometimes, an unblock request doesn’t match those criteria and we have to decide what to accept from the set of proposed changes. Generally, new features and non-fixes (read new upstream versions) changes are not the kind of changes that we would accept during the freeze. Some of them could be accepted if they are not intrusive, easy and well defended. When, I’m not sure I try to ask other members of the Release Team to see if they share my opinion or if I missed something important during the review. The key point is to have a clear idea on what’s the benefit of the proposed update, and compare it to the current situation. For example, accepting a new upstream release (even if it fixes some critical bugs) is taking a risk to break other features and that’s why we (usually) ask for a backported fix.

It’s also worth noticing that (most of the time) we don’t decide what goes in, but (more specifically) what version of a given package goes in and try to give to the contributors an idea on what kind of changes are acceptable during the freeze. There are some exceptions though. Most of them are to fix a critical package or feature.

Do you have plans to improve the release process for Debian Wheezy?

We do have plans to improve every bit in Debian. Wheezy will be the best release ever. We just don’t know the details yet :)

During our last meeting in Paris last October, the Release Team agreed to organize a meeting after Squeeze’s release to discuss (among other questions) Wheezy’s cycle. But the details of the meeting are not fixed yet (we still have plenty of time to organize it… and other more important tasks to care about). We would like to be able to announce a clear roadmap for Wheezy and enhance our communication with the rest of the project. We certainly want to avoid what happened for Squeeze. Making things a bit more predictable for developers is one of our goals.

Do you think the Constantly Usable Testing project will help?

The original idea by Joey Hess is great because it allows d-i developers to work with a “stable” version of the archive. It allows them to focus on the new features they want to implement or the parts they want to fix (AIUI). It also allows to have constantly available and working installation images.

Then, there is the idea of having a constantly usable Testing for users. The idea seems nice. People tend to like the idea behind CUT because they miss some software disappearing from Testing and because of the long delays for security fixes to reach Testing.

If the Release Team has decided to remove a package from Testing, I think that there must be a reason for that. It either means that the software is broken, has unfixed security holes or was asked for the removal by its maintainer. I think that we should better try to spend some time to fix those packages, instead of throwing a broken version in a new suite. It could be argued that one could add experimental’s version in CUT (or sid’s) but every user is free to cherry-pick packages from the relevant suite when needed while still following Testing as a default branch.

Besides, it’s quite easy to see what was removed recently by checking the archive of debian-testing-changes or by querying UDD. IMO, It would be more useful to provide a better interface of that archive for our users. We could even imagine a program that alerts the user about installed software that got recently removed from Testing, to keep the user constantly aware any issue that could affect his machine. About the security or important updates, one has to recall the existence of Testing-security and testing-proposed-updates that are used specifically to let fixes reach Testing as soon as possible when it’s not possible to go through Unstable. I’m sure that the security team would appreciate some help to deal with security updates for Testing. We also have ways to speed migrate packages from Unstable to Testing.

I have to admit that I’m not convinced yet by the benefits brought by CUT for our users.


Thank you to Mehdi for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

5 reasons why Debian Unstable does not deserve its name

Debian Unstable (also known as sid) is one of the 3 distributions that Debian provides (along with Stable and Testing).

It’s not conceived as a product for end-users, instead it’s the place where contributors are uploading newer packages. Daily. Yes that means that Unstable is a quickly moving target and it’s not for everybody. But you can use it and your computer won’t explode.

1. It contains mainly stable versions of the software

Yes, you read it right. Unstable is not full of development versions of the various software. It happens on some software but then it’s usually a conscious decision of the maintainer who believes that this specific version is already better than the previous one.

The packages in sid are supposed to migrate to testing, the place where the next Debian stable release is prepared. So maintainers are advised to only upload stuff that is of release quality, the rest should be uploaded to experimental instead.

2. It doesn’t break badly every other day

Breakages happen but they are not a big deal usually. It has been long time since I could not reboot my computer after an upgrade or since the graphical interface was no longer working. The kind of breakages that you have is that one software stops working, or triggers an annoying bug, or that a few packages are uninstallable.

In most cases, you can save yourself by downgrading to the version available in Testing. Or by finding a work-around in the bug tracking system. Or by not upgrading because you have apt-listbugs installed and you have been warned about the problem.

3. It’s the basis of other distributions

If Debian Unstable was really so bad, it would not be a good basis to build a derivative distribution, isn’t it? But Ubuntu and SiduxAptosid (to name only two) are based on Debian Sid.

4. It’s not inherently less secure than Stable or Testing

High impact security vulnerabilities will usually be quickly fixed in Stable and Unstable. The stable upload is done by the security team while the unstable one is made by the maintainer. Testing will usually get the fix through the package uploaded to Unstable, so testing users get security updates with a delay.

For less serious vulnerabilities, it’s entirely possible that stable does not get any update at all. In that case, unstable/testing users are better served since they will get the fix with the next upstream version anyway.

Of course, it happens that maintainers are busy or that something falls through the cracks, but there are other people watching RC bugs who will fix this if the maintainer doesn’t react at all.

5. I use it on my main computer

And many other people do the same. And you can do the same if you meet the criteria below:

  • you can work on the command-line (enough to downgrade a problematic package, to edit configuration files, etc.);
  • you know how to work with APT and multiple distributions in /etc/apt/sources.list;
  • you are able to read/write English so that you can read/file bug reports when needed;
  • you have another computer connected to the Internet that you can use to lookup documentation (or the bug tracking system, or the support mailing lists) when your usual computer is off-line for a reason that you don’t understand.

If you feel you are not ready for the jump, click here to subscribe to this blog (or here via the RSS feed), I’ll surely teach some of the required skills in future articles.

PS: All that said, if you have a working sid installation, do not upgrade it just before an important presentation, or before a trip. It will always break at the most annoying time. Unless you like to live dangerously, of course.