My Debian activities in July 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (170 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

This month passed by very quickly since I attended both the Libre Software Meeting / RMLL and the DebConf.

Libre Software Meeting / RMLL

I attended “only” 3 days out of the 6 but that was a deliberate choice since I was also attending DebConf for a full week later in the month.

During those 3 days I helped with the Debian booth that was already well taken care of by Frédéric Perrenot and Arnaud Gambonnet. Unfortunately we did not have any goodies to sell. We (as in Debian France) should do better in this regard next time.

One of the talks I attended presented EnVenteLibre. This website started as an online shop for two French associations (Ubuntu-fr, Framasoft). They externalize all the logistic to a company and only have to care about ordering goodies and delivering to the warehouse of the logistic company. They can also take some goodies from the warehouse and ship them for a conference, etc. We discussed a bit to see how Debian France could join, they are even ready to study what can be done to operate at the international level (that would be interesting for Debian with all the local associations that we have throughout the world).

Back to the LSM, while I had 3 good days in Strasbourg, it seems to mee that the event is slowly fading out… it’s far from being an international event and the number of talks doesn’t make for a better quality.

BTW, do you remember that Debconf 0 and Debconf 1 were associated to this event while it was in Bordeaux?

dpkg-source improvements

During my time in Strasbourg (and in particular the travel to go there and back!) I implemented some changes to “3.0 (quilt)” source format. It will now fail to build the source package if there are upstream changes that are not properly recorded in a quilt patch:

dpkg-source: info: local changes detected, the modified files are:
 2ping-1.1/README
dpkg-source: info: you can integrate the local changes with dpkg-source --commit
dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/2ping_1.1-1.diff.cki8YB

As the error message hints, there’s a new --commit command supported by dpkg-source that will generate the required quilt patch to fix this. In the process you will have to submit a name and edit the patch header (pre-formatted with DEP3 compatible fields). You can get back the old behavior with the --auto-commit option.

Build flags changes

Ever since we adopted the Ubuntu changes to let dpkg-buildpackage set some build related environment variables (see #465282), many Debian people expressed their concerns with this approach both because it broke some packages and because those variables are not set if you execute debian/rules directly.

In the end, the change was not quickly reverted and we fixed the package that this change broke. Despite this we later decided that the correct approach to inject build flags would be a new interface: dpkg-buildflags.

Before changing dpkg-buildpackage to no longer set the compilation flags, I wanted to ensure dpkg-buildflags had some decent coverage in the archive (to avoid breaking too many packages again). My criteria was that CDBS and dh (of debhelper) should be using it. With the recent debhelper change (see #544844) this has been reached so I changed dpkg-buildpackage accordingly.

Makefile snippets provided by dpkg

At the same time, I also wanted an easy way for maintainers not using dh or CDBS to be able to fix their package easily and go back to injecting the compilation flags in the environment but doing it from the rules files. Starting with the next version of dpkg, this will be possible with something like this:

DPKG_EXPORT_BUILDFLAGS = 1
include /usr/share/dpkg/default.mk

Without DPKG_EXPORT_BUILDFLAGS the variables are not exported in the environment and have no effect unless you use them somewhere.

More than build flags, this will also provide a bunch of other variables that can be useful in a rules files: all the variables provided by dpkg-architecture, vendor related variables/macro and some basic package information (mainly version related).

dpkg-buildflags improvements

Given the renewed importance that dpkg-buildflags will take now that dpkg-buildpackage no longer sets the corresponding environment variables, I thought that I could give it some love by fixing all the open issues and implementing some suggestions I got.

I also had a chat with a few members of the technical committee to discuss how hardening build flags could be enabled in Debian and this also resulted in a few ideas of improvements.

In the end, here are the main changes implemented:

  • new “prepend” directive to inject flags at the start (see commit);
  • new “strip” directive to strip flags from the result returned by dpkg-buildflags (see commit);
  • new environment variables DEB_flag_MAINT_directive that can be set by the maintainer to adjust what dpkg-buildflags will return (see commit);
  • new --export=configure command to inject build flags on the ./configure command line (see commit);
  • new --dump command that is the default (see #603435).

Will all those changes, the complete set of compilation flags can be returned by dpkg-buildflags (before it would only return the default flags and it was expected that the Debian packaging would add whatever else is required afterwards). Now the maintainer just has to use the new environment variables to ensure the returned values correspond to what the package needs.

DebConf: rolling and hardening build flags

I spent a full week in DebConf (from Sunday 24th to Sunday 31th) and as usual, it’s been a pleasure to meet again all my Debian friends. It’s always difficult to find a good balance between attending talks, working in the hacklab and socializing but I’m pretty happy with the result.

I did not have any goal when I arrived, except managing the Rolling Bof (slides and video here) but all the discussions during talks always lead to a growing TODO list. This year was no exception. The technical committee BoF resulted in some discussions of some of the pending issues, in particular one that interests me: how to enable hardening build flags in Debian (see #552688).

We scheduled another discussion on the topic for Tuesday and the outcome is that dpkg-buildflags is the proper interface to inject hardening build flags provided that it offers a mean to drop unwanted flags and a practical way to inject them in the ./configure command line.

Given this I got to work and implemented those new features and worked with Kees Cook to prepare a patch that enables the hardening build flags by default. It’s not ready to be merged but it’s working already (see my last update in the bug log).

A few words about the Rolling BoF too. The room was pretty crowded: as usual the topic generates lots of interest. My goal with the BoF was very limited, I wanted to weigh the importance of the various opinions expressed in the last gigantic discussion on debian-devel.

It turns out a vast majority of attendants believe that testing is already usable. But when you ask them if we must advertise it more, answers are relatively mixed. When asked if we can sustain lots of testing/rolling users, few people feel qualified to reply but those that do tend to say yes.

More dpkg work

Lots of small things done:

  • I did again some bug triaging on Launchpad. But Brian Murray did a lot of it and the result is impressive, we’re down to 154 bugs (from more than 300 a month ago!).
  • I updated my multiarch branch multiple times. I was hoping to meet Guillem during DebConf to make some progress on this front but alas he did not attend. I have been asked a status update multiple times during my time in DebConf.
  • I fixed a regression in update-alternatives (#633627), a test-suite failure when run as root (#634961), a segfault in findbreakcycle. There have been a bunch of minor improvements too (#634510, #633539, #608260, #632937).

Package Tracking System and DEHS

Christoph Berg recently wrote a replacement for DEHS because the latter was not really reliable and not under control of the QA team. This is a centralized system that uses the watch files to detect new upstream versions of the software available in Debian.

I updated the Package Tracking System to use this new tool instead of DEHS. The new thing works well but we’re still lacking the mail notifications that DEHS used to send out. If someone wants to contribute it, that would be great!

Misc packaging work

I did some preliminary work to update the WordPress package to the latest upstream version (3.2). I still have to test the resulting package, replacing upstream shipped copies of javascript/PHP libraries is always a risk and unfortunately all of them had some changes in the integration process.

I also updated nautilus-dropbox to version 0.6.8 released upstream. I also uploaded the previous version (that was in testing at that time) to squeeze-backports. So there’s now an official package in all the Debian distributions (Squeeze, Wheezy, Sid and Experimental)!

Thanks

See you next month for a new summary of my activities.

My Debian activities in May 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work, then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

I have been…

Doing some work towards Debian Rolling

At the start of the month, the discussions about Debian rolling were still very active on debian-devel. Declaring that testing would be rolling did not make it (as I hoped), the argument that some RC bugs last for far too long in that distribution carried the discussion and thus the most consensual proposition ended up being the one of Josselin Mouette were rolling would be testing plus a few selected cherry-picked packages from unstable.

I believe it’s a workable solution if we only care about a subset of architectures. Otherwise the same reasons that keep the fixed packages out of testing would probably also apply for rolling.

Given this, I did setup britney (the software that controls testing) on my laptop to investigate how we can create rolling. It turns out britney is a very specialized software with very few configuration knobs.

At the same time Joachim Breitner made a proposition that immediately grabbed my attention. He suggests to use SAT solvers to find out the set of packages that should migrate from unstable to testing. I thought that rolling would be a good testbed for this new implementation of britney (which he calls SAT-britney) so I jumped right in this project.

I was not at all familiar with this science field, so I looked up quite some documentation: I learned that all SAT solvers expect the problem to be presented in CNF form, and that DIMACS was the file format of choice to represent those boolean constraints. Several SAT solvers are available in Debian and picosat appears to be one of the best.

Then I started some early coding/prototyping to play with the concept. You can find the result in this git repository, you can grab a copy with git clone git://git.debian.org/~hertzog/sat-britney.git.

There’s not much yet, except some Python code to generate a SAT problem that can be fed to a SAT solver. But I really look forward to this project.

Representing Debian during Solutions Linux

During the second week, I spent 3 days in Paris to help manage the Debian booth at Solutions Linux.

We have responded to lots of queries but most visitors already knew Debian, and many of them use it at work and/or at home. We tried to recruit those people as new members for Debian France, the local association. We also sold all our remaining goodies.

The Ubuntu people were interviewed by France 3 (an important TV channel) and we took this opportunity (with the consent of the Ubuntu guys) to show our Debian t-shirts in the background: you can watch the video here (in French), you can see me with Carl Chenet at 1:21.

We have also been interviewed by Intelli’n TV: here and here (both in French). I’m not very good at this exercise. :-)

Improving dpkg triggers

The third week was a vacation week, in theory I should have stayed away from my computer but I really wanted to take this opportunity to improve the state of dpkg triggers in Debian.

I already covered my work in another article: Trying to make dpkg triggers more useful and less painful.

The result is not merged yet, I just asked a question to all package maintainers who are using triggers to be able to decide whether I’ll merge it as is, or if I can make the new behavior the default one.

Supporting users after Alioth’s migration

When I came back from my vacation, many services provided by Alioth.debian.org were non-functional after a migration to a new setup that involves two machines instead of one. Given that I used to be an Alioth admin, I know that in those periods you tend to be get bogged down on many user support requests. So I re-joined #alioth on IRC and tried to help a bit.

I did investigate some of the reported problems and prepared fixes (updated scripts, configuration files, etc.) for some of the issues. I also created a list of remaining issues that should have lasted only a few days but that’s still active because there are still regressions left.

The most important things still missing are:

  • proper support for delegation of rights. We used ACL setup by the admins in the past. With the new FusionForge, each project admin should be able to delegate rights to external “roles”. There’s a Debian Developer role already but trying to grant him right fails…
  • access to the Ultimate Debian Database. Many tools rely on this database to work.
  • anonymous FTP access to download project files.
  • clear guidelines on how we’re supposed to deal with websites that are updated by VCS hooks.
  • clear guidelines on how we’re supposed to deal with personal git repositories

Improving the “3.0 (quilt)” source format

I have made some proposals to change the way the new source format would work. The goals are to be less painful for packagers who are using a VCS, and to avoid unexpected changes slipping through a new patch generated by dpkg-source.

It seems that the proposals are relatively consensual so I’ll implement them at some point.

Missing in action on my blog

I did a lots of stuff for Debian between travel and vacation, and in the remaining time, I did not manage to write many articles for my blog.

In fact, besides the article on my triggers work mentioned above I only published one interview: People behind Debian: Steve Langasek, release wizard.

I’ll try to do better this month!

Thanks

Many thanks to the people who gave me 151.61 € in May.

See you next month for a new summary of my activities.

Do we need project-wide support for Debian rolling?

The discussion about Debian rolling started sooner than expected on debian-devel (see the thread here). I initially wanted to iron out the biggest problems through discussions on my blog and try to submit a somewhat polished proposal… instead we ended up discussing the same things both on -devel and on my blog.

That said it’s not that bad (except for the time I lost to have similar conversations in both places) because the debian-devel discussions included members of the release team and it looks like they are not fundamentally opposed to the idea.

Despite this, the introduction of a “frozen” suite branched off from “testing/rolling” is not really consensual (yet?). But the idea of officially supporting testing on a best-effort basis appears to have almost no opposition.

While some will undoubtedly believe that this is a useless exercise, I believe it would help if the project stated this in a somewhat official manner. My answer to the question in the title is thus:

YES

I am thus considering to submit a general resolution to that effect. My current draft is below.

Title: Debian endorses usage of testing by end-users, and renames it to rolling

The Debian project recognizes that the Debian testing distribution—initially created to make it easier to prepare and test the next stable release—has become a useful product of its own. It satisfies the needs of users who are looking for the latest stable versions of software and who can cope (or even appreciate) a system that’s constantly evolving.

The Debian project decides to endorse this usage and will strive to provide a good experience to users of “testing”. To better communicate this policy change to our users, “testing” will be renamed “rolling”.

While we believe that this is a good move, we would like to remind our users that Debian is a volunteer project and that our resources are not infinite. Package maintainers are contributing to Debian on a best-effort basis. This means that they might not be able to properly support their package(s) in all distributions. In that case, the project recommends that maintainers apply the following priorities:

  1. Support in stable (security updates, release critical bugs)
  2. Preparation of the next stable release
  3. Support in rolling

Note that this general resolution could have amendments with s/rolling/current/ and that would solve the bikeshedding over the name that started several times already.

I deliberately separated “Preparation of the next stable release” and “Support in rolling” so that priorities are clear even if we decide to not freeze “rolling” and to introduce a “frozen” distribution to finalize the next stable release.

I also did not go into too much details on the implications that it might have, it’s best to leave that up to each contributor/team/etc.

I hesitated to add a paragraph stating that we want to try to gradually improve the usability of testing but in the end I think it would be somewhat redundant. We’re always trying to do our best when we decide to take on something.

All comments welcome (even if you just agree and would be willing to second such a GR).

Update: I will tweak the draft included in this article when I get good suggestions. Thanks to Lucas Nussbaum for the first one.

No freeze of Debian’s development, what does it entail?

The main feature of rolling is that it would never freeze. This is not without consequences.

Possible consequences

It can divert developers from working on the release

No freeze means developers are free to continue their work as usual in unstable. Will it be more difficult to release because some people will spend their time working on a new upstream version instead of fixing RC bugs in the version that is frozen? Would we lose the work of the people who do lots of NMU to help with the release?

It makes it more difficult to cherry-pick updates from unstable

No freeze also means that unstable is going to diverge sooner from testing and it will be more difficult to cherry-pick updates from unstable into testing. And the release team likes to cherry-pick updates that have been tested in unstable because updates that comes through testing-proposed-updates have often not been tested and need thus a more careful review.

Frozen earth

My responses to the objections

Those are the two major objections that we’ll have to respond to. Let’s try to analyze them a bit more.

It’s not testing vs rolling

On the first objection I would like to respond that we must not put “testing” and “rolling/unstable” in opposition. The fact that a contributor can’t do its work as usual in unstable does not mean that he will instead choose to work on fixing RC bugs in testing. Probably that some do, but in my experience we simply spend our time differently, either working more on non-Debian stuff or doing mostly hidden work that is then released in big batches at the start of the next cycle (which tends to create problems of its own).

I would also like to argue that by giving more exposure to rolling and encouraging developers to properly support their packages in rolling, it probably means that the overall state of rolling should become gradually better compared to what we’re currently used to with testing.

The objection that rolling would divert resources from getting testing in a releasable shape is difficult to prove and/or disprove. The best way to have some objective data would be to setup a questionnaire and to ask all maintainers. Any volunteer for that?

Unstable as a test-bed for RC bugfixes?

It’s true that unstable will quickly diverge from testing and that it will be more difficult to cherry-pick updates from unstable into testing. This cannot be refuted, it’s a downside given the current workflow of the release team.

But I wonder if the importance of this workflow is not overdone. The reason why they like to cherry-pick from unstable is because it gives them some confidence that the update has not caused other regressions and ensures that testing is improving.

But if they’re considering to cherry-pick an update, it’s because the current package in testing is plagued by an RC bug. Supposing that the updated package has introduced a regression, is it really better to keep the current RC bug compared to trading it for a new regression? It sure depends on the precise bugs involved so that’s why they prefer to know up-front about the regression instead of making a blind bet.

Given this, I think we should use testing-proposed-updates (tpu) as a test-bed for RC bug fixes. We should ask beta-testers to activate this repository and to file RC bugs for any regression. And instead of requiring a full review by a release manager for all uploads to testing-proposed-updates, uploads should be auto-accepted provided that they do not change the upstream version and that they do not add/remove binary packages. Other uploads would still need manual approval by the release managers.

On top of this, we can also add an infrastructure to encourage peer-reviews of t-p-u uploads so that reviews become more opportunistic instead of systematic. Positive reviews would help reduce the aging required in t-p-u before being accepted into testing.

This changes the balance by giving a bit more freedom to maintainers but still keeps the safety net that release managers need to have. It should also reduce the overall amount of work that the release team has to do.

Comments welcome

Do you see other important objections beside the two that I mentioned?

Do you have other ideas to overcome those objections?

What do you think of my responses? Does your experience infirm or confirm my point of view?

Towards Debian rolling: my own Debian CUT manifesto

As you might know, I’m of one the people who promote the idea of having a Constantly Usable Testing (CUT). I will post a set of articles on this topic to help me clarify my ideas and to get some early feedback of other Debian contributors. I plan to use the result of this process to kickstart a broader discussion on debian-devel (where I hope to get the release team involved).

Let’s start by defining my own objectives with Debian CUT. I won’t speak of the part of Debian CUT that plans to make regular snapshots of testing with installable media because that’s not where I will invest my time. I do hope other people will do it though. Instead I will concentrate on changes that would improve Debian testing for end-users. I consider that testing (in its current form) is largely usable already but there are two main obstacles to overcome.

Testing without freezes = rolling

The first one is that testing is no longer testing during freezes. The regular flow of new upstream versions—that makes testing so interesting for many users—is stalled because we’re using testing to finalize the next stable version. That’s why I’d like to introduce a new suite called “rolling” that would work like the current testing except that it never freezes. Testing would no longer be a permanent suite, it would only exist during the freeze and it would be branched off from rolling.

Rolling should be a supported distribution

The second obstacle is not really technical. We must advertise rolling as a distribution that ordinary people can use but we should make it clear that it’s never going to have the same level of polish that a stable release can have. And to back up this assertion, we must empower Debian developers to be able to provide good support of their software in rolling, this probably means using “rolling-proposed-updates” more often to push fixes and security updates when the natural flow of updates is blocked by transitions or other problems. Some maintainers won’t have the time required to provide the same level of support as they currently do for a stable release and that’s okay, it’s not worse than the current testing. The goal is to treat it on a best-effort basis and to try to gradually improve the situation.

My goal for wheezy

This is the the minimal implementation of CUT that I would like to target in the wheezy timeframe. Having testing as a temporary branch of rolling is not a strict requirement, although I’d argue that it’s important to not waste resources towards maintaining two separate yet very similar distributions.

Should Debian embrace those goals?

Ignoring all possible problems that will surface while trying to implement those goals, can we agree that Debian should embrace those goals because it means providing a better service to a class of users that is not satisfied by the current stable release?

I will come back to the expected problems in a subsequent post and we will have the opportunity to discuss them. Here I just want to see whether we can have broad buy-in on the principles behind Debian rolling and CUT.

Can Debian offer a Constantly Usable Testing distribution?

Debian’s “testing” distribution is where Debian developers prepare the next stable distribution. While this is still its main purpose, many users have adopted this version of Debian because it offers them a good trade-off between stability and freshness. But there are downsides to using this distribution and the “Constantly Usable Testing” (CUT) project aims to resolve those. This article will present the project and the challenges involved to make it happen.

About Debian unstable & testing

Debian unstable is the distribution where developers upload new versions of their packages. It happens frequently that some packages are not installable due to changes in other packages or due to transitions not yet completed.

Debian testing, on the contrary, is managed by a tool that ensures the consistency of the whole distribution: it picks updates from unstable only if the package has been enough tested (10 days usually), if it’s free of new release-critical bugs, if it’s available on all supported architectures, and if it doesn’t break any other package already present in testing. The Release Team (RT) controls this tool and provide “hints” to help it find a set of packages that can flow from unstable to testing.

Those rules also ensure that the packages that flow into testing are reasonably free of show-stopper bugs (like a system that doesn’t boot, or X that doesn’t work at all). This makes it very attractive to users who like to regularly get new upstream versions of their software without dealing with the biggest problems associated to them. This is all very attractive, yet several Debian developers advise people to not use testing. Why is that?

Known problems with testing

Disappearing software

The release team use this distribution to prepare the next stable release and from time to time they remove packages from it. Either because it’s needed to ensure that other packages can migrate from unstable to testing, or because they have long-standing release-critical bugs without progress towards a resolution. It also happens that they remove packages on request of the maintainers because they believe that the current version of the software cannot be supported (security-wise) for 2 years or more. The security team also regularly issues such requests.

Long delays for security and important fixes

Despite the 10-day delay in unstable, there are always some annoying bugs (and security bugs are no exceptions) that are only discovered when the package already has migrated to testing. The maintainer might be quick to upload a fixed package in unstable, and might even raise the urgency to allow the package to migrate sooner, but if the packages gets entangled in a large ongoing transition, it will not migrate before the transition is completed. Sometimes it can take weeks for that to happen.

This delay can be avoided by doing direct uploads to testing (through testing-proposed-updates) but this is almost never used, except during a freeze, where targeted bugfixes are the norm.

Not always installable

With testing evolving daily, updates sometimes break the last installation images available (in particular netboot images that get everything from the network). The debian-installer (d-i) packages are usually quickly fixed but they don’t move to testing automatically because the new combination of d-i packages has not necessarily been validated yet. Colin Watson sums up the problem:

Getting new installer code into testing takes too long, and problems remain unfixed in testing for too long. [...] The problem with d-i development at the moment is more that we’re very slow at producing new d-i *releases*. [...] Your choices right now are to work with stable (too old), testing (would be nice except for the way sometimes it breaks and then it tends to take a week to fix anything), unstable (breaks all the time).

CUT’s history

CUT finds its root in an old proposal by Joey Hess: it introduces the idea that the stable release is not Debian’s sole product and that testing could become — with some work — a suitable choice for end-users. Nobody took on that work and there was no visible progress in the last 3 years.

But recently Joey brought up CUT again on the debian-devel mailing list and Stefano Zacchiroli (the Debian project leader) challenged him to setup a BoF on CUT for Debconf10. It turned out to be one of the most heavily attended BoF (video recording is here), there is clearly a lot of interest in the topic.

There’s now a dedicated wiki and an Alioth project with a mailing list. The rest of this article tries to summarize the various options discussed and how they’re supposed to address the problems identified.

The ideas behind CUT

Among all the ideas, there are two main approaches that have been discussed. The first is to regularly snapshot testing at points where it is known to work reasonably well (those snapshots would be named “cut”). The second is to build an improved testing distribution tailored to the needs of users who want a working distribution with daily updates, its name would be “rolling”.

Regular snapshots of testing

There’s general agreement that regular snapshots of testing are required: it’s the only way to ensure that the generated installation media will continue to work until the next snapshot. If tests of the snapshot do not reveal any major problem, then it becomes the latest “cut”. For clarity, the official codename would be date based: e.g. “cut-2010-09″ would be the cut taken during September 2010.

While the frequency has not been fixed yet, the goal is clearly to be on the aggressive side: at the very least every 6 months, but every month has been suggested as well. In order to reach a decision, many aspects have to be balanced.

One of them (and possibly the most important) is the security support. Given that the security team is already overworked, it’s difficult to put more work on their shoulders by declaring that cuts will be supported like any stable release. No official security support sounds bad but it’s not necessarily so problematic as one might imagine. Testing’s security record is generally better than stable’s one (see the security tracker) because fixes flow in naturally with new upstream versions. Stable still get fixes for very important security issues sooner than testing, but on the whole there are less known security-related problems in testing than in stable.

Since it’s only a question of time until the fixed version comes naturally from upstream, more frequent cut releases means that users get security fixes sooner. But Stefan Fritsch, who used to be involved in the Debian testing security team, has also experienced the downside for anyone who tries to contribute security updates:

The updates to testing-security usually stay useful only for a few weeks, until a fixed version migrates from unstable. In stable, the updates stay around for a few years, which gives a higher motivation to spend time on preparing them.

So if it’s difficult to form a dedicated security team, the work of providing security updates comes back to the package maintainer. They are usually quite quick to upload fixed packages in unstable but tend to not monitor whether the packages migrate to testing. They can’t be blamed because testing was created to prepare the next stable release and there is thus no urgency to get the fix in as long as it makes it before the release.

CUT can help in this regard precisely because it changes this assumption: there will be users of the testing packages and they deserve to get security fixes much like the stable users.

Another aspect to consider when picking a release frequency is the amount of associated work that comes with any official release: testing upgrades from the previous version, writing release notes and preparing installation images. It seems difficult to do this every month. With this frequency it’s also impossible to have a new major kernel release for each cut (since they tend to come out only every 2 to 3 months) and the new hardware support that it brings is something worthwhile to many users.

In summary, regular snapshots address the “not always installable” problem and changes the perception of maintainers towards testing, so that hopefully they care more of security updates in that distribution (and in cuts). But they do not solve the problem of disappearing packages. Something else is needed to fix that problem.

A new “rolling” distribution?

Lucas Nussbaum pointed out that regular snapshots of Debian is not really a new concept:

How would this differentiate from other distributions doing 6-month release cycles, and in particular Ubuntu, which can already be seen as Debian snapshots (+ added value)?

In Lucas’s eyes, CUT becomes interesting if it can provide a rolling distribution (like testing) with a “constant flux of new upstream releases”. For him, that would be “something quite unique in the Free Software world”. The snapshots would be used as starting point for the initial installation, but the installed system would point to the rolling distribution and users would then upgrade as often as they want. In this scenario, security support for the snapshots is not so important, what matters is the state of the rolling distribution.

If testing were used as the rolling distribution, the problem of “disappearing packages” would not be fixed. That’s why there have been discussions of introducing a new distribution named “rolling” that would work like testing but with adapted rules, and the cuts would then be snapshots of rolling instead of testing.

The basic proposal is to make a copy of testing and to re-add the packages which have been removed because they are not suited for a long term release while they are perfectly acceptable for a constantly updated release (the most recent example being Chromium).

Then it’s possible to go one step further: during freeze, testing is no longer automatically updated which makes it inappropriate to feed the rolling distribution. That’s why rolling would be reconfigured to grab updates from unstable (but using the same rules than testing).

Given the frequent releases, it’s likely that only a subset of architectures would be officially supported. This is not a real problem because the users who want bleeding edge software tends to be desktop users on mainly i386/amd64 (and maybe armel for tablets and similar mobile products). This choice — if made — opens up the door to even more possibilities: if rolling is configured exactly like testing but with only a subset of the architectures, it’s likely that some packages migrate to rolling before testing when non-mainstream architectures are lagging in terms of auto-building (or have toolchain problems).

While being ahead of testing can be positive for the users, it’s also problematic on several levels. First, managing rolling becomes much more complicated because the transition management work done by the release team can’t be reused as-is. Then it introduces competition between both distributions which can make it more difficult to get a stable release out, for example if maintainers stop caring of the migration to testing once the migration to rolling has been completed.

The rolling distribution is certainly a good idea but the rules governing it must be designed to avoid any conflict with the process of releasing a stable distribution. Lastly, the mere existence of rolling would finally fix the marketing problem plaguing testing: the name “rolling” does not suggest that the software is not yet ready for prime time.

Conclusion

Whether CUT will be implemented remains to be seen, but it’s off for a good start: ftpmaster Joerg Jaspert said that the new archive server can cope with a new distribution, and there’s now a proposal shaping up. The project might start quickly: there is already an implementation plan for the snapshot side of the project. The rolling distribution can always be introduced later, once it is ready. Both approaches can complement each other and provide something useful to different kind of users.

The global proposal is certainly appealing: it would address the concerns of obsolescence of Debian’s stable release by making intermediary releases. Anyone needing something more recent for hardware support can start by installing a cut and follow the subsequent releases until the next stable version. And users who always want the latest version of every software could use rolling after having installed a cut.

From a user point of view, there are similarities with the mix of usual and long term releases of Ubuntu. But from the development side, the process followed would be quite different, and the constraints imposed by having a constantly usable distribution are stronger: any wide-scale change must be designed in a way that it can happen progressively in a transparent manner for the user.

This article was first published in Linux Weekly News. If you want to see more articles like this, join Flattr and click on the flattr button below every article that you like.