Category Archives: Debian News

What’s happening right now in Debian.

Understanding Membership Structures in Debian and Ubuntu

Debian and Ubuntu have a set of official membership roles that can be granted to regular contributors. Those roles come with rights that enable the contributors to do their work and to participate in the project governance (elections and other official decision-making processes). It’s also a way for the distributions to acknowledge the work done: most contributors are proud of the status they reached.

The membership structure plays an important role in the development of a distribution: it defines the kind of contributors that are welcome in the project, it sets expectations of the project towards its contributors and defines their rights. In the end, this shapes the project’s ability to recruit new contributors to keep the project alive and kicking. This article introduces the existing statuses in Debian and Ubuntu, and defines the — sometimes confusing — jargon associated with them.

The Debian Case

Debian only has two types of official members: Debian Developers (DD) and Debian Maintainers (DM). The rights of the developers are codified in the Debian Constitution while those of the maintainers have been defined in a general resolution of 2007. The Debian Maintainer status is still mostly documented in a wiki page. The integration of this new status in Debian’s official processes has been slow to come largely because it was introduced — at that time — without enough negotiation with the involved parties. Nowadays, it is preferred that people get the DM status before applying for DD.

DM is a very limited role: maintainers can only upload packages that already have their name on them (either in the Maintainer or Uploaders field) and a specific flag (DM-Upload-Allowed: yes) that only Debian Developers can add. They have no other rights and limited access to Debian’s resources.

Besides those official roles, there are also maintainers of packages that have no official status within Debian except that they are listed in the “Maintainer” field of the package. They are doing the maintenance work but all uploads are done by a Debian Developer after verification of the work done (this is called “sponsorship” and is the only way to start with official packaging work). Once the DD trusts the maintainer, the developer will typically ask the maintainer to apply for DM status in order to be relieved from the sponsorship work.

In the end, that makes three different kind of package maintainers and a lot of confusion when you discuss membership issues… in particular when the New Maintainer process is the path that you follow to become a Debian Developer. Don’t be fooled by the names when reading Debian’s documentation!

The Ubuntu Case

Ubuntu had, from the start, an official Ubuntu Member status that includes all contributors: developers of course, but also documentation writers, artists, translators, etc. This status notably grants the right to vote in elections of the Community Council, the right to participate on Planet Ubuntu, and the @ubuntu.com email alias.

For developers, the situation is more complicated: the wiki page lists no less than five different statuses. Initially, developers were split between Ubuntu Core Developers and the MOTU (Masters Of The Universe). The latter were responsible of the universe/multiverse sections of the archive while the former also had upload rights for the main/restricted sections. But, inspired by the Debian Maintainers concept and facing concrete problems in terms of archive management, they changed their infrastructure to offer more fine-grained control on package uploads.

Ubuntu can now grant upload permissions on a package-per-package basis, but it can also delegate the right to grant upload permissions with the same granularity. This lead to the new Per-Package Uploader status which is simply an Ubuntu Member with upload rights on a limited set of packages where they have a specific expertise. The more generic Ubuntu Developers status now encompasses members of various development teams that have been delegated the right to manage upload permissions on a (usually large) package set (the current teams are Ubuntu Desktop, Mythubuntu, Kubuntu, and Edubuntu). Those teams can define their own policy to add new members provided they follow the basic rules defined by the Developer Membership Board (see this wiki page).

Ubuntu Contributing Developer is an intermediate status for someone who is not yet ready for one of the other developer statuses but who has still shown enough commitment to be an Ubuntu Member.

All those statuses can be obtained in a similar way: you prepare a wiki page listing your past contributions, you collect testimonials from existing members that you have worked with, you add yourself in the agenda of the next meeting of the board (or council) that grants the status that you seek, and you attend the meeting. The members of the board will decide whether you are ready for the status (or not) based on what you provided in the wiki, based on your answers during the meeting (and on a mailing-list for developers), and based on what others have to say about you.

The most important boards are usually elected by the community while others are commonly appointed by the community council. Those governance bodies include Canonical employees but not as many as one would expect: two out of eight in the Developer Membership Board, two out of eight in the Community Council, but all six members of the Technical Board. The last figure, while not intended, is not surprising given the high expectations set on potential members of the technical board. Mark, as the founder, is the only person to have a permanent seat on both the Community Council and the Technical Board.

Comparison of the Statuses Between Debian and Ubuntu

The following table summarizes the rights given to each developer role in the two projects (Put the mouse over the abbreviations to know what they are referring to).

Rights Debian Ubuntu
DM DD UM PPU/UD MOTU UCD
Package maintenance via sponsorshipYN/AYYYN/A
Official email alias-YYYYY
Participate in votes for members-YYYYY
Participate in votes for developers-Y-YYY
Upload rights restricted to pre-approved packagesY--Y--
Upload rights restricted to a section of the archive----Y-
Unlimited upload rights-Y---Y
Number of contributors (as of 2010-07-27)117904462278563

Please note that the number of contributors are not 100% accurate for Ubuntu. A contributor can have multiple statuses (direct membership to a launchpad group) granted over time (while gaining experience). The problem has been mostly avoided by calculating differences between number of members of the various groups but it’s not perfect and it can’t be: some MOTU are also PPU for packages in main and it’s legitimate (but I only counted them as MOTU and not as PPU). Another limitation is that members of some administrative teams are included indirectly in many teams and thus appear in the count while they should not.

Anyway, this simple table makes it obvious that Ubuntu’s structure offers a broader choice of statuses. They acknowledge the work of all contributors from the start while still giving the most critical rights only to those who have proven that they deserve them. Despite this difference, Debian still has a significant advantage in terms of number of developers. That number does not tell the whole story though: the Ubuntu contributors include many Canonical employees (e.g. 36 out of 63 core developers have a @canonical.com email registered on their launchpad account) that are likely to spend more time working on the distribution than the average Debian member. But even if comparing person-hours would be a challenging thought experiment, in practice it’s of not much interest if both projects continue to cooperate and if more and more of the contributions flow in both directions.

Debian is aware of the shortcomings of its structure. Changes to better accommodate non-packagers have been discussed several times already. The last efforts in that direction were unfortunately perceived as a solution ready to go rather than a proposal to be discussed, and the project got quickly buried by a general resolution (GR). Even if that resolution invited for further discussion and a new proposal, the truth is that when someone’s initiative is “corrected” by way of GR, it usually kills any motivation to go forward.

Possible Evolution?

On the Ubuntu side, the infrastructural changes were completed recently and they don’t expect any further change in the near future. They do plan, however, to expand usage of those new features so that more teams benefit from the possibility to control upload rights on packages that are relevant to them, and so that more individuals developers apply to become Per-Package Uploaders on packages that they know very well.

On the Debian side, a recent discussion on the debian-project list brought back the topic of the bad terminology and it was agreed that the “New Maintainer process” should be renamed into something else (“New Developer process” has been suggested). But Christoph Berg — Debian Account Manager and hence heavily involved in the New Maintainer Team — suggested that Debian would be better off implementing the long-awaited membership changes before trying to update all the documentation. It would certainly imply some more vocabulary updates. Later in the discussion, he confirmed that membership reform is on the top of the TODO list of the new maintainer team (just after the rewrite of the nm.debian.org website).

What can be expected from this reform? The following answers are my own guesses based on my experience of Debian, but the project hasn’t decided anything yet.

  • First of all: a new status for contributors that are not packagers. The tricky part will be defining the process to follow and the rights granted.

  • Changes to the technical implementation of the DM status. The current implementation does not allow to give upload rights to a single DM if two are listed in the Uploaders field of a package (and both might not have the same experience for that package). Furthermore, it suffers from annoying restrictions like the inability to upload new binary packages.

  • A change of the Debian constitution to integrate those new statuses is almost unavoidable.

  • Other more invasive changes have been proposed like replacing the NM process by a simple designation by other DD, but it’s unlikely to happen. The NM process can already be greatly simplified by the application manager if the applicant can show good testimonials from other developers and if he has a track record of real contributions (e.g. as witnessed by changelog entries in Debian packages).

Almost two years have elapsed since the previous efforts in that direction, the new maintainer team has recruited new members and is in a general better shape. Hopefully, the next episode of this saga will have a better outcome.

This article was first published in Linux Weekly News. In a comment, Mark Shuttleworth tried to explain how the Ubuntu community is being setup.
  • Share/Bookmark
Posted in Debian News, News, Ubuntu News | Tagged , , | 10 Comments

Quick news: dpkg, collab-maint, alioth and the future

Dpkg got rid of Perl

Let’s start with the interesting part and the great news: dpkg 1.15.8 (to be uploaded soon) will no longer need perl! After my changes to rewrite update-alternatives in C, Guillem recently pushed the rewrite of dpkg-divert/mksplit in C. Please test it out (binary package for i386 or .dsc).

This is rather exciting news for those who would like to use dpkg in embedded contexts. And it’s great to see this completed in time for Squeeze. In Squeeze+1, we might go one step further and merge cdebconf, the C replacement for debconf.

I got rid of some recurring administrative tasks

I have been administrating the Alioth server since its inception (see the announce I sent in 2003) but I’m no longer enjoying the day-to-day administrative work that it represents. That’s why I just retired from the team. We recently recruited Tollef Fog Heen so the number of admins is still the same (that said, Alioth could benefit from some more help, if you’re a DD and interested, drop a mail to admin@alioth.debian.org or come to #alioth).

Same goes for the collab-maint project. I have dealt with hundreds of requests to add new contributors to the project since it’s the central repository where all Debian developers have write access and where they put the VCS for their packages that do not belong to a more specialized team. The new administrator that will approve the requests is Xavier Oswald and he’s doing the work under the umbrella of the New Maintainer’s Front Desk.

The future

I will continue to spend the same amount of time on Debian, the time freed will quickly be reallocated to other Debian and free software related projects. In fact, I even anticipated a bit by launching Flattr FOSS last week but that’s a relatively simple project. :-)

The other projects that will never all fit in the freed time: I want to spend more time working on dpkg. I do plan to blog more often too, but I’m sure you’ll notice that yourself soon. I would like to see my Debian book translated into English (another post coming on the topic sometimes soon). In my dreams, I could even start yet another software project, I have some ideas that I really would like to see implemented but I don’t see how that could fit in this year’s planning… unless I can convince someone else to implement them! Maybe I should blog about them.

  • Share/Bookmark
Posted in Debian News | Tagged , , , | 5 Comments

Rewriting update-alternatives in C

Among the goals listed in dpkg’s roadmap, there’s the C rewrite of the remaining perl scripts provided by the dpkg binary package (dpkg-dev is not concerned, it will remain a collection of perl scripts). Of the remaining scripts, update-alternatives was the largest piece of code (~1100 lines of perl) and I started converting it to C a few weeks ago (based on preliminary work of Guillem). It’s now 2200 lines of C…

Thanks to the relatively extensive test-suite that I wrote last year, I’m relatively confident that this new update-alternatives won’t break your system. That said, it still needs some real-life usage to ensure everything is really ok (and users actively trying to break it are even better). Thus I would be glad if you could try it out ( binary package for i386 or .dsc) and report back to debian-dpkg@lists.debian.org.

The rewrite of the 2 other remaining scripts is almost completed in a branch of Guillem. Hopefully this can be our last project completed in time for Squeeze as far as dpkg goes. It would be a great achievement for people that would like to use dpkg in embedded environments and avoid perl due to its size.

Note: nobody sponsored that work. But it’s not too late :-)

  • Share/Bookmark
Posted in Debian News | Tagged , | 7 Comments

New source formats allowed in testing/unstable

The ftpmasters merged my dak branch last week during their meeting and have enabled the support of new source formats “3.0 (quilt)” and “3.0 (native)” in testing, unstable and testing-proposed-updates. I have uploaded 3 packages using the new formats already: logidee-tools using “3.0 (native)”, quilt and ftplib using “3.0 (quilt)”. The latter is arch any and has been successfully built on all architectures even those that still use an old version of sbuild (it looks like the fears that the old version would not cope with the new format were unfounded). For logidee-tools I built it with “-Zbzip2” in order to use bzip2 compression on the native tarball.

I have updated the wiki page and the release goal page with latest information. Feel free to convert some of your packages to give it a try. For ftplib, it led me to discover a Debian specific patch that I completely missed when I took the package over. This is precisely the kind of benefit that I expect from generalizing this format, it will encourage us to have separate documented patches instead of keeping everything hidden inside the usual .diff.gz. Combined with DEP-3 (patch tagging guidelines), we have a better infrastructure to share our patches with the rest of the free software community.

The next step is to fix all bugs listed here and make dpkg-source use the new source formats by default (#553928). Feel free to help by preparing patches (and offering NMUs), it’s a release goal to have all packages buildable with new source formats.

  • Share/Bookmark
Posted in Debian News | Tagged , , | 4 Comments

3-way merge of Debian changelog files

I’ve been considering for some time now to create a merge tool specifically suited for debian/changelog files. My goal was to let Git use it automatically thanks to gitattributes.

I’ve just gone ahead, so let me introduce you git-merge-dch. Grab it with git clone git://git.debian.org/~hertzog/git-merge-dch.git, you can build a package if you wish. Beware, you need to have a dpkg-dev 1.15.5 that is not yet published (so you need to build dpkg from its git repository, git clone git://git.debian.org/dpkg/dpkg.git) as I rely on features that I introduced recently… you will also need the libalgorithm-merge-perl package.

Using it in a git repository requires two changes:

  • defining a new merge driver somewhere in the git configuration (in .git/config or ~/.gitconfig for example):
    [merge "git-merge-dch"]
            name = debian/changelog merge driver
            driver = git-merge-dch -m %O %A %B %A
    
  • defining the merge attribute for debian/changelog files either in .gitattributes in the repository itself or in .git/info/attributes:
    debian/changelog merge=git-merge-dch
    

Now you can safely maintain two branches of a package with changelog files evolving separately and merge one into the other without creating undue conflicts. Suppose you created an experimental branch for version 2.28 (you use a version 2.28-1~exp1) when 2.26.2 was current stable in the master branch. In the mean time, 2.26.3 got out and was packaged in master. Next time you merge stable into experimental, the changelog entries for 2.28 and 2.26.3 won’t collide despite being at the same place in the changelog file compared to the common ancestor.

Let’s continue with this example, 2.28 is out. Instead of adding a new changelog entry with “New upstream release” without further changes, you keep the current changelog entry and simply change the version into 2.28-1. While preparing this you discover a branch with fixes that was based on 2.28-1~exp1, if you merge it it will reintroduce a 2.28-1~exp1 entry that you don’t want. Fortunately you can use the --merge-prereleases (-m) option of git-merge-dch so that it strip the prerelease part of the version string and considers 2.28-1~exp1 and 2.28-1 to be the same entry really.

The only limitation is that this merge tool will remove any lines which are not parsed by Dpkg::Changelog (and which in theory are not supposed to be there).

Feel free to test, share your comments, report bugs and send patches!

Update: the script has been merged in dpkg-dev (>= 1.15.7) under the name dpkg-mergechangelogs.

  • Share/Bookmark
Posted in Debian News | Tagged , , , | 6 Comments

DPL election: low participation

This year I have not given any vote recommendation because all candidates would be (IMO) good DPL. The participation stats are a bit strange however: when I got the second call for vote I noticed 176 votes in the first week compared to 135 last year. So I thought “good, participation is on the rise”. But then I got reminded that we have shortened the voting period of the DPL election to two weeks. So the comparison doesn’t hold.

The vote close in two days and we have so far only 283 votes, and last year we got 482 in the end. So we’re likely to have much less participation this year… even if you add a percentage for the people who wish to vote but cannot for various reasons (which proves once more how important it is that the next DPL be determined to fix those recurring problems), you won’t get the same numbers.

So my question is: do we have lower participation because all candidates are good and people do not care who gets elected? or do we have so many DD that follow Debian only every 2.5 weeks?

And if you haven’t voted yet, it’s time to do it. :-)

  • Share/Bookmark
Posted in Debian News | Tagged , | 4 Comments

Debian Documentation Project moved to SVN, webwml might follow

The topic of switching from CVS to something else regularly came forward but nobody did anything. The net result is that several documentation are now maintained outside of the debian-doc repository because their respective maintainers didn’t want to stay with CVS.

After noticing that the developers-reference also switched to SVN, I decided to convert the whole debian-doc CVS repository and import it in the new “ddp” SVN repository on Alioth. This is now done.

Hopefully, the Debian Documentation Project can now again become the central place for writing good documentation about Debian. New contributors can be easily added through the DDP Alioth project. Volunteers are welcome to review what’s in the SVN and move obsolete documentation aside. People who moved away are welcome back. :-)

Another project that also suffers from its CVS usage is the website (and it desperately needs a better design). After yet another round of discussion on #debian-devel, we agreed that discussing endlessly was not an option and that someone had to try the conversion and prepare the scripts for SVN usage. So I proposed to handle the CVS to SVN conversion and ifvoid decided to try to update the scripts. And it looks like things are progressing quite well… we included the CVS revision -> SVN revision mapping in the conversion (option –cvs-revnums of cvs2svn) and this will enable us to script the update of all translations (they encode a CVS revision to know if they are out-of-date or not). Expect to hear from us soon on debian-www@lists.debian.org…

  • Share/Bookmark
Posted in Debian News | Tagged , , | Comments Off

Unexpected move on the DSA front

One should never loose hope apparently. After my previous posts on the topic, Sam put some more pressure and expressed strongly his disappointment in the lack of progress on the DSA front and warned that he might have to look for more radical solution.

Phil Hands (who was really quiet in the discussions) decided that the situation had lasted long enough already and proposed to add Peter Palfrader (weasel@debian.org) to the team if nobody expressed any opposition in 24 hours. And this morning, Phil added weasel to the “adm” group… which make of Peter a full DSA member! :-)

I really didn’t expect this outcome after so many months of negociations and discussions. But I’m really relieved to see some new blood in the DSA team. I wish him good luck and I’ll hope he will be able to foster cooperation with non-DSA people like I tried since the beginning of the year.

Thank you Phil!

  • Share/Bookmark
Posted in Debian News | Tagged , | Comments Off

Planet debian CVS repo superseded by SVN repository

When I discussed the deprecation of cvs.debian.org, I forgot to mention that planet’s CVS repository was also part of the last users of this service.

Thanks to mako, the move to Alioth is underway. The CVS repository has been disabled (crude hack, if you know a proper way to forbid commits to a CVS repository, please leave a comment) and its history has been imported in a new SVN repository

As usual DD have write-access on this repository so that they can add/update their feeds and hackergotchis. Mako will change the live setup from CVS to SVN somewhat later today.

Please start updating any documentation that refers to the CVS repository.

  • Share/Bookmark
Posted in Debian News | Tagged | 2 Comments

Deprecating cvs.debian.org in favor of Alioth

It’s very difficult to discuss with DSA and make things evolve if none of the DSA member express an interest in something related to your goal: here comes an example of a story like another in my desperate quest to try to help the DSA team. :-)

Last time gluck ran out of space, a few non-DSA people (me, taggart, Ganneff, and others I might have forgotten) contacted people to ask them to clean their home directories. Following that discussion we discussed a bit about the opportunity to move some services from gluck on another host. Among the services on gluck, there’s cvs.debian.org. As an Alioth administrator, it struck me that cvs.debian.org is the only VCS service that’s handled by the DSA team. It seems logical to not duplicate the administrative work and have all the VCS repositories handled by the same team.

The logical conclusion is that cvs.debian.org should be deprecated in favor of Alioth. So I made the suggestion in RT ticket #146 (login with guest/readonly). I got exactly zero response from DSA. No support and no opposition. So I went ahead and contacted the last users of cvs.debian.org:

  • webwml: the website team
  • debian-doc: the Debian documentation project
  • debian-admin: the DSA team (this was already suggested in April this year in ticket #44, no response of course… except elmo saying me that he’s in favor. On IRC I also discovered that neuro doesn’t like bzr and is thus not in favor of such a move. Furthermore he visibly wants to keep control on userdir-ldap, thus he probably has not much interest in moving to a distributed system.)
  • buildd: the Packages-arch-specific file is maintained in the dak cvs…

All in all, the debian-doc and debian-www folks are rather supportive of the move, but it requires adjustment to the build infrastructure, in particular to keep track of the status of translations. I have no answer from DSA and the buildd guys however.

The web team started a wiki page to evaluate the VCS that they would switch to. Volunteers would be welcome to organize the conversion of the repositories and to fix the build infrastructure accordingly. This a nice little project for new contributors that want to learn. :-)

  • Share/Bookmark
Posted in Debian News | Tagged | 1 Comment