My Debian Activities in November 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (310.73 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg: Multi-Arch Saga

I know lots of people are waiting the landing of multiarch in Debian unstable, and so am I. Things are progressing, though not as quickly as I hoped. Guillem merged about half of the branch between the 24th October and the 6th of November. After that most of the work happened on his personal repository in his pu/multiarch/master branch.

I verify this repository from time to time because Guillem does not inform me when he has made progress. I noticed changes on his repository on the 10th, 19th, 23th, 28th of November and on the 1th of December.

He announced a long time ago that he had some “interface changes” and up to now only wrote about the switch from the command-line option --foreign-architecture (to put in /etc/dpkg/dpkg.cfg) to the explicit command dpkg --add-architecture that only needs to be called once (see mail here). As of today (December 2th), the promised email for the other interface changes is still not here.

On November 23th, I reviewed Guillem’s work and tried to run the code in his branch. I spent the whole day chasing up regressions and submitted lots of fixes to Guillem. Thanks to the extensive test-suite I wrote when I developed my branch, it has been fairly easy to track them all down.

All the issues I reported have been fixed in the latest version of Guillem’s branch although the fixes are often slightly different from those that I submitted.

Dpkg: Squeeze Backport

At the start of the month, I uploaded what I expected to be a fairly uncontroversial backport of dpkg 1.16.1.1. It turns out I was wrong.

After some discussion, I think we came to an agreement that it was acceptable to backport dpkg-dev and libdpkg-perl only. My goal was not to bring the latest dpkg to users but to make it easier for package maintainers to backport packages using new features provided by dpkg-dev >= 1.16 (such as hardening build flags, the makefile snippets provided in /usr/share/dpkg/, or the improved dpkg-buildflags interface).

Thus I modified the source package uploaded to squeeze-backports to build only dpkg-dev and libdpkg-perl. It has been uploaded on November 23th and it’s waiting in the NEW queue for a backports admin to process it.

Misc Dpkg Work

I merged a patch of Colin Watson to be able to verify build-dependencies for a foreign architecture (taking into account the Multi-Arch status of each package listed).

I released dpkg 1.16.1.2 with two minor fixes that were sitting in the sid branch. I wanted to get rid of this so that the path is clear for a 1.16.2 upload with multiarch. The package just migrated to testing so we’re fine.

I spent another day doing dpkg bug triaging on Launchpad, we’re now down to 77 bugs with many of them tagged as incomplete and likely to expire in 2 months.

The Debian Administrator’s Handbook

eBookWe released a sample chapter so that it’s easier to have an idea of the quality of the book. The chapter covers the APT tools quite extensively. I bet that even you could learn something about apt-get/aptitude…

The crowfunding campaign on Ulule ended on November 28th.
With 673 supporters, we raised 24345 EUR. Of those, 14935 EUR have been put in the liberation fund and the rest corresponds to the various pre-orders and rewards offered.

This means that the translation will happen (we just started) but that the book is currently not going to be released under a free license. Don’t despair… As planned, the liberation campaign is carried on until the 25 K€ target is reached!

Instead of being hosted on Ulule, this permanent campaign is on the project website at debian-handbook.info/liberation/. Note that any contribution of 10 EUR or more means that you get a copy of the ebook as soon as it’s available (even if the liberation target is not reached).

Package Tracking System

At the start of the month, I filed two ideas of improvements for the PTS in the bug tracking system: #647258 is about showing outstanding bugs that relate to a release goal and #647901 is about warning maintainers that the package is affected by a current transition. If you’re a coder and want to start contributing to Debian and its QA team, those bugs could be interesting targets for a start. :-) In both cases, I have been in contact with members of the release team because those ideas require some structured data from the release team as input. Thanks to Meddi Dohguy and Niels Thykier for their help.

Later in the month, the topic of relocating the PTS once again came up. For historical reasons, the PTS was hosted on master.debian.org together with the BTS. Nowadays the BTS has its own host and it made no sense anymore to have the PTS separate from the rest of the QA services hosted on qa.debian.org (currently quantz.debian.org). So together with Martin Zobel Helas we took care to plan the migration and on November 19th we executed the plan. It worked like a charm and almost nobody noticed (only one undocumented dependency was missed, which broke the SOAP interface).

Misc Packaging Work

WordPress was broken in Ubuntu and it was also not properly synchronized with Debian due to an almost useless change on their side. Thus I requested a sync so that the working version from Debian gets imported in Ubuntu.

I sponsored the docbook-xsl 1.76.1 upload that I needed for Publican. Then I updated Publican just to discover that the test-suite triggers a new bug in fop (filed as #649476). I disabled the test-suite temporarily and uploaded Publican 2.8 to unstable. BTW, I also filed 2 upstream bugs with patches for issues I discovered while trying to generate the sample chapter of my book (see here and here).

I uploaded a version 0.7.1 of nautilus-dropbox and fixed #648215 at the same time. I made an NMU of bison to fix a long-standing release critical bug that hit me once more during an upgrade (see #645038).

I uploaded to experimental a new version of gnome-shell-timer compatible with GNOME 3.2. I took the opportunity to install from experimental the few GNOME 3.2 packages which are not yet in unstable…

Thanks

See you next month for a new summary of my activities.

My Debian activities in October 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (130.30 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg work

The month started with fixing newly reported bugs to prepare the 1.16.1.1 release:

  • #644492: there was a flaw in a change I made to the trigger setup code. This resulted in packages being incorrectly marked as configured while they were only unpacked in a new chroot.
  • #642656: dpkg-source’s refusal to build when it detects unrecorded changes broke the (mostly unused, except by the lintian test suite apparently) “2.0” source format.
  • #644412: the Makefile snippet “buildflags.mk” did not respect the new maintainer specific environment variables (like DEB_CFLAGS_MAINT_APPEND) because make does not forward environment variable through $(shell …). Fixed that by manually exporting the required variables.
  • I also disabled dpkg-buildpackage’s output of the build flags since it was confusing several maintainers. dpkg-buildpackage invokes debian/rules and it has no (clean) way to discover the build flags changes that maintainer request by setting the dedicated environment variables in debian/rules. Maintainers expect to see the build flags with the modifications they have requested and not just the default values set by the distribution.

With the help of Guillem, we decided on a proper fix for a race condition sometimes triggered by parallel builds when 2 concurrent dpkg-gencontrol try to update debian/files (see #642608). This ended up requiring a new package (libfile-fcntllock-perl) that the Debian perl team kindly packaged for us. With all this sorted, it was a rather easy fix.

Multiarch progress

I also spent lots of time on multiarch. I fixed an old bug that requested to support the multi-arch paths in case of cross-building (see #595144), the discussion was not really conclusive on which of the two proposed patches was better so I ended up picking my own patch because it was closer to how we currently deal with cross-building. Then I fixed 2 issues that have been reported on Ubuntu’s dpkg. The first one (LP #863675) was rather severe since an installed package ended being “disappeared” in favor of its foreign counterpart that was removed (but that had some config files left). The second one (LP #853679) only affected dselect users (apparently there are still some!) who had a self-conflicting library (Provides: foo, Conflicts: foo) installed for multiple architectures.

But the bulk of the time spent on multiarch has been spent discussing with various parties on how to go forward with multiarch. The release team commented on the schedule of the merge to ensure it makes it into Wheezy, and the Debian project leader also commented on the problems encountered so far.

While not the best course of action I could have hoped for, it certainly helped since Guillem started pushing some reviewed commits. Out of the 66 commits that were in my pu/multiarch/full branch one week ago, 20 have been merged in the master branch already.

Python-django security update and RC bug

Since python-django’s maintainer did not manage to prepare the required security updates, I stepped in and prepared version 1.2.3-3+squeeze2 for Squeeze and 1.0.2-1+lenny3 for Lenny. Unfortunately this security update is an example of how an inactive maintainer is likely to result in a severe delay for the release of security updates.

Furthermore in this specific case, the security team did not want to release the Squeeze security update until the Lenny one had been investigated (which required some time since upstream no longer supports the version in Lenny) but they did not make this very clear.

Later another release critical bug had been filed against the package (#646634) but after investigation, it turned out to be a local configuration problem so I downgraded it. I still forwarded the test suite failure to upstream authors since the test could be enhanced.

In any case, co-maintainers for python-django are welcome. I really preferred the situation where I can quietly sit down as backup maintainer… :-)

WordPress packaging

WordPress sounds similar to python-django. I’m also “only a backup maintainer” but Giuseppe has been inactive for many months and I had to step in August because I wanted the new upstream version. I discovered a bit late that I was not subscribed to wordpress’ bugs and thus the release critical bug #639733 (that I introduced with my new upstream version) went unattended for a rather long time. Once aware, though, I quickly fixed it.

I also took the opportunity to start a discussion on debian-devel about how to deal with embedded javascript libraries and proposed a mechanism of “opportunistic replacement with symlinks”. WordPress is my testbed package for this mechanism, you can check out its debian/dh_linktree that implements the replacement logic.

The discussion has not been very interesting but at least I learned that Debian now requires that each source package shipping minified javascript files includes the original files too. It’s somewhat of a pain since it’s not a license requirement in many cases (many of those libraries are not under the GPL), but just a Debian requirement that many upstreams are not complying with. WordPress is affected and Jakub Wilk thus opened #646729 which is going to be a long-standing RC bug. To give good measures, I spent several hours investigating the case of each javascript file in the WordPress source package and I filed a new ticket on the upstream bugtracker.

Dropbox packaging work

A few months after the introduction of nautilus-dropbox to Debian and Ubuntu, I can say that the decision to only support the download of dropbox in the postinst has been a mistake. Because of this decision I had to make the postinst fail if the download failed. Even if the error message is relatively clear, this lead to many (mostly automated) bug reports on the Ubuntu side. Various other problems cropped up on top of this (trying to start dropbox while the package was not configured would result in an error because the user did not have the required rights to install the software, reinstalling the package while dropbox was running would result in a failure too, etc.).

I have fixed all those issues in the version 0.7.0-2 of the package. Now if the user has to install dropbox, it will use PolicyKit to request the root rights. The postinst will no longer fail if the dropbox download fails since it can be run later by the user. And I fixed the download code to remove the replaced file before unpacking a new file (insead of overwriting the existing file). All this work has been forwarded upstream.

The Debian Administrator’s Handbook Update

I’m glad to tell you that the translation will happen because we reached the minimal funding goal on October 22th with the help of 380 supporters.

Now the fundraising continues, but this time the goal is the liberation of the resulting book. For this to happen, we need to reach 25000 EUR in the liberation fund. So far we’re at 37% of this goal with 9400 EUR in the liberation fund (which means that 59% of the money raised has been put in the liberation fund).

Click here if you want to contribute towards the liberation of this book.

With (less than) 27 days left, it’s going to be a challenge to meet the goal, but we do like challenges, don’t we?

Misc work

  • I filed #644486 against dh-make so that new packages have proper support of dpkg-buildflags from the start.
  • I merged lots of patches from Luca Falavigna in the developers-reference.
  • I discussed debtags integration in the PTS with Enrico Zini and Paul Wise.
  • I updated publican’s packaging for the new upstream version 2.8. I had to write a new patch that I forwarded upstream.
  • I filed an upstream bug on hamster-applet because just running hamster-time-tracker no longer brings its window forward.

Thanks

See you next month for a new summary of my activities.

My Debian activities in September 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (144.3 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg work

While taking care of the last details for the hardening feature in dpkg 1.16.1, I have mailed debian-devel to find volunteers to handle a hardening release goal. The objective is to ensure a large number of packages have been converted/rebuilt to actually use the new hardening build flags.

Then I prepared the draft of the announce of the dpkg 1.16.1 upload (aka Bits of dpkg maintainers sent to debian-devel-announce) which got expanded by Guillem to also cover new features since dpkg 1.15.7.

update-alternatives got some refactoring by Guillem which resulted in a regression that has been fortunately discovered by Sven Joachim. I fixed that regression and did some further cleanup inspired by the root cause of this regression (see top 4 commits here).

Note that Sven is one of the few persons who are running the git version of dpkg. Hopefully the number of tester will increase since I recently documented the APT repositories with autobuilt versions of dpkg in the wiki.

At the end of the month, I started working on a bugfix release (what’s going to be 1.16.1.1) by fixing some of the unavoidable problems discovered after an upload that accumulated more than 4 months worth of work (see top 4 commits here).

The Debian Administrator’s Handbook

I spent countless hours finalizing the launch of the crowdfunding campaign for the Debian Administrator’s Handbook and it went live on September 27th.

So far it’s on good track with more than 63% of the base funding already secured. But we still have a long way to go to reach the liberation goal (we’re at 21%). It’s still worth nothing that more than 55% of the money raised has been put in the liberation fund so there are many persons who care about getting the book freed.

More than 250 persons are supporting the project currently with an average contribution of 38 EUR. I would have expected much less for the average contribution but many more supporters. I still hope we can get more people on board with the perspective of a good DFSG-free Debian ebook.

Did you order your copy? If not, click here and fix this! ;-) By the way Paypal used to be required but it’s no longer the case, you can support the project just with your usual credit card.

Misc blog updates

Over time, I have written many useful articles for Debian users and Debian contributors. But scattered in the history, they are somewhat difficult to find. To fix this I have created some index pages listing them. Check them out:

Two new articles joined those pages this month: How to triage bugs in the Debian Bug Tracking System and Understand dpkg and don’t get stuck with a maintainer script failure.

While writing the first article, I noticed we lacked a good page showing the most buggy packages so I quickly created it (with the help of UDD): http://qa.debian.org/cgi-bin/bugs-by-source

Misc packaging work

I did a small update to the developer’s reference. Luca Falavigna submitted a patch to clarify how one is supposed to deal with meta-packages (cf #569219), I improved it and integrated the result in the SVN repository.

I upgraded nautilus-dropbox to version 0.6.9 and while doing this I discovered a bug in mergechanges (filed as #640782). I uploaded a new release of quilt mainly to add the Multi-Arch: foreign field so that it can satisfy dependencies of foreign packages (i.e. packages of a different architecture).

Django released some security advisories (tracked in #641405) and since the maintainer did not deal with the issue, I stepped up to the task (I’m a backup maintainer) and released the fixed version 1.3.1 to unstable. I took the opportunity to switch from python-support to dh_python2, and do some misc improvements to the packaging (see changelog).

I wanted to update publican to a newer version but it turned out to be not possible because Debian doesn’t have the latest version of docbook-xsl yet. I also discovered some bugs in the test suite and forwarded upstream the patch I created (see upstream bug). On top of this, fop was failing due to some java problem related to the introduction of multiarch. After having reported the bug, the java maintainers quickly released a fixed version.

So now publican is ready in the git repository but it’s waiting on the docbook-xsl update. I got in touch with the maintainer who said he would have the time to take care of it by mid-october.

Thanks

See you next month for a new summary of my activities.

My Debian activities in August 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (91.44 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg work

When I came back from Debconf, I merged my implementation of dpkg-source --commit (already presented last month). I continued some work on the hardening build flags but it’s currently stalled waiting on Kees Cook to provide the required documentation to integrate in dpkg-buildflags(1).

Following a discussion held during DebConf, Michael Prokop has been kind enough to setup a git-triggered auto-builder of dpkg (using Jenkins). You can now help us by testing the latest git version. Follow those instructions:

$ wget -O - http://jenkins.grml.org/debian/C525F56752D4A654.asc | sudo apt-key add -
$ sudo sponge /etc/apt/sources.list.d/dpkg-git <<END
deb http://jenkins.grml.org/debian dpkg main
END
$ sudo apt-get update && sudo apt-get upgrade

On the bug fixing side I took care of #640198 (minor man page update), #638291 (a fix to correctly handle hardlinks of conffiles), #637564 (the simplification logic of union dependencies was broken in some cases) and #631494 (interrupting dpkg-source while building a native source package left some temporary files around that should have been cleaned).

WordPress update

I released WordPress 3.2.1 in unstable (after having taken the time to test the updated package on my blog!) and fixed its RC bug (#625773). In the process I discovered a false positive in lintian (I reported it in 637473).

Gnome-shell-timer package

From time to time, I like to use the Pomodoro Technique. That’s why I was an user of timer-applet in GNOME 2. Now with the switch to GNOME 3, I lost this feature. But I recently discovered gnome-shell-timer, a GNOME Shell extension that provides the same features.

I created a Debian package of it and quickly filed some bugs while I was testing it (two usability issues and an encoding problem)

QA Work

During DebConf I met Giovanni Mascellani and he was interested to help the QA team. He started working on the backlog of bugs concerning the Package Tracking System (PTS) and submitted a bunch of patches. I reviewed them and merged them but since they were good, I quickly got lazy and got him added to the QA team so that he can commit his fixes alone. It also helps to build trust when you have had the opportunity to discuss face to face. :-)

Vacation

That’s not so much compared to usual but to my defense I also took 2 weeks of vacation with my family. But somehow even in vacation I can’t really forget Debian. Here’s my son:

Thanks

See you next month for a new summary of my activities.

Google plus and server to give away

Just a quick note to let you know that (like many free software hackers apparently) I have an account on Google+.

I’m not using it much yet but I like it in general. It’s interesting to see how Google transformed Joindiaspora‘s aspects into “circles”.

When Google will make an API available, I’ll probably setup it like my public Facebook page so that new blog posts are automatically announced. In the mean time, it’s going to be very quiet on my Google+ profile.

That said, I used it twice this week: the first time because I’m looking for a French developer with sysadmin skills, and the second time because I have a server to give away (Pentium IV 3Ghz, 4 Gb RAM, 200 Gb of diskspace in RAID1 Hard). If you take the server for a free software project, it can be hosted for free where it currently is (courtesy of Julien Danjou).

So if you’re also a Google+ user, feel free to add me to one of your circles.

My Debian activities in July 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (170 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

This month passed by very quickly since I attended both the Libre Software Meeting / RMLL and the DebConf.

Libre Software Meeting / RMLL

I attended “only” 3 days out of the 6 but that was a deliberate choice since I was also attending DebConf for a full week later in the month.

During those 3 days I helped with the Debian booth that was already well taken care of by Frédéric Perrenot and Arnaud Gambonnet. Unfortunately we did not have any goodies to sell. We (as in Debian France) should do better in this regard next time.

One of the talks I attended presented EnVenteLibre. This website started as an online shop for two French associations (Ubuntu-fr, Framasoft). They externalize all the logistic to a company and only have to care about ordering goodies and delivering to the warehouse of the logistic company. They can also take some goodies from the warehouse and ship them for a conference, etc. We discussed a bit to see how Debian France could join, they are even ready to study what can be done to operate at the international level (that would be interesting for Debian with all the local associations that we have throughout the world).

Back to the LSM, while I had 3 good days in Strasbourg, it seems to mee that the event is slowly fading out… it’s far from being an international event and the number of talks doesn’t make for a better quality.

BTW, do you remember that Debconf 0 and Debconf 1 were associated to this event while it was in Bordeaux?

dpkg-source improvements

During my time in Strasbourg (and in particular the travel to go there and back!) I implemented some changes to “3.0 (quilt)” source format. It will now fail to build the source package if there are upstream changes that are not properly recorded in a quilt patch:

dpkg-source: info: local changes detected, the modified files are:
 2ping-1.1/README
dpkg-source: info: you can integrate the local changes with dpkg-source --commit
dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/2ping_1.1-1.diff.cki8YB

As the error message hints, there’s a new --commit command supported by dpkg-source that will generate the required quilt patch to fix this. In the process you will have to submit a name and edit the patch header (pre-formatted with DEP3 compatible fields). You can get back the old behavior with the --auto-commit option.

Build flags changes

Ever since we adopted the Ubuntu changes to let dpkg-buildpackage set some build related environment variables (see #465282), many Debian people expressed their concerns with this approach both because it broke some packages and because those variables are not set if you execute debian/rules directly.

In the end, the change was not quickly reverted and we fixed the package that this change broke. Despite this we later decided that the correct approach to inject build flags would be a new interface: dpkg-buildflags.

Before changing dpkg-buildpackage to no longer set the compilation flags, I wanted to ensure dpkg-buildflags had some decent coverage in the archive (to avoid breaking too many packages again). My criteria was that CDBS and dh (of debhelper) should be using it. With the recent debhelper change (see #544844) this has been reached so I changed dpkg-buildpackage accordingly.

Makefile snippets provided by dpkg

At the same time, I also wanted an easy way for maintainers not using dh or CDBS to be able to fix their package easily and go back to injecting the compilation flags in the environment but doing it from the rules files. Starting with the next version of dpkg, this will be possible with something like this:

DPKG_EXPORT_BUILDFLAGS = 1
include /usr/share/dpkg/default.mk

Without DPKG_EXPORT_BUILDFLAGS the variables are not exported in the environment and have no effect unless you use them somewhere.

More than build flags, this will also provide a bunch of other variables that can be useful in a rules files: all the variables provided by dpkg-architecture, vendor related variables/macro and some basic package information (mainly version related).

dpkg-buildflags improvements

Given the renewed importance that dpkg-buildflags will take now that dpkg-buildpackage no longer sets the corresponding environment variables, I thought that I could give it some love by fixing all the open issues and implementing some suggestions I got.

I also had a chat with a few members of the technical committee to discuss how hardening build flags could be enabled in Debian and this also resulted in a few ideas of improvements.

In the end, here are the main changes implemented:

  • new “prepend” directive to inject flags at the start (see commit);
  • new “strip” directive to strip flags from the result returned by dpkg-buildflags (see commit);
  • new environment variables DEB_flag_MAINT_directive that can be set by the maintainer to adjust what dpkg-buildflags will return (see commit);
  • new --export=configure command to inject build flags on the ./configure command line (see commit);
  • new --dump command that is the default (see #603435).

Will all those changes, the complete set of compilation flags can be returned by dpkg-buildflags (before it would only return the default flags and it was expected that the Debian packaging would add whatever else is required afterwards). Now the maintainer just has to use the new environment variables to ensure the returned values correspond to what the package needs.

DebConf: rolling and hardening build flags

I spent a full week in DebConf (from Sunday 24th to Sunday 31th) and as usual, it’s been a pleasure to meet again all my Debian friends. It’s always difficult to find a good balance between attending talks, working in the hacklab and socializing but I’m pretty happy with the result.

I did not have any goal when I arrived, except managing the Rolling Bof (slides and video here) but all the discussions during talks always lead to a growing TODO list. This year was no exception. The technical committee BoF resulted in some discussions of some of the pending issues, in particular one that interests me: how to enable hardening build flags in Debian (see #552688).

We scheduled another discussion on the topic for Tuesday and the outcome is that dpkg-buildflags is the proper interface to inject hardening build flags provided that it offers a mean to drop unwanted flags and a practical way to inject them in the ./configure command line.

Given this I got to work and implemented those new features and worked with Kees Cook to prepare a patch that enables the hardening build flags by default. It’s not ready to be merged but it’s working already (see my last update in the bug log).

A few words about the Rolling BoF too. The room was pretty crowded: as usual the topic generates lots of interest. My goal with the BoF was very limited, I wanted to weigh the importance of the various opinions expressed in the last gigantic discussion on debian-devel.

It turns out a vast majority of attendants believe that testing is already usable. But when you ask them if we must advertise it more, answers are relatively mixed. When asked if we can sustain lots of testing/rolling users, few people feel qualified to reply but those that do tend to say yes.

More dpkg work

Lots of small things done:

  • I did again some bug triaging on Launchpad. But Brian Murray did a lot of it and the result is impressive, we’re down to 154 bugs (from more than 300 a month ago!).
  • I updated my multiarch branch multiple times. I was hoping to meet Guillem during DebConf to make some progress on this front but alas he did not attend. I have been asked a status update multiple times during my time in DebConf.
  • I fixed a regression in update-alternatives (#633627), a test-suite failure when run as root (#634961), a segfault in findbreakcycle. There have been a bunch of minor improvements too (#634510, #633539, #608260, #632937).

Package Tracking System and DEHS

Christoph Berg recently wrote a replacement for DEHS because the latter was not really reliable and not under control of the QA team. This is a centralized system that uses the watch files to detect new upstream versions of the software available in Debian.

I updated the Package Tracking System to use this new tool instead of DEHS. The new thing works well but we’re still lacking the mail notifications that DEHS used to send out. If someone wants to contribute it, that would be great!

Misc packaging work

I did some preliminary work to update the WordPress package to the latest upstream version (3.2). I still have to test the resulting package, replacing upstream shipped copies of javascript/PHP libraries is always a risk and unfortunately all of them had some changes in the integration process.

I also updated nautilus-dropbox to version 0.6.8 released upstream. I also uploaded the previous version (that was in testing at that time) to squeeze-backports. So there’s now an official package in all the Debian distributions (Squeeze, Wheezy, Sid and Experimental)!

Thanks

See you next month for a new summary of my activities.

My Debian activities in May 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work, then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

I have been…

Doing some work towards Debian Rolling

At the start of the month, the discussions about Debian rolling were still very active on debian-devel. Declaring that testing would be rolling did not make it (as I hoped), the argument that some RC bugs last for far too long in that distribution carried the discussion and thus the most consensual proposition ended up being the one of Josselin Mouette were rolling would be testing plus a few selected cherry-picked packages from unstable.

I believe it’s a workable solution if we only care about a subset of architectures. Otherwise the same reasons that keep the fixed packages out of testing would probably also apply for rolling.

Given this, I did setup britney (the software that controls testing) on my laptop to investigate how we can create rolling. It turns out britney is a very specialized software with very few configuration knobs.

At the same time Joachim Breitner made a proposition that immediately grabbed my attention. He suggests to use SAT solvers to find out the set of packages that should migrate from unstable to testing. I thought that rolling would be a good testbed for this new implementation of britney (which he calls SAT-britney) so I jumped right in this project.

I was not at all familiar with this science field, so I looked up quite some documentation: I learned that all SAT solvers expect the problem to be presented in CNF form, and that DIMACS was the file format of choice to represent those boolean constraints. Several SAT solvers are available in Debian and picosat appears to be one of the best.

Then I started some early coding/prototyping to play with the concept. You can find the result in this git repository, you can grab a copy with git clone git://git.debian.org/~hertzog/sat-britney.git.

There’s not much yet, except some Python code to generate a SAT problem that can be fed to a SAT solver. But I really look forward to this project.

Representing Debian during Solutions Linux

During the second week, I spent 3 days in Paris to help manage the Debian booth at Solutions Linux.

We have responded to lots of queries but most visitors already knew Debian, and many of them use it at work and/or at home. We tried to recruit those people as new members for Debian France, the local association. We also sold all our remaining goodies.

The Ubuntu people were interviewed by France 3 (an important TV channel) and we took this opportunity (with the consent of the Ubuntu guys) to show our Debian t-shirts in the background: you can watch the video here (in French), you can see me with Carl Chenet at 1:21.

We have also been interviewed by Intelli’n TV: here and here (both in French). I’m not very good at this exercise. :-)

Improving dpkg triggers

The third week was a vacation week, in theory I should have stayed away from my computer but I really wanted to take this opportunity to improve the state of dpkg triggers in Debian.

I already covered my work in another article: Trying to make dpkg triggers more useful and less painful.

The result is not merged yet, I just asked a question to all package maintainers who are using triggers to be able to decide whether I’ll merge it as is, or if I can make the new behavior the default one.

Supporting users after Alioth’s migration

When I came back from my vacation, many services provided by Alioth.debian.org were non-functional after a migration to a new setup that involves two machines instead of one. Given that I used to be an Alioth admin, I know that in those periods you tend to be get bogged down on many user support requests. So I re-joined #alioth on IRC and tried to help a bit.

I did investigate some of the reported problems and prepared fixes (updated scripts, configuration files, etc.) for some of the issues. I also created a list of remaining issues that should have lasted only a few days but that’s still active because there are still regressions left.

The most important things still missing are:

  • proper support for delegation of rights. We used ACL setup by the admins in the past. With the new FusionForge, each project admin should be able to delegate rights to external “roles”. There’s a Debian Developer role already but trying to grant him right fails…
  • access to the Ultimate Debian Database. Many tools rely on this database to work.
  • anonymous FTP access to download project files.
  • clear guidelines on how we’re supposed to deal with websites that are updated by VCS hooks.
  • clear guidelines on how we’re supposed to deal with personal git repositories

Improving the “3.0 (quilt)” source format

I have made some proposals to change the way the new source format would work. The goals are to be less painful for packagers who are using a VCS, and to avoid unexpected changes slipping through a new patch generated by dpkg-source.

It seems that the proposals are relatively consensual so I’ll implement them at some point.

Missing in action on my blog

I did a lots of stuff for Debian between travel and vacation, and in the remaining time, I did not manage to write many articles for my blog.

In fact, besides the article on my triggers work mentioned above I only published one interview: People behind Debian: Steve Langasek, release wizard.

I’ll try to do better this month!

Thanks

Many thanks to the people who gave me 151.61 € in May.

See you next month for a new summary of my activities.

My Debian activities in April 2011

This is my monthly summary of my Debian related activities. If you’re among the people who support my work, then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

GNOME 3 packaging

Right after the GNOME 3 release, I was eager to try it out so I helped the pkg-gnome team to update some of the packages. I did some uploads of totem, totem-pl-parser, gvfs, mutter, gnome-shell, gnome-screensaver. I also kept people informed via my blog and prepared a pinning file for adventurous users who wanted to try it out from experimental (like me).

One month later, I’m still using GNOME 3. There are rough edges still, but not so many. And I’m starting to get used to it.

Debian Rolling planning

Debian Rolling is a project on my TODO list for quite some time. I decided it was time to do something about it and started a series of articles to help clarify my ideas while getting some early feedback. My goal was to prepare a somewhat polished proposal before posting it to a Debian mailing list.

But as usual with Murphy’s law, my plan did not work out as expected. Almost immediately after my first post the discussion started on debian-devel:

At this point it’s a discussion thread of several hundreds of messages (there are several screens of messages like the one above). Many of the sub-threads have been interesting, but the general discussions mixed too many different things so that there’s no clear outcome yet. Lucas Nussbaum tried to make a summary.

Obviously I must adjust my plan, there’s lots of feedback to process. I accepted to drive a DEP together with Sean Finney to help structure the part of the discussion that focuses on allowing development to continue during freezes. But I’m also eager to fix the marketing problem of testing and have the project recognize that testing is a product in itself and that end-users should be encouraged to use it.

Package Tracking System maintenance

The Package Tracking System is an important tool for Debian developers, and it has been broken by some change on the Bug Tracking System. I worked around it quite quickly so that few people noticed the problem but Cron kept reminding me that I had to properly fix it.

I ended up doing it last week-end. While working on the PTS, I took the opportunity to merge a patch from Jan Dittberner to enhance the news RSS feed that the PTS provides. And I also integrated information from backports.debian.org (thanks to Mehdi Dogguy for reminding me #549115).

Multiarch update

Not much new this month. I fixed two bugs in the multiarch dpkg branch thanks to bug reports from Ubuntu users (LP 767634, LP 756381). I’m still waiting on Guillem Jover finishing his review of the multiarch branch. I’m pinging him from time to time but it looks like multi-arch is no longer in his short term priority list. :-(

I’ve been running this code for more than 2 months and it works fine. I want to see it merged. I’m ready to update my code should anything need to be changed to please Guillem. But without any feedback we’re in a deadlock.

Misc dpkg work

While fixing a bug in update-alternatives (found in one of the valid reports on launchpad), I noticed that there was room for improvements in the error messages output by update-alternatives. I changed them to reuse the same strings that were already used in other parts of dpkg. The result is that there are a few strings less to translate (always a nice thing for the poor translators who have to deal with the thousands of strings that dpkg contains).

I also tried to fix some of the most cryptic error messages in dpkg (see #621763) but that work is stalled at the request of Guillem.

Book update

We (me and Roland Mas) are almost done with the update of our French book for Debian Squeeze. It will hit the shelves in July or September. I’m starting to prepare the fundraising campaign to make an English translation of it. We’ll use ulule.com for this.

On my blog

I have been pleased to interview Meike Reichle, it’s the first women that I have interviewed in the series but it’s certainly not the last one. I also interviewed Adam D. Barratt, one of our tireless release managers.

Thanks

Many thanks to the people who gave me 180.35 € in March and 235.37 € in April. That represents 1.5 and 2 days of work for those months.

See you next month for a new summary of my activities.

March 2011 wrap up

Since I’m soliciting donations to support my Debian work, the least I can do is explain what I do. You can thus expect to see an article like this one every month.

Multi-Arch work

I updated the code to use another layout for the control files stored in /var/lib/dpkg/info/. Instead of using a sub-directory per architecture (arch/package.type), we decided to use package:arch.type but only for packages which are Multi-Arch: same. dpkg is taking care to rename the files the first time it is executed with write rights and then updates /var/lib/dpkg/info/format to remember that the upgrade has been done and that we can rely on the new structure.

I filed a few bugs on packages that are improperly accessing those internal files instead of using the appropriate dpkg-query interface. I sent a heads-up mail on -devel to make other people aware of those problems in the hope to discover most of them as early as possible.

After that, the work stalled because Guillem went away for 2 weeks and thus stopped his review of my work. I hope he will quickly resume the review and that we will get something final this month.

With the arrival of dpkg 1.16.0, it’s now possible to start converting libraries to multi-arch even if full multi-arch support has not yet landed in dpkg proper. See http://wiki.debian.org/Multiarch/Bootstrapping for the detailed plan.

If you’re curious about Multi-Arch, you might want to read this article of Steve Langasek as well.

Bug triage for dpkg in launchpad

At the start of the month, there was close to 500 bugs reported against the dpkg package in Launchpad. Unfortunately most of it is noise… many of the reported bugs are misfiled, they show an upgrade problem of a random package and that upgrade problem confuses update-manager which tries to configure an already configured package. This generates a second error that apport attributes to dpkg and the resulting bug report is thus filed on dpkg. There are literally hundreds of those that have to be reclassified.

Michael Vogt and Brian Murray did some triaging, and I also spend quite some hours on this task. It’s a bit frustrating as I tend to mark many reports “Incomplete” because there’s no way they can be acted upon and many of them are so old that the reporter is unlikely to be able to provide supplementary information.

But in the middle of this noise, there are some useful bug reports, like LP#739179 which enabled me to fix a regression even before it reached Debian Unstable (because Ubuntu runs a snapshot of dpkg with multiarch support).

I subscribed to the Launchpad bugs for dpkg via the Debian Package Tracking System (thanks to the derivatives-bugs keyword) and will try to keep up with the incoming reports.

Misc dpkg work

The ftpmasters came up with a request for a new field (see 619131) in source packages. After a quick discussion and a round of review on debian-policy@l.d.o, I implemented the new Package-List field. This should allow the ftpmasters to save some time in NEW processing, but we deferred the change for the next dpkg version (1.16.1) to ponder a bit more on the design of the field.

I also fixed a bunch of bugs (#619541, #605719, #598922, #616096) and merged a patch of Mark Hymers to recognize the new Built-Using field.

Developers-reference work

The review process for changes to the developers-reference is not working as it should. And I suffered from it while trying to integrate the patch I wrote for the “Developer duties” chapter (see #548867).

We purposely changed the maintainer field from debian-doc to debian-policy in the hope to have more reviews of suggested changes and to seek some sort of consensus before committing anything. But we don’t get more reviews… and deciding to commit a patch is now even harder than it was (except for trivial stuff where personal opinions can’t interfere).

In my case, I only got the feedback of Charles Plessy which was very mixed to say the least. I tried to improve my patch based on what he expressed but I also clearly disagreed with some of his assertions and was convinced that my wording was in line with the dominant point of view within Debian.

We tried to involve the release team in the discussion because most of what I documented was about helping making stable release happen, but nobody of the team answered.

Instead of letting the situation (and my patch) rot, I solicited feedback from the DPL and from another developers-reference editor to see whether my patch was an improvement or not. After some more time, I went ahead and committed it.

It was not pleasant for anyone.

I don’t know how we can improve this. Contrary to the policy, the developers-reference is a document that is not normative, I believe the result is better when we put some “soul” into it. But it’s a real challenge when you seek a consensus and that the interest in reviewing changes is so low.

DVD shop listed on debian.org

In February, I launched a DVD shop whose benefits are used to fund my Debian work. Shortly after the launch I used the official form to be added to the official listing of Debian CD vendors and offered a few suggestions to deal with vendors who are selling unofficial images (with firmware in my case).

A few weeks later, I got no answers: neither for my request nor for my suggestions, I mailed the cdvendors@debian.org team directly asking for a status update and quickly got an answer suggesting that Simon Paillard usually does the work and can’t process the backlog due to some injury. At this point no concerns had been raised about adding me to the list. To save some time and some work for the team, I added myself to the list since I had commit rights and I informed them that I did it, so that they can review it.

Shortly after I did that, Martin Zobel Helas objected to my addition. I cleared some misunderstandings but the discussion also lead to some changes to please everybody: the listing now indicates that some images are unofficial and I have prepared a special landing page for people coming from the Debian website through this listing.

Debian column on OMG! Ubuntu

I have always been a firm believer that it’s important for Debian to reach out to the widest public with its message of freedom. Thus when Benjamin Humphrey contacted the debian-publicity team to find volunteers to write a Debian column on OMG! Ubuntu, I immediately jumped in.

I wrote 4 articles over there. The tone is very different from my articles on my blog and I like that duality. Check out Debian is dying! Oh my word!, Debian or Ubuntu, which is the best place to contribute?, Are you contributing your share? and Ubuntu’s CTO reveals DEX: an effort to close the gap with Debian.

It’s a great win-win situation, OMG! Ubuntu benefits from my articles, Debian’s values are relayed further, and OMG! Ubuntu’s large audience also helps me develop my own blog.

Work on my book

I had lots of paperwork to do this month (annual accounting stuff for my company) and I did not have as much time as I hoped for my book. Still I have a updated a few more chapters of my French book and I certainly hope to complete the update during April.

This means that the work on the English translation could start in may.

Work on my blog

Just like for my book, it has been relatively difficult for me to cope with my policy of two articles every week. But I still managed to get quite some good stuff out.

I interviewed Christian Perrier (Debian’s translation coordinator) and also Bdale Garbee (chair of Debian’s technical committee).

I finished my series of “Debian Cleanup Tips” with 2 supplementary articles:

The removal of firmware is causing troubles to quite some users so I wrote an article explaining how to deal with the problem. A regular reader also asked me to write an article about Jigdo, I executed myself because it was a good idea and that he has been very nice with me: Download ISO images of Debian CD/DVD at light speed with Jigdo.

Last but not least, I shared my package maintainer pledge which inspired my developers-reference patch (see discussion above).

Thanks

Many thanks to all the people who showed their appreciation of my work. The 324.37 EUR that you gave me in February represented 2 days and a half of my time that I have spent working on the above projects.

See you next month for a new summary of my activities.

February 2011 wrap up

February has been again a busy month for me. Here’s a quick summary of what I did:

Multi-Arch work

I have spent many days implementing and refining dpkg’s Multi-Arch support with Guillem Jover (dpkg co-maintainer) and Steve Langasek (beta-tester of my code ;-)). Early testers can try what’s in my latest pu/multiarch/snapshot/* branch in my personal git repository.

A Debian DVD shop

I’m always exploring new options to fund my Debian work (besides direct donations) and this month—with the Debian Squeeze release—I saw an opportunity in selling Debian DVD. Nobody provides DVD with included firmwares and quite a few people would like to avoid the SpaceFun theme. So I built unofficial Debian DVDs that integrate firmware and that install a system with the old theme (MoreBlue Orbit). Click here to learn more about my unofficial DVDs.

On my blog

In my “People behind Debian” series, I interviewed Mike Hommey (Iceweasel maintainer) and Maximiliam Attems (member of the kernel team).

I started a “Debian Cleanup Tip” series and already published 4 installments:

For contributors, I wrote two articles: the first gives a set of (suggested) best practices for sponsoring Debian packages and adapted my article as a patch for the Developers Reference. In the second article, I shared some personal advice for people who are considering participating on Debian mailing list: 7 mistakes to avoid when participating to Debian mailing lists.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.