My Debian Activities in February 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (384.14 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg and multiarch

The month started with a decision of the technical committee which allowed me to proceed with an upload of a multiarch dpkg even if Guillem had not yet finished his review (and related changes). Given this decision, Guillem made the experimental upload himself.

I announced the availability of this test version and invited people to test it. This lead to new discussions on debian-devel.

We learned in those discussions that Guillem changed his mind about the possibility of sharing (identical) files between multiple Multi-Arch: same packages, and that he dropped that feature. But if this point of the multiarch design had been reverted, it would mean that we had to update again all library packages which had already been updated for multi-arch. The discussions mostly stalled at this point with a final note of Guillem explaining that there was a tension between convenience and doing the right things every time that we discuss far-reaching changes.

After a few weeks (and a helpful summary from Russ Allbery), Guillem said that he remained unconvinced but that he put back the feature. He also announced that he’s close to having completed the work and that he would push the remaining parts of the multiarch branch to master this week (with the 1.16.2 upload planned next week).

That’s it for the summary. Obviously I participated in the discussions but I didn’t do much besides this… I have a “mandate” to upload a multiarch dpkg to sid but I did not want to make use of it while those discussions remained pretty unconclusive. Also Guillem made it pretty clear that the multiarch implementation was “buggy”, “not right” and “not finished” and that he had reworked code fixing at least some of the issues… since he never shared that work in progress, I also had no way to help even just by reviewing what he’s doing.

We also got a few multiarch bug reports, but I couldn’t care to get them fixed since Guillem clearly held a lock on the codebase having done many private changes… it’s not quite like this that I expect to collaborate on a free software project but life is full of surprises!

I’ll be relieved once this story is over. In the mean time, I have added one new thing on my TODO list since I made a proposal to handle bin-nmu changelogs and it’s something that could also fix #440094.

Misc dpkg stuff

After a discussion with Guillem, we agreed that copyright notices should only appear in the sources and not in manual pages or --version output, both of which are translated and cause useless work to translators when updated. Guillem already had some code to do it for --version strings, and I took care of the changes for the manual pages.

I merged some minor documentation updates, fixed a bug with a missing manpage. Later I discovered that some recent changes lead to the loss of all the translated manual pages. I suggested an improvement to dh_installman to fix this (and even prepared a patch). In the end, Guillem opted for another way of installing translated manual pages.

Triggered by a discussion on debian-devel, I added a new entry to my TODO list: implementing dpkg-maintscript-helper rm_conffile_if_owner to deal with the case where a conffile is taken over by another package which might (or might not) be installed.

Misc packaging

At the start of the month, I packaged quilt 0.51. The number of Debian specific patches is slowly getting down. With version 0.51, we dropped 5 patches and introduced a new one. Later in the month I submitted 4 supplementary patches upstream which have been accepted for version 0.60.

This new version (just released, I will package it soon) is an important milestone since it’s the first version without any C code (Debian had this for a long time but we were carrying an intrusive patch for this). Upstream developer Jean Delvare worked on this and based his work on our patch, but he went further to make it much more efficient.

Besides quilt, I also uploaded dh-linktree 0.2 (minor doc update), sql-ledger 2.8.36 (new upstream version), logidee-tools 1.2.12 (minor fixes) and publican 2.8-2 (to fix release critical bug #660795).

Debian Consultants

The Debian Project Leader is working on federating Debian Companies. As the owner of Freexian SARL, I was highly interested in it since Freexian “contributes to Debian, offers support for Debian and has a strategic interest in Debian”. There’s only one problem, you need to have at least 2 Debian developers on staff but I have no employees (it’s me only). I tried to argue that I have already worked with multiple Debian developers (as contractors) when projects were too big for me alone (or when I did not have enough time). Alas this argument was not accepted.

Instead, and since our fearless leader is never afraid to propose compromises, he suggested me (and MJ Ray who argued something similar than me) to try to bring life to the Debian Consultants list which (in his mind) would be more appropriate for one-man companies like mine. I accepted to help “animate” the list, and on his side, he’s going to promote both the “Debian Companies” and the “Debian Consultants” lists.

In any case, the list has seen some traffic lately and you’re encouraged to join if you’re a freelancer offering services around Debian. The most promising thing is that James Bromberger offered to implement a real database of consultants instead of the current static page.

Book update

We made quite some progress this month. There’s only one chapter left to translate. I thus decided to start with proofreading. I made a call for volunteers and I submitted one (different) chapter to 5 proofreaders.

The liberation campaign made a nice leap forwards thanks to good coverage on barrapunto.com. We have reached 80% while we were only at 72% at the start of the month (thanks to the 113 new supporters!). There’s thus less than 5000 EUR to raise before the book gets published under a free license.

Looking at the progression in the past months, this is unlikely to be completed on time for the release of the book in April. It would be nice though… so please share the news around you.

Speaking of the book’s release, I’m slowly preparing it. Translating docbook files is not enough, I must be able to generate HTML, ePub and PDF versions of the book. I’m using Publican for most formats, but for the PDF version Publican is moving away of fop and the replacement (webkit-based) is far from being satisfactory to generate a book ready for print. So I plan to use dblatex and get Publican to support dblatex as a backend.

I have hired Benoît Guillon, the upstream author of dblatex, to fix some annoying bugs and to improve it to suit my needs for the book (some results are already in the upstream CVS repository). I’m also working with a professional book designer to get a nice design.

I have also started to look for a Python Django developer to build the website that I will use to commercialize the book. The website will have a larger goal than just this though (“helping to fund free software developers”) but in free software it’s always good to start with your own case. 🙂

Hopefully everything will be ready in April. I’m working hard to meet that deadline (you might have noticed that my blog has been relatively quiet in the last month…).

Thanks

See you next month for a new summary of my activities.

Dpkg with multiarch support available in Debian experimental

As I announced on debian-devel, Guillem Jover uploaded a snapshot of dpkg’s multiarch branch to experimental (version 1.16.2~wipmultiarch). Beware: There will
likely be some small “interface” changes between this version and the version that will be released later in unstable (possibly in the output of dpkg --get-selections, dpkg --list, maybe other commands).

multiarch allows you to install packages from different architectures on the same machine. This can be useful if your computer can run programs from 2 architectures (eg. x86 CPU supporting i386 and amd64), or if you often need to cross-compile software and thus need the libraries of your target architecture.

Test dpkg with multiarch support

If you want to test multiarch support in dpkg, install the package from experimental (apt-get install dpkg/experimental assuming you have experimental in your sources.list).

Then you can add a supplementary architecture to your system by doing sudo dpkg --add-architecture <arch> (e.g. i386 if you are on amd64, and vice-versa). APT will automatically pick up the new architecture and start downloading the Packages file for the new architecture (it uses dpkg --print-foreign-architectures to know about them).

From there on you can install packages from the “foreign” architectures with “apt-get install foo:<arch>“. Many packages will not be installable because some of their dependencies have not yet been updated to work with in a multiarch world (libraries must be installed in a multiarch-compliant path so as to be co-installable, and then marked “Multi-Arch: same“). Other dependencies might need to be marked “Multi-Arch: foreign“. See wiki.debian.org/Multiarch/Implementation for more HOWTO-like explanations.

Now is a good time to see if you can install the foreign packages that you could need in such a setup and to help to convert the required libraries.

You can also read Cyril Brulebois’ article which quickly shows how to hunt for the problematic packages which have not been converted to multiarch (in his sample, “ucf” is not ready. Since it’s an “Architecture: all” package which can run on any architecture, it means that it’s lacking a “Multi-Arch: foreign” field).

Report bugs

If you discover any bug in dpkg’s multiarch implementation, please report it to the Bug Tracking System (against “dpkg” with the version “1.16.2~wipmultiarch”).

If you notice important libraries or packages which are not yet multiarch ready, please open wishlist bug reports requesting the conversion and point the maintainers towards the wiki page linked above. Even better, prepare patches and submit those with your bug reports.

Again, you can follow the lead of Cyril Brulebois who filed 6 bugs!

Review the multiarch implementation

If you’re a C programmer and have some good knowledge of dpkg (or are willing to learn more of it), we would certainly benefit from more eyes reviewing the multiarch branch. If you want to discuss some design issues of the multiarch implementation in dpkg (or have questions related to your review), please get in touch via debian-dpkg@lists.debian.org.

The latest version of the branch is pu/multiarch/master in Guillem’s personal repository. I have my own version of the branch (pu/multiarch/full) which is usually a snapshot of Guillem’s branch with my own submitted fixes.

$ git clone git://git.debian.org/dpkg/dpkg.git
$ cd dpkg
$ git remote add guillem git://git.hadrons.org/git/debian/dpkg/dpkg.git
$ git remote add buxy git://git.debian.org/~hertzog/dpkg.git
$ git fetch guillem && git fetch buxy

If you followed the instructions above, the relevant branches are thus guillem/pu/multiarch/master and buxy/pu/multiarch/full. Both branches are regularly rebased on top of master where Guillem merges progressively the commits from the multi-arch branch as his review progresses.

Thank you in advance for your help bringing multiarch in shape for Debian Wheezy,

My Debian Activities in January 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (213.68 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg

The “biggest change” I made is a small patch that brings to an end years and years of recurring discussions about the build-arch and build-indep targets of debian/rules (see #229357). Last year the technical committee took this issue in its hands (see #629385) but it failed to take any resolution. Fortunately thanks to this we got some concrete numbers on the colateral damages inflicted on the archive for each possible approach. In the end, Guillem and I managed to agree on the way forward.

The remaining of what I did as dpkg maintainer has not much to do with coding. I reviewed the work of Gianluca Ciccarelli on dpkg-maintscript-helper who is trying to provide helper functions to handle migration between directories and symlinks. I also reviewed a 2000-lines patch from Patrick Schoenfeld who’s trying to provide a perl API to parse dpkg log files and extract meaningful data out of them.

I updated the dpkg-architecture manual page to document the Makefile snippet /usr/share/dpkg/architecture.mk and to drop information that’s no longer releveant nowadays.

I reviewed a huge patch prepared by Russ Allbery to update the Debian policy and document the usage of symbols files for libraries. As the author of dpkg-gensymbols, I was keen to see it properly documented at the policy level.

I brought up for discussion a detail that was annoying me for quite some time: some copyright notices were embedded in translatable strings and updating them resulted in useless work for translators. In the end we decided to drop those notices and to keep them only at the source level.

I updated my multiarch branch on top of Guillem’s branch several times, all the fixes that were in my branch have been integrated (often in a modified form).

Unfortunately even if the code works quite well, Guillem doesn’t want to release anything to Debian until he has finished to review everything… and many people are annoyed by the unreasonable delay that it imposes. Cyril Brulebois tried to release a snapshot of the current multiarch branch to experimental but Guillem has been prompt to revert this upload.

I’m somewhat at a loss in this situation. I offered my help to Guillem multiple times but he keeps doing his work in private, he doesn’t share many details of his review except some comments in commit logs or when it affects the public interface. I complained once more of this sad situation.

Debian Package Maintenance Hub

That’s the codename I use for a new infrastructure that I would like to develop to replace the Package Tracking System and the DDPO and several other services. I started to draft a Debian Enhancement Proposal (DEP), see DEP-2, and requested some comments within the QA team.

For now, it looks like that nobody had major objections on the driving idea behind this project. Those who commented were rather enthusiastic. I will continue to improve this DEP within the QA team and at some point I will bring the discussion to a larger audience like debian-devel@lists.debian.org.

Package Tracking System

Even if I started to design its replacement, the PTS will still be used for quite some time so I implemented two new features that I deemed important: displaying a TODO notice when there is (at least) one open bug related to a release goal, displaying a notice when the package is involved in an ongoing or upcoming transition.

Misc packaging tasks

I created and uploaded the dh-linktree package which is a debhelper addon to create symlink trees (useful to replace embedded copies of PHP/JavaScript libraries by symlinks to packaged copies of those files).

I packaged quilt 0.50. I helped the upstream authors to merge a Debian patch that had been forwarded by Martin Quinson (a quilt’s co-maintainer). I packaged a security release of WordPress (3.3.1) and a new upstream release of feed2omb and gnome-shell-timer.

I prepared a new Debian release of python-django with a patch cherry-picked from the upstream SVN repository to fix the RC bug #655666.

Book update

We’re again making decent progress in the translation of the Debian Administrator’s Handbook, about 12 chapters are already translated.

The liberation campaign is also (slowly) going forward. We’re at 72% now (thanks to 63 new supporters!) while we were only at 67% at the start of January.

Thanks

See you next month for a new summary of my activities.

My Debian Activities in December 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (364.18 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg and Multiarch

I had some hope to have a multiarch-enabled dpkg in sid for Christmas as Guillem told me that it was realistic. Alas Guillem got sick. We’re in January and we’re still not there.

While some of Guillem’s commits in December were related to multi-arch, the size of his pu/multiarch/master branch did not really shrink. We still have 36 commits to merge… most of the work he did was refactoring some parts of the code that were already merged. And he initiated some discussion on interface changes. I participated to those discussions hoping to bring them to a quick resolution.

I’m still maintaining my own pu/multiarch/full branch, it is based on Guillem’s branch but with further fixes that I developed and that he has not yet merged and with a change reverted (Guillem’s branch allows crossgrading packages between different architectures while dpkg does not manage this correctly yet).

I can only hope that January will be the last month of this never-ending saga. It’s been one year since I started working on this project. 😐

Misc dpkg work

I reviewed (and later merged) a patch of Kees Cook to enhance dpkg-buildflags so that it can report which hardening features are enabled. This feature might then be used by tools like lintian to detect missing hardening features.

I mentored Gianluca Ciccarelli who is trying to enhance dpkg-maintscript-helper to take care of replacing a directory by a symlink and vice-versa.

I took care of #651993 so that dpkg-mergechangelogs doesn’t fail when it encounters an invalid version in the changelog, and of #652414 so that dpkg-source --commit accepts a relative filename when a patch file is explicitly given.

Guillem also merged a fix I developed for LP#369898.

Packaging work

WordPress 3.3 came out so I immediately packaged it. Despite my upstream bug report, they did not update their GPL compliance page which offers the corresponding sources for what’s bundled in the tarball. So I hunted for the required sources myself, and bundled them in the debian.tar.xz of the Debian source package. It’s a rather crude solution but this allowed me to close the release critical bug #646729 and to reintroduce the Flash files that were dropped in the past… which is great since the Flash-based file uploader is nicer than the one using the browser’s file field.

Quilt 0.50 came out after 2 years of (slow) development. The Debian package has many patches and several of them had to be updated to cope with the new upstream release. Fortunately some of them were also merged upstream. It still took an entire morning to complete this update. I also converted the packaging from CDBS to dh with a short rules file.

Zim 0.54 came out and I immediately updated the package since it fixed a bug that was annoying me.

Review of the ledgersmb packaging

As the sql-ledger maintainer (and a user of this software for my accounting), I have been hoping to get ledgersmb packaged as a possible replacement for it. I have been following the various efforts initiated over time but none of them resulted in a real package in Debian.

This is a real pity so I tried to fix this by offering to sponsor package uploads. That’s why I did a first review of the packaging. It took several hours because you have to explain everything that’s not good enough.

I also filed a wishlist bug against lintian (#652963) to suggest that lintian should detect improper usage of dpkg-statoverride (this is a mistake that was present in the package that I reviewed).

nautilus-dropbox work

I wanted to polish the package in time for the Ubuntu LTS release and since Debian Import Freeze is in January, I implemented some of the important fixes that I wanted.

The Debian package diverges from upstream in that the non-free binaries are installed in /var/lib/dropbox/ instead of $HOME.
Due to a bug, the files were not properly root-owned so I first fixed this (unpacking the tarball as root lead to reuse of the embedded user & group information, and those information changed recently on the Dropbox side apparently).

Then we recently identified other problems related to proxy handling (see #651065). I fixed this too because it’s relatively frequent that the initial download triggered during the package configuration fails… and in that case it’s the user that will re-trigger a package download after having given the appropriate credentials through PackageKit. Without my fix, usage of pkexec would imply the loss of the http_proxy environment variable and thus it would not be possible for a user to download through a proxy.

Last but not least I reorganized the Debian specific patches to better separate what can and should be merged upstream, from the changes that upstream doesn’t want. Unfortunately Dropbox insists on being able to auto-update their non-free binaries, they are, thus, against the installation under /var/lib/dropbox and the corresponding changes.

Book update

We’re making decent progress in the translation of the Debian Administrator’s Handbook, about 6 chapters are already translated (not yet reviewed though).

The liberation campaign is also (slowly) going forward. We’re at 67% now (thanks to 90 new supporters!) while we were only at 60% at the start of December.

Thanks

See you next month for a new summary of my activities.