March 2011 wrap up

Since I’m soliciting donations to support my Debian work, the least I can do is explain what I do. You can thus expect to see an article like this one every month.

Multi-Arch work

I updated the code to use another layout for the control files stored in /var/lib/dpkg/info/. Instead of using a sub-directory per architecture (arch/package.type), we decided to use package:arch.type but only for packages which are Multi-Arch: same. dpkg is taking care to rename the files the first time it is executed with write rights and then updates /var/lib/dpkg/info/format to remember that the upgrade has been done and that we can rely on the new structure.

I filed a few bugs on packages that are improperly accessing those internal files instead of using the appropriate dpkg-query interface. I sent a heads-up mail on -devel to make other people aware of those problems in the hope to discover most of them as early as possible.

After that, the work stalled because Guillem went away for 2 weeks and thus stopped his review of my work. I hope he will quickly resume the review and that we will get something final this month.

With the arrival of dpkg 1.16.0, it’s now possible to start converting libraries to multi-arch even if full multi-arch support has not yet landed in dpkg proper. See http://wiki.debian.org/Multiarch/Bootstrapping for the detailed plan.

If you’re curious about Multi-Arch, you might want to read this article of Steve Langasek as well.

Bug triage for dpkg in launchpad

At the start of the month, there was close to 500 bugs reported against the dpkg package in Launchpad. Unfortunately most of it is noise… many of the reported bugs are misfiled, they show an upgrade problem of a random package and that upgrade problem confuses update-manager which tries to configure an already configured package. This generates a second error that apport attributes to dpkg and the resulting bug report is thus filed on dpkg. There are literally hundreds of those that have to be reclassified.

Michael Vogt and Brian Murray did some triaging, and I also spend quite some hours on this task. It’s a bit frustrating as I tend to mark many reports “Incomplete” because there’s no way they can be acted upon and many of them are so old that the reporter is unlikely to be able to provide supplementary information.

But in the middle of this noise, there are some useful bug reports, like LP#739179 which enabled me to fix a regression even before it reached Debian Unstable (because Ubuntu runs a snapshot of dpkg with multiarch support).

I subscribed to the Launchpad bugs for dpkg via the Debian Package Tracking System (thanks to the derivatives-bugs keyword) and will try to keep up with the incoming reports.

Misc dpkg work

The ftpmasters came up with a request for a new field (see 619131) in source packages. After a quick discussion and a round of review on debian-policy@l.d.o, I implemented the new Package-List field. This should allow the ftpmasters to save some time in NEW processing, but we deferred the change for the next dpkg version (1.16.1) to ponder a bit more on the design of the field.

I also fixed a bunch of bugs (#619541, #605719, #598922, #616096) and merged a patch of Mark Hymers to recognize the new Built-Using field.

Developers-reference work

The review process for changes to the developers-reference is not working as it should. And I suffered from it while trying to integrate the patch I wrote for the “Developer duties” chapter (see #548867).

We purposely changed the maintainer field from debian-doc to debian-policy in the hope to have more reviews of suggested changes and to seek some sort of consensus before committing anything. But we don’t get more reviews… and deciding to commit a patch is now even harder than it was (except for trivial stuff where personal opinions can’t interfere).

In my case, I only got the feedback of Charles Plessy which was very mixed to say the least. I tried to improve my patch based on what he expressed but I also clearly disagreed with some of his assertions and was convinced that my wording was in line with the dominant point of view within Debian.

We tried to involve the release team in the discussion because most of what I documented was about helping making stable release happen, but nobody of the team answered.

Instead of letting the situation (and my patch) rot, I solicited feedback from the DPL and from another developers-reference editor to see whether my patch was an improvement or not. After some more time, I went ahead and committed it.

It was not pleasant for anyone.

I don’t know how we can improve this. Contrary to the policy, the developers-reference is a document that is not normative, I believe the result is better when we put some “soul” into it. But it’s a real challenge when you seek a consensus and that the interest in reviewing changes is so low.

DVD shop listed on debian.org

In February, I launched a DVD shop whose benefits are used to fund my Debian work. Shortly after the launch I used the official form to be added to the official listing of Debian CD vendors and offered a few suggestions to deal with vendors who are selling unofficial images (with firmware in my case).

A few weeks later, I got no answers: neither for my request nor for my suggestions, I mailed the cdvendors@debian.org team directly asking for a status update and quickly got an answer suggesting that Simon Paillard usually does the work and can’t process the backlog due to some injury. At this point no concerns had been raised about adding me to the list. To save some time and some work for the team, I added myself to the list since I had commit rights and I informed them that I did it, so that they can review it.

Shortly after I did that, Martin Zobel Helas objected to my addition. I cleared some misunderstandings but the discussion also lead to some changes to please everybody: the listing now indicates that some images are unofficial and I have prepared a special landing page for people coming from the Debian website through this listing.

Debian column on OMG! Ubuntu

I have always been a firm believer that it’s important for Debian to reach out to the widest public with its message of freedom. Thus when Benjamin Humphrey contacted the debian-publicity team to find volunteers to write a Debian column on OMG! Ubuntu, I immediately jumped in.

I wrote 4 articles over there. The tone is very different from my articles on my blog and I like that duality. Check out Debian is dying! Oh my word!, Debian or Ubuntu, which is the best place to contribute?, Are you contributing your share? and Ubuntu’s CTO reveals DEX: an effort to close the gap with Debian.

It’s a great win-win situation, OMG! Ubuntu benefits from my articles, Debian’s values are relayed further, and OMG! Ubuntu’s large audience also helps me develop my own blog.

Work on my book

I had lots of paperwork to do this month (annual accounting stuff for my company) and I did not have as much time as I hoped for my book. Still I have a updated a few more chapters of my French book and I certainly hope to complete the update during April.

This means that the work on the English translation could start in may.

Work on my blog

Just like for my book, it has been relatively difficult for me to cope with my policy of two articles every week. But I still managed to get quite some good stuff out.

I interviewed Christian Perrier (Debian’s translation coordinator) and also Bdale Garbee (chair of Debian’s technical committee).

I finished my series of “Debian Cleanup Tips” with 2 supplementary articles:

The removal of firmware is causing troubles to quite some users so I wrote an article explaining how to deal with the problem. A regular reader also asked me to write an article about Jigdo, I executed myself because it was a good idea and that he has been very nice with me: Download ISO images of Debian CD/DVD at light speed with Jigdo.

Last but not least, I shared my package maintainer pledge which inspired my developers-reference patch (see discussion above).

Thanks

Many thanks to all the people who showed their appreciation of my work. The 324.37 EUR that you gave me in February represented 2 days and a half of my time that I have spent working on the above projects.

See you next month for a new summary of my activities.

February 2011 wrap up

February has been again a busy month for me. Here’s a quick summary of what I did:

Multi-Arch work

I have spent many days implementing and refining dpkg’s Multi-Arch support with Guillem Jover (dpkg co-maintainer) and Steve Langasek (beta-tester of my code ;-)). Early testers can try what’s in my latest pu/multiarch/snapshot/* branch in my personal git repository.

A Debian DVD shop

I’m always exploring new options to fund my Debian work (besides direct donations) and this month—with the Debian Squeeze release—I saw an opportunity in selling Debian DVD. Nobody provides DVD with included firmwares and quite a few people would like to avoid the SpaceFun theme. So I built unofficial Debian DVDs that integrate firmware and that install a system with the old theme (MoreBlue Orbit). Click here to learn more about my unofficial DVDs.

On my blog

In my “People behind Debian” series, I interviewed Mike Hommey (Iceweasel maintainer) and Maximiliam Attems (member of the kernel team).

I started a “Debian Cleanup Tip” series and already published 4 installments:

For contributors, I wrote two articles: the first gives a set of (suggested) best practices for sponsoring Debian packages and adapted my article as a patch for the Developers Reference. In the second article, I shared some personal advice for people who are considering participating on Debian mailing list: 7 mistakes to avoid when participating to Debian mailing lists.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

My Debian related goals for 2011

Like last year, here’s a list of Debian related projects that I’d like to tackle this year. I might not do them all, but I like to write them down, I am more productive when I have concrete objectives.

  1. Translate my Debian book into English.
    I will run a fundraising campaign with Ulule.com and if enough people are interested, I will spend a few months with Roland Mas to translate the book. Hopefully this project can be completed until the summer.
  2. Finish multiarch support in dpkg.
    I’m working on this with Guillem Jover right now, thanks to Linaro who is sponsoring my work.
  3. Make deb files use XZ compression by default.
    It’s a simple change in dpkg-deb and it would literally save gigabytes of space on mirrors. It’s particularly important so that single CD/DVD can have a more complete set of software. #556407 (on DAK) needs to be fixed first though and a project-wide discussion is in order. Some archs might want to have a different default.
  4. Be more reactive to review/merge dpkg patches.
    While we welcome help, we have a backlog of patches sitting in the BTS and it happened several times that we failed to review/merge submitted branches in a decent time. It’s very frustrating for the contributor. I already tried to improve the situation by creating the Review/Merge Queue but nobody stepped up to help review and update the patches.
    As I am getting more familiar with the C part of dpkg, I hope to be able to jump in when Guillem doesn’t have the time or the interest.
  5. Implement the rolling distribution proposed as part of the CUT project and try to improve the release process.
    I really want the new rolling distribution but it will inevitably have an impact on the release process. It can’t be done as a standalone project. I would also like to see progress in the way we deal with transitions (see discussion here).
  6. Work more regularly on the developers-reference.
    Hopefully I will be able to combine this with my blog writing activities, i.e. write blog articles on the topics that the developers-reference shall cover and then adapt my articles with some docbook markup.

To the above list, I shall add a few supplementary goals related to funding my Debian work:

  1. Write a 10-lesson course called “Smart Package Management”.
    It will delivered by email to my newsletter subscribers.
  2. Create an information product (most likely an ebook or an online training) and sell it on my blog.
    The precise topic has not yet been defined although I have a few ideas. Is there something that you would like me to teach you? Leave your suggestions in a comment.
  3. By the end of the year, have at least 1/3 of my time funded by donations and/or earnings of my information products.
    More concretely it means 700 € each month or a 9-fold increase compared to the current income level (around 80 €/month mostly via Flattr).

That makes up lots of challenges for this year. You can help me reach those goals, join my Debian Supporters Guild and you’ll be informed every time that I start something new or when you can help in specific ways.

What I publish on Planet Debian and Planet Ubuntu

Up to now, Planet Debian and Planet Ubuntu were syndicating the main feed of my blog. I changed this recently (remember, I explained how I created my custom WordPress feed). Read on to learn why and subscribe to the main feed now if you want to get all my Debian/Ubuntu articles in the future.

Planet Debian/Ubuntu are supposed to be windows into the lives of the contributors. Whatever they are doing. Obviously you get a lot of infos about Debian and Ubuntu since most contributors spend a lot of time working on those distributions. Due to this, those planets have also become dedicated medias to exchange between contributors.

The problem I had is that my blog is more than just a place where I speak of what I do within Debian. As part of my evil plan, I set out my blog to become a great documentation resource for Debian/Ubuntu users. So when I wrote articles for users and/or beginners (like 5 reasons why a Debian package is more than a simple file archive or How to rebuild Debian packages), I got some remarks that it was “odd” to have those articles on Planet Debian because most contributors knew those things already.

The bottom line is that from now on I’ll decide on a case by case whether the article is suitable for Planet Debian&Ubuntu based on my feelings (and the feedback I get by email). But if you enjoyed my articles for users, I invite you to directly subscribe to my blog to make sure you’ll get them all in the future:

By the way I would like to thank you all. This blog has been created last July and I’m glad to have already more than 1400 regular followers (spread over RSS/email/Twitter/Identi.ca/Facebook, not counting the thousands that read me via Planet Debian/Ubuntu).

I hope you’ll enjoy 2011 with me, and continue sharing the articles you like.

Update: if you want you can also subscribe to a feed containing only the articles not published on Planet Debian/Ubuntu. It’s here: http://raphaelhertzog.com/feed/notonplanets/.

Review of my Debian related goals for 2010

Last year I shared my “Debian related goals for 2010”. I announced that I would not be able to complete them all and indeed I have not, but I have still done more than what I expected. Let’s have a look.

Translate my Debian book into English and get it published: NOT DONE

Or rather not yet done. It’s still an important project of mine and I will do it this year. When I wrote this last year, I expected to find a publisher that would take care of everything but we failed to find a suitable one so we’re going to do it ourselves.

Cleanup the dpkg perl API and create libdpkg-perl: DONE

libdpkg-perl has been introduced with dpkg 1.15.6.

Create dpkg-buildflags: DONE

dpkg-buildflags has been introduced with dpkg 1.15.7. It’s not widely used yet and it won’t really be until debhelper 7 supports it (see #544844). It would be nice to see progress on this front this year.

Ensure the new source formats continue to gain acceptance: DONE

The adoption rate has been steady, it clearly slowed down since the freeze though. I have implemented quite a few features to satisfy the needs of users, like the possibility to unapply the patches after the build, or the possibility to fail in case of unwanted upstream changes.

Design a generic vcs-buildpackage infrastructure to be integrated in dpkg-dev: NOT DONE

I believe it’s something important on the long term but it never made it to my short-term TODO list and it’s unlikely to change any time soon.

Continue fixing dpkg bugs faster than they are reported: PARTLY DONE

We have dealt with many bugs over the year, but we still have 20 more bugs than at the start of last year (370 vs 350). We’re not doing bad compared to many other Debian teams but we can still benefit from some help. Start here if you’re interested.

Enhance our infrastructure to ease interaction between contributors and to have a better view of how each package is maintained: NOT DONE

I am convinced that we need something to have a clearer idea of the commitments made by each contributor. I don’t put the same amount of care in maintaining smarty-gettext that I do on dpkg. If we had a database of the stuff that we know we don’t do well enough, it’s easier to point new contributors towards those.

Anyway, this project is still unlikely to come to the top of my priorities any time soon.

Work on the developers-reference: NOT DONE

We have switched the Maintainer field to debian-policy@lists.debian.org to have more review of the changes suggested through the bug tracking system but that has not changed much on the global situation.

I still hope to become more active on it sometimes this year. Maybe by trying to make it more fun and creating the text for some of the wishlist bugs as blog articles first.

Rewrite in C the last Perl scripts provided by the dpkg binary package: DONE

Dpkg 1.15.8 was the first version working without Perl, I announced it in July.

Integrate the 3-way merge tool for Debian changelogs in dpkg-dev: DONE

dpkg-mergechangelogs is part of dpkg-dev since 1.15.7.

I enjoy it regularly. Unfortunately it doesn’t work well for cherry-picks. Would be nice to see this fixed, anyone up to the task? :-)

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

Go2Linux interviewed me: the biggest problem of Debian

Guillermo Garron of Go2Linux enjoys a lot the “People behind Debian” interviews that I make. That’s why he interviewed me (with somewhat similar questions) and published the result on his blog.

Click here to read the full interview. I speak of my Debian projects, of the Ubuntu-Debian relationship, and more.

The question that I would like to highlight is “What’s the biggest problem of Debian?”. I answered this:

Our project identity is somewhat minimalistic. It evolves around the social contract and the Debian Free Software Guidelines. Both documents answer the question of what we’re doing but we lack a clear answer to the question of how we’re supposed to work towards our goals. It would be great if Debian could agree on some principles concerning topics like goal setting, collaboration, team work, politeness, respect. We could then advertise those and build on them while recruiting volunteers.

PS: The interview also hit LinuxToday.

Wiki page to share Debian presentations and associated templates

After having created my Debian presentation template, I wanted to share it so that future speakers can reuse my work. Surely there should be a place for this. I quickly found Ubuntu’s dedicated wiki page, and for Debian the relevant place was on the website.

However looking at that page, it’s obvious that the website is not the proper place: the page has not been updated with new talks since 2005. It’s just to much burden for any speaker. A wiki page is much more convenient in that regard. Switching to the wiki means we loose translations but that’s still better than an outdated translated page on the website.

So I went ahead, I created http://wiki.debian.org/Presentations and I filed a bug against www.debian.org to suggest to move the content to the wiki (see #601337) and to leave a link pointing to the new wiki page.

So the content of the website needs to be injected into the wiki. This is an easy task for someone that wishes to start contributing to Debian. Thus it’s your task.

Once completed send a mail to 601337@bugs.debian.org to inform the webmasters that the wiki part of the work has been completed. They just have to drop the old pages and replace them with links to the wiki.

PS: If you have given a Debian talk recently, please put your slides online and link to them on the new wiki page.

Latest features of dpkg-dev: debian packaging tools

I’m attending the mini-Debconf Paris and I just gave a talk about the latest improvement of dpkg-dev—the package providing the basic tools used to build Debian packages. Latest is a bit stretched since it embraces the last 2-3 years of development.

My talk covered the following topics:

  • Support of symbols files by dpkg-shlibdeps, dpkg-gensymbols
  • Support of new source formats by dpkg-source
  • Supplementary options for dpkg-source
  • Cross distribution collaboration with dpkg-vendor
  • Custom compilation flags with dpkg-buildflags
  • Miscellaneous improvements to other tools

The slides are relatively verbose so that you can understand them even if you did not attend the talk. Click here to get the slides.

Related links

This section points to various articles that cover more extensively some of the features mentioned in my talk.

Concerning dpkg-source:

Concerning dpkg-maintscript-helper:

Concerning dpkg-vendor:

Nice OpenOffice.org template for Debian presentations

OpenOffice.org Impress Template for Debian Presentations While preparing for the upcoming mini-debconf Paris, I noticed that we don’t have any good presentation template for OpenOffice.org Impress. I spent a few hours browsing Debian presentations put online by many speakers and was unable to find one that would suit me.

This situation was clearly unacceptable and I decided to spend a few hours to fix this. I selected a very nice wallpaper created by Alexis Younes, used The Gimp to add a translucent white box so that the text can still be read, and combined all this in a OpenOffice.org Impress template.

Click here to download the template. You can also get the Gimp XCF file for the background image.

By the way, the same wallpaper has been used by Sam Hocevar to create nice-looking Debian business cards.

PS: I contacted Alexis Younes by mail and he agreed to put the wallpapers under GPL-2+. It has been clarified on his webpage.

Follow me on Identi.ca, Twitter and Facebook.

Secret figures of a Debian/Ubuntu blogger: what you liked most on raphaelhertzog.com

Chart goes up on screenI launched raphaelhertzog.com this summer (taking over the English content of my former multi-lingual blog), when I decided that I would be more serious about blogging on Debian/Ubuntu related topics. On September, I decided to write 2 articles per week and up to now I managed to keep the schedule.

Two of my articles were published by Linux Weekly News, those are much more researched than the average blog article (they are tagged with [LWN] in the list below).

The most popular articles

Most people read my blog through the RSS feed which happens to be syndicated on Planet Debian and Planet Ubuntu. According to the feedburner’s statistics, the top-5 articles are:

  1. 5 reasons why I still contribute to Debian after 12 years (32700 views)
  2. [LWN] Understanding Membership Structures in Debian and Ubuntu (31700 views)
  3. Social Micropayment Can Foster Free Software, Discover Flattr (30100 views)
  4. Everything you need to know about conffiles: configuration files managed by dpkg (29900 views)
  5. How to make 110.28 EUR in one month with free software and Flattr (29400 views)

But I also have occasional readers visiting my blog because my articles are announced on Identi.ca, Twitter and Facebook (and they circulate on social networks, thanks to those who are sharing them!). The top-5 articles according to the statistics of my website are:

  1. 5 reasons why I still contribute to Debian after 12 years (6000 views)
  2. [LWN] Can Debian offer a Constantly Usable Testing distribution? (5000 views)
  3. Understanding Debian’s release process (1500 views)
  4. Flattr FOSS (1400 views, not an article but I regularly blog about this project)
  5. Can Debian achieve world domination without being on Facebook? (1100 views)

The most flattered

Since I am using Flattr on my blog, it can be interesting to see the articles which generated lots of flattr micro-donations. The top-3 articles are my articles about Flattr (1, 2, 3). Excluding articles related to Flattr, the top-5 is:

  1. 5 reasons why I still contribute to Debian after 12 years (12 flattr)
  2. The secret plan behind the “3.0 (quilt)” Debian source package format (10 flattr)
  3. How to use multiple upstream tarballs in Debian source packages? (5 flattr)
  4. [LWN] Understanding Membership Structures in Debian and Ubuntu (4 flattr)
  5. Do You Want a Free Debian Book? Read on. (4 flattr)

Most articles get 2 to 3 flattr clicks.

The most commented

I usually get 4-5 comments on most articles but some generate much more feedback:

  1. [LWN] Can Debian offer a Constantly Usable Testing distribution? (40 comments)
  2. 5 reasons why I still contribute to Debian after 12 years (22 comments)
  3. Can Debian achieve world domination without being on Facebook? (15 comments)
  4. How to generate different dependencies on Debian and Ubuntu with a common source package (14 comments)
  5. [LWN] Understanding Membership Structures in Debian and Ubuntu (12 comments)

Factoids

Here are my conclusions based on the above figures:

  • Writing about your Debian/Ubuntu work and your long term involvement makes for highly popular content that spreads well.
  • In-depth and well researched articles (like those written for LWN) do not generate more flattr revenues than the average article even if they take 4 to 8 times as long to write.
  • People are more likely to flattr you for your free software contribution than for the value they get out of your article.
  • People care a lot about the Debian release process, and like to discuss the topic.

If you also appreciate the above-linked articles, you should click here to subscribe to my email newsletter.