My Free Software Activities in September 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (86.18 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Package Tracking System^U Distro Tracker

Marko Lalic implemented quite a few interesting features in the last weeks of the Google Summer of Code (support of teams most notably). Unfortunately he didn’t deploy (yet) the latest changes on pts.debian.net.

Given the good work he made over the summer, I marked him as successful in his GSOC. Hopefully he will stick around and continue to contribute, he promised to try to handle some mass renaming that we agreed upon. Effectively, after much bike-shedding, I decided that the software would be called “Distro Tracker”.

Once those last-minute cleanups are done, I plan to request “tracker.debian.org” to Debian System Administrators. This means that it will be deployed in parallel to the current PTS at least until we’re at feature parity in the new codebase.

The new codebase should be much more easy to get started with, so I should do some promotion and invite people to contribute to it… possibly by writing some short “how to get started” documentation.

I started by creating a dedicated wiki page: http://wiki.debian.org/qa.debian.org/distro-tracker

Misc packaging

I got two REJECTs from ftpmasters this month (one for galette, one for dolibarr). I took care of fixing the various issues in galette and the package has been promptly accepted afterwards. For dolibarr, I mentored the upstream maintainer about the various problems and got him to fix it. It took a bit more time and the package is thus still in NEW.

I packaged wordpress 3.6, and then wordpress 3.6.1 (security update). python-django also had multiple security updates this month, I took care of one or two uploads but Luke Faraone dealt with most of them (including backports to Squeeze!).

I packaged Publican 3.6.1 and uploaded dh-linktree 0.4 to fix a FTBFS issue introduced with Perl 5.18.

Of anecdotal importance, but I also filed bug #721849 after seeing how much energy was spent to ensure debian/rules didn’t contain an improper copyright statement.

Thanks

See you next month for a new summary of my activities.

Finding a new name for the Package Tracking System

The Google Summer of Code rewriting the Package Tracking System is approaching its end and I’m starting to think about deploying it on debian.org. Its scope has expanded over the years and the rewritten PTS will continue this trend by bringing some new features for teams (like the possibility to subscribe to all packages of a team).

I believe that its current hostname (and name) doesn’t reflect properly the role of the PTS. Add to this the fact that there’s still some work left to be done to reach feature-parity with the current PTS, I’m considering deploying it in parallel to the current PTS under a new name.

“Package Tracking System” is also a bit too long for a name, and sounds more like a description than a name…

But if I get rid of “packages.qa.debian.org” and “Package Tracking System”, how should we call the new PTS? :-)

The PTS is a sort of central place that brings together information from many parts of Debian. It’s currently mainly a consumer/dispatcher of information but I expect to integrate some of the external services that are useful for all Debian derivatives, and it will thus become more and more a producer of first-hand information as well.

To replace packages.qa.debian.org, Stefano Zacchiroli suggested me hub.debian.org and I must say I like it, it’s short and relatively close to what the PTS actually is (and reminds me of DEP-2 — the new PTS will be an asset to make it a reality). My other ideas were devel.debian.org, inside.debian.org, watch.debian.org, track.debian.org, … do you have better suggestions? what’s your preference?

Finding a better name is harder, but there’s room to build on the hub concept and similar images. I would like a full name that’s not too long and an associated abbreviation/short name for the top-level Python package (currently we use “pts” for that Python package). Can you come up with something original and satisfactory?

My latest thoughts end up with “DistroHub” as full name and “dhub” as Python package name. Still boring…

So, dear lazy web, I heard that we’re good at bikeshedding in Debian, so can you come up with something better? Share your suggestions in the comments!

My Free Software Activities in August 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (47.50 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Package Tracking System

There are only 2-3 weeks left in the summer of code project dedicated to rewrite the package tracking system. We have come a long way during August… check it out yourself in pts.debian.net.

The rewrite doesn’t have all the features of the old PTS yet, but I opted to keep some of the easy and less interesting features for others to re-implement. Instead I asked Marko to work in the coming weeks on new features that will bring more value, like the possibility to have user accounts with the possibility to easily review and tweak all your subscriptions on the web, and like the possibility to subscribe to groups of packages (i.e. those managed by a team).

Our main problem right now is that exim has a pretty poor default behavior of forking hundreds of processes if you get hundreds of mails (in a batch) to an address that delivers via a pipe (postfix is saner, it serializes the deliveries on pipes). The new PTS is much more modular and its memory footprint is bigger (about 3 times more for the process that delivers mails, 30Mb instead of 10Mb), and in such a situation we managed to run out of memory… for now we worked around the situation with an exim setting that queues mails once the load gets too high but it’s a poor workaround IMO. We could obviously implement our own queue and a daemon but I’d like to avoid this. So who knows how to tell exim to behave? :-)

On the positive side, Marko has gotten some feedback from people who like the new PTS and are using it daily already. And several persons have expressed their interest to work on the new codebase already.

On my side, I created a package so that it’s easy to deploy for derivatives. In this process, I revamped the way we manage the Django settings (for development and for production). The package is not finished yet, but it’s mostly usable already. But I still want to do some cleanup/refactoring in the models before others start deploying it. We must also enable South to make it possible to upgrade easily afterwards.

DebConf 13 in Vaumarcus

From August 10th to 17th, I was attending DebConf 13. It matched the only week of vacation that my wife had this summer so we went there with the whole family (that is with a 3 years old son, and 6 months old one). Thus I could not immerse myself in Debconf and missed all the nice things that happen outside of the talk rooms. I picked 3-4 interesting talks per day and I spent the rest with my family.

On the positive side, I was pleased that my wife could meet (or at least see) some other Debian people. She knows quite a few (of you) by name because I have been telling her Debian stories for years now…

Debian France

Debian France sold quite some merchandise during Debconf but I didn’t take care of that. It was supervised by Sylvestre Ledru but fortunately he got the help of multiple persons, both to bring everything there, to sell it, and to bring back the rest.

The good news of the month is that the upstream author of galette published a new version with all the features that we ordered him a few months ago. We send now automatic reminders to members who must renew their subscription, we have automatic update of our accounting books (in a ledger file in a git repository) when we people donate or pay their subscription via the paypal form on our website.

I was so pleased to finally have this that I took some hours to finalize the packaging of galette, so that it could be uploaded to Debian. It’s now waiting in the NEW queue. I also spent multiple hours to write the python script that is executed by galette and that updates the accounting files.

Misc Debian stuff

Debian Packaging. I did two uploads of logidee-tools to fix bugs #718671 and #718836. I created a package for Dolibarr a PHP-based CRM and ERP software (it doesn’t do accounting however), it’s sitting in the NEW queue for almost a month already. I forwarded #719000 to the upstream Publican developers. I filed #720393 to request a new upstream version of libphp-mailer.

git-multimail. After its deployment on Alioth last month, Niels Thykier reported me a case where it lead to bounces, I filed this as a new upstream ticket and in fact I fixed it myself a few days after. I got the fixed version installed on Alioth.

dpkg. I investigated why the the automatic builds of dpkg were no longer happening and asked Michael Prokop if he could install a newer version of gettext in the build chroot. He told me that he would need a backport for that so I asked Santiago Vila if he was willing to provide it and he kindly accepted. A few days after, the package was in backports and I’m now again running the latest dpkg out of git thanks to the nice service provided by Michael.

Misc discussions. The thread about “user planets” drifted into a discussion of how to avoid “promotional posts” on such planets and in that context someone again brought up the Debian Machine Usage Policy as a way to shut down any kind of (self-)promotional content on planet if there’s money involved. This always irritates me and this time I opted to ask James Troup about the origin of that clause in the DMUP. So who is willing to work with DSA to fix the DMUP so that people stop abusing it in contexts where it doesn’t make sense?

I also participated in some discussions concerning dgit. I like the ideas behind the tool, but I’m saddened by the behavior of Ian Jackson. I helped him to fill his gap of knowledge about new sources formats but he keeps on bashing about the “3.0 (quilt)” source format both in the manual page and in the output of the program. He believes that dgit is no longer an experiment but the truth is that it’s still a poorly commented Perl script doing lots of hackish things.

Kali Linux

Between Debconf and all, I haven’t done much for Kali except a couple of fixes. There’s a nice story of how I tracked a bug in live-installer on the Kali blog. That fix has been committed to Debian. I also improved live-build to include xfsprogs/jfsutils on the ISO image when you include the debian-installer (so that you don’t end up in problems when you pick JFS or XFS as file systems for your installation).

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in July 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (167.67 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

The Debian Administrator’s Handbook

After the successful crowdfunding campaign, I had a bunch of rewards to ship: I subcontracted most of the job but I had to take care of the books with dedication. I also dealt regularly with books/stickers coming back to the sender (due to invalid address or people not picking up their parcels in the post-office).

After the rewards, we had to take care to actually finalize the liberation of the French translation. I merged the translations we had in Git and Roland updated/translated a few strings that weren’t in the original book in French. Then I have put the book online.

Last but not least, I started to work on updating the English book for Debian 7 (Roland started way before me) and we have put some updated chapters up for review.

Debian France

Elections. After Debian France’s general assembly, the new board of administrators voted the officers: I have been re-elected as President, Sylvestre continues as Treasurer but we have a new Secretary in the person of Alexandre Delanoë. Welcome Alexandre!

I did the administrative work to register the new board/officers in the « Tribunal d’instance » and to give access to the internal git repositories to the new members.

Galette. I also did a bunch of tests on Galette’s new features that Debian France ordered to the upstream author. They should all land in the next upstream release due in the next weeks. \o/

Accounting. I worked on the accounting to bring it up-to-date so that Sylvestre can pick up the work from now on. We’re learning how to best use ledger for our needs.

PTS rewrite

I continued to spend about 12 hours a week to mentor Marko Lalic who is rewriting the Package Tracking System. I’m pretty happy with the results so far so I marked him as “pass” for the mid-term evaluation required by Google. You can have a look at the documentation and the web interface is starting to show some content.

The email interface is fully working and I have configured the real PTS to forward all mails to our test instance (pts.debian.net) so that you can use the rewritten PTS for real-life work. Mail your subscription commands to control@pts.debian.net and start using it!

Thanks to the test driven development methodology we’re using, we’re pretty confident that it works reasonably well! :-)

I also packaged python-django-jsonfield (still in NEW) since Marko has been using this python module in his code, and filed bug #717900 on sqlite3 to raise a limit that we have hit with queries made by the PTS.

Kali Linux

I used the Calxeda Highbank node donated to Debian by Offensive Security to test the new -armmp kernel flavor on it. It seemed to work except for a missing network driver (filed in #717269).

Misc Debian work

Issues with social networks. With the move of identi.ca to pump.io, we don’t have any possibility to auto-post status updates based on RSS feeds. Identi.ca’s @debian account was also configured to push updates to the @debian account on twitter.com (and from there it was grabbed in the Debian page on Facebook). This is also gone… so to limit the damage, I setup twitterfeed.com so that the twitter/facebook accounts continue to have updates). If you’re looking for a development project, here’s an area that is not well covered by free software! We need code to do what twitterfeed does… and we need that code to also support pump.io.

Dpkg work. It’s been a long time since I last pushed some code to dpkg’s git repository. I took care of reworking and merging a patch submitted by Steve Langasek to fix #716948 (an issue with dpkg-maintscript-helper rm_conffile messing with conffiles that the package no longer owns).

Git mail notification. When I was still administrator of Alioth, I wrote git-commit-notice (a fork of Git’s post-receive-email) and many packaging projects are using this hook script to send commit notices to mailing lists. This script has not been updated for multiple years and it started spewing warnings recently due to deprecated features in Wheezy’s git. So I looked at updating it and while doing so I discovered a much better replacement with git-multimail. Thus I adapted git-commit-notice to work on top of this new script. The result has now been installed on git.debian.org (this is to be properly announced in the next DeveloperNews).

Misc work. I packaged sql-ledger 3.0.5-1, forwarded #714739 on publican, and I participated in discussions to move the French Debian planets to planet.debian.org.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in June 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (106.39 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Debian France

I handled some subscriptions that we collected during Solutions Linux at the end of last month and I also worked on updating the accounts in preparation of our annual general assembly. As the current president, I wrote a “moral assessment” of the organization and I helped Sylvestre Ledru (the treasurer) to prepare an overview of our financial situation (which is rather good).

The plan for this general assembly included the renewal of 6 of our 9 board members so I tried to motivate some persons to candidate and I’m pretty happy to see that we managed to have 9 candidates for 6 seats, a real election for once. :-)

And as usual, I updated Galette to version 0.7.4.2 and I filed a couple of tickets (#621, #622).

Debian Package Tracking System

We’re getting close to one month of work on the Google Summer of Code Project that rewrites the Package Tracking System. On average I spent a good 1h30 per day on reviewing the code, giving my feedback, and preparing the user stories for the next iteration, plus a 2 hour meeting each Wednesday afternoon. This is more than I expected but at I’m rather happy with the result so far.

We have a live test instance updated each week (after the end of the iteration) at pts.debian.net. The web interface is empty because the bulk of the work has been spent on the email interface for now, but it’s now feature complete and already has seen some improvements compared to the current implementation (fixing #340863 for example). You can interact with it by sending mails to control@pts.debian.net.

The Debian Administrator’s Handbook

The fundraising for the liberation of the original book in French met its initial target very quickly and we added new targets with supplementary rewards (more ebook from Eyrolles, donation to Debian). All the supplementary targets were met, although the last one took until the last day!

Thanks to all the supporters, we now have a French translation of the Debian Administrator’s Handbook which is free and we have collected about 2700 EUR for Debian.

Debian Packaging

I got a couple of release critical bug reports that I had to fix:

  • #713636: cpputest FTBFS due to -Werror. Dropped -Werror in Debian and submitted the patch upstream.
  • #713527: logidee-tools FTBFS due to reorganizations in the TeXLive meta-packages. Fixed the (build-)dependencies.
  • #713947: multiple security issues in WordPress. Packaged new upstream version 3.5.2 into unstable and wheezy-security (and Yves-Alexis Perez took care of squeeze-security).

And trying to anticipate future issues, I filed an upstream ticket on feed2omb (#33) to see whether upstream had plans to support the new pump.io API that identi.ca is supposed to use soonish.

Kali Linux

I filed a couple of Debian bugs: #711866 to request a new upstream version of w3af in Debian, #711044 to report a missing dependency in LSB meta-informations of the vsftpd init script.

I worked on packaging passing-the-hash. Basically this is a fork of several software just to override some NTLM authentication functions. In some cases, I managed to avoid the fork by transforming the patches in LD_PRELOAD libraries that override the specific symbol of the patched functions.

Among the forked software, there was winexe which wasn’t packaged yet. So I packaged it but we quickly ran into some segfaults (which was already reported to upstream since a few months). I investigated the problem and prepared a patch. It has been submitted to upstream and merged.

Thanks

See you next month for a new summary of my activities.

My Debian Activities in July 2012

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (72.65 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

This month has been a short one since I have been away for 2 weeks of vacation.

Dpkg

My dpkg work encompasses a bunch of small tasks:

  • I uploaded dpkg 1.16.7 with an important bugfix for a regression.
  • I initiated a new round of discussions about how we were going to solve the problem that source packages with “Multi-Arch: same” binary packages can’t be individually bin-nmued.
  • Following that discussion, I opened a bunch of bugs to plan/discuss the transition of changelog/copyright files within the package metadata (#681289 on debian-policy, #681293 on apt-listchanges, #681295 on www.debian.org for packages.debian.org).
  • I also filed #681292 on sbuild to get it to use dpkg’s new syntax for bin-nmus. It will allow us to do binary-only rebuilds with arbitrary versions (instead of only “+b1″ suffixes). Ubuntu could use this for their +rebuild1, we could use this to build backports which do not require source changes (and share the common source package instead of duplicating it). It can also be useful if we ever get to the situation where transitions are prepared in external repositories and where we want bin-nmus in those repositories to have unique versions (even though the same package might be bin-nmued in multiple repositories in case of concurrent transitions).
  • I filed an unblock request for dpkg once it was almost 10 days old.
  • I did reconsider the bug #316521 where dpkg looses track of some shared directories with manually created files and proposed an updated patch. No words from Guillem on the patch yet. Fixing this would help to fix a bunch of piuparts issues.
  • And just before my vacation, I filed many bugs against dpkg itself, effectively moving some of the items that accumulated in my TODO in a public place where others can help (I’ll be happy to mentor anyone who wants to tackle one of these):
    • #681443: dpkg-source –commit should be able to merge changes in an existing patch
    • #681470: dpkg-shlibdeps: should also scan Build-Depends-Arch for minimal versions
    • #681474: Dpkg::Vendor: should support /etc/os-release and /etc/os-release.d/*
    • #681477: dpkg-vendor: implement –select-closest command
    • #681480: base-files: Provide HOME_URL, SUPPORT_URL and BUG_REPORT_URL in /etc/os-release
    • #681489: base-files: Add /etc/os-release.d/debian and make it easy to provide supplementary /etc/os-release.d/* files
  • In #595112 we discussed the specifics of a new dpkg-mainstscript-helper feature to move a conffile from one package to another.

Packaging

I updated nautilus-dropbox to version 1.4.0 and python-django-registration to version 0.8. Both have been uploaded to unstable and I initially wanted to request an unblock for the latter, but it turns out it has gained reverse dependencies and version 0.8 introduces API changes so it’s not an option at this point of the freeze.

QA work

I investigated and fixed #678356 where it had been reported that the PTS static news were no longer working as expected.

At the start of the month, I also unblocked the mostly-unknown but important “mole” service… it was out of date of several weeks and several people were annoyed that the information about new upstream versions was no longer up-to-date.

Vacation

Almost no Debian work during my vacation but the lack of Wifi nearby made me look for solutions to connect my computer through my Nokia N900 3G/GPRS connection. I discovered the “Mobile Hotspot” application (homepage) and it worked like a charm (although it required Maemo’s non-default devel repository to be able to install the alternative kernel “for power users”).

The Debian Handbook

Michal Čihař proposed us to host a Weblate instance to help translate the book with a web interface. He kindly agreed to implement some improvements to better suit my requirements. Those have been completed and the weblate instance is now live at debian.weblate.org.

There’s no requirement to use Weblate for translations teams but for those that do, it sure makes it easier to recruit volunteers who have no prior knowledge of Git and PO files. If you want to help, please checkout this page first though, you should not start using Weblate without getting in touch with the respective translations teams.

Apart from translations, I also had the pleasure to merge some patches from Philipp Kern who improved the section covering IPv6 and a few other parts. We can make the book even better if more people share their expertise in the part of the book where they know better than me and Roland. :-)

Thanks

See you next month for a new summary of my activities.

My Debian Activities in November 2011

This is my monthly summary of my Debian related activities. If you’re among the people who made a donation to support my work (310.73 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Dpkg: Multi-Arch Saga

I know lots of people are waiting the landing of multiarch in Debian unstable, and so am I. Things are progressing, though not as quickly as I hoped. Guillem merged about half of the branch between the 24th October and the 6th of November. After that most of the work happened on his personal repository in his pu/multiarch/master branch.

I verify this repository from time to time because Guillem does not inform me when he has made progress. I noticed changes on his repository on the 10th, 19th, 23th, 28th of November and on the 1th of December.

He announced a long time ago that he had some “interface changes” and up to now only wrote about the switch from the command-line option --foreign-architecture (to put in /etc/dpkg/dpkg.cfg) to the explicit command dpkg --add-architecture that only needs to be called once (see mail here). As of today (December 2th), the promised email for the other interface changes is still not here.

On November 23th, I reviewed Guillem’s work and tried to run the code in his branch. I spent the whole day chasing up regressions and submitted lots of fixes to Guillem. Thanks to the extensive test-suite I wrote when I developed my branch, it has been fairly easy to track them all down.

All the issues I reported have been fixed in the latest version of Guillem’s branch although the fixes are often slightly different from those that I submitted.

Dpkg: Squeeze Backport

At the start of the month, I uploaded what I expected to be a fairly uncontroversial backport of dpkg 1.16.1.1. It turns out I was wrong.

After some discussion, I think we came to an agreement that it was acceptable to backport dpkg-dev and libdpkg-perl only. My goal was not to bring the latest dpkg to users but to make it easier for package maintainers to backport packages using new features provided by dpkg-dev >= 1.16 (such as hardening build flags, the makefile snippets provided in /usr/share/dpkg/, or the improved dpkg-buildflags interface).

Thus I modified the source package uploaded to squeeze-backports to build only dpkg-dev and libdpkg-perl. It has been uploaded on November 23th and it’s waiting in the NEW queue for a backports admin to process it.

Misc Dpkg Work

I merged a patch of Colin Watson to be able to verify build-dependencies for a foreign architecture (taking into account the Multi-Arch status of each package listed).

I released dpkg 1.16.1.2 with two minor fixes that were sitting in the sid branch. I wanted to get rid of this so that the path is clear for a 1.16.2 upload with multiarch. The package just migrated to testing so we’re fine.

I spent another day doing dpkg bug triaging on Launchpad, we’re now down to 77 bugs with many of them tagged as incomplete and likely to expire in 2 months.

The Debian Administrator’s Handbook

eBookWe released a sample chapter so that it’s easier to have an idea of the quality of the book. The chapter covers the APT tools quite extensively. I bet that even you could learn something about apt-get/aptitude…

The crowfunding campaign on Ulule ended on November 28th.
With 673 supporters, we raised 24345 EUR. Of those, 14935 EUR have been put in the liberation fund and the rest corresponds to the various pre-orders and rewards offered.

This means that the translation will happen (we just started) but that the book is currently not going to be released under a free license. Don’t despair… As planned, the liberation campaign is carried on until the 25 K€ target is reached!

Instead of being hosted on Ulule, this permanent campaign is on the project website at debian-handbook.info/liberation/. Note that any contribution of 10 EUR or more means that you get a copy of the ebook as soon as it’s available (even if the liberation target is not reached).

Package Tracking System

At the start of the month, I filed two ideas of improvements for the PTS in the bug tracking system: #647258 is about showing outstanding bugs that relate to a release goal and #647901 is about warning maintainers that the package is affected by a current transition. If you’re a coder and want to start contributing to Debian and its QA team, those bugs could be interesting targets for a start. :-) In both cases, I have been in contact with members of the release team because those ideas require some structured data from the release team as input. Thanks to Meddi Dohguy and Niels Thykier for their help.

Later in the month, the topic of relocating the PTS once again came up. For historical reasons, the PTS was hosted on master.debian.org together with the BTS. Nowadays the BTS has its own host and it made no sense anymore to have the PTS separate from the rest of the QA services hosted on qa.debian.org (currently quantz.debian.org). So together with Martin Zobel Helas we took care to plan the migration and on November 19th we executed the plan. It worked like a charm and almost nobody noticed (only one undocumented dependency was missed, which broke the SOAP interface).

Misc Packaging Work

WordPress was broken in Ubuntu and it was also not properly synchronized with Debian due to an almost useless change on their side. Thus I requested a sync so that the working version from Debian gets imported in Ubuntu.

I sponsored the docbook-xsl 1.76.1 upload that I needed for Publican. Then I updated Publican just to discover that the test-suite triggers a new bug in fop (filed as #649476). I disabled the test-suite temporarily and uploaded Publican 2.8 to unstable. BTW, I also filed 2 upstream bugs with patches for issues I discovered while trying to generate the sample chapter of my book (see here and here).

I uploaded a version 0.7.1 of nautilus-dropbox and fixed #648215 at the same time. I made an NMU of bison to fix a long-standing release critical bug that hit me once more during an upgrade (see #645038).

I uploaded to experimental a new version of gnome-shell-timer compatible with GNOME 3.2. I took the opportunity to install from experimental the few GNOME 3.2 packages which are not yet in unstable…

Thanks

See you next month for a new summary of my activities.