My Free Software Activities in August 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (47.50 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Package Tracking System

There are only 2-3 weeks left in the summer of code project dedicated to rewrite the package tracking system. We have come a long way during August… check it out yourself in pts.debian.net.

The rewrite doesn’t have all the features of the old PTS yet, but I opted to keep some of the easy and less interesting features for others to re-implement. Instead I asked Marko to work in the coming weeks on new features that will bring more value, like the possibility to have user accounts with the possibility to easily review and tweak all your subscriptions on the web, and like the possibility to subscribe to groups of packages (i.e. those managed by a team).

Our main problem right now is that exim has a pretty poor default behavior of forking hundreds of processes if you get hundreds of mails (in a batch) to an address that delivers via a pipe (postfix is saner, it serializes the deliveries on pipes). The new PTS is much more modular and its memory footprint is bigger (about 3 times more for the process that delivers mails, 30Mb instead of 10Mb), and in such a situation we managed to run out of memory… for now we worked around the situation with an exim setting that queues mails once the load gets too high but it’s a poor workaround IMO. We could obviously implement our own queue and a daemon but I’d like to avoid this. So who knows how to tell exim to behave? :-)

On the positive side, Marko has gotten some feedback from people who like the new PTS and are using it daily already. And several persons have expressed their interest to work on the new codebase already.

On my side, I created a package so that it’s easy to deploy for derivatives. In this process, I revamped the way we manage the Django settings (for development and for production). The package is not finished yet, but it’s mostly usable already. But I still want to do some cleanup/refactoring in the models before others start deploying it. We must also enable South to make it possible to upgrade easily afterwards.

DebConf 13 in Vaumarcus

From August 10th to 17th, I was attending DebConf 13. It matched the only week of vacation that my wife had this summer so we went there with the whole family (that is with a 3 years old son, and 6 months old one). Thus I could not immerse myself in Debconf and missed all the nice things that happen outside of the talk rooms. I picked 3-4 interesting talks per day and I spent the rest with my family.

On the positive side, I was pleased that my wife could meet (or at least see) some other Debian people. She knows quite a few (of you) by name because I have been telling her Debian stories for years now…

Debian France

Debian France sold quite some merchandise during Debconf but I didn’t take care of that. It was supervised by Sylvestre Ledru but fortunately he got the help of multiple persons, both to bring everything there, to sell it, and to bring back the rest.

The good news of the month is that the upstream author of galette published a new version with all the features that we ordered him a few months ago. We send now automatic reminders to members who must renew their subscription, we have automatic update of our accounting books (in a ledger file in a git repository) when we people donate or pay their subscription via the paypal form on our website.

I was so pleased to finally have this that I took some hours to finalize the packaging of galette, so that it could be uploaded to Debian. It’s now waiting in the NEW queue. I also spent multiple hours to write the python script that is executed by galette and that updates the accounting files.

Misc Debian stuff

Debian Packaging. I did two uploads of logidee-tools to fix bugs #718671 and #718836. I created a package for Dolibarr a PHP-based CRM and ERP software (it doesn’t do accounting however), it’s sitting in the NEW queue for almost a month already. I forwarded #719000 to the upstream Publican developers. I filed #720393 to request a new upstream version of libphp-mailer.

git-multimail. After its deployment on Alioth last month, Niels Thykier reported me a case where it lead to bounces, I filed this as a new upstream ticket and in fact I fixed it myself a few days after. I got the fixed version installed on Alioth.

dpkg. I investigated why the the automatic builds of dpkg were no longer happening and asked Michael Prokop if he could install a newer version of gettext in the build chroot. He told me that he would need a backport for that so I asked Santiago Vila if he was willing to provide it and he kindly accepted. A few days after, the package was in backports and I’m now again running the latest dpkg out of git thanks to the nice service provided by Michael.

Misc discussions. The thread about “user planets” drifted into a discussion of how to avoid “promotional posts” on such planets and in that context someone again brought up the Debian Machine Usage Policy as a way to shut down any kind of (self-)promotional content on planet if there’s money involved. This always irritates me and this time I opted to ask James Troup about the origin of that clause in the DMUP. So who is willing to work with DSA to fix the DMUP so that people stop abusing it in contexts where it doesn’t make sense?

I also participated in some discussions concerning dgit. I like the ideas behind the tool, but I’m saddened by the behavior of Ian Jackson. I helped him to fill his gap of knowledge about new sources formats but he keeps on bashing about the “3.0 (quilt)” source format both in the manual page and in the output of the program. He believes that dgit is no longer an experiment but the truth is that it’s still a poorly commented Perl script doing lots of hackish things.

Kali Linux

Between Debconf and all, I haven’t done much for Kali except a couple of fixes. There’s a nice story of how I tracked a bug in live-installer on the Kali blog. That fix has been committed to Debian. I also improved live-build to include xfsprogs/jfsutils on the ISO image when you include the debian-installer (so that you don’t end up in problems when you pick JFS or XFS as file systems for your installation).

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in June 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (106.39 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Debian France

I handled some subscriptions that we collected during Solutions Linux at the end of last month and I also worked on updating the accounts in preparation of our annual general assembly. As the current president, I wrote a “moral assessment” of the organization and I helped Sylvestre Ledru (the treasurer) to prepare an overview of our financial situation (which is rather good).

The plan for this general assembly included the renewal of 6 of our 9 board members so I tried to motivate some persons to candidate and I’m pretty happy to see that we managed to have 9 candidates for 6 seats, a real election for once. :-)

And as usual, I updated Galette to version 0.7.4.2 and I filed a couple of tickets (#621, #622).

Debian Package Tracking System

We’re getting close to one month of work on the Google Summer of Code Project that rewrites the Package Tracking System. On average I spent a good 1h30 per day on reviewing the code, giving my feedback, and preparing the user stories for the next iteration, plus a 2 hour meeting each Wednesday afternoon. This is more than I expected but at I’m rather happy with the result so far.

We have a live test instance updated each week (after the end of the iteration) at pts.debian.net. The web interface is empty because the bulk of the work has been spent on the email interface for now, but it’s now feature complete and already has seen some improvements compared to the current implementation (fixing #340863 for example). You can interact with it by sending mails to control@pts.debian.net.

The Debian Administrator’s Handbook

The fundraising for the liberation of the original book in French met its initial target very quickly and we added new targets with supplementary rewards (more ebook from Eyrolles, donation to Debian). All the supplementary targets were met, although the last one took until the last day!

Thanks to all the supporters, we now have a French translation of the Debian Administrator’s Handbook which is free and we have collected about 2700 EUR for Debian.

Debian Packaging

I got a couple of release critical bug reports that I had to fix:

  • #713636: cpputest FTBFS due to -Werror. Dropped -Werror in Debian and submitted the patch upstream.
  • #713527: logidee-tools FTBFS due to reorganizations in the TeXLive meta-packages. Fixed the (build-)dependencies.
  • #713947: multiple security issues in WordPress. Packaged new upstream version 3.5.2 into unstable and wheezy-security (and Yves-Alexis Perez took care of squeeze-security).

And trying to anticipate future issues, I filed an upstream ticket on feed2omb (#33) to see whether upstream had plans to support the new pump.io API that identi.ca is supposed to use soonish.

Kali Linux

I filed a couple of Debian bugs: #711866 to request a new upstream version of w3af in Debian, #711044 to report a missing dependency in LSB meta-informations of the vsftpd init script.

I worked on packaging passing-the-hash. Basically this is a fork of several software just to override some NTLM authentication functions. In some cases, I managed to avoid the fork by transforming the patches in LD_PRELOAD libraries that override the specific symbol of the patched functions.

Among the forked software, there was winexe which wasn’t packaged yet. So I packaged it but we quickly ran into some segfaults (which was already reported to upstream since a few months). I investigated the problem and prepared a patch. It has been submitted to upstream and merged.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in April 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (102.70 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Debian France

Work on Galette. I spent quite some time on Debian France’s galette installation (the web application handling its member database), first converting its Postgres database to UTF-8, then upgrading to 0.7.4 while working-around many known problems.

I also created Debian packages of three Galette plugins that we have been using (galette-plugin-paypal, galette-plugin-admintools, galette-plugin-fullcard).

But every time I use galette, I tend to find something to report. This month I filed 5 tickets:

  • #588: galette should offer a way to send a test mail while setting up the mail notifications
  • #589: CSV export page contains an invalid download link
  • #590: confirmation page of a successful PayPal payment contains empty fields
  • #591: problem with the selection of recipients of a mailing
  • #595: galette should put a proper recipient in the “To:” field of automatically generated mails.

I tested quite some fixes prepared by the upstream author (3 of the above bugs are already fixed) and this lead to the 0.7.4.1 bugfix release.

Preliminary work on new bylaws. I have setup a git repository to make it easier to collaborate on new versions of our bylaws and internal rules. The goal is to make Debian France a trusted organization of Debian and to update everything to be compliant with the “association 1901” law (we currently have a special statute reserved to associations from Alsace/Moselle).

Kali Linux

Improve accessibility support in Debian Wheezy. Offensive Security wanted Kali Linux to be fully accessible to disabled people. Since Wheezy was suffering from some serious regressions in that area, we hired Emilio Pozuelo Monfort to fix #680636 and #689559 in gdm3. On my side, I updated debian-installer’s finish-install to correctly pre-configure the system when you make an install with speech synthesis (patch submitted in #705599).
Thanks to accommodating release managers, this work has already been integrated in Wheezy and won’t have to wait the first point release.

Fix bugs in Debian’s live desktop installer. We also wanted to enable the desktop installer in the Kali live DVD. While our first tries a few months ago failed, this time it worked almost out of the box (thanks to Ben Armstrong who fixed it). I still identified a few issues that I fixed in debian-installer-launcher’s git repository.

Packaging and misc Debian work

  • I reviewed the work of Charles Plessy who drafted an important update of the Debian Policy to document dpkg triggers (see #582109)
  • I reviewed the libwebsockets package prepared by Peter Pentchev (ITP 697671)
  • I discovered Tanglu and joined their mailing list because I want to watch its evolution (and maybe use it as a test-bed for some future infrastructure developments).
  • I reviewed and committed a patch of Robert Spencer on debian-cd (see #703431).
  • I packaged version 3.3 of cpputest (in experimental). I tested a new upstream snapshot converted to autotools.

I also spent a significant number of hours to answer questions of students who want to participate in Google’s summer of code and who are interested by the rewrite of the Package Tracking System with Python and Django. Some of the discussions happened on debian-qa@lists.debian.org.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in March 2013

This is my monthly summary of my free software related activities. If you’re among the people who made a donation to support my work (114.19 €, thanks everybody!), then you can learn how I spent your money. Otherwise it’s just an interesting status update on my various projects.

Simple-CDD and debian-cd

I tried to use wheezy’s version of debian-cd and simple-cdd to generate an automatic installer. In this process, I filed a couple of bugs on simple-cdd (#701963: type-handling package is gone and should not be listed in default.downloads, and #701998: the --keyboard parameter is not working with wheezy’s debian-installer) and I commited fixes for a few issues in debian-cd:

  • r2518: adjust Makefile for new xorriso requirement
  • r2520: add missing depends on dosfstools
  • r2521: use --no-check-gpg when querying debootstrap
  • r2522: make debian-cd work with a mirror without sources)

Debian France

I completed the new website for Debian France and I put it online. Later I merged some supplementary enhancements prepared by Tanguy Ortolo (and I gave him commits rights at the same time).

I tried to update our Galette installation to the latest upstream version but I reverted to the former version after having encountered two problems (filed here and here). In the process, I created a Debian package for galette (you can grab it on git.debian.org).

I also suggested an idea of improvement for Galette’s paypal plugin and it has been quickly implemented. Thus I updated the plugin installed on france.debian.net.

Kali related work

It’s been a few months that I have been helping the Kali team to prepare this new Debian derivative. Now that the derivative has gone public, I can attribute some of my Debian work to my collaboration with the Kali team.

This month I contributed a few features and fixes to debian-installer and live-build:

After the launch, we registered Kali in the derivative census. Paul Wise quickly reported some misfiled bugs from early Kali users and I discovered that reportbug was not behaving properly even though we correctly updated base-files (see #703678 on reportbug and #703677 on lsb-release).

Misc packaging work

  • I sponsored a new upstream version of dnsjava because it’s required by Jitsi.
  • I prepared rebuild 0.4.1.1 and uploaded it to testing-proposed-updates for a RC bug fix.
  • I uploaded Publican 3.1.5 to experimental and filed #703514 to request a new upstream version of docbook-xsl that is needed by Publican.
  • I filed #703995 to fix apt-setup’s handling of the apt-setup/multiarch preseed option.

DPL election

I also spent quite some time to read and participate to the discussions on debian-vote since it was campaigning time for the DPL candidates

Thanks

This was a rather active month if you take into account the fact that I got a second son — Lucas — on March 6th.

See you next month for a new summary of my activities.

Kali Linux 1.0, a new Debian derivative

Today, during Blackhat Europe, Offensive Security announced the availability of Kali Linux 1.0, which aims to be the most advanced, robust, and stable penetration testing distribution to date. It is the successor of Backtrack Linux.

kali

Kali’s choice of Debian

Kali’s release is a significant event in the security auditing and penetration testing field, and I’m proud to see that Debian was retained as the best distribution to create this new product. Here’s what Mati Aharoni of Offensive Security told me:

Debian provides a reliable base to build a new distribution and yet can easily be customized to add bleeding edge features, thanks to the unstable and experimental distributions.

Kali’s development policies

Even though Kali was prepared in secret, from now on Kali’s development happens in the open in public git repositories. There are repositories for all the packages that have been created (or forked) as well as for the ISO images creation script.

Debian packages are maintained with git-buildpackage, pristine-tar and the associated helper tools, making it easy to integrate the latest changes of Debian.

Kali packaged several hundreds tools that relate to their field and they intend to contribute those which are DFSG-free back to Debian.

Kali’s technical infrastructure

In the last year, I have been working within the Kali team to setup large parts of their infrastructure as a proper Debian derivative.

Kali’s main ISO images are built with live-build. All the bugfixes that I contributed to Debian Live were the direct result of my work for Kali.

The git repositories are managed with gitolite. The package repositories are built with reprepro. The build daemons use rebuildd and sbuild.

The (push) mirrors are synchronized with the same tools than Debian (based on rsync), but there’s also a central server which redirects to a mirror close to you (and which is used by default everywhere). This one runs mirrorbrain (and not Raphaël Geissert’s redirector).

The ARM build daemons (armel/armhf) run on machines powered by Calxeda’s Highbank (4 cores, 4 GB RAM) that work pretty well. Even better, Offensive Security is willing to dedicate one node of this “cluster” for Debian’s own usage.

The future

This first release is not an end. It’s only the start of a journey. Not all applications have been packaged yet and there’s lot of work left to integrate everything in Debian.

I’m really looking forward to continue my collaboration with the Kali team as this has been one of the most interesting project I ever had as a Debian consultant. And also one of the few where I could really contribute something back to Debian.