My Free Software Activities in June 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

Debian packaging

Django and Python. I uploaded Django 1.9.7 and filed an upstream ticket (#26755) for a failure seen in its DEP-8 tests.

I packaged/sponsored python-django-modeltranslation and python-paypal. I opened a pull request on model-translation to fix failing tests in the Debian package build.

I packaged a new python-django-jsonfield (1.0.0), filed a bug and discovered some regression in its PostgreSQL support. I helped on the upstream ticket and I have been granted commit rights. I used this opportunity to do some bug triage and push a few fixes. I also discussed the future of the module and ended up starting a discussion on Django’s developer list about the possibility to add a JSONField to the core.

CppUTest. I uploaded a new upstream version (3.8) with more than a year of work. I found out that make install does not install a required header so I opened a ticket with a patch. The package ended up not compiling on quite a few architectures so I opened a ticket and prepared a fix for some of those failures with the help of the upstream developers. I also added a DEP-8 tests after having uploaded a broken (untested) package…

systemd support in net-snmp and postfix. I worked on adding native systemd service units to net-snmp (#782243) and postfix (#715188). In both cases, the maintainers have not been very reactive so far so I uploaded my changes as delayed NMU.

pkg-security team. The team that I started quietly a few months ago is now growing, both with new members and new packages. I created the required Teams/pkg-security wiki page. I sponsored xprobe, hydra, made an upload of medusa to merge Kali changes into Debian (and at the same time submitting the patch to upstream).

fontconfig. After having read Jonathan McDowell’s analysis of a bug that I experienced multiple times (and that many Kali users had too), I opened bug #828037 to get it fixed once for all. Unfortunately, nothing happened yet.

DebConf 16

I spent some time to prepare the 2 talks and the BoF that I will give/manage in Cape Town next week:

  • Kali Linux’s Experience https://debconf16.debconf.org/talks/39/
  • 2 Years of Work of Paid Contributors in the Debian LTS Project https://debconf16.debconf.org/talks/40/
  • Using Debian Money to Fund Debian Projects https://debconf16.debconf.org/talks/41/

Distro Tracker

I continued to mentor Vladimir Likic who managed to finish his first patch. He is now working on documentation for new contributors based on his recent experience.

I enhanced the tox configuration to run tests with Django 1.8 LTS with fatal warnings (python -Werror) so as to ensure that I’m not relying on any deprecated feature and so that I can be sure that the codebase will work on the next Django LTS release (1.11). Thanks to this, I did discover quite a few places where I have been using deprecated API and I fixed them all (the JSONField update to 1.0.0 I mentionned above was precisely to fix such a warning).

I also fixed a few more issues with folded mail headers that you can’t inject back in a new Message object and with messages lacking the subject field. All those have been caught through real (spam) email generating exceptions wich are then mailed to me.

Kali related work

I uploaded a new live-boot (5.20160608) to Debian to fix a bug where the boot process was blocking on some timeout.

I forwarded a Kali bug against libatk-wrapper-java (#827741) which turned out to be an OpenJDK bug.

I filed #827749 against reprepro to request a way to remove selected internal file references. This is required if you want to be able to make a file disappear and if that file is part of a snapshot that you want to keep despite this. But in truth, my real need is to be able to replace the .orig.tar.gz used by Kali by the orig.tar.gz used by Debian… those conflicts break the mirroring/import script.

Salt

I have been using salt to deploy a new service, and I developed patches for a few issues in salt formulas. I also created a new letsencrypt-sh formula to manage TLS certificates with the letsencrypt.sh ACME client.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in May 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

Debian LTS

Due to some departure and increased workload, I wanted to find out a few new paid contributors for Debian LTS. So I sent a mail to debian-jobs@lists.debian.org and contrary to last time (where I posted the announce only here on my blog), I had plenty of replies… I ended up enrolling 6 new contributors and refusing 3 persons that did not have the required profile.

All new contributors are supposed to handle at least one LTS update on their free time to get up to speed. But from the 6 new contributors only 3 managed to handle their “training update” in May. 🙁

During the month I spent quite some time providing guidance to the new contributors both in private mails and on the debian-lts mailing list.

I also reviewed a xen update where I had (rightfully) some doubts about the work done.

Packaging work

fonts-cantarell. After having diagnosed the problem last month, I got annoyed enough by the lack of a fixed package that I found a way to package a newer upstream release of fonts-cantarell without requiring a fontforge update that was likely to take some time still… so I prepared and uploaded 0.0.24-1.

cpputest. Bug #823711 reported some license issues with some of the files. I immediately forwarded this upstream (issue 961) and fixed it in Debian by repacking the upstream tarball. Fortunately upstream has been quick to handle his and there’s a new upstream release (3.8) where the problematic files have been dropped.

live-boot. Kali’s live images were no longer booting (stuck in the initrd) and with the help of Ben Hutchings we diagnosed this back to #823069 which I fixed in live-boot 20160511.

udev. I filed #824025 to request that the rule defining the MAC-based name of USB network interfaces be isolated in its own file so that it can be easily disabled (we do that in Kali).

Misc stuff. I packaged Django 1.8.13 in jessie-backports. I filed
#824165 against sbuild being broken with “$apt_allow_unauthenticated = 1;” in .sbuildrc. I filed a wishlist bug #824168 against apt-listchanges to suggest that it ignores news from auto-installed packages. I filed #825923 to report a regression in python-nltk (discovered in Kali first).

Infrastructure work

packages.debian.org. A few months ago, I wrote a patch for packages.debian.org so that it forwards emails to tracker.debian.org instead of packages.qa.debian.org. At that time, I was in touch with Rhonda and was hoping that she would apply it rather quickly (the patch is rather short). After a few more pings, she made it clear that she was not alone and that I should rather file a proper request so that someone else can also process it. So I filed #824085 and tried to find someone else to apply my patch. Most of the members of pkg_maint said that they were part of the group only due to generic webmaster involvement but that they did not want to touch that part. Fortunately, Martin Zobel Helas was more receptive to my request and helped me to deploy my changes. I committed my change and Martin pulled it in the live checkout on picconi.debian.org.

This update is also a first step towards the possibility to use foo@packages.debian.org and/or teams+foo@tracker.debian.org in the Maintainer field of a package. With this we can get rid of dedicated mailing lists that just duplicate the work of the package tracker. And we no longer need to care about the fact that the Maintainer is handled differently than Uploaders since all (human) co-maintainers would then be listed in Uploaders only (and the package tracker would deal appropriately with mails sent to the Maintainer).

Distro Tracker. I improved the import process to be able to force a new processing of source packages that were already imported. This was useful to let it recognize architectures which were newly added in its database (and that were ignored and thus not displayed up to now).

I also made a first review of the AppStream patch submitted by Matthias Klump in #806740.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in April 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

Debian LTS

I handled a new LTS sponsor that wanted to see wheezy keep supporting armel and armhf. This was not part of our initial plans (set during last Debconf) and I thus mailed all teams that were impacted if we were to collectively decide that it was OK to support those architectures. While I was hoping to get a clear answer rather quickly, it turns out that we never managed to get an answer to the question from all parties. Instead the discussion drifted on the more general topic of how we handle sponsorship/funding in the LTS project.

Fortunately, the buildd maintainers said they were OK with this and the ftpmasters had no objections, and they both implicitly enacted the decision: Ansgar Burchardt kept the armel/armhf architectures in the wheezy/updates suite when he handled the switch to the LTS team, and Aurélien Jarno also configured wanna-build to keep building armel/armhf for the suite. The DSA team did not confirm that this change was not interfering with one of their plans to decommission some hardware. Build daemons are a shared resource anyway and a single server is likely to handle builds for multiple releases.

DebConf 16

This month I registered for DebConf 16 and submitted multiple talk/BoF proposals:

  • Kali Linux’s Experience of a Debian Derivative Based on Testing (Talk)
  • 2 Years of Work of Paid Contributors in the Debian LTS Project (Talk)
  • Using Debian Money to Fund Debian Projects (BoF)

I want to share the setup we use in Kali as it can be useful for other derivatives and also for Debian itself to help smooth the relationship with derivatives.

I also want to open again the debate on the usage of money within Debian. It’s a hard topic but we should really strive to take some official position on what’s possible and what’s not possible. With Debian LTS and its sponsorship we have seen that we can use money to some extent without hurting the Debian project as a whole. Can this be transposed to other teams or projects? What are the limits? Can we define a framework and clear rules? I expect the discussion to be very interesting in the BoF. Mehdi Dogguy has agreed to handle this BoF with me.

Packaging

Django. I uploaded 1.8.12 to jessie-backports and 1.9.5 to unstable. I filed two upstream bugs (26473 and 26474) for two problems spotted by lintian.

Unfortunately, when I wanted to upload it to unstable, the test suite did not ran. I pinned this down to a sqlite regression. Chris Lamb filed #820225 and I contacted the SQLite and Django upstream developers by email to point them to this issue. I helped the SQLite upstream author (Richard Hipp) to reproduce the issue and he was quick to provide a patch which landed in 3.12.1.

Later in the month I made another upload to fix an upgrade bug (#821789).

GNOME 3.20. As for each new version, I updated gnome-shell-timer to ensure it works with the new GNOME. This time I spent a bit more time to fix a regression (805347) that dates back to a while and that would never be fixed otherwise since the upstream author orphaned this extension (as he no longer uses GNOME).

I have also been bitten by display problems where accented characters would be displayed below the character that follows. With the help of members of the GNOME team, we found out that this was a problem specific to the cantarell font and was only triggered with Harfbuzz 1.2. This is tracked in Debian with #822682 on harfbuzz and #822762 in fonts-cantarell. There’s a new upstream release (with the fix) ready to be packaged but unfortunately it is blocked by the lack of a recent fontforge in Debian. I thus mailed debian-mentors in the hope to find volunteers to help the pkg-fonts team to package a newer version…

Misc Debian/Kali work

Distro Tracker. I started to mentor Vladimir Likic who contacted me because he wants to contribute to Distro Tracker. I helped him to setup his development environment and we fixed a few issues in the process.

Bug reports. I filed many bug reports, most of them due to my work on Kali:

  • #820288: a request to keep the wordpress package installable in older releases (due to renaming of many php packages)
  • #820660: request support of by-hash indices in reprepro
  • #820867: possibility to apply overrides on already installed packages in reprepro
  • #821070: jessie to stretch upgrade problem with samba-vfs-modules
  • #822157: python-future hides and breaks python-configparser
  • #822669: dh_installinit inserts useless autoscript for System V init script when package doesn’t contain any
  • #822670: dh-systemd should be merged into debhelper, we have systemd by default and debhelper should have proper support for it by default

I also investigated #819958 that was affecting testing since it has been reported to Kali as well. And I made an NMU of dh-make-golang to fix #819472 that I reported earlier.

Thanks

See you next month for a new summary of my activities.

My Free Software Activities in February and March 2016

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it’s one of the best ways to find volunteers to work with me on projects that matter to me.

I skipped my monthly report last time so this one will cover two months. I will try to list only the most important things to not make it too long. 🙂

The Debian Handbook

I worked with Ryuunosuke Ayanokouzi to prepare a paperback version of the Japanese translation of my book. Thanks to the efforts of everybody, it’s now available. Unfortunately, Lulu declined to take it in “distribution” program so it won’t be available on traditional bookstores (like Amazon, etc.). The reason is that they do not support non-latin character sets in the meta-data.

I tried to cheat a little bit by inputting the description in English (still explaining that the book was in Japanese) but they rejected it nevertheless because the English title could mislead people. So the paperback is only available on lulu.com. Fortunately, the shipping costs are reasonable if you pick the most economic offer.

Following this I invited the Italian, Spanish and Brazilian Portuguese translators to complete the work (they were close will all the strings already translated, mainly missing translated screenshots and some backcover content) so that we can also release paperback versions in those languages. It’s getting close to completion for them. Hopefully we will have those available until next month.

Distro Tracker

In early February, I tweaked the configuration to send (by email) exceptions generated by incoming mails and by routine task. Before this they were logged but I did not take the time to look into them. This quickly brought a few issues into light and I fixed them as they appeared: for instance the bounce handling code was getting confused when the character case was not respected, and it appears that some emails come back to us after having been lowercased. Also the code was broken when the “References” field used more than one line on incoming control emails.

This brought into light a whole class of problems with the database storing twice the same email with only differing case. So I did further work to merge all those duplicate entries behind a single email entry.

Later, the experimental Sources files changed and I had to tweak the code to work with the removal of the Files field (relying instead on Checksums-* to find out the various files part of the entry).

At some point, I also fixed the login form to not generate an exception when the user submits an empty form.

I also decided that I no longer wanted to support Django 1.7 in distro tracker as Django 1.8 is the current LTS version. I asked the Debian system administrators to update the package on tracker.debian.org with the version in jessie-backports. This allowed me to fix a few deprecation warnings that I kept triggering because I wanted the code to work with Django 1.7.

One of those warnings was generated by django-jsonfield though and I could not fix it immediately. Instead I prepared a pull request that I submitted to the upstream author.

Oh, and a last thing, I tweaked the CSS to densify the layout on the package page. This was one of the most requested changes from the people who were still preferring packages.qa.debian.org over tracker.debian.org.

Kali and new pkg-security team

As part of my Kali work, I have been fixing RC bugs in Debian packages that we use in Kali. But in many cases, I stumbled upon packages whose maintainers were really missing in action (MIA). Up to now, we were only doing non-maintainers upload (NMU) but I want to be able to maintain those packages more effectively so we created a new pkg-security team (we’re only two right now and we have no documentation yet, but if you want to join, you’re welcome, in particular if you maintain a package which is useful in the security field).

arm64 work. The first 3 packages that we took over (ssldump, sucrack, xprobe) are actually packages that were missing arm64 builds. We just started our arm64 port on Kali and we fixed them for that architecture. Since they were no longer properly maintained, in most cases it was just a matter of using dh_autoreconf to get up-to-date config.{sub,guess} files.

We still miss a few packages on arm64: vboot-utils (that we will likely take over soon since it’s offered for adoption), ruby-libv8 and ruby-therubyracer, ntopng (we have to wait a new luajit which is only in experimental right now). We also noticed that dh-make-golang was not available on arm64, after some discussion on #debian-buildd, I filed two bugs for this: #819472 on dh-make-golang and #819473 on dh-golang.

RC bug fixing. hdparm was affected by multiple RC bugs and the release managers were trying to get rid of it from testing. This removed multiple packages that were used by Kali and its users. So I investigated the situation of that package, convinced the current maintainers to orphan it, asked for new maintainers on debian-devel, reviewed multiple updates prepared by the new volunteers and sponsored their work. Now hdparm is again RC-bug free and has the latest upstream version. We also updated jsonpickle to 0.9.3-1 to fix RC bug #812114 (that I forwarded upstream first).

Systemd presets support in init-system-helpers. I tried to find someone (to hire) to implement the system preset feature I requested in #772555 but I failed. Still Andreas Henriksson was kind enough to give it a try and sent a first patch. I tried it and found some issues so I continued to improve it and simplify it… I submitted an updated patch and pinged Martin Pitt. He pointed me to the DEP-8 test failures that my patch was creating. I quickly fixed those afterwards. This patch is in use in Kali and lets us disable network services by default. I would like to see it merged in Debian so that everybody can setup systemd preset file and have their desire respected at installation time.

Misc bug reports. I filed #813801 to request a new upstream release of kismet. Same for masscan in #816644 and for wkhtmltopdf in #816714. We packaged (before Debian) a new upstream release of ruby-msgpack and found out that it was not building on armel/armhf so we filed two upstream tickets (with a suggested fix). In #814805, we asked the pyscard maintainer to reinstate python-pyscard that was dropped (keeping only the Python3 version) as we use the Python 2 version in Kali.

And there’s more: I filed #816553 (segfault) and #816554 against cdebootstrap. I asked for dh-python to have a better behaviour after having being bitten by the fact that “dh –with python3” was not doing what I expected it to do (see #818175). And I reported #818907 against live-build since it is failing to handle a package whose name contains an upper case character (it’s not policy compliant but dpkg supports them).

Misc packaging

I uploaded Django 1.9.2 to unstable and 1.8.9 to jessie-backports. I provided the supplementary information that Julien Cristau asked me in #807654 but despite this, this jessie update has been ignored for the second point release in a row. It is now outdated until I update it to include the security fixes that have been released in the mean time but I’m not yet sure that I will do it… the lack of cooperation of the release team for that kind of request is discouraging.

I sponsored multiple uploads of dolibarr (on security update notably) and tcpdf (to fix one RC bug).

Thanks

See you next month for a new summary of my activities.