20 Things to Learn About APT With the Free Chapter of the Debian Administrator’s Handbook

We just released a sample chapter of the Debian Administrator’s Handbook. It covers the APT family of tools: apt-get, aptitude, synaptic, update-manager, etc.


Click here to get your free sample chapter

I’m sure you will enjoy it. There are many interesting things to learn:

  • How to customize the sources.list file
  • The various APT repositories that Debian offers (Security Updates, Stable Updates, Proposed Updates, Backports, Experimental, etc.)
  • How to select the best Debian mirror for you
  • How to find old package versions
  • How to install the same selection of packages on multiple computers
  • How to install and remove a package on a single command-line
  • How to reinstall packages and how to install a specific version of a package
  • How to pass options to dpkg via APT
  • How to configure a proxy for APT
  • How to set priorities to various package sources (APT pinning)
  • How to safely mix packages from several distributions on a single system
  • How to use aptitude’s text-mode graphical interface
  • How to use the tracking of automatically installed packages to keep a clean system
  • How APT checks the authenticity of packages that it downloads
  • How to add supplementary GnuPG keys to APT’s trusted keyring
  • How to upgrade from one stable distribution to the next
  • How to handles problems after an upgrade
  • How to keep a system up-to-date
  • How to automate upgrades
  • How to find the package that you’re looking for

If you liked this chapter, click here to contribute a few euros towards the liberation of the whole book. That way you’ll get a copy of the ebook as soon as it’s available. Thank you!

I also invite you to share this sample chapter as widely as possible. We’re only at 40% of the liberation fund and there’s less than 2 weeks left. I hope this book extract will convince enough people that the book is going to be great, and that it really deserves to be liberated and bundled with Debian!

Understanding dpkg’s file overwrite error

This is probably one of the most common errors. You’re very likely to encounter it, in particular if you tend to mix packages from various origins/distributions, or if you’re using unstable. It looks like this:

Unpacking gbonds-data (from .../gbonds-data_2.0.3-2_all.deb) ...
dpkg: error processing /var/cache/apt/archives/gbonds-data_2.0.3-2_all.deb (--unpack):
 trying to overwrite '/usr/share/omf/gbonds/gbonds-C.omf', which is also in package gbonds 2.0.2-9
dpkg-deb: subprocess paste killed by signal (Broken pipe)

A given file can only be provided by a single package. So if you try to install a package that provides a file that is already part of another installed package, it will fail with a message similar to the above one.

Sometimes this failure will be meaningful because dpkg prevented you to install two unrelated packages that happen to have a real file conflict. In other cases, like in the example above, this failure is just the result of a mistake.

Folder with gears

The version 2.0.3-1 of gbonds split the architecture independent files in a separate package called gbonds-data but the maintainer forgot to add the required control field in gbonds-data (Replaces: gbonds (<< 2.0.3-1)). That field allows dpkg to take over files from the listed packages.

If you want to ignore the file conflict and let dpkg take over the file (even without the Replaces), you can pass the --force-overwrite command-line option.

But you’re not using dpkg directly, you’re probably using an APT frontend (like apt-get or aptitude). Don’t worry, there’s a simple way to define custom dpkg options to use:

# apt-get -o Dpkg::Options::="--force-overwrite" install gbonds-data

The syntax is a bit weird, but the “::” after “Options” is important, it’s the syntax that defines a list item value instead of a single value. And you can effectively pass multiple options to dpkg by putting multiple -o Dpkg::Options::="…".

If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

Deciphering one of dpkg’s weirdest errors: short read on buffer copy

As a Debian/Ubuntu user, you’re likely to be exposed at some point to an error reported by dpkg. In a series of articles, I’ll explain some of the errors that you might encounter.

Some error messages can be confusing at times. Most of the error strings do not appear very often and developers thus tend to use very terse description of the underlying problem. In other cases the architecture of the software makes it difficult to pin-point the real problem because the part that displays the error is several layers above the one that generated the initial error.

This is for example the case with this error of dpkg:

Unpacking replacement xulrunner-1.9.2 ...
dpkg-deb (subprocess): data: internal gzip read error: '<fd:0>: too many length or distance symbols'
dpkg-deb: error: subprocess <decompress> returned error exit status 2
dpkg: error processing /var/cache/apt/archives/xulrunner-1.9.2_1.9.2.17+build3+nobinonly-0ubuntu1_amd64.deb (--unpack):
 short read on buffer copy for backend dpkg-deb during `./usr/lib/xulrunner-1.9.2.17/components/libdbusservice.so'

First, the decompression layer discovers something unexpected in the data read in the .deb file and dpkg-deb outputs the error message coming from zlib (“too many length or distance symbols”). This causes the premature end of dpkg-deb --fsys-tarfile that dpkg had executed to extract the .data.tar archive from the deb file. In turn, dpkg informs us that dpkg-deb did not send all the data that were announced (and hence the “short read” in the error message) and that were meant to be part of the file ‘/usr/lib/xulrunner-1.9.2.17/components/libdbusservice.so’.

That’s all nice but it doesn’t help you much in general. What you must understand from the above is that the .deb file is corrupted (sometimes just truncated). In theory it should not happen since APT verifies the checksums of files when they are downloaded. But computers are not infallible and even if the downloaded data was good, it can have been corrupted when stored on disk (for example cheap SSD disks are known to not last very well).

Try removing the file (usually with apt-get clean since it’s stored in APT’s cache) and let APT download it again. Chances are that it will work on the second try. Otherwise consider doing a memory and HDD check as something is probably broken in your computer.

Join my free newsletter and learn more tips for users. Or click here to support my work on dpkg with Flattr, consider subscribing for a few months.

apt-get, aptitude, … pick the right Debian package manager for you

This is a frequently asked question: “What package manager shall I use?”. And my answer is “the one that suits your needs”. In my case, I even use different package managers depending on what I’m trying to do.

APT vs dpkg, which one is the package manager?

In the Debian world, we’re usually thinking of APT-based software when we’re referring to a “package manager”. But in truth, the real package manager is dpkg. It’s the low-level tool that takes a .deb file and extracts its content on the disk, or that takes the name of a package to remove the associated files, etc.

APT is better known because it’s the part of the packaging infrastructure that matters to the user. APT makes collection of software available to the user and does the dirty work of downloading all the required packages and installing them by calling dpkg in the correct order to respect the dependencies.

But APT is not a simple program, it’s a library and several different APT frontends have been developed on top of that library. The most widely known is apt-get since it’s the oldest one, and it’s provided by APT itself.

Graphical APT front-ends

update-manager is a simple frontend useful to install security updates and other trivial daily upgrades (if you’re using testing or sid). It’s the one that you get when you click in the desktop notification that tells you that updates are available. In cases, where the upgrade is too complicated for update-manager, it will suggest to run synaptic which is full featured package manager. You can browse the list on installed/available packages in numerous ways, you can mark packages for installation/upgrade/removal/purge and then run in one go all the recorded actions.

software-center aims to be an easy to use application installer, it will hide most of the packaging details and will only present installed/available applications (as defined by a .desktop file). It’s very user friendly and has been developed by Ubuntu.

Of the graphical front-ends, I use mainly synaptic and only when I’m reviewing what I have installed to trim the system down.

Console-based GUI APT front-ends

In this category, I’ll cite only aptitude. Run without parameter, it will start a powerful console-based GUI. Much like synaptic, you can have multiple views of the installed/available packages and mark packages for installation/upgrade/removal/purge before executing everything at once.

Command-line based package managers and APT front-ends

This is where the well known apt-get fits, but there are several other alternatives: aptitude, cupt, wajig. Wajig and cupt are special cases as they don’t use libapt: the former wraps several tools including apt-get, and the latter is a (partial) APT reimplementation (versions 1.x were in Perl, 2.x are now is C++).

You’re welcome to try them out and find out which one you prefer, but I have never felt the need to use something else than apt-get and aptitude.

apt-get or aptitude?

First I want to make it clear that you can use both and mix them without problems. It used to be annoying when apt-get did not track which packages were automatically installed while aptitude did, but now that both packages share this list, there’s no reason to avoid switching back and forth.

I would recommend apt-get for the big upgrades (i.e. dist-upgrade from one stable to the next) because it will always find quickly a relatively good solution while aptitude can find several convoluted solutions (or none) and it’s difficult to decide which one should be used.

On the opposite for regular upgrades in unstable (or testing), I would recommend “aptitude safe-upgrade“. It does a better job than apt-get at keeping on hold packages which are temporarily broken due to some not yet finished changes while still installing new packages when required. With aptitude it’s also possible to tweak dynamically the suggested operations while apt-get doesn’t allow this. And aptitude’s command line is probably more consistent: with apt-get you have to switch between apt-get and apt-cache depending on the operation that you want to do, aptitude on the other hand does everything by itself.

Take some time to read their respective documentation and to try them.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

Debian Cleanup Tip #6: Remove automatically installed packages that are no longer needed

Last week we learned how to identify cruft on your Debian system. This week, for the last article in this series, we’ll learn more about automatically installed packages and how to get rid of them when you don’t need them any longer.

APT tracks automatically installed packages

When you install a new package with apt-get/aptitude/synaptic, it’s very common to end up installing many more packages: those are the dependencies of the installed package. Here’s an example:

$ sudo apt-get install pino
[...]
The following extra packages will be installed:
  libdbusmenu-glib1 libgee2 libindicate4 libnotify1 notification-daemon
The following NEW packages will be installed:
  libdbusmenu-glib1 libgee2 libindicate4 libnotify1 notification-daemon pino
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 478 kB of archives.
After this operation, 2531 kB of additional disk space will be used.
Do you want to continue [Y/n]? 

After this installation, the 5 “extra packages” will be marked as “automatically installed”. What does this mean? It means that you have not explicitly requested their installation and that the system should be free to remove them as soon as they are no longer needed.

You can verify that this is effectively the case with “apt-mark showauto” (it returns a list of the automatically installed packages).

$ apt-mark showauto |grep libdbusmenu
libdbusmenu-glib1
$ apt-mark showauto |grep pino
$

Aptitude shows this information with the “A” letter in its interactive interface and in the “aptitude search” output. “aptitude show” has a dedicated field for this:

$ aptitude show libdbusmenu-glib1
Package: libdbusmenu-glib1               
New: yes
State: installed
Automatically installed: yes
Version: 0.3.7-1
[...]

In Synaptic, it’s not very visible but once you have selected an installed package, you can verify in the “Package” menu whether “Automatically installed” is checked or not.

APT tells you which packages are no longer needed

Over time, some of those automatically installed packages become unnecessary because the packages that depended on them no longer do. It might be that they are using a newer version of the same library, or they switched to use something else, or they are able to do the task themselves.

Whatever the reason, the original dependency has vanished and the automatically installed package is no longer needed on the system.

Aptitude will automatically remove those unneeded packages the next time you run it but apt-get and synaptic do not.

Apt-get will inform you that some packages are no longer needed and will even tell you how you can get rid of them:

$ sudo apt-get remove pino
[...]
The following packages were automatically installed and are no longer required:
  notification-daemon libdbusmenu-glib1 libnotify1 libgee2 libindicate4
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
  pino
0 upgraded, 0 newly installed, 1 to remove and 219 not upgraded.
After this operation, 1225 kB disk space will be freed.
Do you want to continue [Y/n]? 
[...]
$ sudo apt-get autoremove
[...]
The following packages will be REMOVED:
  libdbusmenu-glib1 libgee2 libindicate4 libnotify1 notification-daemon
0 upgraded, 0 newly installed, 5 to remove and 219 not upgraded.
After this operation, 1307 kB disk space will be freed.
Do you want to continue [Y/n]? 
[...]

Synaptic will show you the packages that can be removed in a new section name “Installed (auto removable)” if you select the “Status” button in the bottom-left pane.

It’s thus a good habit to get rid of those unneeded package from time to time.

Use this feature to trim down your system

While APT usually sets the “Automatically installed” flag, you can also set it manually. It’s a very simple way to tell the system “I don’t need this package directly, feel free to remove it if nothing else requires it”.

# With apt-get
$ sudo apt-mark markauto libxml-simple-perl
# Or with aptitude
$ sudo aptitude markauto libxml-simple-perl

You can also do it in the interactive interface of aptitude with the key “M” (and “m” for unmarking). To do it in Synaptic, you have to use the menu entry “Package > Automatically installed”.

Many users like to have a minimal set of packages installed but they don’t really know which packages are really important and trying to remove every package to look what happens is cumbersome.

Thanks to this feature, you don’t try removing packages but you flag them as automatically installed. There is few risks in doing so when it concerns libraries (including python/perl modules). If the package is not indirectly needed by one of your important packages, it will be removed by apt-get autoremove, otherwise it’s kept for as long as it’s needed.

I would suggest to not mark as such packages of priority higher or equal to important to avoid nasty surprises (although I say this to not be blamed in case you remove too much, in theory the system should not remove essential components and all dependencies should be complete).

Also be aware of the consequences when you mark “task” packages like “gnome” as automatically installed… it will suggest you to remove your whole desktop. If you want to trim down the default desktop, you should “unmark” the desktop packages that you want to keep:

$ sudo apt-mark unmarkauto gnome-session gnome-panel

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

People behind Debian: Michael Vogt, synaptic and APT developer

Michael and his daughter Marie

Michael has been around for more than 10 years and has always contributed to the APT software family. He’s the author of the first real graphical interface to APT—synaptic. Since then he created “software-center” as part of his work for Ubuntu. Being the most experienced APT developer, he’s naturally the coordinator of the APT team. Check out what he has to say about APT’s possible evolutions.

My questions are in bold, the rest is by Michael.

Who are you?

My name is Michael Vogt, I’m married and have two little daughters. We live in Germany (near to Trier) and I work for Canonical as a software developer. I joined Debian as a developer in early 2000 and started to contribute to Ubuntu in 2004.

What’s your biggest achievement within Debian or Ubuntu?

I can not decide on a single one so I will just be a bit verbose.

From the very beginning I was interested in improving the package manager experience and the UI on top for our users. I’m proud of the work I did with synaptic. It was one of the earliest UIs on top of apt. Because of my work on synaptic I got into apt development as well and fixed bugs there and added new features. I still do most of the uploads here, but nowadays David Kalnischkies is the most active developer.

I also wrote a bunch of tools like gdebi, update-notifier, update-manager, unattended-upgrade and software-properties to make the update/install situation for the user easier to deal with. Most of the tools are written in python so I added a lot of improvements to python-apt along the way, including the initial high level “apt” interface and a bunch of missing low-level apt_pkg features. Julian Andres Klode made a big push in this area recently and thanks to his effort the bindings are fully complete now and have good documentation.

My most recent project is software-center. Its aim is to provide a UI strongly targeted for end-users. The goal of this project is to make finding and installing software easy and beautiful. We have a fantastic collection of software to offer and software-center tries to present it well (including screenshots, instant search results and soon ratings&reviews). This builds on great foundations like aptdaemon by Sebastian Heinlein, screenshots.debian.net by Christoph Haas, ddtp.debian.org by Michael Bramer, apt-xapian-index by Enrico Zini and many others (this is what I love about free software, it usually “adds”, rarely “takes away”).

What are your plans for Debian Wheezy?

For apt I would love to see a more plugable architecture for the acquire system. It would be nice to be able to make apt-get update (and the frontends that use this from libapt) be able to download additional data (like debtags or additional index file that contains more end-user targeted information). I also want to add some scripts so that apt (optionally) creates btrfs snapshots on upgrade and provide some easy way to rollback in case of problems.

There is also some interesting work going on around making the apt problem resolver a more plugable part. This way we should be able to do much faster development.

software-center will get ratings&reviews in the upstream branch, I really hope we can get that into Wheezy.

If you could spend all your time on Debian, what would you work on?

In that case I would start with a refactor of apt to make it more robust about ABI breaks. It would be possible to move much faster once this problem is solved (its not even hard, it just need to be done). Then I would add a more complete testsuite.

Another important problem to tackle is to make maintainer scripts more declarative. I triaged a lot of upgrade bug reports (mostly in ubuntu though) and a lot of them are caused by maintainer script failures. Worse is that depending on the error its really hard for the user to solve the problem. There is also a lot of code duplication. Having a central place that contains well tested code to do these jobs would be more robust. Triggers help us a lot here already, but I think there is still more room for improvement.

What’s the biggest problem of Debian?

That’s a hard question :) I mostly like Debian the way it is. What frustrated me in the past were flamewars that could have been avoided. To me being respectful to each other is important, I don’t like flames and insults because I like solving problems and fighting like this rarely helps that. The other attitude I don’t like is to blame people and complain instead of trying to help and be positive (the difference between “it sucks because it does not support $foo” instead of “it would be so helpful if we had $foo because it enables me to let me do $bar”).

For a long time, I had the feeling you were mostly alone working on APT and were just ensuring that it keeps working. Did you also had this feeling and are things better nowadays ?

I felt a bit alone sometimes :) That being said, there were great people like Eugene V. Lyubimkin and Otavio Salvador during my time who did do a lot of good work (especially at release crunch times) and helped me with the maintenance (but got interested in other area than apt later). And now we have the unstoppable David Kalnischkies and Julian Andres Klode.

Apt is too big for a single person, so I’m very happy that especially David is doing superb work on the day-to-day tasks and fixes (plus big project like multiarch and the important but not very thankful testsuite work). We talk about apt stuff almost daily, doing code reviews and discuss bugs. This makes the development process much more fun and healthy. Julian Andres Klode is doing interesting work around making the resolver more plugable and Christian Perrier is as tireless as always when it comes to the translations merging.

I did a quick grep over the bzr log output (including all branch merges) and count around ~4300 total commits (including all revisions of branches merged). Of that there ~950 commits from me plus an additional ~500 merges. It was more than just ensuring that it keeps working but I can see where this feeling comes from as I was never very verbose. Apt also was never my “only” project, I am involved in other upstream work like synaptic or update-manager or python-apt etc). This naturally reduced the time available to hack on apt and spend time doing the important day-to-day bug triage, response to mailing list messages etc.

One the python-apt side Julian Andres Klode did great work to improve the code and the documentation. It’s a really nice interface and if you need to do anything related to packages and love python I encourage you to try it. Its as simple as:

import apt
cache = apt.Cache()
cache["update-manager"].mark_install()
cache.commit()

Of course you can do much more with it (update-manager, software-center and lots of more tools use it). With “pydoc apt” you can get a good overview.

The apt team always welcomes contributors. We have a mailing list and a irc channel and it’s a great opportunity to solve real world problems. It does not matter if you want to help triage bugs or write documentation or write code, we welcome all contributors.

You’re also an Ubuntu developer employed by Canonical. Are you satisfied with the level of cooperation between both projects? What can we do to get Ubuntu to package new applications developed by Canonical directly in Debian?

Again a tricky question :) When it comes to cooperation there is always room for improvement. I think (with my Canonical hat on) we do a lot better than we did in the past. And it’s great to see the current DPL coming to Ubuntu events and talking about ways to improve the collaboration. One area that I feel that Debian would benefit is to be more positive about NMUs and shared source repositories (collab-maint and LowThresholdNmu are good steps here). The lower the cost is to push a patch/fix (e.g. via direct commit or upload) the more there will be.

When it comes to getting packages into Debian I think the best solution is to have a person in Debian as a point of contact to help with that. Usually the amount of work is pretty small as the software will have a debian/* dir already with useful stuff in it. But it helps me a lot to have someone doing the Debian uploads, responding to the bugmail etc (even if the bugmail is just forwarded as upstream bugreports :) IMO it is a great opportunity especially for new packagers as they will not have to do a lot of packaging work to get those apps into Debian. This model works very well for me for e.g. gdebi (where Luca Falavigna is really helpful on the Debian side).

Is there someone in Debian that you admire for his contributions?

There are many people I admire. Probably too many to mention them all. I always find it hard to single out individual people because the project as a whole can be so proud of their achievements.

The first name that comes to my mind is Jason Gunthorpe (the original apt author) who I’ve never met. The next is Daniel Burrows who I met and was inspired by. David Kalnischkies is doing great work on apt. From contributing his first (small) patch to being able to virtually fix any problem and adding big features like multiarch support in about a year. Sebastian Heinlein for aptdaemon.

Christian Perrier has always be one of my heroes because he cares so much about i18n. Christoph Haas for screenshots.debian.net, Michael Bramer for his work on debian translated package descriptions.


Thank you to Michael for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

Howto to rebuild Debian packages

Being able to rebuild an existing Debian package is a very useful skill. It’s a prerequisite for many tasks that an admin might want to perform at some point: enable a feature that is disabled in the official Debian package, rebuild a source package for another suite (for example build a Debian Testing package for use on Debian Stable, we call that backporting), include a bug fix that upstream developers prepared, etc. Discover the 4 steps to rebuild a Debian package.

1. Download the source package

The preferred way to download source packages is to use APT. It can download them from the source repositories that you have configured in /etc/apt/sources.list, for example:

deb-src http://ftp.debian.org/debian unstable main contrib non-free
deb-src http://ftp.debian.org/debian testing main contrib non-free
deb-src http://ftp.debian.org/debian stable main contrib non-free

Note that the lines start with “deb-src” instead of the usual “deb”. This tells APT that we are interested in the source packages and not in the binary packages.

After an apt-get update you can use apt-get source publican to retrieve the latest version of the source package “publican”. You can also indicate the distribution where the source package must be fetched with the syntax “package/distribution“. apt-get source publican/testing will grab the source package publican in the testing distribution and extract it in the current directory (with dpkg-source -x, thus you need to have installed the dpkg-dev package).

$ apt-get source publican/testing
Reading package lists... Done
Building dependency tree       
Reading state information... Done
NOTICE: 'publican' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/collab-maint/publican.git
Need to get 727 kB of source archives.
Get:1 http://nas/debian/ squeeze/main publican 2.1-2 (dsc) [2253 B]
Get:2 http://nas/debian/ squeeze/main publican 2.1-2 (tar) [720 kB]
Get:3 http://nas/debian/ squeeze/main publican 2.1-2 (diff) [4728 B]
Fetched 727 kB in 0s (2970 kB/s)  
dpkg-source: info: extracting publican in publican-2.1
dpkg-source: info: unpacking publican_2.1.orig.tar.gz
dpkg-source: info: unpacking publican_2.1-2.debian.tar.gz
$ ls -dF publican*
publican-2.1/                 publican_2.1-2.dsc
publican_2.1-2.debian.tar.gz  publican_2.1.orig.tar.gz

If you don’t want to use APT, or if the source package is not hosted in an APT source repository, you can download a complete source package with dget -u dsc-url where dsc-url is the URL of the .dsc file representing the source package. dget is provided by the devscripts package. Note that the -u option means that the origin of the source package is not verified before extraction.

2. Install the build-dependencies

Again APT can do the grunt work for you, you just have to use apt-get build-dep foo to install the build-dependencies for the last version of the source package foo. It supports the same syntactic sugar than apt-get source so that you can run apt-get build-dep publican/testing to install the build-dependencies required to build the testing version of the publican source package.

If you can’t use APT for this, enter the directory where the source package has been unpacked and run dpkg-checkbuilddeps. It will spit out a list of unmet build dependencies (if there are any, otherwise it will print nothing and you can go ahead safely). With a bit of copy and paste and a “apt-get install” invocation, you’ll install the required packages in a few seconds.

3. Do whatever changes you need

I won’t detail this step since it depends on your specific goal with the rebuild. You might have to edit debian/rules, or to apply a patch.

But one thing is sure, if you have made any change or have recompiled the package in a different environment, you should really change its version number. You can do this with “dch --local foo” (again from the devscripts package), replace “foo” by a short name identifying you as the supplier of the updated version. It will update debian/changelog and invite you to write a small entry documenting your change.

4. Build the package

The last step is also the simplest one now that everything is in place. You must be in the directory of the unpacked source package.
Now run either “debuild -us -uc” (recommended, requires the devscripts package) or directly “dpkg-buildpackage -us -uc”. The “-us -uc” options avoid the signature step in the build process that would generate a (harmless) failure at the end if you have no GPG key matching the name entered in the top entry of the Debian changelog.

$ cd publican-2.1
$ debuild -us -uc
 dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): 
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor): 
dpkg-buildpackage: source package publican
dpkg-buildpackage: source version 2.1-2rh1
dpkg-buildpackage: source changed by Raphaël Hertzog 
 dpkg-source --before-build publican-2.1
dpkg-buildpackage: host architecture i386
[...]
dpkg-deb: building package `publican' in `../publican_2.1-2rh1_all.deb'.
 dpkg-genchanges  >../publican_2.1-2rh1_i386.changes
dpkg-genchanges: not including original source code in upload
 dpkg-source --after-build publican-2.1
dpkg-buildpackage: binary and diff upload (original source NOT included)
Now running lintian...
Finished running lintian.

The build is over, the updated source and binary packages have been generated in the parent directory.

$ cd ..
$ ls -dF publican*
publican-2.1/                    publican_2.1-2rh1.dsc
publican_2.1-2.debian.tar.gz     publican_2.1-2rh1_i386.changes
publican_2.1-2.dsc               publican_2.1-2rh1_source.changes
publican_2.1-2rh1_all.deb        publican_2.1.orig.tar.gz
publican_2.1-2rh1.debian.tar.gz

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

People behind Debian: David Kalnischkies, an APT developer

The two first interviews were dedicated to long-time Debian developers. This time I took the opposite approach, I interviewed David Kalnischkies who is not (yet) a Debian developer. But he’s contributing to one of the most important software within Debian—the APT package manager—since 2009. You can already see him in many places in Debian sharing his APT knowledge when needed.

English is not his native language and he’s a bit shy, but he accepted the interview nevertheless. I would like to thank him for the efforts involved and I hope his story can inspire some people to take the leap and just start helping… My questions are in bold, the rest is by David.

Who are you?

I am David Kalnischkies, 22 years old, living in the small town Erbach near Wiesbaden in Germany and I’m studying computer science at the TU Darmstadt. Furthermore I am—for more than half a decade now—young group leader of my hometown.

I never intended to get into this position, but it has similarities with my “career” in this internet-thingy here. I don’t remember why, but in April 2009 I was at a stage that some simple bugs in APT annoyed me so much that I grabbed the source, and most importantly—I don’t know why I did it— but I published my changes in Mai with #433007, a few more bugs and even a branch on launchpad. And this public branch got me into all this trouble in June: I got a mail from “Mr. package managment” Michael Vogt regarding this branch…

A few days later I joined an IRC session with him and closely after that my name appeared for the first time in a changelog entry. It’s a strange but also addicting feeling to read your own name in an unfamiliar place. And even now after many IRC discussions, bugfixes and features, three Ubuntu Developer Summits and a Google Summer of Code in Debian, my name still appear in places I have never even thought about—e.g. in an interview.

What’s your biggest achievement within Debian?

I would like to answer “MultiArch in APT” as it was my Google Summer of Code project, but as it has (not much) use for the normal user at this point—will hopefully change for wheezy—I chose three smaller things in squeeze’s APT that many people don’t even know yet:

  • “MMap run out of room”, the most famous error related to the APT::Cache-Limit option is gone. The DynamicMMap in APT now deserves its name :)
  • Download and usage of multiple Translations of Descriptions
  • crazy stuff like apt-cache show ^apt-.*$/experimental works

If your impression is now that I only do APT stuff: that’s completely right, but that’s already more than enough for me for now as the toolchain behind the short name APT contains so many tools and use cases that you always have something different.

You’re an active member of the APT development team. Are there plans for APT in Debian Wheezy? What features can we expect?

That’s very hard to answer, as the team is too small to be able to really plan something. I mean, you can have fancy plans and everything and half a second later someone arrives on the mailing list with a “small” question which eats days of development time just for debugging…

But right now the TODO list contains (in no particular order):

  • MultiArch – managing packages built for different architectures like i386 on amd64 to ease the usage of packages which only work on specific architectures or cross-compiling.
  • rewriting the subsystem responsible for downloading files to e.g. enable tools like apt-file or debtags to reuse the infrastructure by plugging into it.
  • splitting long descriptions out of the Packages file to possibly decrease the size to download on an “apt-get update”. Includes mostly pulling the button only to enable it by the archive masters
  • backporting popular features of some frontends to enable all frontends to use them, e.g. “apt-get changelog” or “apt-get download”
  • support research on resolver improvements by implementing support for a distribution independent upgrade problem format (CUDF).
  • an additional single binary for users wrapping apt-get, apt-cache and co.
  • getting in contact (again) with various forks like apt-rpm and derivatives like maemo or telesphoreo to join forces instead of splitting them by working on different (and possibily very old) heads of the dragon

We will see what will get real for wheezy and what is postponed, but one thing is sure: more will be done for wheezy if you help!

If you could spend all your time on Debian, what would you work on?

I would spend it on APT’s debbugs count… zero would be cool to look at! We make progress in this regard, but with the current velocity we will reach it in ten years or so.

Reading more mailing lists would be interesting, as I am kind of an information junky. Maintaining a package could be interesting to share the annoyance of a maintainer with handcrafted dependencies just to notice that APT doesn’t get it in the way I intended it to be. Through, to make it feel real I need to train a few new APT contributors before so they can point my mistake out, but this unfortunately doesn’t depend so much on time but on victims… Maybe I could even be working on getting an official status.

Beside that, I would love to be able to “apt-get dist-upgrade” the increasing mass of systems I and many others carry around in their pockets. In regards to my phone, this is already fixed, but there is much room for improvements.

What’s the biggest problem of Debian?

You need to be lucky. You need to talk at the right time to the right person. That’s not really a debian-only problem as such, but in a global project full of volunteers you can see it clearly as there are plenty of opportunities to be unlucky.

For example, it’s unlikely that an interview would be made with me now if Michael had not contacted me in June 2009. In a big project like Debian, you are basically completely lost without a mentor guiding you, so things like the debian-mentors list are good projects, but I am pretty certain they could benefit from some more helping hands.

The other thing which I consider a problem is that—and I read from time to time—some people don’t care for translations. That’s bad. Yes, a developer is able to read English, otherwise s/he couldn’t write code or participate on the mailinglists.

Still, I personally prefer to use a translated application if I have the chance as it’s simply easier for me to read in my mother tongue, not only because I am dyslexic, but because my mind still thinks in German and not in English. Yes, I could personally fix that by thinking in English only from now on, but its a quite big problem to convince my family—which is not really familiar with tech-stuff—to use something if they can’t understand what is written on screen.

It was hard enough to tell my mother how to write an SMS in a German interface. My phone with English words all over the place would be completely unusable for her—despite the fact that my phone is powered by Debian and better for the task from a technical point of view.

You are not yet an official Debian developer/maintainer, but you’re already perceived in the community as one the most knowledgeable person about APT. It’s a great start! What’s your advice to other people who want to start contributing to Debian in general, and to APT in particular?

It was never a goal in my life to “start contributing”. My goal was and still is to make my life easier by letting the computer work for me. At some point APT hindered the success of this goal, so it needed to be fixed. I didn’t expect to open pandora’s box.

So, my advice is simple: Just start. Ignore the warning signs telling you that this is not easy. They are just telling you that you do something useful. Only artificial problems are easy. Further more, contribution to APT, dpkg or any other existing package is in no way harder than opening an ITP and working on your own, and it’s cooler as you have a similar minded team around you to talk to. :)

“APT didn’t accept release codenames as target release” was one of the first things I fixed. If I had asked someone if that would be a good starting point the answer would have been a clear “no”, but I didn’t search for a good starting point…

As a kid I can start playing football by just walking on the field and play or I can sit near the field, watching the others play, while analyzing which position would be the best for me to start ruling out one by one as the technical requirements seem too high… “Oh – bicycle kick – that sounds complicated… I can’t do that”

Julian Andreas Klode is working on a APT replacement, there’s also Cupt by Eugene V. Lyubimkin. Both projects started because their authors are not satisfied with APT, they find APT’s code difficult to hack partly due to the usage of C++. Do you share their concerns and what’s your opinion on those projects?

I don’t think C++ is a concern in this regard, after all cupt is currently rewritten to C++0x and APT2 started in vala and is now C + glib—last time I checked at least. I personally think that something is wrong if we need to advertise an application by saying in which language it is written…

The major “problem” for APT is probably that the code is “old”: APT does its job for more than 12 years now, under different maintainers with an always changing environment around it: so there are lines in APT which date from a time when nobody knew what a “Breaks” dependency is, that packages can have long descriptions which can be translated or even that package archives can be signed with a gpg key! And yet we take all those for granted today. APT has proven to adapt to these changes in the environment and became in this process very popular. So I don’t think the point is near (if it will come at all) that APT can go into retirement as it is completely replaced by something else.

The competitors one the other hand have their first 12 years still to go. And it will be interesting to see how they will evolve and what will be the state of the art in 2022…

But you asked what I think about the competitors: I prefer the “revolution from inside” simply because I can see effects faster as more users will profit from it now. Cupt and co. obviously prefer the “normal” revolution. The goal is the same, creating the best package manager tool, but the chosen way to the goal is different. aptitude and cupt have an interactive resolver for example: that’s something I dislike personally, for others that is the ultimate killer feature. cupt reading the same preference file as APT will have a different pinning result, which we should consider each time someone mentions the word “drop-in replacement”.

APT2 isn’t much more than the name—which I completely dislike—currently from a user point of view, so I can’t really comment on that. All of them make me sad as each line invested in boilerplate code like configuration file parsing would be in my eyes better be spent in a bugfix or new feature instead, but I am not here to tell anyone what they should do in their free time…

But frankly, I don’t see them really as competitors: I use the tools I use, if other do that too that’s good, if not that’s their problem. :) The thing that annoys me really are claims like “plan is to remove APT by 2014″ as this generates a “vi vs. emacs” like atmosphere we don’t need. If some people really think emacs is a good editor… who cares?

I really hope we all can drink a beer in 2022 in Milliways, the restaurant at the end of the package universe, remembering the good old 2010… ;)

Is there someone in Debian that you admire for his contributions?

No, not one, many!

Michael Vogt who has nearly the monopole of “package manager maintainer” by being upstream of APT, synaptics and software center to name only the biggest and still has the time to answer even the dumbest of my questions. :) Jason Gunthorpe for being one of the initial developers behind “deity” who I will probably never meet in person beside in old comments and commit logs. Christian Perrier for caring so much about translations. Obey Arthur Liu as a great admin for Debian’s participation in Google’s Summer of Code. Paul Wise for doing countless reviews on debian-mentors which are a good source of information—not only for the maintainer of the package under review.

I guess I need to stop here because you asked for just one. So let’s end with some big words instead: I am just a little cog in the big debian wheel…


Thank you to David Kalnischkies for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.