20 Things to Learn About APT With the Free Chapter of the Debian Administrator’s Handbook

We just released a sample chapter of the Debian Administrator’s Handbook. It covers the APT family of tools: apt-get, aptitude, synaptic, update-manager, etc.


Click here to get your free sample chapter

I’m sure you will enjoy it. There are many interesting things to learn:

  • How to customize the sources.list file
  • The various APT repositories that Debian offers (Security Updates, Stable Updates, Proposed Updates, Backports, Experimental, etc.)
  • How to select the best Debian mirror for you
  • How to find old package versions
  • How to install the same selection of packages on multiple computers
  • How to install and remove a package on a single command-line
  • How to reinstall packages and how to install a specific version of a package
  • How to pass options to dpkg via APT
  • How to configure a proxy for APT
  • How to set priorities to various package sources (APT pinning)
  • How to safely mix packages from several distributions on a single system
  • How to use aptitude’s text-mode graphical interface
  • How to use the tracking of automatically installed packages to keep a clean system
  • How APT checks the authenticity of packages that it downloads
  • How to add supplementary GnuPG keys to APT’s trusted keyring
  • How to upgrade from one stable distribution to the next
  • How to handles problems after an upgrade
  • How to keep a system up-to-date
  • How to automate upgrades
  • How to find the package that you’re looking for

If you liked this chapter, click here to contribute a few euros towards the liberation of the whole book. That way you’ll get a copy of the ebook as soon as it’s available. Thank you!

I also invite you to share this sample chapter as widely as possible. We’re only at 40% of the liberation fund and there’s less than 2 weeks left. I hope this book extract will convince enough people that the book is going to be great, and that it really deserves to be liberated and bundled with Debian!

apt-get, aptitude, … pick the right Debian package manager for you

This is a frequently asked question: “What package manager shall I use?”. And my answer is “the one that suits your needs”. In my case, I even use different package managers depending on what I’m trying to do.

APT vs dpkg, which one is the package manager?

In the Debian world, we’re usually thinking of APT-based software when we’re referring to a “package manager”. But in truth, the real package manager is dpkg. It’s the low-level tool that takes a .deb file and extracts its content on the disk, or that takes the name of a package to remove the associated files, etc.

APT is better known because it’s the part of the packaging infrastructure that matters to the user. APT makes collection of software available to the user and does the dirty work of downloading all the required packages and installing them by calling dpkg in the correct order to respect the dependencies.

But APT is not a simple program, it’s a library and several different APT frontends have been developed on top of that library. The most widely known is apt-get since it’s the oldest one, and it’s provided by APT itself.

Graphical APT front-ends

update-manager is a simple frontend useful to install security updates and other trivial daily upgrades (if you’re using testing or sid). It’s the one that you get when you click in the desktop notification that tells you that updates are available. In cases, where the upgrade is too complicated for update-manager, it will suggest to run synaptic which is full featured package manager. You can browse the list on installed/available packages in numerous ways, you can mark packages for installation/upgrade/removal/purge and then run in one go all the recorded actions.

software-center aims to be an easy to use application installer, it will hide most of the packaging details and will only present installed/available applications (as defined by a .desktop file). It’s very user friendly and has been developed by Ubuntu.

Of the graphical front-ends, I use mainly synaptic and only when I’m reviewing what I have installed to trim the system down.

Console-based GUI APT front-ends

In this category, I’ll cite only aptitude. Run without parameter, it will start a powerful console-based GUI. Much like synaptic, you can have multiple views of the installed/available packages and mark packages for installation/upgrade/removal/purge before executing everything at once.

Command-line based package managers and APT front-ends

This is where the well known apt-get fits, but there are several other alternatives: aptitude, cupt, wajig. Wajig and cupt are special cases as they don’t use libapt: the former wraps several tools including apt-get, and the latter is a (partial) APT reimplementation (versions 1.x were in Perl, 2.x are now is C++).

You’re welcome to try them out and find out which one you prefer, but I have never felt the need to use something else than apt-get and aptitude.

apt-get or aptitude?

First I want to make it clear that you can use both and mix them without problems. It used to be annoying when apt-get did not track which packages were automatically installed while aptitude did, but now that both packages share this list, there’s no reason to avoid switching back and forth.

I would recommend apt-get for the big upgrades (i.e. dist-upgrade from one stable to the next) because it will always find quickly a relatively good solution while aptitude can find several convoluted solutions (or none) and it’s difficult to decide which one should be used.

On the opposite for regular upgrades in unstable (or testing), I would recommend “aptitude safe-upgrade“. It does a better job than apt-get at keeping on hold packages which are temporarily broken due to some not yet finished changes while still installing new packages when required. With aptitude it’s also possible to tweak dynamically the suggested operations while apt-get doesn’t allow this. And aptitude’s command line is probably more consistent: with apt-get you have to switch between apt-get and apt-cache depending on the operation that you want to do, aptitude on the other hand does everything by itself.

Take some time to read their respective documentation and to try them.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

Debian Cleanup Tip #6: Remove automatically installed packages that are no longer needed

Last week we learned how to identify cruft on your Debian system. This week, for the last article in this series, we’ll learn more about automatically installed packages and how to get rid of them when you don’t need them any longer.

APT tracks automatically installed packages

When you install a new package with apt-get/aptitude/synaptic, it’s very common to end up installing many more packages: those are the dependencies of the installed package. Here’s an example:

$ sudo apt-get install pino
[...]
The following extra packages will be installed:
  libdbusmenu-glib1 libgee2 libindicate4 libnotify1 notification-daemon
The following NEW packages will be installed:
  libdbusmenu-glib1 libgee2 libindicate4 libnotify1 notification-daemon pino
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 478 kB of archives.
After this operation, 2531 kB of additional disk space will be used.
Do you want to continue [Y/n]? 

After this installation, the 5 “extra packages” will be marked as “automatically installed”. What does this mean? It means that you have not explicitly requested their installation and that the system should be free to remove them as soon as they are no longer needed.

You can verify that this is effectively the case with “apt-mark showauto” (it returns a list of the automatically installed packages).

$ apt-mark showauto |grep libdbusmenu
libdbusmenu-glib1
$ apt-mark showauto |grep pino
$

Aptitude shows this information with the “A” letter in its interactive interface and in the “aptitude search” output. “aptitude show” has a dedicated field for this:

$ aptitude show libdbusmenu-glib1
Package: libdbusmenu-glib1               
New: yes
State: installed
Automatically installed: yes
Version: 0.3.7-1
[...]

In Synaptic, it’s not very visible but once you have selected an installed package, you can verify in the “Package” menu whether “Automatically installed” is checked or not.

APT tells you which packages are no longer needed

Over time, some of those automatically installed packages become unnecessary because the packages that depended on them no longer do. It might be that they are using a newer version of the same library, or they switched to use something else, or they are able to do the task themselves.

Whatever the reason, the original dependency has vanished and the automatically installed package is no longer needed on the system.

Aptitude will automatically remove those unneeded packages the next time you run it but apt-get and synaptic do not.

Apt-get will inform you that some packages are no longer needed and will even tell you how you can get rid of them:

$ sudo apt-get remove pino
[...]
The following packages were automatically installed and are no longer required:
  notification-daemon libdbusmenu-glib1 libnotify1 libgee2 libindicate4
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
  pino
0 upgraded, 0 newly installed, 1 to remove and 219 not upgraded.
After this operation, 1225 kB disk space will be freed.
Do you want to continue [Y/n]? 
[...]
$ sudo apt-get autoremove
[...]
The following packages will be REMOVED:
  libdbusmenu-glib1 libgee2 libindicate4 libnotify1 notification-daemon
0 upgraded, 0 newly installed, 5 to remove and 219 not upgraded.
After this operation, 1307 kB disk space will be freed.
Do you want to continue [Y/n]? 
[...]

Synaptic will show you the packages that can be removed in a new section name “Installed (auto removable)” if you select the “Status” button in the bottom-left pane.

It’s thus a good habit to get rid of those unneeded package from time to time.

Use this feature to trim down your system

While APT usually sets the “Automatically installed” flag, you can also set it manually. It’s a very simple way to tell the system “I don’t need this package directly, feel free to remove it if nothing else requires it”.

# With apt-get
$ sudo apt-mark markauto libxml-simple-perl
# Or with aptitude
$ sudo aptitude markauto libxml-simple-perl

You can also do it in the interactive interface of aptitude with the key “M” (and “m” for unmarking). To do it in Synaptic, you have to use the menu entry “Package > Automatically installed”.

Many users like to have a minimal set of packages installed but they don’t really know which packages are really important and trying to remove every package to look what happens is cumbersome.

Thanks to this feature, you don’t try removing packages but you flag them as automatically installed. There is few risks in doing so when it concerns libraries (including python/perl modules). If the package is not indirectly needed by one of your important packages, it will be removed by apt-get autoremove, otherwise it’s kept for as long as it’s needed.

I would suggest to not mark as such packages of priority higher or equal to important to avoid nasty surprises (although I say this to not be blamed in case you remove too much, in theory the system should not remove essential components and all dependencies should be complete).

Also be aware of the consequences when you mark “task” packages like “gnome” as automatically installed… it will suggest you to remove your whole desktop. If you want to trim down the default desktop, you should “unmark” the desktop packages that you want to keep:

$ sudo apt-mark unmarkauto gnome-session gnome-panel

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

Debian Cleanup Tip #3: get rid of third-party packages

Last week, we learned how to get rid of obsolete packages. This time, we’re going to learn how to bring back your computer to a state close to a “pure” Ubuntu/Debian installation.

Thanks to the power of APT, it’s easy to add new external repositories and install supplementary software. Unfortunately some of those are not very well maintained. They might contain crappy packages or they might simply not be updated. An external package which was initially working well, can become a burden on system maintenance because it will be interfering with regular updates (for example by requiring a package that should be removed in newer versions of the system).

So my goal for today is to teach you how to identify the packages on your system that are not coming from Debian or Ubuntu. So that you can go through them from time to time and keep only those that you really need. Obsolete packages are a subset of those, but I’ll leave them alone. We took care of them last week.

Each (well-formed) APT repository comes with a “Release” file describing it (example). They provide some values that can be used by APT to identify packages contained in the repository. All official Debian repositories are documented with Origin=Debian (and Origin=Ubuntu for Ubuntu). You can verify the origin value associated to each repository (if any) in the output of apt-cache policy:

[...]
 500 http://ftp.debian.org/debian/ lenny/main i386 Packages
     release v=5.0.8,o=Debian,a=stable,n=lenny,l=Debian,c=main
     origin ftp.debian.org
[...]

From there on, we can simply ask aptitude to compute a list of packages which are both installed and not available in an official Debian repository:

$ aptitude search '?narrow(?installed, !?origin(Debian))!?obsolete'
or
$ aptitude search '~S ~i !~ODebian !~o'

You can replace “search” with “purge” or “remove” if you want to get rid of all the packages listed. But you’re more likely to want to remove only a subset of carefully chosen packages… you’re probably still using some of the software that you installed from external repositories.

With synaptic, you can also browse the content of each repository. Click on the “Origin” button and you have a list of repositories. You can go through the non-Debian repositories and look which packages are installed and up-to-date.

But you can do better, you can create a custom view. Click on the menu entry “Settings > Filter”. Click on “New” to create a new filter and name it “External packages”. Unselect everything in the “Status” tab and keep only “Installed”.

Go in the “Properties” tab and here add a new entry “Origin” “Excludes” “ftp.debian.org”. In fact you must replace “ftp.debian.org” with the hostname of your Debian/Ubuntu mirror. The one that appears on the “origin” line in the output of apt-cache policy (see the excerpt quoted above in this article).

Note that the term “Origin” is used to refer to two different things, a field in the release file but also the name of the host for an APT repository. It’s a bit confusing if you don’t pay attention.

Close the filters window with OK. You now have a new listing of “External packages” under the “Custom Filters” screen. You can see which packages are installed and up-to-date and decide whether you really want to keep it. If the package is also provided by Debian/Ubuntu and you want to go back to the version provided by your distribution, you can use the “Package > Force version…” menu entry.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

Debian Cleanup Tip #2: Get rid of obsolete packages

Last week, we learned to remove useless configuration files. This week, we’re going to take care of obsolete packages.

An obsolete package is a package who is no longer provided by any of the APT repositories listed in /etc/apt/source.lists (and /etc/apt/sources.list.d/). There can be multiple reasons why a package is no longer available in the repository (or at least not under the same name) :

  • the upstream author stopped maintaining the software a long time ago, nobody else took over and the Debian maintainer preferred to remove the package from Debian. Usually there are alternatives in the Debian archive.
  • the package was orphaned in Debian since a long time, nobody took over and it had very few users. The Debian QA team might have asked its removal.
  • the latest version of the software might have been packaged under a new package name. Either because the amount of changes was so important that it was preferred to not upgrade automatically to the latest version (it has been the case with request-tracker and nagios, they both embed a version number in their package names), or simply because the maintainer wants to let the user install several versions at the same time (that’s the case for example with the Linux kernel, the python interpreter and many libraries).
  • the software has been renamed, the maintainer renamed the packages and kept transitional packages under the old name for one release. Then the transitional packages have been removed.

In any case, it’s never a good idea to keep obsolete packages around: they do not benefit from security updates and they might cause problems during upgrades if they depend on other packages that should be removed to complete the upgrade.

You could blindly remove them with aptitude purge ~o (or aptitude purge ?obsolete) but you might want to first verify what those package are. There might be some packages that you have manually installed, that are not part of any current APT repository, and that you want to keep around nevertheless (I have skype, dropbox and a few personal packages for example). You can get the list with aptitude search ?obsolete

With the graphical package manager (Synaptic), you can find the list of obsolete packages by clicking on the “Status” button and selecting “Installed (local or obsolete)”. You can then go through the list and decide for each package whether you want to keep it or not.

Follow me on Identi.ca, Twitter and Facebook. Or subscribe to this blog by RSS or by email.

Debian Cleanup Tip #1: Get rid of useless configuration files

If you like to keep your place clean, you probably want to do the same with your computer. I’m going to show you a few tips over the next 4 weeks so that you can keep your Debian/Ubuntu system free of dust!

Over time the set of packages that is installed on your system changes, either because you install and remove stuff, or because the distribution evolved (and you upgraded your system to the latest version).

But the Debian packaging system is designed to keep configuration files when a package is removed. That way if you reinstall it, you won’t have to redo the configuration. That’s a nice feature but what if you will never reinstall those packages?

Then those configuration files become clutter that you would rather get rid of. In some cases, those files lying around might have unwanted side-effects (recent example: it can block the switch to a dependency-based boot sequence because obsolete init scripts without the required dependencies are still present).

The solution is to “purge” all packages which are in the “config-files” state. With aptitude you can do aptitude purge ~c (or aptitude purge ?config-files). Replace “purge” by “search” if you only want to see a list of the affected packages.

If you want a machine-friendly list of the packages in that state, you could use one of those commands (and then pass the result to apt-get if you don’t have aptitude available):

$ grep-status -n -sPackage -FStatus config-files
[...]
$ dpkg-query -f '${Package} ${Status}\n' -W | grep config-files$ | cut -d" " -f1
[...]

Note that grep-status is part of the dctrl-tools package.

Of course you can also use graphical package managers, like Synaptic. Click on the “Status” button on the bottom left, then on “Not installed (residual config)” and you have a list of packages that you can purge. You can select them all, right click and pick “Mark for Complete Removal”. See the screenshot below. The last step is to click on “Apply” to get the packages purged.

Synaptic purging residul config files

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

People behind Debian: Michael Vogt, synaptic and APT developer

Michael and his daughter Marie

Michael has been around for more than 10 years and has always contributed to the APT software family. He’s the author of the first real graphical interface to APT—synaptic. Since then he created “software-center” as part of his work for Ubuntu. Being the most experienced APT developer, he’s naturally the coordinator of the APT team. Check out what he has to say about APT’s possible evolutions.

My questions are in bold, the rest is by Michael.

Who are you?

My name is Michael Vogt, I’m married and have two little daughters. We live in Germany (near to Trier) and I work for Canonical as a software developer. I joined Debian as a developer in early 2000 and started to contribute to Ubuntu in 2004.

What’s your biggest achievement within Debian or Ubuntu?

I can not decide on a single one so I will just be a bit verbose.

From the very beginning I was interested in improving the package manager experience and the UI on top for our users. I’m proud of the work I did with synaptic. It was one of the earliest UIs on top of apt. Because of my work on synaptic I got into apt development as well and fixed bugs there and added new features. I still do most of the uploads here, but nowadays David Kalnischkies is the most active developer.

I also wrote a bunch of tools like gdebi, update-notifier, update-manager, unattended-upgrade and software-properties to make the update/install situation for the user easier to deal with. Most of the tools are written in python so I added a lot of improvements to python-apt along the way, including the initial high level “apt” interface and a bunch of missing low-level apt_pkg features. Julian Andres Klode made a big push in this area recently and thanks to his effort the bindings are fully complete now and have good documentation.

My most recent project is software-center. Its aim is to provide a UI strongly targeted for end-users. The goal of this project is to make finding and installing software easy and beautiful. We have a fantastic collection of software to offer and software-center tries to present it well (including screenshots, instant search results and soon ratings&reviews). This builds on great foundations like aptdaemon by Sebastian Heinlein, screenshots.debian.net by Christoph Haas, ddtp.debian.org by Michael Bramer, apt-xapian-index by Enrico Zini and many others (this is what I love about free software, it usually “adds”, rarely “takes away”).

What are your plans for Debian Wheezy?

For apt I would love to see a more plugable architecture for the acquire system. It would be nice to be able to make apt-get update (and the frontends that use this from libapt) be able to download additional data (like debtags or additional index file that contains more end-user targeted information). I also want to add some scripts so that apt (optionally) creates btrfs snapshots on upgrade and provide some easy way to rollback in case of problems.

There is also some interesting work going on around making the apt problem resolver a more plugable part. This way we should be able to do much faster development.

software-center will get ratings&reviews in the upstream branch, I really hope we can get that into Wheezy.

If you could spend all your time on Debian, what would you work on?

In that case I would start with a refactor of apt to make it more robust about ABI breaks. It would be possible to move much faster once this problem is solved (its not even hard, it just need to be done). Then I would add a more complete testsuite.

Another important problem to tackle is to make maintainer scripts more declarative. I triaged a lot of upgrade bug reports (mostly in ubuntu though) and a lot of them are caused by maintainer script failures. Worse is that depending on the error its really hard for the user to solve the problem. There is also a lot of code duplication. Having a central place that contains well tested code to do these jobs would be more robust. Triggers help us a lot here already, but I think there is still more room for improvement.

What’s the biggest problem of Debian?

That’s a hard question :) I mostly like Debian the way it is. What frustrated me in the past were flamewars that could have been avoided. To me being respectful to each other is important, I don’t like flames and insults because I like solving problems and fighting like this rarely helps that. The other attitude I don’t like is to blame people and complain instead of trying to help and be positive (the difference between “it sucks because it does not support $foo” instead of “it would be so helpful if we had $foo because it enables me to let me do $bar”).

For a long time, I had the feeling you were mostly alone working on APT and were just ensuring that it keeps working. Did you also had this feeling and are things better nowadays ?

I felt a bit alone sometimes :) That being said, there were great people like Eugene V. Lyubimkin and Otavio Salvador during my time who did do a lot of good work (especially at release crunch times) and helped me with the maintenance (but got interested in other area than apt later). And now we have the unstoppable David Kalnischkies and Julian Andres Klode.

Apt is too big for a single person, so I’m very happy that especially David is doing superb work on the day-to-day tasks and fixes (plus big project like multiarch and the important but not very thankful testsuite work). We talk about apt stuff almost daily, doing code reviews and discuss bugs. This makes the development process much more fun and healthy. Julian Andres Klode is doing interesting work around making the resolver more plugable and Christian Perrier is as tireless as always when it comes to the translations merging.

I did a quick grep over the bzr log output (including all branch merges) and count around ~4300 total commits (including all revisions of branches merged). Of that there ~950 commits from me plus an additional ~500 merges. It was more than just ensuring that it keeps working but I can see where this feeling comes from as I was never very verbose. Apt also was never my “only” project, I am involved in other upstream work like synaptic or update-manager or python-apt etc). This naturally reduced the time available to hack on apt and spend time doing the important day-to-day bug triage, response to mailing list messages etc.

One the python-apt side Julian Andres Klode did great work to improve the code and the documentation. It’s a really nice interface and if you need to do anything related to packages and love python I encourage you to try it. Its as simple as:

import apt
cache = apt.Cache()
cache["update-manager"].mark_install()
cache.commit()

Of course you can do much more with it (update-manager, software-center and lots of more tools use it). With “pydoc apt” you can get a good overview.

The apt team always welcomes contributors. We have a mailing list and a irc channel and it’s a great opportunity to solve real world problems. It does not matter if you want to help triage bugs or write documentation or write code, we welcome all contributors.

You’re also an Ubuntu developer employed by Canonical. Are you satisfied with the level of cooperation between both projects? What can we do to get Ubuntu to package new applications developed by Canonical directly in Debian?

Again a tricky question :) When it comes to cooperation there is always room for improvement. I think (with my Canonical hat on) we do a lot better than we did in the past. And it’s great to see the current DPL coming to Ubuntu events and talking about ways to improve the collaboration. One area that I feel that Debian would benefit is to be more positive about NMUs and shared source repositories (collab-maint and LowThresholdNmu are good steps here). The lower the cost is to push a patch/fix (e.g. via direct commit or upload) the more there will be.

When it comes to getting packages into Debian I think the best solution is to have a person in Debian as a point of contact to help with that. Usually the amount of work is pretty small as the software will have a debian/* dir already with useful stuff in it. But it helps me a lot to have someone doing the Debian uploads, responding to the bugmail etc (even if the bugmail is just forwarded as upstream bugreports :) IMO it is a great opportunity especially for new packagers as they will not have to do a lot of packaging work to get those apps into Debian. This model works very well for me for e.g. gdebi (where Luca Falavigna is really helpful on the Debian side).

Is there someone in Debian that you admire for his contributions?

There are many people I admire. Probably too many to mention them all. I always find it hard to single out individual people because the project as a whole can be so proud of their achievements.

The first name that comes to my mind is Jason Gunthorpe (the original apt author) who I’ve never met. The next is Daniel Burrows who I met and was inspired by. David Kalnischkies is doing great work on apt. From contributing his first (small) patch to being able to virtually fix any problem and adding big features like multiarch support in about a year. Sebastian Heinlein for aptdaemon.

Christian Perrier has always be one of my heroes because he cares so much about i18n. Christoph Haas for screenshots.debian.net, Michael Bramer for his work on debian translated package descriptions.


Thank you to Michael for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.