HOWTO – Page 2

Debian Cleanup Tip #3: get rid of third-party packages

February 14, 2011 by Raphaël Hertzog

Last week, we learned how to get rid of obsolete packages. This time, we’re going to learn how to bring back your computer to a state close to a “pure” Ubuntu/Debian installation.

Thanks to the power of APT, it’s easy to add new external repositories and install supplementary software. Unfortunately some of those are not very well maintained. They might contain crappy packages or they might simply not be updated. An external package which was initially working well, can become a burden on system maintenance because it will be interfering with regular updates (for example by requiring a package that should be removed in newer versions of the system).

So my goal for today is to teach you how to identify the packages on your system that are not coming from Debian or Ubuntu. So that you can go through them from time to time and keep only those that you really need. Obsolete packages are a subset of those, but I’ll leave them alone. We took care of them last week.

Each (well-formed) APT repository comes with a “Release” file describing it (example). They provide some values that can be used by APT to identify packages contained in the repository. All official Debian repositories are documented with Origin=Debian (and Origin=Ubuntu for Ubuntu). You can verify the origin value associated to each repository (if any) in the output of apt-cache policy:

[...]
 500 http://ftp.debian.org/debian/ lenny/main i386 Packages
     release v=5.0.8,o=Debian,a=stable,n=lenny,l=Debian,c=main
     origin ftp.debian.org
[...]

From there on, we can simply ask aptitude to compute a list of packages which are both installed and not available in an official Debian repository:

$ aptitude search '?narrow(?installed, !?origin(Debian))!?obsolete'
or
$ aptitude search '~S ~i !~ODebian !~o'

You can replace “search” with “purge” or “remove” if you want to get rid of all the packages listed. But you’re more likely to want to remove only a subset of carefully chosen packages… you’re probably still using some of the software that you installed from external repositories.

With synaptic, you can also browse the content of each repository. Click on the “Origin” button and you have a list of repositories. You can go through the non-Debian repositories and look which packages are installed and up-to-date.

But you can do better, you can create a custom view. Click on the menu entry “Settings > Filter”. Click on “New” to create a new filter and name it “External packages”. Unselect everything in the “Status” tab and keep only “Installed”.

Go in the “Properties” tab and here add a new entry “Origin” “Excludes” “ftp.debian.org”. In fact you must replace “ftp.debian.org” with the hostname of your Debian/Ubuntu mirror. The one that appears on the “origin” line in the output of apt-cache policy (see the excerpt quoted above in this article).

Note that the term “Origin” is used to refer to two different things, a field in the release file but also the name of the host for an APT repository. It’s a bit confusing if you don’t pay attention.

Close the filters window with OK. You now have a new listing of “External packages” under the “Custom Filters” screen. You can see which packages are installed and up-to-date and decide whether you really want to keep it. If the package is also provided by Debian/Ubuntu and you want to go back to the version provided by your distribution, you can use the “Package > Force version…” menu entry.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

Debian Cleanup Tip #2: Get rid of obsolete packages

February 7, 2011 by Raphaël Hertzog

Last week, we learned to remove useless configuration files. This week, we’re going to take care of obsolete packages.

An obsolete package is a package who is no longer provided by any of the APT repositories listed in /etc/apt/source.lists (and /etc/apt/sources.list.d/). There can be multiple reasons why a package is no longer available in the repository (or at least not under the same name) :

the upstream author stopped maintaining the software a long time ago, nobody else took over and the Debian maintainer preferred to remove the package from Debian. Usually there are alternatives in the Debian archive.
the package was orphaned in Debian since a long time, nobody took over and it had very few users. The Debian QA team might have asked its removal.
the latest version of the software might have been packaged under a new package name. Either because the amount of changes was so important that it was preferred to not upgrade automatically to the latest version (it has been the case with request-tracker and nagios, they both embed a version number in their package names), or simply because the maintainer wants to let the user install several versions at the same time (that’s the case for example with the Linux kernel, the python interpreter and many libraries).
the software has been renamed, the maintainer renamed the packages and kept transitional packages under the old name for one release. Then the transitional packages have been removed.

In any case, it’s never a good idea to keep obsolete packages around: they do not benefit from security updates and they might cause problems during upgrades if they depend on other packages that should be removed to complete the upgrade.

You could blindly remove them with aptitude purge ~o (or aptitude purge ?obsolete) but you might want to first verify what those package are. There might be some packages that you have manually installed, that are not part of any current APT repository, and that you want to keep around nevertheless (I have skype, dropbox and a few personal packages for example). You can get the list with aptitude search ?obsolete…

With the graphical package manager (Synaptic), you can find the list of obsolete packages by clicking on the “Status” button and selecting “Installed (local or obsolete)”. You can then go through the list and decide for each package whether you want to keep it or not.

Follow me on Identi.ca, Twitter and Facebook. Or subscribe to this blog by RSS or by email.

Debian Cleanup Tip #1: Get rid of useless configuration files

January 31, 2011 by Raphaël Hertzog

If you like to keep your place clean, you probably want to do the same with your computer. I’m going to show you a few tips over the next 4 weeks so that you can keep your Debian/Ubuntu system free of dust!

Over time the set of packages that is installed on your system changes, either because you install and remove stuff, or because the distribution evolved (and you upgraded your system to the latest version).

But the Debian packaging system is designed to keep configuration files when a package is removed. That way if you reinstall it, you won’t have to redo the configuration. That’s a nice feature but what if you will never reinstall those packages?

Then those configuration files become clutter that you would rather get rid of. In some cases, those files lying around might have unwanted side-effects (recent example: it can block the switch to a dependency-based boot sequence because obsolete init scripts without the required dependencies are still present).

The solution is to “purge” all packages which are in the “config-files” state. With aptitude you can do aptitude purge ~c (or aptitude purge ?config-files). Replace “purge” by “search” if you only want to see a list of the affected packages.

If you want a machine-friendly list of the packages in that state, you could use one of those commands (and then pass the result to apt-get if you don’t have aptitude available):

$ grep-status -n -sPackage -FStatus config-files
[...]
$ dpkg-query -f '${Package} ${Status}\n' -W | grep config-files$ | cut -d" " -f1
[...]

Note that grep-status is part of the dctrl-tools package.

Of course you can also use graphical package managers, like Synaptic. Click on the “Status” button on the bottom left, then on “Not installed (residual config)” and you have a list of packages that you can purge. You can select them all, right click and pick “Mark for Complete Removal”. See the screenshot below. The last step is to click on “Apply” to get the packages purged.

Synaptic purging residul config files

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

3 ways to not clutter your Debian source package with autogenerated files

January 28, 2011 by Raphaël Hertzog

It’s quite common that the upstream build system generates/updates some files but does not clean them up properly when you call make clean. In that case, when you rebuild the package a second time in the same tree, the generated Debian source package will contain those changes.

You usually don’t want those changes. They make your package harder to review because they contain unneeded modifications (either directly in the .diff.gz with the old source format, or in a new patch in debian/patches/debian-changes-<ver> with the “3.0 (quilt)” source format).

I’ll show you 3 ways to avoid this problem. They are all workarounds, the proper fix would be to improve the upstream build system to really clean up the generated files. This is usually possible for files that are “created”, but it’s much more cumbersome for files that are “updated” (you would have to keep a backup of the original file so that you can restore it).

The traditional fix

Instead of relying on the upstream build system to do the work, we modify the clean target in debian/rules to remove the files that are left-over. Since “debian/rules clean” is always called before a source package is built, those generated files are not included as changes compared to what upstream provided.

A common work-around: always build from a clean state

As you have noted, the problem only happens when you build (source and binaries) twice in a row in the same tree. Some VCS-helper tools always build the Debian package in a temporary tree which is exported from the VCS. This is the case of svn-buildpackage by default and of git-buildpackage if you use its --git-export-dir option.

I don’t like this solution because it solves the problem only for the maintainer. Anyone else who is working on top of the package without using the same VCS-helper tool would be affected by the problem.

A new way to avoid the problem

Since it’s now possible to store dpkg-source options in the source package itself, we can conveniently have everybody use the --extend-diff-ignore option. It tells dpkg-source to ignore some files when checking whether we have made changes to upstream files.

For example if you want to ignore changes made on the files “config.sub”, “config.guess” and “Makefile” you could put this in debian/source/options:

# Don't store changes on autogenerated files
extend-diff-ignore = "(^|/)(config\.sub|config\.guess|Makefile)$"

You need to know a bit about Perl regular expressions since that’s what is used by dpkg-source to match the filenames to exclude.

Note that this approach always works, even when you can’t remove the file. So it saves you having to make a backup of the unmodified file just to be able to restore it before the next build.

Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.

The traditional fix

A common work-around: always build from a clean state

A new way to avoid the problem

Tags

Recent Posts