Last week, we learned how to get rid of obsolete packages. This time, we’re going to learn how to bring back your computer to a state close to a “pure” Ubuntu/Debian installation.
Thanks to the power of APT, it’s easy to add new external repositories and install supplementary software. Unfortunately some of those are not very well maintained. They might contain crappy packages or they might simply not be updated. An external package which was initially working well, can become a burden on system maintenance because it will be interfering with regular updates (for example by requiring a package that should be removed in newer versions of the system).
So my goal for today is to teach you how to identify the packages on your system that are not coming from Debian or Ubuntu. So that you can go through them from time to time and keep only those that you really need. Obsolete packages are a subset of those, but I’ll leave them alone. We took care of them last week.
Each (well-formed) APT repository comes with a “Release” file describing it (example). They provide some values that can be used by APT to identify packages contained in the repository. All official Debian repositories are documented with Origin=Debian (and Origin=Ubuntu for Ubuntu). You can verify the origin value associated to each repository (if any) in the output of apt-cache policy:
[...]
500 http://ftp.debian.org/debian/ lenny/main i386 Packages
release v=5.0.8,o=Debian,a=stable,n=lenny,l=Debian,c=main
origin ftp.debian.org
[...]
From there on, we can simply ask aptitude to compute a list of packages which are both installed and not available in an official Debian repository:
$ aptitude search '?narrow(?installed, !?origin(Debian))!?obsolete' or $ aptitude search '~S ~i !~ODebian !~o'
You can replace “search” with “purge” or “remove” if you want to get rid of all the packages listed. But you’re more likely to want to remove only a subset of carefully chosen packages… you’re probably still using some of the software that you installed from external repositories.
With synaptic, you can also browse the content of each repository. Click on the “Origin” button and you have a list of repositories. You can go through the non-Debian repositories and look which packages are installed and up-to-date.
But you can do better, you can create a custom view. Click on the menu entry “Settings > Filter”. Click on “New” to create a new filter and name it “External packages”. Unselect everything in the “Status” tab and keep only “Installed”.
Go in the “Properties” tab and here add a new entry “Origin” “Excludes” “ftp.debian.org”. In fact you must replace “ftp.debian.org” with the hostname of your Debian/Ubuntu mirror. The one that appears on the “origin” line in the output of apt-cache policy (see the excerpt quoted above in this article).
Note that the term “Origin” is used to refer to two different things, a field in the release file but also the name of the host for an APT repository. It’s a bit confusing if you don’t pay attention.
Close the filters window with OK. You now have a new listing of “External packages” under the “Custom Filters” screen. You can see which packages are installed and up-to-date and decide whether you really want to keep it. If the package is also provided by Debian/Ubuntu and you want to go back to the version provided by your distribution, you can use the “Package > Force version…” menu entry.
Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.
On a related note, aptitude can also check for packages which exist in Debian but not with the version you have: ?not(?narrow(?version(CANDIDATE), ?archive(unstable)))
Replace “unstable” with “testing” or “stable” if you use those. This will also show you any packages you have installed from experimental; if you want to exclude those too, repeat the whole stanza once for each distribution you use.
Or rather, if you want to find all installed packages which are not coming from the main Debian repository that are you’re tracking, you can use this (example with “testing” as the main repo):
Beware it will show any package that is installed and that doesn’t match the version available in testing, including packages that you have not yet upgraded but also packages that you might have grabbed elsewhere (sid, experimental, etc.)
Great articles. Thanks a lot.
If you dont mind i have a request.
Could you do a round up of all Debian packaging related utilities you can think of when you’re done with this series? There are a lot of them in the repositories and its very hard to keep track. For example utilites like dctrl-tools you mentioned in the first article, deborphan and useful utilities like those? Maybe even a series with examples of usage for each one. Thanks again!
Thanks for the feedback Greg, I have noted your suggestion and I might do it at some point.
Elegant and very useful, thanks!
Wonderful, once again.
Thanks so much!
I usually use a combination of deborphan and
apt-show-versions |grep -v testingfor the same purpose. Works good enough for testing-based installation + a few packages from unstable and experimental + a couple from external sources, and much easier to remember 😉BTW, Raphaël, on my system, your aptitude search command actually returns a list of manually recompiled official packages, and I definitely would not want to do a purge for them 😉
Very interesting again Raphaël, but while reading through … this time … something regarding to repositories came to my mind. I have three different Ubuntu instalations, each of them with around 20 (yes twenty) external repositories defined. The thing is that every six months, after each version upgrade, I have to redefine them all to properly work … around 60 redefinitions … my godness !!!
I think that one of your future tips (say tip number x) could be related to this issue and explain us a way to mitigate this lack of repository definitions’ flexibility after an upgrade.
Thank you and regards.
Hi Ana, I’m not sure I understand your request… it’s easy to edit /etc/apt/sources.list and do a search & replace on the distribution name for example.
It might not be so convenient in the graphical interface however. 🙂
But using 20 external repositories is a lot, are they really needed? It’s a failure of Debian/Ubuntu if so many useful applications are missing from the main repositories (or from backports if you need newer versions).
Well … it is more related to ‘up to date’ applications than to missing applications.
Some of the repositories doesn’t hold just one updated application but many dozens of them. So that you define some very useful and well known repositories … and in some way you’re granting having last stable updates from many apps … it’s so nice …
Other repos are just upstream ppas … which are very useful too. One example (there’re lots) is … if you want having last Shutter 0.87 … you define (I do it through GUI) its ppa (ppa:shutter/ppa) … and you will have the last stable software. But coming back to my initial request, these ppa definitions turn unusable after an Ubuntu version upgrade. I have to delete them all and redefine them from the scratch (60 deletions and 60 definitions … 120 updates … things are getting worse 😉
You suggested to edit /etc/apt/sources.list … but I can find there only the ‘oficial’ repos. I don’t know where the ‘manual added’ repository definitions are kept … so 120 updates … using GUI … ONE AT A TIME … … … yes … it’s a bit frustrating …
Look into /etc/apt/sources.list.d/ there’s probably one file with your repositories.
I know it’s nice to have the latest versions, but unfortunately it’s also a source of problems because you’re using unofficial packages and upgrades are only tested with the official packages. If you encounter problems, you’re then on your own. So it’s a trade-off to consider…
Yes … it seems they are here (/etc/apt/sources.list.d/) … they are 48 elements whose names are blablabla-bla-maverick.list and blablabla-bla-maverick.list.save (and whose content also points to maverick) … I guess that’s why they will turn unusable when I migrate from maverick to natty.
You’re right when you say there’s a trade-off … in my particular case, my ONLY drawback so far is the 120 manual updates every single semester.
From an end-user point of view … yes , I understand the trade-off you mean, but … even though this trade-off, I have been given the opportunity (and the power) to choose … using ‘apt’ utility … and this is great … there’s a trade-off but you still have the chance to choose. And … in my particular case … I am NOT having neither consistency nor logic problems with these ‘manually added’ software … my problem is structural … the freedom ‘apt’ utility is giving me … is broken when I am upgrading version release.
I appretiate the freedom ‘apt’ utility is giving me (to be ‘out of the law’) … and I would like, in the future … one day … this freedom could be kept even while upgrading version releases.
Thank you and regards Raphaël.
It looks like this is the only one of the Debian Cleanup Series that hasn’t the HOWTO tag, there maybe a reason, but just wanted to mention it in case you forgot it.
Fixed, thanks.