apt-get install debian-wizard

5 reasons why Debian Unstable does not deserve its name

December 20, 2010 by Raphaël Hertzog

Debian Unstable (also known as sid) is one of the 3 distributions that Debian provides (along with Stable and Testing).

It’s not conceived as a product for end-users, instead it’s the place where contributors are uploading newer packages. Daily. Yes that means that Unstable is a quickly moving target and it’s not for everybody. But you can use it and your computer won’t explode.

1. It contains mainly stable versions of the software

Yes, you read it right. Unstable is not full of development versions of the various software. It happens on some software but then it’s usually a conscious decision of the maintainer who believes that this specific version is already better than the previous one.

The packages in sid are supposed to migrate to testing, the place where the next Debian stable release is prepared. So maintainers are advised to only upload stuff that is of release quality, the rest should be uploaded to experimental instead.

2. It doesn’t break badly every other day

Breakages happen but they are not a big deal usually. It has been long time since I could not reboot my computer after an upgrade or since the graphical interface was no longer working. The kind of breakages that you have is that one software stops working, or triggers an annoying bug, or that a few packages are uninstallable.

In most cases, you can save yourself by downgrading to the version available in Testing. Or by finding a work-around in the bug tracking system. Or by not upgrading because you have apt-listbugs installed and you have been warned about the problem.

3. It’s the basis of other distributions

If Debian Unstable was really so bad, it would not be a good basis to build a derivative distribution, isn’t it? But Ubuntu and ~~Sidux~~Aptosid (to name only two) are based on Debian Sid.

4. It’s not inherently less secure than Stable or Testing

High impact security vulnerabilities will usually be quickly fixed in Stable and Unstable. The stable upload is done by the security team while the unstable one is made by the maintainer. Testing will usually get the fix through the package uploaded to Unstable, so testing users get security updates with a delay.

For less serious vulnerabilities, it’s entirely possible that stable does not get any update at all. In that case, unstable/testing users are better served since they will get the fix with the next upstream version anyway.

Of course, it happens that maintainers are busy or that something falls through the cracks, but there are other people watching RC bugs who will fix this if the maintainer doesn’t react at all.

5. I use it on my main computer

And many other people do the same. And you can do the same if you meet the criteria below:

you can work on the command-line (enough to downgrade a problematic package, to edit configuration files, etc.);
you know how to work with APT and multiple distributions in /etc/apt/sources.list;
you are able to read/write English so that you can read/file bug reports when needed;
you have another computer connected to the Internet that you can use to lookup documentation (or the bug tracking system, or the support mailing lists) when your usual computer is off-line for a reason that you don’t understand.

If you feel you are not ready for the jump, click here to subscribe to this blog (or here via the RSS feed), I’ll surely teach some of the required skills in future articles.

PS: All that said, if you have a working sid installation, do not upgrade it just before an important presentation, or before a trip. It will always break at the most annoying time. Unless you like to live dangerously, of course.

Avoid a newbie packager mistake: don’t build your Debian packages with dpkg -b

December 17, 2010 by Raphaël Hertzog

In the last years, I have seen many people try to use dpkg --build to create Debian packages. Indeed, if you look up dpkg’s and dpkg-deb‘s manual pages, this option seems to be what you have to use:

-b, --build directory [archive|directory]

Creates a debian archive from the filesystem tree stored in directory. directory must have a DEBIAN subdirectory, which contains the control information files such as the control file itself. This directory will not appear in the binary package’s filesystem archive, but instead the files in it will be put in the binary package’s control information area.

And indeed, dpkg-deb is what ultimately creates the .deb files (aka binary packages). But it’s a low-level tool that you should not call yourself. If you want to properly package a new software, you should rather create a Debian source package that will transform upstream source code into policy-compliant binary packages.

Creating a source package also involves preparing a directory tree (but with a “debian” sub-directory), it’s probably more complicated than calling dpkg -b on a manually crafted directory. But the result is much more versatile: the tools used bring value by dynamically analyzing/modifying the files within your package (for example, the dependencies on C libraries that your package needs are automatically inserted).

If this is news to you, you might want to check out the New Maintainers’ Guide and the Debian Policy.

Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.

Howto to rebuild Debian packages

December 15, 2010 by Raphaël Hertzog

Being able to rebuild an existing Debian package is a very useful skill. It’s a prerequisite for many tasks that an admin might want to perform at some point: enable a feature that is disabled in the official Debian package, rebuild a source package for another suite (for example build a Debian Testing package for use on Debian Stable, we call that backporting), include a bug fix that upstream developers prepared, etc. Discover the 4 steps to rebuild a Debian package.

1. Download the source package

The preferred way to download source packages is to use APT. It can download them from the source repositories that you have configured in /etc/apt/sources.list, for example:

deb-src http://ftp.debian.org/debian unstable main contrib non-free
deb-src http://ftp.debian.org/debian testing main contrib non-free
deb-src http://ftp.debian.org/debian stable main contrib non-free

Note that the lines start with “deb-src” instead of the usual “deb”. This tells APT that we are interested in the source packages and not in the binary packages.

After an apt-get update you can use apt-get source publican to retrieve the latest version of the source package “publican”. You can also indicate the distribution where the source package must be fetched with the syntax “package/distribution“. apt-get source publican/testing will grab the source package publican in the testing distribution and extract it in the current directory (with dpkg-source -x, thus you need to have installed the dpkg-dev package).

$ apt-get source publican/testing
Reading package lists... Done
Building dependency tree       
Reading state information... Done
NOTICE: 'publican' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/collab-maint/publican.git
Need to get 727 kB of source archives.
Get:1 http://nas/debian/ squeeze/main publican 2.1-2 (dsc) [2253 B]
Get:2 http://nas/debian/ squeeze/main publican 2.1-2 (tar) [720 kB]
Get:3 http://nas/debian/ squeeze/main publican 2.1-2 (diff) [4728 B]
Fetched 727 kB in 0s (2970 kB/s)  
dpkg-source: info: extracting publican in publican-2.1
dpkg-source: info: unpacking publican_2.1.orig.tar.gz
dpkg-source: info: unpacking publican_2.1-2.debian.tar.gz
$ ls -dF publican*
publican-2.1/                 publican_2.1-2.dsc
publican_2.1-2.debian.tar.gz  publican_2.1.orig.tar.gz

If you don’t want to use APT, or if the source package is not hosted in an APT source repository, you can download a complete source package with dget -u dsc-url where dsc-url is the URL of the .dsc file representing the source package. dget is provided by the devscripts package. Note that the -u option means that the origin of the source package is not verified before extraction.

2. Install the build-dependencies

Again APT can do the grunt work for you, you just have to use apt-get build-dep foo to install the build-dependencies for the last version of the source package foo. It supports the same syntactic sugar than apt-get source so that you can run apt-get build-dep publican/testing to install the build-dependencies required to build the testing version of the publican source package.

If you can’t use APT for this, enter the directory where the source package has been unpacked and run dpkg-checkbuilddeps. It will spit out a list of unmet build dependencies (if there are any, otherwise it will print nothing and you can go ahead safely). With a bit of copy and paste and a “apt-get install” invocation, you’ll install the required packages in a few seconds.

3. Do whatever changes you need

I won’t detail this step since it depends on your specific goal with the rebuild. You might have to edit debian/rules, or to apply a patch.

But one thing is sure, if you have made any change or have recompiled the package in a different environment, you should really change its version number. You can do this with “dch --local foo” (again from the devscripts package), replace “foo” by a short name identifying you as the supplier of the updated version. It will update debian/changelog and invite you to write a small entry documenting your change.

4. Build the package

The last step is also the simplest one now that everything is in place. You must be in the directory of the unpacked source package.
Now run either “debuild -us -uc” (recommended, requires the devscripts package) or directly “dpkg-buildpackage -us -uc”. The “-us -uc” options avoid the signature step in the build process that would generate a (harmless) failure at the end if you have no GPG key matching the name entered in the top entry of the Debian changelog.

$ cd publican-2.1
$ debuild -us -uc
 dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): 
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor): 
dpkg-buildpackage: source package publican
dpkg-buildpackage: source version 2.1-2rh1
dpkg-buildpackage: source changed by Raphaël Hertzog 
 dpkg-source --before-build publican-2.1
dpkg-buildpackage: host architecture i386
[...]
dpkg-deb: building package `publican' in `../publican_2.1-2rh1_all.deb'.
 dpkg-genchanges  >../publican_2.1-2rh1_i386.changes
dpkg-genchanges: not including original source code in upload
 dpkg-source --after-build publican-2.1
dpkg-buildpackage: binary and diff upload (original source NOT included)
Now running lintian...
Finished running lintian.

The build is over, the updated source and binary packages have been generated in the parent directory.

$ cd ..
$ ls -dF publican*
publican-2.1/                    publican_2.1-2rh1.dsc
publican_2.1-2.debian.tar.gz     publican_2.1-2rh1_i386.changes
publican_2.1-2.dsc               publican_2.1-2rh1_source.changes
publican_2.1-2rh1_all.deb        publican_2.1.orig.tar.gz
publican_2.1-2rh1.debian.tar.gz

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

People behind Debian: David Kalnischkies, an APT developer

December 10, 2010 by Raphaël Hertzog

The two first interviews were dedicated to long-time Debian developers. This time I took the opposite approach, I interviewed David Kalnischkies who is not (yet) a Debian developer. But he’s contributing to one of the most important software within Debian—the APT package manager—since 2009. You can already see him in many places in Debian sharing his APT knowledge when needed.

English is not his native language and he’s a bit shy, but he accepted the interview nevertheless. I would like to thank him for the efforts involved and I hope his story can inspire some people to take the leap and just start helping… My questions are in bold, the rest is by David.

Who are you?

I am David Kalnischkies, 22 years old, living in the small town Erbach near Wiesbaden in Germany and I’m studying computer science at the TU Darmstadt. Furthermore I am—for more than half a decade now—young group leader of my hometown.

I never intended to get into this position, but it has similarities with my “career” in this internet-thingy here. I don’t remember why, but in April 2009 I was at a stage that some simple bugs in APT annoyed me so much that I grabbed the source, and most importantly—I don’t know why I did it— but I published my changes in Mai with #433007, a few more bugs and even a branch on launchpad. And this public branch got me into all this trouble in June: I got a mail from “Mr. package managment” Michael Vogt regarding this branch…

A few days later I joined an IRC session with him and closely after that my name appeared for the first time in a changelog entry. It’s a strange but also addicting feeling to read your own name in an unfamiliar place. And even now after many IRC discussions, bugfixes and features, three Ubuntu Developer Summits and a Google Summer of Code in Debian, my name still appear in places I have never even thought about—e.g. in an interview.

What’s your biggest achievement within Debian?

I would like to answer “MultiArch in APT” as it was my Google Summer of Code project, but as it has (not much) use for the normal user at this point—will hopefully change for wheezy—I chose three smaller things in squeeze’s APT that many people don’t even know yet:

“MMap run out of room”, the most famous error related to the APT::Cache-Limit option is gone. The DynamicMMap in APT now deserves its name 🙂
Download and usage of multiple Translations of Descriptions
crazy stuff like apt-cache show ^apt-.*$/experimental works

If your impression is now that I only do APT stuff: that’s completely right, but that’s already more than enough for me for now as the toolchain behind the short name APT contains so many tools and use cases that you always have something different.

You’re an active member of the APT development team. Are there plans for APT in Debian Wheezy? What features can we expect?

That’s very hard to answer, as the team is too small to be able to really plan something. I mean, you can have fancy plans and everything and half a second later someone arrives on the mailing list with a “small” question which eats days of development time just for debugging…

But right now the TODO list contains (in no particular order):

MultiArch – managing packages built for different architectures like i386 on amd64 to ease the usage of packages which only work on specific architectures or cross-compiling.
rewriting the subsystem responsible for downloading files to e.g. enable tools like apt-file or debtags to reuse the infrastructure by plugging into it.
splitting long descriptions out of the Packages file to possibly decrease the size to download on an “apt-get update”. Includes mostly pulling the button only to enable it by the archive masters
backporting popular features of some frontends to enable all frontends to use them, e.g. “apt-get changelog” or “apt-get download”
support research on resolver improvements by implementing support for a distribution independent upgrade problem format (CUDF).
an additional single binary for users wrapping apt-get, apt-cache and co.
getting in contact (again) with various forks like apt-rpm and derivatives like maemo or telesphoreo to join forces instead of splitting them by working on different (and possibily very old) heads of the dragon

We will see what will get real for wheezy and what is postponed, but one thing is sure: more will be done for wheezy if you help!

If you could spend all your time on Debian, what would you work on?

I would spend it on APT’s debbugs count… zero would be cool to look at! We make progress in this regard, but with the current velocity we will reach it in ten years or so.

Reading more mailing lists would be interesting, as I am kind of an information junky. Maintaining a package could be interesting to share the annoyance of a maintainer with handcrafted dependencies just to notice that APT doesn’t get it in the way I intended it to be. Through, to make it feel real I need to train a few new APT contributors before so they can point my mistake out, but this unfortunately doesn’t depend so much on time but on victims… Maybe I could even be working on getting an official status.

Beside that, I would love to be able to “apt-get dist-upgrade” the increasing mass of systems I and many others carry around in their pockets. In regards to my phone, this is already fixed, but there is much room for improvements.

What’s the biggest problem of Debian?

You need to be lucky. You need to talk at the right time to the right person. That’s not really a debian-only problem as such, but in a global project full of volunteers you can see it clearly as there are plenty of opportunities to be unlucky.

For example, it’s unlikely that an interview would be made with me now if Michael had not contacted me in June 2009. In a big project like Debian, you are basically completely lost without a mentor guiding you, so things like the debian-mentors list are good projects, but I am pretty certain they could benefit from some more helping hands.

The other thing which I consider a problem is that—and I read from time to time—some people don’t care for translations. That’s bad. Yes, a developer is able to read English, otherwise s/he couldn’t write code or participate on the mailinglists.

Still, I personally prefer to use a translated application if I have the chance as it’s simply easier for me to read in my mother tongue, not only because I am dyslexic, but because my mind still thinks in German and not in English. Yes, I could personally fix that by thinking in English only from now on, but its a quite big problem to convince my family—which is not really familiar with tech-stuff—to use something if they can’t understand what is written on screen.

It was hard enough to tell my mother how to write an SMS in a German interface. My phone with English words all over the place would be completely unusable for her—despite the fact that my phone is powered by Debian and better for the task from a technical point of view.

You are not yet an official Debian developer/maintainer, but you’re already perceived in the community as one the most knowledgeable person about APT. It’s a great start! What’s your advice to other people who want to start contributing to Debian in general, and to APT in particular?

It was never a goal in my life to “start contributing”. My goal was and still is to make my life easier by letting the computer work for me. At some point APT hindered the success of this goal, so it needed to be fixed. I didn’t expect to open pandora’s box.

So, my advice is simple: Just start. Ignore the warning signs telling you that this is not easy. They are just telling you that you do something useful. Only artificial problems are easy. Further more, contribution to APT, dpkg or any other existing package is in no way harder than opening an ITP and working on your own, and it’s cooler as you have a similar minded team around you to talk to. 🙂

“APT didn’t accept release codenames as target release” was one of the first things I fixed. If I had asked someone if that would be a good starting point the answer would have been a clear “no”, but I didn’t search for a good starting point…

As a kid I can start playing football by just walking on the field and play or I can sit near the field, watching the others play, while analyzing which position would be the best for me to start ruling out one by one as the technical requirements seem too high… “Oh – bicycle kick – that sounds complicated… I can’t do that”

Julian Andreas Klode is working on a APT replacement, there’s also Cupt by Eugene V. Lyubimkin. Both projects started because their authors are not satisfied with APT, they find APT’s code difficult to hack partly due to the usage of C++. Do you share their concerns and what’s your opinion on those projects?

I don’t think C++ is a concern in this regard, after all cupt is currently rewritten to C++0x and APT2 started in vala and is now C + glib—last time I checked at least. I personally think that something is wrong if we need to advertise an application by saying in which language it is written…

The major “problem” for APT is probably that the code is “old”: APT does its job for more than 12 years now, under different maintainers with an always changing environment around it: so there are lines in APT which date from a time when nobody knew what a “Breaks” dependency is, that packages can have long descriptions which can be translated or even that package archives can be signed with a gpg key! And yet we take all those for granted today. APT has proven to adapt to these changes in the environment and became in this process very popular. So I don’t think the point is near (if it will come at all) that APT can go into retirement as it is completely replaced by something else.

The competitors one the other hand have their first 12 years still to go. And it will be interesting to see how they will evolve and what will be the state of the art in 2022…

But you asked what I think about the competitors: I prefer the “revolution from inside” simply because I can see effects faster as more users will profit from it now. Cupt and co. obviously prefer the “normal” revolution. The goal is the same, creating the best package manager tool, but the chosen way to the goal is different. aptitude and cupt have an interactive resolver for example: that’s something I dislike personally, for others that is the ultimate killer feature. cupt reading the same preference file as APT will have a different pinning result, which we should consider each time someone mentions the word “drop-in replacement”.

APT2 isn’t much more than the name—which I completely dislike—currently from a user point of view, so I can’t really comment on that. All of them make me sad as each line invested in boilerplate code like configuration file parsing would be in my eyes better be spent in a bugfix or new feature instead, but I am not here to tell anyone what they should do in their free time…

But frankly, I don’t see them really as competitors: I use the tools I use, if other do that too that’s good, if not that’s their problem. 🙂 The thing that annoys me really are claims like “plan is to remove APT by 2014” as this generates a “vi vs. emacs” like atmosphere we don’t need. If some people really think emacs is a good editor… who cares?

I really hope we all can drink a beer in 2022 in Milliways, the restaurant at the end of the package universe, remembering the good old 2010… 😉

Is there someone in Debian that you admire for his contributions?

No, not one, many!

Michael Vogt who has nearly the monopole of “package manager maintainer” by being upstream of APT, synaptics and software center to name only the biggest and still has the time to answer even the dumbest of my questions. 🙂 Jason Gunthorpe for being one of the initial developers behind “deity” who I will probably never meet in person beside in old comments and commit logs. Christian Perrier for caring so much about translations. Obey Arthur Liu as a great admin for Debian’s participation in Google’s Summer of Code. Paul Wise for doing countless reviews on debian-mentors which are a good source of information—not only for the maintainer of the package under review.

I guess I need to stop here because you asked for just one. So let’s end with some big words instead: I am just a little cog in the big debian wheel…

Thank you to David Kalnischkies for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.