How Ubuntu builds up on Debian

I have been asked how Ubuntu relates to Debian, and how packages flow from one to the other. So here’s my attempt at clarifying the whole picture.

Where do the packages come from?

Most packages are created by Debian contributors and they are uploaded in Debian unstable (or Debian experimental). New packages are reviewed by the Debian ftpmasters before being accepted in the official archive. The packages are held in the NEW queue until the review is over, and the time spent there varies between a few hours and a few months (usually they are processed within one week or two).

Ubuntu imports all the official Debian packages, but they also add some packages of their own. About 7% of the Ubuntu packages are third-party software that have been packaged for Ubuntu but not for Debian.

What are the changes made by Ubuntu?

From all the source packages coming from Debian, 17% have additional changes made by Ubuntu. Many of them are part of the “main” repository, which is actively maintained by Canonical and Ubuntu core developers. The “universe” repository is usually closer to the official Debian packages.

Many of the changes made by Ubuntu are the results of the decisions taken during the Ubuntu Developer Summit in order to reach specific goals: provide a better user interface, offer faster boot times, become a better platform for third-party software developers, offer a good integration with their online services (Launchpad, Ubuntu One), etc. Other changes are simply the result of fixing bugs reported by Ubuntu users.

Note that even non-modified source packages will result in different binary packages for Ubuntu. That’s because Ubuntu has made changes to the build environment. They only support Intel-based computers with a 686-class (or newer) CPU, they enable some compiler options that Debian doesn’t, etc. And all binary packages are modified by a program called pkgbinarymangler.

Ubuntu’s release cycle and the relation with Debian

Ubuntu releases every 6 months (that’s what time based releases is about). Debian has a very different schedule. How does Ubuntu manage to reuse Debian’s work?

Ubuntu imports packages from Debian unstable (even experimental sometimes) to get the newest packages. If the Ubuntu package already has Ubuntu-specific changes, they merge their changes in the updated Debian package. Otherwise the Debian package is simply grabbed and rebuilt in Ubuntu. This works well because Debian unstable is much more usable than the name suggests. And this process only goes on during the first 2 months of the cycle (until the Debian Import Freeze), so there’s plenty of time afterward to fix the biggest problems.

In the third and fourth month, it’s still possible to pick updated packages from Debian but it must be requested by a developer, it won’t be done automatically. At the end of the fourth month, the feature freeze is put in place.

The 2 months left are dedicated to bug fixing and polishing the distribution. There are various sub-freezes that happen in this period, you can check the Natty release schedule as an example. Picking updated packages from Debian is now the exception, it will only be allowed if the update on the Debian side is a bug-fix only release.

Credits: some figures taken from a talk of Lucas Nussbaum, they were collected based on the packages available in the Lucid Lynx release of Ubuntu.

Click here to subscribe to my newsletter and get my monthly update on what’s going on in Debian and Ubuntu.

4 tips to maintain a “3.0 (quilt)” Debian source package in a VCS

Most Debian packages are managed with a version control system (VCS) like git, subversion, bazaar or mercurial. The particularities of the 3.0 (quilt) source format are not without consequences in terms of integration with the VCS. I’ll give you some tips to have a smoother experience.

All the samples given in the article assume that you use git as version control system.

1. Add .pc to the VCS ignore list

.pc is the directory used by quilt to store its internal data (list of applied patches, backup of modified files). It’s also created by dpkg-source so that quilt knows that the patches are in debian/patches (and not in patches which is the default directory used by quilt). For that reason, the directory is kept even if you unapply all the patches.

However you don’t want to store this directory in your repository, so it’s best to put it in the VCS ignore list. With git you simply do:

$ echo ".pc" >>.gitignore
$ git add .gitignore
$ git commit -m "Ignore quilt dir"

The .gitignore file is ignored by dpkg-source, so you’re not adding any noise to the generated source package.

2. Unapply patches after the build

If you store upstream sources with non-applied patches (most people do), and if you don’t build packages in a temporary build directory, then you probably want to unapply the patches after the build so that your repository is again in a clean status.

This is now the default since dpkg-source will unapply any patch that it had to apply by itself. Thus if you start the build with a clean tree, you’ll end up with a clean tree.

But you can still force dpkg-source to unapply patches by adding “unapply-patches” to debian/source/local-options:

$ echo "unapply-patches" >>debian/source/local-options
$ git add debian/source/local-options
$ git commit -m "Unapply patches after build"

svn-buildpackage always builds in a temporary directory so the repository is left exactly like it was before the build, this option is thus useless. git-buildpackage can also be told to build in a temporary directory with --git-export-dir=../build-area/ (the directory ../build-area/ is the one used by svn-buildpackage, so this option makes git-buildpackage behave like svn-buildpackage in that respect).

3. Manage your quilt patches as a git branch

Instead of using quilt to manage the Debian-specific patches, it’s possible to use git itself. git-buildpackage comes with gbp-pq (“Git-BuildPackage Patch Queue”): it can export the quilt serie in a git branch that you can manipulate like you want. Each commit represents a patch, so you want to rebase that branch to edit intermediary commits. Check out the upstream documentation of this tool to learn how to work with it.

There’s an alternative tool as well: git-dpm. Its website explains the principle very well. It’s a more complicated than gbp-pq but it has the advantage of keeping the history of all branches used to generate the quilt series of all Debian releases. You might want to read a review made by Sam Hartman, it explains the limits of this tool.

4. Document how to review the changes

One of the main benefit of this new source format is that it’s easy to review changes because upstream changes are kept as separate patches properly documented (ideally using the DEP-3 format). With the tools above, the commit message becomes the patch header. Thus it’s important to write meaningful commit messages.

This works well as long as your workflow considers the Debian patches as a branch that you rebase on top of the upstream sources at each release. Some maintainers don’t like this workflow and prefer to have the Debian changes applied directly in the packaging branch. They switch to a new upstream version by merging it in their packaging branch. In that case, it’s difficult to generate a quilt serie out of the VCS. Instead, you should instruct dpkg-source to store all the changes in a single patch (which is then similar to the good old .diff.gz) and document in the header of that patch how the changes can be better reviewed, for example in the VCS web interface. You do the former with the --single-debian-patch option and the latter by writing the header in debian/source/patch-header:

$ echo "single-debian-patch" >> debian/source/local-options
$ cat >debian/source/patch-header <<END
This patch contains all the Debian-specific
changes mixed together. To review them
separately, please inspect the VCS history
at http://git.debian.org/?=collab-maint/foo.git

END

Subscribe to this blog by RSS, by email or on Facebook.

Save disk space by excluding useless files with dpkg

Most packages contain files that you don’t need: for example translations in languages that you don’t understand, or documentation that you don’t read. Wouldn’t it be nice if you could get rid of them and save a few megabytes? Good news: since dpkg 1.15.8 you can!

dpkg has two options --path-include=glob-pattern and --path-exclude=glob-pattern that control what files are installed or not. The pattern work the same than what you’re used to on the shell (see the glob(7) manual page).

Passing those options on the command-line would be impractical, so the best way to use them is to put them in a file in /etc/dpkg/dpkg.cfg.d/. Beware, the order of the options does matter: when a file matches several options, the last one makes the decision.

A typical usage is to first exclude a directory and then to re-include parts of that directory that you want to keep. For example if you want to drop gettext translations and translated manual pages except French, you could put this in /etc/dpkg/dpkg.cfg.d/excludes:

# Drop locales except French
path-exclude=/usr/share/locale/*
path-include=/usr/share/locale/fr/*
path-include=/usr/share/locale/locale.alias

# Drop translated manual pages except French
path-exclude=/usr/share/man/*
path-include=/usr/share/man/man[1-9]/*
path-include=/usr/share/man/fr*/*

Note that the files will vanish progressively every time that a package is upgraded. If you want to save space immediately, you have to reinstall the packages present in your system. aptitude reinstall or apt-get --reinstall install might help. In theory with aptitude you can even do aptitude reinstall ~i but it tends to not work because one package is not available (either because it was installed manually or because the installed version has been superseded by a newer version on the mirror).

Found it useful? Click here to see how you can encourage me to provide more articles like this one.

5 reasons why a Debian package is more than a simple file archive

Folder with gearsYou’re probably manipulating Debian packages everyday, but do you know what those files are? This article will show you their bowels… Surely they are more than file archives otherwise we would just use TAR archives (you know those files ending with .tar.gz). Let’s have a look!

1. It’s two TAR file archives in an AR file archive!

A .deb file is actually an archive using the AR format, you can manipulate it with the ar command. This archive contains 3 files, you can check it yourself, download any .deb file and run “ar t” on it:

$ ar t gwibber_2.31.91-1_all.deb
debian-binary
control.tar.gz
data.tar.gz

debian-binary is a text file indicating the version of the format of the .deb file, the current version is “2.0”.

$ ar p gwibber_2.31.91-1_all.deb debian-binary
2.0

data.tar.gz contains the real files of the package, the content of that archive gets installed in your root directory when you run “dpkg --unpack“.

But the most interesting part—which truly makes .deb files more than a file archive—is the last file. control.tar.gz contains meta-information used by the package manager. What are they?

$ ar p gwibber_2.31.91-1_all.deb control.tar.gz | tar tzf -
./
./postinst
./prerm
./preinst
./postrm
./conffiles
./md5sums
./control

2. It contains meta-information defining the package and its relationships

The control file within the control.tar.gz archive is the most fundamental file. It contains basic information about the package like its name, its version, its description, the architecture it runs on, who is maintaining it and so on. It also contains dependency fields so that the package manager can ensure that everything needed by the package is installed before-hand. If you want to learn more about those fields, you can check Binary control files in the Debian Policy.

Those information end up in /var/lib/dpkg/status once the package is installed.

3. It contains maintainer scripts so that everything can just work out of the box

At various steps of the installation/upgrade/removal process, dpkg is executing the maintainer scripts provided by the package:

  • postinst: after installation
  • preinst: before installation
  • postrm: after removal
  • prerm: before removal

Note that this description is largely simplified. In fact the scripts are executed on many other occasions with different parameters. There’s an entire chapter of the Debian Policy dedicated to this topic. But you might find this wiki page easier to grasp: http://wiki.debian.org/MaintainerScripts.

While this looks scary, it’s a very important feature. It’s required to cope with non-backwards compatible upgrades, to provide automatic configuration, to create system users on the fly, etc.

4. Configuration files are special files

Unpacking a file archive overwrites the previous version of the files. This is the desired behavior when you upgrade a package, except for configuration files. You prefer not to loose your customizations, don’t you?

That’s why packages can list configuration files in the conffiles file provided by control.tar.gz. That way dpkg will deal with them in a special way.

5. You can always add new meta-information

And in fact many tools already exploit the possibility to provide supplementary files in control.tar.gz:

  • debsums use the md5sums file to ensure no files were accidentally modified
  • dpkg-shlibdeps uses shlibs and symbols files to generate dependencies on libraries
  • debconf uses config scripts to collect configuration information from the user

Once installed, those files are kept by dpkg in /var/lib/dpkg/info/package.* along with maintainer scripts.

If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

Managing distribution-specific patches with a common source package

In the comments of the article explaining how to generate different dependencies on Debian and Ubuntu with a common source package, I got asked if it was possible to apply a patch only in some distribution. And indeed it is.

The source package format 3.0 (quilt) has a neat feature for this. Instead of unconditionally using debian/patches/series to look up patches, dpkg-source first tries to use debian/patches/vendor.series (where vendor is ubuntu, debian, etc.). Note that dpkg-source does not stack patches from multiple series file, it uses a single series file, the first that exists.

So what’s the best way to use this? Debian should always provide debian/patches/series, they are supposed to provide the default set of patches to use. Any derivative cooperating with Debian can maintain their own series files within the common VCS repository used for package maintenance. They can drop Debian-specific patches (say branding patches for example), and they can add their own on top of the remaining Debian patches.

It’s worth noting that it’s the job of the maintainers to keep both series files in sync when needed. dpkg-source offers no way to have stacked series files (or dependencies between them).

If you want to use quilt to edit an alternate series file, you can temporarily set the QUILT_SERIES environment variable to “vendor.series”. Just make sure to start from a clean state, i.e. no patches applied. Otherwise quilt will be confused by the sudden mismatch between the series file and its internal data (stored in the .pc directory).

Found it useful? Click here to see how you can encourage me to provide more articles like this one.

Understanding Debian’s release process

Currently, the main product of the Debian project is its stable release[1]. Those release come out approximately every 18-24 months. This article gives a short overview of the process leading to the next stable release.

Creating a new distribution

Immediately after a stable release, a new distribution is created in the Debian archive. Its initial content is a copy of the (just released) stable distribution. Its codename is decided by the release managers and there’s a tradition of picking a character’s name from the Toy Story movie.

As an example, the “wheezy” distribution will be created once “squeeze” (aka Debian 6.0) is out.

For simplicity there’s a generic name to refer to the distribution used to prepare the next stable release: it’s testing. In the Debian archive, testing is just a symbolic link pointing to the right directory (squeeze currently).

Updating packages, working on release goals

During most of the cycle, developers work on packaging new upstream versions and implementing release goals. They upload their packages in the unstable distribution.

From there packages trickle to the testing distribution once they satisfy some quality checks: they must not have new release-critical bugs, they must have been built on all architectures that were previously supported, they must not break any dependency in testing, and they must have spent at least 10 days in unstable.

This minimal period ensures the package has been tested and gives enough time to users to file bugs if the package is suffering from problems. If the problems are deemed release-critical, they will block the migration of the package to testing.

During this part of the release cycle, the main work of the release team is ensuring that updated packages flow from unstable to testing. It can be a tricky task: package dependencies frequently tie packages together so that they can only migrate to testing together. If only one of the tied packages is not ready (for example if a new revision has been uploaded and has not spent 10 days in unstable yet), then none of them can migrate.

Stabilizing, polishing, fixing release-critical issues

The constant churn of new packages makes it very difficult to build a very polished release. That’s why, at some point, release managers freeze the testing distribution: automatic updates are stopped and they vet every single update made to testing. They have strong requirements, the goal is to only allow updates fixing release critical bugs, or those which are low-risk and bring significant value to the user experience (like new translations, updated documentation, etc.).

During freeze, some packages are also removed because the current upstream version can’t be supported for the lifetime of the stable release.

The freeze tends to slow down the pace of changes in unstable. Many maintainers opt to push new upstream versions in experimental instead so that if they need to update their packages in testing, they can still do it through unstable. This procedure is recommended by the release managers because it means that updates that they unblock have been tested as usual. It’s not the case for updates uploaded directly to testing (through testing-proposed-updates).

This behavior is rather annoying for the bleeding-edge users that use testing or unstable like a rolling release.

Release time

Once release managers are satisfied of the quality of the new distribution, some last minute work is needed, like generating the CD images. In the Debian archive, the release is made official by pointing the “stable” symbolic link to the new distribution (and the “oldstable” one to the previous distribution).

Now it’s party time, the cycle is over, and a new one can start. :-)

[1] The Constantly Usable Testing project aims to make testing a first-class product like stable—but with a very different update policy.

Follow me on Identi.ca, Twitter and Facebook.

Correctly renaming a conffile in Debian package maintainer scripts

After having dealt with the removal of obsolete conffiles, I’ll now explain what you should do when a configuration file managed by dpkg must be renamed.

The problem

Let’s suppose that version 1.2 of the software stopped providing /etc/foo.conf. Instead it provides /etc/bar.conf because the configuration file got renamed. If you do nothing special, the new conffile will be installed with the default configuration, and the old one will stay around. Any customization made by the administrator are lost in the process (in fact they are not lost, they are still in foo.conf but they are unused).

Of course, you could do mv /etc/foo.conf /etc/bar.conf in the pre-installation script. But that’s not satisfactory: it will generate a spurious conffile prompt that the end-user will not understand.

The solution

In the preinst script, you have to verify if the old conffile has been modified by the administrator. If yes, you want to keep the file around. Otherwise you know you will be able to ditch it once the upgrade is over, and you rename it to /etc/foo.conf.dpkg-remove to remember this fact.

In the postinst script, you remove /etc/foo.conf.dpkg-remove. If the old conffile (/etc/foo.conf) still exists, it’s because it was modified by the administrator. You make a backup of the new conffile in /etc/bar.conf.dpkg-dist and rename the old one into /etc/bar.conf.

In the postrm, when called to abort an upgrade, you move /etc/foo.conf.dpkg-remove back to its original name.

In practice, use dpkg-maintscript-helper

dpkg-maintscript-helper can automate all those tasks. You just have to put the following snippet in the maintainer scripts (postinst, postrm, preinst):

if dpkg-maintscript-helper supports mv_conffile 2>/dev/null; then
    dpkg-maintscript-helper mv_conffile /etc/foo.conf /etc/bar.conf 1.1-3 -- "$@"
fi

In this example, I assumed that version 1.1-3 was the last version of the package that contained /etc/foo.conf (i.e. the last version released before 1.2-1 was packaged).

You can avoid the preliminary test if you pre-depend on “dpkg (>= 1.15.7.2)” or if enough time has passed to assume that everybody has a newer version anyway. You can learn all the details in dpkg-maintscript-helper’s manual page.

Debhelper support

Debhelper makes it easy to inject those commands in the maintainer scripts. You just have to provide debian/*.maintscript files. See dh_installdeb’s manual page for details.

Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email (just check “Send me blog updates”).

The right way to remove an obsolete conffile in a Debian package

A conffile is a configuration file managed by dpkg, I’m sure you remember the introductory article about conffiles. When your package stops providing a conffile, the file stays on disk and it’s recorded as obsolete by the package manager. It’s only removed during purge. If you want the file to go away, you have to remove it yourself within your package’s configuration scripts. You will now learn how to do this right.

When is that needed?

dpkg errs on the side of safety by not removing the file until purge but in most cases it’s best to remove it sooner so as to not confuse the user. In some cases, it’s even required because keeping the file could break the software (for example if the file is in a .d configuration directory, and if it contains directives that are either no longer supported by the new version or in conflict with other new configuration files).

What’s complicated in “rm”?

So you want to remove the conffile. Adding an “rm” command in debian/postinst sounds easy. Except it’s not the right thing to do. The conffile might contain customizations made by the administrator and you don’t want to wipe those. Instead you want to keep the file around so that he can get his changes back and do whatever is required with those.

The correct action is thus to move the file away in the prerm, to ensure it doesn’t disturb the new version. At the same time, you need to verify whether the conffile has been modified by the administrator and remember it for later. In the postinst, you need to remove the file if it’s unmodified, or keep it under a different name that doesn’t interfere with the software. In many cases adding a simple .dpkg-bak suffix is enough. For instance, run-parts ignore files that contain a dot, and many other software are configured to only include files with a certain extension—say *.conf. In the postrm, you have to remove the obsolete conffiles that were kept due to local changes and you should also restore the original conffile in case the upgrade obsoleting the conffile is aborted.

Automating everything with dpkg-maintscript-helper

Phewww… that’s a lot of things to do for a seemingly simple task. Fortunately everything can be automated with dpkg-maintscript-helper. Let’s assume you want to remove /etc/foo/conf.d/bar because it’s obsolete and you’re going to prepare a new version 1.2-1 with the appropriate code to remove the file on upgrade. You just have to put this snippet in the 3 relevant scripts (preinst, postinst, postrm):

if dpkg-maintscript-helper supports rm_conffile 2>/dev/null; then
    dpkg-maintscript-helper rm_conffile /etc/foo/conf.d/bar 1.2-1 -- "$@"
fi

You can avoid the preliminary test if you pre-depend on “dpkg (>= 1.15.7.2)” or if enough time has passed to assume that everybody has a newer version anyway. You can learn all the details in dpkg-maintscript-helper’s manual page.

Debhelper integration

debhelper makes it easy to inject those commands for you. You can provide debian/*.maintscript files. See dh_installdeb’s manual page for details.

I hope you found this article helpful. You can follow me on Identi.ca, Twitter and Facebook.

How to generate different dependencies on Debian and Ubuntu with a common source package

There are situations in which a given package needs to have different dependencies in Debian and in Ubuntu. Despite this difference it’s possible to keep a single source package that will build both variants of the package. Continue reading to discover a step-by-step explanation.

1. When is that needed?

While it is possible to have different dependencies depending on the distribution on which the package is built, it should be usually avoided when possible. This infrastructure should only be used as a last resort when there are no better alternatives.

Here are some examples of when it might be needed:

  • Ubuntu has some packages that Debian does not have (or vice-versa), and the resulting package would benefit from having them installed.
  • The package names differ between Ubuntu and Debian and it’s on purpose (i.e. the difference is a justified choice and not a mistake because both distributions failed to coordinate).
  • The packages are built differently in both distributions, and the run time dependencies are not the same due to this. Maybe some associated patches are only applied in Ubuntu.

2. Using substitution variables in debian/control

The dependency that varies between both distributions can’t be hardcoded in debian/control, instead you should put a substitution variable (substvar) that will be replaced at build time by dpkg-gencontrol. You can name it ${dist:Depends} for example:

[...]
Depends: bzip2, ${shlibs:Depends}, ${misc:Depends}, ${dist:Depends}
[...]

Note that you typically already have other substitution variables (${shlibs:Depends} comes from dpkg-shlibdeps and ${misc:Depends} from debhelper and its dh_* scripts).

3. dpkg-gencontrol needs a -V option

The value used to replace this new variable needs to be communicated to dpkg-gencontrol. You can use the -V option for this, the syntax would be something like this:

dpkg-gencontrol [...] -Vdist:Depends="foo (>= 2), bar"

If you use debhelper, you have to pass the option to dh_gencontrol after two dashes (--):

dh_gencontrol -- -Vdist:Depends="foo (>= 2), bar"

If you use CDBS, you can set the DEB_DH_GENCONTROL_ARGS_ALL make variable:

include /usr/share/cdbs/1/rules/debhelper.mk
DEB_DH_GENCONTROL_ARGS_ALL = -- -Vdist:Depends="foo (>= 2), bar"

The value given to dpkg-gencontrol is static in all those examples, now let’s see how we can use give a different value depending on the distribution that we’re targetting.

4. Using dpkg-vendor in debian/rules

dpkg-vendor is a small tool (provided by the dpkg-dev package) that parses the /etc/dpkg/origins/default file (provided by the base-files package) to know the current distribution and its ancestry. It can be used in debian/rules to adjust the behavior depending on the current distribution. You can check its man-page to learn about the various options supported but we’re only going to use --derives-from <vendor> in this sample. With this option the script exits with zero if the current distribution is or derives from the indicated distribution, or with 1 otherwise.

Now combining all together, we can use dpkg-vendor to dynamically define the content of the substitution variable in debian/rules. Let’s suppose that you want ${dist:Depends} to be “foo (>= 2)” on Ubuntu (and its derivatives) and “bar” everywhere else. Using debhelper’s 7 tiny rules file, this example could be:

ifeq ($(shell dpkg-vendor --derives-from Ubuntu && echo yes),yes)
	SUBSTVARS = -Vdist:Depends="foo (>= 2)"
else
	SUBSTVARS = -Vdist:Depends="bar"
endif

%:
	dh $@

override_dh_gencontrol:
	dh_gencontrol -- $(SUBSTVARS)

If you use CDBS, it could be this:

include /usr/share/cdbs/1/rules/debhelper.mk
ifeq ($(shell dpkg-vendor --derives-from Ubuntu && echo yes),yes)
	DEB_DH_GENCONTROL_ARGS_ALL = -- -Vdist:Depends="foo (>= 2)"
else
	DEB_DH_GENCONTROL_ARGS_ALL = -- -Vdist:Depends="bar"
endif

Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

Everything you need to know about conffiles: configuration files managed by dpkg

The Debian policy dictates that package upgrades must take care of preserving user changes to configuration files. This article will explain you how most packages ensure this. This is important knowledge for anyone who has to manage upgrades: knowing how it works lets you easily automate most of it and deal correctly with the fallout.

How dpkg manages configuration files

Most packages rely on dpkg to properly install configuration files. Dpkg keeps a checksum of the last installed version of configuration file. When it must install a new version, it calculates the checksum of the currently installed file and if it doesn’t match anymore, it knows that the user has edited the file. In that case, instead of overwriting the configuration file, it asks the user what to do. You probably already have seen those questions, they look like this:

Configuration file `/etc/bash.bashrc'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** bash.bashrc (Y/I/N/O/D/Z) [default=N] ? 

In this specific example, if you answer “Y” or “I” (for “yes” or “install”), dpkg will install the new version of /etc/bash.bashrc but it will also backup the current version in /etc/bash.bashrc.dpkg-old. If you answer “N” or “O” (for “no” or “old”), dpkg will install the new version in /etc/bash.bashrc.dpkg-dist and /etc/bash.bashrc is left untouched. The two other answers allow you to examine the differences before taking a decision. Note that if you choose to start a shell, the new version is currently available as /etc/bash.bashrc.dpkg-new (and since Squeeze there are convenient environment variables $DPKG_CONFFILE_OLD and $DPKG_CONFFILE_NEW in case you want to create a custom review script).

All configurations files managed by dpkg are called “conffiles” because that’s the name of the field where they are recorded in the dpkg database. You can display the list of conffiles for any package:

$ dpkg --status bash
[...]
Conffiles:
 /etc/skel/.profile ecb6d3479ac3823f1da7f314d871989b
 /etc/skel/.bashrc 2afdd6c53990f2387a7ef9989af0bc07
 /etc/skel/.bash_logout 22bfb8c1dd94b5f3813a2b25da67463f
 /etc/bash.bashrc 5b3c3bc73d236e4e1b6f9b6c1ed5964e
[...]

The command “dpkg-query --showformat='${Conffiles}\n' --show bash” can give you the same information if you need to retrieve only that field. The 32 characters after the filename are the MD5 checksum of the original configuration file provided by the package.

Avoiding the conffile prompt

Every time that dpkg must install a new conffile that you have modified (and a removed file is only a particular case of a modified file in dpkg’s eyes), it will stop the upgrade and wait your answer. This can be particularly annoying for major upgrades. That’s why you can give predefined answers to dpkg with the help of multiple --force-conf* options:

  • --force-confold: do not modify the current configuration file, the new version is installed with a .dpkg-dist suffix. With this option alone, even configuration files that you have not modified are left untouched. You need to combine it with --force-confdef to let dpkg overwrite configuration files that you have not modified.
  • --force-confnew: always install the new version of the configuration file, the current version is kept in a file with the .dpkg-old suffix.
  • --force-confdef: ask dpkg to decide alone when it can and prompt otherwise. This is the default behavior of dpkg and this option is mainly useful in combination with --force-confold.
  • --force-confmiss: ask dpkg to install the configuration file if it’s currently missing (for example because you have removed the file by mistake).

If you use Apt, you can pass options to dpkg with a command-line like this:

$ apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade

You can also make those options permanent by creating /etc/apt/apt.conf.d/local:

Dpkg::Options {
   "--force-confdef";
   "--force-confold";
}

Bringing up the conffile prompt at any time

The conffile prompt is only displayed when dpkg detects that the package provides an new version of the conffile. Thus reinstalling the same package will not bring up the prompt. But you can instruct dpkg to ask nevertheless with the --force-confask option. This is a new feature in Debian Squeeze. It will only ask for files that are locally modified.

If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.