Documentation – Page 9

4 tips to maintain a “3.0 (quilt)” Debian source package in a VCS

November 18, 2010 by Raphaël Hertzog

Most Debian packages are managed with a version control system (VCS) like git, subversion, bazaar or mercurial. The particularities of the 3.0 (quilt) source format are not without consequences in terms of integration with the VCS. I’ll give you some tips to have a smoother experience.

All the samples given in the article assume that you use git as version control system.

1. Add .pc to the VCS ignore list

.pc is the directory used by quilt to store its internal data (list of applied patches, backup of modified files). It’s also created by dpkg-source so that quilt knows that the patches are in debian/patches (and not in patches which is the default directory used by quilt). For that reason, the directory is kept even if you unapply all the patches.

However you don’t want to store this directory in your repository, so it’s best to put it in the VCS ignore list. With git you simply do:

$ echo ".pc" >>.gitignore
$ git add .gitignore
$ git commit -m "Ignore quilt dir"

The .gitignore file is ignored by dpkg-source, so you’re not adding any noise to the generated source package.

2. Unapply patches after the build

If you store upstream sources with non-applied patches (most people do), and if you don’t build packages in a temporary build directory, then you probably want to unapply the patches after the build so that your repository is again in a clean status.

This is now the default since dpkg-source will unapply any patch that it had to apply by itself. Thus if you start the build with a clean tree, you’ll end up with a clean tree.

But you can still force dpkg-source to unapply patches by adding “unapply-patches” to debian/source/local-options:

$ echo "unapply-patches" >>debian/source/local-options
$ git add debian/source/local-options
$ git commit -m "Unapply patches after build"

svn-buildpackage always builds in a temporary directory so the repository is left exactly like it was before the build, this option is thus useless. git-buildpackage can also be told to build in a temporary directory with --git-export-dir=../build-area/ (the directory ../build-area/ is the one used by svn-buildpackage, so this option makes git-buildpackage behave like svn-buildpackage in that respect).

3. Manage your quilt patches as a git branch

Instead of using quilt to manage the Debian-specific patches, it’s possible to use git itself. git-buildpackage comes with gbp-pq (“Git-BuildPackage Patch Queue”): it can export the quilt serie in a git branch that you can manipulate like you want. Each commit represents a patch, so you want to rebase that branch to edit intermediary commits. Check out the upstream documentation of this tool to learn how to work with it.

There’s an alternative tool as well: git-dpm. Its website explains the principle very well. It’s a more complicated than gbp-pq but it has the advantage of keeping the history of all branches used to generate the quilt series of all Debian releases. You might want to read a review made by Sam Hartman, it explains the limits of this tool.

4. Document how to review the changes

One of the main benefit of this new source format is that it’s easy to review changes because upstream changes are kept as separate patches properly documented (ideally using the DEP-3 format). With the tools above, the commit message becomes the patch header. Thus it’s important to write meaningful commit messages.

This works well as long as your workflow considers the Debian patches as a branch that you rebase on top of the upstream sources at each release. Some maintainers don’t like this workflow and prefer to have the Debian changes applied directly in the packaging branch. They switch to a new upstream version by merging it in their packaging branch. In that case, it’s difficult to generate a quilt serie out of the VCS. Instead, you should instruct dpkg-source to store all the changes in a single patch (which is then similar to the good old .diff.gz) and document in the header of that patch how the changes can be better reviewed, for example in the VCS web interface. You do the former with the --single-debian-patch option and the latter by writing the header in debian/source/patch-header:

$ echo "single-debian-patch" >> debian/source/local-options
$ cat >debian/source/patch-header <<END
This patch contains all the Debian-specific
changes mixed together. To review them
separately, please inspect the VCS history
at http://git.debian.org/?=collab-maint/foo.git

END

Subscribe to this blog by RSS, by email or on Facebook.

Save disk space by excluding useless files with dpkg

November 15, 2010 by Raphaël Hertzog

Most packages contain files that you don’t need: for example translations in languages that you don’t understand, or documentation that you don’t read. Wouldn’t it be nice if you could get rid of them and save a few megabytes? Good news: since dpkg 1.15.8 you can!

dpkg has two options --path-include=glob-pattern and --path-exclude=glob-pattern that control what files are installed or not. The pattern work the same than what you’re used to on the shell (see the glob(7) manual page).

Passing those options on the command-line would be impractical, so the best way to use them is to put them in a file in /etc/dpkg/dpkg.cfg.d/. Beware, the order of the options does matter: when a file matches several options, the last one makes the decision.

A typical usage is to first exclude a directory and then to re-include parts of that directory that you want to keep. For example if you want to drop gettext translations and translated manual pages except French, you could put this in /etc/dpkg/dpkg.cfg.d/excludes:

# Drop locales except French
path-exclude=/usr/share/locale/*
path-include=/usr/share/locale/fr/*
path-include=/usr/share/locale/locale.alias

# Drop translated manual pages except French
path-exclude=/usr/share/man/*
path-include=/usr/share/man/man[1-9]/*
path-include=/usr/share/man/fr*/*

Note that the files will vanish progressively every time that a package is upgraded. If you want to save space immediately, you have to reinstall the packages present in your system. aptitude reinstall or apt-get --reinstall install might help. In theory with aptitude you can even do aptitude reinstall ~i but it tends to not work because one package is not available (either because it was installed manually or because the installed version has been superseded by a newer version on the mirror).

Found it useful? Click here to see how you can encourage me to provide more articles like this one.

5 reasons why a Debian package is more than a simple file archive

November 8, 2010 by Raphaël Hertzog

Folder with gears You’re probably manipulating Debian packages everyday, but do you know what those files are? This article will show you their bowels… Surely they are more than file archives otherwise we would just use TAR archives (you know those files ending with .tar.gz). Let’s have a look!

1. It’s two TAR file archives in an AR file archive!

A .deb file is actually an archive using the AR format, you can manipulate it with the ar command. This archive contains 3 files, you can check it yourself, download any .deb file and run “ar t” on it:

$ ar t gwibber_2.31.91-1_all.deb
debian-binary
control.tar.gz
data.tar.gz

debian-binary is a text file indicating the version of the format of the .deb file, the current version is “2.0”.

$ ar p gwibber_2.31.91-1_all.deb debian-binary
2.0

data.tar.gz contains the real files of the package, the content of that archive gets installed in your root directory when you run “dpkg --unpack“.

But the most interesting part—which truly makes .deb files more than a file archive—is the last file. control.tar.gz contains meta-information used by the package manager. What are they?

$ ar p gwibber_2.31.91-1_all.deb control.tar.gz | tar tzf -
./
./postinst
./prerm
./preinst
./postrm
./conffiles
./md5sums
./control

2. It contains meta-information defining the package and its relationships

The control file within the control.tar.gz archive is the most fundamental file. It contains basic information about the package like its name, its version, its description, the architecture it runs on, who is maintaining it and so on. It also contains dependency fields so that the package manager can ensure that everything needed by the package is installed before-hand. If you want to learn more about those fields, you can check Binary control files in the Debian Policy.

Those information end up in /var/lib/dpkg/status once the package is installed.

3. It contains maintainer scripts so that everything can just work out of the box

At various steps of the installation/upgrade/removal process, dpkg is executing the maintainer scripts provided by the package:

postinst: after installation
preinst: before installation
postrm: after removal
prerm: before removal

Note that this description is largely simplified. In fact the scripts are executed on many other occasions with different parameters. There’s an entire chapter of the Debian Policy dedicated to this topic. But you might find this wiki page easier to grasp: http://wiki.debian.org/MaintainerScripts.

While this looks scary, it’s a very important feature. It’s required to cope with non-backwards compatible upgrades, to provide automatic configuration, to create system users on the fly, etc.

4. Configuration files are special files

Unpacking a file archive overwrites the previous version of the files. This is the desired behavior when you upgrade a package, except for configuration files. You prefer not to loose your customizations, don’t you?

That’s why packages can list configuration files in the conffiles file provided by control.tar.gz. That way dpkg will deal with them in a special way.

5. You can always add new meta-information

And in fact many tools already exploit the possibility to provide supplementary files in control.tar.gz:

debsums use the md5sums file to ensure no files were accidentally modified
dpkg-shlibdeps uses shlibs and symbols files to generate dependencies on libraries
debconf uses config scripts to collect configuration information from the user

Once installed, those files are kept by dpkg in /var/lib/dpkg/info/package.* along with maintainer scripts.

If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

Managing distribution-specific patches with a common source package

November 5, 2010 by Raphaël Hertzog

In the comments of the article explaining how to generate different dependencies on Debian and Ubuntu with a common source package, I got asked if it was possible to apply a patch only in some distribution. And indeed it is.

The source package format 3.0 (quilt) has a neat feature for this. Instead of unconditionally using debian/patches/series to look up patches, dpkg-source first tries to use debian/patches/vendor.series (where vendor is ubuntu, debian, etc.). Note that dpkg-source does not stack patches from multiple series file, it uses a single series file, the first that exists.

So what’s the best way to use this? Debian should always provide debian/patches/series, they are supposed to provide the default set of patches to use. Any derivative cooperating with Debian can maintain their own series files within the common VCS repository used for package maintenance. They can drop Debian-specific patches (say branding patches for example), and they can add their own on top of the remaining Debian patches.

It’s worth noting that it’s the job of the maintainers to keep both series files in sync when needed. dpkg-source offers no way to have stacked series files (or dependencies between them).

If you want to use quilt to edit an alternate series file, you can temporarily set the QUILT_SERIES environment variable to “vendor.series”. Just make sure to start from a clean state, i.e. no patches applied. Otherwise quilt will be confused by the sudden mismatch between the series file and its internal data (stored in the .pc directory).