Auto Mounting Windows Shares in GNOME with Gigolo and gvfs-fuse

The traditional way to mount Windows (or Samba) shares involves hardcoding the credentials in a plain-text file and some /etc/fstab entry to mount it automatically at boot time. If you don’t want to store a plain-text copy of your password, you’re bound to mount your shares interactively.

This is clearly sub-optimal and I thought that there must be a better way to handle this in the context of a GNOME desktop (which already supports connecting to such shares in the file browser). So I looked for a solution and after a bit of googling I found one.

Start by installing a few packages:

$ sudo apt-get install gigolo gvfs-fuse

Launch Gigolo, setup your shares as bookmarks and mark them as “Auto-Connect”.

During (first) connection, you will be prompted for your password and you have the possibility to store it in the GNOME keyring (and here it’s encrypted, not in plain-text!).

You should also configure the Gigolo preferences so that it starts minimized in the system tray (because we’re going to run it on session startup):

The last step is to ensure that Gigolo is executed at the start of each GNOME session. Unfortunately this GNOME feature is no longer accessible from the control center so you have to execute gnome-session-properties manually (from a terminal or the command line accessible via Alt+F2). Click on “Add” to add a new startup program:

You’re done!

The main limitation is that those shares are not real mounts, instead they are available within GNOME’s virtual file system (GVFS). If you use only 100% GNOME application, then it’s not a problem but otherwise it’s pretty annoying. You can’t “cd” in those shares from a terminal for example.

There’s a workaround though, it’s called “gvfs-fuse” and you installed it right at the start of this HOWTO. This service hooks into GVFS and exports all the virtual filesystem(s) in a real fuse-based mount that is automatically setup in ~/.gvfs/. However for this to work, the user must be in the “fuse” group. So you should run something like this:

$ sudo adduser $USER fuse

By the way, I haven’t found a way to use a non-hidden directory so if you want this directory to be more visible, I suggest that you create a symlink pointing to it.

Do you want to read more HOWTO like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

Test Driven Development with CppUTest, now in Debian

I have recently read Test Driven Development with Embedded C by James W. Grenning and published by Pragmatic Programmers.

I really enjoyed the book: while I was aware of the huge benefits of having a comprehensive test suite, I never studied seriously the principles behind Test Driven Development (TDD) and this book makes a good introduction to the topic. At the same time it focuses on the C language and contains lots of examples on how you can create tests even for projects which have to interact with hardware or other unpredictable components (the key is to create many abstractions) using all the possibilities that C offers.

The author convincingly argues that developing code with TDD forces you to create a modular design that is easier to evolve when the underlying requirements change. He also highlights how the tests serve as reference documentation of the API.

James W. Grenning recommends CppUTest as his xUnit test framework of choice. When I wanted to try this test framework, I discovered that it was not available in Debian. I decided to package it because it has some interesting features not offered by the contenders (at least not to my knowledge). It’s now available in Debian and in Ubuntu.

First, it doesn’t require any explicit registration of tests and has a very lightweight syntax. The small downside is that CppUTest requires the usage of C++ for the tests. But C++ is compatible with C so it doesn’t matter much if you have a C++ compiler for your target. On the contrary, usage of variables and methods scoped to the test group makes it easy to write clear tests. Here’s a short sample of test code:

extern "C" {
#include "timer.h"
#include "timefn.h"
}
 
#include "CppUTest/TestHarness.h"
 
static Time the_time;
static const int start_sec = 123;
static const int start_nsec = 456789000;
static const int delay_sec = 8;
static const int delay_nsec = 111111000; // start_nsec + delay_nsec < 10^9
 
TEST_GROUP(Timer)
{
    /* Class variables available to all tests in the group */
    Timer timer;
    Delay remaining;
 
    /* Standard setup/teardown methods of xUnit tests */
    void setup() {
        timer = timer_new();
        time_set(&the_time, start_sec, start_nsec);
        /* [...] */
    }
 
    void teardown() {
        timer_free(timer);
        /* [...] */
    }
 
    /* Helper functions specific to the test group */
    void start_timer_with_delay(long sec, long nsec)
    {
        timer_set_real_delay(timer, sec, nsec);
        timer_start(timer);
    }
 
    void ensure_remaining_is(long sec, long nsec)
    {
        CHECK_EQUAL(sec, delay_get_seconds(remaining));
        CHECK_EQUAL(nsec, delay_get_nanoseconds(remaining));
    }
};
 
TEST(Timer, NewIsNotStarted)
{
    CHECK(!timer->started);
}
/* [...] */
TEST(Timer, GetRemainingTimeWithNanosecondPrecision_ShiftOfSeconds)
{
    start_timer_with_delay(delay_sec, delay_nsec);
    time_set(&the_time, start_sec + delay_sec - 5, start_nsec + delay_nsec + 1000);
 
    remaining = timer_get_remaining_time(timer);
 
    ensure_remaining_is(4, 999999000);
}

To run those tests, you just need this boilerplate code in a main.cpp:

#include "CppUTest/CommandLineTestRunner.h"
 
int main(int argc, char** argv)
{
   return CommandLineTestRunner::RunAllTests(argc, argv);
}

Another interesting feature is its integrated memory leak detection system. Any test that hasn’t released allocated memory at the end of the “teardown” process will be marked as failed.

The upstream developers have made some unusual choices (static library only, installation in a private directory) but this will likely change with the switch to an automake and autoconf-based build system. I have reported the oddities that I found and I requested them to provide a pkg-config file to make it easier to compile and link unit tests exploiting CppUTest.

I already used CppUTest to develop a small application running on an embedded Linux. At some point, I might try to use CppUTest for dpkg development. I believe that it makes for a good fit. dpkg is already C++ ready since dselect is written in C++ and reuses a good part of dpkg’s code.

In any case, if you like Test Driven Development and are writing C or C++ based applications, I invite you to try CppUTest.

People Behind Debian: Mark Shuttleworth, Ubuntu’s founder

I probably don’t have to present Mark Shuttleworth… he was already a Debian developer when he became millionaire after having sold Thawte to Verisign in 1999. Then in 2002 he became the first African (and first Debian developer) in space. 2 years later, he found another grandiose project to pursue: bring the Microsoft monopoly to an end with a new alternative operating system named Ubuntu (see bug #1).

I have met Mark during Debconf 6 in Oaxtepec (Mexico), we were both trying to find ways to enhance the collaboration between Debian and Ubuntu. The least I can say is that Mark is opinionated but any leader usually is, and in particular the self-appointed ones! :-)

Read on to discover his view on the Ubuntu-Debian relationship and much more.

Raphael: Who are you?

Mark: At heart I’m an explorer, inventor and strategist. Change in technology, society and business is what fascinates me, and I devote almost all of my time and wealth to the catalysis of change in a direction that I hope improves society and the environment.

I’m 38, studied information systems and finance at the University of Cape Town. My ‘hearts home’ is Cape Town, and I’ve lived there and in Star City and in London, now I live in the Isle of Man with my girlfriend Claire and 14 precocious ducks. I joined Debian in around 1995 because I was helping to setup web servers for as many groups as possible, and I thought Debian’s approach to packaging was very sensible but there was no package for Apache. In those days, the NM process was a little easier ;-)

Raphael: What was your initial motivation when you decided to create Ubuntu 7 years ago?

Mark: Ubuntu is designed to fulfill a dream of change; a belief that the potential of free software was to have a profound impact on the economics of software as well as its technology. It’s obvious that the technology world is enormously influenced by Linux, GNU and the free software ecosystem, but the economics of software are still essentially unchanged.

Before Ubuntu, we have a two-tier world of Linux: there’s the community world (Debian, Fedora, Arch, Gentoo) where you support yourself, and the restricted, commercial world of RHEL and SLES/SLED. While the community distributions are wonderful in many regards, they don’t and can’t meet the needs of the whole of society; one can’t find them pre-installed, one can’t get certified and build a career around them, one can’t expect a school to deploy at scale a platform which is not blessed by a wide range of institutions. And the community distributions cannot create the institutions that would fix that.

Ubuntu brings those two worlds together, into one whole, with a commercial-grade release (inheriting the goodness of Debian) that is freely available but also backed by an institution.

The key to that dream is economics, and as always, a change in economics; it was clear to me that the flow of money around personal software would change from licensing (“buying Windows”) to services (“paying for your Ubuntu ONE storage”). If that change was coming, then there might be room for a truly free, free software distribution, with an institution that could make all the commitments needed to match the commercial Linux world. And that would be the achievement of a lifetime. So I decided to dedicate a chunk of my lifetime to the attempt, and found a number of wonderful people who shared that vision to help with the attempt.

It made sense to me to include Debian in that vision; I knew it well as both a user and insider, and believed that it would always be the most rigorous of the community distributions. I share Debian’s values and those values are compatible with those we set for Ubuntu.

“Debian would always be the most rigorous of the community distributions.”

Debian on its own, as an institution, could not be a partner for industry or enterprise. The bits are brilliant, but the design of an institution for independence implies making it difficult to be decisive counterparty, or contractual provider. It would be essentially impossible to achieve the goals of pre-installation, certification and support for third-party hardware and software inside an institution that is designed for neutrality, impartiality and independence.

However, two complementary institutions could cover both sides of this coin.

So Ubuntu is the second half of a complete Debian-Ubuntu ecosystem. Debian’s strengths complement Ubuntu’s, Ubuntu can achieve things that Debian cannot (not because its members are not capable, but because the institution has chosen other priorities) and conversely, Debian delivers things which Ubuntu cannot, not because its members are not capable, but because it chooses other priorities as an institution.

Many people are starting to understand this: Ubuntu is Debian’s arrow, Debian is Ubuntu’s bow. Neither instrument is particularly useful on its own, except in a museum of anthropology ;)

“Ubuntu is Debian’s arrow, Debian is Ubuntu’s bow.”

So the worst and most frustrating attitude comes from those who think Debian and Ubuntu compete. If you care about Debian, and want it to compete on every level with Ubuntu, you are going to be rather miserable; you will want Debian to lose some of its best qualities and change some of its most important practices. However, if you see the Ubuntu-Debian ecosystem as a coherent whole, you will celebrate the strengths and accomplishments of both, and more importantly, work to make Debian a better Debian and Ubuntu a better Ubuntu, as opposed to wishing Ubuntu was more like Debian and vice versa.

Raphael: The Ubuntu-Debian relationship was rather hectic at the start, it took several years to “mature”. If you had to start over, would you do some things differently?

Mark: Yes, there are lessons learned, but none of them are fundamental. Some of the tension was based on human factors that cannot really be altered: some of the harshest DD critics of Canonical and Ubuntu are folk who applied for but were not selected for positions at Canonical. I can’t change that, and wouldn’t change that, and would understand the consequences are, emotionally, what they are.

Nevertheless, it would have been good to be wiser about the way people would react to some approaches. We famously went to DebConf 5 in Porto Allegre and hacked in a room at the conference. It had an open door, and many people popped a head in, but I think the not-a-cabal collection of people in there was intimidating and the story became one of exclusion. If we’d wanted to be exclusive, we would have gone somewhere else! So I would have worked harder to make that clear at the time if I’d known how many times that story would be used to paint Canonical in a bad light.

As for engagement with Debian, I think the situation is one of highs and lows. As a high, it is generally possible to collaborate with any given maintainer in Debian on a problem in which there is mutual interest. There are exceptions, but those exceptions are as problematic within Debian as between Debian and outsiders. As a low, it is impossible to collaborate with Debian as an institution, because of the design of the institution.

“It is generally possible to collaborate with any given maintainer […] [but] it is impossible to collaborate with Debian as an institution.”

In order to collaborate, two parties must make and keep commitments. So while one Debian developer and one Ubuntu developer can make personal commitments to each other, Debian cannot make commitments to Ubuntu, because there is no person or body that can make such commitments on behalf of the institution, on any sort of agile basis. A GR is not agile ;-). I don’t say this as a critique of Debian; remember, I think Debian has made some very important choices, one of those is the complete independence of its developers, which means they are under no obligation to follow a decision made by anyone else.

It’s also important to understand the difference between collaboration and teamwork. When two people have exactly the same goal and produce the same output, that’s just teamwork. When two people have different goals and produce different product, but still find ways to improve one anothers product, that’s collaboration.

So in order to have great collaboration between Ubuntu and Debian, we need to start with mutual recognition of the value and importance of the differences in our approach. When someone criticises Ubuntu because it exists, or because it does not do things the same way as Debian, or because it does not structure every process with the primary goal of improving Debian, it’s sad. The differences between us are valuable: Ubuntu can take Debian places Debian cannot go, and Debian’s debianness brings a whole raft of goodness for Ubuntu.

Raphael: What’s the biggest problem of Debian?

Mark: Internal tension about the vision and goals of Debian make it difficult to create a harmonious environment, which is compounded by an unwillingness to censure destructive behaviour.

Does Debian measure its success by the number of installs? The number of maintainers? The number of flamewars? The number of packages? The number of messages to mailing lists? The quality of Debian Policy? The quality of packages? The “freshness” of packages? The length and quality of maintenance of releases? The frequency or infrequency of releases? The breadth of derivatives?

Many of these metrics are in direct tension with one another; as a consequence, the fact that different DD’s prioritise all of these (and other goals) differently makes for… interesting debate. The sort of debate that goes on and on because there is no way to choose between the goals when everyone has different ones. You know the sort of debate I mean :-)

Raphael: Do you think that the Debian community improved in the last 7 years? If yes, do you think that the coopetition with Ubuntu partly explains it?

Mark: Yes, I think some of the areas that concern me have improved. Much of this is to do with time giving people the opportunity to consider a thought from different perspectives, perhaps with the benefit of maturity. Time also allows ideas to flow and and of course introduces new people into the mix. There are plenty of DD’s now who became DD’s after Ubuntu existed, so it’s not as if this new supernova has suddenly gone off in their galactic neighbourhood. And many of them became DD’s because of Ubuntu. So at least from the perspective of the Ubuntu-Debian relationship, things are much healthier.

We could do much better. Now that we are on track for four consecutive Ubuntu LTS releases, on a two-year cadence, it’s clear we could collaborate beautifully if we shared a freeze date. Canonical offered to help with Squeeze on that basis, but institutional commitment phobia reared its head and scotched it. And with the proposal to put Debian’s first planned freeze exactly in the middle of Ubuntu’s LTS cycle, our alignment in interests will be at a minimum, not a maximum. Pure <facepalm />.

Raphael: What would you suggest to people (like me) who do not feel like joining Canonical and would like to be paid to work on improving Debian?

Mark: We share the problem; I would like to be paid to work on improving Ubuntu, but that’s also a long term dream ;-)

Raphael: What about using the earnings of the dormant Ubuntu Foundation to fund some Debian projects?

Mark: The Foundation is there in the event of Canonical’s failure to ensure that commitments, like LTS maintenance, are met. It will hopefully be dormant for good ;-)

Raphael: The crowdfunding campaign for the Debian Administrator’s Handbook is still going on and I briefly envisioned the possibility to create the Ubuntu Administrator’s Handbook. What do you think of this project?

Mark: Crowdfunding is a great match for free software and open content, so I hope this works out very well for you. I also think you’d find a bigger market for an Ubuntu book, not because Ubuntu is any more important than Debian but because it is likely to appeal to people who are more inclined to buy or download a book than to dive into the source.

Again, this is about understanding the difference in audiences, not judging the projects or the products.

Raphael: Is there someone in Debian that you admire for their contributions?

Mark: Zack is the best DPL since 1995; it’s an impossible job which he handles with grace and distinction. I hope praise from me doesn’t tarnish his reputation in the project!


Thank you to Mark for the time spent answering my questions. I hope you enjoyed reading his answers as I did.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook

.

20 Things to Learn About APT With the Free Chapter of the Debian Administrator’s Handbook

We just released a sample chapter of the Debian Administrator’s Handbook. It covers the APT family of tools: apt-get, aptitude, synaptic, update-manager, etc.


Click here to get your free sample chapter

I’m sure you will enjoy it. There are many interesting things to learn:

  • How to customize the sources.list file
  • The various APT repositories that Debian offers (Security Updates, Stable Updates, Proposed Updates, Backports, Experimental, etc.)
  • How to select the best Debian mirror for you
  • How to find old package versions
  • How to install the same selection of packages on multiple computers
  • How to install and remove a package on a single command-line
  • How to reinstall packages and how to install a specific version of a package
  • How to pass options to dpkg via APT
  • How to configure a proxy for APT
  • How to set priorities to various package sources (APT pinning)
  • How to safely mix packages from several distributions on a single system
  • How to use aptitude’s text-mode graphical interface
  • How to use the tracking of automatically installed packages to keep a clean system
  • How APT checks the authenticity of packages that it downloads
  • How to add supplementary GnuPG keys to APT’s trusted keyring
  • How to upgrade from one stable distribution to the next
  • How to handles problems after an upgrade
  • How to keep a system up-to-date
  • How to automate upgrades
  • How to find the package that you’re looking for

If you liked this chapter, click here to contribute a few euros towards the liberation of the whole book. That way you’ll get a copy of the ebook as soon as it’s available. Thank you!

I also invite you to share this sample chapter as widely as possible. We’re only at 40% of the liberation fund and there’s less than 2 weeks left. I hope this book extract will convince enough people that the book is going to be great, and that it really deserves to be liberated and bundled with Debian!

Debian joins Dropbox’s officially supported platforms along with Fedora and Ubuntu

If you checkout Dropbox’s Linux download page, you will see that Debian packages are provided. Up to a few days ago, they only provided packages for Ubuntu and Fedora.

I’m glad to see that packaging nautilus-dropbox for Debian and being in touch with them due to this led to this result.

Another positive outcome is that the version 0.7.0 now ensures the origin of the downloaded binaries with GPG.

What about creating The Ubuntu Administrator’s Handbook?

I am currently running a crowdfunding campaign whose ultimate goal is to liberate the English translation of a French book that I have written. This book will be named The Debian Administrator’s Handbook because it has primarily been written for Debian.

Creating a new Ubuntu book based on The Debian Administrator’s Handbook

But since Ubuntu is based on Debian, a large part of its content applies equally well to Ubuntu. While discussing with Mark Shuttleworth, he suggested me to reuse those parts and to create a new book dedicated to Ubuntu. It would also cover the latest cloud technologies that Ubuntu has been delivering (since this is a topic that the current book does not cover).

This is something that I have been envisioning for a while and something that I would be ready to try if we manage to complete the liberation of the current book. This project would then bring a truly free book to the Ubuntu ecosystem.

Why? The official Ubuntu books are not really free

There’s a policy in place that ensures that official Ubuntu books use a free software/culture license and they are effectively available under the terms of a Creative Commons Share Alike license. But try to create a derivative book… you won’t find the “sources” (LaTeX or DocBook usually with most big books). You can only find a few PDF copies if you google for it. But this is really not the preferred form of modification for such a book.

Those books are also not packaged. Ubuntu much like Debian deserves to have a good book embodying the values of free software that can be shipped together with its product.

When I speak of liberation of the book, I really mean it in the way that free software hackers are used to: a public Git repository containing the DocBook sources, the pictures and the .dia files for the various schemas.

Help Ubuntu by spreading the word

I understand that at this point this proposed Ubuntu book is really hypothetical (“vaporware” one could say) but we need to go step by step to make it a reality. And the first step is to ensure that we manage to liberate the Debian Administrator’s Handbook.

For this I am seeking the support of the Ubuntu community to promote the current fundraising campaign. If the perspective of the Ubuntu book is not enough to convince you, you’ll be glad to learn that I also commit to give back to Ubuntu 15% of the money raised via the link below (once VAT has been subtracted).

Click here to go to the crowdfunding campaign page and pledge a few euros. Then share this article (or the link http://debian-handbook.info/go/ulule-ubuntu/) and convince others to participate.

At this point, the liberation target is entirely reachable with your help and the help of the community: the remaining 18 K€ needed in the liberation fund represent 720 persons giving 25 EUR each or 1800 persons giving 10 EUR each.

Thank you very much for your support and your help in this project!

Understand dpkg and don’t get stuck with a maintainer script failure

Continuing my series of articles on dpkg’s errors, this time I’ll cover a pretty common one which has several variations:

Setting up acpid (1:2.0.12-1) ...
rm: cannot remove `/etc/rc1.d/K20acpid': No such file or directory
dpkg: error processing acpid (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 acpid

Even if dpkg is failing and outputting the error message, the real problem is not in dpkg but in the installed package (acpid in the example above). As we already learned, a package contains not only files but also “maintainer scripts” that are executed at various points of the installation process (see some useful graphics to understand how they are called, thanks to Margarita Manterola).

Maintainer scripts in a package upgrade

In the introductory example it was acpid’s “post-installation script” that failed, and dpkg is only forwarding that failure back to the caller. The maintainer scripts are stored in /var/lib/dpkg/info/. You can thus inspect them and even modify them if you hit a bug and want to work around it (do this only if you understand what you do!).

One common modification is to add “set -x” at the start of the script and to retry the failing operation. That way you can see what’s executed exactly. Here’s what the output could look like after the addition of “set -x” to /var/lib/dpkg/info/acpid.postinst:

$ sudo dpkg --configure acpid
Setting up acpid (1:2.0.12-1) ...
+ dpkg --compare-versions 1:2.0.11-1 lt-nl 1.0.10-3
+ dpkg --compare-versions 1:2.0.11-1 lt-nl 1.0.6-16
+ dpkg --compare-versions 1:2.0.11-1 lt 1.0.6-6
+ rm /etc/rc1.d/K20acpid
rm: cannot remove `/etc/rc1.d/K20acpid': No such file or directory
dpkg: error processing acpid (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 acpid

This output helps you locate the command that is actually failing. Here’s it’s relatively easy since we have an error message from “rm”. And the fix is trivial too, we replace “rm” with “rm -f” so that it doesn’t fail when the file doesn’t exist (this is a fake bug I made up for this article—I just added a failing rm call—but it’s inspired by real bugs I experienced).

Maintainer scripts are supposed to be idempotent: we should be able to execute them several times in a row without bad consequences. It happens from time to time that the maintainer gets this wrong… on the first try it works, so he uploads his package and we discover the problem only later once someone ended up executing the same code twice for some reason.

Follow me on Identi.ca, Twitter, Facebook and Google+. Or subscribe to this blog by RSS or by email.

Understanding dpkg’s file overwrite error

This is probably one of the most common errors. You’re very likely to encounter it, in particular if you tend to mix packages from various origins/distributions, or if you’re using unstable. It looks like this:

Unpacking gbonds-data (from .../gbonds-data_2.0.3-2_all.deb) ...
dpkg: error processing /var/cache/apt/archives/gbonds-data_2.0.3-2_all.deb (--unpack):
 trying to overwrite '/usr/share/omf/gbonds/gbonds-C.omf', which is also in package gbonds 2.0.2-9
dpkg-deb: subprocess paste killed by signal (Broken pipe)

A given file can only be provided by a single package. So if you try to install a package that provides a file that is already part of another installed package, it will fail with a message similar to the above one.

Sometimes this failure will be meaningful because dpkg prevented you to install two unrelated packages that happen to have a real file conflict. In other cases, like in the example above, this failure is just the result of a mistake.

Folder with gears

The version 2.0.3-1 of gbonds split the architecture independent files in a separate package called gbonds-data but the maintainer forgot to add the required control field in gbonds-data (Replaces: gbonds (<< 2.0.3-1)). That field allows dpkg to take over files from the listed packages.

If you want to ignore the file conflict and let dpkg take over the file (even without the Replaces), you can pass the --force-overwrite command-line option.

But you’re not using dpkg directly, you’re probably using an APT frontend (like apt-get or aptitude). Don’t worry, there’s a simple way to define custom dpkg options to use:

# apt-get -o Dpkg::Options::="--force-overwrite" install gbonds-data

The syntax is a bit weird, but the “::” after “Options” is important, it’s the syntax that defines a list item value instead of a single value. And you can effectively pass multiple options to dpkg by putting multiple -o Dpkg::Options::="…".

If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

Deciphering one of dpkg’s weirdest errors: unable to open ‘/path/to/foo.dpkg-new’

We already studied one weird error of dpkg, let’s do another one:

Unpacking replacement libexo-common ...
dpkg: error processing /var/cache/apt/archives/libexo-common_0.6.1-1_all.deb (--unpack):
 unable to open '/usr/share/doc/exo/html/C/images/exo-preferred-applications-internet.png.dpkg-new': No such file or directory

Rather difficult to understand on the first look right? Let’s see in detail what’s usually happening when you get this error.

The first hint comes from the file extension “.dpkg-new”. This extension is used by dpkg to unpack the updated files near the old files. When everything has been unpacked, they are renamed over the old files.

The failure happens precisely when dpkg tries to rename the file (in fact when it tries to fsync() it before the rename)… but why does it fail?

Usually because there are unexpected symlinks (or bind mounts) involved that resulted in two different files being installed in the same directory. For example consider package that provides /dir1/a-file and /dir2/a-file. Now imagine that on the target system, /dir1 is a real directory but /dir2 is a symbolic link that points to /dir1.

When dpkg processes /dir1/a-file.dpkg-new everything is fine, but when it tries to process /dir2/a-file.dpkg-new it will fail because that file is the same than /dir1/a-file.dpkg-new which has already been renamed.

Diagnosing further the problem requires to understand why there’s a symlink instead of a real directory. It might be two packages that were badly coordinated, or a problem in the package itself because it lacks some code that drops the symlink in the preinst (so that dpkg installs the real directory instead of keeping the symlink).

There might be variations in the way two files end up sharing the same directory, but this simple example should have clarified the nature of the underlying problem.

Subscribe to this blog by RSS, by email or on Facebook.

How to prepare patches for Debian packages

You want to start contributing to Debian and/or Ubuntu, you decided to help a package maintainer and you’re now looking for how to change a source package and how to submit your changes.

1. Retrieve the source package and install build-dependencies

The first step is to retrieve the latest version of the source package and to install the required build-dependencies. I already covered how to do this with apt-get in the article explaining how to rebuild a source package.

If you prefer, you can use dget (from the devscripts package) to directly grab the source package. You can find the URL of the .dsc files in the Package Tracking System for example.

Sometimes apt-get will warn you that the source package is maintained in a VCS repository, like this:

$ apt-get source wordpress
[...]
NOTICE: 'wordpress' packaging is maintained in the 'Git' version control system at:
git://git.debian.org/git/collab-maint/wordpress.git
[...]

In that case, you can use debcheckout to retrieve the VCS repository instead (provided that you have the corresponding VCS installed):

$ debcheckout wordpress
declared git repository at git://git.debian.org/git/collab-maint/wordpress.git
git clone git://git.debian.org/git/collab-maint/wordpress.git wordpress ...
Cloning into wordpress...

Note however that some maintainers use their VCS in a way that’s not really compatible with the explanations that I will give below.

It’s also a good idea to install the package “packaging-dev”. It’s a meta-package depending on the most common tools that are used for Debian packaging work.

2. Do the changes

Execute dch --nmu to record the fact that you’re working on an update prepared by someone who is not the maintainer (NMU means Non Maintainer Upload). This also ensures that if we build the package, we won’t overwrite the original source package that we downloaded, thus making it possible to generate a “diff” between both versions.

2.1. Modify Debian packaging files

Now fire your text editor and do the required changes in the “debian” sub-directory. You will probably run dch -a multiple times to document each subsequent change.

2.2. Modify upstream files

If you have to modify upstream files, the proper way to do it depends on the source package format (“1.0″ vs “3.0 (quilt)” vs “3.0 (native)”, see the debian/source/format file) and on the presence or not of a patch system (the what-patch can help you identify it). In this explanation, I’ll assume that the package is using the recommended format: “3.0 (quilt)”. (It also works for “1.0” if quilt is used and if you configured ~/.quiltrc as recommended by /usr/share/doc/quilt/README.source).

First you should ensure that all patches have been applied with quilt push -a. If there’s no patch yet, you want to create the debian/patches directory (with mkdir debian/patches). Note that you should better invoke quilt from the root of the source package (and the examples below assume this).

2.2.1. Import a patch

If the upstream changes are already in a patch file (say /tmp/patch that you downloaded from the upstream VCS repository) you can import that patch like this:

$ quilt import -P fix-foobar.patch /tmp/patch
Importing patch /tmp/patch (stored as fix-foobar.patch)
$ quilt push
Applying patch fix-foobar.patch
[...]
Now at patch fix-foobar.patch

The -P option allows to select the name of the patch file created in debian/patches/. As you see, the new patch file is recorded in debian/patches/series but not applied by default, we’re thus doing it with quilt push.

2.2.1. Create a new patch

If the upstream changes that you want to make are not in a patch yet, you should tell quilt that you’re going to create one:

$ quilt new fix-foobar.patch
Patch fix-foobar.patch is now on top

Then you must record every file that you’re going to modify with a quilt add invocation. quilt then makes a backup of those files so that it can generate the patch later on. If you’re going to modify the files with your text editor you can just do quilt edit file-to-modify, it’s the same than quilt add file-to-modify followed by sensible-editor file-to-modify.

$ quilt edit foobar.c
File foobar.c added to patch fix-foobar.patch

The last step is tell quilt to generate the patch:

$ quilt refresh
Refreshed patch fix-foobar.patch

3. Test your changes

You should build your modified package with “debuild -us -uc”. You can easily install the resulting package with “debi”. Verify that everything works as expected. If not, continue your modifications until you’re satisfied with the result.

4. Generate a patch and mail it

If you followed the instructions, you should have two .dsc files in the parent directory, like this:

$ cd ..
$ ls wordpress_*.dsc
../wordpress_3.0.5+dfsg-1.1.dsc
../wordpress_3.0.5+dfsg-1.dsc

Generating the patch to send to the maintainer is then just a matter of running debdiff:

$ debdiff wordpress_3.0.5+dfsg-1.dsc wordpress_3.0.5+dfsg-1.1.dsc >/tmp/wp-debdiff

You can send the /tmp/wp-debdiff file to the wordpress maintainer. Usually you send it via the bugreport that your update is fixing and you add the “patch” tag to the report.

This can be automated with the nmudiff utility. By default it assumes that you’re using mutt but it can also directly feed the resulting mail to sendmail. The default text that nmudiff proposes assumes that you’re actually performing an NMU and that the result has been uploaded. If that’s not the case, you should edit the text and make it clear that you’re just sending a patch.

If you have been working in a VCS repository, instead of using debdiff you can simply use the diff feature integrated in your VCS (git diff, svn diff, etc.). But note that with a distributed VCS (like git/bzr/mercurial, unlike svn) you should probably have committed all individual changes in separate changesets. And instead of sending a single patch, you’re probably going to send a series of patches (though it might be easier to just upload your branch in a public repository and give the corresponding URL to the maintainer).

Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.