Missing firmware in Debian? Learn how to deal with the problem

You know it already, since Debian 6.0 non-free firmware are no longer provided by a standard Debian installation. This will cause some troubles to users who need them. I’m thus going to do a small overview on the topic and teach you what you need to know to deal with the problem.

What are firmware and how are they used?

From the user’s point of view, a firmware is just some data that is needed by some piece of hardware in order to function properly. The driver for that hardware typically loads the firmware on the device as part of its initialization.

In the Linux kernel, the drivers are all using a standardized interface (request_firmware) to retrieve the firmware before sending it to the device. Thanks to this standardization, it’s possible to either embed the firmware in the kernel or to have it request the firmware from user-space when needed.

Debian (like most distributions) has selected the latter option. Thus when the kernel needs a firmware, it sends out a request to user-space. udev is getting the request with the name of the firmware, and in its default configuration (see /lib/udev/rules.d/80-drivers.rules) it executes /lib/udev/firmware.agent in response.

Where are firmware stored?

firmware.agent is a simple shell script that tries to locate a firmware before sending it back to the kernel through a sysfs entry. It looks into the following directories:

  • /lib/firmware/$(uname -r)
  • /lib/firmware
  • /usr/local/lib/firmware
  • /usr/lib/hotplug/firmware

Firmware provided by packages are thus usually in /lib/firmware and you can use /usr/local/lib/firmware for manually installed firmware.

How do I know whether I need a firmware?

First of, you can notice messages from the kernel telling you that it tried to load a firmware but it failed. They look like this:

e100: eth0: e100_request_firmware: Failed to load firmware "e100/d101m_ucode.bin": -2

But you might be informed sooner. When you install a new version of the Linux kernel with the official Debian packages, the post-installation script will go through all loaded modules (those listed by lsmod) and it will verify whether this module as provided by the newly installed kernel might require firmware files. This information can be retrieved with modinfo:

$ modinfo -F firmware /lib/modules/2.6.32-5-amd64/kernel/drivers/net/e100.ko
e100/d102e_ucode.bin
e100/d101s_ucode.bin
e100/d101m_ucode.bin

If one (or more) of those firmware is (are) not yet available on the system, you will get a warning message similar to this one:

update-initramfs will also generate a similar warning on the terminal:

update-initramfs: Generating /boot/initrd.img-2.6.32-5-amd64
W: Possible missing firmware /lib/firmware/e100/d102e_ucode.bin for module e100
W: Possible missing firmware /lib/firmware/e100/d101s_ucode.bin for module e100
W: Possible missing firmware /lib/firmware/e100/d101m_ucode.bin for module e100

The Debian installer also detects when you have hardware that might require a missing firmware file. You have the option to supply the missing files on a USB stick (either directly or through the corresponding package).

How do I find and install the missing firmware?

Now that you have the name of the firmware file that you want, it’s relatively easy to identify the package that provides the required file. You can use “apt-cache search <filename>” because the firmware packages embed the list of firmware files in their description. You can also use “apt-file” (provided by the package of the same name) or the web interface at packages.debian.org.

$ apt-cache search d101m_ucode.bin
firmware-linux-nonfree - Binary firmware for various drivers in the Linux kernel
$ apt-file search d101m_ucode.bin
firmware-linux-nonfree: /lib/firmware/e100/d101m_ucode.bin

If the above commands return nothing, you probably need to enable the “non-free” repository in your /etc/apt/sources.list (you can also enable it within synaptic). And you also want to run “sudo apt-file update” to have the latest information.

Now you can install the right package, in the example above it was firmware-linux-nonfree.

How do I install all firmware just to be sure I don’t miss any?

There’s no meta-package depending on all firmware packages so there’s no easy answer to this question. Furthermore not all firmware packages respect the naming convention “firmware-*” (there’s for example zd1211-firmware).

So your best bet is to look up all the packages with a generic search like this one:

$ apt-file --package-only search /lib/firmware/
atmel-firmware
[...]

And then install them all.

Are there DVD or CD images with non-free firmware?

Yes. Debian provides an unofficial netinst image for i386/amd64/powerpc with the non-free firmware, you can find it here: http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/.

I provide complete DVD sets with firmware, and Multi-Arch CD/DVD with firmware, it’s over there in my DVD shop: http://raphaelhertzog.com/go/debian-cd/ (i386/amd64 only)

When using those installation discs, the Debian-installer will be able to find the required firmware immediately. There’s no need for you to provide them on a USB stick.

Other questions?

I think I covered the most important things you have to know about firmware on Debian. But should you still have a question, feel free to share it in the comments so that I can improve this article.

Click here to subscribe to my free newsletter and get my monthly analysis on what’s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

Discover my Debian DVD shop

After a private launch (with discounted prices) for my newsletter subscribers, it’s now time to open my Debian DVD shop to the public.

I did not want to become yet another DVD reseller, so my DVDs are different and better. Here’s why you want to get one (or more):

  1. it’s easier to install Debian with my DVDs since they provide all the (non-free) firmwares that have been stripped and that you’re supposed to provide on a USB key;
  2. the installed system features the former theme (MoreBlue Orbit) and not SpaceFun (although you can reactivate SpaceFun easily if you prefer it);
  3. 100% of the benefits are reinvested into Debian (90% to fund my Debian work, 10% given back to Debian to fund work meetings)
  4. they are provided in a beautiful DVD case and despite this they are not expensive (between $3.49 and $5.49)

Click here to learn more about my DVD offer.

PS: Click here and join my newsletter to not miss other opportunities.

People behind Debian: Maximilian Attems, member of the kernel team

Maximilian, along with the other members of the Debian kernel team, has the overwhelming job of maintaining the Linux kernel in Debian. It’s one of the largest package and certainly one where dealing with bug reports is really difficult as most of them are hardware-specific, and thus difficult to reproduce.

He’s very enthusiastic and energetic, and does not fear criticizing when something doesn’t please him. You’ll see.

My questions are in bold, the rest is by Maximilian.

Who are you?

My name is Maximilian Attems. I am a theoretical physicist in my last year of PhD at the Technical University of Vienna. My main research area is the early phase of a Quark-Gluon Plasma as produced in heavy ion collisions at the LHC at CERN. I am developing simulations that take weeks on the Vienna Scientific Cluster (in the TOP 500 list). The rest of the lab is much less fancy and boils down to straight intel boxes without any binary blobs or external drivers (although lately we add radeon graphics for decent free 3D). Mathematica and Maple are the rare exceptions to the many dev tools of Debian (LaTeX, editors, git, IDE’s, Open MPI, ..) found at the institute, as those are unfortunately yet unmatched in Free Software for symbolic computations. The lab mostly runs a combination of Debian stable (testing starting from freeze) for desktops and oldstable/stable for servers. Debian is in use for more than 10 years. So people in the institute know some ups and downs of the project. Newcomers like my room neighbors are always surprised how functional a free Debian Desktop is. :)

What’s your biggest achievement within Debian?

Building lots and lots of kernels together with an growing uptake of the officially released linux images.

I joined the Debian kernel team shortly after Herbert Xu departed. I had been upstream Maintainer of the linux-2.6 janitor project for almost a year brewing hundreds of small cleanups with quilt in a tree named kjt for early linux-2.6. In Debian we had lots of fun in sorting out the troubles that the long 2.5 freeze had imposed: Meaning we were sitting on a huge diverging “monolithic” semi-good patchset. It was great fun to prepare 2.6.8 for Sarge with a huge team enthusiastic in shipping something real close to mainline (You have to imagine that back then you had no stable or longterm release nor any useful free tools like git. This involved passing patches around, hand editing them and seeing what the result does.)

From the Sarge install reports a common pattern emerged that the current Debian early userspace was causing lots of boot failures. This motivated me to develop an alternative using the new upstream initramfs features. So I got involved in early userspace. Thanks to large and active development team initramfs-tools got a nice ecosystem. It still tries to be as generic and flexible as possible and thus gains many nice features. Also H. Peter Anvin (hpa) gave me the official co-maintenance of klibc. klibc saw uptake and good patches from Google in the last 2 years. I am proud that the early userspace is working out fairly well these days, meaning you can shuffle discs around and see your box boot.

Later on we focused on 2.6.18 for Etch, which turned out to a be good release and picked up by several other distributions. Only very much later we would see such a “sync” again. With 2.6.26 for Lenny we got somehow unlucky as we just missed the new longterm release by one release. We also pushed for another update very late (during freeze) in the release cycle, which turned out to semi-work as too much things depend on linux-2.6.

For Squeeze 2.6.32 got picked thanks to discussions at Portland Linux Plumbers and it turned out to be a good release picked up by many distributions and external patchsets. The long-term support is going very well. Greg KH is doing a great job in collecting various needed fixes for it. Somehow we had hoped that the Squeeze freeze would start sooner and that the freeze duration would be shorter, since we were ready for a release starting from the actual freeze on. The only real big bastard on the cool 2.6.32 “sync” is Red Hat. Red Hat Enterprise 6.0 is shipping the linux-2.6 2.6.32 in obfuscated form. They released their linux-2.6 as one big tarball clashing with the spirit of the GPL. One can only mildly guess from the changelog which patches get applied. This is in sharp contrast to any previous Red Hat release and has not yet generated the sharp and snide comments in press it deserves. Red Hat should really step back and not make such stupid management moves. Next to them even the semi-maintained Oracle “Unbreakable” 2.6.32 branch looks better: It is git fetchable.

What are your plans and those of the kernel team for Debian Wheezy?

Since 2.6.32 many of the used patches landed upstream or are on the way (speakup, Kbuild Debian specific targets, ..). The proper vfs based unionfs is something we’d be looking forward. We haven’t yet picked the next upstream release we will base Wheezy on, so currently we can happily jump to the most recent ones. There are plans for better interaction with Debian Installer thanks to generating our udebs properly in linux-2.6 source itself. Also we are looking forward to using git as tool of maintenance. We’d hope that this will also allow for even better cross distribution collaboration.

Concerning early userspace I plan to release an initramfs-tools with more generic userspace for the default case and finally also a klibc only for embedded or tuning cases.

What do you like most in Debian?

For one thing I do like the 2 year release cycle. It is not too long to have completely outdated software and on the other hand it gives enough time to really see huge progress from release to release. Also at my institute the software is is recent enough without too much admin overhead. For servers the three years support are a bit short, but on the manageable side.

I do enjoy a lot the testing distribution. For my personal use it is very stable and thus I mainly run testing on my desktop and work boxes. (Occasionally mixing in things from sid for unbreaking transition or newer security fixes).

Debian is independent and not a commercial entity. I think this is its main force and even more important these days. I enjoy using the Debian platform a lot at work thus in return this motivates me to contribute to Debian itself. I also like the fact that we strive for technical correctness.

Is there some recurrent problem that hinders the progress of Debian?

The “New Maintainer process” is a strange way to discourage people to contribute to Debian. It is particularly bureaucratic and a huge waste of time both for the applicant and his manager. It should be completely thrown overboard.

One needs a more scalable approach for trust and credibility that also enhances the technical knowledge for coding and packaging of the applicant.

NM is currently set in stone as any outside critics is automatically rejected. Young and energetic people are crucial for Debian and the long-term viability of the project, this is the reason why I’d consider the “New Maintainer process” as Debian’s biggest problem.

Note from Raphaël Hertzog: I must say I do not share this point of view on the New Maintainer process, I have witnessed lots of improvements lately thanks to the addition of the Debian Maintainer status, and to the fact that a good history of contribution can easily subsume the annoying Tasks & Skills questionnaire.

Another thing I miss is professional graphics’ input both for the desktop theme and the website. I know that effort has been done there lately and it is good to see movement there, but the end result is still lacking.

Another trouble of Debian is its marketing capabilities. It should learn to better sell itself. It is the distribution users want to run and use—not the rebranded copies of itself with lock-in “sugar”. Debian is about choice and it offers plenty of it: it is a great default Desktop.

Linus Torvalds doesn’t find Debian (and/or Ubuntu) a good platform to hack on the kernel. Do you know why and what can we do about this?

The Fedora linux-2.6 receives contributions from several Red Hat employed upstream sub-Maintainers. Thus it typically carries huge patches which are not yet upstream. As a consequence eventual userland troubles get revealed quite quickly and are often seen there first. The cutting edge nature of Fedora rawhide is appealing for many developers.

The usual Debian package division of library development files and the library itself is traditionally an entry barrier for dev on Debian. Debian got pretty easily usable these days, although we could and should again improve a lot more in this sector. Personally I think that Linus hasn’t tried Debian for years.

I have the feeling that the implication of the Debian Kernel team in LKML has been on the rise. Is that true and how do you explain this?

Ben Hutchings is the Nr.1 contributor for 2.6.33. He also is top listed as author of patches on stable 2.6.32. Debian is not listed as organization as many send their linux-2.6 patches from their corporate or personal email address and thus it won’t be attributed to Debian.

There is currently no means to see how many patches get forwarded for the stable tree, but I certainly forwarded more then fifty patches. I was very happy when Greg KH personally thanked me in the 2.6.32.12 release.

In the Squeeze kernel, the firmwares have been stripped and moved into separate packages in the non-free section. What should a user do to ensure his system keeps working?

There is a debconf warning on linux-2.6 installation. It is quite clear that the free linux-2.6 can’t depend on the firmware of the non-free archive (also there is no strict dependency there technically).

On the terminal you’d also see warnings by update-initramfs on the initramfs generation for drivers included in the initramfs.

The debconf warning lists the filename(s) of the missing firmware(s). One can then “apt-cache search” for the firmware package name and install it via the non-free repository. The check runs against the current loaded modules. The match is not 100% accurate for special cases as the one where the device might be handled well by this driver without firmware, but is accurate enough to warrant the warning.

The set of virtualization technologies that the official Debian kernel supports seems to change regularly. Which of the currently available options would you recommend to users who want to build on something that will last?

KVM has been a smooth ride from day zero. It almost got included instantly upstream. The uptake it has is great as it sees both dev from Intel and AMD. Together with libvirt it’s management is easy. Also the performance of virtio is very good.

The linux containers are the thing we are looking forward for enhanced chroots in the Wheezy schedule. They are also manageable by libvirt.

Xen being the “bad outside boy” has an incredible shrinking patchset, thus is fair to expect to see it for Wheezy and beyond. For many it may come a bit late, but for old hardware without relevant CPU it is there.

Many tend to overstate the importance of the virtualization tech. I’d be much more looking forward to the better Desktop support in newer linux-2.6. The Desktop is important for linux and something that is in heavy use. The much better graphics support of the radeon and nouveau drivers:

  • For Nouveau in 32 with 33 drm we could only deliver a first taste meaning something better then rusty nv.
  • For Radeon the support for Evergreen and newer is only been shaping now.

The performance optimizations thanks to dcache scalability work and the neat automatic task-grouping for the CPU scheduler are very promising features for the usability of the linux desktop. Another nice to have feature is the online defrag of ext4 and its faster mkfs. Even cooler would be better scalability in ext4 (This side seems to have seen not enough effort lately).

Is there someone in Debian that you admire for their contributions?

Hans Peter Anvin and Ted Tso are a huge source of deep linux-2.6 knowledge and personal wisdom. I do enjoy all sorts of interactions with them.

Christoph Hellwig with Matthew Wilcox and also William Irwin for setting up the Debian kernel Team.

Several Debian leaders including the previous and the current one for their engagement, which very often happens behind the scene.

The Debian Gnome Team work is great, also the interactions have always been always easy and a pleasure.

Martin Michlmayr and previously Thiemo Seufer do an incredible job in porting Debian on funny and interesting ARM and MIPS boxes. Debian has a lot of upcoming potential on this area. I’m looking forward to other young enthusiastic people in that area.

Colin Watson is bridging Debian and Ubuntu, which is an immense task.

Michael Prokop bases on Debian an excellent recovery boot CD: http://www.grml.org. I’d be happy if any Debian Developer would work as carefully coding and working.


Thank you to Maximilian for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.