This article originally appeared in issue 95 of Linux User & Developer magazine. Subscribe and save more than 30% and receive our exclusive money back guarantee – click here to find out more.

We’re now free of the Big Kernel Lock (in many configurations – it’s a config option that will hide other not-yet-compatible options when used) and I have been running systems BKL-free for some time now. Arnd Bergman and others have done an excellent job to rid us of this last vestige of truly ancient non-scalable Linux and unless you need a V4L (Video-4-Linux – TV tuner, webcam etc) device, you can probably run BKL-free today too. It is hoped that V4L will be fixed soon, maybe in time for 2.6.37. You probably won’t notice a huge performance benefit of running without the BKL unless you happen to have something more high end than a desktop, but it’s still pretty cool to know that you could get higher performance if only you could afford to have a system with dozens of CPUs to take advantage of it.

Scalability is great on the high end, but another more impressive feature for those working with more down-to-earth systems is (at last) near-complete support for running as a Xen ‘Dom0’ (or host kernel) under the Xen hypervisor. For years, the support for Xen host kernels lived in patches separate from the mainline kernel and had to be added separately. This constrained which Xen kernels could be used and made life more difficult for those using it in their virtualisation setups. It’s not (yet) possible to fully run an official Linus kernel without any patches as a Dom0 host kernel, but the remaining extra driver pieces and other work should be complete in time for 2.6.38. This incidentally prompted some folks in the Fedora kernel community to wonder about scheduling. They would like this to land in Fedora 15 (for those who want to use Xen instead of hardware-based KVM virtualisation) but are unwilling to accept large patches for things not yet in the official kernel (especially given historical experiences with having to maintain large patches for Xen). Only time can tell what will happen there.

Virtualisation also came up in the context of 3D graphics this month. Specifically, the idea is being floated that it might be time to implement a fully virtualised GPU (graphics processor) that provides a hybrid of OpenGL and DirectX to virtual machines, safely translating their operations on it into operations on the host GPU. Right now, there are some options for virtual machines to do 3D graphics, but they can involve more direct use of the host GPU, and this is not always safe against abuse, since graphics chips were never designed with such use cases in mind. Most of the current options involve hacks like giving each virtual machine its own GL context and trying hard not to allow guests to interfere with each other. But until the problem is solved nicely, various interim solutions such as virtio-gl do make sharing the host GPU a little easier to pull off in some kind of a clean fashion.

Google’s Android ‘suspend blockers’ code came up in discussion several times this month. These are a feature unique to Android wherein the Android kernel will attempt to suspend aggressively (in order to save as much power on the mobile embedded devices it has been designed primarily for use within) and driver code must explicitly request not to be suspended when necessary – for example, when in the middle of a phone call. The existing heavyweight Google solution was not liked by upstream developers, but several new alternative suggestions have started to bubble up and we might yet see something happen wherein a palatable solution can be found for enough developers to bring a generic Linux solution Android can also use in the future. There was more fun from Google in the way of nifty kernel features – look for coverage in my write-up of the 2010 Linux Plumbers Conference, also in this issue.
Finally, it looks as if those of us in embedded Linux space are finally headed in a similar direction to those in the enterprise Linux space by declaring certain releases of the upstream kernel to be ‘long term supported’ (LTS) for use in various projects (and products). Sony, Google, and the MeeGo and Linaro projects have gotten behind an effort spearheaded by Tim Bird, who is a Sony employee and former head of the Consumer Electronics Linux Forum (now part of the Linux Foundation) and who terms such supported versions to be ‘embedded flag’ releases. 2.6.35 is the first such kernel release.

Debugging regressions in the Intel graphics drivers
As one who has an interest in kernel development, I recently acquired a netbook intended solely to run the latest upstream kernels and distributions. The Eee PC model 1015PEM features a slightly more exciting revision to the Intel Atom processor (the Atom 470), Intel graphics and the SSD disk I shoehorned into the ‘non-user serviceable’ system after much yanking with screwdrivers. It runs Fedora Rawhide, the unstable work-in-progress distribution, and whatever kernel Linus (Torvalds) has just finished shoving out the door. In spite of all that apparently instability, it really doesn’t break horribly all that often.

Recently, I installed the 2.6.37-rc1 kernel on the netbook, only to find that the display was shifted about an inch (2.5cm) down, such that the top had a blank bar and the bottom was cut off. The display is a TFT panel attached to an Intel i915 within the Atom processor over a LVDS (low-voltage differential signalling) link. Modern displays built this century all implement the EDID specification, which is a protocol used by graphics chipsets like the i915 to read display geometry and colour information from a TFT panel or other screen device. EDID (one day maybe replaced by DisplayID) traditionally used several spare lines on a VGA connector to do I2C signalling and carried information like the ‘dotclock’ (maximum scan rate of a CRT) and manufacturer detail.

These days, we don’t worry about magnets and cathode rays, or the maximum speed at which we can drive them without them catching fire. Instead, we now deal in pixels and the maximum rate at which we can update them. But the display still needs to convey its mode (pixel resolution etc) capabilities to the graphics chipset, which needs to inform the driver, which needs to inform the kernel of the possible valid choices for mode to set. This was clearly not working correctly in my case, as reported by running the xrandr and other EDID parsing tools on my system. I knew there was a bug in the driver (i915) or a related part of the kernel GPU subsystem – as evidenced by the visual – but I didn’t have any idea where to look for this. Fortunately, Linux kernel engineers have a secret weapon in the fight against such
bugs: Git.
Far from being merely the kernel source code version control utility, Git can also be used to do many more advanced things, like tracking down bugs through binary bisection of code, repeating until a bad patch is found. In a bisection, Git will take a last known good kernel and produce a source tree that represents the halfway point between good and bad. Depending upon the results of a subsequent test build and boot, Git will then pick another halfway point, repeating until a single bad source code commit (patch) is found. This does require many kernel compiles/test boots, but it’s a good way to track down where a problem was introduced, and compile times are now fast enough as to make the entire exercise of building 10+ kernels doable in under an hour of effort. This is especially true when using a config made with ‘make localmodconfig’ to only build actually needed drivers.

Bisection really requires only three commands:

# git bisect start
# git bisect good
# git bisect bad

The first starts a bisection, the second marks the current point as good, and the third indicates a ‘bad’ kernel. A ‘git bisect reset’ resets back to the master or head of the Git tree from whence you started bisecting. Using this process, it became apparent that the bug I was seeing had been added in some cleanups to the i915 driver cacheing logic (which attempts to cache EDID data read from fixed panels like my netbook display that should not change). Unfortunately, that cleanup was incomplete, but with the aid of Git and a few resulting emails to the maintainer, a fix was soon in place. I really recommend you try out Git bisection sometime. Steven Rostedt even posted a script recently, called ktest.pl, that can be used to automate the process of running the build and boot tests, making bisection even easier to do than it already was, helping you nail down the next annoying bug.

Linux User & Developer is the magazine for the GNU Generation
Click here to try 3 issues for £1

Return to the homepage

Linux User

Ubuntu Linux Complete Edition 8.04

• Full functioning version of Ubuntu Hardy Herron 8.04
• Getting Started Technical Support — 60 days of unlimited customer support
• FireFox – Safe, Easy and Tabbed Web Browsing, OpenOffice – A Complete Productivity Suite, Evolution — Simple Email & Calendaring
• Gaim -Instant Messaging Compatible with AOL, Yahoo, MSN and others, F-spot – Easy Editing and Uploading of Photos, Gimp — A Powerful Graphics Program
• Compatible with Intel-based MACs and PCs.

Ubuntu gives users an operating system that is perfect for laptops and desktops and allows users to surf the web, read email, create documents and spreadsheets, edit images and much more. Take your old computer and give it a new life with Ubuntu. There’s so much to like about Ubuntu, it’s hard to know where to begin. I’ve now got two Linux machines that are purring along and doing exactly what they should without throwing me any curveballs. Getting Started Technical Support: 60 days of unlimited

Rating: (out of reviews)

List Price: $ 19.99

Price: $ 1.50

This article is due to appear in issue 96 of Linux User & Developer magazine. Subscribe and save more than 30% and receive our exclusive money back guarantee – click here to find out more.

Along the way, we also had two huge flame wars, several major security holes (one of which had been around for many years) were fixed, and many regressions were tracked and squashed. But if the year had to be summarised in a word, that word would be scalability. 2010 saw work on almost every aspect of the kernel – from Nick Piggin’s VFS (virtual file system) scalability patches to Arnd Bergmann’s BKL (Big Kernel Lock) removal work, and countless other efforts in between. You will see that recurring theme in the following month-by-month summary. If you’d like to see some more figures to accompany this timeline, visit the Linux Foundation website (linuxfoundation.org). Their annual kernel report focuses on overall numbers, not on individual features as we do here.

January
The year began with the rc3 release of kernel 2.6.33. This was the first kernel since the major Enterprise Linux vendors had chosen to base their next products on 2.6.32. New patches posted for discussion (but not necessarily merging into the kernel) included Mel Gorman’s Memory Compaction patches that literally ‘defragment’ physical memory to allow for larger contiguous regions to be used in certain allocations where contiguousness is required, for example in DMA operations withindevice drivers.
January also saw a brief discussion around kernel mode setting. Since modern kernels increasingly include in-kernel support for setting up graphics modes, it isn’t necessary for user-space software like Xorg to do this heavy lifting now. But Linus Torvalds was in no rush to remove user-space support for mode setting entirely, preferring to wait several years until all the kinks are gone and nobody is left using an older incompatible distribution.

February
February saw the release of 2.6.33. The new kernel included support for the reverse-engineered Nvidia driver (Nouveau) and the Data Replicated Block Device (DRBD), which apparently is now even in use on the kernel.org servers for replicating out content. Linus, who was growing increasingly unhappy with developers posting patches late in the ‘merge window’ (the period of time in which features are allowed into a new release) began making noises about reducing the merge window for 2.6.34, even closing it randomly in order to force good behaviour of posting patches early. This is a threat he did actually partially carry out in the following month.
The first nasty security hole of 2010 reared its ugly head in February. A problem was discovered in the kernel’s compatibility code for running a 32-bit distribution ‘user space’ on a 64-bit kernel. The exploit involved 32-bit code attempting (and failing) to run 64-bit code, which confused the kernel enough that it could be abused into causing a panic (crash).

March
March saw the closing (early, as threatened… then partially relaxed as Linus admitted he was “too much of a softie”) of the 2.6.34 merge window. New features that were accepted included Rafael J Wysocki’s excellent work on ‘asynchronous suspend and resume’, allowing the kernel to optimise the process of shutting down unused devices and the time taken for suspend. There was also a massive flame war around the merging of an incompatible set of patches for the Nouveau driver. Nouveau had previously been shipping (as a separate patch) in Fedora for some time. Linus was still very unhappy it hadn’t been merged sooner and blew a fuse when a patch that would have required distribution updates to Fedora 12 (which were not forthcoming as work was happening in the more experimental ‘rawhide’ area of Fedora) in order to test landed in Nouveau. Linus said, “I’m not going to release a kernel that I can’t test.” It got sorted out (not entirely cleanly) with special libraries built for Linus by Dave Airlie, the poor graphics subsystem maintainer.

Continue to: Page 2 – April to December 2010

Return to the homepage

Linux User

Linux in a Nutshell

• ISBN13: 9780596154486
• Condition: New
• Notes: BRAND NEW FROM PUBLISHER! BUY WITH CONFIDENCE, Over one million books sold! 98% Positive feedback. Compare our books, prices and service to the competition. 100% Satisfaction Guaranteed

Everything you need to know about Linux is in this book. Written by Stephen Figgins, Ellen Siever, Robert Love, and Arnold Robbins — people with years of active participation in the Linux community — Linux in a Nutshell, Sixth Edition, thoroughly covers programming tools, system and network administration tools, the shell, editors, and LILO and GRUB boot loaders.

This updated edition offers a tighter focus on Linux system essentials, as well as more coverage of new capabilities such a

Rating: (out of reviews)

List Price: $ 49.99

Price: $ 27.48

In order to thoroughly understand what makes Linux tick and why it works so well on a wide variety of systems, you need to delve deep into the heart of the kernel. The kernel handles all interactions between the CPU and the external world, and determines which programs will share processor time, in what order. It manages limited memory so well that hundreds of processes can share the system efficiently, and expertly organizes data transfers so that the CPU isn’t kept waiting any (more…)