ATI driver madness in Kernel 2.6.25 - A bit of detective work

As I mentioned before in the guide to Fedora 9 Preview release, the proprietary ATI drivers don't work on Fedora 9. Compiling the driver by using the buildpkg argument to the driver installer generates the following output:

make[1]: Entering directory `/usr/src/kernels/2.6.25-8.fc9.i686'
CC [M] /tmp/ATI-fglrx-8.476-1-10249-januz/tmp/ATI-fglrx-8.476-1.f9-root-januz/lib/modules/fglrx/build_mod/2.6.x/firegl_public.o
/tmp/ATI-fglrx-8.476-1-10249-januz/tmp/ATI-fglrx-8.476-1.f9-root-januz/lib/modules/fglrx/build_mod/2.6.x/firegl_public.c: In function 'KCL_PosixSecurityCapGetEffectiveVector':
/tmp/ATI-fglrx-8.476-1-10249-januz/tmp/ATI-fglrx-8.476-1.f9-root-januz/lib/modules/fglrx/build_mod/2.6.x/firegl_public.c:1888: error: implicit declaration of function 'cap_t'
/tmp/ATI-fglrx-8.476-1-10249-januz/tmp/ATI-fglrx-8.476-1.f9-root-januz/lib/modules/fglrx/build_mod/2.6.x/firegl_public.c: In function 'KCL_PosixSecurityCapSetEffectiveVector':
/tmp/ATI-fglrx-8.476-1-10249-januz/tmp/ATI-fglrx-8.476-1.f9-root-januz/lib/modules/fglrx/build_mod/2.6.x/firegl_public.c:1896: error: lvalue required as left operand of assignment
make[2]: *** [/tmp/ATI-fglrx-8.476-1-10249-januz/tmp/ATI-fglrx-8.476-1.f9-root-januz/lib/modules/fglrx/build_mod/2.6.x/firegl_public.o] Error 1
make[1]: *** [_module_/tmp/ATI-fglrx-8.476-1-10249-januz/tmp/ATI-fglrx-8.476-1.f9-root-januz/lib/modules/fglrx/build_mod/2.6.x] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.25-8.fc9.i686'
make: *** [kmod_build] Error 2
build failed with return value 2


So it seems like cap_t is the culprit here. I am not a kernel hacker, so I have no idea what cap_t is supposed to accomplish. A Google search found this blog post, that describes exactly the same problem. The author has traced back the steps to actually find the kernel patch that caused the errors to crop up !. Apparently cap_t used to be defined as an int, but in the new kernel it has become an int array. The blog post contained two patches for the source files but there was no mention about how to apply them. ATI drivers come as a single executable package with no other files. Therefore getting to the source files was a problem.
Out of curiosity, I opened up ati-driver-installer-8-4-x86.x86_64.run in vim to glance through the script and I discovered a function that listed one of the valid arguments to the script as 'extract'.
MS_Help()
{
cat <<>&2
Makeself version 2.1.3
1) Getting help or info about $0 :
$0 -h|--help Print this message
$0 -i|--info Print embedded info : title, default target directory, embedded script
$0 -l|--list Print the list of files in the archive
$0 -c|--check Checks integrity of the archive
$0 --extract NewDirectory Extract this package to NewDirectory only

2) Running $0 :
$0 [options] [additional arguments to embedded script] with following options (in that order)
--keep Do not erase target directory after running the embedded script
Following arguments will be passed to the embedded script:
--install Install the driver(default)
--listpkg List all the generatable packages
--buildpkg package Build "package" if generatable ("package" as returned by --listpkg)
EOH
}


Running the installer package with the extract command as
ati-driver-installer-8-4-x86.x86_64.run --extract ati_src
created the directory ati_src with all the scripts and source files included in the driver package.

Patching the source tree was done easily. I copied the two patch sources from http://sarah-a-happy.livejournal.com/90345.html in to two files named patch1.patch and patch2.patch which I saved in the ati_src direcotry. Then all that was left to do was the actual patching itself !
patch -p0 < patch1.patch
patch -p0 < patch2.patch

To test whether the patches were working, I needed to run the installer again. But now since everything was extracted, the process was not quite obvious at first. I took my chances with the ati-installer.sh script that I found on the ati_src directory and ran it as follows:
./ati-installer.sh --buildpkg Fedora/F9
==================================================
ATI Technologies Linux Driver Installer/Packager
==================================================
Unrecognized parameter 'Fedora/F9' to ati-installer.sh
This script supports the following arguments:
--help : print help messages
--listpkg : print out a list of generatable packages
--buildpkg package : if generatable, the package will be created
--install : install the driver


So obviously the script was expecting more arguments. I opened up the script in vim to look through the code and discovered the following set of lines
DRV_RELEASE=$1
ACTION=$2

# Process input command
status=0
case "${ACTION}" in

--install)

SIGNATURE=$3

Aha ! The first argument to the script is the driver release version. Since this is the release 8.4 of the driver, I was sure that it is what the script was looking for, but to be doubly sure I opened up ati-driver-installer-8-4-x86.x86_64.run again to look through the source. The script invocation looks like this:
#extract the driver release version information from ${label}, which was set when we use makeself.sh to create
#this .run archive
drv_release=`echo $label | cut -d'-' -f2`


I found the declaration of $label at the very beginning of the script
label="ATI Proprietary Linux Driver-8.476"

So it was quite simple to execute the cut command inside my head and figure out that the expected argument was 8.476. With that knowledge, the drivers can now be compiled.

./ati-installer.sh 8.476 --buildpkg Fedora/F9

Yowza ! The drivers compile with no errors now !!

This is where the fun ends. I installed the drivers and restarted X, only to get a nasty X error that complains about a missing symbol named miZeroLineScreenIndex. A look through http://forums.fedoraforum.org/ revealed that X server 1.5 has a different ABI and any code written for older versions of X will fail to work. Fedora 9 Preview release ships with X server 1.4.99, the final test release before X 1.5 is officially released. Therefore, in order to get the drivers working, I would need to downgrade the X server. Now the main reason that Fedora is my distro of choice is because it represents the bleeding edge in the Linux world. While it makes getting things to work a nightmare, the rush you get from successfully getting things to work is unbelievable. So keeping to that philosophy, I decided to stick with the current X server and forget about ATI drivers for now. Besides, the open source radeon drivers that ship with Fedora work really well and I only missed the proprietary drivers when I attempted to get compiz-fusion working. But compiz can wait a few more days.

If you really want to get the ATI driver working on Fedora 9, downgrade Xorg as follows:

wget http://download.fedora.redhat.com/pub/fedora/linux/updates/8/i386/xorg-x11-server-Xorg-1.3.0.0-44.fc8.i386.rpm
sudo rpm -Uvh --oldpackage xorg-x11-server-Xorg-1.3.0.0-44.fc8.i386.rpm

There might be some dependencies on Xorg 1.4 that will prevent the downgrade. I haven't gone through the process, so I don't know what it entails. Good luck !!!