Thursday, January 1, 2009

NVIDIA broken AGAIN

I love NVIDIA's proprietary binary driver. It gives me so many hours of troubleshooting fun. Here's the latest installment:

Ubuntu 8.10 Kernel upgrade 2.6.27-9 = No more EDID from my monitor

Without the EDID information from the monitor, NVIDIA has a meltdown and only lets me have 640x480... sometimes... Other times it just blows up.

After several hours of hacking with the xorg.conf in an attempt to replicate the modelines properly, I have an epiphany.. Why not FAKE the EDID!

Option "CustomEDID" "DFP-0:/etc/X11/edid.bin"

This allows me to give NVIDIA a bin file and tell it to use this instead of asking my monitor for the EDID... So now I just pull the EDID from the monitor and we're done, right?

Download read-edid 2.0

> sudo get-edid > edid.bin

VBE/DDC service about to be called
Read EDID

Performing real mode VBE call
Interrupt 0x10 ax=0x4f15 bx=0x1 cx=0x0
Function supported
Call failed

The EDID data should not be trusted as the VBE call failed

The EDID from get-edid on amd64 systems comes out with a bad checksum! NVIDIA won't load an EDID with a bad checksum! AGGHHH!!

But wait... read-edid provides us with another tool... parse-edid... Normally this will spit out xorg.conf like data regarding your monitor specifics.. but it also has a bit of code that reads the 8 bit checksum.... but it doesn't tell us what the checksum IS... YET.

open up parse-edid.c and navigate to line 249.

this bit:
MSG( "EDID checksum failed - data is corrupt. Continuing anyway." );
Needs to look like this:
printf( "EDID checksum failed - data is corrupt. Continuing anyway. %d", checksum );

And recompile.

Then, when we run parse-edid against the bin we got out of get-edid.. we see this:

EDID checksum failed - data is corrupt. Continuing anyway. 19


That 19 is important. The last hex value in the .bin file needs to NEGATE this number.. It needs to equal -19 as a signed 8 bit integer...

Grab your favorite hex editor and make that very last value in the file equal to the opposite of your checksum value!

wait, what's -19 in hex?

Well, it's an 8 bit checksum.. so.. add -19 to the max positive signed 8 bit value of 256 and we get 237... now convert 237 into hexadecimal..

The Lazy way

So now I stick ED into the last bit of the file, stick in in the xorg.conf, and viola. No more EDID problems. Works just like before.

Piece of cake, right?

Advice regarding my previous post

Just go here and buy this. It's much better than tora... Ok, it's not FOSS, but it's nice.

DB Visualizer