Phoenix BIOS Developer Blog

The Amazing Curative Powers Of BIOS...

I was just reading a system review over at techzine.nl and came across this quote which made me chuckle:

...Whether this requires a simple BIOS update or else a rework of the hardware...

How many times has BIOS covered up some nasty hardware problem or motherboard design faux pas? I remember one board I worked on where two of the memory lines had been swizzled. Performance was terrible. But the customer had made a million of the boards. So the good ol' BIOS boys had to go in and switch all of the standard DIMM detection code, just for that one board. Or how about the time when the chipset vendor got the polarity for the NMI disable bit backwards? Or the time that we used periodic SMIs to correct a fault in the real-time clock logic?

Couldn't these be fixed in an OS driver? Often they could. But you can make one BIOS fix or you can make one driver fix per OS. Plus, OEMs have more BIOS engineers than they have driver writers. 

That's BIOS-the band-aid. Or more like duct tape.

Tim

UEFI PI 1.0: Firmware File System

File systems have always fascinated me. My first published software was in an Apple II hacker magazine (Hardcore Computing) , where I rewrote the the way the 'catalog' instruction worked. When FAT12 (the IBM floppy format) came out, I was fascinated by the hard-sectored low-level hardware which didn't require "sync" bytes. During my career at Award I hand-disassembled a new boot sector virus which halted our development for a week because we thought it was a BIOS bug. FAT16? FAT32? NTFS? No problem.

File systems for flash devices, such as those used by the BIOS, are nothing new. Consider the Mars Rover, where NASA scientists were able to download firwmare updates from millions of miles away. But what happens if there is a power failure during the update? You have a very expensive brick. Your PC can be the same way. Ever see those warnings "Don't turn off your PC" when updating the flash? That' s because, if you did, you might lose your BIOS. No BIOS =  Useless PC.

Isn't there a way around this? Sure. That's fault tolerance. Most file systems have some way to recover from power failures to a greater or lesser extent. In some cases, it comes at the expense of performance, since some speed enhancements, like write caching, mean that the physical media (flash, hard disk, etc.) aren't updated immediately. Just go check out the properties of your USB hard drive in the Windows Device Manager, where you can select for "Performance" or "Data Protection"

Flash devices present some interesting problems for file systems. First, the performance is relatively slow (which is why folks want to cache!). Second, writes to a flash part can only change a bit value once before have to erase and re-write some larger portion of the flash device (e.g. 4KB or 64KB). Third, flash devices wear out over time if you do too many writes or, on some devices, too many reads.

Microsoft documents their own variant of the FAT16 file system (called TFAT or Transaction-Safe File Allocation Table) here. The UEFI Platform Initialization Firmware File System specification (available here) also documents a standard file system as well as a means of extending support to other file systems.

But back up a second: Why does UEFI PI need to define a file system at all? Well, interoperability. In order for 3rd parties to be able to smoothly integrate their drivers and applications into a flash device containing Phoenix BIOS, we have to have a file format which everybody recognizes. Otherwise, they would have to recompile their drivers using Phoenix tools.

As I mentioned before, one of the key aspects of file systems is fault-tolerance. If I yank out the power cord while my BIOS is being updated, I still want to the BIOS to survive. For the UEFI PI firmware file system, fault-tolerance is achieved using well defined state transitions. The file system is stable before the state transition and the file system is stable after the state transition. In between the before-state and the after-state are atomic actions, which the file system software assumes will always complete once initiated.

For the UEFI PI firmware file system, there is only one atomic action: write-one-bit. Each state transition is marked by a single bit being changed. In UEFI PI, there are six file states, each represented by a single bit. If more than one bit is set, then only the highest state is considered:

1. File Header Start Write - File system has started writing the file's header. The file header is invalid.
2. File Header End Write - File system has completed writing the file's header and has started writing the file's data. The file header except for the file data checksum is valid but the file's data is not valid.
3. File Data End Write - File system has completed writing the file's data. The file's header and data are valid.
4. File Being Updated - File system is created an updated version of this file. The file's header and data are valid.
5. File Deleted - The file has been deleted.
6. File Invalid - The file's header is invalid.

File Being Updated is particularly interesting. It means that the file system started to create an updated version of this file somewhere in the same file volume. When that updated version has been written completely (File Data Written), then this version will be moved to File Deleted. But what happens if the power fails after the updated version has been written but before this one is deleted? Well, if two versions of the same file are found and one is File Being Updated and the other is File Data Written then the latter is ignored. Usually this state is detected when the firmware volume is mounted and the latter is moved to Deleted.

Here's the typical firmware volume layout:

Next time we'll talk a little bit more about the actual UEFI PI file and volume formats.

Tim

UEFI EFI Byte Code Plugfest

Talk about special purpose plug fests! This plug fest focuses just on the virtual machine architecture which is built into every UEFI machine. This virtual machine was designed so that EFI drivers could be executed on any UEFI compliant platform.  This is especially important for plug-in PCI cards. Today, you either need a hefty ROM on the board to hold the option ROM for all the different platforms you support. Or, you can create a special SKU for each processor architecture you ship on.

EBC itself is a fairly complicated encoding which is more akin to a x86 architecture than anything else. There is only one EBC compiler available, from Intel, for a cool $995 (see product page here). On the side, I'm putting together a translator that converts EBC into a normal assembly language file.

EBC has received some attention recently within UEFI because, as an interpreted byte code, it is a little easier to secure. In the EFI 1.10 specification, you can see some hints of this in the UGA specification because Microsoft originally intended to use the EFI UGA driver as an emergency video driver. The UEFI Security Sub-Team recently revisited it when talking about creating de-privileged driver. It is certainly a lighter-weight approach than, say, user/supervisor (Ring 0/Ring 3) in UEFI.

From the invitation letter:

On behalf of the UEFI Testing Working Group (UTWG) from the Unified EFI Forum (UEFI), Phoenix Technologies invites you to the EFI Byte Code (EBC) Summit. The EBC Summit will be held from February 27 to March 1, 2007 at the Phoenix facility in Milpitas, California USA.

This event provides the industry with an opportunity to conduct focused interoperability testing on the EBC functionality portions of the UEFI 2.0 specification. The EFI Byte Code allows UEFI drivers and applications to execute on a wide range of CPU architectures without recompilation.

This event is open and free to all attendees from companies that are UEFI members, but registration is required. We encourage companies who are not yet UEFI adopters or contributors to sign up at http://www.uefi.org and participate.

Participants in the event include UEFI members who are Add-in card Vendors, Operating System Vendors, Independent Software Vendors, Independent BIOS Vendors and Original Equipment / Design Manufacturers (OEM / ODM) platform vendors.

I'll be there, of course, since it is in my company headquarters :-)  See you there.

Tim

UEFI PI 1.0: The Life Cycle Of A PC

Did you know the PC has a life cycle? No, I don't mean from manufacture to landfill.  No, rather from power-on until power-off.  When the power button is pressed, the average PC begins a journey starting with a flash chip with the BIOS firmware on the motherboard, progressing through the load of your favorite OS flavor and ending with the power being removed, either gracefully or through forcible disconnect from the power grid. 

With only the flash of a logo and a few LED flickers, the PC actually passes through several important phases before the operating system starts. Seldom described in detail, the UEFI PI 1.0 specification (available here) describes these phases in great detail. Each phase has unique characteristics and operational limitations which are essential to understand before trying to drive in and write PI drivers.

This diagram gives an overview of the process. Those in purple (BDS, TSL and RT) are governed by UEFI's UEFI Specification Working Group (USWG). Those in blue (SEC, PEI, DXE & AL) are governed by UEFI's Platform Initialization Working Group (PIWG). Why the distinction? Well, in the original UEFI specification, there was a big black hole called "Platform Initialization" which dealt with stuff before the UEFI Boot Manager was launched. So PIWG was created to fill the black hole.

So here are the different phases:

SEC - The SEC phase (don't ask me why the phase named for Security doesn't actually deal with security, it's a long and tedious story) actually brings the system from reset of the CPU through the time when temporary RAM is available. Temporary RAM is a small amount of memory (say 8KB - 64KB) available to create some global memory for stack and data storage.

PEI - The PEI (Pre-EFI Initialization) phase takes the system up through the point when permanent RAM (i.e. normal DRAM) is available. It also determines the boot mode (normal boot, S3 resume, S4 resume, etc.).

DXE - The DXE (Driver Execution Environment) phase takes the system from the point when memory is available to the point when the firmware is ready to look for a boot device.

BDS - The BDS (Boot Device Selection) phase selects a boot device and boots it. It also usually allows the user to enter Setup and possibly boot the shell or choose a different boot device.

TSL  - The TSL (Transient System Load) phase refers to an alternate class of applications where the system hasn't booted an OS, but it could. For example, there is a shell where numerous applications can be run. When done, you can either return to the BDS phase or proceeed on to RT or just hang around.

RT  - The RT (Run Time) phase refers to the period after an operating system loader has called the ExitBootServices() interface, officially terminating the boot role of the BIOS. After this point, there are still BIOS functions lying around (GetVariable, SetVariable, etc.), but this is primarily the OS landscape.

AL - The AL (Afterlife) is a nebulous phase which refers to times when the firmware actually takes control back from the OS after the OS has shutdown. This happens many times on platforms when entering a low power state (say ACPI's S3, S4,S5).

Well this is just a quick tour. Next time we'll look at how these phases manage all of the different drivers without stepping on toes.

Tim

2TB Hard Drives Force BIOS Changes

Earlier today I put together an external SATA box which can hold 2 500GB drives. The box hooks up to my notebook using either USB 2.0 or eSATA. And then I came across this article at Slashdot here, where Hitachi will ship a 1TB (1,000 gigabytes) drive this year and Seagate predicts a 37TB drive in the next few years. That's my entire DVD collection.

So, what does this have to do with BIOS? Well, it goes back to one of those older data structures used by the BIOS to boot an operating system: the master boot record (MBR). The master boot record keeps track of up to 4 partitions on a hard drive. Each partition can have up to 2^32 blocks, with each logical block having 512 bytes. That gives you a maximum storage limit of ~2TB for a hard drive.

What's the plan? Well, UEFI already had a plan for this in the GUID Partition Table (GPT), which is essentially an alternate to the MBR. So folks said: we could either try to fix the MBR OR we could just use the UEFI stuff.  UEFI allows volumes to be 2^64 blocks (each block is still 512 bytes), which gives you 8,589,934,592TB. That gives us a little more room to grow.

That drive will be sufficient to record my genome and every chess position possible. :-)

Tim

My 2007 Wish List For Drivers

What would I like to see in 2007? Other than a new gaming rig and fun with my kids, you mean?

I must be a BIOS geek, but what I really want are UEFI and PI drivers  (UEFI web site) that are easy to deliver. Over the past month, I've pulled a few all-nighters trying to import silicon vendor drivers into TrustedCore and it is a real pain. Every vendor has a different way of building them and customizing them. But here's my top 5 wish list:

1. Provide X86 and X64 versions.  By the time Longhorn Server comes out, no mainstream processor will be shipping X86 (32-bit) only.  I expect most UEFI implementations to be X64 only. So a pile of 32-bit assembly language is about as useful as my pile of Apple IIgs diskettes. Yes, it means some BIOS engineers are going to have to learn C/C++ or X64.

2. Use smaller source trees.  This week, I saw one driver, less than 4KB, which pulled in files from 400 directories. At Phoenix we call this "directory sprawl"  One driver was nested eight directory levels deep, and that's before it was integrated into our source tree. So when we deliver one driver, we have to pull all the directories and give them to the customer.

3. Simpler customization. UEFI and PI are going in the right direction with binary editability. Already things like dependency expressions, driver names and even user interface can be managed with external tools. Right now, when we put together a system BIOS, we are often integrating drivers from 5 or more vendors. If the only way to customize them is to search through each vendor's source code for the "magic file" it just slows down my ability to get a working system. Worse yet, many vendors require a "magic driver" to be written and don't document what the driver is supposed to do or what the interface looks like. What's the goal: plug-and-play BIOS. Hand me the drivers, make a couple of tweaks related to my motherboard and (poof) the BIOS works. That's the holy grail of BIOS deployment. But we can't get there if everybody hides the tweaks in an obscure source file or readme.

There are a lot of cool things we can do in BIOS (more on that in a future post). But we can't do them if we have to spend all of our time with needle and thread trying to patch together just the basic BIOS.  And what's the use of my new gaming rig if I'm stuck searching for the magic readme file?

Tim