Last week, I wrote an introduction to Architectural Event Trace, an extremely powerful JTAG-based Trace facility within current Intel silicon. Using this technology in conjunction with SourcePoint gives the firmware developer unprecedented insight into program execution and events. This week, I look at some of the use cases.
The cool thing about AET is that it generates debug information in real-time, while the platform is running. As such, it is significantly different from what we normally call “run-control”, which intrusively is used to halt a CPU, set a breakpoint, single-step through code, and such. And it’s also different from Intel Processor Trace, that shows the exact path a core took through a region of code. AET is better suited to seeing the big picture of interactions between cores, the BIOS, device drivers, and board peripherals.
AET uses the Intel Trace Hub, which is logic block with the ability to collect and funnel event data. If you want to learn more about how we program the Trace Hub, have a look here at the Intel Trace Hub Developers Manual. A block diagram looks like this:
The nice thing about the Trace Hub is that it acts as a PCI device, and is thus visible to software. AET, on the other hand, requires probe-mode (JTAG, or run-control) MSRs to be programmed with the desired AET configuration.
Setting up AET in SourcePoint is a two-step process. First, you must configure and enable the Trace Hub to collect and transmit trace data:
Then, set up AET to collect the trace events you want:
You’ll note that we’ve set up the Trace Routing for the Trace Hub to go to “MTB”. What is that? It stands for “Micro Storage Controller Trace Buffer”, and note from the architectural block diagram that it does not show up as one of the Output Ports. MTB is a small 4kB trace buffer in the Trace Hub hardware that allows for trace before system memory is available. This is extremely powerful, as it allows for platform debug before system memory is initialized.
The trick to using the Trace Hub is that, being a PCI device, most BIOS implementations “hide” it past a certain point in the boot process. I’m not exactly sure why, but it may have something to do with OS drivers messing it up past BDS. This is unfortunate, since it makes using it a little less intuitive. So, when I use the Trace Hub, I follow these steps:
- Power on the emulator
- Open SourcePoint
- Power on the target, and stop at the reset vector
- Use JTAG to halt the platform at post code ‘A1’.
- Configure the Trace Hub
- Have fun tracing!
Editor’s Note: some BIOS menus allow you to turn off the “Hide Trace Hub” feature:
Let’s look at these above steps in a little more detail.
In the diagram below, I’ve halted the CPU just past the reset vector:
Then, I use the SourcePoint command language to step through each post code until I hit A1 with this command:
P0> while(ax != 0xa1) {go til 80io; wait; ax}
This really shows off the power of the SourcePoint command language. You can concatenate all of these commands on a single line, or you could put them into a macro and simply invoke it from the command line. What we’re doing is breaking at each Port 80 write, suspending command execution until a breakpoint is encountered, and displaying the value of register AX (the post code number); and continuing until we hit post code x’A1’. Post code A1 is where UPI initialization starts, x’0706’ is where registers are unlocked, and x’B0’ is the start of MRC, then it jumps to BF signifying completion of MRC:
Then, configure the Trace Hub and AET as per the top two diagrams in this blog.
When you configure the Trace Hub, SourcePoint will prompt you:
If you’re lucky (that is, something is not astray with your BIOS, in terms of having access to the Trace Hub), it will come back with a Success message.
Then, it’s time to configure AET. For now, we’ll just configure the MTB to capture MSR and I/O reads/writes, as per the diagram at the top of this article.
Then we run to the next post code with a {go til 80io; wait; ax} and we see the trace output (note: it’s a good idea to disable all Reset and Init breakpoints starting this step):
And this is where the real power comes in; we right-click in the Event Trace window to open up a Tracking Trace window, and then click on each of the black lines in the Event Trace window to see the exact code which is invoking the IN or OUT instruction:
Amazing! It may be that the routine doing the IN or OUT is one we don’t have source code for.
This is just a simple example. Wait until we get to read/write of MSRs, and some of the other powerful events we can trap on.
