Capturing Boot Debug on an IBM Blade

Question
This document can be used to assist in capturing boot debug information which may help determine why a system is hanging during IPL. These steps are valid for AIX 5L and AIX 6.1.



Answer
Enabling debug boot output will cause the debug information to be directed to the console of the blade. In this case we will use ssh to connect to the BladeCenter Management Module, then connect to the particular blade console using SOL (Serial Over Lan).
1. If SOL is not set up to this blade already, please do so before continuing in this document.

2. Configure an SSH client (eg putty) to log session output to a local file on the user's PC.

3. Open a connection to the BladeCenter Management Module and log in an root or another account with administrator functionality.

4. You will be logged in to the CLI at the "system" level. Use the 'list' command to list out hardware at that level and 2 levels down:

system> list -l 2
system
mm[1] primary
power[1]
power[2]
power[3]
power[4]
blower[1]
blower[2]
switch[1]
switch[2]
switch[3]
switch[4]
blade[1] CJT-HS21
blade[2] VMware HS21 XM
blade[3] JS21
blade[4] JS20_2
blade[5] SN#YK339074T12M
blade[6] SN#YK339074T159
blade[7] QS22
blade[8] SN#YK105387R14B
mt[1]

For these next comands you will have to know which blade in the list you are going to operate on.

In my example here we'll pick blade[4].

5. Switch the environment over to that blade:
system> env -T blade[4]
OK
system:blade[4]>

6. Check the current power state of the blade.

system:blade[4]> power -state
Off

If the power is off, we are ready to proceed. If not then power off the blade:

system:blade[4]> power -off

7. Power on the blade and attach to the SOL console:

system:blade[4]> power -on -c
Establishing an sol connection. This may take a few minutes.
E20F
E200
E20B

At this point you will see the LED codes scrolling up the left hand side of the console screen.

8. We now have to stop the boot sequence to get into Open Firmware. As the blades do not have SSM mode, we have to jump directly into Open Firmware.

On some blades you may see the standard "IBM IBM IBM" across the screen at LED E1F1. On others you may only see the LED codes.

At LED E1F1 hit the number 8 key on the keyboard (not on the numeric keypad, but above the letters U and I). This will give you the Open Firmware prompt:

ok
0 >


9. Now boot AIX with the 'verbose' flag enabled to gather debug output. This will track every exec() during the boot cycle and the running system.

0 > boot -s verbose

You should see the cursor spin at the end of the line as the system boots. This output will be captured in the putty log on the PC.

The verbose debug mode will disable automatically at next boot.