Service processor problems

1. Check the management-module event log for entries that were made around the time that the PS703 or PS704 blade server shut down.
2. Resolve any problems.
3. Remove the blade from the BladeCenter unit and then reinsert the blade server.
4. Power on the blade server.
5. Monitor the blade server operation to verify that the problem is solved.
6. If the BladeCenter unit is functioning normally, but the 1xxx2670 problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. Replace the management card, as described in Removing the tier 2 management card and Installing the tier 2 management card.
2. After replacing the card and installing the blade server in the chassis unit and before rebooting the blade server or performing other operations, ensure that the initialization of the management card VPD occurs by waiting for the management module to discover the blade server. Otherwise, the system might fail to IPL.

1. Reseat the PCIe expansion card.
2. If the problem persists, replace the expansion card.
3. If the problem persists, go to Checkout procedure.
4. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

If the SRC is B1xxB107 or B1xxB108

The system has detected a problem with a clock card.

1. Replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

If the SRC is B1xxB106

The system has detected that the planars are deconfigured.

1. Replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

If the SRC is B1xxB110 or B1xxB111

The system has detected that all of the I/O bridges are deconfigured.

1. Replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. Replace the management card, as described in Removing the tier 2 management card and Installing the tier 2 management card.
2. After replacing the card and installing the blade server in the chassis unit and before rebooting the blade server or performing other operations, ensure that the initialization of the management card VPD occurs by waiting for the management module to discover the blade server. Otherwise, the system might fail to IPL.
3. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. Verify that the operating system is running. If it is running, perform an in-band firmware update, as described in Updating the firmware.
2. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. Verify that the operating system is running. If it is running, perform an in-band firmware update, as described in Updating the firmware.
2. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. Use the chdate command to set the Virtual I/O Server date and time, using one of the following syntaxes:

   chdate [-year YYyy] 
          [-month mm] 
          [-day dd] 
          [-hour HH] 
          [-minute MM] 
          [-timezone TZ] 
   
   
   chdate mmddHHMM[YYyy|yy] 
          [-timezone TZ]

2. If the problem persists, replace the battery, as described in Removing the battery and Installing the battery.

1. If there is only one CRU called out after this FSPSP10 call:
   1. Verify that the CRU is installed, connected, and seated properly.
   2. If the CRU is seated properly and the problem persists, replace the CRU.
   3. If the CRU is missing, add the CRU.
2. If multiple CRUs are called out, they have identical serial numbers. Remove all but one of the CRUs.

1. Verify that the operating system is running. If it is running, perform an in-band firmware update, as described in Updating the firmware.
2. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. View the event log in the management module to locate the system reference code (SRC) and the time that the event was logged. See Error logs.

   If progress codes are being displayed, the server firmware was able to reset the service processor and solve the problem.

2. Record the time the log was created or when you first noticed this SRC.
3. If progress codes are not being displayed, examine the management module event log to see if an A7006995 SRC has been displayed.

   If an A7006995 SRC has been displayed, the blade server is powering off partitions and attempting a server dump. Follow the action in the A7006995 SRC description if the partitions do not terminate as requested.

4. If an A7006995 SRC has not been displayed, has the A1xx SRC remained for more than 40 minutes?

   If so, the server firmware could not begin terminating the partitions. Contact your next level of support to assist in attempting to terminate any remaining partitions and forcing a server dump. Collect the dump for support and power off and power on the blade server.

5. If an A1xx SRC has not remained more than 40 minutes, call IBM Support.

• Look for other serviceable events.
• Use the SRCs that those events call out to determine and fix any problems.

1. Verify that the operating system is running. If it is running, perform an in-band firmware update, as described in Updating the firmware.
2. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. Save the service processor dump to storage by using the partition dump pin control on the control panel.
2. Once the dump is complete, attempt to re-IPL the system.
3. Call IBM Support.

1. Power off the blade server, as described in Turning off the blade server.
2. Remove the blade server from the BladeCenter unit and reinsert the blade server into the BladeCenter unit.
3. Power on the blade server, as described in Turning on the blade server.
4. If the problem persists, replace the CRU that is called out after this procedure.

1. Find another callout that reads "FSPxxxx" where xxxx is a 4-digit hex number that represents the resource ID. Record the resource ID and the model of the system.
2. Call IBM Support to find out what CRU the resource ID represents.
3. Replace the CRU that the resource ID represents.

This error occurred before VPD collection was completed, so no location codes have been created.

1. Replace the management card, as described in Removing the tier 2 management card and Installing the tier 2 management card.
2. After replacing the card and installing the blade server in the chassis unit and before rebooting the blade server or performing other operations, ensure that the initialization of the management card VPD occurs by waiting for the management module to discover the blade server. Otherwise, the system might fail to IPL.
3. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. When the system reaches the SMS, set the system VPD values that are required, which automatically resets the service processor.
2. Power on the blade server, as described in Turning on the blade server.

• Reason code A46F
  1. Verify that the operating system is running. If it is running, perform an in-band firmware update.
  2. If the problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.
  3. If the problem persists, call IBM Support.
• Reason code A460
  1. Set the enclosure serial number using SMS, which automatically resets the service processor.
  2. If the problem persists, call IBM Support.
• Reason code A45F
  1. Set the enclosure feature code using SMS, which automatically resets the service processor.
  2. If the problem persists, call IBM Support.

If you do not see your reason code listed, call IBM Support.

Look for the following error codes in order. Follow the procedure for the first code you find.

SRC B1xx C02A A memory card is missing from the system.

The additional parts in the CRU callout list include all memory cards in the group with the missing card. To correct the error, visually check the system to determine which card is missing, and add the card.

SRC B1xx C029 A memory card is a different type than the other memory cards in the same group.

The additional parts in the CRU callout list include all memory cards in the group that contain the error. To correct the error, exchange the memory cards of the incorrect type with those of the correct type.

SRC B1xx C02B A group of memory cards are missing and are required so that other memory cards on the board can be configured.

The additional parts in the CRU callout list include all the missing memory cards in the group. To correct the error, add or move the memory cards to the correct locations.

SRC B1xx C036 A memory card is not supported in this system.

The additional parts in the CRU callout list include all memory cards in the group that contain the unsupported cards. To correct the error, remove the unsupported cards from the system or replace them with the correct type.

1. Power off the blade server, as described in Turning off the blade server.
2. Remove the blade server from the BladeCenter unit and reinsert the blade server.
3. Power on the blade server, as described in Turning on the blade server.

1. Measure the ambient room temperature to see if it is in within the upper limit of the normal operating range. The upper limit is less than 35 degrees C or 95 degrees F. If the temperature exceeds this limit, you must bring down the room temperature until it is within the limit. When the temperature is within range, retry the operation.
2. If the temperature is within the acceptable range, check the front and rear of the BladeCenter unit to verify that the each is free of obstructions that would impede the airflow. If there are obstructions, you must clear the obstructions. Also clean the air inlets and exits in the BladeCenter unit drawer as required. If you cleared obstructions, retry the operation.
3. Verify that the fans in the BladeCenter unit are working correctly. If not, replace fans that are not turning or that are turning slowly. If you replace fans, wait for the unit to cool and retry the operation.
4. If the cooling components are functioning correctly, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.

1. Using SRC word 6, determine which I/O module has been identified as having a problem:

   0x00000003

   Blade center I/O slot 3

   0x00000004

   Blade center I/O slot 4

   0x00000005

   Blade center I/O slot 7

   0x00000006

   Blade center I/O slot 8

   0x00000007

   Blade center I/O slot 9

   0x00000008

   Blade center I/O slot 10

2. Verify that the I/O module installed in the blade center I/O slot identified by word 6 of the SRC is marked compatible in the blade center advanced management module (AMM) interface. Click System Status > I/O Compatibility.
3. If the I/O module is marked as compatible, continue with the next step. If the I/O module is not marked as compatible, select an I/O module slot that is compatible with the current blade configuration as identified by the AMM. Select the I/O module for Ethernet traffic by clicking Blade Tasks > Configuration > Interface Management > blade > Management Network > Route Traffic Through. This ends the procedure.
4. Reseat the I/O module identified in step 1. If the problem persists, continue with the next step.
5. Reseat the blade system. If the problem persists, continue with the next step.
6. Using SRC word 7, determine the FRU to replace:

   0x00000001

   CIOv form-factor expansion card

   0x00000002

   CFFh expansion card

   0x00000003

   Blade system

   This ends the procedure.

1. Replace the failing CRU:

   DIMM 1 or 17 (Px-C1)

   P1-C1 is memory module 1;

   P2-C1 is memory module 17.

   DIMM 2 or 18 (Px-C2)

   P1-C2 is memory module 2;

   P2-C2 is memory module 18.

   DIMM 3 or 19 (Px-C3)

   P1-C3 is memory module 3;

   P2-C3 is memory module 19.

   DIMM 4 or 20 (Px-C4)

   P1-C4 is memory module 4;

   P2-C4 is memory module 20.

   DIMM 5 or 21 (Px-C5)

   P1-C5 is memory module 5;

   P2-C5 is memory module 21.

   DIMM 6 or 22 (Px-C6)

   P1-C6 is memory module 6;

   P2-C6 is memory module 22.

   DIMM 7 or 23 (Px-C7)

   P1-C7 is memory module 7;

   P2-C7 is memory module 23.

   DIMM 8 or 24 (Px-C8)

   P1-C8 is memory module 8;

   P2-C8 is memory module 24.

   DIMM 9 or 25 (Px-C9)

   P1-C9 is memory module 9;

   P2-C9 is memory module 25.

2. See Removing a memory module for location information and the removal procedure.
3. Install new memory DIMMs, as described in Installing a memory module.

See Supported DIMMs for more information.

1. Replace the failing CRU:

   DIMM 10 or 26 (Px-C10)

   P1-C10 is memory module 10;

   P2-C10 is memory module 26.

   DIMM 11 or 27 (Px-C11)

   P1-C11 is memory module 11;

   P2-C11 is memory module 27.

   DIMM 12 or 28 (Px-C12)

   P1-C12 is memory module 12;

   P2-C12 is memory module 28.

   DIMM 13 or 29 (Px-C13)

   P1-C13 is memory module 13;

   P2-C13 is memory module 29.

   DIMM 14 or 30 (Px-C14)

   P1-C14 is memory module 14;

   P2-C14 is memory module 30.

   DIMM 15 or 31 (Px-C15)

   P1-C15 is memory module 15;

   P2-C15 is memory module 31.

   DIMM 16 or 32 (Px-C16)

   P1-C16 is memory module 16;

   P2-C16 is memory module 32.

2. See Removing a memory module for location information and the removal procedure.
3. Install new memory DIMMs, as described in Installing a memory module.

See Supported DIMMs for more information.

1. Check the management-module event log for entries that indicate a power problem with the BladeCenter unit.
2. Resolve any problems.
3. Remove the blade from the BladeCenter unit and then reinsert the blade server.
4. Power on the blade server.
5. Monitor the blade server operation to verify that the problem is solved.
6. If the BladeCenter unit is functioning normally, but the 1xxx2647 problem persists, replace the system board and chassis assembly, as described in Replacing the FRU system-board and chassis assembly.