IBM AIX 7 Open Beta Program

IBM AIX 7 Open Beta Program
Welcome to the open beta for IBM’s premier UNIX operating system, AIX 7. AIX 7 is binary compatible with previous releases of AIX including AIX 6, 5.3, 5.2 and 5.1. AIX 7 extends the leadership features of AIX to include exciting new capabilities for vertical scalability, virtualization and manageability.
The open beta for AIX 7 is intended to give our clients, independent software vendors and business partners the opportunity to gain early experience with this new release of AIX prior to general availability. This open beta can be run on any Power Systems, IBM System p or eServer pSeries system that is based on POWER4, PPC970, POWER5, POWER6, or POWER7 processors.
Key features of AIX 7 included in this beta:
Virtualization
AIX 5.2 Workload Partitions for AIX 7 - This new enhancement to WPAR technology allows a client to backup an LPAR running AIX V5.2 and restore it into a WPAR running on AIX 7 on POWER7. This capability is designed to allow clients to easily consolidate smaller workloads running on older hardware onto larger, more efficient POWER7 systems. Although this capability is designed specifically for POWER7, it can be tested on older POWER processors during the open beta. Please note that this capability will only work with AIX 5.2
Support for Fibre Channel adapters in a Workload Partition AIX 7 includes support to allow a physical or virtual fibre channel adapter to a WPAR. This allows WPAR to directly own SAN devices including tape devices using the ‘atape” device type. This capability is designed to expand the capabilities of a Workload Partition and simplify management of storage devices.
Security
Domain Support in Role Based Access Control - This enhancement to RBAC allows a security policy to restrict administrative access to a specific set of similar resources, such as a subset of the available network adapters. This allows IT organizations that host services for multiple tenants to restrict administrator access to only the resources associated with a particular tenant. Domains can be used to control access to Volume Groups, Filesystems, files, devices (in /dev)
Manageability
NIM thin server Network Installation Management (NIM) support for thin servers has been enhanced to support NFSV4 and IPv6. Thin Servers are diskless or dataless AIX instances that boot from a common AIX image via NFS.
Networking
Etherchannel enhancements - Support for the 802.3AD Etherchannel has been enhanced to insure that a link is Link Aggregation Control Protocol (LACP) ready before sending data packets.
Product plans referenced in this document may change at any time at IBM’s sole discretion, and are not intended to be a commitment to future product or feature availability. All statements regarding IBM future direction, plans, product names or intent are subject to change or withdrawal without notice and represent goals and objectives only. All information is provided on an as is basis, without any warranty of any kind.

Links for AIX 7.0

IBM AIX 7 Open Beta Program
The following links provide additional valuable resources related to this AIX 7 Open Beta.
AIX 7 On-line Information Center
The official IBM statement on AIX binary compatibility
IBM articles, tutorials, and technical resources for AIX and UNIX users
A full range of IBM POWER System solutions to match your business needs
A full range of IBM POWER System hardware to match your business needs
Discover the POWER of IBM POWER System servers and solutions
PartnerWorld for AIX has resources and support for IBM Business Partners looking to exploit and learn about AIX
A one stop shop to learn about the benefits, resources and support available to IBM Business Partners for IBM Systems, servers and storage

New Features in AIX Version 7

New Features in AIX Version 7

IBM announced AIX version 7. http://www-03.ibm.com/systems/power/software/aix/v71/preview.html
Several new features were mentioned in the launch, but there were two new features that I found particularly interesting:

- AIX 5.2 WPARs for AIX 7
- Cluster Aware AIX

AIX 5.2 WPARs for AIX 7

In AIX version 7, administrators will now have the capability to create Workload Partitions (WPARs) that can run AIX 5.2, inside an AIX 7 operating system instance. This will be supported on the POWER7 server platform. This is pretty cool. IBM have done this to allow some customers, that are unable to migrate to later generations of AIX and Power, to move to POWER7 whilst keeping their legacy AIX 5.2 systems operational. So for those clients that MUST stay on AIX 5.2 (for various reasons such as Application support) but would like to run their systems on POWER7, this feature may be very attractive. It will help to reduce the effort required when consolidating older AIX 5.2 systems onto newer hardware. It may also reduce some of the risk associated with migrating applications from one version of the AIX operating system to another.

To migrate an existing AIX 5.2 system to an AIX 7 WPAR, administrators will first need to take a mksysb of the existing system. Then they can simply restore the mksysb image inside the AIX 7 WPAR. IBM will also offer limited defect and how-to support for the AIX 5.2 operating system in an AIX 7 WPAR. These WPARs can, of course, be managed via IBM Systems Director with the Workload Partitions Manager plug-in.

The following figure provides a visualization of how these AIX 5.2 systems will fit into an AIX 7 WPAR. The WPARs in blue are native AIX 7 WPARs, while the WPARs in orange are AIX 5.2 WPARs running in the same AIX 7 instance. Pretty amazing really!

Cluster Aware AIX

Another very interesting feature of AIX 7 is a new technology known as “Cluster Aware AIX”. Believe it or not, administrators will now be able to create a cluster of AIX systems using features of the new AIX 7 kernel. IBM have introduced this “in built” clustering to the AIX OS in order to simplify the configuration and management of highly available clusters. This new AIX clustering has been designed to allow for:

- The easy creation of clusters of AIX instances for scale-out computing or high availability.
- Significantly simplify cluster configuration, construction, and maintenance.- Improve availability by reducing the time to discover failures.
- Capabilities such as common device naming to help simplify administration.
- Built in event management and monitoring.
- A foundation for future AIX capabilities and the next generation of PowerHA SystemMirror.

This does not replace PowerHA but it does change the way in which AIX traditionally integrates with cluster software like HACMP and PowerHA. A lot of the HA cluster functionality is now available in the AIX 7 kernel itself. However, the mature RSCT technology is still a component of the AIX and PowerHA configuration. I’m looking forward to reading more about this new technology and it’s capabilities.

These are just two of the many features introduced in AIX 7. I’m eagerly looking forward to what these features and others mean for the future of the AIX operating system. It’s exciting to watch this operating system grow and strengthen over time. I can’t wait to get my hands on an AIX 7 system so that I can trial these new features.

And speaking of trialing AIX 7, there is good news. IBM plan on running another AIX Open Beta program for AIX 7 mid 2010. Just as they did with AIX Version 6, customers will be given the opportunity to download a beta version of AIX 7 and trial it on their own systems in their own environment. This is very exciting and I’m really looking forward to it.

=================================================================

Clustering infrastructureAIX 7 (which some are calling Cluster Aware AIX) will be the first AIX release that will provide for built-in clustering. This promises to simplify high-availability application management with PowerHA SystemMirror.
It should be noted: This innovation isn’t being targeted as a replacement of PowerHA, but it’s supposed to change the way in which AIX integrates with it. Much of the PowerHA cluster functionality will now be available in the actual kernel. It’s simply designed to more easily construct and manage clusters for scale-out and high-availability applications.
Furthermore, AIX 7 will have features that will help reduce the time to discover failures, along with common device naming, to help systems administrators simplify cluster administration. It will also provide for event management and monitoring.
I am excited about this tighter integration between PowerHA and AIX, because anything that provides greater transparency between high-availability software and the OS further eases the burden of system administrators who architect, install and configure high-availability software.
Vertical ScalabilityAIX 7 will allow you to scale up to 1,024 threads or 256 cores in a single partition. This is simply outstanding; No other Unix OS can come close to this.
Profile-Based Configuration ManagementIBM Systems Director enhancements will simplify AIX systems-configuration management. IBM is calling this facility profile-based configuration management.
At a high level it’ll provide simplified discovery, application, update and AIX configuration-verification properties across multiple systems. It’ll be particularly helpful in terms of cloning out changes to ‘pools’ of systems. After populating a profile into a file (XML), it can then be deployed to the other servers in the pool (see Figure 1).
AIX 5.2 and WPARsAIX 7 will now provide the capability to run AIX 5.2 inside of a Workload Partition (WPAR). This will allow for further IT consolidation and flexible deployment opportunities (such as moving up to the POWER7 architecture) to folks who are still on older AIX OSs. In an easy way, it also allows you to backup an existing environment and restore it inside an AIX 7 WPAR. Furthermore, it will allow you to do this through IBM Systems Director’s Workload Partitions Manager.
I’m particularly impressed with this feature. Most companies look to discontinue support for their older operating systems as soon as they can. On the other hand, IBM continues to listen to their customers and provide additional features to folks on older versions of their systems. For example, AIX 7 will also support older hardware, including POWER4 processor-based servers. While this type of compatibility is critical to those who want to take advantage of the feature/functionality improvements of AIX but can’t afford to upgrade their hardware, it should also be noted AIX 7 will include exploitation features that take full advantage of POWER7 processor-based servers. Additionally, AIX 7 will have full binary compatibility for application programs developed on prior versions of AIX—as long as these programs comply with reasonable programming standards.

AIX7 WPAR support

AIX7 WPAR support
Besides adding AIX 5.2 support to WPAR's (workload partitions) AIX7 is also adding more virtual device support and security to the WPAR virtualization engine.
AIX WPAR support will add Fibre Channel support- or exporting a virtual (NPIV) or physical fibre channel adapter.Fibre channel tape systems using the "atape" driver are also
supported inside the WPAR in this configuration.
With the next releases AIX, VIO SCSI disks are now supported in a WPAR in the same manner as Fibre Channel disks. This feature is available on both AIX V7.1 and AIX V6.1 with the 6100-06 Technology Level.
Trusted Kernel Extension Loading and Config from WPAR (AIX 7.1 Only)
AIX V7.1 provides the capability for a Global administrator to export specific kernel extensions for a WPAR administrator to have the ability to load and configure from inside the WPAR.

Workload Management in AIX: WLM, DLPAR and now WPAR

Workload Manager - WLM
Over the years several methods of Workload Management have been developed as means to enhance resource utilization of systems. Some might say the Workload Management is a form of Performance Management – but that is only true in that Performance Management is actually Resource Management. In this sense, Workload Management is the collection of services and resource management applications that are used to monitor and regulate the resources a workload is permitted at any particular time.
Legacy UNIX systems had a very simple model of workload resource management. This was also known as sizing the box. Generally, the workload was the collection of all applications or processes running on the box. Various tools could be used – either system or application tools – to tune the application(s) to best fit the box. In other words, the amount of resources available to the workload was constant – whatever the box had.
With the release of AIX 4.3.3 in 1999 AIX included a new system component – AIX Workload Manager (WLM). This component made it possible to define collections of applications and processes into a workload, or workload class. A workload class was given resource entitlement (CPU, Memory, and starting with AIX 5.1 local I/O) in terms of shares. If there were four (4) classes active, and a resource was saturated (being used 100%) then AIX would compute a resource entitlement percentage based on the total active shares. If the four (4) classes had, respectively 15, 30, 45, and 60 shares the classes would be entitled to respectively – 10, 20, 30 and 40% of the resource concerned. As long as a resource was not constrained (less than 100% usage) WLM, by default, would not restrict a class resource entitlement.
The primary advantages of WLM are that it is included in the AIX base at no extra charge and is a software solution requiring no special hardware. However, performance specialists seem to have found it difficult to think in terms of performance management on a system which is regularly going to need more resources than it has. In other words, the legacy UNIX workload management model dominates most system administrations view of resource management.
Firmware Partitioning as Resource Management
Parallel with the development of WLM, a software solution for workload resource monitoring and control, the use of dividing a single system in to several separate system definitions commonly referred to as partitions. Virtualization in UNIX had become. Unlike WLM, partitioning required specific hardware features. For AIX, partitioning was introduced with the p690 POWER4 system.
Partitioning is a technique used to define multiple systems from a single system. A partition is allocated a specific amount of resources that it has to use as it desires. Individual partitions resources are isolated via firmare (Logical Partitions, or LPAR) or by the hardware component assembly (Physical Partition, or PPAR).
Initially, resource assignment was static. To change the resource allocation a halt and a (re)activation of the partition was required. Starting with AIX 5.2 the acronym DLPAR (dynamic LPAR) was introduced. This enhancement enables dynamic resource allocation to a partition, that is, a partition can have it's allocation of resources increased or decreased without a system, i.e. partition, halt and reactivation. With POWER5 the resource virtualization continued with the introduction of the firmware hypervisor, micropartitions, virtual Ethernet and virtual SCSI.
The advantages of partitioning are the flexibility in allocation of resources and the isolation quaranteed by the hypervisor firmware. However, partitioning requires specific hardware. Also, an administrator needs extra training to create and manage partition resources.
AIX 6.1 introduces Workload Partitions
A workload partition is a virtual system environment created using software tools. A workload partition is hosted by an AIX software environment. Applications and users working within the workload partition see the workload partition as if it was a regular system. Although less than a firmware created and managed partition – workload partition processes, signals and even file systems are isolated from the hosting environment as well as from other workload partitions. Additionally, workload partitions can have their own users, groups and dedicated network addresses. Interpprocess communication is limited to processes running within the workload partition.
AIX supports two kinds of Workload Partiions (WPARs).
A System WPAR is an environment that can be best compared to a stand-alone system. This WPAR runs it owns services and does not share writeable file systems with another WPAR or the AIX hosting (global) system.
An Application WPAR has all the process isolation a system WPAR has. The defining charactgeristic is that it shares file system name space with the global system and applications defined within the application WPAR environment.
Both types of WPARs can be configured for mobility to allow running insttances of the WPAR to be moved between physical systems or LPARs using the AIX Workload Partitin Manager LPP.
Summary
With the addition of WPAR (workload partitions) to AIX workload management has an intermediate level of flexibility and isolation of applications, users and data. Using WLM all process share the same environment with only CPU, memory and I/O resource allocation being managed when a resource is saturated. Firmware based virtualization of partitions starting with POWER4 hardware provides both hard allocation resource levels as well as complete isolation of services, network addresses, devices, etc. from all other partitions. Workload partitions, or WPAR, are a software based virtualization of partitions supporting a high degree of isolation and enhanced mobility over supporting global systems.

Powerpath CLI Commands

Powerpath CLI Commands
Command
Description
powermt
Manages a PowerPath environment
powercf
Configures PowerPath devices
emcpreg -install
Manages PowerPath license registration
emcpminor
Checks for free minor numbers
emcpupgrade
Converts PowerPath configuration files
powermt command
Command
Description
powermt check
Checks for, and optionally removes, dead paths.
powermt check_ registration
Checks the state of the PowerPath license.
powermt config
Configures logical devices as PowerPath devices.
powermt display
powermt watch
Displays the state of HBAs configured for PowerPath.
powermt watch is deprecated.
powermt display options
Displays the periodic autorestore setting.
powermt load
Loads a PowerPath configuration.
powermt remove
Removes a path from the PowerPath configuration.
powermt restore
Tests and restores paths.
powermt save
Saves a custom PowerPath configuration.
powermt set mode
Sets paths to active or standby mode.
powermt set
periodic_autorestore
Enables or disables periodic autorestore.
powermt set policy
Changes the load balancing and failover policy.
powermt set priority
Sets the I/O priority
powermt version
Returns the number of the PowerPath version for which powermt was created.
powermt command examples
powermt display: # powermt display paths class=all
# powermt display ports dev=all
# powermt display dev=all
powermt set: To disable a HBA from passing I/O # powermt set mode=standby adapter=
To enable a HBA from passing I/O # powermt set mode=active adapter=
To set or validate the Load balancing policy
To see the current load-balancing policy and I/Os run the following command
# powermt display dev=
so = Symmetrix Optimization (default)
co = Clariion Optimization
li = Least I/Os (queued)
lb = Least Blocks (queued)
rr = Round Robin (one path after another)
re = Request (failover only)
nr = No Redirect (no load-balancing or failover)
To set to no load balancing # powermt set policy=nr dev=
To set the policy to default Symmetrix Optimization # powermt set policy=so dev=
To set the policy to default Clariion Optimization # powermt set policy=co dev=
pprootdev
To bring the rootvg devices under powerpath control # pprootdev on
To bring back the rootvg disks back to hdisk control # pprootdev off
To temporarily bring the rootvg disks to hdisk control for running "bosboot" # pprootdev fix
powermt command examples with output
To validate the installation # powermt check_registration
Key B3P3-HB43-CFMR-Q2A6-MX9V-O9P3
Product: PowerPath
Capabilities: Symmetrix CLARiiON
To display each device's path, state, policy and average I/O information
# powermt display dev=emcpower6a
Pseudo name=emcpower6a
Symmetrix ID=000184503070
Logical device ID=0021
state=alive; policy=SymmOpt; priority=0; queued-IOs=0

---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors

0 sbus@2,0/fcaw@2,0 c4t25d225s0 FA 13bA active dead 0 1
1 sbus@6,0/fcaw@1,0 c5t26d225s0 FA 4bA active alive 0 0
To show the paths and dead paths to the storage port
# powermt display paths
Symmetrix logical device count=20

----- Host Bus Adapters --------- ------ Storage System ----- - I/O Paths -
### HW Path ID Interface Total Dead

0 sbus@2,0/fcaw@2,0 000184503070 FA 13bA 20 20
1 sbus@6,0/fcaw@1,0 000184503070 FA 4bA 20 0
CLARiiON logical device count=0

----- Host Bus Adapters --------- ------ Storage System ----- - I/O Paths -
### HW Path ID Interface Total Dead

To display the storage ports information
# powermt display ports
Storage class = Symmetrix

----------- Storage System --------------- -- I/O Paths -- --- Stats ---
ID Interface Wt_Q Total Dead Q-IOs Errors

000184503070 FA 13bA 256 20 20 0 20
000184503070 FA 4bA 256 20 0 0 0
Storage class = CLARiiON

----------- Storage System --------------- -- I/O Paths -- --- Stats ---
ID Interface Wt_Q Total Dead Q-IOs Errors

Moving an LPAR to another frame

Moving an LPAR to another frame

Steps for migrating LPAR from ONE Frame to Another IBM Frame

1.Have Storage zone the LPARs disk to the new HBA(s). Also have them add an additional 40GB drive for the new boot disk. By doing this we have a back out to the old boot disk on the old frame.

2. Collect data from the current LPAR:

a. Network information – Write down IP and ipv4 alias(s) for each interface

b. Run “oslevel –r” - will need this when setting up NIM for the mksysb recovery

c. Is the LPAR running AIO, if so will need to configure after the mksysb recovery

d. Run “lspv”, save this output, contains volume group and PVID information

e. Any other customizations you deem neccessary


3. create mksysb backup of this LPAR

4. Reconfigure the NIM machine for this LPAR, with new Ethernet MAC address. Foolproof method is to remove the machine and re-create it.

5. In NIM, configure the LPAR for a mksysb recovery. Select the appropriate SPOT and LPP Source, base on “oslevel –r” data collected in step 2.

6. Shut down the LPAR on the old frame (Halt the LPAR)

7. Move network cables, fibre cables, disk, zoning

8. if needed, to the LPAR on the new frame

9. On the HMC, bring up the LPAR on the new frame in SMS mode and select a network boot. Verify SMS profile has only a single HBA (if Clarrion attached, zoned to a single SP), otherwise the recovery will fail with a 554.

10. Follow prompts for building a new OS. Select the new 40GB drive for the boot disk (use lspv info collected in Step 2 to identify the correct 40GB drive). Leave defaults for remaining questions NO (shrink file systems, recover devices, and import volume groups).

11. After the LPAR has booted, from the console (the network interface may be down):

a. lspv Note the hdisk# of the bootdisk

b. bootlist –m normal –o Verify boot list is set – if not, set it

bootlist –m normal –o hdisk#

c. ifconfig en0 down If interface got configured, down it

d. ifconfig en0 detach and remove it


e. lsdev –Cc adapter Note Ethernet interfaces (ex. ent0, ent1)

f. rmdev –dl Remove all en devices

g. rmdev –dl Remove all ent devices

h. cfgmgr Will rediscover the en/ent devices

i. chdev –l -a media_speed=100_Full_Duplex Set on each interface unless

running GIG, leave defaults


j. Configure the network interfaces and aliases Use info recorded from step 2 mktcpip –h -a -m -i -g -A no –t N/A –s

chdev –l en# -a alias4=,


k. Verify that the network is working.


12. If LPAR was running AIO (data collected in Step 2), verify it is running (smitty aio)

13. Check for any other customizations which may have been made on this LPAR

14. Vary on the volume groups, use the “lspv” data collected in Step 2 to identify by PVID a hdisk in each volume group. Run for each volume group:

a. importvg –y hdisk# Will vary on all hdisk in the volume group

b. varyonvg

c. mount all Verify mounts are good

15. Verify paging space is configured appropriately

a. lsps –a Look for Active and Auto set to yes

b. chps –ay pagingXX Run for each paging space, sets Auto

c. swapon /dev/pagingxx Run for each paging space, sets Active


16. Verify LPAR is running 64 bit

a. bootinfo –K If 64, you are good


b. ln –sf /usr/lib/boot/unix_64 /unix If 32, change to run 64 bit

c. ln –sf /usr/lib/boot/unix_64 /usr/lib/boot/unix

d. bosboot –ak /usr/lib/boot/unix_64


17. If LPAR has Power Path

a. Run “powermt config” Creates the powerpath0 device

b. Run “pprootdev on” Sets Power Path control of the boot disk

c. If Clariion, make configuration changes to enable SP failover


chdev -l powerpath0 -Pa QueueDepthAdj=1

chdev –l fcsX –Pa num_cmd_elems=2048 For each fiber adapter

chdev –l fscsiX –Pa fc_err_recov=fast_fail For each fiber adapter

d. Halt the LPAR

e. Activate the Normal profile If Sym/DMX – verify two HBA’s in profile

f. If Clarrion attached, have Storage add zone to 2nd SP

i. Run cfgmgr Configure the 2nd set of disk


g. Run “pprootdev fix” Put rootdisk pvid’s back on hdisk

h. lspv grep rootvg Get boot disk hdisk#

i. bootlist –m normal –o hdisk# hdisk# Set the boot list with both hdisk


20. From the HMC, remove the LPAR profile from the old frame

21. Pull cables from the old LPAR (Ethernet and fiber), deactivate patch panel ports

22. Update documentation, Server Master, AIX Hardware spreadsheet, Patch Panel spreadsheet

23. Return the old boot disk to storage.

Unique VLAN ID for SEA failover control channel setup

Unique VLAN ID for SEA failover control channel setup

Always select unique VLAN ID – which dosn’t exist on any of your organization network to avoid conflict when setting up dual VIOS with a control channel for SEA failover.. failure to follow this may result in a network storm. ( Very important and I couldn’t find any note on IBM site about it)


Requirements for Configuring SEA Failover

One SEA on one VIOS acts as the primary (active) adapter and the second SEA on the second VIOS acts as a backup (standby) adapter.
Each SEA must have at least one virtual Ethernet adapter with the “Access external network” flag (previously known as “trunk” flag) checked. This enables the SEA to provide bridging functionality between the two VIO servers.
This adapter on both the SEAs has the same PVID, but will have a different priority value.
A SEA in ha_mode (Failover mode) might have more than one trunk adapters, in which case all should have the same priority value.
The priority value defines which of the two SEAs will be the primary and which will be the backup. The lower the priority value, the higher the priority, e.g. an adapter with priority 1 will have the highest priority.
An additional virtual Ethernet adapter , which belongs to a unique VLAN on the system, is used to create the control channel between the SEAs, and must be specified in each SEA when configured in ha_mode.
The purpose of this control channel is to communicate between the two SEA adapters to determine when a fail over should take place.

Upgrading PowerPath in a dual VIO server environment

Upgrading PowerPath in a dual VIO server environment

When upgrading PowerPath in a dual Virtual I/O (VIO) server environment, the devices need to be unmapped in order to maintain the existing mapping information.

To upgrade PowerPath in a dual VIO server environment:
1. On one of the VIO servers, run lsmap -all.
This command displays the mapping between physical, logical,
and virtual devices.

$ lsmap -all
SVSA Physloc Client Partition ID
————— ————————————– ——————–
vhost1 U8203.E4A.10B9141-V1-C30 0×00000000
VTD vtscsi1
Status Available
LUN 0×8100000000000000
Backing device hdiskpower5
Physloc U789C.001.DQD0564-P1-C2-T1-L67

2. Log in on the same VIO server as the padmin user.

3. Unconfigure the PowerPath pseudo devices listed in step 1 by
running:
rmdev -dev -ucfg
where is the virtual target device.
For example, rmdev -dev vtscsil -ucfg
The VTD status changes to Defined.
Note: Run rmdev -dev -ucfg for all VTDs displayed in step 1.

4. Upgrade PowerPath

=======================================================================

1. Close all applications that use PowerPath devices, and vary off all
volume groups except the root volume group (rootvg).

In a CLARiiON environment, if the Navisphere Host Agent is
running, type:
/etc/rc.agent stop

2. Optional. Run powermt save in PowerPath 4.x to save the
changes made in the configuration file.

Run powermt config.
5. Optional. Run powermt load to load the previously saved
configuration file.
When upgrading from PowerPath 4.x to PowerPath 5.3, an error
message is displayed after running powermt load, due to
differences in the PowerPath architecture. This is an expected
result and the error message can be ignored.
Even if the command succeeds in updating the saved
configuration, the following error message is displayed by
running powermt load:
host1a 5300-08-01-0819:/ #powermt load Error loading auto-restore value
Warning:Error occurred loading saved driver state from file /etc/powermt.custom

host1a 5300-08-01-0819:/ #powermt load Error loading auto-restore value
Warning:Error occurred loading saved driver state from file /etc/powermt.custom
…
Loading continues…
Error loading auto-restore value
When you upgrade from an unlicensed to a licensed version of
PowerPath, the load balancing and failover device policy is set to
bf/nr (BasicFailover/NoRedirect). You can change the policy by
using the powermt set policy command.

=======================================================================

5. Run powermt config.

6. Log in as the padmin user and then configure the VTD
unconfigured from step 3 by running:
cfgdev -dev
Where is the virtual target device.
For example, cfgdev -dev vtscsil
The VTD status changes to Available.
Note: Run cfgdev -dev for all VTDs unconfigured in step 3

7. Run lspath -h on all clients to verify all paths are Available.

8. Perform steps 1 through 7 on the second VIO server.

Recovering emc dead path

# powermt display dev=allAnd you notice that there are "dead" paths, then these are the commands to run in order to set these paths back to "alive" again, of course, AFTER ensuring that any SAN related issues are resolved. To have PowerPath scan all devices and mark any dead devices as alive, if it finds that a device is in fact capable of doing I/O commands, run:
# powermt restoreTo delete any dead paths, and to reconfigure them again:
# powermt reset# powermt configOr you could run:
# powermt check

EMC - MPIO

You can run into an issue with EMC storage on AIX systems using MPIO (No Powerpath) for your boot disks:After installing the ODM_DEFINITONS of EMC Symmetrix on your client system, the system won't boot any more and will hang with LED 554 (unable to find boot disk). The boot hang (LED 554) is not caused by the EMC ODM package itself, but by the boot process not detecting a path to the boot disk if the first MPIO path does not corresponding to the fscsiX driver instance where all hdisks are configured. Let me explain that more in detail: Let's say we have an AIX system with four HBAs configured in the following order:

# lscfg -v grep fcsfcs2 (wwn 71ca) -> no devices configured behind this fscsi2 driver instance

(path only configured in CuPath ODM table)fcs3 (wwn 71cb) -> no devices configured behind this fscsi3 driver instance (path only configured in CuPath ODM table)fcs0 (wwn 71e4) -> no devices configured behind this fscsi0 driver instance (path only configured in CuPath ODM table)fcs1 (wwn 71e5) -> ALL devices configured behind this fscsi1 driver instance Looking at the MPIO path configuration, here is what we have for the rootvg disk:

# lspath -l hdisk2 -H -F"name parent path_id connection status" name parent path_id connection status hdisk2 fscsi0 0 5006048452a83987,33000000000000 Enabled hdisk2 fscsi1 1 5006048c52a83998,33000000000000 Enabled hdisk2 fscsi2 2 5006048452a83986,33000000000000 Enabled hdisk2 fscsi3 3 5006048c52a83999,33000000000000 Enabled The fscsi1 driver instance is the second path (pathid 1), then remove the 3 paths keeping only the path corresponding to fscsi1 :

# rmpath -l hdisk2 -p fscsi0 -d # rmpath -l hdisk2 -p fscsi2 -d # rmpath -l hdisk2 -p fscsi3 -d # lspath -l hdisk2 -H -F"name parent path_id connection status"Afterwards, do a savebase to update the boot lv hd5. Set up the bootlist to hdisk2 and reboot the host. It will come up successfully, no more hang LED 554. When checking the status of the rootvg disk, a new hdisk10 has been configured with the correct ODM definitions as shown below:

# lspv hdisk10 0003027f7f7ca7e2 rootvg active # lsdev -Cc disk hdisk2 Defined 00-09-01 MPIO Other FC SCSI Disk Drive hdisk10 Available 00-08-01 EMC Symmetrix FCP MPIO Raid6 To summarize, it is recommended to setup ONLY ONE path when installing an AIX to a SAN disk, then install the EMC ODM package then reboot the host and only after that is complete, add the other paths. Dy doing that we ensure that the fscsiX driver instance used for the boot process has the hdisk configured behind

EMC Grab

EMC GRAB

EMC Grab is a utility that is run locally on each host and gathers storage-specific information (driver version, storage-technical details, etc). The EMC Grab report creates a zip file. This zip file can be used by EMC support. You can download the "Grab Utility" from the following locations:
ftp://ftp.emc.com/pub/emcgrab/Unix/ When you�ve downloaded EMCgrab, and stored in a temporary location on the server like /tmp/emc, untar it using:
tar -xvf *tarThen run:
/tmp/emc/emcgrab/emcgrab.shThe script is interactive and finishes after a couple of minutes.

RESET RESERVATION BIT

If you run into not being able to access an hdiskpowerX disk, you may need to reset the reservation bit on it:
# /usr/lpp/EMC/Symmetrix/bin/emcpowerreset fscsiX hdiskpowerX

EMC PowerPath key installation

EMC PowerPath key installation
This describes how to configure the EMC PowerPath registration keys.

First, check the current configuration of PowerPath:

# powermt config
Warning: all licenses for storage systems support are missing or expired.
The install the keys:
# emcpreg -install

=========== EMC PowerPath Registration ===========
Do you have a new registration key or keys to enter?[n] y
Enter the registration keys(s) for your product(s),
one per line, pressing Enter after each key.
After typing all keys, press Enter again.

Key (Enter if done): P6BV-4KDB-QET6-RF9A-QV9D-MN3V
1 key(s) successfully added.
Key successfully installed.

Key (Enter if done):
1 key(s) successfully registered.
(Note: the license key used in this example is not valid).

EMC ODM definations cleanup

Before making any changes, collect host logs to document the current configuration. At a minimum, save the following: inq, lsdev -Cc disk, lsdev -Cc adapter, lspv, and lsvg
Shutdown the application(s), unmount the file system(s), and varyoff all volume groups except for rootvg. Do not export the volume groups.

# varyoffvg Check with lsvg -o (confirm that only rootvg is varied on)If no

PowerPath, skip all steps with power names.
For CLARiiON configuration, if Navisphere Agent is running, stop it:
# /etc/rc.agent stop
Remove paths from Powerpath configuration:
# powermt remove hba=all

Delete all hdiskpower devices:
# lsdev -Cc disk -Fname grep power xargs -n1 rmdev -dl
Remove the PowerPath driver instance:
# rmdev -dl powerpath0
Delete all hdisk devices:For Symmetrix devices, use this command:
# lsdev -CtSYMM* -Fname xargs -n1 rmdev -dlFor CLARiiON devices, use this command:
# lsdev -CtCLAR* -Fname xargs -n1 rmdev -dl
Confirm with lsdev -Cc disk that there are no EMC hdisks or hdiskpowers.
Remove all Fiber driver instances:
# rmdev -Rdl fscsiX(X being driver instance number, i.e. 0,1,2, etc.)
Verify through lsdev -Cc driver that there are no more fiber driver instances (fscsi).
Change the adapter instances in Defined state
# rmdev -l fcsX(X being adapter instance number, i.e. 0,1,2, etc.)
Create the hdisk entries for all EMC devices:
# emc_cfgmgror
# cfgmgr -vl fcsx(x being each adapter instance which was rebuilt). Skip this part if no PowerPath.
Configure all EMC devices into PowerPath:
# powermt config
Check the system to see if it now displays correctly:
# powermt display# powermt display dev=all# lsdev -Cc disk# /etc/rc.agent start

Configuring MPIO for Virtual AIX client

Configuring MPIO for the virtual AIX client
This document describes the procedure to set up Multi-Path I/O on the AIX clients of
the virtual I/O server.
Procedure:
This procedure assumes that the disks are already allocated to both the VIO servers
involved in this configuration.
· Creating Virtual Server and Client SCSI Adapters
First of all, via HMC create SCSI server adapters on the two VIO servers and
then two virtual client SCSI adapters on the newly created client partition, each
mapping to one of the VIO servers´ server SCSI adapter.
An example:
Here is an example of configuring and exporting an ESS LUN from both the
VIO servers to a client partition:
· Selecting the disk to export
You can check for the ESS LUN that you are going to use for MPIO by
running the following command on the VIO servers.
On the first VIO server:
$ lsdev -type disk
name status description
..
hdisk3 Available MPIO Other FC SCSI Disk Drive
hdisk4 Available MPIO Other FC SCSI Disk Drive
hdisk5 Available MPIO Other FC SCSI Disk Drive
..
$lspv
..
hdisk3 00c3e35c99c0a332 None
hdisk4 00c3e35c99c0a51c None
hdisk5 00c3e35ca560f919 None
..
In this case hdisk5 is the ESS disk that we are going to use for MPIO.
Then run the following command to list the attributes of the disk that you choose for MPIO:
$lsdev -dev hdisk5 -attr
..
algorithm fail_over Algorithm True
..
lun_id 0x5463000000000000 Logical Unit Number ID False
..
..
pvid 00c3e35ca560f9190000000000000000 Physical volume identifier
False
..
reserve_policy single_path Reserve Policy True
Note down the lun_id, pvid and the reserve_policy of the hdisk4.
· Command to change reservation policy on the disk
You see that the reserve policy is set to single_path.
Change this to no_reserve by running the following command:
$ chdev -dev hdisk5 -attr reserve_policy=no_reserve
hdisk4 changed
On the second VIO server:
On the second VIO server too, find the hdisk# that has the same pvid, it could
be a different one than the one on the first VIO server, but the pvid should the
same.
$ lspv
..
hdisk7 00c3e35ca560f919 None
..
The pvid of the hdisk7 is the same as the hdisk5 on the first VIO server.
$ lsdev -type disk
name status description
..
hdisk7 Available MPIO Other FC SCSI Disk Drive
..
$lsdev -dev hdisk7 -attr
..
algorithm fail_over Algorithm True
..
lun_id 0x5463000000000000 Logical Unit Number ID False
..
pvid 00c3e35ca560f9190000000000000000 Physical volume identifier
False
..
reserve_policy single_path Reserve Policy True
You will note that the lun_id, pvid of the hdisk7 on this server are the same as
the hdisk4 on the first VIO server.
$ chdev -dev hdisk7 -attr reserve_policy=no_reserve
hdisk6 changed
· Creating the Virtual Target Device
Now on both the VIO servers run the mkvdev command using the appropriate
hdisk#s respectively.
$ mkvdev -vdev hdisk# -vadapter vhost# -dev vhdisk#
The above command might have failed when run on the second VIO server, if
the reserve_policy was not set to no_reserve on the hdisk.
After the above command runs succesfully on both the servers, we have
same LUN exported to the client with mkvdev command on both servers.
· Check for correct mapping between the server and the client

Double check the client via the HMC that the correct slot numbers match the
respective slot numbers on the servers.
In the example, the slot number 4 for the client virtual scsi adapter maps to
slot number 5 of the VIO server VIO1_nimtb158.
And the slot number 5 for the client virtual SCSI adapter maps to the slot
number 5 of the VIO server VIO1_nimtb159.
· On the client partition
Now you are ready to install the client. You can install the client using any of
the following methods described in the red book on virtualization at
http://www.redbooks.ibm.com/redpieces/abstracts/sg247940.html:
1. NIM installation
2. Alternate disk installation
3. using the CD media
Once you install the client, run the following commands to check for MPIO:
# lsdev -Cc disk
hdisk0 Available Virtual SCSI Disk Drive
# lspv
hdisk0 00c3e35ca560f919 rootvg active
# lspath
Enabled hdisk0 vscsi0
Enabled hdisk0 vscsi1
· Dual Path
When one of the VIO servers goes down, the path coming from that server
shows as failed with the lspath command.
# lspath
Failed hdisk0 vscsi0
Enabled hdisk0 vscsi1
· Path Failure Detection
The path shows up in the "failed" mode, even after the VIO server is up
again. We need to either change the status with the “chpath” command to
“enabled” state or set the the attributes “hcheck_interval” and “hcheck_mode” to
“60” and “nonactive” respectively for a path failure to be detected automatically.
· Setting the related attributes
Here is the command to be run for setting the above attributes on the client
partition:
$ chdev -l hdisk# -a hcheck_interval=60 –a hcheck_mode=nonactive -P
The VIO AIX client needs to be rebooted for hcheck_interval attribute to take
effect.
· EMC for Storage
In case of using EMC device as the storage device attached to VIO server,
then make sure of the following:
1. Powerpath version 4.4. is installed on the VIO servers.
2. Create hdiskpower devices which are shared between both the VIO
servers.
· Additional Information
Another thing to take note of is that you cannot have the same name for
Virtual SCSI Server Adapter and Virtual Target Device. The mkvdev command
will error out if the same name for both is used.
$ mkvdev -vdev hdiskpower0 -vadapter vhost0 -dev hdiskpower0
Method error (/usr/lib/methods/define -g -d):
0514-013 Logical name is required.
The reserve attribute is named differently for an EMC device than the attribute
for ESS or FasTt storage device. It is “reserve_lock”.
Run the following command as padmin for checking the value of the
attribute.
$ lsdev -dev hdiskpower# -attr reserve_lock
Run the following command as padmin for changing the value of the attribute.
$ chdev -dev hdiskpower# -attr reserve_lock=no
· Commands to change the Fibre Channel Adapter attributes
And also change the following attributes of the fscsi#, fc_err_recov to “fast_fail”
and dyntrk to “yes”
$ chdev -dev fscsi# -attr fc_err_recov=fast_fail dyntrk=yes –perm
The reason for changing the fc_err_recov to “fast_fail” is that if the Fibre
Channel adapter driver detects a link event such as a lost link between a storage
device and a switch, then any new I/O or future retries of the failed I/Os will be
failed immediately by the adapter until the adapter driver detects that the device
has rejoined the fabric. The default setting for this attribute is 'delayed_fail’.
Setting the dyntrk attribute to “yes” makes AIX tolerate cabling changes in the
SAN.
The VIOS needs to be rebooted for fscsi# attributes to take effect

Delayed SEA failover

Question
Why is my Shared Ethernet Adapter (SEA) failover delayed?



Cause
Spanning tree is turned on at the switch port with portfast disabled.


Answer
SEA failover, from primary to backup, the delay is sometimes due to Spanning Tree Protocol being enabled on the switch ports.
To ensure prompt recovery times when you enable the Spanning Tree Protocol on the switch ports connected to the physical adapters of the Shared Ethernet Adapter, you can also enable the portfast option on those ports. The portfast option allows the switch to immediately forward packets on the port without first completing the Spanning Tree Protocol. (Spanning Tree Protocol blocks the port completely until it is finished.)On SEA failback, from backup to primary, there is an additional issue:

The switches are sometimes not ready to read transmit and receive packets even after declaring the link as up. Hence, it leads to packet loss. This type of problem can be avoided or reduce failback time by disabling Spanning Tree Protocol all together.

Here are the 5 supported methods to test SEA failover:

Scenario 1, Manual SEA Failover
On VIO server:
$ lsdev -type adapter
or
$ oem_setup_env
# lsdev -Cc adapter |grep ent –> Note which ent is the SEA
# entstat -d entX | grep State –> Check for the state (PRIMARY, or BACKUP)

Set ha_mode to standby on primary VIOS with chdev command:
# chdev -l entX -a ha_mode=standby
or
$ chdev -dev -attr ha_mode=standby

Reset it back to auto and the SEA should fail back to the primary VIOS:
# chdev -l entX -a ha_mode=auto
or
$ chdev -dev -attr ha_mode=auto

Scenario 2, Primary VIOS Shutdown
Reboot the primary VIOS for fail over to backup SEA adapter.
When the primary VIOS is up again, it should fail back to the primary SEA adapter.

Scenario 3, Primary VIOS Error
Deactivate primary VIOS from the HMC for fail over to backup SEA adapter.
Activate the primary VIOS for the fail back to the primary SEA adapter again.

Scenario 4, Physical Link Failure
Unplug the cable of the physical ethernet adapter on primary VIOS for the failover to the backup VIOS.
Replug the cable of the physical ethernet adapter on primary VIOS for the failback to the primary VIOS.

Scenario 5, Reverse Boot Sequence
Shut down both the VIO servers.
Activate the VIOS with backup SEA until the adapter becomes active.
Activate the VIOS with primary SEA. The configuration should fail back to
the primary SEA.

NOTE: When we force a manual failover in Scenario 1, we bring down the link to the switch connected to VIO1, thus asking the switch to modify its MAC tables accordingly. The backup VIOS is able to take over immediately since it is up and running but was just not being used as yet. Now, during failback, the same situation occurs. Less delay happens because we forced the failover while the primary VIOS is up and running.

In Scenario 2, when primary VIO1 is shutdown, the failover is also immediate. However, the failback to VIO 1 takes more time because the switch connected to VIO1 takes more time to start requeing packets.

The fact that the delay is shorter for manual failover and longer for VIOS shutdown, implies that the delay is happening because some switches don’t start transmitting and receiving packets for some time even after declaring that the link as up. From IBM’s side, if the link is up when TCPIP is started, then we assume the switch is ready to start sending and receiving packets even though it may not actually be ready

Capturing Boot Debug on an IBM Blade

Question
This document can be used to assist in capturing boot debug information which may help determine why a system is hanging during IPL. These steps are valid for AIX 5L and AIX 6.1.



Answer
Enabling debug boot output will cause the debug information to be directed to the console of the blade. In this case we will use ssh to connect to the BladeCenter Management Module, then connect to the particular blade console using SOL (Serial Over Lan).
1. If SOL is not set up to this blade already, please do so before continuing in this document.

2. Configure an SSH client (eg putty) to log session output to a local file on the user's PC.

3. Open a connection to the BladeCenter Management Module and log in an root or another account with administrator functionality.

4. You will be logged in to the CLI at the "system" level. Use the 'list' command to list out hardware at that level and 2 levels down:

system> list -l 2
system
mm[1] primary
power[1]
power[2]
power[3]
power[4]
blower[1]
blower[2]
switch[1]
switch[2]
switch[3]
switch[4]
blade[1] CJT-HS21
blade[2] VMware HS21 XM
blade[3] JS21
blade[4] JS20_2
blade[5] SN#YK339074T12M
blade[6] SN#YK339074T159
blade[7] QS22
blade[8] SN#YK105387R14B
mt[1]

For these next comands you will have to know which blade in the list you are going to operate on.

In my example here we'll pick blade[4].

5. Switch the environment over to that blade:
system> env -T blade[4]
OK
system:blade[4]>

6. Check the current power state of the blade.

system:blade[4]> power -state
Off

If the power is off, we are ready to proceed. If not then power off the blade:

system:blade[4]> power -off

7. Power on the blade and attach to the SOL console:

system:blade[4]> power -on -c
Establishing an sol connection. This may take a few minutes.
E20F
E200
E20B

At this point you will see the LED codes scrolling up the left hand side of the console screen.

8. We now have to stop the boot sequence to get into Open Firmware. As the blades do not have SSM mode, we have to jump directly into Open Firmware.

On some blades you may see the standard "IBM IBM IBM" across the screen at LED E1F1. On others you may only see the LED codes.

At LED E1F1 hit the number 8 key on the keyboard (not on the numeric keypad, but above the letters U and I). This will give you the Open Firmware prompt:

ok
0 >


9. Now boot AIX with the 'verbose' flag enabled to gather debug output. This will track every exec() during the boot cycle and the running system.

0 > boot -s verbose

You should see the cursor spin at the end of the line as the system boots. This output will be captured in the putty log on the PC.

The verbose debug mode will disable automatically at next boot.

Client-specific SEA statistics

To gather network statistics at a per-client level, advanced accounting can be enabled on the SEA to provide more information about its network traffic.
To enable per-client statistics, the VIOS administrator can set the SEA accounting attribute to “enabled” (the default value is “disabled”, so that no per-client statistics are gathered by default). When advanced accounting is enabled, the SEA keeps track of the hardware (MAC) addresses of all of the packets it receives from the LPAR clients, and increments packet and byte counts for each client independently. After advanced accounting is enabled on the SEA, the VIOS administrator can generate a report to view per-client statistics by running the seastat command

Client-specific SEA statistics
To gather network statistics at a per-client level, advanced accounting can be enabled on the SEA to provide more information about its network traffic.
To enable per-client statistics, the VIOS administrator can set the SEA accounting attribute to “enabled” (the default value is “disabled”, so that no per-client statistics are gathered by default). When advanced accounting is enabled, the SEA keeps track of the hardware (MAC) addresses of all of the packets it receives from the LPAR clients, and increments packet and byte counts for each client independently. After advanced accounting is enabled on the SEA, the VIOS administrator can generate a report to view per-client statistics by running the seastat command.


Note:
Advanced accounting must be enabled on the SEA before the seastat command can print any statistics

To enable advanced accounting on the SEA, enter the following command:

chdev -dev -attr accounting=enabledThe following command displays per-client SEA statistics. The optional -n flag disables name resolution on IP addresses.

seastat -d [-n]The following command clears (zeroes out) all of the per-client SEA statistics
that have been gathered:

seastat -d -c

Unable to See New VSCSI Disk on Existing vhost

Technote (FAQ)

Question
I added a new virtual target device to a vhost adapter that already had target devices on it, but the client cannot see the new virtual SCSI disk and a VSCSI_HOST error has been logged on the client. Why?



Answer
This is likely to be a problem with the max_transfer parameter. The maximum transfer parameter determines the size, in bytes, of the largest single data transfer request that can be handled by this adapter. As devices are added to a vhost adapter, the config method determines the maximum transfer each device is capable of by getting the maximum transfer size for that device and for the adapter to which that device is attached and using the smaller of those values. When the client adapter connects to
the host adapter, the host reports to the client the smaller value of all the devices defined to that host. The client then reports that value to the disk head driver on the client. When the disk head driver on the
client receives a request that is larger than the maximum transfer size, the disk head driver must break that request up into several smaller requests.

If a device is added to a host adapter after that host adapter has reported the maximum transfer size to the client, and the new device has a smaller transfer size than the value already reported, if the host showed that device to the client, the host could receive a request for that device that is larger than the device could handle. Therefore, the host does not show the device to the client and instead logs the informative error:


LABEL: VSCSI_HOST
IDENTIFIER: DF63A4FE

Date/Time: Mon Mar 5 11:47:21 2007
Sequence Number: 2205
Machine Id: 00C5763E4C00
Node Id: vios1
Class: S
Type: TEMP
Resource Name: vhost2

Description
Virtual SCSI Host Adapter detected an error

Probable Causes
Virtual SCSI Host Adapter Driver is an Undefined State

Failure Causes
Virtual SCSI Host Adapter Driver is an Undefined State

Recommended Actions
Remove Virtual SCSI Host Adapter Instance, then Configure the same instance

Detail Data
ADDITIONAL INFORMATION
module: target_report_luns rc: 00000000FFFFFFD8 location:00000001
data: 2 8F00000000000000


The best solution is to group devices according to maximum transfer sizes on host adapters. That is, if you know you will have devices with three different maximum transfer sizes in use by a single client, configure three host/client adapter pairs for that client and put the big transfer size devices on one, the medium size devices on the second, and the small devices on the third. That way, each device can run with the appropriate maximum transfer size and enjoy optimum throughput.

The second solution is to unconfigure the client adapter and all devices attached to it ("rmdev -l vscsiN -R") and reconfigure ("cfgmgr"), or reboot the client which does the same thing. However, the reconfigured client will now be using the smaller maximum transfer size so throughput will not be as good as it might be otherwise.

If a backing device (i.e. hdisk or logical volume) is exported through a vhost adapter, and its max_transfer size is greater than the max_transfer size of an already associated backing device of that vhost adapter, then the new virtual disk is not presented to the client partition. For example, suppose vhost1 has a backing device with a max_transfer size of 256K (0x40000), and you are trying to export a backing device whose max_transfer size is 1 MB (0x100000), when you run cfgmgr on the client, the client will not see the new virtual disk.

Therefore, a third alternative is to change the max_transfer size for the new device you are trying to export through that same vhost adapter to match the "largest" max_transfer size of the existing backing device(s). Then export that logical volume or disk as a backing device by running mkvdev command, and run cfgmgr on the client.

Note: The max_transfer size is a disk attribute. So, if you are exporting a logical volume as a backing device, you must check the max_transfer size of the underlying physical disk (i.e. if the logical volume, lv00, you are trying to export is on hdisk1, check the max_transfer size for hdisk1).

Example:

To check the max_transfer size for hdisk1, run

$ lsdev -dev hdisk1 -attr
...
max_transfer 0x100000 Maximum TRANSFER Size True
...
To change the max_transfer value, run


$ chdev -dev hdisk# -attr max_transfer=
where value is in hex (i.e. 0x40000)

To find out what physical volume the logical volume, rootvg_lv resides on, run


$ lslv -pv rootvg_lv
rootvg_lv:N/A
PV COPIES IN BAND DISTRIBUTION
hdisk0 001:000:000 100% 000:001:000:000:000

Note: If your VIOS level is FP11.1, and you have ruled out the max_transfer size as previously discussed, it is possible you may be running into a known defect addressed with IZ29603, which is included in FP11.1 SP 1. Consequently, the recommendation is to update the VIOS to that SP level or higher.

Sympthoms may include:

- existing client unable to see new virtual disk on existing and new virtual SCSI adapter
- existing VIO client may fail to see boot device
- new client unable to see virtual disk during the Installation menu

Setting Up and Using JFS2 Filesystem Quotas

Technote (FAQ)

Question
How do I set up and manage quotas on a JFS2 filesystem?



Answer
Enable quotas on an existing JFS2 filesystem, or create a new filesystem with quotas enabled.
1. To change an existing filesystem:
# chfs -a quota=userquota /myfs
The other option is quota=groupquota, or both can be added

2. mount the filesystem
# mount /myfs

3. Make sure it mounted with quotas enabled:
# lsfs -q /myfs
Name Nodename Mount Pt VFS Size Options Auto Accounting
/dev/fslv02 -- /myfs jfs2 2097152 rw no no
(lv size: 2097152, fs size: 2097152, block size: 4096, sparse files: yes, inline log: no, inline log size: 0, EAformat: v1, Quota: userquota, DMAPI: no, VIX: no)

Set up a new class of quotas for a user

# j2edlimit -e /myfs

This will load the current quota class into a 'vi' editor session:

User Limits Classes for file system /myfs
Block Limits units: g=gigabyte, m=megabyte, or optional k=kilobyte
Grace Period units: d=days, h=hours, m=minutes, or s=seconds
Limits Class ID 0 is the default class.

Block Limits File Limits Grace Period
ID soft hard soft hard block file

0 0 0 0 0 0 0

Add a new class by creating a new line, using ID=+ and setting your limits:

+ 1g 2g 12000 15000 7d 7d

Save the file using :wq or ZZ.

The command should assign ID 1 to your new class, and list it out for you after the vi session. You can also list it using:

# j2edlimit -l /myfs
User Limits Classes for file system /myfs
Block Limits units: g=gigabyte, m=megabyte, or optional k=kilobyte
Grace Period units: d=days, h=hours, m=minutes, or s=seconds
Limits Class ID 0 is the default class.

Block Limits File Limits Grace Period
ID soft hard soft hard block file

0 0 0 0 0 0 0
1 1g 2g 12000 15000 7d 7d


Assign a user to your new limits class

# j2edlimit -a 1 -u bob /myfs
Where: 1 is the ID of the class you wish to assign
bob is the username

To report on a user's quota usage in a filesystem

# repquota /myfs
Block limits File limits
Group used soft hard grace used soft hard grace
system -- 64 0 0 4 0 0
staff -- 18756 0 0 1 0 0

Block limits File limits
User used soft hard grace used soft hard grace
root -- 32 0 0 4 0 0
bob -- 18756 1048576 2097152 1 12000 15000


To remove a user from a particular class

Reassign that user to class 0, which is unlimited.

# j2edlimit -a 0 bob /myfs
User Limits Classes for file system /myfs
Block Limits units: g=gigabyte, m=megabyte, or optional k=kilobyte
Grace Period units: d=days, h=hours, m=minutes, or s=seconds
Limits Class ID 0 is the default class.

Block Limits File Limits Grace Period
ID soft hard soft hard block file

0 0 0 0 0 0 0

Considerations while upgrading AIX with SDDPCM installed

Considerations while upgrading AIX with SDDPCM installed

SDDPCM (Subsystem Device Driver Path Control Module) is the multi-pathing plug-in for AIX 5.3 and 6.1.

Customers who use IBM SVC, DS6000, DS8000 and/or ESS800 use this package to allow the operating system to handle multiple paths from OS to storage.
When you are upgrading AIX with SDDPCM installed, don’t forget to plan for SDDPCM too.
1) Planning an upgrade from AIX 5.3 to 6.1Before the AIX OS is upgraded, SDDPCM must be uninstalled and then reinstalled after the upgrade. There are cases when the host attachment script must also be uninstalled and reinstalled. This is explained in the SDD Users Guide found here:

2) Already upgraded AIX to 6.1 but still using AIX 5.3 version of SDDPCM?

You may need help from IBM before you can upgrade your SDDPCM to the AIX 6.1 version. This will come in the form of some special scripts.

3) General SDDPCM maintenance

There are quite a few SDDPCM flashes out there right now. You need to check these out and ensure you are not exposed to the issues that are corrected by later versions of SDDPCM. Check out the flashes listed here (or read my previous blog entry):

http://www-01.ibm.com/support/search.wss?rs=540&tc=ST52G7&dc=D600&dtm

What about SDD (Subsystem Device Driver) for AIX?
Prior to AIX 5.2 FP5, AIX did not offer native multi-pathing (MPIO). This meant that each hardware vendor had to offer their own third-party software to handle multiple paths. To achieve this with the ESS (Shark), IBM released a product called DPO (Data Path Optimiser). This product became SDD and was made available for a wide variety of operating systems.

When AIX offered MPIO, IBM then also offered a vendor plug-in (Path Contol Module) for native AIX MPIO which IBM called SDDPCM. This means you have two choices with AIX: SDD or SDDPCM. If your considering which is best, SDDPCM is my preference. This is because it is native to the operating system and also better supports the possibility of co-existence of multiple PCMs. Note that migrating from SDD to SDDPCM is not supported by the VIOS at this time, so if your running VIOS you will need to stay put for now.

EMC Powerpath and MPIO in a same box

You can run into an issue with EMC storage on AIX systems using MPIO (No Powerpath) for your boot disks:

After installing the ODM_DEFINITONS of EMC Symmetrix on your client system, the system won’t boot any more and will hang with LED 554 (unable to find boot disk).

The boot hang (LED 554) is not caused by the EMC ODM package itself, but by the boot process not detecting a path to the boot disk if the first MPIO path does not corresponding to the fscsiX driver instance where all hdisks are configured. Let me explain that more in detail:

Let’s say we have an AIX system with four HBAs configured in the following order:

# lscfg -v | grep fcs

fcs2 (wwn 71ca) -> no devices configured behind this fscsi2 driver
instance (path only configured in CuPath ODM table)

fcs3 (wwn 71cb) -> no devices configured behind this fscsi3 driver
instance (path only configured in CuPath ODM table)

fcs0 (wwn 71e4) -> no devices configured behind this fscsi0 driver
instance (path only configured in CuPath ODM table)

fcs1 (wwn 71e5) -> ALL devices configured behind this fscsi1 driver
instance

Looking at the MPIO path configuration, here is what we have for the rootvg disk:

# lspath -l hdisk2 -H -F”name parent path_id connection status”

name parent path_id connection status

hdisk2 fscsi0 0 5006048452a83987,33000000000000 Enabled

hdisk2 fscsi1 1 5006048c52a83998,33000000000000 Enabled

hdisk2 fscsi2 2 5006048452a83986,33000000000000 Enabled

hdisk2 fscsi3 3 5006048c52a83999,33000000000000 Enabled

The fscsi1 driver instance is the second path (pathid 1), then remove the 3 paths keeping only the path corresponding to fscsi1 :

# rmpath -l hdisk2 -p fscsi0 -d

# rmpath -l hdisk2 -p fscsi2 -d

# rmpath -l hdisk2 -p fscsi3 -d
# lspath -l hdisk2 -H -F”name parent path_id connection status”

Afterwards, do a savebase to update the boot lv hd5. Set up the bootlist to hdisk2 and reboot the host.

It will come up successfully, no more hang LED 554.

When checking the status of the rootvg disk, a new hdisk10 has been configured with the correct ODM definitions as shown below:

# lspv

hdisk10 0003027f7f7ca7e2 rootvg active

# lsdev -Cc disk

hdisk2 Defined 00-09-01 MPIO Other FC SCSI Disk Drive

hdisk10 Available 00-08-01 EMC Symmetrix FCP MPIO Raid6

To summarize, it is recommended to setup ONLY ONE path when installing an AIX to a SAN disk, then install the EMC ODM package then reboot the host and only after that is complete, add the other paths. Dy doing that we ensure that the fscsiX driver instance used for the boot process has the hdisk configured behind.

Creating VLAN Tagged IEEE 802.1Q Virtual Ethernet Adapters in IVM

Technote (FAQ)

Question
Is there a method for creating VLAN tagged virtual Ethernet adapters using the integrated virtual manager (IVM)?



Cause
The IVM GUI in v2.1 and lower does not provide GUI advanced options for creating multiple VLAN tagged Ethernet adapters. The limitation of the IVM GUI does not allow you to specify VLAN tagging when creating virtual ethernet devices, but you can achieve the configuration using CLI.


Answer
The IVM command line interface (CLI) does allow creating VLAN tagged (IEEE 802.1Q) virtual Ethernet adapters. There are two commands that will need to be used:

(1) chhwres is used on the VIO server to dynamically modify the VIO server LPAR only.

(2) chsyscfg is used to modify the client LPAR profile and can only be used when client is not activated.


Following example of creating virtual Ethernet adapters that support VLAN configurations illustrate using the chhwres and chsyscfg commands. The example shows commands for listing output of profile definitions of a client LPAR and then adding definitions for a new virtual adapter to the LPAR profile.

lssyscfg -m Server-9111-520-SN104797E -r prof --filter lpar_ids=2

Example output:
name=ivmclient,
lpar_name=ivmclient,
lpar_id=2,
os_type=aixlinux,
all_resources=0,
min_mem=256,desired_mem=256,max_mem=256,
proc_mode=shared,
min_proc_units=0.10,desired_proc_units=0.10,max_proc_units=2.00,
min_procs=1,desired_procs=1,max_procs=2,
sharing_mode=uncap,uncap_weight=128,
io_slots=none,
lpar_io_pool_ids=none,
max_virtual_slots=6,
"virtual_serial_adapters=0/server/1/any//any/1,1/server/1/any//any/1",
virtual_scsi_adapters=2/client/1/10-4797E_vios/11/1,
virtual_eth_adapters=5/0/1//0/0,
boot_mode=norm,conn_monitoring=0,auto_start=0,power_ctrl_lpar_ids=none

Note: The bold text output for virtual_eth_adapter variable in the example above shows the virtual device is in slot 5. The properties of the device are separated by a slash and designate parameters such as.
slot_num
ieee_virtual_eth
port_vlan_id
addl_vlan_ids
is_trunk
trunk_priority

Example of modifying the profile of an LPAR to add an additional virtual Ethernet adapter which will support the ability to receive VLAN 200 tagged traffic. The client LPAR needs to be shutdown before its profile can be changed.

chsyscfg -m Server-9111-520-SN104797E -r prof -i 'lpar_id=2,"virtual_eth_adapters=4/1/14/200/0/0,5/0/1//0/0"'

Checking the profile of LPAR 2 to see if changes were made can be done using the lssyscfg command..

lssyscfg -m Server-9111-520-SN104797E -r prof --filter lpar_ids=2 -F lpar_id,virtual_eth_adapters

Example output:
2,"4/1/14/200/0/0,5/0/1//0/0"

On the VIO server LPAR, when you do issue a command to modify the hardware. you can get the changed to dynamically take affect, but that is the special nature of a IVM environment. For example, the following IVM command dynamically alters the VIO server configuration.

chhwres -m Server-9111-520-SN104797E -p 10-4797E_vios -o a -r virtualio --rsubtype eth -s 14 -a
"ieee_virtual_eth=1,port_vlan_id=14,addl_vlan_ids=200,is_trunk=1,trunk_priority=1" -d 5

lshwres -m Server-9111-520-SN104797E -r virtualio --rsubtype eth --level lpar --filter lpar_ids=1
Example output:
lpar_name=10-4797E_vios,lpar_id=1,slot_num=3,state=1,ieee_virtual_eth=0,port_vlan_id=1,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001003
lpar_name=10-4797E_vios,lpar_id=1,slot_num=4,state=1,ieee_virtual_eth=0,port_vlan_id=2,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001004
lpar_name=10-4797E_vios,lpar_id=1,slot_num=5,state=1,ieee_virtual_eth=0,port_vlan_id=3,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001005
lpar_name=10-4797E_vios,lpar_id=1,slot_num=6,state=1,ieee_virtual_eth=0,port_vlan_id=4,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001006
lpar_name=10-4797E_vios,lpar_id=1,slot_num=14,state=1,ieee_virtual_eth=1,port_vlan_id=14,addl_vlan_ids=200,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA29000100F

On the VIO server, the chhwres command also invokes other commands that will cause the server profile to be modified to for the new device so it will be available on next boot. You can also verify the profile was changed using the lssyscfg command.

lssyscfg -m Server-9111-520-SN104797E -r prof --filter lpar_ids=1 -F lpar_ids,virutal_eth_adapters

Example output:
1,"3/0/1//1/0,4/0/2//1/0,5/0/3//1/0,6/0/4//1/0,14/1/14/200/1/0"

The IVM GUI or CLI interface can then be used to setup a virtual Ethernet bridge (SEA) between a physical adapter and a virtual adapter that was created with the additional VLAN properties.

IBM Power system microcode (firmware) naming conversion

Difference between p5, p6 and p7 system firmware naming conventions?

On power5 systems all firmware levels began with SFXXX…. regardless of p5 system model. Starting in p6 and continuing on p7 servers they have different streams for the pSeries servers like EHXXX, ELXXX, EMXXX, ALXXX, AMXXX.

Does anyone have a paper, or a URL or can explain what servers will have these firmware naming conventions and why? Also, is the p6 firmware level naming convention the same as p7? Both p6 and p7 are definitely different than p5.

Explaination

Answer:
Not sure if all of this is the “formal” naming, but generally:

POWER5:
SF means “Squadrons Firmware”.

POWER6:
EH is Enterprise High-End
EM is Enterprise Mid-Range (formerly Intermediate-High)
EL is Enterprise Low-End

POWER7:
750 and 755 should be AL and 770/780 should be AM.
775, 790 and 795, or whatever comes out for the ultra high end would be AH.

I’m not sure, but A is probably Advanced. POWER7 shares architecture with their PowerPC A2 “wire-speed” products. It’s a substantial advancement on the NUMA technology they’ve been using since POWER5 (think Sequent acquisition).

Redundancy configuration using virtual fibre channel adapters With N_Port ID Virtualization (NPIV)

Redundancy configurations help protect your network from physical adapter failures as well as Virtual I/O Server failures.

With N_Port ID Virtualization (NPIV), you can configure the managed system so that multiple logical partitions can access independent physical storage through the same physical fibre channel adapter. Each virtual fibre channel adapter is identified by a unique worldwide port name (WWPN), which means that you can connect each virtual fibre channel adapter to independent physical storage on a SAN.

Similar to virtual SCSI redundancy, virtual fibre channel redundancy can be achieved using Multi-path I/O (MPIO) and mirroring at the client partition. The difference between traditional redundancy with SCSI adapters and the NPIV technology using virtual fibre channel adapters, is that the redundancy occurs on the client, because only the client recognizes the disk. The Virtual I/O Server is essentially just a pipe. The second example below uses multiple Virtual I/O Server logical partitions to add redundancy at the Virtual I/O Server level as well.

Example: Host bus adapter failover
This example uses Host bus adapter (HBA) failover to provide a basic level of redundancy for the client logical partition. The figure shows the following connections:

The storage area network (SAN) connects physical storage to two physical fibre channel adapters located on the managed system.
The physical fibre channel adapters are assigned to the Virtual I/O Server and support NPIV.
The physical fibre channel ports are each connected to a virtual fibre channel adapter on the Virtual I/O Server. The two virtual fibre channel adapters on the Virtual I/O Server are connected to ports on two different physical fibre channel adapters in order to provide redundancy for the physical adapters.
Each virtual fibre channel adapter on the Virtual I/O Server is connected to one virtual fibre channel adapter on a client logical partition. Each virtual fibre channel adapter on each client logical partition receives a pair of unique WWPNs. The client logical partition uses one WWPN to log into the SAN at any given time. The other WWPN is used when you move the client logical partition to another managed system.
The virtual fibre channel adapters always has a one-to-one relationship between the client logical partitions and the virtual fibre channel adapters on the Virtual I/O Server logical partition. That is, each virtual fibre channel adapter that is assigned to a client logical partition must connect to only one virtual fibre channel adapter on the Virtual I/O Server, and each virtual fibre channel on the Virtual I/O Server must connect to only one virtual fibre channel adapter on a client logical partition.

The client can write to the physical storage through client virtual fibre channel adapter 1 or 2. If a physical fibre channel adapter fails, the client uses the alternative path. This example does not show redundancy in the physical storage, but rather assumes it would be built into the SAN.

Note: It is recommended that you configure virtual fibre channel adapters from multiple logical partitions to the same HBA, or you configure virtual fibre channel adapters from the same logical partition to different HBAs.
Example: HBA and Virtual I/O Server failover
This example uses HBA and Virtual I/O Server failover to provide a more advanced level of redundancy for the client logical partition. The figure shows the following connections:

The storage area network (SAN) connects physical storage to two physical fibre channel adapters located on the managed system.
There are two Virtual I/O Server logical partitions to provide redundancy at the Virtual I/O Server level.
The physical fibre channel adapters are assigned to their respective Virtual I/O Server and support NPIV.
The physical fibre channel ports are each connected to a virtual fibre channel adapter on the Virtual I/O Server. The two virtual fibre channel adapters on the Virtual I/O Server are connected to ports on two different physical fibre channel adapters in order to provide redundancy for the physical adapters. A single adapter could have multiple ports.
Each virtual fibre channel adapter on the Virtual I/O Server is connected to one virtual fibre channel adapter on a client logical partition. Each virtual fibre channel adapter on each client logical partition receives a pair of unique WWPNs. The client logical partition uses one WWPN to log into the SAN at any given time. The other WWPN is used when you move the client logical partition to another managed system.
The client can write to the physical storage through virtual fibre channel adapter 1 or 2 on the client logical partition through VIOS 2. The client can also write to physical storage through virtual fibre channel adapter 3 or 4 on the client logical partition through VIOS 1. If a physical fibre channel adapter fails on VIOS 1, the client uses the other physical adapter connected to VIOS 1 or uses the paths connected through VIOS 2. If VIOS 1 fails, then the client uses the path through VIOS 2. This example does not show redundancy in the physical storage, but rather assumes it would be built into the SAN.

Considerations
These examples can become more complex as you add physical storage redundancy and multiple clients, but the concepts remain the same. Consider the following points:

To avoid configuring the physical fibre channel adapter to be a single point of failure for the connection between the client logical partition and its physical storage on the SAN, do not connect two virtual fibre channel adapters from the same client logical partition to the same physical fibre channel adapter. Instead, connect each virtual fibre channel adapter to a different physical fibre channel adapter.
Consider load balancing when mapping a virtual fibre channel adapter on the Virtual I/O Server to a physical port on the physical fiber channel adapter.
Consider what level of redundancy already exists in the SAN to determine whether to configure multiple physical storage units.
Consider using two Virtual I/O Server logical partitions. Since the Virtual I/O Server is central to communication between logical partitions and the external network, it is important to provide a level of redundancy for the Virtual I/O Server. Multiple Virtual I/O Server logical partitions require more resources as well, so you should plan accordingly.
NPIV technology is useful when you want to move logical partitions between servers. For example, in active Partition Mobility, if you use the redundancy configurations above, in combination with physical adapters, you can stop all the I/O activity through the dedicated, physical adapter and direct all traffic through a virtual fibre channel adapter until the logical partition is successfully moved. The dedicated physical adapter would need to be connected to the same storage as the virtual path. Since you cannot migrate a physical adapter, all I/O activity is routed through the virtual path while you move the partition. After the logical partition is moved successfully, you need to set up the dedicated path (on the destination logical partition) if you want to use the same redundancy configuration you had configured on the original logical partition. Then the I/O activity can resume through the dedicated adapter, using the virtual fibre channel adapter as a secondary path.

PLM Configuration on AIX

PLM can be easily configured using the instructions available from the Hardware InfoCenter. The main document title and links to all the other related steps is as follows.
Configuring resource management for AIX partitions with the Partition Load Manager:

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/iphat_p5/iphbkconfigureplm.htm

This URL has links to key topic to assist in the
installation and configuration of PLM.

The topics are:

- Preparing to install the Partition Load Manager

- Installing OpenSSH software tools

- Installing the Partition Load Manager server

- Configuring the policy file

- Querying partition status

- Configuring Resource Monitoring and Control (RMC)

- Starting and stopping the Partition Load Manager server

- Commands for the Partition Load Manager

PLM is only valid for use with System p power 5 servers and typically this will also require a HMC at v6r1.x or lower. The latest patches for PLM should be installed prior to trying to start the PLM server. Additional information regarding PLM can be found at following URL.

Partition Load Manager
Support for System p
http://www14.software.ibm.com/webapp/set2/sas/f/plm/home.html



Related information
Partition Load Manager Support for System p