Santosh Gupta's passion for AIX

2011-10-26T23:37:00.000+05:30

AIX Version 5.3

The IBM AIX 5L Version 5.3 has been withdrawn from the market, effective April 29, 2011.
Highlights
• Well-proven, scalable, open, standards-based UNIX® operating system
• IBM POWER5™ technology and Virtualization Engine™ enablement help deliver power, increase utilization, ease administration and reduce total cost
• Rock-solid security and availability to help protect IT assets and keep businesses running
• Linux® affinity enables fast, cost-effective development of cross-platform applications
Accept no limits, make no compromises
In today’s on demand world, clients need a safe, secure, stable and flexible operating environment to run their organizations. That is why more and more businesses large and small are choosing AIX 5L™ for POWER™, IBM’s industrial-strength UNIX operating system (OS), for their mission-critical applications. With its proven scalability, reliability and manageability, the AIX 5L OS is an excellent choice for building a flexible IT infrastructure and is the only UNIX operating system that leverages IBM experience in building solutions that run businesses worldwide. And only one UNIX operating system leads the industry in vision and delivery of advanced support for 64-bit scalability, virtualization and affinity for Linux. That operating system is AIX 5L.
AIX 5L is an open, standards-based OS that conforms to The Open Group’s Single UNIX Specification Version 3. It provides fully integrated support for 32- and 64-bit applications. AIX 5L supports the IBM System p5™, IBM eServer™ p5, IBM eServer pSeries®, IBM eServer i5 and IBM RS/6000® server product lines, as well as IBM BladeCenter® JS2x blades and IntelliStation® POWER and RS/6000 workstations. In addition to compliance with UNIX standards, AIX 5L includes commands and application programming interfaces to ease the porting of applications from Linux to AIX 5L.
AIX 5L Version 5.3 offers new levels of innovative self-management technologies. It continues to exploit current 64-bit system and software architecture to support advanced virtualization options, as well as IBM POWER5 and POWER5+™ processors with simultaneous multithreading capability for improved performance and system utilization. AIX 5L V5.3 is enhanced to support the IBM Virtualization Engine systems technology innovations available on POWER5 and POWER5+ systems, including Micro-Partitioning™ and Virtual I/O Server support.
AIX 5L V5.3 also includes the advanced distributed file system NFSv4. NFSv4 is an open, standards-based distributed file system that offers superior security, interoperability and scalability. AIX 5L was the first commercial UNIX vendor to include NFSv4. IBM includes advanced NFSv4 file system federation and replication management capabilities.
AIX 5L V5.3 provides improved system security, enhanced performance analysis and tuning tools and added system management tools. This AIX 5L release underscores IBM’s firm commitment to long-term UNIX innovations that deliver business value.

AIX videos Links

2011-10-21T23:56:00.003+05:30

http://www.ibm.com/developerworks/wikis/display/WikiPtype/Movies#Movies-power7

Welcome to the POWER6/POWER7 and AIX6 Hands-On Technical Product Demos
The idea is to provide the "cook book" information to get your started with these new interesting technologies and to answer some basic questions:

•What is it about?
•How do I get started?
•What are a few typical first good uses I could start with?
•How easy is it to use?
•How could this save me time or money?
•Where can I get more information?
We hope you find these movies interesting and let you make a flying start.

Currently, the movies add up to 20.6 hours of free education on the hottest topics.

Quick links to the main sections:

1.POWER7 Processor
2.AIX Workload Partitions
3.AIX6 and AIX7 Operating System Features
4.POWER6 Processor Features
5.Integrated Virtualization Manager (IVM)
6.Other Cool & Interesting Stuff
7.IBM System Director 6 on AIX
8.Thirteen More Director 6 Movies
9.Back to POWER Basics
10.New Virtualisation Features
11.PowerHA SystemMirror 7.1 for AIX
The latest movies added are:

•2nd Sept 2010 - How Systems Director Saves Me Time - movie 84
•12th Jan 2011 - Shared Storage Pools Hands-On - movie 85
•28th Jan 2011 - Shared Storage Pools Intro - movie 86
•March 2011 - HACMP = PowerHA System Mirror
◦On this Techdocs website the famous Shawn Bodily, Power/AIX Advanced Technical Skills, USA presents four technical movies on AIX High Availability. These are in .mov format. I had to download Apple QuickTime to view them as other players don't work (mostly audio problems).
•PowerHA SystemMirror 7.1 for AIX by HACMP Guru Alex Abderrazag - this includes a set of 6 movies:
1.PowerHA Introduction to a typical environment used in the movies
2.PowerHA Configuration via SMIT
3.PowerHA The "clmgr" command
4.PowerHA High Availability in Action
5.PowerHA SAN Communications
6.PowerHA Application Monitoring
Notes on getting the movies to work on your PC:

•These movies are in Windows Movie Format (.wmv) to make them small enough to watch over the internet or download but this means some quality has been lost from the Audio Video Interleave (.avi) originals which are 60 MB to 90 MBs in size.
•When tested on some PCs it took 4 to 5 minutes to start the movie - please be patient and don't just assume its broken - some browsers download the entire movie before they start playing it.
•Other browsers handle the media file differently - some start Windows Media Player and some start it within the browser itself. Also I have found that some auto resize the movie to fit the window - so start the movie in a suitable sized browser window. The movies where first recorded at 1024x768 but later ones at 800x600 but higher resolution. Sorry but I rather create new movies than try to regenerate them all to one size. If the movie does not fit your screen the best fix is to upgrade your screen to at least 1280x1024
•If all else fails try to download the .wmv file and play locally on your machine: using Right Click on the Download link below and selecting "Save Link as" or "Save Target as". This may highlight your PC does not support this format (good luck sorting that out!).
•Linux workstation users - ideas please, can Linux handle the .mwv format? If so, how or a good alternative solution is welcome.
◦I am told that Linux can indeed play this format - have a look at this website for hints Ubuntu - Installing Mplayer Codecs and installing OpenSUSE codecs is really simple too.
•Windows 7 users - some of the older movies do not work with Windows 7 Media Player. This appears to be missing CODEC's from Windows 7 that were in early Windows versions send your comments to Microsoft. We fixed this by downloading the ACELP CODEC from http://www.voiceage.com/acelp_eval.php - strictly at your own risk. I installed the Vista-64 version as I run Windows 7. Then watching the movie via the Windows Media Center (not the Player).
•For Windows 7 these movies have been remastered (August 2010) to fix Windows 7 problems of lack of certain CODECs found in earlier Windows versions, poor audio or hangs half way through: DFP, HMC7 Partition Mobility, Memory Keys, Partition Priority, CPU Pools and Monitoring Pools, Ganglia and PowerVM LX86.
•Feed back and further ideas for movies to Nigel Griffiths - nag at uk dot ibm dot com

Migration of AIX LPAR from one hardware to other

2011-10-01T02:01:00.000+05:30

Supported Methods of Duplicating an AIX System

Technote (FAQ)

Question
I would like to move, duplicate, or clone an AIX system onto another partition or hardware. How can I accomplish this?

Answer
This document describes the supported methods of duplicating, or cloning, an AIX instance to create new systems based on an existing one. It also describes methods known to us that are not supported and will not work.
Why Duplicate A System?
Duplicating an installed and configured AIX system has some advantages over installing AIX from scratch, and can be a faster way to get a new LPAR or system up and running.

Using this method customized configuration files, installation of additional AIX filesets, application configurations and tuning parameters can be set up once and then installed on another system or partition.

Supported Methods

1. Cloning a system via mksysb backup from one system and restore to new system.

This can either be a mksysb backup of the rootvg from the source system to tape, DVD, or a file on a NIM server.

If the mksysb is going to be used to create a new machine, make sure to set 'recover devices' to NO when it is restored. This will insure that devices existing on the source machine aren't added to the ODM of the target machine.

2. Using the alt_disk_copy command.

If you have extra disks on your system, or have disks you would like to associate with one system, load a rootvg, then remove them and associate with a new system, this is a good way to copy the rootvg to them.

The basic command to do this would be:

# alt_disk_copy -BOd hdiskx

The -B option tells alt_disk_copy not to change the bootlist to this new copy of rootvg, the -O option will remove devices from your customized ODM database.

From the alt_disk_copy man page:

-O
Performs a device reset on the target altinst_rootvg. This causes
the alternate disk install to not retain any user-defined device
configurations. This flag is useful if the target disk or disks
become the rootvg of a different system (such as in the case of
logical partitioning or system disk swap).

When the disks containing this altinst_rootvg are moved to another host and then booted from, AIX will run cfgmgr and probe for any hardware, adding ODM information at that time.

3. Using alt_disk_mksysb to install a mksysb image on another disk.

Using this technique a mksysb image is first created, either to a file, on CD or DVD or tape.

Then that mksysb image is restored to unused disks in the current system using alt_disk_mksysb, again using the -O option to perform a device reset.

After this the disks could be removed and placed in a new system, or via fibre rezoned to a new system, and the rootvg booted up.

Advanced Techniques

1. Live Partition Mobility

Using the Live Partition Mobility feature of AIX you can migrate an AIX LPAR and applications from one LPAR to another while it is up and running. Please see the AIX Manual for further information:

http://publib.boulder.ibm.com/infocenter/aix/v6r1/topic/com.ibm.aix.baseadmn/doc/baseadmndita/lpm_overview.htm

2. Higher Availability Using SAN Services

There are methods not described here, which have been documented by DeveloperWorks.
Please refer to the document "AIX higher availability using SAN services" for details.

http://www.ibm.com/developerworks/aix/library/au-AIX_HA_SAN/index.html

Unsupported Methods

1. Using a bitwise copy of a rootvg disk to another disk.

This bitwise copy can be a one-time snapshot copy such as flashcopy, from one disk to another, or a continuously-updating copy method, such as Metro Mirror.

While these methods will give you an exact duplicate of the installed AIX operating system, the copy of the OS may not be bootable.

2. Removing the rootvg disks from one system and inserting into another.

This also applies to re-zoning SAN disks that contain the rootvg so another host can see them and attempt to boot from them.

Why don't these methods work?

The reason for this is there are many objects in an AIX system that are unique to it; Hardware location codes, World-Wide Port Names, partition identifiers, and Vital Product Data (VPD) to name a few. Most of these objects or identifiers are stored in the ODM and used by AIX commands.

If a disk containing the AIX rootvg in one system is copied bit-for-bit (or removed), then inserted in another system, the firmware in the second system will describe an entirely different device tree than the AIX ODM expects to find, because it is operating on different hardware. Devices that were previously seen will show missing or removed, and usually the system will typically fail to boot with LED 554 (unknown boot disk).

Moving file systems from one volume group to another

2010-10-21T16:20:00.002+05:30

Moving file systems from one volume group to another

ATTENTION: Make sure a full backup exists of any data you intend to migrate before using these procedures.

In AIX, storage allocation is performed at the volume group level. Storage cannot span volume groups. If space within a volume group becomes constrained, then space that is available in other volume groups cannot be used to resolve storage issues.

The solution to this problem is to add more physical volumes to the relevant volume group. This may not be an option in all environments. If other volume groups contain the required free space, the alternative is to move the required logical volumes to the desired volume group and expand them as needed.

The source logical volume can be moved to another volume group with the cplv command. The following steps achieve this.

ATTENTION: The logical volume should be inactive during these steps to prevent incomplete or inconsistent data. If the logical volume contains a mounted file system, then that file system should be unmounted first. If this logical volume is being used as a RAW storage device, then the application using this logical volume should close the device or be shut down.

1.Copy the source logical volume to the desired volume group with the cplv command.

For example, where myvg is the new volume group and mylv is the name of the user's logical volume, enter:

cplv -v myvg mylv

This will return the name of the new logical volume, such as lv00.

If this logical volume was being used for RAW storage, skip to to step 6. If this is a JFS or JFS2 file system, proceed to step 2. Note that RAW storage devices should NOT use the first 512 bytes of the RAW device. This is reserved for the LVCB or logical volume control block. cplv will not copy the first 512 bytes of the RAW logical volume, but it will update fields in the new logical volume's LVCB.

2.All JFS and JFS2 file systems require a log device. This will be a logical volume with a type of jfslog or jfs2log for JFS2 file systems. Run the lsvg -l command on your destination volume group. If a JFS or JFS2 log DOES NOT already exist on the new volume group, create one by using the mklv and logform commands as detailed below. If a JFS or JFS2 log DOES exist, proceed to
step 3

With a JFS2 filesystem, you also have the option of using an inline log. With inline logs, the jfs2log exists on the filesyster itself. After the cplv command is ran on a JFS2 inline log filesystem, run:

logform /dev/lvname

You should receive a message about formatting the inline log. If you do not receive a message about an inline log, then this filesystem is not a JFS2 inline log filesystem and you should treat it as a regular JFS2 filesystem. After hitting y on formatting the inline log, continue to step 3.

To make a new JFS log, enter the following command, where myvg is the name of the new volume group, enter:

mklv -t jfslog myvg 1

To make a new JFS2 log, enter: mklv -t jfs2log myvg 1

This will return a new logical volume of either type jfslog or jfs2log, such as loglv00. This new logical volume will need to be formatted with the logform command in order to function properly as either a JFS or JFS2 log. For example:

logform /dev/loglv00

Answer yes to destroy.

3.Change the filesystem to reference a log device that exists in the new volume group and the new logical volume with the chfs command.

For example, where myfilesystem is the name of the user's filesystem, enter:

chfs -a dev=/dev/lv00 -a log=/dev/loglv00 /myfilesystem

With inline logs on JFS2 filesystems this command is also different:

chfs -a dev=/dev/lv00 -a log=INLINE /myfilesystem

4.Run fsck to ensure filesystem integrity. Enter:

fsck -p /dev/lv00

NOTE: It is common to receive errors after running fsck -p /dev/lvname prior to mounting the filesystem. These errors are due to a known bug that development is currently aware of and which will be resolved in a future release of AIX. Once the filesystem is mounted, a future fsck with the filesystem unmounted should no longer produce an error.

Mount the file system.

For example,
where myfilesystem is the name of the user's file system, enter:

mount /myfilesystem

At this point, the migration is complete, and any applications or users can now access the data in this filesystem. To change the logical volume name, proceed to the following step.

NOTE: If you receive errors from the preceding step, do not continue. Contact you AIX support center.

6.Remove the source logical volume with the rmlv command.

For example,

where mylv is the name of the user's logical volume, enter:

rmlv mylv

Rename and reset any needed attributes on the new logical volume with the chlv or chmod commands. In order to rename the logical volume, the filesystem or raw logical volume must be in a closed state.

For example, where mylv is the new name you wish to change lv00 to be, enter:

chlv -n mylv lv00

Logical volumes specific to rootvg

The following logical volumes and file systems are specific to the rootvg volume group and cannot be moved to other volume groups

Logical Volume File System or Description ------------------------------------------------------
hd2 /usr
hd3 /tmp
hd4 /
hd5
hd6
hd8
hd9var /var

Moving file systems from one volume group to another

2010-10-21T16:18:00.001+05:30

Moving file systems from one volume group to another
ATTENTION: Make sure a full backup exists of any data you intend to migrate before using these procedures.
In AIX, storage allocation is performed at the volume group level. Storage cannot span volume groups. If space within a volume group becomes constrained, then space that is available in other volume groups cannot be used to resolve storage issues.
The solution to this problem is to add more physical volumes to the relevant volume group. This may not be an option in all environments. If other volume groups contain the required free space, the alternative is to move the required logical volumes to the desired volume group and expand them as needed.
The source logical volume can be moved to another volume group with the cplv command. The following steps achieve this.
ATTENTION: The logical volume should be inactive during these steps to prevent incomplete or inconsistent data. If the logical volume contains a mounted file system, then that file system should be unmounted first. If this logical volume is being used as a RAW storage device, then the application using this logical volume should close the device or be shut down.
Copy the source logical volume to the desired volume group with the cplv command.
For example, where myvg is the new volume group and mylv is the name of the user's logical volume, enter: cplv -v myvg mylv
This will return the name of the new logical volume, such as lv00.
If this logical volume was being used for RAW storage, skip to to step 6. If this is a JFS or JFS2 file system, proceed to step 2. Note that RAW storage devices should NOT use the first 512 bytes of the RAW device. This is reserved for the LVCB or logical volume control block. cplv will not copy the first 512 bytes of the RAW logical volume, but it will update fields in the new logical volume's LVCB.
All JFS and JFS2 file systems require a log device. This will be a logical volume with a type of jfslog or jfs2log for JFS2 file systems. Run the lsvg -l command on your destination volume group. If a JFS or JFS2 log DOES NOT already exist on the new volume group, create one by using the mklv and logform commands as detailed below. If a JFS or JFS2 log DOES exist, proceed to step 3.
With a JFS2 filesystem, you also have the option of using an inline log. With inline logs, the jfs2log exists on the filesyster itself. After the cplv command is ran on a JFS2 inline log filesystem, run: logform /dev/lvname
You should receive a message about formatting the inline log. If you do not receive a message about an inline log, then this filesystem is not a JFS2 inline log filesystem and you should treat it as a regular JFS2 filesystem. After hitting y on formatting the inline log, continue to step 3.
To make a new JFS log, enter the following command, where myvg is the name of the new volume group, enter: mklv -t jfslog myvg 1
To make a new JFS2 log, enter: mklv -t jfs2log myvg 1
This will return a new logical volume of either type jfslog or jfs2log, such as loglv00. This new logical volume will need to be formatted with the logform command in order to function properly as either a JFS or JFS2 log. For example: logform /dev/loglv00
Answer yes to destroy.

Change the filesystem to reference a log device that exists in the new volume group and the new logical volume with the chfs command.
For example, where myfilesystem is the name of the user's filesystem, enter: chfs -a dev=/dev/lv00 -a log=/dev/loglv00 /myfilesystem
With inline logs on JFS2 filesystems this command is also different: chfs -a dev=/dev/lv00 -a log=INLINE /myfilesystem
Run fsck to ensure filesystem integrity. Enter: fsck -p /dev/lv00
NOTE: It is common to receive errors after running fsck -p /dev/lvname prior to mounting the filesystem. These errors are due to a known bug that development is currently aware of and which will be resolved in a future release of AIX. Once the filesystem is mounted, a future fsck with the filesystem unmounted should no longer produce an error.
Mount the file system.
For example, where myfilesystem is the name of the user's file system, enter: mount /myfilesystem
At this point, the migration is complete, and any applications or users can now access the data in this filesystem. To change the logical volume name, proceed to the following step.
NOTE: If you receive errors from the preceding step, do not continue. Contact you AIX support center.
Remove the source logical volume with the rmlv command.
For example, where mylv is the name of the user's logical volume, enter: rmlv mylv
Rename and reset any needed attributes on the new logical volume with the chlv or chmod commands. In order to rename the logical volume, the filesystem or raw logical volume must be in a closed state.
For example, where mylv is the new name you wish to change lv00 to be, enter: chlv -n mylv lv00
Logical volumes specific to rootvg
The following logical volumes and file systems are specific to the rootvg volume group and cannot be moved to other volume groups: Logical Volume File System or Description
------------------------------------------------------
hd2 /usr
hd3 /tmp
hd4 /
hd5
hd6
hd8
hd9var /var

Error when install softwares giving bosboot verification failure

2010-09-11T18:22:00.000+05:30

Not able to install softwares giving bosboot verification failure

when we try to install software on aix box it gives me error of bosboot verification failed.

We check and found that /dev/ipldevice was not present. this file is a symlink of /dev/hdisk0 ( boot disk ).

so recreate the file /dev/ipldevice and make a hardlink of /dev/hdisk0

ln /dev/hdisk0 /dev/ipldevice

and then do bosboot -ad /dev/hdisk0

and then i tried to install software it works

IBM VIOS Installation over NIM

2010-09-11T12:46:00.001+05:30

Prerequisites
IBM VIOS Installation DVD
IBM AIX Installation CD Disk 1 (I used AIX 5.3)
AIX NIM Server (I used AIX 5.3)
Power Series 5

Step 1. Prepare Installation files:
AIX File Limit SizeYou must ensure that your file size security limitation isn't going to stop you from copying your mksysb image from your cdrom to your hard drive. On your NIM server, go to the /etc/security directory and edit the limits file. Change the fsize to -1 or something large enough to ensure the mksysb image will copy over. You will need to reboot your system for this to take place, or you can log out and log in again.
cd /etc/securityvi limitsfsize = -1reboot or logout
Insert and Mount VIOS DVD
smitty mountfsFILE SYSTEM name: /dev/cd0DIRECTORY over which to mount: /cdromTYPE of file system: cdrfsMount as a READ-ONLY system? yes(or mkdir /cdrommount -v cdrfs -o ro /dev/cd0 /cdrom )
Copy installation files from cdrom:
mkdir /export/VIOScd /cdrom/nimol/ioserver_res
-rw-r--r-- 1 root system 11969032 Jul 05 07:07 booti.chrp.mp.ent.Z
-rw-r--r-- 1 root system 951 Jul 05 07:07 bosinst.data
-rw-r--r-- 1 root system 40723208 Jul 05 07:07 ispot.tar.Z
lrwxrwxrwx 1 root system 38 Jul 05 07:07 mksysb -> ../../usr/sys/inst.images/mksysb_image
cp bosinst.data /export/VIOScd /cdrom/usr/sys/inst.images
-rw-r--r-- 1 root system 1101926400 Jul 05 06:52 mksysb_image
cp mksysb_image /export/VIOS
For newer versions of vio like 1.5.2 & 2.1 you need to do the following:cp mksysb_image2 /export/VIOScd /export/VIOScat mksysb_image2 >> mksysb_image
Step 2. Define NIM Resources:
Define the mksysb_image resource object
nim -o define -t mksysb -a location=/export/VIOS/mksysb_image -a server=master vios_mksysb
Define the SPOT resource object
mkdir /export/VIOSSPOTnim -o define -t spot -a server=master -a location=/export/VIOS/VIOSSPOT -a source=vios_mksysb vios_spot
# nim -o define -t spot -a server=master -a location=/export/VIOS/VIOSSPOT -a so
urce=vios_mksysb vios_spot
Creating SPOT in "/export/VIOS/VIOSSPOT" on machine "master" from "vios_mksysb"
...
Restoring files from BOS image. This may take several minutes ...
Checking filesets and network boot images for SPOT "vios_spot".
This may take several minutes ...
Define the bosinst resource object
nim -o define -t bosinst_data -a location=/export/VIOS/bosinst.data -a server=master vios_bosinst
Define the lpp_source resource object.( You might skip this step if you wish as lpp_source provides extra filesets. But you should be able to install/runvio without lpp_source, same as AIX. Also note that different VIO version is based on different AIX version.You need to find which AIX version you need to create the lpp_source. Run lsnim -l vios_mksysb and you willsee the AIX version. You need that CD to create the lpp_source. For example for VIO 1.5 you need AIX 5.3 TL7 CD1, for 1.5.2 you need AIX 5.3 TL8 CD1 for 2.1 you need AIX 6.1 TL2. But always run lsnim -l command on the mksysb or the spot you just created to find which AIX CD you need.)
Insert the first disk of the AIX installation. NOTE: When trying to use the VIOS lpp_source, when trying to NIM an LPAR, you get a missing simages error. So instead, we will use the AIX installation CDs, which works just fine.
umount /cdrommkdir /export/VIOS/lppsourcenim -o define -t lpp_source -a source=/dev/cd0 -a server=master -a location=/export/VIOS/lppsource vios_lppsource
Step 3. Create VIOS LPAR:
NOTE: I don't have any pictures of this part of the setup, but it should be obvious how this is doneNOTE: I give specifications for a typical VIOS server. Your environment may vary.
On the Power 5 HMC, right click on Partitions and select Create -> Logical Partition
Enter a Parition ID and a Partition name. Under Partition environment, select Virtual I/O server.
Select Next.
Configure the workload group, otherwise select No. Select Next.
Enter a Profile Name. Select Next.
Enter select the amount of Minimum memory, Desired memory, and Maximum memory. I usually use 2 GB throughout all three. Select Next.
Select a Processing mode. I use Dedicated. Select Next.
If using Dedicated, enter the Minimum processors, Desired processors, and Maximum processors. I usually use 4 processors throughout all three. Select Next.
Select your Hardware Configuration that you wish to use for your environment. Select Next.
Configure I/O pools - Leave these as the default. Select Next.
Configure Virtual I/O adapters - I typically configure this part later. Select Next.
Configure Power Controlling Partitions - Leave these as the default settings. Select Next.
Optional Settings - Leave these as the default settings. Select Next.
Verify settings and Select Finish.
Step 4. NIM VIOS LPAR:
On the NIM server, start NIM: smit nim
Network Installation Management
Move cursor to desired item and press Enter.
Configure the NIM Environment
Perform NIM Software Installation and Maintenance Tasks
Perform NIM Administration Tasks
Create IPL ROM Emulation Media
Esc+1=Help Esc+2=Refresh Esc+3=Cancel Esc+8=Image
Esc+9=Shell Esc+0=Exit Enter=Do
Select Perform NIM Software Installation and Maintenance Tasks
Perform NIM Software Installation and Maintenance Tasks
Move cursor to desired item and press Enter.
Install and Update Software
List Software and Related Information
Software Maintenance and Utilities
Alternate Disk Installation
Manage Diskless/Dataless Machines
Esc+1=Help Esc+2=Refresh Esc+3=Cancel Esc+8=Image
Esc+9=Shell Esc+0=Exit Enter=Do
Select Install and Update Software
Install and Update Software
Move cursor to desired item and press Enter.
Install the Base Operating System on Standalone Clients
Install Software
Update Installed Software to Latest Level (Update All)
Install Software Bundle
Update Software by Fix (APAR)
Install and Update from ALL Available Software
Esc+1=Help Esc+2=Refresh Esc+3=Cancel Esc+8=Image
Esc+9=Shell Esc+0=Exit Enter=Do
Select Install the Base Operating System on Standalone Client
Install and Update Software
Move cursor to desired item and press Enter.
Install the Base Operating System on Standalone Clients
Install Software
Update Installed Software to Latest Level (Update All)
Install Software Bundle
Update Software by Fix (APAR)
Install and Update from ALL Available Software
+--------------------------------------------------------------------------+
Select a TARGET for the operation

Move cursor to desired item and press Enter.

reg-05 machines standalone

Esc+1=Help Esc+2=Refresh Esc+3=Cancel
Esc+8=Image Esc+0=Exit Enter=Do
Es /=Find n=Find Next
Es+--------------------------------------------------------------------------+
Select the machine to install VIOS on. If nothing appears, make sure you have created a standalone system.
Install and Update Software
Move cursor to desired item and press Enter.
Install the Base Operating System on Standalone Clients
Install Software
Update Installed Software to Latest Level (Update All)
Install Software Bundle
Update Software by Fix (APAR)
Install and Update from ALL Available Software
+--------------------------------------------------------------------------+
Select the installation TYPE

Move cursor to desired item and press Enter.

rte - Install from installation images
mksysb - Install from a mksysb
spot - Install a copy of a SPOT resource

Esc+1=Help Esc+2=Refresh Esc+3=Cancel
Esc+8=Image Esc+0=Exit Enter=Do
Es /=Find n=Find Next
Es+--------------------------------------------------------------------------+
Select mksysb - Install from a mksysb
Install and Update Software
Move cursor to desired item and press Enter.
Install the Base Operating System on Standalone Clients
Install Software
Update Installed Software to Latest Level (Update All)
Install Software Bundle
Update Software by Fix (APAR)
Install and Update from ALL Available Software
+--------------------------------------------------------------------------+
Select the MKSYSB to use for the installation

Move cursor to desired item and press Enter.

vios_mksysb resources mksysb

Esc+1=Help Esc+2=Refresh Esc+3=Cancel
Esc+8=Image Esc+0=Exit Enter=Do
Es /=Find n=Find Next
Es+--------------------------------------------------------------------------+
Select the vios_mksysb resource.
Install and Update Software
Move cursor to desired item and press Enter.
Install the Base Operating System on Standalone Clients
Install Software
Update Installed Software to Latest Level (Update All)
Install Software Bundle
Update Software by Fix (APAR)
Install and Update from ALL Available Software
+--------------------------------------------------------------------------+
Select the SPOT to use for the installation

Move cursor to desired item and press Enter.

vios_spot resources spot

Esc+1=Help Esc+2=Refresh Esc+3=Cancel
Esc+8=Image Esc+0=Exit Enter=Do
Es /=Find n=Find Next
Es+--------------------------------------------------------------------------+
Select vios_spot resource.
Select the vios_lppsource resource.
Select the vios_bosinst resource.
Install the Base Operating System on Standalone Clients
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[TOP] [Entry Fields]
* Installation Target reg-05
* Installation TYPE mksysb
* SPOT vios_spot
LPP_SOURCE [vios_lppsource] +
MKSYSB vios_mksysb
BOSINST_DATA to use during installation [vios_bosinst] +
IMAGE_DATA to use during installation [] +
RESOLV_CONF to use for network configuration [] +
Customization SCRIPT to run after installation [] +
Customization FB Script to run at first reboot [] +
ACCEPT new license agreements? [no] +
Remain NIM client after install? [yes] +
PRESERVE NIM definitions for resources on [yes] +
this target?
FORCE PUSH the installation? [no] +
[MORE...31]
Esc+1=Help Esc+2=Refresh Esc+3=Cancel Esc+4=List
Esc+5=Reset Esc+6=Command Esc+7=Edit Esc+8=Image
Esc+9=Shell Esc+0=Exit Enter=Do
NOTE: Setting the "Remain as NIM client after install" as YES can cause errors when configuring your shared ethernet adapters after install.
Press Enter to start the NIM process.

Assiging original PVID to hdisk in AIX

2010-09-11T12:32:00.000+05:30

Problem:
--------

I am using AIX 5.3L with EMC Symmetrix storage, establishing BCV's and then
splitting them and mounting them to the same host. I can mount the BCV's to
the same host using the 'recreatevg' command, but the problem I'm having is
when I'm restoring a BCV back to the standard. When the BCV is restored and
I do an 'lsvg vg1' where vg1's original PV was hdiskpower33 (the standard) it
is now hdiskpower35 (the BCV). I do not want this to happen and suspect the
problem is that the BCV's PVID was changed during the recreatevg. I want to
assign the original PVID to the BCV so that it will not remove hdiskpower33
from vg1. If I do 'rmdev -dl hdiskpower35' and then do 'lsvg -p vg1' I get
an error stating that the PVID was not found, and hdiskpower33 is not listed
as being a member of the vg1 volume group. I've tried doing:

chdev -l hdiskpower35 -a pv={original pvid}

but am told it is an illegal parameter. Is there another way to do this?

Solution:
---------

Use at your own risk:

1) BACKUP old disk critical information

# dd if=/dev/hdisk9 of=/tmp/hdisk9.save bs=4k count=1

If something were to go wrong and the head information got damaged
use the following to RECOVER the origional PVID and head information

RECOVERY
# dd if=/tmp/hdisk9.save of=/dev/hdisk9 bs=4k count=1

2) Find the origional PVID. This might be seen with lspv importvg, or
varyonvg. Our example origional PVID is "0012a3e42bc908f3"

# lqueryvg -Atp /dev/hdisk9
...
Physical: 0012a3e42bc908f3 2 0
00ffffffc9cc5f99 1 0
...

3) Verify that the disk sees an invalid PVID. The first 2 data fields
of offset 80 contain the PVID.

# lquerypv -h /dev/hdisk9 80 10
00000080 00001155 583CD4B0 00000000 00000000 ...UX<.......... ^^^^^^PVID^^^^^^^ 4) Translate the ORIGIONAL PVID into the octal version. Take every 2 digits of the hex PVID and translate it to octal. This can be done by hand, calculator, script, or web page. 00012a3e42bc908f3 -> 00 12 a3 e4 2b c9 08 f3
Octal version -> 000 022 243 344 053 311 010 363

5) Write the binary version of the PVID to a file by using the octal
values. Each octal char is lead with a backslash-Zero "\0". Do
not use spaces or any other characters except for the final \c to
keep from issuing a hard return.

# echo "\0000\0022\0243\0344\0053\0311\0010\0363\c" >/tmp/oldpvid

6) Verify that the binary pvid was written correctly. The origional
hex PVID should be seen AND the final address should be "0000010"
If EITHER of these is incorrect, try again, make sure there are no
spaces in the echo and the echo ends with a "\c".

# od -x /tmp/oldpvid
0000000 0012 a3e4 2bc9 08f3
0000010

7) Restore the PVID to the disk. You sould see 8 records in and out.
If there are more or less, restore the origional 4K block by using
the recovery method in step 1.

# cat /tmp/oldpvid dd of=/dev/hdisk9 bs=1 seek=128
8+0 records in.
8+0 records out.

8) Verify that the PVID was written correctly

#lquerypv -h /dev/hdisk9 80 10
00000080 0012A3E4 2BC908F3 00000000 00000000 ....+...........

9) Reconfigure the disk definitions on all systems attaching to that disk.
The ODM information for that drive will NOT be updated until the
disk is removed and reconfigured. Until that reconfigure commands
like `lspv` will still be incorrect.

Script to delete failed path MPIO in AIX

2010-09-11T12:25:00.001+05:30

for disk in `lspv awk '{ print $1 }'`
do
for path in `lspath -l $disk -F "status connection" grep Failed awk '{ print $2 }'`
do
echo $disk
rmpath -l $disk -w $path -d
done
done

IBM AIX 7 Open Beta Program

2010-08-22T15:55:00.000+05:30

IBM AIX 7 Open Beta Program
Welcome to the open beta for IBM’s premier UNIX operating system, AIX 7. AIX 7 is binary compatible with previous releases of AIX including AIX 6, 5.3, 5.2 and 5.1. AIX 7 extends the leadership features of AIX to include exciting new capabilities for vertical scalability, virtualization and manageability.
The open beta for AIX 7 is intended to give our clients, independent software vendors and business partners the opportunity to gain early experience with this new release of AIX prior to general availability. This open beta can be run on any Power Systems, IBM System p or eServer pSeries system that is based on POWER4, PPC970, POWER5, POWER6, or POWER7 processors.
Key features of AIX 7 included in this beta:
Virtualization
AIX 5.2 Workload Partitions for AIX 7 - This new enhancement to WPAR technology allows a client to backup an LPAR running AIX V5.2 and restore it into a WPAR running on AIX 7 on POWER7. This capability is designed to allow clients to easily consolidate smaller workloads running on older hardware onto larger, more efficient POWER7 systems. Although this capability is designed specifically for POWER7, it can be tested on older POWER processors during the open beta. Please note that this capability will only work with AIX 5.2
Support for Fibre Channel adapters in a Workload Partition AIX 7 includes support to allow a physical or virtual fibre channel adapter to a WPAR. This allows WPAR to directly own SAN devices including tape devices using the ‘atape” device type. This capability is designed to expand the capabilities of a Workload Partition and simplify management of storage devices.
Security
Domain Support in Role Based Access Control - This enhancement to RBAC allows a security policy to restrict administrative access to a specific set of similar resources, such as a subset of the available network adapters. This allows IT organizations that host services for multiple tenants to restrict administrator access to only the resources associated with a particular tenant. Domains can be used to control access to Volume Groups, Filesystems, files, devices (in /dev)
Manageability
NIM thin server Network Installation Management (NIM) support for thin servers has been enhanced to support NFSV4 and IPv6. Thin Servers are diskless or dataless AIX instances that boot from a common AIX image via NFS.
Networking
Etherchannel enhancements - Support for the 802.3AD Etherchannel has been enhanced to insure that a link is Link Aggregation Control Protocol (LACP) ready before sending data packets.
Product plans referenced in this document may change at any time at IBM’s sole discretion, and are not intended to be a commitment to future product or feature availability. All statements regarding IBM future direction, plans, product names or intent are subject to change or withdrawal without notice and represent goals and objectives only. All information is provided on an as is basis, without any warranty of any kind.

Links for AIX 7.0

IBM AIX 7 Open Beta Program
The following links provide additional valuable resources related to this AIX 7 Open Beta.
AIX 7 On-line Information Center
The official IBM statement on AIX binary compatibility
IBM articles, tutorials, and technical resources for AIX and UNIX users
A full range of IBM POWER System solutions to match your business needs
A full range of IBM POWER System hardware to match your business needs
Discover the POWER of IBM POWER System servers and solutions
PartnerWorld for AIX has resources and support for IBM Business Partners looking to exploit and learn about AIX
A one stop shop to learn about the benefits, resources and support available to IBM Business Partners for IBM Systems, servers and storage

New Features in AIX Version 7

2010-08-22T15:50:00.002+05:30

New Features in AIX Version 7

IBM announced AIX version 7. http://www-03.ibm.com/systems/power/software/aix/v71/preview.html
Several new features were mentioned in the launch, but there were two new features that I found particularly interesting:

- AIX 5.2 WPARs for AIX 7
- Cluster Aware AIX

AIX 5.2 WPARs for AIX 7

In AIX version 7, administrators will now have the capability to create Workload Partitions (WPARs) that can run AIX 5.2, inside an AIX 7 operating system instance. This will be supported on the POWER7 server platform. This is pretty cool. IBM have done this to allow some customers, that are unable to migrate to later generations of AIX and Power, to move to POWER7 whilst keeping their legacy AIX 5.2 systems operational. So for those clients that MUST stay on AIX 5.2 (for various reasons such as Application support) but would like to run their systems on POWER7, this feature may be very attractive. It will help to reduce the effort required when consolidating older AIX 5.2 systems onto newer hardware. It may also reduce some of the risk associated with migrating applications from one version of the AIX operating system to another.

To migrate an existing AIX 5.2 system to an AIX 7 WPAR, administrators will first need to take a mksysb of the existing system. Then they can simply restore the mksysb image inside the AIX 7 WPAR. IBM will also offer limited defect and how-to support for the AIX 5.2 operating system in an AIX 7 WPAR. These WPARs can, of course, be managed via IBM Systems Director with the Workload Partitions Manager plug-in.

The following figure provides a visualization of how these AIX 5.2 systems will fit into an AIX 7 WPAR. The WPARs in blue are native AIX 7 WPARs, while the WPARs in orange are AIX 5.2 WPARs running in the same AIX 7 instance. Pretty amazing really!

Cluster Aware AIX

Another very interesting feature of AIX 7 is a new technology known as “Cluster Aware AIX”. Believe it or not, administrators will now be able to create a cluster of AIX systems using features of the new AIX 7 kernel. IBM have introduced this “in built” clustering to the AIX OS in order to simplify the configuration and management of highly available clusters. This new AIX clustering has been designed to allow for:

- The easy creation of clusters of AIX instances for scale-out computing or high availability.
- Significantly simplify cluster configuration, construction, and maintenance.- Improve availability by reducing the time to discover failures.
- Capabilities such as common device naming to help simplify administration.
- Built in event management and monitoring.
- A foundation for future AIX capabilities and the next generation of PowerHA SystemMirror.

This does not replace PowerHA but it does change the way in which AIX traditionally integrates with cluster software like HACMP and PowerHA. A lot of the HA cluster functionality is now available in the AIX 7 kernel itself. However, the mature RSCT technology is still a component of the AIX and PowerHA configuration. I’m looking forward to reading more about this new technology and it’s capabilities.

These are just two of the many features introduced in AIX 7. I’m eagerly looking forward to what these features and others mean for the future of the AIX operating system. It’s exciting to watch this operating system grow and strengthen over time. I can’t wait to get my hands on an AIX 7 system so that I can trial these new features.

And speaking of trialing AIX 7, there is good news. IBM plan on running another AIX Open Beta program for AIX 7 mid 2010. Just as they did with AIX Version 6, customers will be given the opportunity to download a beta version of AIX 7 and trial it on their own systems in their own environment. This is very exciting and I’m really looking forward to it.

=================================================================

Clustering infrastructureAIX 7 (which some are calling Cluster Aware AIX) will be the first AIX release that will provide for built-in clustering. This promises to simplify high-availability application management with PowerHA SystemMirror.
It should be noted: This innovation isn’t being targeted as a replacement of PowerHA, but it’s supposed to change the way in which AIX integrates with it. Much of the PowerHA cluster functionality will now be available in the actual kernel. It’s simply designed to more easily construct and manage clusters for scale-out and high-availability applications.
Furthermore, AIX 7 will have features that will help reduce the time to discover failures, along with common device naming, to help systems administrators simplify cluster administration. It will also provide for event management and monitoring.
I am excited about this tighter integration between PowerHA and AIX, because anything that provides greater transparency between high-availability software and the OS further eases the burden of system administrators who architect, install and configure high-availability software.
Vertical ScalabilityAIX 7 will allow you to scale up to 1,024 threads or 256 cores in a single partition. This is simply outstanding; No other Unix OS can come close to this.
Profile-Based Configuration ManagementIBM Systems Director enhancements will simplify AIX systems-configuration management. IBM is calling this facility profile-based configuration management.
At a high level it’ll provide simplified discovery, application, update and AIX configuration-verification properties across multiple systems. It’ll be particularly helpful in terms of cloning out changes to ‘pools’ of systems. After populating a profile into a file (XML), it can then be deployed to the other servers in the pool (see Figure 1).
AIX 5.2 and WPARsAIX 7 will now provide the capability to run AIX 5.2 inside of a Workload Partition (WPAR). This will allow for further IT consolidation and flexible deployment opportunities (such as moving up to the POWER7 architecture) to folks who are still on older AIX OSs. In an easy way, it also allows you to backup an existing environment and restore it inside an AIX 7 WPAR. Furthermore, it will allow you to do this through IBM Systems Director’s Workload Partitions Manager.
I’m particularly impressed with this feature. Most companies look to discontinue support for their older operating systems as soon as they can. On the other hand, IBM continues to listen to their customers and provide additional features to folks on older versions of their systems. For example, AIX 7 will also support older hardware, including POWER4 processor-based servers. While this type of compatibility is critical to those who want to take advantage of the feature/functionality improvements of AIX but can’t afford to upgrade their hardware, it should also be noted AIX 7 will include exploitation features that take full advantage of POWER7 processor-based servers. Additionally, AIX 7 will have full binary compatibility for application programs developed on prior versions of AIX—as long as these programs comply with reasonable programming standards.

AIX7 WPAR support

2010-08-22T15:48:00.001+05:30

AIX7 WPAR support
Besides adding AIX 5.2 support to WPAR's (workload partitions) AIX7 is also adding more virtual device support and security to the WPAR virtualization engine.
AIX WPAR support will add Fibre Channel support- or exporting a virtual (NPIV) or physical fibre channel adapter.Fibre channel tape systems using the "atape" driver are also
supported inside the WPAR in this configuration.
With the next releases AIX, VIO SCSI disks are now supported in a WPAR in the same manner as Fibre Channel disks. This feature is available on both AIX V7.1 and AIX V6.1 with the 6100-06 Technology Level.
Trusted Kernel Extension Loading and Config from WPAR (AIX 7.1 Only)
AIX V7.1 provides the capability for a Global administrator to export specific kernel extensions for a WPAR administrator to have the ability to load and configure from inside the WPAR.

Workload Management in AIX: WLM, DLPAR and now WPAR

2010-08-22T15:44:00.000+05:30

Workload Manager - WLM
Over the years several methods of Workload Management have been developed as means to enhance resource utilization of systems. Some might say the Workload Management is a form of Performance Management – but that is only true in that Performance Management is actually Resource Management. In this sense, Workload Management is the collection of services and resource management applications that are used to monitor and regulate the resources a workload is permitted at any particular time.
Legacy UNIX systems had a very simple model of workload resource management. This was also known as sizing the box. Generally, the workload was the collection of all applications or processes running on the box. Various tools could be used – either system or application tools – to tune the application(s) to best fit the box. In other words, the amount of resources available to the workload was constant – whatever the box had.
With the release of AIX 4.3.3 in 1999 AIX included a new system component – AIX Workload Manager (WLM). This component made it possible to define collections of applications and processes into a workload, or workload class. A workload class was given resource entitlement (CPU, Memory, and starting with AIX 5.1 local I/O) in terms of shares. If there were four (4) classes active, and a resource was saturated (being used 100%) then AIX would compute a resource entitlement percentage based on the total active shares. If the four (4) classes had, respectively 15, 30, 45, and 60 shares the classes would be entitled to respectively – 10, 20, 30 and 40% of the resource concerned. As long as a resource was not constrained (less than 100% usage) WLM, by default, would not restrict a class resource entitlement.
The primary advantages of WLM are that it is included in the AIX base at no extra charge and is a software solution requiring no special hardware. However, performance specialists seem to have found it difficult to think in terms of performance management on a system which is regularly going to need more resources than it has. In other words, the legacy UNIX workload management model dominates most system administrations view of resource management.
Firmware Partitioning as Resource Management
Parallel with the development of WLM, a software solution for workload resource monitoring and control, the use of dividing a single system in to several separate system definitions commonly referred to as partitions. Virtualization in UNIX had become. Unlike WLM, partitioning required specific hardware features. For AIX, partitioning was introduced with the p690 POWER4 system.
Partitioning is a technique used to define multiple systems from a single system. A partition is allocated a specific amount of resources that it has to use as it desires. Individual partitions resources are isolated via firmare (Logical Partitions, or LPAR) or by the hardware component assembly (Physical Partition, or PPAR).
Initially, resource assignment was static. To change the resource allocation a halt and a (re)activation of the partition was required. Starting with AIX 5.2 the acronym DLPAR (dynamic LPAR) was introduced. This enhancement enables dynamic resource allocation to a partition, that is, a partition can have it's allocation of resources increased or decreased without a system, i.e. partition, halt and reactivation. With POWER5 the resource virtualization continued with the introduction of the firmware hypervisor, micropartitions, virtual Ethernet and virtual SCSI.
The advantages of partitioning are the flexibility in allocation of resources and the isolation quaranteed by the hypervisor firmware. However, partitioning requires specific hardware. Also, an administrator needs extra training to create and manage partition resources.
AIX 6.1 introduces Workload Partitions
A workload partition is a virtual system environment created using software tools. A workload partition is hosted by an AIX software environment. Applications and users working within the workload partition see the workload partition as if it was a regular system. Although less than a firmware created and managed partition – workload partition processes, signals and even file systems are isolated from the hosting environment as well as from other workload partitions. Additionally, workload partitions can have their own users, groups and dedicated network addresses. Interpprocess communication is limited to processes running within the workload partition.
AIX supports two kinds of Workload Partiions (WPARs).
A System WPAR is an environment that can be best compared to a stand-alone system. This WPAR runs it owns services and does not share writeable file systems with another WPAR or the AIX hosting (global) system.
An Application WPAR has all the process isolation a system WPAR has. The defining charactgeristic is that it shares file system name space with the global system and applications defined within the application WPAR environment.
Both types of WPARs can be configured for mobility to allow running insttances of the WPAR to be moved between physical systems or LPARs using the AIX Workload Partitin Manager LPP.
Summary
With the addition of WPAR (workload partitions) to AIX workload management has an intermediate level of flexibility and isolation of applications, users and data. Using WLM all process share the same environment with only CPU, memory and I/O resource allocation being managed when a resource is saturated. Firmware based virtualization of partitions starting with POWER4 hardware provides both hard allocation resource levels as well as complete isolation of services, network addresses, devices, etc. from all other partitions. Workload partitions, or WPAR, are a software based virtualization of partitions supporting a high degree of isolation and enhanced mobility over supporting global systems.

Powerpath CLI Commands

2010-08-07T14:07:00.001+05:30

Powerpath CLI Commands
Command
Description
powermt
Manages a PowerPath environment
powercf
Configures PowerPath devices
emcpreg -install
Manages PowerPath license registration
emcpminor
Checks for free minor numbers
emcpupgrade
Converts PowerPath configuration files
powermt command
Command
Description
powermt check
Checks for, and optionally removes, dead paths.
powermt check_ registration
Checks the state of the PowerPath license.
powermt config
Configures logical devices as PowerPath devices.
powermt display
powermt watch
Displays the state of HBAs configured for PowerPath.
powermt watch is deprecated.
powermt display options
Displays the periodic autorestore setting.
powermt load
Loads a PowerPath configuration.
powermt remove
Removes a path from the PowerPath configuration.
powermt restore
Tests and restores paths.
powermt save
Saves a custom PowerPath configuration.
powermt set mode
Sets paths to active or standby mode.
powermt set
periodic_autorestore
Enables or disables periodic autorestore.
powermt set policy
Changes the load balancing and failover policy.
powermt set priority
Sets the I/O priority
powermt version
Returns the number of the PowerPath version for which powermt was created.
powermt command examples
powermt display: # powermt display paths class=all
# powermt display ports dev=all
# powermt display dev=all
powermt set: To disable a HBA from passing I/O # powermt set mode=standby adapter=
To enable a HBA from passing I/O # powermt set mode=active adapter=
To set or validate the Load balancing policy
To see the current load-balancing policy and I/Os run the following command
# powermt display dev=
so = Symmetrix Optimization (default)
co = Clariion Optimization
li = Least I/Os (queued)
lb = Least Blocks (queued)
rr = Round Robin (one path after another)
re = Request (failover only)
nr = No Redirect (no load-balancing or failover)
To set to no load balancing # powermt set policy=nr dev=
To set the policy to default Symmetrix Optimization # powermt set policy=so dev=
To set the policy to default Clariion Optimization # powermt set policy=co dev=
pprootdev
To bring the rootvg devices under powerpath control # pprootdev on
To bring back the rootvg disks back to hdisk control # pprootdev off
To temporarily bring the rootvg disks to hdisk control for running "bosboot" # pprootdev fix
powermt command examples with output
To validate the installation # powermt check_registration
Key B3P3-HB43-CFMR-Q2A6-MX9V-O9P3
Product: PowerPath
Capabilities: Symmetrix CLARiiON
To display each device's path, state, policy and average I/O information
# powermt display dev=emcpower6a
Pseudo name=emcpower6a
Symmetrix ID=000184503070
Logical device ID=0021
state=alive; policy=SymmOpt; priority=0; queued-IOs=0

---------------- Host --------------- - Stor - -- I/O Path - -- Stats ---
### HW Path I/O Paths Interf. Mode State Q-IOs Errors

0 sbus@2,0/fcaw@2,0 c4t25d225s0 FA 13bA active dead 0 1
1 sbus@6,0/fcaw@1,0 c5t26d225s0 FA 4bA active alive 0 0
To show the paths and dead paths to the storage port
# powermt display paths
Symmetrix logical device count=20

----- Host Bus Adapters --------- ------ Storage System ----- - I/O Paths -
### HW Path ID Interface Total Dead

0 sbus@2,0/fcaw@2,0 000184503070 FA 13bA 20 20
1 sbus@6,0/fcaw@1,0 000184503070 FA 4bA 20 0
CLARiiON logical device count=0

----- Host Bus Adapters --------- ------ Storage System ----- - I/O Paths -
### HW Path ID Interface Total Dead

To display the storage ports information
# powermt display ports
Storage class = Symmetrix

----------- Storage System --------------- -- I/O Paths -- --- Stats ---
ID Interface Wt_Q Total Dead Q-IOs Errors

000184503070 FA 13bA 256 20 20 0 20
000184503070 FA 4bA 256 20 0 0 0
Storage class = CLARiiON

----------- Storage System --------------- -- I/O Paths -- --- Stats ---
ID Interface Wt_Q Total Dead Q-IOs Errors

Moving an LPAR to another frame

2010-08-07T14:04:00.002+05:30

Moving an LPAR to another frame

Steps for migrating LPAR from ONE Frame to Another IBM Frame

1.Have Storage zone the LPARs disk to the new HBA(s). Also have them add an additional 40GB drive for the new boot disk. By doing this we have a back out to the old boot disk on the old frame.

2. Collect data from the current LPAR:

a. Network information – Write down IP and ipv4 alias(s) for each interface

b. Run “oslevel –r” - will need this when setting up NIM for the mksysb recovery

c. Is the LPAR running AIO, if so will need to configure after the mksysb recovery

d. Run “lspv”, save this output, contains volume group and PVID information

e. Any other customizations you deem neccessary

3. create mksysb backup of this LPAR

4. Reconfigure the NIM machine for this LPAR, with new Ethernet MAC address. Foolproof method is to remove the machine and re-create it.

5. In NIM, configure the LPAR for a mksysb recovery. Select the appropriate SPOT and LPP Source, base on “oslevel –r” data collected in step 2.

6. Shut down the LPAR on the old frame (Halt the LPAR)

7. Move network cables, fibre cables, disk, zoning

8. if needed, to the LPAR on the new frame

9. On the HMC, bring up the LPAR on the new frame in SMS mode and select a network boot. Verify SMS profile has only a single HBA (if Clarrion attached, zoned to a single SP), otherwise the recovery will fail with a 554.

10. Follow prompts for building a new OS. Select the new 40GB drive for the boot disk (use lspv info collected in Step 2 to identify the correct 40GB drive). Leave defaults for remaining questions NO (shrink file systems, recover devices, and import volume groups).

11. After the LPAR has booted, from the console (the network interface may be down):

a. lspv Note the hdisk# of the bootdisk

b. bootlist –m normal –o Verify boot list is set – if not, set it

bootlist –m normal –o hdisk#

c. ifconfig en0 down If interface got configured, down it

d. ifconfig en0 detach and remove it

e. lsdev –Cc adapter Note Ethernet interfaces (ex. ent0, ent1)

f. rmdev –dl Remove all en devices

g. rmdev –dl Remove all ent devices

h. cfgmgr Will rediscover the en/ent devices

i. chdev –l -a media_speed=100_Full_Duplex Set on each interface unless

running GIG, leave defaults

j. Configure the network interfaces and aliases Use info recorded from step 2 mktcpip –h -a -m -i -g -A no –t N/A –s

chdev –l en# -a alias4=,

k. Verify that the network is working.

12. If LPAR was running AIO (data collected in Step 2), verify it is running (smitty aio)

13. Check for any other customizations which may have been made on this LPAR

14. Vary on the volume groups, use the “lspv” data collected in Step 2 to identify by PVID a hdisk in each volume group. Run for each volume group:

a. importvg –y hdisk# Will vary on all hdisk in the volume group

b. varyonvg

c. mount all Verify mounts are good

15. Verify paging space is configured appropriately

a. lsps –a Look for Active and Auto set to yes

b. chps –ay pagingXX Run for each paging space, sets Auto

c. swapon /dev/pagingxx Run for each paging space, sets Active

16. Verify LPAR is running 64 bit

a. bootinfo –K If 64, you are good

b. ln –sf /usr/lib/boot/unix_64 /unix If 32, change to run 64 bit

c. ln –sf /usr/lib/boot/unix_64 /usr/lib/boot/unix

d. bosboot –ak /usr/lib/boot/unix_64

17. If LPAR has Power Path

a. Run “powermt config” Creates the powerpath0 device

b. Run “pprootdev on” Sets Power Path control of the boot disk

c. If Clariion, make configuration changes to enable SP failover

chdev -l powerpath0 -Pa QueueDepthAdj=1

chdev –l fcsX –Pa num_cmd_elems=2048 For each fiber adapter

chdev –l fscsiX –Pa fc_err_recov=fast_fail For each fiber adapter

d. Halt the LPAR

e. Activate the Normal profile If Sym/DMX – verify two HBA’s in profile

f. If Clarrion attached, have Storage add zone to 2nd SP

i. Run cfgmgr Configure the 2nd set of disk

g. Run “pprootdev fix” Put rootdisk pvid’s back on hdisk

h. lspv grep rootvg Get boot disk hdisk#

i. bootlist –m normal –o hdisk# hdisk# Set the boot list with both hdisk

20. From the HMC, remove the LPAR profile from the old frame

21. Pull cables from the old LPAR (Ethernet and fiber), deactivate patch panel ports

22. Update documentation, Server Master, AIX Hardware spreadsheet, Patch Panel spreadsheet

23. Return the old boot disk to storage.

Unique VLAN ID for SEA failover control channel setup

2010-08-07T14:02:00.001+05:30

Unique VLAN ID for SEA failover control channel setup

Always select unique VLAN ID – which dosn’t exist on any of your organization network to avoid conflict when setting up dual VIOS with a control channel for SEA failover.. failure to follow this may result in a network storm. ( Very important and I couldn’t find any note on IBM site about it)

Requirements for Configuring SEA Failover

One SEA on one VIOS acts as the primary (active) adapter and the second SEA on the second VIOS acts as a backup (standby) adapter.
Each SEA must have at least one virtual Ethernet adapter with the “Access external network” flag (previously known as “trunk” flag) checked. This enables the SEA to provide bridging functionality between the two VIO servers.
This adapter on both the SEAs has the same PVID, but will have a different priority value.
A SEA in ha_mode (Failover mode) might have more than one trunk adapters, in which case all should have the same priority value.
The priority value defines which of the two SEAs will be the primary and which will be the backup. The lower the priority value, the higher the priority, e.g. an adapter with priority 1 will have the highest priority.
An additional virtual Ethernet adapter , which belongs to a unique VLAN on the system, is used to create the control channel between the SEAs, and must be specified in each SEA when configured in ha_mode.
The purpose of this control channel is to communicate between the two SEA adapters to determine when a fail over should take place.

Upgrading PowerPath in a dual VIO server environment

2010-08-07T14:01:00.001+05:30

Upgrading PowerPath in a dual VIO server environment

When upgrading PowerPath in a dual Virtual I/O (VIO) server environment, the devices need to be unmapped in order to maintain the existing mapping information.

To upgrade PowerPath in a dual VIO server environment:
1. On one of the VIO servers, run lsmap -all.
This command displays the mapping between physical, logical,
and virtual devices.

$ lsmap -all
SVSA Physloc Client Partition ID
————— ————————————– ——————–
vhost1 U8203.E4A.10B9141-V1-C30 0×00000000
VTD vtscsi1
Status Available
LUN 0×8100000000000000
Backing device hdiskpower5
Physloc U789C.001.DQD0564-P1-C2-T1-L67

2. Log in on the same VIO server as the padmin user.

3. Unconfigure the PowerPath pseudo devices listed in step 1 by
running:
rmdev -dev -ucfg
where is the virtual target device.
For example, rmdev -dev vtscsil -ucfg
The VTD status changes to Defined.
Note: Run rmdev -dev -ucfg for all VTDs displayed in step 1.

4. Upgrade PowerPath

=======================================================================

1. Close all applications that use PowerPath devices, and vary off all
volume groups except the root volume group (rootvg).

In a CLARiiON environment, if the Navisphere Host Agent is
running, type:
/etc/rc.agent stop

2. Optional. Run powermt save in PowerPath 4.x to save the
changes made in the configuration file.

Run powermt config.
5. Optional. Run powermt load to load the previously saved
configuration file.
When upgrading from PowerPath 4.x to PowerPath 5.3, an error
message is displayed after running powermt load, due to
differences in the PowerPath architecture. This is an expected
result and the error message can be ignored.
Even if the command succeeds in updating the saved
configuration, the following error message is displayed by
running powermt load:
host1a 5300-08-01-0819:/ #powermt load Error loading auto-restore value
Warning:Error occurred loading saved driver state from file /etc/powermt.custom

host1a 5300-08-01-0819:/ #powermt load Error loading auto-restore value
Warning:Error occurred loading saved driver state from file /etc/powermt.custom
…
Loading continues…
Error loading auto-restore value
When you upgrade from an unlicensed to a licensed version of
PowerPath, the load balancing and failover device policy is set to
bf/nr (BasicFailover/NoRedirect). You can change the policy by
using the powermt set policy command.

=======================================================================

5. Run powermt config.

6. Log in as the padmin user and then configure the VTD
unconfigured from step 3 by running:
cfgdev -dev
Where is the virtual target device.
For example, cfgdev -dev vtscsil
The VTD status changes to Available.
Note: Run cfgdev -dev for all VTDs unconfigured in step 3

7. Run lspath -h on all clients to verify all paths are Available.

8. Perform steps 1 through 7 on the second VIO server.

Recovering emc dead path

2010-08-07T13:52:00.000+05:30

# powermt display dev=allAnd you notice that there are "dead" paths, then these are the commands to run in order to set these paths back to "alive" again, of course, AFTER ensuring that any SAN related issues are resolved. To have PowerPath scan all devices and mark any dead devices as alive, if it finds that a device is in fact capable of doing I/O commands, run:
# powermt restoreTo delete any dead paths, and to reconfigure them again:
# powermt reset# powermt configOr you could run:
# powermt check

EMC - MPIO

2010-08-07T13:49:00.002+05:30

# lscfg -v grep fcsfcs2 (wwn 71ca) -> no devices configured behind this fscsi2 driver instance

(path only configured in CuPath ODM table)fcs3 (wwn 71cb) -> no devices configured behind this fscsi3 driver instance (path only configured in CuPath ODM table)fcs0 (wwn 71e4) -> no devices configured behind this fscsi0 driver instance (path only configured in CuPath ODM table)fcs1 (wwn 71e5) -> ALL devices configured behind this fscsi1 driver instance Looking at the MPIO path configuration, here is what we have for the rootvg disk:

# lspath -l hdisk2 -H -F"name parent path_id connection status" name parent path_id connection status hdisk2 fscsi0 0 5006048452a83987,33000000000000 Enabled hdisk2 fscsi1 1 5006048c52a83998,33000000000000 Enabled hdisk2 fscsi2 2 5006048452a83986,33000000000000 Enabled hdisk2 fscsi3 3 5006048c52a83999,33000000000000 Enabled The fscsi1 driver instance is the second path (pathid 1), then remove the 3 paths keeping only the path corresponding to fscsi1 :

# rmpath -l hdisk2 -p fscsi0 -d # rmpath -l hdisk2 -p fscsi2 -d # rmpath -l hdisk2 -p fscsi3 -d # lspath -l hdisk2 -H -F"name parent path_id connection status"Afterwards, do a savebase to update the boot lv hd5. Set up the bootlist to hdisk2 and reboot the host. It will come up successfully, no more hang LED 554. When checking the status of the rootvg disk, a new hdisk10 has been configured with the correct ODM definitions as shown below:

# lspv hdisk10 0003027f7f7ca7e2 rootvg active # lsdev -Cc disk hdisk2 Defined 00-09-01 MPIO Other FC SCSI Disk Drive hdisk10 Available 00-08-01 EMC Symmetrix FCP MPIO Raid6 To summarize, it is recommended to setup ONLY ONE path when installing an AIX to a SAN disk, then install the EMC ODM package then reboot the host and only after that is complete, add the other paths. Dy doing that we ensure that the fscsiX driver instance used for the boot process has the hdisk configured behind

EMC Grab

2010-08-07T13:46:00.002+05:30

EMC GRAB

EMC Grab is a utility that is run locally on each host and gathers storage-specific information (driver version, storage-technical details, etc). The EMC Grab report creates a zip file. This zip file can be used by EMC support. You can download the "Grab Utility" from the following locations:
ftp://ftp.emc.com/pub/emcgrab/Unix/ When you�ve downloaded EMCgrab, and stored in a temporary location on the server like /tmp/emc, untar it using:
tar -xvf *tarThen run:
/tmp/emc/emcgrab/emcgrab.shThe script is interactive and finishes after a couple of minutes.

RESET RESERVATION BIT

If you run into not being able to access an hdiskpowerX disk, you may need to reset the reservation bit on it:
# /usr/lpp/EMC/Symmetrix/bin/emcpowerreset fscsiX hdiskpowerX

EMC PowerPath key installation

2010-08-07T13:45:00.001+05:30

EMC PowerPath key installation
This describes how to configure the EMC PowerPath registration keys.

First, check the current configuration of PowerPath:

# powermt config
Warning: all licenses for storage systems support are missing or expired.
The install the keys:
# emcpreg -install

=========== EMC PowerPath Registration ===========
Do you have a new registration key or keys to enter?[n] y
Enter the registration keys(s) for your product(s),
one per line, pressing Enter after each key.
After typing all keys, press Enter again.

Key (Enter if done): P6BV-4KDB-QET6-RF9A-QV9D-MN3V
1 key(s) successfully added.
Key successfully installed.

Key (Enter if done):
1 key(s) successfully registered.
(Note: the license key used in this example is not valid).

EMC ODM definations cleanup

2010-08-07T13:41:00.002+05:30

Before making any changes, collect host logs to document the current configuration. At a minimum, save the following: inq, lsdev -Cc disk, lsdev -Cc adapter, lspv, and lsvg
Shutdown the application(s), unmount the file system(s), and varyoff all volume groups except for rootvg. Do not export the volume groups.

# varyoffvg Check with lsvg -o (confirm that only rootvg is varied on)If no

PowerPath, skip all steps with power names.
For CLARiiON configuration, if Navisphere Agent is running, stop it:
# /etc/rc.agent stop
Remove paths from Powerpath configuration:
# powermt remove hba=all

Delete all hdiskpower devices:
# lsdev -Cc disk -Fname grep power xargs -n1 rmdev -dl
Remove the PowerPath driver instance:
# rmdev -dl powerpath0
Delete all hdisk devices:For Symmetrix devices, use this command:
# lsdev -CtSYMM* -Fname xargs -n1 rmdev -dlFor CLARiiON devices, use this command:
# lsdev -CtCLAR* -Fname xargs -n1 rmdev -dl
Confirm with lsdev -Cc disk that there are no EMC hdisks or hdiskpowers.
Remove all Fiber driver instances:
# rmdev -Rdl fscsiX(X being driver instance number, i.e. 0,1,2, etc.)
Verify through lsdev -Cc driver that there are no more fiber driver instances (fscsi).
Change the adapter instances in Defined state
# rmdev -l fcsX(X being adapter instance number, i.e. 0,1,2, etc.)
Create the hdisk entries for all EMC devices:
# emc_cfgmgror
# cfgmgr -vl fcsx(x being each adapter instance which was rebuilt). Skip this part if no PowerPath.
Configure all EMC devices into PowerPath:
# powermt config
Check the system to see if it now displays correctly:
# powermt display# powermt display dev=all# lsdev -Cc disk# /etc/rc.agent start

Configuring MPIO for Virtual AIX client

2010-08-07T13:32:00.003+05:30

Configuring MPIO for the virtual AIX client
This document describes the procedure to set up Multi-Path I/O on the AIX clients of
the virtual I/O server.
Procedure:
This procedure assumes that the disks are already allocated to both the VIO servers
involved in this configuration.
· Creating Virtual Server and Client SCSI Adapters
First of all, via HMC create SCSI server adapters on the two VIO servers and
then two virtual client SCSI adapters on the newly created client partition, each
mapping to one of the VIO servers´ server SCSI adapter.
An example:
Here is an example of configuring and exporting an ESS LUN from both the
VIO servers to a client partition:
· Selecting the disk to export
You can check for the ESS LUN that you are going to use for MPIO by
running the following command on the VIO servers.
On the first VIO server:
$ lsdev -type disk
name status description
..
hdisk3 Available MPIO Other FC SCSI Disk Drive
hdisk4 Available MPIO Other FC SCSI Disk Drive
hdisk5 Available MPIO Other FC SCSI Disk Drive
..
$lspv
..
hdisk3 00c3e35c99c0a332 None
hdisk4 00c3e35c99c0a51c None
hdisk5 00c3e35ca560f919 None
..
In this case hdisk5 is the ESS disk that we are going to use for MPIO.
Then run the following command to list the attributes of the disk that you choose for MPIO:
$lsdev -dev hdisk5 -attr
..
algorithm fail_over Algorithm True
..
lun_id 0x5463000000000000 Logical Unit Number ID False
..
..
pvid 00c3e35ca560f9190000000000000000 Physical volume identifier
False
..
reserve_policy single_path Reserve Policy True
Note down the lun_id, pvid and the reserve_policy of the hdisk4.
· Command to change reservation policy on the disk
You see that the reserve policy is set to single_path.
Change this to no_reserve by running the following command:
$ chdev -dev hdisk5 -attr reserve_policy=no_reserve
hdisk4 changed
On the second VIO server:
On the second VIO server too, find the hdisk# that has the same pvid, it could
be a different one than the one on the first VIO server, but the pvid should the
same.
$ lspv
..
hdisk7 00c3e35ca560f919 None
..
The pvid of the hdisk7 is the same as the hdisk5 on the first VIO server.
$ lsdev -type disk
name status description
..
hdisk7 Available MPIO Other FC SCSI Disk Drive
..
$lsdev -dev hdisk7 -attr
..
algorithm fail_over Algorithm True
..
lun_id 0x5463000000000000 Logical Unit Number ID False
..
pvid 00c3e35ca560f9190000000000000000 Physical volume identifier
False
..
reserve_policy single_path Reserve Policy True
You will note that the lun_id, pvid of the hdisk7 on this server are the same as
the hdisk4 on the first VIO server.
$ chdev -dev hdisk7 -attr reserve_policy=no_reserve
hdisk6 changed
· Creating the Virtual Target Device
Now on both the VIO servers run the mkvdev command using the appropriate
hdisk#s respectively.
$ mkvdev -vdev hdisk# -vadapter vhost# -dev vhdisk#
The above command might have failed when run on the second VIO server, if
the reserve_policy was not set to no_reserve on the hdisk.
After the above command runs succesfully on both the servers, we have
same LUN exported to the client with mkvdev command on both servers.
· Check for correct mapping between the server and the client

Double check the client via the HMC that the correct slot numbers match the
respective slot numbers on the servers.
In the example, the slot number 4 for the client virtual scsi adapter maps to
slot number 5 of the VIO server VIO1_nimtb158.
And the slot number 5 for the client virtual SCSI adapter maps to the slot
number 5 of the VIO server VIO1_nimtb159.
· On the client partition
Now you are ready to install the client. You can install the client using any of
the following methods described in the red book on virtualization at
http://www.redbooks.ibm.com/redpieces/abstracts/sg247940.html:
1. NIM installation
2. Alternate disk installation
3. using the CD media
Once you install the client, run the following commands to check for MPIO:
# lsdev -Cc disk
hdisk0 Available Virtual SCSI Disk Drive
# lspv
hdisk0 00c3e35ca560f919 rootvg active
# lspath
Enabled hdisk0 vscsi0
Enabled hdisk0 vscsi1
· Dual Path
When one of the VIO servers goes down, the path coming from that server
shows as failed with the lspath command.
# lspath
Failed hdisk0 vscsi0
Enabled hdisk0 vscsi1
· Path Failure Detection
The path shows up in the "failed" mode, even after the VIO server is up
again. We need to either change the status with the “chpath” command to
“enabled” state or set the the attributes “hcheck_interval” and “hcheck_mode” to
“60” and “nonactive” respectively for a path failure to be detected automatically.
· Setting the related attributes
Here is the command to be run for setting the above attributes on the client
partition:
$ chdev -l hdisk# -a hcheck_interval=60 –a hcheck_mode=nonactive -P
The VIO AIX client needs to be rebooted for hcheck_interval attribute to take
effect.
· EMC for Storage
In case of using EMC device as the storage device attached to VIO server,
then make sure of the following:
1. Powerpath version 4.4. is installed on the VIO servers.
2. Create hdiskpower devices which are shared between both the VIO
servers.
· Additional Information
Another thing to take note of is that you cannot have the same name for
Virtual SCSI Server Adapter and Virtual Target Device. The mkvdev command
will error out if the same name for both is used.
$ mkvdev -vdev hdiskpower0 -vadapter vhost0 -dev hdiskpower0
Method error (/usr/lib/methods/define -g -d):
0514-013 Logical name is required.
The reserve attribute is named differently for an EMC device than the attribute
for ESS or FasTt storage device. It is “reserve_lock”.
Run the following command as padmin for checking the value of the
attribute.
$ lsdev -dev hdiskpower# -attr reserve_lock
Run the following command as padmin for changing the value of the attribute.
$ chdev -dev hdiskpower# -attr reserve_lock=no
· Commands to change the Fibre Channel Adapter attributes
And also change the following attributes of the fscsi#, fc_err_recov to “fast_fail”
and dyntrk to “yes”
$ chdev -dev fscsi# -attr fc_err_recov=fast_fail dyntrk=yes –perm
The reason for changing the fc_err_recov to “fast_fail” is that if the Fibre
Channel adapter driver detects a link event such as a lost link between a storage
device and a switch, then any new I/O or future retries of the failed I/Os will be
failed immediately by the adapter until the adapter driver detects that the device
has rejoined the fabric. The default setting for this attribute is 'delayed_fail’.
Setting the dyntrk attribute to “yes” makes AIX tolerate cabling changes in the
SAN.
The VIOS needs to be rebooted for fscsi# attributes to take effect

Delayed SEA failover

2010-08-01T01:09:00.001+05:30

Question
Why is my Shared Ethernet Adapter (SEA) failover delayed?

Cause
Spanning tree is turned on at the switch port with portfast disabled.

Answer
SEA failover, from primary to backup, the delay is sometimes due to Spanning Tree Protocol being enabled on the switch ports.
To ensure prompt recovery times when you enable the Spanning Tree Protocol on the switch ports connected to the physical adapters of the Shared Ethernet Adapter, you can also enable the portfast option on those ports. The portfast option allows the switch to immediately forward packets on the port without first completing the Spanning Tree Protocol. (Spanning Tree Protocol blocks the port completely until it is finished.)On SEA failback, from backup to primary, there is an additional issue:

The switches are sometimes not ready to read transmit and receive packets even after declaring the link as up. Hence, it leads to packet loss. This type of problem can be avoided or reduce failback time by disabling Spanning Tree Protocol all together.

Here are the 5 supported methods to test SEA failover:

Scenario 1, Manual SEA Failover
On VIO server:
$ lsdev -type adapter
or
$ oem_setup_env
# lsdev -Cc adapter |grep ent –> Note which ent is the SEA
# entstat -d entX | grep State –> Check for the state (PRIMARY, or BACKUP)

Set ha_mode to standby on primary VIOS with chdev command:
# chdev -l entX -a ha_mode=standby
or
$ chdev -dev -attr ha_mode=standby

Reset it back to auto and the SEA should fail back to the primary VIOS:
# chdev -l entX -a ha_mode=auto
or
$ chdev -dev -attr ha_mode=auto

Scenario 2, Primary VIOS Shutdown
Reboot the primary VIOS for fail over to backup SEA adapter.
When the primary VIOS is up again, it should fail back to the primary SEA adapter.

Scenario 3, Primary VIOS Error
Deactivate primary VIOS from the HMC for fail over to backup SEA adapter.
Activate the primary VIOS for the fail back to the primary SEA adapter again.

Scenario 4, Physical Link Failure
Unplug the cable of the physical ethernet adapter on primary VIOS for the failover to the backup VIOS.
Replug the cable of the physical ethernet adapter on primary VIOS for the failback to the primary VIOS.

Scenario 5, Reverse Boot Sequence
Shut down both the VIO servers.
Activate the VIOS with backup SEA until the adapter becomes active.
Activate the VIOS with primary SEA. The configuration should fail back to
the primary SEA.

NOTE: When we force a manual failover in Scenario 1, we bring down the link to the switch connected to VIO1, thus asking the switch to modify its MAC tables accordingly. The backup VIOS is able to take over immediately since it is up and running but was just not being used as yet. Now, during failback, the same situation occurs. Less delay happens because we forced the failover while the primary VIOS is up and running.

In Scenario 2, when primary VIO1 is shutdown, the failover is also immediate. However, the failback to VIO 1 takes more time because the switch connected to VIO1 takes more time to start requeing packets.

The fact that the delay is shorter for manual failover and longer for VIOS shutdown, implies that the delay is happening because some switches don’t start transmitting and receiving packets for some time even after declaring that the link as up. From IBM’s side, if the link is up when TCPIP is started, then we assume the switch is ready to start sending and receiving packets even though it may not actually be ready

Capturing Boot Debug on an IBM Blade

2010-08-01T01:08:00.001+05:30

Question
This document can be used to assist in capturing boot debug information which may help determine why a system is hanging during IPL. These steps are valid for AIX 5L and AIX 6.1.

Answer
Enabling debug boot output will cause the debug information to be directed to the console of the blade. In this case we will use ssh to connect to the BladeCenter Management Module, then connect to the particular blade console using SOL (Serial Over Lan).
1. If SOL is not set up to this blade already, please do so before continuing in this document.

2. Configure an SSH client (eg putty) to log session output to a local file on the user's PC.

3. Open a connection to the BladeCenter Management Module and log in an root or another account with administrator functionality.

4. You will be logged in to the CLI at the "system" level. Use the 'list' command to list out hardware at that level and 2 levels down:

system> list -l 2
system
mm[1] primary
power[1]
power[2]
power[3]
power[4]
blower[1]
blower[2]
switch[1]
switch[2]
switch[3]
switch[4]
blade[1] CJT-HS21
blade[2] VMware HS21 XM
blade[3] JS21
blade[4] JS20_2
blade[5] SN#YK339074T12M
blade[6] SN#YK339074T159
blade[7] QS22
blade[8] SN#YK105387R14B
mt[1]

For these next comands you will have to know which blade in the list you are going to operate on.

In my example here we'll pick blade[4].

5. Switch the environment over to that blade:
system> env -T blade[4]
OK
system:blade[4]>

6. Check the current power state of the blade.

system:blade[4]> power -state
Off

If the power is off, we are ready to proceed. If not then power off the blade:

system:blade[4]> power -off

7. Power on the blade and attach to the SOL console:

system:blade[4]> power -on -c
Establishing an sol connection. This may take a few minutes.
E20F
E200
E20B

At this point you will see the LED codes scrolling up the left hand side of the console screen.

8. We now have to stop the boot sequence to get into Open Firmware. As the blades do not have SSM mode, we have to jump directly into Open Firmware.

On some blades you may see the standard "IBM IBM IBM" across the screen at LED E1F1. On others you may only see the LED codes.

At LED E1F1 hit the number 8 key on the keyboard (not on the numeric keypad, but above the letters U and I). This will give you the Open Firmware prompt:

ok
0 >

9. Now boot AIX with the 'verbose' flag enabled to gather debug output. This will track every exec() during the boot cycle and the running system.

0 > boot -s verbose

You should see the cursor spin at the end of the line as the system boots. This output will be captured in the putty log on the PC.

The verbose debug mode will disable automatically at next boot.

Client-specific SEA statistics

2010-08-01T01:06:00.000+05:30

To gather network statistics at a per-client level, advanced accounting can be enabled on the SEA to provide more information about its network traffic.
To enable per-client statistics, the VIOS administrator can set the SEA accounting attribute to “enabled” (the default value is “disabled”, so that no per-client statistics are gathered by default). When advanced accounting is enabled, the SEA keeps track of the hardware (MAC) addresses of all of the packets it receives from the LPAR clients, and increments packet and byte counts for each client independently. After advanced accounting is enabled on the SEA, the VIOS administrator can generate a report to view per-client statistics by running the seastat command

Client-specific SEA statistics
To gather network statistics at a per-client level, advanced accounting can be enabled on the SEA to provide more information about its network traffic.
To enable per-client statistics, the VIOS administrator can set the SEA accounting attribute to “enabled” (the default value is “disabled”, so that no per-client statistics are gathered by default). When advanced accounting is enabled, the SEA keeps track of the hardware (MAC) addresses of all of the packets it receives from the LPAR clients, and increments packet and byte counts for each client independently. After advanced accounting is enabled on the SEA, the VIOS administrator can generate a report to view per-client statistics by running the seastat command.

Note:
Advanced accounting must be enabled on the SEA before the seastat command can print any statistics

To enable advanced accounting on the SEA, enter the following command:

chdev -dev -attr accounting=enabledThe following command displays per-client SEA statistics. The optional -n flag disables name resolution on IP addresses.

seastat -d [-n]The following command clears (zeroes out) all of the per-client SEA statistics
that have been gathered:

seastat -d -c

Unable to See New VSCSI Disk on Existing vhost

2010-08-01T01:05:00.001+05:30

Technote (FAQ)

Question
I added a new virtual target device to a vhost adapter that already had target devices on it, but the client cannot see the new virtual SCSI disk and a VSCSI_HOST error has been logged on the client. Why?

Answer
This is likely to be a problem with the max_transfer parameter. The maximum transfer parameter determines the size, in bytes, of the largest single data transfer request that can be handled by this adapter. As devices are added to a vhost adapter, the config method determines the maximum transfer each device is capable of by getting the maximum transfer size for that device and for the adapter to which that device is attached and using the smaller of those values. When the client adapter connects to
the host adapter, the host reports to the client the smaller value of all the devices defined to that host. The client then reports that value to the disk head driver on the client. When the disk head driver on the
client receives a request that is larger than the maximum transfer size, the disk head driver must break that request up into several smaller requests.

If a device is added to a host adapter after that host adapter has reported the maximum transfer size to the client, and the new device has a smaller transfer size than the value already reported, if the host showed that device to the client, the host could receive a request for that device that is larger than the device could handle. Therefore, the host does not show the device to the client and instead logs the informative error:

LABEL: VSCSI_HOST
IDENTIFIER: DF63A4FE

Date/Time: Mon Mar 5 11:47:21 2007
Sequence Number: 2205
Machine Id: 00C5763E4C00
Node Id: vios1
Class: S
Type: TEMP
Resource Name: vhost2

Description
Virtual SCSI Host Adapter detected an error

Probable Causes
Virtual SCSI Host Adapter Driver is an Undefined State

Failure Causes
Virtual SCSI Host Adapter Driver is an Undefined State

Recommended Actions
Remove Virtual SCSI Host Adapter Instance, then Configure the same instance

Detail Data
ADDITIONAL INFORMATION
module: target_report_luns rc: 00000000FFFFFFD8 location:00000001
data: 2 8F00000000000000

The best solution is to group devices according to maximum transfer sizes on host adapters. That is, if you know you will have devices with three different maximum transfer sizes in use by a single client, configure three host/client adapter pairs for that client and put the big transfer size devices on one, the medium size devices on the second, and the small devices on the third. That way, each device can run with the appropriate maximum transfer size and enjoy optimum throughput.

The second solution is to unconfigure the client adapter and all devices attached to it ("rmdev -l vscsiN -R") and reconfigure ("cfgmgr"), or reboot the client which does the same thing. However, the reconfigured client will now be using the smaller maximum transfer size so throughput will not be as good as it might be otherwise.

If a backing device (i.e. hdisk or logical volume) is exported through a vhost adapter, and its max_transfer size is greater than the max_transfer size of an already associated backing device of that vhost adapter, then the new virtual disk is not presented to the client partition. For example, suppose vhost1 has a backing device with a max_transfer size of 256K (0x40000), and you are trying to export a backing device whose max_transfer size is 1 MB (0x100000), when you run cfgmgr on the client, the client will not see the new virtual disk.

Therefore, a third alternative is to change the max_transfer size for the new device you are trying to export through that same vhost adapter to match the "largest" max_transfer size of the existing backing device(s). Then export that logical volume or disk as a backing device by running mkvdev command, and run cfgmgr on the client.

Note: The max_transfer size is a disk attribute. So, if you are exporting a logical volume as a backing device, you must check the max_transfer size of the underlying physical disk (i.e. if the logical volume, lv00, you are trying to export is on hdisk1, check the max_transfer size for hdisk1).

Example:

To check the max_transfer size for hdisk1, run

$ lsdev -dev hdisk1 -attr
...
max_transfer 0x100000 Maximum TRANSFER Size True
...
To change the max_transfer value, run

$ chdev -dev hdisk# -attr max_transfer=
where value is in hex (i.e. 0x40000)

To find out what physical volume the logical volume, rootvg_lv resides on, run

$ lslv -pv rootvg_lv
rootvg_lv:N/A
PV COPIES IN BAND DISTRIBUTION
hdisk0 001:000:000 100% 000:001:000:000:000

Note: If your VIOS level is FP11.1, and you have ruled out the max_transfer size as previously discussed, it is possible you may be running into a known defect addressed with IZ29603, which is included in FP11.1 SP 1. Consequently, the recommendation is to update the VIOS to that SP level or higher.

Sympthoms may include:

- existing client unable to see new virtual disk on existing and new virtual SCSI adapter
- existing VIO client may fail to see boot device
- new client unable to see virtual disk during the Installation menu

Setting Up and Using JFS2 Filesystem Quotas

2010-08-01T01:04:00.001+05:30

Technote (FAQ)

Question
How do I set up and manage quotas on a JFS2 filesystem?

Answer
Enable quotas on an existing JFS2 filesystem, or create a new filesystem with quotas enabled.
1. To change an existing filesystem:
# chfs -a quota=userquota /myfs
The other option is quota=groupquota, or both can be added

2. mount the filesystem
# mount /myfs

3. Make sure it mounted with quotas enabled:
# lsfs -q /myfs
Name Nodename Mount Pt VFS Size Options Auto Accounting
/dev/fslv02 -- /myfs jfs2 2097152 rw no no
(lv size: 2097152, fs size: 2097152, block size: 4096, sparse files: yes, inline log: no, inline log size: 0, EAformat: v1, Quota: userquota, DMAPI: no, VIX: no)

Set up a new class of quotas for a user

# j2edlimit -e /myfs

This will load the current quota class into a 'vi' editor session:

User Limits Classes for file system /myfs
Block Limits units: g=gigabyte, m=megabyte, or optional k=kilobyte
Grace Period units: d=days, h=hours, m=minutes, or s=seconds
Limits Class ID 0 is the default class.

Block Limits File Limits Grace Period
ID soft hard soft hard block file

0 0 0 0 0 0 0

Add a new class by creating a new line, using ID=+ and setting your limits:

+ 1g 2g 12000 15000 7d 7d

Save the file using :wq or ZZ.

The command should assign ID 1 to your new class, and list it out for you after the vi session. You can also list it using:

# j2edlimit -l /myfs
User Limits Classes for file system /myfs
Block Limits units: g=gigabyte, m=megabyte, or optional k=kilobyte
Grace Period units: d=days, h=hours, m=minutes, or s=seconds
Limits Class ID 0 is the default class.

Block Limits File Limits Grace Period
ID soft hard soft hard block file

0 0 0 0 0 0 0
1 1g 2g 12000 15000 7d 7d

Assign a user to your new limits class

# j2edlimit -a 1 -u bob /myfs
Where: 1 is the ID of the class you wish to assign
bob is the username

To report on a user's quota usage in a filesystem

# repquota /myfs
Block limits File limits
Group used soft hard grace used soft hard grace
system -- 64 0 0 4 0 0
staff -- 18756 0 0 1 0 0

Block limits File limits
User used soft hard grace used soft hard grace
root -- 32 0 0 4 0 0
bob -- 18756 1048576 2097152 1 12000 15000

To remove a user from a particular class

Reassign that user to class 0, which is unlimited.

# j2edlimit -a 0 bob /myfs
User Limits Classes for file system /myfs
Block Limits units: g=gigabyte, m=megabyte, or optional k=kilobyte
Grace Period units: d=days, h=hours, m=minutes, or s=seconds
Limits Class ID 0 is the default class.

Block Limits File Limits Grace Period
ID soft hard soft hard block file

0 0 0 0 0 0 0

Considerations while upgrading AIX with SDDPCM installed

2010-08-01T01:02:00.000+05:30

Considerations while upgrading AIX with SDDPCM installed

SDDPCM (Subsystem Device Driver Path Control Module) is the multi-pathing plug-in for AIX 5.3 and 6.1.

Customers who use IBM SVC, DS6000, DS8000 and/or ESS800 use this package to allow the operating system to handle multiple paths from OS to storage.
When you are upgrading AIX with SDDPCM installed, don’t forget to plan for SDDPCM too.
1) Planning an upgrade from AIX 5.3 to 6.1Before the AIX OS is upgraded, SDDPCM must be uninstalled and then reinstalled after the upgrade. There are cases when the host attachment script must also be uninstalled and reinstalled. This is explained in the SDD Users Guide found here:

2) Already upgraded AIX to 6.1 but still using AIX 5.3 version of SDDPCM?

You may need help from IBM before you can upgrade your SDDPCM to the AIX 6.1 version. This will come in the form of some special scripts.

3) General SDDPCM maintenance

There are quite a few SDDPCM flashes out there right now. You need to check these out and ensure you are not exposed to the issues that are corrected by later versions of SDDPCM. Check out the flashes listed here (or read my previous blog entry):

http://www-01.ibm.com/support/search.wss?rs=540&tc=ST52G7&dc=D600&dtm

What about SDD (Subsystem Device Driver) for AIX?
Prior to AIX 5.2 FP5, AIX did not offer native multi-pathing (MPIO). This meant that each hardware vendor had to offer their own third-party software to handle multiple paths. To achieve this with the ESS (Shark), IBM released a product called DPO (Data Path Optimiser). This product became SDD and was made available for a wide variety of operating systems.

When AIX offered MPIO, IBM then also offered a vendor plug-in (Path Contol Module) for native AIX MPIO which IBM called SDDPCM. This means you have two choices with AIX: SDD or SDDPCM. If your considering which is best, SDDPCM is my preference. This is because it is native to the operating system and also better supports the possibility of co-existence of multiple PCMs. Note that migrating from SDD to SDDPCM is not supported by the VIOS at this time, so if your running VIOS you will need to stay put for now.

EMC Powerpath and MPIO in a same box

2010-08-01T00:59:00.002+05:30

You can run into an issue with EMC storage on AIX systems using MPIO (No Powerpath) for your boot disks:

After installing the ODM_DEFINITONS of EMC Symmetrix on your client system, the system won’t boot any more and will hang with LED 554 (unable to find boot disk).

The boot hang (LED 554) is not caused by the EMC ODM package itself, but by the boot process not detecting a path to the boot disk if the first MPIO path does not corresponding to the fscsiX driver instance where all hdisks are configured. Let me explain that more in detail:

Let’s say we have an AIX system with four HBAs configured in the following order:

# lscfg -v | grep fcs

fcs2 (wwn 71ca) -> no devices configured behind this fscsi2 driver
instance (path only configured in CuPath ODM table)

fcs3 (wwn 71cb) -> no devices configured behind this fscsi3 driver
instance (path only configured in CuPath ODM table)

fcs0 (wwn 71e4) -> no devices configured behind this fscsi0 driver
instance (path only configured in CuPath ODM table)

fcs1 (wwn 71e5) -> ALL devices configured behind this fscsi1 driver
instance

Looking at the MPIO path configuration, here is what we have for the rootvg disk:

# lspath -l hdisk2 -H -F”name parent path_id connection status”

name parent path_id connection status

hdisk2 fscsi0 0 5006048452a83987,33000000000000 Enabled

hdisk2 fscsi1 1 5006048c52a83998,33000000000000 Enabled

hdisk2 fscsi2 2 5006048452a83986,33000000000000 Enabled

hdisk2 fscsi3 3 5006048c52a83999,33000000000000 Enabled

The fscsi1 driver instance is the second path (pathid 1), then remove the 3 paths keeping only the path corresponding to fscsi1 :

# rmpath -l hdisk2 -p fscsi0 -d

# rmpath -l hdisk2 -p fscsi2 -d

# rmpath -l hdisk2 -p fscsi3 -d
# lspath -l hdisk2 -H -F”name parent path_id connection status”

Afterwards, do a savebase to update the boot lv hd5. Set up the bootlist to hdisk2 and reboot the host.

It will come up successfully, no more hang LED 554.

When checking the status of the rootvg disk, a new hdisk10 has been configured with the correct ODM definitions as shown below:

# lspv

hdisk10 0003027f7f7ca7e2 rootvg active

# lsdev -Cc disk

hdisk2 Defined 00-09-01 MPIO Other FC SCSI Disk Drive

hdisk10 Available 00-08-01 EMC Symmetrix FCP MPIO Raid6

To summarize, it is recommended to setup ONLY ONE path when installing an AIX to a SAN disk, then install the EMC ODM package then reboot the host and only after that is complete, add the other paths. Dy doing that we ensure that the fscsiX driver instance used for the boot process has the hdisk configured behind.

Creating VLAN Tagged IEEE 802.1Q Virtual Ethernet Adapters in IVM

2010-08-01T00:59:00.001+05:30

Technote (FAQ)

Question
Is there a method for creating VLAN tagged virtual Ethernet adapters using the integrated virtual manager (IVM)?

Cause
The IVM GUI in v2.1 and lower does not provide GUI advanced options for creating multiple VLAN tagged Ethernet adapters. The limitation of the IVM GUI does not allow you to specify VLAN tagging when creating virtual ethernet devices, but you can achieve the configuration using CLI.

Answer
The IVM command line interface (CLI) does allow creating VLAN tagged (IEEE 802.1Q) virtual Ethernet adapters. There are two commands that will need to be used:

(1) chhwres is used on the VIO server to dynamically modify the VIO server LPAR only.

(2) chsyscfg is used to modify the client LPAR profile and can only be used when client is not activated.

Following example of creating virtual Ethernet adapters that support VLAN configurations illustrate using the chhwres and chsyscfg commands. The example shows commands for listing output of profile definitions of a client LPAR and then adding definitions for a new virtual adapter to the LPAR profile.

lssyscfg -m Server-9111-520-SN104797E -r prof --filter lpar_ids=2

Example output:
name=ivmclient,
lpar_name=ivmclient,
lpar_id=2,
os_type=aixlinux,
all_resources=0,
min_mem=256,desired_mem=256,max_mem=256,
proc_mode=shared,
min_proc_units=0.10,desired_proc_units=0.10,max_proc_units=2.00,
min_procs=1,desired_procs=1,max_procs=2,
sharing_mode=uncap,uncap_weight=128,
io_slots=none,
lpar_io_pool_ids=none,
max_virtual_slots=6,
"virtual_serial_adapters=0/server/1/any//any/1,1/server/1/any//any/1",
virtual_scsi_adapters=2/client/1/10-4797E_vios/11/1,
virtual_eth_adapters=5/0/1//0/0,
boot_mode=norm,conn_monitoring=0,auto_start=0,power_ctrl_lpar_ids=none

Note: The bold text output for virtual_eth_adapter variable in the example above shows the virtual device is in slot 5. The properties of the device are separated by a slash and designate parameters such as.
slot_num
ieee_virtual_eth
port_vlan_id
addl_vlan_ids
is_trunk
trunk_priority

Example of modifying the profile of an LPAR to add an additional virtual Ethernet adapter which will support the ability to receive VLAN 200 tagged traffic. The client LPAR needs to be shutdown before its profile can be changed.

chsyscfg -m Server-9111-520-SN104797E -r prof -i 'lpar_id=2,"virtual_eth_adapters=4/1/14/200/0/0,5/0/1//0/0"'

Checking the profile of LPAR 2 to see if changes were made can be done using the lssyscfg command..

lssyscfg -m Server-9111-520-SN104797E -r prof --filter lpar_ids=2 -F lpar_id,virtual_eth_adapters

Example output:
2,"4/1/14/200/0/0,5/0/1//0/0"

On the VIO server LPAR, when you do issue a command to modify the hardware. you can get the changed to dynamically take affect, but that is the special nature of a IVM environment. For example, the following IVM command dynamically alters the VIO server configuration.

chhwres -m Server-9111-520-SN104797E -p 10-4797E_vios -o a -r virtualio --rsubtype eth -s 14 -a
"ieee_virtual_eth=1,port_vlan_id=14,addl_vlan_ids=200,is_trunk=1,trunk_priority=1" -d 5

lshwres -m Server-9111-520-SN104797E -r virtualio --rsubtype eth --level lpar --filter lpar_ids=1
Example output:
lpar_name=10-4797E_vios,lpar_id=1,slot_num=3,state=1,ieee_virtual_eth=0,port_vlan_id=1,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001003
lpar_name=10-4797E_vios,lpar_id=1,slot_num=4,state=1,ieee_virtual_eth=0,port_vlan_id=2,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001004
lpar_name=10-4797E_vios,lpar_id=1,slot_num=5,state=1,ieee_virtual_eth=0,port_vlan_id=3,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001005
lpar_name=10-4797E_vios,lpar_id=1,slot_num=6,state=1,ieee_virtual_eth=0,port_vlan_id=4,addl_vlan_ids=none,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA290001006
lpar_name=10-4797E_vios,lpar_id=1,slot_num=14,state=1,ieee_virtual_eth=1,port_vlan_id=14,addl_vlan_ids=200,is_trunk=1,trunk_priority=1,is_required=0,mac_addr=EEA29000100F

On the VIO server, the chhwres command also invokes other commands that will cause the server profile to be modified to for the new device so it will be available on next boot. You can also verify the profile was changed using the lssyscfg command.

lssyscfg -m Server-9111-520-SN104797E -r prof --filter lpar_ids=1 -F lpar_ids,virutal_eth_adapters

Example output:
1,"3/0/1//1/0,4/0/2//1/0,5/0/3//1/0,6/0/4//1/0,14/1/14/200/1/0"

The IVM GUI or CLI interface can then be used to setup a virtual Ethernet bridge (SEA) between a physical adapter and a virtual adapter that was created with the additional VLAN properties.

IBM Power system microcode (firmware) naming conversion

2010-08-01T00:57:00.000+05:30

Difference between p5, p6 and p7 system firmware naming conventions?

On power5 systems all firmware levels began with SFXXX…. regardless of p5 system model. Starting in p6 and continuing on p7 servers they have different streams for the pSeries servers like EHXXX, ELXXX, EMXXX, ALXXX, AMXXX.

Does anyone have a paper, or a URL or can explain what servers will have these firmware naming conventions and why? Also, is the p6 firmware level naming convention the same as p7? Both p6 and p7 are definitely different than p5.

Explaination

Answer:
Not sure if all of this is the “formal” naming, but generally:

POWER5:
SF means “Squadrons Firmware”.

POWER6:
EH is Enterprise High-End
EM is Enterprise Mid-Range (formerly Intermediate-High)
EL is Enterprise Low-End

POWER7:
750 and 755 should be AL and 770/780 should be AM.
775, 790 and 795, or whatever comes out for the ultra high end would be AH.

I’m not sure, but A is probably Advanced. POWER7 shares architecture with their PowerPC A2 “wire-speed” products. It’s a substantial advancement on the NUMA technology they’ve been using since POWER5 (think Sequent acquisition).

Redundancy configuration using virtual fibre channel adapters With N_Port ID Virtualization (NPIV)

2010-08-01T00:49:00.005+05:30

Redundancy configurations help protect your network from physical adapter failures as well as Virtual I/O Server failures.

With N_Port ID Virtualization (NPIV), you can configure the managed system so that multiple logical partitions can access independent physical storage through the same physical fibre channel adapter. Each virtual fibre channel adapter is identified by a unique worldwide port name (WWPN), which means that you can connect each virtual fibre channel adapter to independent physical storage on a SAN.

Similar to virtual SCSI redundancy, virtual fibre channel redundancy can be achieved using Multi-path I/O (MPIO) and mirroring at the client partition. The difference between traditional redundancy with SCSI adapters and the NPIV technology using virtual fibre channel adapters, is that the redundancy occurs on the client, because only the client recognizes the disk. The Virtual I/O Server is essentially just a pipe. The second example below uses multiple Virtual I/O Server logical partitions to add redundancy at the Virtual I/O Server level as well.

Example: Host bus adapter failover
This example uses Host bus adapter (HBA) failover to provide a basic level of redundancy for the client logical partition. The figure shows the following connections:

The storage area network (SAN) connects physical storage to two physical fibre channel adapters located on the managed system.
The physical fibre channel adapters are assigned to the Virtual I/O Server and support NPIV.
The physical fibre channel ports are each connected to a virtual fibre channel adapter on the Virtual I/O Server. The two virtual fibre channel adapters on the Virtual I/O Server are connected to ports on two different physical fibre channel adapters in order to provide redundancy for the physical adapters.
Each virtual fibre channel adapter on the Virtual I/O Server is connected to one virtual fibre channel adapter on a client logical partition. Each virtual fibre channel adapter on each client logical partition receives a pair of unique WWPNs. The client logical partition uses one WWPN to log into the SAN at any given time. The other WWPN is used when you move the client logical partition to another managed system.
The virtual fibre channel adapters always has a one-to-one relationship between the client logical partitions and the virtual fibre channel adapters on the Virtual I/O Server logical partition. That is, each virtual fibre channel adapter that is assigned to a client logical partition must connect to only one virtual fibre channel adapter on the Virtual I/O Server, and each virtual fibre channel on the Virtual I/O Server must connect to only one virtual fibre channel adapter on a client logical partition.

The client can write to the physical storage through client virtual fibre channel adapter 1 or 2. If a physical fibre channel adapter fails, the client uses the alternative path. This example does not show redundancy in the physical storage, but rather assumes it would be built into the SAN.

Note: It is recommended that you configure virtual fibre channel adapters from multiple logical partitions to the same HBA, or you configure virtual fibre channel adapters from the same logical partition to different HBAs.
Example: HBA and Virtual I/O Server failover
This example uses HBA and Virtual I/O Server failover to provide a more advanced level of redundancy for the client logical partition. The figure shows the following connections:

The storage area network (SAN) connects physical storage to two physical fibre channel adapters located on the managed system.
There are two Virtual I/O Server logical partitions to provide redundancy at the Virtual I/O Server level.
The physical fibre channel adapters are assigned to their respective Virtual I/O Server and support NPIV.
The physical fibre channel ports are each connected to a virtual fibre channel adapter on the Virtual I/O Server. The two virtual fibre channel adapters on the Virtual I/O Server are connected to ports on two different physical fibre channel adapters in order to provide redundancy for the physical adapters. A single adapter could have multiple ports.
Each virtual fibre channel adapter on the Virtual I/O Server is connected to one virtual fibre channel adapter on a client logical partition. Each virtual fibre channel adapter on each client logical partition receives a pair of unique WWPNs. The client logical partition uses one WWPN to log into the SAN at any given time. The other WWPN is used when you move the client logical partition to another managed system.
The client can write to the physical storage through virtual fibre channel adapter 1 or 2 on the client logical partition through VIOS 2. The client can also write to physical storage through virtual fibre channel adapter 3 or 4 on the client logical partition through VIOS 1. If a physical fibre channel adapter fails on VIOS 1, the client uses the other physical adapter connected to VIOS 1 or uses the paths connected through VIOS 2. If VIOS 1 fails, then the client uses the path through VIOS 2. This example does not show redundancy in the physical storage, but rather assumes it would be built into the SAN.

Considerations
These examples can become more complex as you add physical storage redundancy and multiple clients, but the concepts remain the same. Consider the following points:

To avoid configuring the physical fibre channel adapter to be a single point of failure for the connection between the client logical partition and its physical storage on the SAN, do not connect two virtual fibre channel adapters from the same client logical partition to the same physical fibre channel adapter. Instead, connect each virtual fibre channel adapter to a different physical fibre channel adapter.
Consider load balancing when mapping a virtual fibre channel adapter on the Virtual I/O Server to a physical port on the physical fiber channel adapter.
Consider what level of redundancy already exists in the SAN to determine whether to configure multiple physical storage units.
Consider using two Virtual I/O Server logical partitions. Since the Virtual I/O Server is central to communication between logical partitions and the external network, it is important to provide a level of redundancy for the Virtual I/O Server. Multiple Virtual I/O Server logical partitions require more resources as well, so you should plan accordingly.
NPIV technology is useful when you want to move logical partitions between servers. For example, in active Partition Mobility, if you use the redundancy configurations above, in combination with physical adapters, you can stop all the I/O activity through the dedicated, physical adapter and direct all traffic through a virtual fibre channel adapter until the logical partition is successfully moved. The dedicated physical adapter would need to be connected to the same storage as the virtual path. Since you cannot migrate a physical adapter, all I/O activity is routed through the virtual path while you move the partition. After the logical partition is moved successfully, you need to set up the dedicated path (on the destination logical partition) if you want to use the same redundancy configuration you had configured on the original logical partition. Then the I/O activity can resume through the dedicated adapter, using the virtual fibre channel adapter as a secondary path.

PLM Configuration on AIX

2010-08-01T00:46:00.001+05:30

PLM can be easily configured using the instructions available from the Hardware InfoCenter. The main document title and links to all the other related steps is as follows.
Configuring resource management for AIX partitions with the Partition Load Manager:

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=/iphat_p5/iphbkconfigureplm.htm

This URL has links to key topic to assist in the
installation and configuration of PLM.

The topics are:

- Preparing to install the Partition Load Manager

- Installing OpenSSH software tools

- Installing the Partition Load Manager server

- Configuring the policy file

- Querying partition status

- Configuring Resource Monitoring and Control (RMC)

- Starting and stopping the Partition Load Manager server

- Commands for the Partition Load Manager

PLM is only valid for use with System p power 5 servers and typically this will also require a HMC at v6r1.x or lower. The latest patches for PLM should be installed prior to trying to start the PLM server. Additional information regarding PLM can be found at following URL.

Partition Load Manager
Support for System p
http://www14.software.ibm.com/webapp/set2/sas/f/plm/home.html

Related information
Partition Load Manager Support for System p

New Stuff coming soon... Not getting time to blogging...

2010-02-01T15:19:00.000+05:30

2008-09-03T17:36:00.001+05:30

Data availability

GPFS is fault tolerant and can be configured for continued access to data
even if cluster nodes or storage systems fail. This is accomplished though
robust clustering features and support for data replication.
GPFS continuously monitors the health of the file system components.
When failures are detected appropriate recovery action is taken automatically.
Extensive logging and recovery capabilities are provided which maintain
metadata consistency when application nodes holding locks or performing
services fail. Data replication is available for journal logs, metadata and data.
Replication allows for continuous operation even if a path to a disk or a disk
itself fails.
GPFS Version 3.2 further enhances clustering robustness with connection
retries. If the LAN connection to a node fails GPFS will automatically try and
reestablish the connection before making the node unavailable. This provides
for better uptime in environments experiencing network issues.
Using these features along with a high availability infrastructure ensures a
reliable enterprise storage solution.

GPFS Administration

2008-09-03T17:34:00.000+05:30

Administration

GPFS provides an administration model that is consistent with standard
AIX and Linux file system administration while providing extensions for the
clustering aspects of GPFS. These functions support cluster management
and other standard file system administration functions such as quotas,
snapshots, and extended access control lists.
GPFS provides functions that simplify cluster-wide tasks. A single GPFS
command can perform a file system function across the entire cluster and
most can be issued from any node in the cluster. These commands are
typically extensions to the usual AIX and Linux file system commands.
Rolling upgrades allow you to upgrade individual nodes in the cluster while
the file system remains online. With GPFS Version 3.1 you could mix GPFS
3.1 nodes with different patch levels. Continuing that trend in GPFS Version
3.2 you can run a cluster with a mix of GPFS Version 3.1 and GPFS Version
3.2 nodes.
Quotas enable the administrator to control and monitor file system usage
by users and groups across the cluster. GPFS provides commands to
generate quota reports including user, group and fileset inode and data block
usage. In addition to traditional quota management, GPFS has an API that
provides high performance metadata access enabling custom reporting
options on very large numbers of files.
A snapshot of an entire GPFS file system may be created to preserve the
file system's contents at a single point in time. This is a very efficient
mechanism because a snapshot contains a map of the file system at the time
it was taken and a copy of only the file system data that has been changed
since the snapshot was created. This is done using a copy-on-write technique.
The snapshot function allows a backup program, for example, to run
concurrently with user updates and still obtain a consistent copy of the file

system as of the time that the snapshot was created. Snapshots provide an
online backup capability that allows files to be recovered easily from common
problems such as accidental deletion of a file.
An SNMP interface is introduced in GPFS Version 3.2 to allow monitoring
by network management applications. The SNMP agent provides information
on the GPFS cluster and generates traps in the event a file system is mounted,
modified or if a node fails. In GPFS Version 3.2 the SNMP agent runs only on
Linux. You can monitor a mixed cluster of AIX and Linux nodes as long as the
agent runs on a Linux node.
GPFS provides support for the Data Management API (DMAPI) interface
which is IBM’s implementation of the X/Open data storage management API.
This DMAPI interface allows vendors of storage management applications
such as IBM Tivoli® Storage Manager (TSM) to provide Hierarchical Storage
Management (HSM) support for GPFS.
GPFS enhanced access control protects directories and files by providing a
means of specifying who should be granted access. On AIX, GPFS supports
NFS V4 access control lists (ACLs) in addition to traditional ACL support.
Traditional GPFS ACLs are based on the POSIX model. Access control lists
(ACLs) extend the base permissions, or standard file access modes, of read
(r), write (w), and execute (x) beyond the three categories of file owner, file
group, and other users, to allow the definition of additional users and user
groups. In addition, GPFS introduces a fourth access mode, control (c), which
can be used to govern who can manage the ACL itself.
In addition to providing application file service GPFS file systems may be
exported to clients outside the cluster through NFS or Samba. GPFS has
been used for a long time as the base for a scalable NFS file service
infrastructure. Now that feature is integrated in GPFS Version 3.2 and is called
clustered NFS. Clustered NFS provides all the tools necessary to run a GPFS
Linux cluster as a scalable NFS file server. This allows a GPFS cluster to
provide scalable file service by providing simultaneous access to a common
set of data from multiple nodes. The clustered NFS tools include monitoring of
file services, load balancing and IP address fail over.

GPFS file system

2008-09-03T17:27:00.000+05:30

The file system

A GPFS file system is built from a collection of disks which contain the file
system data and metadata. A file system can be built from a single disk or
contain thousands of disks, storing Petabytes of data. A GPFS cluster can
contain up to 256 mounted file systems. There is no limit placed upon the
number of simultaneously opened files within a single file system. As an
example, current GPFS customers are using single file systems up to 2PB in
size and others containing tens of millions of file.

Application interfaces

Applications can access files through standard UNIX® file system
interfaces or through enhanced interfaces available for parallel programs.
Parallel and distributed applications can be scheduled on GPFS clusters to
take advantage of the shared access architecture. This makes GPFS a key
component in many grid-based solutions. Parallel applications can
concurrently read or update a common file from multiple nodes in the cluster.
GPFS maintains the coherency and consistency of the file system using a
sophisticated byte level locking, token (lock) management and logging.
In addition to standard interfaces GPFS provides a unique set of extended
interfaces which can be used to provide high performance for applications with
demanding data access patterns. These extended interfaces are more
efficient for traversing a file system, for example, and provide more features
than the standard POSIX interfaces.

Performance and scalability

GPFS provides unparalleled performance especially for larger data objects
and excellent performance for large aggregates of smaller objects. GPFS
achieves high performance I/O by:
• Striping data across multiple disks attached to multiple nodes.
• Efficient client side caching.
• Supporting a large block size, configurable by the administrator, to fit
I/O requirements.
• Utilizing advanced algorithms that improve read-ahead and writebehind
file functions.
• Using block level locking based on a very sophisticated scalable token
management system to provide data consistency while allowing
multiple application nodes concurrent access to the files.
GPFS recognizes typical access patterns like sequential, reverse
sequential and random and optimizes I/O access for these patterns.
GPFS token (lock) management coordinates access to shared disks
ensuring the consistency of file system data and metadata when different
nodes access the same file. GPFS has the ability for multiple nodes to act as
token managers for a single file system. This allows greater scalability for high
transaction workloads.
Along with distributed token management, GPFS provides scalable
metadata management by allowing all nodes of the cluster accessing the file
system to perform file metadata operations. This key and unique feature
distinguishes GPFS from other cluster file systems which typically have a
centralized metadata server handling fixed regions of the file namespace. A
centralized metadata server can often become a performance bottleneck for
metadata intensive operations and can represent a single point of failure.
GPFS solves this problem by managing metadata at the node which is using
the file or in the case of parallel access to the file, at a dynamically selected
node which is using the file.

2008-09-03T17:26:00.001+05:30

What is GPFS?
The IBM General Parallel File System (GPFS) provides unmatched
performance and reliability with scalable access to critical file data. GPFS
distinguishes itself from other cluster file systems by providing concurrent
high-speed file access to applications executing on multiple nodes of an AIX
cluster, a Linux cluster, or a heterogeneous cluster of AIX and Linux nodes. In
addition to providing file storage capabilities, GPFS provides storage
management, information life cycle tools, centralized administration and
allows for shared access to file systems from remote GPFS clusters.
GPFS provides scalable high-performance data access from a two node
cluster providing a high availability platform supporting a database application,
for example, to 2,000 nodes or more used for applications like modeling
weather patterns. Up to 512 Linux nodes or 128 AIX nodes with access to
one or more file systems are supported as a general statement and larger
configurations exist by special arrangements with IBM. The largest existing
configurations exceed 2,000 nodes. GPFS has been available on AIX since
1998 and Linux since 2001. It has proven time and again on some of the
world's most powerful supercomputers1 to provide efficient use of disk
bandwidth.
GPFS was designed from the beginning to support high performance
computing (HPC) and has been proven very effective for a variety of
applications. It is installed in clusters supporting relational databases, digital
media and scalable file serving. These applications are used across many
industries including financial, retail and government applications. Being tested
in very demanding large environments makes GPFS a solid solution for any
size application.
GPFS supports various system types including the IBM System p™ family
and machines based on Intel® or AMD processors such as an IBM System
x™ environment. Supported operating systems for GPFS Version 3.2 include
AIX V5.3 and selected versions of Red Hat and SUSE Linux distributions.
This paper introduces a number of GPFS features and describes core
concepts. This includes the file system, high availability features, information
lifecycle management (ILM) tools and various cluster architectures.

IBM General Parallel File System ( GPFS )

2008-09-03T16:26:00.004+05:30

GPFS (General Parallel File System) is a high-performance shared-disk clustered file system developed by IBM.

Like some other cluster filesystems, GPFS provides concurrent high-speed file access to applications executing on multiple nodes of clusters. It can be used with AIX 5L clusters, Linux clusters, or a heterogeneous cluster of AIX and Linux nodes. In addition to providing filesystem storage capabilities, GPFS provides tools for management and administration of the GPFS cluster and allows for shared access to file systems from remote GPFS clusters.

GPFS has been available on AIX since 1998 and on Linux since 2001, and is offered as part of the IBM System Cluster 1350.

Versions of GPFS.

Versions:

GPFS 3.2, September 2007
GPFS 3.2.1-2, April 2008
GPFS 3.2.1-4, July 2008
GPFS 3.1
GPFS 2.3.0-29

Architecture

GPFS provides high performance by allowing data to be accessed over multiple computers at once. Most existing file systems are designed for a single server environment, and adding more file servers does not improve performance. GPFS provides higher input/output performance by "striping" blocks of data from individual files over multiple disks, and reading and writing these blocks in parallel. Other features provided by GPFS include high availability, support for heterogeneous clusters, disaster recovery, security, DMAPI, HSM and ILM.

Restoring the Virtual I/O Server

2008-08-16T20:21:00.000+05:30

As there are 4 different ways to backup the Virtual I/O Server, so there are 4 ways to restore it.

Restoring from a tape or DVD

To restore the Virtual I/O Server from tape or DVD, follow these steps:

1. specify the Virtual I/O Server partition to boot from the tape or DVD by
using the bootlist command or by altering the bootlist in SMS menu.
2. insert the tape/DVD into the drive.
3. from the SMS menu, select to install from the tape/DVD drive.
4. follow the installation steps according to the system prompts

Restoring the Virtual I/O Server from a remote file system using a nim_resources.tar file

To restore the Virtual I/O Server from a nim_resources.tar image in a file system, perform the following steps:

1. run the installios command without any flag from the HMC command line.
a) Select the Managed System where you want to restore your Virtual I/O Server
from the objects of type "managed system" found by installios command.
b) Select the VIOS Partition where you want to restore your system from the
objects of type "virtual I/O server partition" found

c) Select the Profile from the objects of type "profile" found.
d) Enter the source of the installation images [/dev/cdrom]:
server:/exported_dir
e) Enter the client's intended IP address:
f) Enter the client's intended subnet mask:
g) Enter the client's gateway:
h) Enter the client's speed [100]:
i) Enter the client's duplex [full]:
j) Would you like to configure the client's network after the installation
[yes]/no?

2. when the restoration is finished, open a virtual terminal connection (for
example, using telnet) to the Virtual I/O Server that you restored. Some
additional user input might be required

Note: The ability to run the installios command from the NIM server against the nim_resources.tar file is enabled with APAR IY85192.

Restoring the Virtual I/O Server from a remote file system using a mksysb image
To restore the Virtual I/O Server from a mksysb image in a file system using NIM, complete the following tasks:

1. define the mksysb file as a NIM object, by running the nim command.
#nim -o define -t mksysb -a server=master –a
location=/export/ios_backup/filename.mksysb objectname
objectname is the name by which NIM registers and recognizes the mksysb
file.
2. define a SPOT resource for the mksysb file by running the nim command.
#nim -o define -t spot -a server=master -a location=/export/ios_backup/
SPOT -a source=objectname SPOTname
SPOTname is the name of the SPOT resource for the mksysb file.
3. install the Virtual I/O Server from the mksysb file using the smit command.
#smit nim_bosinst
The following entry fields must be filled:
“Installation type” => mksysb
“Mksysb” => the objectname chosen in step1
“Spot” => the SPOTname chosen in step2
4. start the Virtual I/O Server logical partition.
a) On the HMC, right-click the partition to open the menu.
b) Click Activate. The Activate Partition menu opens with a selection of
partition profiles. Be sure the correct profile is highlighted.
c) Select the Open a terminal window or console session check box to open a
virtual terminal (vterm) window.
d) Click (Advanced...) to open the advanced options menu.
e) For the Boot mode, select SMS.
f) Click OK to close the advanced options menu.
g) Click OK. A vterm window opens for the partition.
h) In the vterm window, select Setup Remote IPL (Initial Program Load).
i) Select the network adapter that will be used for the installation.
j) Select IP Parameters.
k) Enter the client IP address, server IP address, and gateway IP address.
Optionally, you can enter the subnet mask. After you have entered these
values, press Esc to return to the Network Parameters menu.
l) Select Ping Test to ensure that the network parameters are properly
configured. Press Esc twice to return to the Main Menu.
m) From the Main Menu, select Boot Options.
n) Select Install/Boot Device.
o) Select Network.
p) Select the network adapter whose remote IPL settings you previously
configured.
q) When prompted for Normal or Service mode, select Normal.
r) When asked if you want to exit, select Yes.

Integrated Virtualization Manager (IVM) Consideration

If your Virtual I/O Server is managed by the IVM, prior to backup of your system, you need to backup your partition profile data for the management partition and its clients as IVM is integrated with Virtual I/O Server, but the LPARs profile is not saved with the backupios command.

There are two ways to perform this backup:
From the IVM Web Interface
1) From the Service Management menu, click Backup/Restore
2) Select the Partition Configuration Backup/Restore tab
3) Click Generate a backup

From the Virtual I/O Server CLI
1) Run the following command
#bkprofdata -o backup

Both these ways generate a file named profile.bak with the information about the LPARs configuration. While using the Web Interface, the default path for the file is /home/padmin. But if you perform the backup from CLI, the default path will be /var/adm/lpm. This path can be changed using the –l flag. Only ONE file can be present on the system, so each time the bkprofdata is issued or the Generate a Backup button is pressed, the file is overwritten.

To restore the LPARs profile you can use either the GUI or the CLI

From the IVM Web Interface
1) From the Service Management menu, click Backup/Restore
2) Select the Partition Configuration Backup/Restore tab
3) Click Restore Partition Configuration

From the Virtual I/O Server CLI
1) Run the following command
#rstprofdata –l 1 –f /home/padmin/profile.bak

It is not possible to restore a single partition profile. In order to restore LPARs profile, none of the LPARs profile included in the profile.bak must be defined in the IVM.

Backup of Virtual I/O Server

2008-08-16T20:07:00.001+05:30

Backing up the Virtual I/O Server

There are 4 different ways to backup/restore the Virtual I/O Server as illustrated in the following table.

Backup method Restore method
To tape From bootable tape
To DVD From bootable DVD
To remote file system From HMC using the NIMoL facility and installios
To remote file system From an AIX NIM server

Backing up to a tape or DVD-RAM

To backup the Virtual I/O Server to a tape or a DVD-RAM, the following steps must be performed

1. check the status and the name of the tape/DVD drive
#lsdev | grep rmt (for tape)
#lsdev | grep cd (for DVD)

2. if it is Available, backup the Virtual I/O Server with the following command
#backupios –tape rmt#
#backupios –cd cd#

If the Virtual I/O Server backup image does not fit on one DVD, then the backupios command provides instructions for disk replacement and removal until all the volumes have been created. This command creates one or more bootable DVDs or tapes that you can use to restore the Virtual I/O Server

Backing up the Virtual I/O Server to a remote file system by creating a nim_resources.tar file

The nim_resources.tar file contains all the necessary resources to restore the Virtual I/O Server, including the mksysb image, the bosinst.data file, the network boot image, and SPOT resource.
The NFS export should allow root access to the Virtual I/O Server, otherwise the backup will fail with permission errors.

To backup the Virtual I/O Server to a filesystem, the following steps must be performed

1. Create a mount directory where the backup file will be written
#mkdir /backup_dir

2. Mount the exported remote directory on the directory created in step 1.
#mount server:/exported_dir /backup_dir

3. Backup the Virtual I/O Server with the following command
#backupios –file /backup_dir

The above command creates a nim_resources.tar file that you can use to restore the Virtual I/O Server from the HMC.

Note: The ability to run the installios command from the NIM server against the nim_resources.tar file is enabled with APAR IY85192.

The backupios command empties the target_disk_data section of bosinst.data and sets RECOVER_DEVICES=Default. This allows the mksysb file generated by the command to be cloned to another logical partition. If you plan to use the nim_resources.tar image to install to a specific disk, then you need to repopulate the target_disk_data section of bosinst.data and replace this file in the nim_resources.tar. All other parts of the nim_resources.tar image must remain unchanged.

Procedure to modify the target_disk_data in the bosinst.data

1. Extract from the nim_resources.tar the bosinst.data
#tar -xvf nim_resources.tar ./bosinst.data

2. The following is an example of the target_disk_data stanza of the bosinst.data generated by backupios.
target_disk_data:
LOCATION =
SIZE_MB =
HDISKNAME =

3. Fill the value of HDISKNAME with the name of the disk to which you want to restore to

4. Put back the modified bosinst.data in the nim_resources.tar image
#tar -uvf nim_resources.tar ./bosinst.data

If you don't remember on which disk your Virtual I/O Server was previously installed, you can also view the original bosinst.data and look at the target_disk_data stanza.
Use the following steps

1. extract from the nim_resources.tar the bosinst.data
#tar -xvf nim_resources.tar ./bosinst.data
2. extract the mksysb from the nim_resources.tar
#tar -xvf nim_resources.tar ./5300-00_mksysb
3. extract the original bosinst.data
#restore -xvf ./5300-00_mksysb ./var/adm/ras/bosinst.data
4. view the original target_disk_data
#grep -p target_disk_data ./var/adm/ras/bosinst.data

The above command displays something like the following:

target_disk_data:
PVID = 00c5951e63449cd9
PHYSICAL_LOCATION = U7879.001.DQDXYTF-P1-T14-L4-L0
CONNECTION = scsi1//5,0
LOCATION = 0A-08-00-5,0
SIZE_MB = 140000
HDISKNAME = hdisk0

5. replace ONLY the target_disk_data stanza in the ./bosinst_data with the original one
6. add the modified file to the nim_resources.tar
#tar -uvf nim_resources.tar ./bosinst.data

Backing up the Virtual I/O Server to a remote file system by creating a mksysb image

You could also restore the Virtual I/O Server from a NIM server. One of the ways to restore from a NIM server is from the mksysb image of the Virtual I/O Server. If you plan to restore the Virtual I/O Server from a NIM server from a mksysb image, verify that the NIM server is at the latest release of AIX.

To backup the Virtual I/O Server to a filesystem the following steps must be performed

1. Create a mount directory where the backup file will be written
#mkdir /backup_dir
2. Mount the exported remote directory on the just created directory
#mount NIM_server:/exported_dir /backup_dir
3. Backup the Virtual I/O Server with the following command
#backupios –file /backup_dir/filename.mksysb -mksysb

Backup of Virtual I/O Server

2008-08-16T20:07:00.000+05:30

Updating VIO server Patch level update

2008-08-16T19:48:00.000+05:30

Applying updates from a local hard disk

To apply the updates from a directory on your local hard disk, follow one of these two procedures, depending on your currently installed level of VIOS.

A. If the current level of the VIOS is earlier than V1.2.0.0 (V1.0 or V1.1):

NOTE:
If you are updating from VIOS level 1.1, you must update to the 10.1 level of the Fix Pack before updating to the 11.1 level of Fix Pack. In other words, if you are at level 1.1, updating to the 11.1 Fix Pack is a two-step process: First, update to version 10.1 Fix Pack, and then update to the 11.1 Fix Pack.

Contact your IBM Service Representative to obtain the VIOS 10.1 Fix Pack.

After you install the 10.1 Fix Pack, follow these steps to install the 11.1 Fix Pack.

Login to the VIOS as the user padmin.
Create a directory on the Virtual I/O Server.
$ mkdir
Using ftp, transfer the update file(s) to the directory you created.
Apply the update by running the updateios command
$ updateios -accept -dev
Verify that update was successful by checking results of the updateios command and running the ioslevel command. It should indicate the ioslevel is now V1.5.2.1-FP-11.1.
$ ioslevel

B. If the current level of the VIOS is V1.2 through V1.5:

Login to the VIOS as the user padmin.
Create a directory on the Virtual I/O Server.
$ mkdir
Using ftp, transfer the update file(s) to the directory you created.
Apply the update by running the updateios command
$ updateios -accept -install -dev
Verify that update was successful by checking results of the updateios command and running the ioslevel command. It should indicate the ioslevel is now V1.5.2.1-FP-11.1.
$ ioslevel

NOTE:- If you are updating from an ioslevel prior to 1.3.0.1, the updateios command may indicate several failures (i.e. missing requisites) during fix pack installation. These messages are expected. Proceed with the update if you are prompted to "Continue with the installation [y/n]".

Applying updates from a remotely mounted file system

If the remote file system is to be mounted read-only, follow one of these two procedures, depending on your currently installed level of VIOS.

A. If the current level of the VIOS is earlier than V1.2.0.0 (V1.0 or V1.1):
NOTE:
If you are updating from VIOS level 1.1, you must update to the 10.1 level of the Fix Pack before updating to the 11.1 level of Fix Pack. In other words, if you are at level 1.1, updating to the 11.1 Fix Pack is a two-step process: First, update to version 10.1 Fix Pack, and then update to the 11.1 Fix Pack.

Contact your IBM Service Representative to obtain the VIOS 10.1 Fix Pack.

After you install the 10.1 Fix Pack, follow these steps to install the 11.1 Fix Pack.

Login to the VIOS as the user padmin.
Mount the remote directory onto the Virtual I/O Server.
$ mount remote_machine_name:directory /mnt
Apply the update by running the updateios command.
$ updateios -accept -dev /mnt
Verify that update was successful by checking results of the updateios command and running the ioslevel command. It should indicate the ioslevel is now V1.5.2.1-FP-11.1.
$ ioslevel
B. If the current level of the VIOS is V1.2 through V1.5:

Login to the VIOS as the user padmin.
Mount the remote directory onto the Virtual I/O Server.
$ mount remote_machine_name:directory /mnt
Apply the update by running the updateios command
$ updateios -accept -install -dev /mnt
Verify that update was successful by checking results of the updateios command and running the ioslevel command. It should indicate the ioslevel is now V1.5.2.1-FP-11.1.
$ ioslevel
Back to top
Applying updates from the CD/DVD driveThis fix pack can be burned onto a CD by using the ISO image file(s). After the CD has been created, follow one of these two procedures, depending on your currently installed level of VIOS.

A. If the current level of the VIOS is earlier than V1.2.0.0 (V1.0 or V1.1):
NOTE:
If you are updating from VIOS level 1.1, you must update to the 10.1 level of the Fix Pack before updating to the 11.1 level of Fix Pack. In other words, if you are at level 1.1, updating to the 11.1 Fix Pack is a two-step process: First, update to version 10.1 Fix Pack, and then update to the 11.1 Fix Pack.

Contact your IBM Service Representative to obtain the VIOS 10.1 Fix Pack.

After you install the 10.1 Fix Pack, follow these steps to install the 11.1 Fix Pack.

Login to the VIOS as the user padmin.
Place the CD-ROM into the drive assigned to VIOS.
Apply the update by running the updateios command
$ updateios -accept -dev /dev/cdX
where X is the device number 0-N assigned to VIOS
Verify that update was successful by checking results of the updateios command and running the ioslevel command. It should indicate the ioslevel is now V1.5.2.1-FP-11.1.
$ ioslevel
B. If the current level of the VIOS is V1.2 through V1.5:

Login to the VIOS as the user padmin
Place the CD-ROM into the drive assigned to VIOS
Apply the update by running the following updateios command:
$ updateios -accept -install -dev /dev/cdX
where X is the device number 0-N assigned to VIOS
Verify that update was successful by checking results of the updateios command and running the ioslevel command. It should indicate the ioslevel is now V1.5.2.1-FP-11.1.
$ ioslevel

Expanding rootvg disk in VIO environment where 2 VIO servers have been implemented for redundancy.

2008-08-16T13:24:00.000+05:30

This article depicts the procedure for expanding a rootvg volume group for a POWER5 LPAR where two VIO Servers have been implemented for redundancy. It also assumes that the rootvg is mirrored across both VIO Servers. This procedure is not supported by IBM, but does work.
POWER5 LPAR:
• Begin by unmirroring your rootvg and remove hdisk1 from the rootvg volume group. If there are any swap or dump devices on this disk you may need to remove them first before you can remove hdisk1 from the rootvg volume group.

• Once the disk has been removed from the rootvg, remove it from the LPAR by executing the following:
#rmdev -l hdisk1 - d

• Now you execute the bosboot command and update your bootlist now that hdisk1 has been removed and is no longer part of the system:
#bosboot -a bootlist -o -m normal hdisk0

VIO Server (where hdisk1 was created):

• Remove the device from the VIO Server using the rmdev command:
#rmdev -dev < bckcnim_hdisk1 >

• Next you will need to access the AIX* OS part of the VIO Server by executing:
#oem_setup_env

• Now you have two options: you can extend the existing logical volume or create a new one if there is more than enough disk space left. In this example I will be using bckcnim_lv. smitty extendlv and add additonal LP's or smitty mklv

• Exit out of oem_setup_env by just typing "exit" at the OS prompt.

• Now that you are back within the restricted shell of the VIO Server, execute the following command. You can use whatever device name you wish. I used bckcnim_hdisk1 just for example purposes:
#mkvdev -vdev bckcnim_lv -vadapter < vhost# > -dev bckcnim_hdisk1

POWER5 LPAR:
• Execute cfgmgr to add the new hdisk1 back to LPAR:
#cfgmgr

• Add hdisk1 back to the rootvg volume group using the extendvg or smitty extendvg.

• Mirror rootvg using the mirrorvg command or smitty mirrorvg

• Sync the mirroring process to the background and wait to complete. This is very important and must complete before dealing with what represents the hdisk0 logical volume.

• Now you must execute bosboot again and update the bootlist again:
#bosboot -a
#bootlist -o -m normal hdisk0 hdisk1

Recovering a Failed VIO Disk

2008-08-15T12:34:00.000+05:30

Recovering a Failed VIO Disk

Here is a recovery procedure for replacing a failed client disk on a Virtual IO
server. It assumes the client partitions have mirrored (virtual) disks. The
recovery involves both the VIO server and its client partitions. However,
it is non disruptive for the client partitions (no downtime), and may be
non disruptive on the VIO server (depending on disk configuration). This
procedure does not apply to Raid5 or SAN disk failures.

The test system had two VIO servers and an AIX client. The AIX client had two
virtual disks (one disk from each VIO server). The two virtual disks
were mirrored in the client using AIX's mirrorvg. (The procedure would be
the same on a single VIO server with two disks.)

The software levels were:

p520: Firmware SF230_145 VIO Version 1.2.0 Client: AIX 5.3 ML3

We had simulated the disk failure by removing the client LV on one VIO server. The
padmin commands to simulate the failure were:

#rmdev -dev vtscsi01 # The virtual scsi device for the LV (lsmap -all)
#rmlv -f aix_client_lv # Remove the client LV

This caused "hdisk1" on the AIX client to go "missing" ("lsvg -p rootvg"....The
"lspv" will not show disk failure...only the disk status at the last boot..)

The recovery steps included:

VIO Server

Fix the disk failure, and restore the VIOS operating system (if necessary)mklv -lv aix_client_lv rootvg 10G # recreate the client LV mkvdev -vdev aix_client_lv -vadapter vhost1 # connect the client LV to the appropriate vhost

AIX Client

# cfgmgr # discover the new virtual hdisk2
replacepv hdisk1 hdisk2
# rebuild the mirror copy on hdisk2
# bosboot -ad /dev/hdisk2 ( add boot image to hdisk2)
# bootlist -m normal hdisk0 hdisk2 ( add the new disk to the bootlist)

# rmdev -dl hdisk1 ( remove failed hdisk1)

The "replacepv" command assigns hdisk2 to the volume group, rebuilds the mirror, and
then removes hdisk1 from the volume group.

As always, be sure to test this procedure before using in production.

Configuring MPIO for the virtual AIX client

2008-08-15T12:31:00.000+05:30

Virtual SCSI Server Adapter and Virtual Target Device.
The mkvdev command will error out if the same name for both is used.

$ mkvdev -vdev hdiskpower0 -vadapter vhost0 -dev hdiskpower0
Method error (/usr/lib/methods/define -g -d):
0514-013 Logical name is required.

The reserve attribute is named differently for an EMC device than the attribute
for ESS or FasTt storage device. It is “reserve_lock”.

Run the following command as padmin for checking the value of the attribute.
$ lsdev -dev hdiskpower# -attr reserve_lock

Run the following command as padmin for changing the value of the attribute.
$ chdev -dev hdiskpower# -attr reserve_lock=no

•Commands to change the Fibre Channel Adapter attributes And also change the following attributes of the fscsi#, fc_err_recov to “fast_fail” and dyntrk to “yes”

$ chdev -dev fscsi# -attr fc_err_recov=fast_fail dyntrk=yes –perm

The reason for changing the fc_err_recov to “fast_fail” is that if the Fibre
Channel adapter driver detects a link event such as a lost link between a storage
device and a switch, then any new I/O or future retries of the failed I/Os will be
failed immediately by the adapter until the adapter driver detects that the device
has rejoined the fabric. The default setting for this attribute is 'delayed_fail’.
Setting the dyntrk attribute to “yes” makes AIX tolerate cabling changes in the
SAN.

The VIOS needs to be rebooted for fscsi# attributes to take effect.

VIO VLAN Setup

2008-08-15T12:24:00.001+05:30

HA VIO server setup

2008-08-15T12:22:00.000+05:30

VIO server Detail

2008-08-15T12:21:00.000+05:30

VIO Server General setup

2008-08-15T12:18:00.000+05:30

VIO Commands

2008-07-02T19:48:00.007+05:30

VIO Server Commands

lsdev –virtual (list all virtual devices on VIO server partitions)
lsmap –all (lists mapping between physical and logical devices)
oem_setup_env (change to OEM [AIX] environment on VIO server)

Create Shared Ethernet Adapter (SEA) on VIO Server

mkvdev –sea{physical adapt} –vadapter {virtual eth adapt} –default {dflt virtual adapt} –defaultid {dflt vlan ID}
SEA Failover
ent0 – GigE adapter
ent1 – Virt Eth VLAN1 (Defined with a priority in the partition profile)
ent2 – Virt Eth VLAN 99 (Control)
mkvdev –sea ent0 –vadapter ent1 –default ent1 –defaultid 1 –attr ha_mode=auto ctl_chan=ent2
(Creates ent3 as the Shared Ethernet Adapter)

Create Virtual Storage Device Mapping

mkvdev –vdev {LV or hdisk} –vadapter {vhost adapt} –dev {virt dev name}
Sharing a Single SAN LUN from Two VIO Servers to a Single VIO Client LPAR
hdisk = SAN LUN (on vioa server)
hdisk4 = SAN LUN (on viob, same LUN as vioa)
chdev –dev hdisk3 –attr reserve_policy=no_reserve (from vioa to prevent a reserve on the disk)
chdev –dev hdisk4 –attr reserve_policy=no_reserve (from viob to prevent a reserve on the disk)
mkvdev –vdev hdisk3 –vadapter vhost0 –dev hdisk3_v (from vioa)
mkvdev –vdev hdisk4 –vadapter vhost0 –dev hdisk4_v (from viob)
VIO Client would see a single LUN with two paths.
spath –l hdiskx (where hdiskx is the newly discovered disk)
This will show two paths, one down vscsi0 and the other down vscsi1.

VIO command from HMC
#viosvrcmd -m -p -c "lsmap -all

(this works only with IBM VIO Server)

see man viosvrcmd for more information

VIO Server Installation & Configuration

2008-07-02T19:48:00.006+05:30

IBM Virtual I/O Server
The Virtual I/O Server is part of the IBM eServer p5 Advanced Power Virtualization hardware feature. Virtual I/O Server allows sharing of physical resources between LPARs including virtual SCSI and virtual networking. This allows more efficient utilization of physical resources through sharing between LPARs and facilitates server consolidation.

Installation
You have two options to install the AIX-based VIO Server:
1. Install from CD
2. Install from network via an AIX NIM-Server

Installation method
#1 is probably the more frequently used method in a pure Linux environment as installation method #2 requires the presence of an AIX NIM (Network Installation Management) server. Both methods differ only in the initial boot step and are then the same. They both lead to the following installation screen:

IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM STARTING SOFTWARE IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM PLEASE WAIT... IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMIBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBMElapsed time since release of system processors: 51910 mins 20 secs

------------------------------------------------------------------------------- Welcome to the Virtual I/O Server. boot image timestamp: 10:22 03/23 The current time and date: 17:23:47 08/10/2005 number of processors: 1 size of memory: 2048MB boot device: /pci@800000020000002/pci@2,3/ide@1/disk@0:\ppc\chrp\bootfile.exeSPLPAR info: entitled_capacity: 50 platcpus_active: 2This system is SMT enabled: smt_status: 00000007; smt_threads: 2 kernel size: 10481246; 32 bit kernel
-------------------------------------------------------------------------------

The next step then is to define the system console. After some time you should see the following screen:

******* Please define the System Console. *******Type a 1 and press Enter to use this terminal as the system console.

Then Choose language of installation

>>> 1 Type 1 and press Enter to have English during install.

This is the main installation menu of the AIX-based VIO-Server:

Welcome to Base Operating System
Installation and Maintenance
Type the number of your choice and press Enter. Choice is indicated by >>>.>>>

1 Start Install Now with Default Settings
2 Change/Show Installation Settings and Install
3 Start Maintenance Mode for System Recovery

88 Help ? 99 Previous Menu

>>> Choice [1]:

Select Hard disk where you need to install VIO base operating system as we do in AIX Base operating system.

Once the installation is over. You will get login Prompt similar to AIX server.

VIO server is nothing but AIX on top of that Virtualisation software loaded on it. Generally on VIO server we do not host any application. Its basically used for sharing I/O resources ( DISK & Network ) to the client LPAR hosted in same Physical server.

Initial setup
After the reboot you are presented with the VIO-Server login prompt. You can't login as user root as you have to use the special user id padmin. No initial default password is set. Immediately after login you are forced to set a new password.

Before you can do anything you have to accept the I/O Server license.
This is done with the license command

#license -accept

Once you are logged in as user padmin you find yourself in a restricted Korn shell with only a limited set of commands. You can see all available commands with the command help. All these commands are shell aliases to a single SUID-binary called ioscli which is located in the directory /usr/ios/cli/bin. If you are familiar with AIX you will recognize most commands but most command line parameters differ from the AIX versions.
As there are no man pages available you can see all options for each command separately by issueing the command help . Here is an example for the command lsmap:

$ help lsmap
Usage: lsmap {-vadapter ServerVirtualAdapter -plc PhysicalLocationCode
-all}
[-net] [-fmt delimiter]
Displays the mapping between physical and virtual devices.
-all Displays mapping for all the server virtual adapter
devices.
-vadapter Specifies the server virtual adapter device
by device name.
-plc Specifies the server virtual adapter device
by physical location code.
-net Specifies supplied device is a virtual server
Ethernet adapter.
-fmt Divides output by a user-specified delimiter.

A very important command is oem_setup_env which gives you access to the regular AIX command line interface. This is provided solely for the installation of OEM device drivers

Virtual SCSI setup

To map a LV
# mkvg: creates the volume group, where a new LV will be created using the mklv command
# lsdev: shows the virtual SCSI server adapters that could be used for mapping with the LV
# mkvdev: maps the virtual SCSI server adapter to the LV
# lsmap -all: shows the mapping information

To map a physical disk
# lsdev: shows the virtual SCSI server adapters that could be used for mapping with a physical disk
# mkvdev: maps the virtual SCSI server adapter to a physical disk
# lsmap -all: shows the mapping information

Client partition commands

No commands needed, the Linux kernel is notified immediately

Create new volume group datavg with member disk hdisk1
# mkvg -vg datavg hdisk1

Create new logical volume vdisk0 in volume group
# mklv -lv vdisk0 datavg 10G

Maps the virtual SCSI server adapter to the logical volume
# mkvdev -vdev vdisk0 -vadapter vhost0

Display the mapping information
#lsmap -all

Virtual Ethernet setup

To list all virtual and physical adapters use the lsdev -type adapter command.

$ lsdev -type adapter

name status description
ent0 Available 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent1 Available 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent2 Available Virtual I/O Ethernet Adapter (l-lan)
ide0 Available ATA/IDE Controller Device
sisscsia0 Available PCI-X Dual Channel Ultra320 SCSI Adapter
vhost0 Available Virtual SCSI Server Adapter
vhost1 Available Virtual SCSI Server Adapter
vhost2 Available Virtual SCSI Server Adapter
vhost3 Available Virtual SCSI Server Adapter
vsa0 Available LPAR Virtual Serial Adapter

Choose the virtual Ethernet adapter we want to map to the physical Ethernet adapter.

$ lsdev -virtualname status description
ent2 Available Virtual I/O Ethernet Adapter (l-lan)
vhost0 Available Virtual SCSI Server Adapter
vhost1 Available Virtual SCSI Server Adapter
vhost2 Available Virtual SCSI Server Adapter
vhost3 Available Virtual SCSI Server Adapter
vsa0 Available LPAR Virtual Serial Adapter

The command mkvdev maps a physical adapter to a virtual adapter, creates a layer 2 network bridge and defines the default virtual adapter with its default VLAN ID. It creates a new Ethernet interface, e.g., ent3.
Make sure the physical and virtual interfaces are unconfigured (down or detached).

Scenario A (one VIO server)
Create a shared ethernet adapter ent3 with a physical one (ent0) and a virtual one (ent2) with PVID 1:

$ mkvdev -sea ent0 -vadapter ent2 -default ent2 -defaultid 1
ent3 Available
en3
et3

This has created a new shared ethernet adapter ent3 (you can verify that with the lsdev command). Now configure the TCP/IP settings for this new shared ethernet adapter (ent3). Please note that you have to specify the interface (en3) and not the adapter (ent3).

$ mktcpip -hostname op710-1-vio -inetaddr 9.156.175.231 -interface en3 -netmask 255.255.255.0 -gateway 9.156.175.1 -nsrvaddr 9.64.163.21 -nsrvdomain ibm.com

Scenario B (two VIO servers)
Create a shared ethernet adapter ent3 with a physical one (ent0) and a virtual one (ent2) with PVID 1:

$ mkvdev -sea ent0 -vadapter ent2 -default ent2 -defaultid 1

Configure the TCP/IP settings for the new shared ethernet adapter (ent3):

$mktcpip -hostname op710-1-vio -inetaddr 9.156.175.231 -interface en3 -netmask 255.255.255.0 -gateway 9.156.175.1 -nsrvaddr 9.64.163.21 -nsrvdomain ibm.com

Client partition commands
No new commands needed just the typical TCP/IP configuration is done on the virtual Ethernet interface that it is defined in the client partition profile on the HMC

Creating LPAR from command line from HMC

2008-06-19T18:51:00.000+05:30

Create new LPAR using command line

mksyscfg -r lpar -m MACHINE -i name=LPARNAME, profile_name=normal, lpar_env=aixlinux, shared_proc_pool_util_auth=1,min_mem=512, desired_mem=2048, max_mem=4096, proc_mode=shared, min_proc_units=0.2, desired_proc_units=0.5,max_proc_units=2.0, min_procs=1, desired_procs=2, max_procs=2, sharing_mode=uncap, uncap_weight=128,boot_mode=norm, conn_monitoring=1, shared_proc_pool_util_auth=1

Note :- Use man mksyscfg command for all flag information.

Onother method of creating LPAR through configuration file we need to create more than one lPAR at same time

Here is an example for 2 LPARs, each definition starting at new line:

name=LPAR1,profile_name=normal,lpar_env=aixlinux,all_resources=0,min_mem=1024,desired_mem=9216,max_mem=9216,proc_mode=shared,min_proc_units=0.3,desired_proc_units=1.0,max_proc_units=3.0,min_procs=1,desired_procs=3,max_procs=3,sharing_mode=uncap,uncap_weight=128,lpar_io_pool_ids=none,max_virtual_slots=10,"virtual_scsi_adapters=6/client/4/vio1a/11/1,7/client/9/vio2a/11/1","virtual_eth_adapters=4/0/3//0/1,5/0/4//0/1",boot_mode=norm,conn_monitoring=1,auto_start=0,power_ctrl_lpar_ids=none,work_group_id=none,shared_proc_pool_util_auth=1
name=LPAR2,profile_name=normal,lpar_env=aixlinux,all_resources=0,min_mem=1024,desired_mem=9216,max_mem=9216,proc_mode=shared,min_proc_units=0.3,desired_proc_units=1.0,max_proc_units=3.0,min_procs=1,desired_procs=3,max_procs=3,sharing_mode=uncap,uncap_weight=128,lpar_io_pool_ids=none,max_virtual_slots=10,"virtual_scsi_adapters=6/client/4/vio1a/12/1,7/client/9/vio2a/12/1","virtual_eth_adapters=4/0/3//0/1,5/0/4//0/1",boot_mode=norm,conn_monitoring=1,auto_start=0,power_ctrl_lpar_ids=none,work_group_id=none,shared_proc_pool_util_auth=1

Copy this file to HMC and run:

mksyscfg -r lpar -m SERVERNAME -f /tmp/profiles.txt

where profiles.txt contains all LPAR informations as mentioned above.

To change setting of your Lpar use chsyscfg command as mentioned below.

Virtual scsi creation & Mapping Slots
#chsyscfg -m Server-9117-MMA-SNXXXXX -r prof -i 'name=server_name,lpar_id=xx,"virtual_scsi_adapters=301/client/4/vio01_server/301/0,303/client/4/vio02/303/0,305/client/4/vio01_server/305/0,307/client/4/vio02_server/307/0"'

IN Above mentioned command we are creating Virtual scsi adapter for client LPAR & doing Slot mapping with VIO servers. In above scenario there is two VIO servers for redundancy.

Slot Mapping

Vio01_server ( VSCSI server slot) Client ( Vscsi client Slot)
Slot 301 Slot 301
Slot 303 Slot 303

VIO02_server (VSCSI sever Slot) Client ( VSCSI client Slot)
Slot 305 Slot 305
Slot 307 Slot 307

These Slot are mapped in such a way if Any disk or logical volume are mapped to Virtuals scsi adapter through VIO command "mkvdev".

Syntax for Virtual scsi adapter

virtual-slot-number/client-or-server/supports-HMC/remote-lpar-ID/remote-lpar-name/remote-slot-number/is-required

As in command above mentioned command mksyscfg "virtual_scsi_adapters=301/client/4/vio01_server/301/0"

means

301 - virtual-slot-number
client-or-server - client (Aix_client)
4 -- Partiotion Id ov VIO_01 server (remote-lpar-ID)
vio01_server - remote-lpar-name
301 -- remote-slot-number (VIO server_slot means virtual server scsi slot)
1 -- Required slot in LPAR ( It cannot be removed from DLPAR operations )
0 --means desired ( it can be removed by DLPAR operations)

To add Virtual ethernet adapter & slot mapping for above created profile

#chsyscfg -m Server-9117-MMA-SNxxxxx -r prof -i 'name=server_name,lpar_id=xx,"virtual_eth_adapters=596/1/596//0/1,506/1/506//0/1,"'

Syntax for Virtual ethernet adapter

slot_number/is_ieee/port_vlan_id/"additional_vlan_id,additional_vlan_id"/is_trunk(number=priority)/is_required

means

So the adapter with this setting 596/1/596//0/1 would say it is in slot_number 596, Its is ieee, the port_vlan_id is 1, it has no VLAN id assigned, It is not a trunk adapter and it is required.

Listing LPAR information from HMC command line interface

2008-06-12T14:53:00.003+05:30

To list managed system (CEC) managed by HMC

# lssyscfg -r sys -F name

To list number of LPAR defined on the Managed system (CEC)

# lssyscfg -m SYSTEM(CEC) -r lpar -F name,lpar_id,state

To list LPAR created in your system use lsyscfg command as mentioned below.

# lssyscfg -r prof -m SYSTEM(CEC) --filter "lpar_ids=X, profiles_names=normal"

Flags

m-> Managed System name
lpar_ids -> Lpar ID (numeric Id for each LPAR created in the Managed system (CEC)
profile_name -> To choose profile of LPAR

To start console of LPAR from HMC

# mkvterm -m SYSTEM(CEC) --id X

m- > managed system (ex -p5-570_xyz)
id - > LPAR ID

To finish a VTERM, simply press ~ followed by a dot .!

To disconnect console of LPAR from HMC

# rmvterm -m SYSTEM(CEC) --id x

To access LPAR console for diffrent Managed system from HMC

#vtmenu

Activating Partition

hscroot@hmc-570:~> lssyscfg -m Server-9110-510-SN100xxx -r lpar -F name,lpar_id,state,default_profile VIOS1.3-FP8.0,1,Running,default linux_test,2,Not Activated,client_default hscroot@hmc-570:~> chsysstate -m Server-9110-510-SN100xxxx -r lpar -o on -b norm --id 2 -f client_default

The above example would boot the partition in normal mode. To boot it into SMS menu use -b sms and to boot it to the OpenFirmware prompt use -b of.

To restart a partition the chsysstate command would look like this:

hscroot@hmc-570:~> chsysstate -m Server-9110-510-SN100xxxx -r lpar --id 2 -o shutdown --immed --restart

And to turn it off - if anything else fails - use this:
hscroot@hmc-570:~> chsysstate -m Server-9110-510-SN100xxxx -r lpar --id 2 -o shutdown --immed
hscroot@hmc-570:~> lssyscfg -m Server-9110-510-SN100xxxx -r lpar -F name,lpar_id,state
VIOS1.3-FP8.0,1,Running
linux_test,2,Shutting Down

Deleting Partition

hscroot@hmc-570:~> lssyscfg -m Server-9110-510-SN100xxxx -r lpar -F name,lpar_id
VIOS1.3-FP8.0,1
linux_test,2
hscroot@hmc-570:~> rmsyscfg -m Server-9110-510-SN100xxxx -r lpar --id 2
hscroot@hmc-570:~> lssyscfg -m Server-9110-510-SN100xxxx -r lpar -F name,lpar_id
VIOS1.3-FP8.0,1

Enabling the Advanced POWER Virtualization Feature

2008-05-03T18:48:00.000+05:30

Enabling the Advanced POWER Virtualization Feature

Before we could use the virtual I/O, we had to determine whether the machine was enabled to use the feature. To do this, we right-clicked on the name of the target server in the HMC’s ‘Server and Partition’ view and looked at that server’s properties. Figure 4 shows it did not have the feature enabled.

Users can enable this feature by obtaining a key code from their IBM sales representative using information that the HMC gathers about their machine when the user navigates to Show Code Information in the HMC. Figure 5 shows how to navigate there as well as how to get to the HMC dialog box used to enter the activation code which renders the system VIO-capable. We obtained an access code and entered it in the dialog box in Figure.

VIO server setup example

2008-05-03T18:32:00.001+05:30

Virtual I/O Example
A user who currently runs applications on a POWER4 system may want to upgrade to a POWER5 system running AIX 5.3 in order to take advantage of virtual I/O. If so, do these three things:
y Create a Virtual I/O Server. y Add virtual LANs. y Define virtual SCSI devices.
In our example, we had an IBM eServer p5 550 Express with four CPUs that was running one AIX 5.3 database server LPAR, and we needed to create a second application server LPAR that uses a virtual SCSI disk as its boot disk. We wanted to share one Ethernet adapter between the database and application server LPARs and use this shared adapter to access an external network. Finally, we needed a private network between the two LPARs and we decided to implement it using virtual Ethernet devices (see Figure 3). We followed these steps to set up our system:

1. Enabled the Advanced POWER Virtualization feature.

2.Installed the Virtual I/O Server. y Created the Public Ethernet VLAN.

3. Installed the Virtual SCSI Devices.

4. Installed the Private Ethernet VLAN.

Virtual I/O Server installation overview

2008-05-03T17:47:00.003+05:30

The Virtual I/O Server The Virtual I/O Server is a dedicated partition that runs a special operating system called IOS. This special type of partition has physical resources assigned to it in its HMC profile. The administrator issues server partition IOS commands to create virtual resources which present virtual LAN, virtual SCSI adapters, and virtual disk drives client partitions. The client partition’s operating systems recognize these resources as physical devices. The Virtual I/O Server is responsible for managing the interaction between the client LPAR and the physical device supporting the virtualized service. Once the administrator logs in to the Virtual I/O Server as the user padmin, he or she has access to a restricted Korn shell session. The administrator uses IOS commands to create, change, and remove these physical and virtual devices as well as to configure and manage the VIO server. Executing the help command on the VIO server command line lists the commands that are available in padmin’s restricted Korn Shell session

Virtual I/O Server installation

VIO Server code is packaged and shipped as an AIX mksysb image
on a VIO DVD
Installation methods
– DVD install
– HMC install - Open rshterm and type “installios”; follow the
prompts
– Network Installation Manager (NIM)
VIO Server can support multiple client types
– AIX 5.3
– SUSE Linux Enterprise Server 9 or 10 for POWER
– Red Hat Enterprise Linux AS for POWER Version 3 and 4

Virtual I/O Server Administration
The VIO server uses a command line interface running in a restricted shell
– no smitty or GUI
There is no root login on the VIO Server
A special user – padmin – executes VIO server commands
First login after install, user padmin is prompted to change password
After that, padmin runs the command “license –accept”
Slightly modified commands are used for managing devices, networks,
code installation and maintenance, etc.
The padmin user can start a root AIX shell for setting up third-party
devices using the command “oem_setup_env”

We can get all commands by executing help on padmin user id

$ help
Install Commands
Physical Volume Commands
Security Commands
updateios
lspv
lsgcl
lssw
migratepv
cleargcl
ioslevel
lsfailedlogin
remote_management
Logical Volume Command
oem_setup_env
lslv
UserID Commands
oem_platform_level
mklv
mkuser
license
extendlv
rmuser
rmlv
lsuser
LAN Commands
mklvcopy
passwd
mktcpip
rmlvcopy
chuser
hostname
cfglnagg
netstat
Volume Group Commands
Maintenance Commands
entstat
lsvg
chlang
cfgnamesrv
mkvg
diagmenu
traceroute
chvg
shutdown
ping
extendvg
fsck
optimizenet
reducevg
backupios
lsnetsvc
mirrorios
savevgstruct
unmirrorios
restorevgstruct
Device Commands
activatevg
starttrace
mkvdev
deactivatevg
stoptrace
lsdev
importvg
cattracerpt
lsmap
exportvg
bootlist
chdev
syncvg
snap
rmdev
startsysdump

cfgdev
topas
mkpath
mount
chpath
unmount
lspath
showmount
rmpath
startnetsvc
errlog
stopnetsvc

Virtual I/O Server Overview

2008-05-03T17:26:00.003+05:30

What is Advanced POWER Virtualization (APV)
APV – the hardware feature code for POWER5 servers that enables:
– Micro-partitioning – fractional CPU entitlements from a shared pool of
processors, beginning at one-tenth of a CPU
– Partition Load Manager (PLM) – a policy-based, dynamic CPU and
memory reallocation tool
– Physical disks can be shared as virtual disks to client partitions
– Shared Ethernet Adapter (SEA) – A physical adapter or EtherChannel in
a VIO Server can be shared by client partitions. Clients use virtual
Ethernet adapters
Virtual Ethernet – a LPAR-to-LPAR Virtual LAN within a POWER5 Server
– Does not require the APV feature code

Why Virtual I/O Server?
POWER5 systems will support more partitions than physical I/O slots
available
– Each partition still requires a boot disk and network connection, but
now they can be virtual instead of physical
VIO Server allows partitions to share disk and network adapter resources
– The Fibre Channel or SCSI controllers in the VIO Server can be
accessed using Virtual SCSI controllers in the clients
– A Shared Ethernet Adapter in the VIO Server can be a layer 2 bridge
for virtual Ethernet adapters in the clients
The VIO Server further enables on demand computing and server
consolidation

Virtualizing I/O saves:
– Gbit Ethernet Adapters
– 2 Gbit Fibre Channel Adapters
– PCI slots
– Eventually, IO drawers
– Server frames?
– Floor space?
– Electric, HVAC?
– Ethernet switch ports
– Fibre channel switch ports
– Logistics, scheduling, delays of physical Ethernet, SAN attach
Some servers run 90% utilization all the time – everyone knows which
ones.
Average utilization in the UNIX server farm is closer to 25%. They don’t
all maximize their use of dedicated I/O devices
VIO is departure from “new project, new chassis” mindset

Virtual I/O Server Characteristics

Requires AIX 5.3 and POWER5 hardware with APV feature
Installed as a special purpose, AIX-based logical partition
Uses a subset of the AIX Logical Volume Manager and attaches
to traditional storage subsystems
Inter-partition communication (client-server model) provided via
the POWER Hypervisor
Clients “see” virtual disks as traditional AIX SCSI hdisks, although
they may be a physical disk or logical volume on the VIO Server
One physical disk on a VIO server can provide logical volumes for
several client partitions

Virtual Ethernet
Virtual Ethernet
– Enable inter-lpar communications without a physical adapter
– IEEE-compliant Ethernet programming model
– Implemented through inter-partition, in-memory communication
VLAN splits up groups of network users on a physical network onto
segments of logical networks
Virtual switch provides support for multiple (up to 4K) VLANs
– Each partition can connect to multiple networks, through one or more adapters
– VIO server can add VLAN ID tag to the Ethernet frame as appropriate.
Ethernet switch restricts frames to ports that are authorized to receive frames
with specific VLAN ID
Virtual network can connect to physical network through “routing"
partitions – generally not recommended

Why Multiple VIO Servers?
Second VIO Server adds extra protection to client LPARS
Allows two teams to learn VIO setup on single system
Having Multiple VIO Servers will:
– Provide you Multiple paths to your OS/Data Virtual disks
– Provide you Multiple paths to your network
Advantages:
– Highest superior availability to other virtual I/O solutions
– Allows VIO Server updates without shutting down client LPAR’s

Virtualization VIO basics

2008-03-01T08:34:00.005+05:30

The Virtual I/O Server is part of the IBM System p Advanced Power Virtualization hardware feature. Virtual I/O Server allows sharing of physical resources between LPARs including virtual SCSI and virtual networking. This allows more efficient utilization of physical resources through sharing between LPARs and facilitates server consolidation.

The Virtual I/O Server is software that is located in a logical partition. This software facilitates the sharing of physical I/O resources between AIX® and Linux® client logical partitions within the server. The Virtual I/O Server provides virtual SCSI target and Shared Ethernet Adapter capability to client logical partitions within the system, allowing the client logical partitions to share SCSI devices and Ethernet adapters. The Virtual I/O Server software requires that the logical partition be dedicated solely for its use.
The Virtual I/O Server is available as part of the Advanced POWER™ Virtualization hardware feature.
Using the Virtual I/O Server facilitates the following functions:
-->Sharing of physical resources between logical partitions on the system
-->Creating logical partitions without requiring additional physical I/O resources
-->Creating more logical partitions than there are I/O slots or physical devices available with the ability for partitions to have dedicated I/O, virtual I/O, or both
-->Maximizing use of physical resources on the system
-->Helping to reduce the Storage Area Network (SAN) infrastructure
The Virtual I/O Server supports client logical partitions running the following operating systems:
-->AIX 5.3 or later
-->SUSE Linux Enterprise Server 9 for POWER (or later)
-->Red Hat® Enterprise Linux AS for POWER Version 3 (update 2 or later)
-->Red Hat Enterprise Linux AS for POWER Version 4 (or later)
For the most recent information about devices that are supported on the Virtual I/O Server, to download Virtual I/O Server fixes and updates, and to find additional information about the Virtual I/O Server, see the Virtual I/O Server Web site.
The Virtual I/O Server comprises the following primary components:
-->Virtual SCSI
-->Virtual Networking
-->Integrated Virtualization Manager
The following sections provide a brief overview of each of these components.

Virtual SCSI
Physical adapters with attached disks or optical devices on the Virtual I/O Server logical partition can be shared by one or more client logical partitions. The Virtual I/O Server offers a local storage subsystem that provides standard SCSI-compliant logical unit numbers (LUNs). The Virtual I/O Server can export a pool of heterogeneous physical storage as an homogeneous pool of block storage in the form of SCSI disks.
Unlike typical storage subsystems that are physically located in the SAN, the SCSI devices exported by the Virtual I/O Server are limited to the domain within the server. Although the SCSI LUNs are SCSI compliant, they might not meet the needs of all applications, particularly those that exist in a distributed environment.
The following SCSI peripheral-device types are supported:
-->Disks backed by a logical volume
-->Disks backed by a physical volume
-->Optical devices (DVD-RAM and DVD-ROM)

Virtual networking
Shared Ethernet Adapter allows logical partitions on the virtual local area network (VLAN) to share access to a physical Ethernet adapter and to communicate with systems and partitions outside the server. This function enables logical partitions on the internal VLAN to share the VLAN with stand-alone servers.

Integrated Virtualization Manager
The Integrated Virtualization Manager provides a browser-based interface and a command-line interface that you can use to manage IBM® System p5™ and IBM eServer™ pSeries® servers that use the IBM Virtual I/O Server. On the managed system, you can create logical partitions, manage the virtual storage and virtual Ethernet, and view service information related to the server. The Integrated Virtualization Manager is packaged with the Virtual I/O Server, but it is activated and usable only on certain platforms and where no Hardware Management Console (HMC) is present.

Introduction to VIO

2008-03-01T02:50:00.006+05:30

Prior to the introduction of POWER5 systems, it was only possible to create as many separate logical partitions (LPARs) on an IBM system as there were physical processors. Given that the largest IBM eServer pSeries POWER4 server, the p690, had 32 processors, 32 partitions were the most anyone could create. A customer could order a system with enough physical disks and network adapter cards to so that each LPAR would have enough disks to contain operating systems and enough network cards to allow users to communicate with each partition.
The Advanced POWER Virtualization™ feature of POWER5 platforms1 makes it possible to allocate fractions of a physical CPU to a POWER5 LPAR. Using virtual CPU's and virtual I/O a user can create many more LPARs on a p5 system than there are CPU's or I/O slots. The Advanced POWER Virtualization feature accounts for this by allowing users to create shared network adapters and virtual SCSI disks. Customers can use these virtual resources to provide disk space and network adapters for each LPAR they create on their POWER5 system
(see Figure ).

There are three components of the Advanced POWER Virtualization feature: Micro-Partitioning™, shared Ethernet adapters, and virtual SCSI. In addition, AIX 5L Version
5.3 allows users to define virtual Ethernet adapters permitting inter-LPAR communication. This paper provides an overview of how each of these components works and then shows the details of how to set up a simple three-partition system where one partition is a Virtual I/O Server and the other two partitions use virtual Ethernet and virtual SCSI to differing degrees. What follows is a practical guide to help a new POWER5 customer set up simple systems where high availability is not a concern, but becoming familiar with this new technology in a development environment is the primary goal.

Micro-Partitioning
An element of the IBM POWER Virtualization feature called Micro-Partitioning can divide a single processor into many different processors. In POWER4 systems, each physical processor is dedicated to an LPAR. This concept of dedicated processors is still present in POWER5 systems, but so is the concept of shared processors. A POWER5 system administrator can use the Hardware Management Console (HMC) to place processors in
a shared processor pool. Using the HMC, the administrator can assign fractions of a CPU to individual partitions. If one LPAR is defined to use processors in the shared processor pool, when those CPUs are idle, the POWER Hypervisor™ makes them available to other partitions. This ensures that these processing resources are not wasted. Also, the ability to assign fractions of a CPU to a partition means it is possible to partition POWER5 servers into many different partitions. Allocation of physical processor and memory resources on POWER5 systems is managed by a system firmware component called the POWER Hypervisor.

Virtual Networking
Virtual networking on POWER5 hardware consists of two main capabilities. One capability is provided by a software IEEE 802.1q (VLAN) switch that is implemented in the Hypervisor on POWER5 hardware. Users can use the HMC to add Virtual Ethernet adapters to their partition definitions. Once these are added and the partitions booted, the new adapters can be configured just like real physical adapters, and the partitions can communicate with each other without having to connect cables between the LPARs. Users can separate traffic from different VLANs by assigning different VLAN IDs to each virtual Ethernet adapter. Each AIX 5.3 partition can support up to 256 Virtual Ethernet adapters

In addition, a part of the Advanced POWER virtualization virtual networking feature allows users to share physical adapters between logical partitions. These shared adapters, called Shared Ethernet Adapters (SEAs), are managed by a Virtual I/O Server partition which maps physical adapters under its control to virtual adapters. It is possible to map many physical Ethernet adapters to a single virtual Ethernet adapter thereby eliminating a single physical adapter as a point of failure in the architecture.
There are a few things users of virtual networking need to consider before implementing it. First, virtual networking ultimately uses more CPU cycles on the POWER5 machine than when physical adapters are assigned to a partition. Users should consider assigning a physical adapter directly to a partition when heavy network traffic is predicted over a certain adapter. Secondly, users may want to take advantage of larger MTU sizes that virtual Ethernet allows if they know that their applications will benefit from the reduced fragmentation and better performance that larger MTU sizes offer. The MTU size limit for SEA is smaller than Virtual Ethernet adapters, so users will have to carefully choose an MTU size so that packets are sent to external networks with minimum fragmentation.

Virtual SCSI
The Advanced POWER Virtualization feature called virtual SCSI allows access to physical disk devices which are assigned to the Virtual I/O Server (VIOS). The system administrator uses VIOS logical volume manager commands to assign disks to volume groups. The administrator creates logical volumes in the Virtual I/O Server volume groups. Either these logical volumes or the physical disks themselves may ultimately appear as physical disks (hdisks) to the Virtual I/O Server’s client partitions once they are associated with virtual SCSI host adapters. While the Virtual I/O Server software is
packaged as an additional software bundle that a user purchases separately from the AIX 53 distribution, the virtual I/O client software is a part of the AIX 5.3 base installation media so an administrator does not need to install any additional filesets on a Virtual SCSI client partition. Srikrishnan provides more details on how the Virtual SCSI feature works

Step 32 & 33 Check for cluster Stabilize & VG varied on

2008-02-29T08:18:00.001+05:30

Wait for the cluster to stabilize. You can check when the cluster is up by following
commands
a. netstat –i
b. ifconfig –a : look-out for service ip. It will show on each node if the cluster is up.

Check whether the VGs under cluster’s RGs are varied-ON and the filesystems in the
VGs are mounted after the cluster start.

Here test1vg and test2vg are VGs which are varied-ON when the cluster is started and
Filesystems /test2 and /test3 are mounted when the cluster starts.
/test2 and /test3 are in test2vg which is part of the RG which is owned by this node.
32. Perform all the tests such as resource take-over, node failure, n/w failure and verify
the cluster before releasing the system to the customer.

step 30 & 31 Synchronize & start Cluster

2008-02-29T08:15:00.002+05:30

Synchronize the cluster:
This will sync the info from one node to second node.
Smitty cl_sync

That’s it. Now you are ready to start the cluster.
Smitty clstart

You can start the cluster together on both nodes or start individually on each node.

You can start the cluster together on both nodes or start individually on each node.

step 29 Adding IP label & RG owned by Node

2008-02-29T08:12:00.001+05:30

Add the service IP label for the owner node and also the VGs owned by the owner node
Of this resource group.

Continue similarly for all the resource groups.

step 28 Setting attributes of Resource group

2008-02-29T08:11:00.002+05:30

Set attributes of the resource groups already defined:
Here you have to actually assign the resources to the resource groups.
smitty hacmp -> Extended Configuration -> Extended Resource Configuration ->
HACMP extended resource group configuration

step 27 Adding Resource Group

2008-02-29T08:06:00.002+05:30

Add Resource Groups:
smitty hacmp -> Extended Configuration -> Extended Resource Configuration ->
HACMP extended resource group configuration

Continue similarly for all the resource groups.
The node selected first while defining the resource group will be the primary owner of
that resource group. The node after that is secondary node.
Make sure you set primary node correctly for each resource group. Also set the failover/fallback policies as per the requirement of the setup

step 26 Defining IP labels

2008-02-29T08:05:00.001+05:30

Define the service IP labels for both nodes.
smitty hacmp -> Extended Configuration -> Extended Resource Configuration ->
HACMP extended resource configuration -> Configure HACMP service IP label

step 25 Adding Persistent IP labels

2008-02-29T08:01:00.001+05:30

Add a persistent ip label for both nodes.

step 24 Adding persistent IP

2008-02-29T07:57:00.001+05:30

Add the persistent IPs:

smitty hacmp -> Extended Configuration -> Extended Topology Configuration ->
Configure HACMP persistent nodes IP label/Addresses

step23 Adding boot IP & Disk heart beat information

2008-02-29T07:51:00.001+05:30

Include all the four boot ips (2 for each nodes) in this ether interface already defined.Then include the disk for heartbeat on both the nodes in the diskhb already defined

step 22 Adding device for Disk Heart Beat

2008-02-29T07:47:00.001+05:30

Include the interfaces/devices in the ether n/w and diskhb already defined.
smitty hacmp -> Extended Configuration -> Extended Topology Configuration ->
Configure HACMP communication interfaces/devices -> Add communication
Interfaces/devices.

Step21 Adding Communication interface

2008-02-29T07:45:00.001+05:30

Add HACMP communication interfaces. (Ether interfaces.)
smitty hacmp -> Extended Configuration -> Extended Topology Configuration ->
Configure HACMP networks -> Add a network to the HACMP cluster.
Select ether and Press enter.
Then select diskhb and Press enter. Diskhb is your non-tcpip heartbeat.

Step20 Discover HACMP config for Network settings

2008-02-29T07:41:00.001+05:30

22. Discover HACMP config: This will import for both nodes all the node info, boot ips,
service ips from the /etc/hosts
smitty hacmp -> Extended configurations -> Discover hacmp related information

Step 19 Define Cluster Nodes

2008-02-29T07:39:00.001+05:30

19. Define the cluster nodes. #smitty hacmp -> Extended Configuration -> Extended topology configuration -> Configure an HACMP node - > Add a node to an HACMP cluster Define both the nodes on after the other.

Step 18 to configure HACMP

2008-02-28T09:48:00.001+05:30

18. Define cluster name.

Steps 1 to 17 to configure HACMP

2008-02-28T01:59:00.000+05:30

Steps to configure HACMP:

1. Install the nodes, make sure the redundancy is maintained for power supplies, n/w and
fiber n/ws. Then Install AIX on the nodes.
2. Install all the HACMP filesets except HAview and HATivoli.
Install all the RSCT filesets from the AIX base CD.
Make sure that the AIX, HACMP patches and server code are at the latest level (ideally
recommended).
4. Check for fileset bos.clvm to be present on both the nodes. This is required to make the
VGs enhanced concurrent capable.
5. V.IMP: Reboot both the nodes after installing the HACMP filesets.
6. Configure shared storage on both the nodes. Also in case of a disk heartbeat, assign a
1GB shared storage LUN on both nodes.
7. Create the required VGs only on the first node. The VGs can be either normal VGs or
Enhanced concurrent VGs. Assign particular major number to each VGs while creating
the VGs. Record the major no. information.
To check the Majar no. use the command:
ls –lrt /dev grep
Mount automatically at system restart should be set to NO.
8. Varyon the VGs that was just created.
9. V.IMP: Create log LV on each VG first before creating any new LV. Give a unique
name to logLV.
Destroy the content of logLV by: logform /dev/loglvname
Repeat this step for all VGs that were created.
10. Create all the necessary LVs on each VG.
11. Create all the necessary file systems on each LV created…..you can create mount pts
as per the requirement of the customer,
Mount automatically at system restart should be set to NO.
12. umount all the filesystems and varyoff all the VGs.

13. chvg –an ---All VGs will be set to do not mount automatically at
System restart.
14. Go to node 2 and run cfgmgr –v to import the shared volumes.
15. Import all the VGs on node 2
use smitty importvg -----import with the same major number as assigned on node
16. Run chvg –an for all VGs on node 2.
17. V.IMP: Identify the boot1, boot2, service ip and persistent ip for both the nodes
and make the entry in the /etc/hosts.

HACMP v5.x Disk Heartbeat device configuration

2008-02-20T03:35:00.002+05:30

Creating a Disk Heartbeat device in HACMP v5.x

Introduction
This document is intended to supplement existing documentation on how to configure, test, and monitor a disk heartbeat device and network in HACMP/ES V 5.x. This feature is new in V5.1, and it provides another alternative for non-ip based heartbeats. The intent of this document is to provide step-by-step directions as they are currently sketchy in the HACMP v5.1 pubs. This will hopefully clarify several misconceptions that have been brought to my attention.
This example consists of a two-node cluster (nodes GT40 & SL55) with shared ESS vpath devices. If more than two nodes exist in your cluster, you will need N number or non-ip heartbeat networks. Where N represents the number of nodes in the cluster. (i.e. three node cluster requires 3 non-ip heartbeat networks). This creates a heartbeat ring.
It’s worth noting that one should not confuse concurrent volume groups with concurrent resource groups. And note, there is a difference between concurrent volume groups and enhanced concurrent volume groups. A concurrent resource group is one which may be active on more than one node at a type. A concurrent volume group also shares the characteristic that it may be active on more than one node at a time. This is also true for an enhanced concurrent VG; however, in a non-concurrent resource group, the enhanced concurrent VG, while it may be active and not have a SCSI reserve residing on the disk, it’s data is only normally accessed by one system at a time.

Pre-Reqs

In this document, it is assumed that the shared storage devices are already made available and configured to AIX, and that the proper levels of RSCT and HACMP are already installed. Since utilizing enhanced-concurrent volume groups, it is also necessary to make sure that bos.clvm.enh is installed. This is not normally installed as part of a HACMP installation via the installp command.
Disk Heartbeat Details
This provides the ability to use existing shared disks, regardless of disk type, to provide a serial network like heartbeat path. A benefit of this is that one need not dedicate the integrated serial ports for HACMP heartbeats (if supported on the subject systems) or purchase an 8-port asynchronous adapter.
This feature utilizes a special area on the disk previously reserved for “Concurrent Capable” volume groups (traditionally only for SSA disks). Since AIX 5.2 dropped support for the SSA concurrent volume groups, this fit makes it available for use. This also means that the disk chosen for serial heartbeat can be part of a data volume group. (Note Performance Concerns below)

The disk heart beating code went into the 2.2.1.30 version of RSCT. Some recommended APARs bring that to 2.2.1.31. If you've got that level installed, and HACMP 5.1, you can use disk heart beating. The relevant file to look for is /usr/sbin/rsct/bin/hats_diskhb_nim. Though it is supported mainly through RSCT, we recommend AIX 5.2 when utilizing disk heartbeat.

To use disk heartbeats, no node can issue a SCSI reserve for the disk. This is because both nodes using it for heart beating must be able to read and write to that disk. It is sufficient that the disk be in an enhanced concurrent volume group to meet this requirement. (It should also be possible to use a disk that is in no volume group for disk heart beating. RSCT certainly won't care; but HACMP SMIT panels may not be particularly helpful in setting this up.)

Now, in HACMP 5.1 with AIX 5.1, enhanced concurrent mode volume groups can be used only in concurrent (or "online on all available nodes") resource groups. This means that disk heart beating is useful only to people running concurrent configurations, or who can allocate such a volume group/disk (which is certainly possible, though perhaps an expensive approach). In other words, at HACMP 5.1 and AIX 5.1, typical HACMP clusters (with a server and idle standby) will require an additional concurrent resource group with a disk in an enhanced concurrent VG dedicated for heartbeat use. At AIX 5.2, disk heartbeats can exist on an enhanced concurrent VG that resides in a non-concurrent resource group. At AIX 5.2, one may also use the fast disk takeover feature in non-concurrent resource groups with enhanced concurrent volume groups. With HACMP 5.1 and AIX 5.2, enhanced concurrent mode volume groups can be used in serial access configurations for fast disk takeover, along with disk heart beating. (AIX 5.2 requires RSCT 2.3.1.0 or later) That is, the facility becomes usable to the average customer, without committment of additional resource, since disk heart beating can occur on a volume group used for ordinary filesystem and logical volume activity.

Performance Concerns with Disk Heart Beating

Most modern disks take somewhere around 15 milliseconds to service an IO request, which means that they can't do much more than 60 seeks per second. The sectors used for disk heart beating are part of the VGDA, which is at the outer edge of the disk, and may not be near the application data. This means that every time a disk heart beat is done, a seek will have to be done. Disk heart beating will typically (with the default parameters) require four (4) seeks per second. That is each of two nodes will write to the disk and read from the disk once/second, for a total of 4 IOPS. So, if possible, a disk should be selected as a heart beat path that does not normally do more than about 50 seeks per second. The filemon tool can be used to monitor the seek activity on a disk.

In cases where a disk must be used for heart beating that already has a high seek rate, it may be necessary to change the heart beat timing parameters to prevent long write delays from being seen as a failure.

The above cautions as stated apply to JBOD configurations, and should be modified based on the technology of the disk subsystem:
• If the disk used for heart beating is in a controller that provides large amounts of cache - such as the ESS - the number of seeks per second can be much larger
• If the disk used for heart beating is part of a RAID set without a caching front end controller, the disk may be able to support fewer seeks, due to the extra activity required by RAID operations
Pros & Cons of using Disk Heart Beating
Pros:
1. No additional hardware needed.
2. Easier to span greater distances.
3. No loss in usable storage space and can use existing data volume groups.
4. Uses enhanced concurrent vgs which also allows for fast-disk takeover.

Cons:
1. Must be aware of the devices diskhb uses and administer devices properly*
2. Lose the forced down option of stopping cluster services because of enhanced concurrent vg usage.

*I have had a customer delete all their disk definitions and run cfgmgr again to clean up number holes in their device definition list. When they did, obviously , the device names did not come back in the same order as they were before. So the diskhb device assigned to HACMP, was no longer valid as a different device was configured using the old device name and it was not part of an enhanced concurrent vg. Hence diskhb no longer worked, and since the customer did not monitor their cluster either, they were unaware that the diskhb no longer worked.

Configuring Disk Heartbeat

As mentioned previously, disk heartbeat utilizes enhanced-concurrent volume groups. If starting with a new configuration of disks, you will want to create enhanced-concurrent volume groups, either manually, or by utilizing C-SPOC. My example shows using C-SPOC which is the best practice to use here.

If you plan to use an existing volume group for disk heartbeats that is not enhanced concurrent, then you will have to convert them to such using the chvg command. We recommend that the VG be active on only one node, and that the application not be running when making this change run chvg –C vgname to change the VG to enhanced concurrent mode. Vary it off, then run importvg –L vgname on the other node to make it aware that the vg is now enhanced concurrent capable. If using this method, you can skip to the “Creating Disk Heartbeat Devices and Network” section of this document.

Disk and VG Preparation

To be able to use C-SPOC successfully, it is required that some basic IP based topology already exists, and that the storage devices have their PVIDs in both system’s ODMs. This can be verified by running lspv on each system. If a PVID does not exist on each system, it is necessary to run chdev -l -a pv=yes on each system. This will allow C-SPOC to match up the device(s) as known shared storage devices.
In this example, vpath0 on GT40 is the same virtual disk as vpath3 on SL55.
Use C-SPOC to create an Enhanced Concurrent volume group. In the following example, since vpath devices are being used, the following smit screen paths were used.
smitty cl_adminGo to HACMP Concurrent Logical Volume Management Concurrent Volume Groups Create a Concurrent Volume Group with Data Path Devices and press Enter

Choose the appropriate nodes, and then choose the appropriate shared storage devices based on pvids (vpath0 and vpath3 in this example). Choose a name for the VG , desired PP size, make sure that Enhanced Concurrent Mode is set to true and press Enter. (enhconcvg in this example). This will create the shared enhanced-concurrent vg needed for our disk heartbeat. .

It’s a good idea to verify via lspv once this has completed to make sure the device and vg is show appropriately as follows:

GT40#/ lspv
vpath0 000a7f5af78e0cf4 enhconcvg

SL55#/lspv
vpath3 000a7f5af78e0cf4 enhconcvg

Creating Disk Heartbeat Devices and Network

There are two different ways to do this. Since we have already created the enhanced concurrent vg, we can use the discovery method (1) and let HA find it for us. Or we can do this manually via the Pre-defined devices method (2). Following is an example of each.

1) Creating via Discover Method: (See Note)
Enter smitty hacmpExtended ConfigurationDiscover HACMP-related Information from Configured NodesPress Enter

This will run automatically and create a clip_config file that contains the information it has discovered. Once completed, go back to the Extended Configuration menu and chose:

Extended Topology ConfigurationConfigure HACMP Communication Interfaces/DevicesAdd Communication Interfaces/DevicesAdd Discovered Communication Interface and DevicesCommunication Devices Choose appropriate devices (ex. vpath0 and vpath3)

Select Point-to-Point Pair of Discovered Communication Devices to Add
Move cursor to desired item and press F7. Use arrow keys to scroll.
ONE OR MORE items can be selected.
Press Enter AFTER making all selections.
# Node Device Device Path Pvid
> nodeGT40 vpath0 /dev/vpath0 000a7f5af78
> nodeSL55 vpath3 /dev/vpath3 000a7f5af78

Note: Base HA 5.1 appears to have a problem when using the Discovered Devices this method. If you get this error: "ERROR: Invalid node name 000a7f5af78e0cf4".
Then you will need apar IY51594. Otherwise you will have to create via the Pre-Defined Devices method. Once corrected, this section will be completed
2) Creating via Pre-Defined Devices Method

When using this method, it is necessary to create a diskhb network first, then assign the disk-node pair devices to the network. Create the diskhb network as follows:

smitty hacmp  Extended Configuration  Extended Topology Configuration Configure HACMP Networks Add a Network to the HACMP cluster  choose diskhb  Enter desired network name (ex. disknet1)--press Enter

smitty hacmp  Extended Configuration  Extended Topology Configuration  Configure HACMP Communication Interfaces/Devices  Add Communication Interfaces/Devices Add Pre-Defined Communication Interfaces and Devices 
Communication Devices  Choose your diskhb Network Name 

Add a Communication Device

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
* Device Name [GT40_hboverdisk]
* Network Type diskhb
* Network Name disknet1
* Device Path [/dev/vpath0]
* Node Name [GT40]

For Device Name, that is a unique name you can chose. It will show up in your topology under this name, much like serial heartbeat and ttys have in the past.

For the Device Path, you want to put in /dev/. Then choose the corresponding node for this device and device name (ex. GT40). Then press Enter.

You will repeat this process for the other node (ex. SL55) and the other device (vpath3). This will complete both devices for the diskhb network.

Testing Disk Heartbeat Connectivity

Once the device and network definitions have been created, it is a good idea to test it and make sure communications is working properly. If the volume group is varied on in normal mode on one of the nodes, the test will probably not work.

/usr/sbin/rsct/bin/dhb_read is used to test the validity of a diskhb connection. The usage of dhb_read is as follows:

dhb_read -p devicename //dump diskhb sector contents
dhb_read -p devicename -r //receive data over diskhb network
dhb_read -p devicename -t //transmit data over diskhb network

To test that disknet1, in the example configuration, can communicate from nodeB(ex. SL55) to nodeA (ex. GT40), you would run the following commands:

On nodeA, enter:

dhb_read -p rvpath0 -r

On nodeB, enter:

dhb_read -p rvpath3 -t

Note: That the device name is raw device as designated with the “r” proceeding the device name.

If the link from nodeB to nodeA is operational, both nodes will display:

Link operating normally.

You can run this again and swap which node transmits and which one receives. To make the network active, it is necessary to sync up the cluster. Since the volume group has not been added to the resource group, we will sync up once instead of twice.

Add Shared Disk as a Shared Resource

In most cases you would have your diskhb device on a shared data vg. It is necessary to add that vg into your resource group and synchronize the cluster.

smitty hacmp Extended Configuration Extended Resource Configuration > Extended Resource Group Configuration Change/Show Resources and Attributes for a Resource Group and press Enter.

Choose the appropriate resource group, enter the new vg (enhconcvg) into the volume group list and press Enter.

Return to the top of the Extended Configuration menu and synchronize the cluster.

Monitor Disk Heartbeat

Once the cluster is up and running, you can monitor the activity of the disk (actually all) heartbeats via lssrc -ls topsvcs. An example of the output follows:

Subsystem Group PID Status
topsvcs topsvcs 32108 active

Network Name Indx Defd Mbrs St Adapter ID Group ID
disknet1 [ 3] 2 2 S 255.255.10.0 255.255.10.1
disknet1 [ 3] rvpath3 0x86cd1b02 0x86cd1b4f
HB Interval = 2 secs. Sensitivity = 4 missed beats
Missed HBs: Total: 0 Current group: 0
Packets sent : 229 ICMP 0 Errors: 0 No mbuf: 0
Packets received: 217 ICMP 0 Dropped: 0
NIM's PID: 28724

Be aware that there is a grace period for heartbeats to start processing. This is normally around 60 seconds. So if you run this command quickly after starting the cluster, you may not see anything at all until heartbeat processing is started after the grace period time has elapsed.

HACMP failover scenario

2008-02-16T08:31:00.000+05:30

HA failover scenarios

1. Graceful
For graceful failover, you can run “smitty clstop” then select graceful option. This will not change anything except stopping the cluster on that node.
Note: If you stop the cluster, check the status using lssrc –g cluster, sometimes clstrmgrES daemon will take long time to stop, DO NOT KILL THIS DAEMON.It will stop automatically after a while.
You can do this on both the nodes

2. Takeover
For takeover, run “smitty clstop” with takeover option, this will stop the cluster on that node and the standby node will take over the pakage
You can do this on both the nodes

3. Soft Pakckage Failover
Run smitty cm_hacmp_resource_group_and_application_management_menu >>>Move a Resource Group to Another Node >>>>select the package name and node name >>>enter
This will move the package from that node to the node that you have selected in the above menu. This method will give lot of troubles in HA 4.5 whereas it runs good on HA 5.2 unless we have any apps startup issues.
You can do this on both the nodes
│
4. Failover Network Adapter(s):
For this type of testing , run “ifconfig enx down” , then package IP will failover to primary adapter. You can not even see any outage or anything.

We can manually (ifconfig enx up) bring it back to original adapter , but better to reboot the server to bring the package back to the original node

5. Hardware Failure (crash):
This is a standard type of testing; run the command “reboot –q” then the node will godown without stopping any apps and come up immediately. The package will failover to the standby node with in 2 min os downtime (Even tough HA failover is fast, some apps will take long time to start

Specifying the default gateway on a specific interface in HACMP

2008-02-15T05:59:00.001+05:30

Specifying the default gateway on a specific interface

When you're using HACMP, you usually have multiple network adapters installed and thus multiple network interface to handle with. If AIX configured the default gateway on a wrong interface (like on your management interface instead of the boot interface), you might want to change this, so network traffic isn't sent over the management interface. Here's how you can do this:

First, stop HACMP or do a take-over of the resource groups to another node; this will avoid any problems with applications when you start fiddling with the network configuration.

Then open up a virtual terminal window to the host on your HMC. Otherwise you would loose the connection, as soon as you drop the current default gateway.

Now you need to determine where your current default gateway is configured. You can do this by typing: lsattr -El inet0 and netstat -nr. The lsattr command will show you the current default gateway route and the netstat command will show you the interface it is configured on. You can also check the ODM: odmget -q"attribute=route" CuAt.

Now, delete the default gateway like this:
lsattr -El inet0 | awk '$2 ~ /hopcount/ { print $2 }' | read GW
chdev -l inet0 -a delroute=${GW}

If you would now use the route command to specifiy the default gateway on a specific interface, like this:
route add 0 [ip address of default gateway: xxx.xxx.xxx.254] -if enX
You will have a working entry for the default gateway. But... the route command does not change anything in the ODM. As soon as your system reboots; the default gateway is gone again. Not a good idea.

A better solution is to use the chdev command:
chdev -l inet0 -a addroute=net,-hopcount,0,,0,[ip address of default gateway]
This will set the default gateway to the first interface available.

To specify the interface use:
chdev -l inet0 -a addroute=net,-hopcount,0,if,enX,,0,[ip address of default gateway]
Substitute the correct interface for enX in the command above.

If you previously used the route add command, and after that you use chdev to enter the default gateway, then this will fail. You have to delete it first by using route delete 0, and then give the chdev command.

Afterwards, check with lsattr -El inet0 and odmget -q"attribute=route" CuAt if the new default gateway is properly configured. And ofcourse, try to ping the IP address of the default gateway and some outside address. Now reboot your system and check if the default gateway remains configured on the correct interface. And startup HACMP again!

HACMP topology & usefull commands

2008-02-15T05:55:00.002+05:30

Hacmp can be configured in 3 ways.

1. Rotating
2. Cascading
3. Mutual Failover

The cascading and rotating resource groups are the “classic”, pre-HA 5.1 types. The new “custom” type of resource group has been introduced in HA 5.1 onwards.

Cascading resource group:
Upon node failure, a cascading resource group falls over to the available node with the next priority in the node priority list.
Upon node reintegration into the cluster, a cascading resource group falls back to its home node by default.

Cascading without fallback
Thisoption, this means whenever a primary node fails, the package will failover to the next available node in the list and when the primary node comes online then the package will not fallback automatically. We need to move package to its home node at a convenient time.

Rotating resource group:
This is almost similar to Cascading without fallback, whenever package failover to the standby nodes it will never fallback to the primary node automatically, we need to move it manually at our convenience.

Mutual takeover:
Mutual takeover option, which means both the nodes in this type are active-active mode. Whenever fail over happens the package on the failed node will move to the other active node and will run with already existing package. Once the failed node comes online we can move the package manually to that node.

Useful HACMP commands

clstat - show cluster state and substate; needs clinfo.
cldump - SNMP-based tool to show cluster state
cldisp - similar to cldump, perl script to show cluster state.
cltopinfo - list the local view of the cluster topology.
clshowsrv -a - list the local view of the cluster subsystems.
clfindres (-s) - locate the resource groups and display status.
clRGinfo -v - locate the resource groups and display status.
clcycle - rotate some of the log files.
cl_ping - a cluster ping program with more arguments.
clrsh - cluster rsh program that take cluster node names as argument.
clgetactivenodes - which nodes are active?
get_local_nodename - what is the name of the local node?
clconfig - check the HACMP ODM.
clRGmove - online/offline or move resource groups.
cldare - sync/fix the cluster.
cllsgrp - list the resource groups.
clsnapshotinfo - create a large snapshot of the hacmp configuration.
cllscf - list the network configuration of an hacmp cluster.
clshowres - show the resource group configuration.
cllsif - show network interface information.
cllsres - show short resource group information.
lssrc -ls clstrmgrES - list the cluster manager state.
lssrc -ls topsvcs - show heartbeat information.
cllsnode - list a node centric overview of the hacmp configuration.

HACMP Basics

2008-02-10T10:13:00.002+05:30

HACMP Basics

History
IBM's HACMP exists for almost 15 years. It's not actually an IBM product, they bought it from CLAM, which was later renamed to Availant and is now called LakeViewTech. Until august 2006, all development of HACMP was done by CLAM. Nowadays IBM does it's own development of HACMP in Austin, Poughkeepsie and Bangalore

IBM's high availability solution for AIX, High Availability Cluster Multi Processing (HACMP), consists of two components:

•High Availability: The process of ensuring an application is available for use through the use of duplicated and/or shared resources (eliminating Single Points Of Failure – SPOF's)

.Cluster Multi-Processing: Multiple applications running on the same nodes with shared or concurrent access to the data.

A high availability solution based on HACMP provides automated failure detection, diagnosis, application recovery and node reintegration. With an appropriate application, HACMP can also provide concurrent access to the data for parallel processing applications, thus offering excellent horizontal scalability.

What needs to be protected? Ultimately, the goal of any IT solution in a critical environment is to provide continuous service and data protection.

The High Availability is just one building block in achieving the continuous operation goal. The High Availability is based on the availability hardware, software (OS and its components), application and network components.

The main objective of the HACMP is to eliminate Single Points of Failure (SPOF's)

“…A fundamental design goal of (successful) cluster design is the elimination of single points of failure (SPOFs)…”

Eliminate Single Point of Failure (SPOF)
Cluster Eliminated as a single point of failure

Node Using multiple nodes
Power Source Using Multiple circuits or uninterruptible
Network/adapter Using redundant network adapters
Network Using multiple networks to connect nodes.
TCP/IP Subsystem Using non-IP networks to connect adjoining nodes & clients
Disk adapter Using redundant disk adapter or multiple adapters
Disk Using multiple disks with mirroring or RAID
Application Add node for takeover; configure application monitor
Administrator Add backup or every very detailed operations guide
Site Add additional site.

Cluster Components

Here are the recommended practices for important cluster components.

Nodes

HACMP supports clusters of up to 32 nodes, with any combination of active and standby nodes. While it
is possible to have all nodes in the cluster running applications (a configuration referred to as "mutual
takeover"), the most reliable and available clusters have at least one standby node - one node that is normally
not running any applications, but is available to take them over in the event of a failure on an active
node.

Additionally, it is important to pay attention to environmental considerations. Nodes should not have a
common power supply - which may happen if they are placed in a single rack. Similarly, building a cluster
of nodes that are actually logical partitions (LPARs) with a single footprint is useful as a test cluster, but
should not be considered for availability of production applications.
Nodes should be chosen that have sufficient I/O slots to install redundant network and disk adapters.
That is, twice as many slots as would be required for single node operation. This naturally suggests that
processors with small numbers of slots should be avoided. Use of nodes without redundant adapters
should not be considered best practice. Blades are an outstanding example of this. And, just as every cluster
resource should have a backup, the root volume group in each node should be mirrored, or be on a

RAID device.
Nodes should also be chosen so that when the production applications are run at peak load, there are still
sufficient CPU cycles and I/O bandwidth to allow HACMP to operate. The production application
should be carefully benchmarked (preferable) or modeled (if benchmarking is not feasible) and nodes chosen
so that they will not exceed 85% busy, even under the heaviest expected load.
Note that the takeover node should be sized to accommodate all possible workloads: if there is a single
standby backing up multiple primaries, it must be capable of servicing multiple workloads. On hardware
that supports dynamic LPAR operations, HACMP can be configured to allocate processors and memory to
a takeover node before applications are started. However, these resources must actually be available, or
acquirable through Capacity Upgrade on Demand. The worst case situation – e.g., all the applications on
a single node – must be understood and planned for.

Networks

HACMP is a network centric application. HACMP networks not only provide client access to the applications
but are used to detect and diagnose node, network and adapter failures. To do this, HACMP uses
RSCT which sends heartbeats (UDP packets) over ALL defined networks. By gathering heartbeat information
on multiple nodes, HACMP can determine what type of failure has occurred and initiate the appropriate
recovery action. Being able to distinguish between certain failures, for example the failure of a network
and the failure of a node, requires a second network! Although this additional network can be “IP
based” it is possible that the entire IP subsystem could fail within a given node. Therefore, in addition
there should be at least one, ideally two, non-IP networks. Failure to implement a non-IP network can potentially
lead to a Partitioned cluster, sometimes referred to as 'Split Brain' Syndrome. This situation can
occur if the IP network(s) between nodes becomes severed or in some cases congested. Since each node is
in fact, still very alive, HACMP would conclude the other nodes are down and initiate a takeover. After
takeover has occurred the application(s) potentially could be running simultaneously on both nodes. If the
shared disks are also online to both nodes, then the result could lead to data divergence (massive data corruption).
This is a situation which must be avoided at all costs.

The most convenient way of configuring non-IP networks is to use Disk Heartbeating as it removes the
problems of distance with rs232 serial networks. Disk heartbeat networks only require a small disk or
LUN. Be careful not to put application data on these disks. Although, it is possible to do so, you don't want
any conflict with the disk heartbeat mechanism!

Adapters

As stated above, each network defined to HACMP should have at least two adapters per node. While it is
possible to build a cluster with fewer, the reaction to adapter failures is more severe: the resource group
must be moved to another node. AIX provides support for Etherchannel, a facility that can used to aggregate
adapters (increase bandwidth) and provide network resilience. Etherchannel is particularly useful for
fast responses to adapter / switch failures. This must be set up with some care in an HACMP cluster.
When done properly, this provides the highest level of availability against adapter failure. Refer to the IBM
techdocs website: http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD101785 for further
details.
Many System p TM servers contain built-in Ethernet adapters. If the nodes are physically close together, it
is possible to use the built-in Ethernet adapters on two nodes and a "cross-over" Ethernet cable (sometimes
referred to as a "data transfer" cable) to build an inexpensive Ethernet network between two nodes for
heart beating. Note that this is not a substitute for a non-IP network.
Some adapters provide multiple ports. One port on such an adapter should not be used to back up another
port on that adapter, since the adapter card itself is a common point of failure. The same thing is true
of the built-in Ethernet adapters in most System p servers and currently available blades: the ports have a
common adapter. When the built-in Ethernet adapter can be used, best practice is to provide an additional
adapter in the node, with the two backing up each other.
Be aware of network detection settings for the cluster and consider tuning these values. In HACMP terms,
these are referred to as NIM values. There are four settings per network type which can be used : slow,
normal, fast and custom. With the default setting of normal for a standard Ethernet network, the network
failure detection time would be approximately 20 seconds. With todays switched network technology this
is a large amount of time. By switching to a fast setting the detection time would be reduced by 50% (10
seconds) which in most cases would be more acceptable. Be careful however, when using custom settings,
as setting these values too low can cause false takeovers to occur. These settings can be viewed using a variety
of techniques including : lssrc –ls topsvcs command (from a node which is active) or odmget
HACMPnim |grep –p ether and smitty hacmp.

Applications
The most important part of making an application run well in an HACMP cluster is understanding the
application's requirements. This is particularly important when designing the Resource Group policy behavior
and dependencies. For high availability to be achieved, the application must have the ability to
stop and start cleanly and not explicitly prompt for interactive input. Some applications tend to bond to a
particular OS characteristic such as a uname, serial number or IP address. In most situations, these problems
can be overcome. The vast majority of commercial software products which run under AIX are well
suited to be clustered with HACMP.

Application Data Location
Where should application binaries and configuration data reside? There are many arguments to this discussion.
Generally, keep all the application binaries and data were possible on the shared disk, as it is easy
to forget to update it on all cluster nodes when it changes. This can prevent the application from starting or
working correctly, when it is run on a backup node. However, the correct answer is not fixed. Many application
vendors have suggestions on how to set up the applications in a cluster, but these are recommendations.
Just when it seems to be clear cut as to how to implement an application, someone thinks of a new
set of circumstances. Here are some rules of thumb:
If the application is packaged in LPP format, it is usually installed on the local file systems in rootvg. This
behavior can be overcome, by bffcreate’ing the packages to disk and restoring them with the preview option.
This action will show the install paths, then symbolic links can be created prior to install which point
to the shared storage area. If the application is to be used on multiple nodes with different data or configuration,
then the application and configuration data would probably be on local disks and the data sets on
shared disk with application scripts altering the configuration files during fallover. Also, remember the
HACMP File Collections facility can be used to keep the relevant configuration files in sync across the cluster.
This is particularly useful for applications which are installed locally.

Start/Stop Scripts
Application start scripts should not assume the status of the environment. Intelligent programming should
correct any irregular conditions that may occur. The cluster manager spawns theses scripts off in a separate
job in the background and carries on processing. Some things a start script should do are:
First, check that the application is not currently running! This is especially crucial for v5.4 users as
resource groups can be placed into an unmanaged state (forced down action, in previous versions).
Using the default startup options, HACMP will rerun the application start script which may cause
problems if the application is actually running. A simple and effective solution is to check the state
of the application on startup. If the application is found to be running just simply end the start script
with exit 0.
Verify the environment. Are all the disks, file systems, and IP labels available?
If different commands are to be run on different nodes, store the executing HOSTNAME to variable.
Check the state of the data. Does it require recovery? Always assume the data is in an unknown state
since the conditions that occurred to cause the takeover cannot be assumed.
Are there prerequisite services that must be running? Is it feasible to start all prerequisite services
from within the start script? Is there an inter-resource group dependency or resource group sequencing
that can guarantee the previous resource group has started correctly? HACMP v5.2 and later has
facilities to implement checks on resource group dependencies including collocation rules in
HACMP v5.3.
Finally, when the environment looks right, start the application. If the environment is not correct and
error recovery procedures cannot fix the problem, ensure there are adequate alerts (email, SMS,
SMTP traps etc) sent out via the network to the appropriate support administrators.
Stop scripts are different from start scripts in that most applications have a documented start-up routine
and not necessarily a stop routine. The assumption is once the application is started why stop it? Relying
on a failure of a node to stop an application will be effective, but to use some of the more advanced features
of HACMP the requirement exists to stop an application cleanly. Some of the issues to avoid are:

Be sure to terminate any child or spawned processes that may be using the disk resources. Consider
implementing child resource groups.
Verify that the application is stopped to the point that the file system is free to be unmounted. The
fuser command may be used to verify that the file system is free.
In some cases it may be necessary to double check that the application vendor’s stop script did actually
stop all the processes, and occasionally it may be necessary to forcibly terminate some processes.
Clearly the goal is to return the machine to the state it was in before the application start script was run.
Failure to exit the stop script with a zero return code as this will stop cluster processing. * Note: This is not the case with start scripts!
Remember, most vendor stop/starts scripts are not designed to be cluster proof! A useful tip is to have stop
and start script verbosely output using the same format to the /tmp/hacmp.out file. This can be achieved
by including the following line in the header of the script: set -x && PS4="${0##*/}"'[$LINENO]

AIX Security Checklist

2008-02-10T01:07:00.000+05:30

AIX Security Checklist

AIX Environment Procedures

The best way to approach this portion of the checklist is to do a comprehensive physical inventory of the servers. Serial numbers and physical location would be sufficient.

____Record server serial numbers
____Physical location of the servers

Next we want to gather a rather comprehensive list of both the AIX and pseries inventories. By running these next 4 scripts we can gather the information for analyze.

____Run these 4 scripts: sysinfo, tcpchk, nfsck and nethwchk. (See Appendix A for scripts)
____sysinfo:
____Determine active logical volume groups on the servers: lsvg -o
____List physical volumes in each volume group: lsvg –p "vgname"
____List logical volumes for each volume group: lsvg –l "vgname"
____List physical volumes information for each hard disk
____lspv hdiskx
____lspv –p hdiskx
____lspv –l hdiskx
____List server software inventory: lslpp -L
____List server software history: lslpp –h
____List all hardware attached to the server: lsdev –C | sort –d
____List system name, nodename, LAN network number, AIX release, AIX version and machine ID: uname –x
____List all system resources on the server: lssrc –a
____List inetd services: lssrc –t 'service name' –p 'process id'
____List all host entries on the servers: hostent -S
____Name all nameservers the servers have access to: namerslv –Is
____Show status of all configured interfaces on the server: netstat –i
____Show network addresses and routing tables: netstat –nr
____Show interface settings: ifconfig
____Check user and group system variables
____Check users: usrck –t ALL
____Check groups: grpck –t ALL
____Run tcbck to verify if it is enabled: tcbck
____Examine the AIX failed logins: who –s /etc/security/failedlogin
____Examine the AIX user log: who /var/adm/wtmp
____Examine the processes from users logged into the servers: who –p /var/adm/wtmp
____List all user attributes: lsuser ALL | sort –d
____List all group attributes: lsgroup ALL
____tcpchk:
____Confirm the tcp subsystem installed: lslpp –l | grep bos.net
____Determine if it is running: lssrc –g tcpip
____Search for .rhosts and .netrc files: find / -name .rhosts -print ; find / -name .netrc –print
____Checks for rsh functionality on host: cat /etc/hosts.equiv
____Checks for remote printing capability: cat /etc/hosts.lpd | grep v #
____nfschk:
____Verify NFS is installed: lslpp -L | bin/grep nfs
____Check NFS/NIS status: lssrc -g nfs | bin/grep active
____Checks to see if it is an NFS server and what directories are exported: cat /etc/xtab
____Show hosts that export NFS directories: showmount
____Show what directories are exported: showmount –e
____nethwchk
____Show network interfaces that are connected: lsdev –Cc if
____Display active connection on boot: odmget -q value=up CuAt | grep name|cut -c10-12
___Show all interface status: ifconfig ALL

Root level access

____Limit users who can su to another UID: lsuser –f ALL
____Audit the sulog: cat /var/adm/sulog
____Verify /etc/profile does not include current directory
____Lock down cron access
____To allow root only: rm –i /var/adm/cron/cron.deny and rm –I /var/adm/cron/cron.allow
____To allow all users: touch cron.allow (if file does not already exist)
____To allow a user access: touch /var/adm/cron/cron.allow then echo "UID">/var/adm/cron/cron.allow
____To deny a user access: touch /var/adm/cron/cron.deny then echo "UID">/var/adm/cron/cron.deny
____Disable direct herald root access: add rlogin=false to root in /etc/security/user file or through smit

____Limit the $PATH variable in /etc/environment. Use the users .profile instead.

Authorization/authentication administration

____Report all password inconsistencies and not fix them: pwdck –n ALL
____Report all password inconsistencies and fix them: pwdck –y ALL
____Report all group inconsistencies and not fix them: grpck –n ALL
____Report all group inconsistencies and fix them: grpck –y ALL
____Browse the /etc/shadow, etc/password and /etc/group file weekly

SUID/SGID

____Review all SUID/SGID programs owned by root, daemon, and bin.
____Review all SETUID programs: find / -perm -1000 –print
____Review all SETGID programs: find / -perm -2000 –print
____Review all sticky bit programs: find / -perm -3000 –print
____Set user .profile in /etc/security/.profile

Permissions structures

____System directories should have 755 permissions at a minimum
____Root system directories should be owned by root
____Use the sticky bit on the /tmp and /usr/tmp directories.
____Run checksum (md5) against all /bin, /usr/bin, /dev and /usr/sbin files.
____Check device file permissions:
____disk, storage, tape, network (should be 600) owned by root.
____tty devices (should be 622) owned by root.
____/dev/null should be 777.
____List all hidden files in there directories ( the .files).
____List all writable directories (use the find command).
____$HOME directories should be 710
____$HOME .profile or .login files should be 600 or 640.
____Look for un-owned files on the server: find / -nouser –print.
Note: Do not remove any /dev files.
____Do not use r-type commands: rsh, rlogin, rcp and tftp or .netrc or .rhosts files.
____Change /etc/host file permissions to 660 and review its contents weekly.

____Check for both tcp/udp failed connections to the servers: netstat –p tcp; netstat –p udp.
____Verify contents of /etc/exports (NFS export file).
____If using ftp, make this change to the /etc/inetd.conf file to enable logging.
ftp stream tcp6 nowait root /usr/sbin/ftpd ftpd –l
____Set NFS mounts to –ro (read only) and only to the hosts that they are needed.
____Consider using extended ACL's (please review the tcb man page).
____Before making network connection collect a full system file listing and store it off-line:
ls -Ra -la>/tmp/allfiles.system
____Make use of the strings command to check on files: strings /etc/hosts | grep Kashmir

Recommendations

Remove unnecessary services

By default the Unix operating system gives us 1024 services to connect to, we want to parse this down to a more manageable value. There are 2 files in particular that we want to parse. The first is the /etc/services file itself. A good starting point is to eliminate all unneeded services and add services as you need them. Below is a screenshot of an existing ntp server etc/services file on one of my lab servers.

#
# Network services, Internet style
#
ssh 22/udp
ssh 22/tcp mail
auth 113/tcp authentication
sftp 115/tcp
ntp 123/tcp # Network Time Protocol
ntp 123/udp # Network Time Protocol
#
# UNIX specific services
#
login 513/tcp
shell 514/tcp cmd # no passwords used

Parse /etc/rc.tcpip file

This file starts the daemons that we will be using for the tcp/ip stack on AIX servers. By default the file will start the sendmail, snmp and other daemons. We want to parse this to reflect what
functionality we need this server for. Here is the example for my ntp server.

# Start up the daemons
#
echo "Starting tcpip daemons:"
trap 'echo "Finished starting tcpip daemons."' 0
# Start up syslog daemon (for error and event logging)
start /usr/sbin/syslogd "$src_running"

# Start up Portmapper

start /usr/sbin/portmap "$src_running"

# Start up socket-based daemons
start /usr/sbin/inetd "$src_running"

# Start up Network Time Protocol (NTP) daemon
start /usr/sbin/xntpd "$src_running"

This helps also to better understand what processes are running on the server.

Remove unauthorized /etc/inittab entries

Be aware of what is in the /etc/inittab file on the AIX servers. This file works like the registry in a Microsoft environment. If an intruder wants to hide an automated script, he would want it launched here or in the cron file. Monitor this file closely.

Parse /etc/inetd.conf file

This is the AIX system file that starts system services, like telnet, ftp, etc. We also want to closely watch this file to see if there are any services that have been enabled without authorization. If you are using ssh for example this is what the inetd.con file should look like. Because we are using other internet connections, this file is not used in my environment and should not be of use to you. This is why ssh should be used for all administrative connections into the environment. It provides an encrypted tunnel so connection traffic is secure. In the case of telnet, it is very trivial to sniff the UID and password.

## protocol. "tcp" and "udp" are interpreted as IPv4.
##
## service socket protocol wait/ user server server program
## name type nowait program arguments
##

Edit /etc/rc.net

This is network configuration file used by AIX. This is the file you use to set your default network route along your no (for network options) attributes. Because the servers will not be used as routers to forward traffic and we do not want to use loose source routing at you, we will be making a few changes in this file. A lot of them are to protect from DOS and DDOS attacks from the internet. Also protects from ACK and SYN attacks on the internal network.

##################################################################
##################################################################
# Changes made on 06/07/02 to tighten up socket states on this

# server.

##################################################################
if [ -f /usr/sbin/no ] ; then
/usr/sbin/no -o udp_pmtu_discover=0 # stops autodiscovery of MTU
/usr/sbin/no -o tcp_pmtu_discover=0 # on the network interface
/usr/sbin/no -o clean_partial_conns=1 # clears incomplete 3-way conn.
/usr/sbin/no -o bcastping=0 # protects against smurf icmp attacks
/usr/sbin/no -o directed_broadcast=0 # stops packets to broadcast add.
/usr/sbin/no -o ipignoreredirects=1 # prevents loose
/usr/sbin/no -o ipsendredirects=0 # source routing
/usr/sbin/no -o ipsrcrouterecv=0 # attacks on
/usr/sbin/no -o ipsrcrouteforward=0 # our network
/usr/sbin/no -o ip6srcrouteforward=0 # from using indirect
/usr/sbin/no -o icmpaddressmask=0 # dynamic routes
/usr/sbin/no -o nonlocsrcroute=0 # to attack us from
/usr/sbin/no -o ipforwarding=0 # Stops server from acting like a router
fi

Securing root

Change the /etc/motd banner

This computer system is the private property of XYZ Insurance. It is for authorized use only. All users (authorized or non-authorized) have no explicit or implicit expectations of privacy.

Any or all users of this system and all the files on this system may be intercepted, monitored, recorded, copied, audited, inspected and disclosed to XYZ Insurance's management personnel.

By using this system, the end user consents to such interception, monitoring, recording, copying, auditing, inspection and disclosure at the discretion of such personnel. Unauthorized or improper use of this system may result in civil and/or criminal penalities and administrative or disciplinary action, as deemed appropriate by said actions. By continuing to use this system, the individual indicates his/her awareness of and consent to these terms and conditions of use.

LOG OFF IMMEDIATELY if you do not agree to the provisions stated in this warning banner.

Modify /etc/security/user

root:
loginretries = 5 – failed retries until account locks
rlogin = false – Disables remote herald access to a root shell. Need to su from another UID.
admgroups = system
minage = 0 – minimum aging is no time value
maxage = 4 – maximum aging is set to 30 days or 4 weeks
umask = 22

Tighten up /etc/security/limits

This is an attribute that should be changed due to a runaway resource hog. This orphaned process can grow to use
an exorbinate amount of disk space. To provent this we can set the ulimit value here.

default:
#fsize = 2097151
fsize = 8388604 – sets the soft file block size to a max of 8 Gig.

Variable changes in /etc/profile

Set the $TMOUT variable in /etc/profile. This will cause an open shell to close after 15 minutes of inactivity. It works in conjunction with the screensaver, to prevent an open session to be used to either delete the server or worse corrupt data on the server.

# Automatic logout, include in export line if uncommented
TMOUT=900

4.6.5 Sudo is your friend….

This is a nice piece of code that the system administrators can use in order to allow "root-like" functionality. It allows a non-root user to run system binaries or commands. The /etc/sudoers file is used to configure exactly what the user can do. The service is configured and running on ufxcpidev. The developers are running a script called changeperms in order to tag there .ear files with there own ownership attributes.

First we setup sudo to allow root-like or superuser doer access to sxnair.

# sudoers file.
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the sudoers man page for the details on how to write a sudoers file.
#
# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification
root ALL=(ALL) ALL
sxnair,jblade,vnaidu ufxcpidev=/bin/chown * /usr/WebSphere/AppServer/installedApps/*
#
#
# Override the built in default settings
Defaults syslog=auth

Defaults logfile=/var/log/sudo.log

For more details, please see the XYZ Company Insurance Work Report that I compiled, or visit this
URL: http://www.courtesan.com/sudo/.

Tighten user/group attributes

Change /etc/security/user

These are some of the changes to the /etc/security/user file that will promote a more heightened
configuration of default user attributes at your company.

default:

umask = 077 – defines umask values – 22 is readable only for that UID
pwdwarntime = 7 – days of password expiration warnings
loginretries = 5 – failed login attempts before account is locked
histexpire = 52 – defines how long a password cannot be re-used
histsize = 20 – defines how many previous passwords the system remembers
minage = 2 – minimum number of weeks a password is valid
maxage = 8 – maximum number of weeks a password is valid
maxexpired = 4 – maximum time in weeks a password can be changed after it exp

HACMP log files

2008-02-08T23:21:00.000+05:30

/usr/sbin/cluster/etc/rhosts --- to accept incoming communication from clcomdES (cluster communucation enahanced security)
/usr/es/sbin/cluster/etc/rhosts

Note: If there is an unresolvable label in the /usr/es/sbin/cluster/etc/rhosts file,
then all clcomdES connections from remote nodes will be denied.

cluster manager clstrmgrES
cluster lock Daemon (clockdES)
cluster multi peer extension communication daemon (clsmuxpdES)

The clcomdES is used for cluster configuration operations such as cluster synchronisation
cluster management (C-SPoC) * Dynamic re-configuration DARE configuration. (DARE ) operation.

For clcomdES there should be atleast 20 MB free space in /var file system.
/var/hacmp/clcomd/clcomd.log --it requires 2 MB
/var/hacmp/clcomd/clcomdiag.log --it requires 18MB
Additional 1 MB required for
/var/hacmp/odmcache directory

clverfify.log also present in /var directory
/var/hacmp/clverify/current//* contains log for mcurrent execution of clverify
/var/hacmp/clverify/pass//* contains logs from the last passed verification
/var/hacmp/clverify/pass.prev//* contains log from the second last passed verification

AIX 6.1 WPAR Commands

2008-01-13T23:10:00.000+05:30

Most of these commands are only runable by root user for the Global LPAR (Global
Environment).

chwpar Modifies dynamically the configuration options of the WPAR even when it is
running.

clogin Provides a mechanism for the root user to log in or run a command within a
workload partition. The “clogin” command uses a pseudo-terminal allocated on
the Global Environment (see in the mobility part the impact of this) and creates
a login shell running which belongs to the workload partition.

lswpar Prints information about WPARs

mkwpar Specific for System WPAR - Builds the infrastructure and the configuration
file. Then it creates and populates the WPARs file systems. Allocate the
specified resources as well as the WLM information (if any) and eventually
starts it. Specifying whether the workload partitions should be automatically
started on system boot or when /etc/rc.wpars is executed.

startwpar Activates the System WPAR defined with mkwpar

wparexec Specific for Application WPAR – Builds, create specification file and
eventually starts the Application WPAR

rebootwpar Stops and resumes a System Workload Partition. It cannot be run within a
WPAR

rmwpar Deletes the specified Workload Partition from the system as well as its
configuration and its WLM profile

stopwpar Kills the WPAR and deallocate all resources belonging to it. It removes the
Application WPAR

syncwpar Specific to System WPAR - Synchronizes the software installed in the global
shared parts (usually /usr and /opt) with the workload partition's root part

Note: The mkwpar and chwpar can update the global file /etc/wpars:

Note: /etc/rc.wpars: Specific to System WPAR - this script is run at main system boot it
invokes the startwpar command on all workload partitions with the autostart option.
Some commands have different or enhanced behavior in a WPAR environment. The list
includes acctcom, acctctl, df, domainname, hosted, hostname, ifconfig, ioo, ipcrm, ipcs,
mkclass, mount, netstat, nfso, no, projctl, ps , schedo, uname, vmo, wlmstat, wlmtune,
wlmcntrl. This change is described in the AIX 6 documentation.

The simplest way to administrate WPARs using command lines (CLI) is to issue the
command smitty wpar which will provide the interface to these commands.

Server Snapshots: IBM p570 Power 6

2008-01-06T18:09:00.002+05:30

The IBM POWER6 processor-based System p570 is a midrange server that aims to deliver outstanding price/performance while providing mainframe-like reliability and availability. This 19-inch rack-mount system, which can handle up to 16 POWER6 cores, is best-suited to database and application serving, as well as server consolidation.
"The first server to use the new POWER6 processor is the System p570," said Jeff Howard, director of System p offering management for IBM (Armonk, N.Y.). "The p570 leverages the chip's many breakthroughs in energy conservation and virtualization technology."

POWER6 Packs a Punch
The modular p570 takes over from its predecessor, the IBM POWER5+ processor-based System p5 570 server. POWER6 processors can run 64-bit applications while concurrently supporting 32-bit applications. They feature simultaneous multithreading, allowing two application threads to be run at the same time.
"At 4.7 GHz, the dual-core POWER6 processor doubles the speed of the previous generation, POWER5, while using the same amount of electricity to run and cool it," said Howard. "This means customers can use the new processor to either increase their performance by 100 percent or cut their power consumption in half."
Dan Olds, principal of Oregon-based Gabriel Consulting Group, believes the p570/POWER6 combo is an impressive one. The new processors come in 3.5 GHz, 4.2 GHz and 4.7 GHz flavors. IBM has effectively doubled the frequency over its predecessor without increasing the number of cores on the chip.

p570 close up

The POWER6 [processor] has two cores, each of which can simultaneously execute two threads, an approach known as simultaneous multithreading (SMT)," said Gordon Haff, an analyst at Illuminata (Nashua, N.H.). "POWER6 processors also have integrated memory controllers in order to improve memory access times and thereby application performance."
He pointed out that changes in the way POWER6 processors connect to each other and to the rest of the system flatten latency differences. There are also new specialized execution units and reliability features. Compute power is able to double largely because POWER6 has more than 790 million transistors compared to 276 million in the POWER5+. The amount of Level 2 Cache has gone up to 8 MB from just under 2 MB.
More to Come
While POWER6 has been released and is functioning currently in data centers, it is not quite a finished article. IBM is introducing more features later this year, such as Live Partition Mobility and Live Partition Migration. Live Partition Mobility, for example, will move a running Logical Partition (LPAR) from one physical server to another. It functions below the level of the operating system, at the hypervisor level; works only with the upcoming AIX 6 (currently in beta), AIX 5.3, and Linux; and the servers must all be on the same network subnet.
Haff said AIX 6 contains several major new features, including role-based access controls, a Trusted installation option and new "Workload Partitions" (WPAR). WPAR are resource groups similar to Solaris Containers or virtual servers that can be migrated from one server to another.
"I think that enhancements to the overall system, particularly the Live Partition Mobility feature, are big news and will provide the most business value to customers," said Olds. "With Live Partition Mobility, IBM users can move active workloads from one physical server to another — with no application interruption, no loss of transactions, and no user impact."
Gaining Ground
With such functions added to the already impressive POWER6, Olds believes the IBM p570 will gain major ground in the marketplace.
"This arguably means the end of planned application downtime and can eliminate perhaps half of overall application downtime, which is a big deal," said Olds. "This is much more sophisticated than VMware's VMotion and is a feature that their competitors will need to address."
Howard of IBM boasted that the POWER6 chip is nearly three times faster than the latest HP Itanium processor that runs much of HP's midrange server line. Thus, it will likely reinforce IBM's lead in the Unix servers market. According to research firm IDC (Framingham, Mass.), the company has a 29.6 percent revenue share in Unix.
"The p570 is the first server to hold all four major benchmark speed records for business and technical performance," said Howard. "In fact, the System p 570 now holds more than 20 benchmark records."
The p570 currently supports AIX 5L (v5.2 or later), the upcoming AIX 6, as well as Linux distributions from Red Hat (RHEL 4 Update 5 or later) and SUSE Linux (v10 SP1 or later).
The starting price for a basic version of the server is $60,000. IBM was reluctant to give out more details of pricing and configurations. Per its Web site, the minimum is a 2-core version operating at 3.5 GHz, so presumably that is what is obtained for this price. Each p570 building block accommodates two processor cards, each of which provides two 64-bit POWER6 processor cores, 8 MB of Level 2 (L2) cache and 32 MB of Level 3 (L3) cache. Each processor card has 12 slots for DDR2 (double data rate 2) ECC memory DIMMs, providing memory capacity of up to 192 GB per building block.

p570 Close Up

Name p 570
Vendor IBM
Platform Power6
Dimensions 4U rackmount; 19" by 32.4" by 6.85"; Weight: 140 lb
Processor Details 2- to 16-core POWER6
Hard Drives Six hot-swappable SAS disk bays per building block provide up to 7.2 TB of internal disk storage, up to four building blocks
Operating Systems Supported by AIX 5L (V5.2 or later) and Linux distributions from Red Hat (RHEL 4 Update 5 or later) and SUSE Linux (V10 SP1 or later) operating systems; AIX6 is in beta
Configuration Options The base configuration starts at $60,000 for a 2-core processor, 3.5 GHz, 64MB L3 cache, 16 GB system memory, and 2 x 73.4GB SAS 15K rpm internal storage. A high-end, system begins at $2 million before IBM discounts. This is a 16-core version running at 4.7 GHz.
GA Now available
Warranty Three years

Changing WPAR properties & starting it in Verbose mode.

2008-01-06T18:09:00.000+05:30

====================================================

1 To modify the host name of the workload partition called roy, enter the following command:

chwpar -h roy.com roy
2 To remove a network address from the workload partition called dale, enter the following command:

chwpar -K -N address=219.81.45.65 dale
3 To disable resource controls in the workload partition called wayne while retaining the settings for future use, enter the following command:

chwpar -R active=no wayne
4 To remove the device /dev/foo, exported by default through the devexportsFile, from a workload partition, enter the following command:

chwpar -D globaldev=/dev/foo export=no moore
5 To export the device /dev/bar, permitted by the devexportsFile but not exported, enter the following command:

chwpar -D globaldev=/dev/bar export=yes moore
6 To rename the workload partition from moore to hart, enter the following command:

chwpar -n hart moore
=========================================================

To start WPAR in verbose mode.

tarting workload partition MyTestWpar3.
Mounting all workload partition file systems.
Mounting /wpars/MyTestWpar3
Mounting /wpars/MyTestWpar3/home
Mounting /wpars/MyTestWpar3/opt
Mounting /wpars/MyTestWpar3/proc
Mounting /wpars/MyTestWpar3/tmp
Mounting /wpars/MyTestWpar3/usr
Mounting /wpars/MyTestWpar3/var
Loading workload partition.
$corral_t = {
'name' => 'MyTestWpar3',
'wlm_cpu' => [
undef,
undef,
undef,
undef
],
'path' => '/wpars/MyTestWpar3',
'hostname' => 'MyTestWpar3',
'wlm_procVirtMem' => [
-1,
undef
],
'wlm_mem' => [
undef,
undef,
undef,
undef
],
'key' => 5,
'vips' => [
[
'en0',
'172.29.138.24',
'255.255.192.0',
'172.29.191.255'
]
],
'wlm_rset' => undef,
'opts' => 4,
'id' => 0
};
Exporting workload partition devices.
Starting workload partition subsystem cor_MyTestWpar3.
0513-059 The cor_MyTestWpar3 Subsystem has been started. Subsystem PID is 536778.
Verifying workload partition startup.
Return Status = SUCCESS.
lpar47p682e_pub[/] >
-------------------------------------------------------

To remove WPAR from command line.

2008-01-06T18:03:00.000+05:30

par47p682e_pub[/] > rmwpar TestWpar3
rmwpar: Removing file system /wpars/TestWpar3/var.
rmlv: Logical volume fslv15 is removed.
rmwpar: Removing file system /wpars/TestWpar3/usr.
rmwpar: Removing file system /wpars/TestWpar3/tmp.
rmlv: Logical volume fslv14 is removed.
rmwpar: Removing file system /wpars/TestWpar3/proc.
rmwpar: Removing file system /wpars/TestWpar3/opt.
rmwpar: Removing file system /wpars/TestWpar3/home.
rmlv: Logical volume fslv13 is removed.
rmwpar: Removing file system /wpars/TestWpar3.
rmlv: Logical volume fslv12 is removed.

To remove WPAR first we need to stop WPAR in order to remove it.
After removing WPAR all filesystem related to that WPAR gets deleted.