Friday, February 15, 2008

Specifying the default gateway on a specific interface in HACMP

Specifying the default gateway on a specific interface

When you're using HACMP, you usually have multiple network adapters installed and thus multiple network interface to handle with. If AIX configured the default gateway on a wrong interface (like on your management interface instead of the boot interface), you might want to change this, so network traffic isn't sent over the management interface. Here's how you can do this:

First, stop HACMP or do a take-over of the resource groups to another node; this will avoid any problems with applications when you start fiddling with the network configuration.

Then open up a virtual terminal window to the host on your HMC. Otherwise you would loose the connection, as soon as you drop the current default gateway.

Now you need to determine where your current default gateway is configured. You can do this by typing: lsattr -El inet0 and netstat -nr. The lsattr command will show you the current default gateway route and the netstat command will show you the interface it is configured on. You can also check the ODM: odmget -q"attribute=route" CuAt.

Now, delete the default gateway like this:
lsattr -El inet0 | awk '$2 ~ /hopcount/ { print $2 }' | read GW
chdev -l inet0 -a delroute=${GW}

If you would now use the route command to specifiy the default gateway on a specific interface, like this:
route add 0 [ip address of default gateway:] -if enX
You will have a working entry for the default gateway. But... the route command does not change anything in the ODM. As soon as your system reboots; the default gateway is gone again. Not a good idea.

A better solution is to use the chdev command:
chdev -l inet0 -a addroute=net,-hopcount,0,,0,[ip address of default gateway]
This will set the default gateway to the first interface available.

To specify the interface use:
chdev -l inet0 -a addroute=net,-hopcount,0,if,enX,,0,[ip address of default gateway]
Substitute the correct interface for enX in the command above.

If you previously used the route add command, and after that you use chdev to enter the default gateway, then this will fail. You have to delete it first by using route delete 0, and then give the chdev command.

Afterwards, check with lsattr -El inet0 and odmget -q"attribute=route" CuAt if the new default gateway is properly configured. And ofcourse, try to ping the IP address of the default gateway and some outside address. Now reboot your system and check if the default gateway remains configured on the correct interface. And startup HACMP again!


cbish68 said...

Thanks for the default gateway tip. I moved a server and changed its IP address. When it came back up in the new location, I realized I had forgotten to change the deafult route prior to moving it. The interface was configured correctly, but you could not ping the server or get network services like NIS to work. As soon as I used chdev to add my default route, bingo! Everything started talking. Again, thank you for taking the time to write this down.

Bharath said...

Hi Santosh,

I have a question regarding the default gateway. From your explaination it looks like our HACMP node's default gateway is not configured properly. Below is a small description of our environment

Environment : 3 LPARs ( host 1a, host 1b and host2).
Host 1a and 1b are hacmp nodes with host 1a being the active node

Host 1a has 3 network interfaces (en0, en1 and en2)
en1 – persistent ip
en2 – service ip allocated to it.
Host 1a and 1b are on subnet A and host 2 is on subnet B.

Issue: can’t mount a FS exported from host 1a on host2 using service ip address of host 1a.

On host 2 need to run:(example)
mount service_ip_host_1a:/home/Bharath/file1 /tmp

Host1a routing table: default route is going through en1 (persistent ip) interface. So what ever packet transmitted to host2 goes through en1 (even if host 1a receives packet on interface en2-service ip)

any suggestion for this