Though this task is well documented (see [1] and [2]) there are still some pitfalls and quirks that can cause problems. Such as.. Should a particular task be done with the root or oracle user? Should it be performed on both nodes? This is a step-by-step description how to perform the change of the public IP address in the Oracle two node RAC. The Oracle version is 10.2.0.3. The OS used in this example is Linux Centos 2.6.9-42.ELsmp Note that due to the change of the private IP address the virtual IP address (VIP) may be required to change as well as it must remain in the same subnet as the public IP address. The configuration | Host | Current IP Address | Changed IP Address | Private IP | hst1 | 192.168.1.101 | 192.168.2.121 | Private IP | hst2 | 192.168.1.100 | 192.168.2.120 | VIP | hst1-vip | 192.168.1.111 | 192.168.2.111 | VIP | hst2-vip | 192.168.1.110 | 192.168.2.110 |
Note that the actual value of IP addresses and host names was changed. The basic steps The change of the private IP address is performed in the following basic steps 1) Shut down everything except the CRS stack 2) Change the public interface 3) Modify the VIP address 4) Shut down CRS 5) Modify IP address on OS level and reconfigure /etc/hosts, listener,.. 6) Restart The order of the steps is very important. E.g. step 3) can be performed only if the CRS stack is up. If you perform step 5) prematurely, CRS will go down automatically (probably trying some reboots beforehand). This will effectively block step 3) to be done. Shut Down Everything Except the CRS Stack The database and nodeapps (on all nodes) are stopped. [oracle@hst1 ~]$ srvctl stop database -d mydb [oracle@hst1 ~]$ srvctl stop nodeapps -n hst1 [oracle@hst1 ~]$ srvctl stop nodeapps -n hst2 Note that if you use ASM, it must be stopped as well. After that we verify the status. [oracle@hst1 ~]$ srvctl status database -d mydb Instance MYDB1 is not running on node hst2 Instance MYDB2 is not running on node hst1 [oracle@hst1 ~]$ srvctl status nodeapps -n hst1 VIP is not running on node: hst1 GSD is not running on node: hst1 Listener is not running on node: hst1 ONS daemon is not running on node: hst1 [oracle@hst1 ~]$ srvctl status nodeapps -n hst2 VIP is not running on node: hst2 GSD is not running on node: hst2 Listener is not running on node: hst2 ONS daemon is not running on node: hst2 Everything is OK, we can go to the next step. Change the Public Interface First let us have a look on the actual status [oracle@hst1 ~]$ oifcfg getif eth0 192.168.1.0 global public eth1 192.168.2.0 global cluster_interconnect We need to change the interface eth0. As there is no modify command, we will delete and redefine the interface. [oraclu@hst1 ~]$ oifcfg delif -global eth0 [oraclu@hst1 ~]$ oifcfg setif -global eth0/192.168.2.0:public the exact syntax of the setif command can be found in [3] Note that the CRS installation user (here oraclu) must be used for this operation. Otherwise an error message is issued: PROC-5: User does not have permission to perform a cluster registry operation on this key. Authentication error [User [oraclu] does not match with initialized u ser] [0] PRIF-11: cluster registry error Finally, we check if the action was done successfully [oracle@hst1 ~]$ oifcfg getif eth0 192.168.2.0 global public eth1 192.168.2.0 global cluster_interconnect Modify the VIP Address As already mentioned, due to the fact that we changed the subnet of the public IP address, we must change the VIP address as well. The following modify statement should be used. [root@hst1 ~]# /appl/oracle/product/10.2.0/db_1/bin/srvctl modify nodeapps -n hst2 -A 192.168.2.110/255.255.255.0/eth0 [root@hst1 ~]# /appl/oracle/product/10.2.0/db_1/bin/srvctl modify nodeapps -n hst1 -A 192.168.2.111/255.255.255.0/eth0 Note that the root user should be used for this action. The variable ORACLE_HOHE must be initialised. Otherwise one of the following errors will be raised. PRKO-2117 : This command should be executed as the system privilege user. ****ORACLE_HOME environment variable not set! ORACLE_HOME should be set to the main directory that contains Oracle products. Set and export ORACLE_HOME, then re-run. Shut Down CRS The stop command must be performed on all nodes: crsctl stop crs Modify the IP Address on OS Level Modify the public IP address (eth0). In Centos with application / system setting / network Perform the change on all nodes. Modify /etc/hosts and listener.ora files if required. Restart Reboot all nodes and verify the status. Shortly after reboot we can see... [oracle@hst2 ~]$ /sbin/ifconfig -a | egrep '(eth|Mask)' eth0 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:FE inet addr:192.168.2.120 Bcast:192.168.2.255 Mask:255.255.255.0 eth0:1 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:FE inet addr:192.168.2.111 Bcast:192.168.2.255 Mask:255.255.255.0 eth0:2 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:FE inet addr:192.168.2.110 Bcast:192.168.2.255 Mask:255.255.255.0 eth1 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:F1 inet addr:192.168.2.100 Bcast:192.168.2.255 Mask:255.255.255.0 inet addr:127.0.0.1 Mask:255.0.0.0 This means both VIP's eth0:1 and eth0:2 are switched to one host. This is not good, normally this appears if one host is down and the second host takes over the VIP. Fortunately, after a few seconds we see.. [oracle@hst2 ~]$ /sbin/ifconfig -a | egrep '(eth|Mask)' eth0 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:FE inet addr:192.168.2.120 Bcast:192.168.2.255 Mask:255.255.255.0 eth0:2 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:FE inet addr:192.168.2.110 Bcast:192.168.2.255 Mask:255.255.255.0 eth1 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:F1 inet addr:192.168.2.100 Bcast:192.168.2.255 Mask:255.255.255.0 inet addr:127.0.0.1 Mask:255.0.0.0 [oracle@hst1 ~]$ /sbin/ifconfig -a | egrep '(eth|Mask)' eth0 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:F2 inet addr:192.168.2.121 Bcast:192.168.2.255 Mask:255.255.255.0 eth0:1 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:F2 inet addr:192.168.2.111 Bcast:192.168.2.255 Mask:255.255.255.0 eth1 Link encap:Ethernet HWaddr CA:FE:CA:FE:CA:F3 inet addr:192.168.2.101 Bcast:192.168.2.255 Mask:255.255.255.0 inet addr:127.0.0.1 Mask:255.0.0.0 I.e. the VIP's are distributed correctly. The configuration of IP addresses is as required. Let’s have a final look at the status of the RAC. [oracle@hst1 ~]$ srvctl status database -d mydb Instance MYDB1 is running on node hst2 Instance MYDB2 is running on node hst1 [oracle@hst1 ~]$ srvctl status nodeapps -n hst1 VIP is running on node: hst1 GSD is running on node: hst1 Listener is running on node: hst1 ONS daemon is running on node: hst1 [oracle@hst2 ~]$ srvctl status database -d mydb Instance MYDB1 is running on node hst2 Instance MYDB2 is running on node hst1 [oracle@hst2 ~]$ srvctl status nodeapps -n hst1 VIP is running on node: hst1 GSD is running on node: hst1 Listener is running on node: hst1 ONS daemon is running on node: hst1 [root@hst1 ~]# /appl/oraclu/product/10.2.0/crs/bin/crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora....B1.inst application ONLINE ONLINE hst2 ora....B2.inst application ONLINE ONLINE hst1 ora....DB1.srv application ONLINE ONLINE hst2 ora....DB2.srv application ONLINE ONLINE hst1 ora....BTAF.cs application ONLINE ONLINE hst2 ora.MYDB.db application ONLINE ONLINE hst2 ora....B2.lsnr application ONLINE ONLINE hst2 ora.hst2.gsd application ONLINE ONLINE hst2 ora.hst2.ons application ONLINE ONLINE hst2 ora.hst2.vip application ONLINE ONLINE hst2 ora....B1.lsnr application ONLINE ONLINE hst1 ora.hst1.gsd application ONLINE ONLINE hst1 ora.hst1.ons application ONLINE ONLINE hst1 ora.hst1.vip application ONLINE ONLINE hst1 Everything is up. We are ready! References [1] http://forums.oracle.com/forums/thread.jspa?messageID=2161448 [2] Metalink Note:276434.1 https://metalink.oracle.com/metalink/plsql/showdoc?db=NOT&id=276434.1&blackframe=1 [3] Metalink Note:283684.1 https://metalink.oracle.com/metalink/plsql/showdoc?db=NOT&id=283684.1&blackframe=1 (Account is required for Metalink access) Jaromir D.B. Nemec is a freelancer specializing in data warehouse and integration solutions. He can be reached under http://www.db-nemec.com |