ORACLE 10g RAC(ASM) Install on AIX6.1

I will describe step by step the architecture Oracle 10g CRS (Cluster Ready Service) on raw disks and database on ASM(Automatic Storage Management) on AIX 6.1 with VIO.

I will not go into details to explain but i try to explain what we have done from AIX administrator point of view.

SETUP DETAILS:

ORACLE 10g Release 2 (10.2)
AIX 6.1 (VIO server 2.1.2.10) LPAR on 2 different servers
OS / DataBase disk from SAN (DS8000)

2 Nodes (LPAR) with 2 network card and one Alias IP (eg: for en0
one Node IP 
AIX 6.1 Certification
Starting with Oracle Database 10g release 2 (10.2.0.3), AIX 6.1 (64 bit) operating system is supported. Refer to “Preinstallation Requirements” for more information

The latest information on patches and support for this release is available in
OracleMetaLink document 282036.1.

ODBC Certification

Starting with Oracle Database 10g release 2 (10.2.0.4), ODBC driver is supported on AIX operating systems.

3 Software Contents on Installation Media

The following products and documentation are located on the first DVD:

 Oracle Database 10g release 2 (10.2) software and documentation

 Oracle Database Companion Products 10g release 2 (10.2) software and documentation

 Oracle Database 10g release 2 (10.2) Documentation Library The following
products and documentation are located on the second DVD:

 Oracle Clusterware 10g release 2 (10.2) software and documentation

 Oracle Database Client 10g release 2 (10.2) software and documentation

 Oracle Gateway 10g release 2 (10.2) software and documentation

 Oracle Database 10g release 2 (10.2) Documentation Library


4 Preinstallation Requirements

In addition to the information in the installation guides, the following section contains the system requirements for AIX 6.1.

You must review the following sections before installing Oracle Database on AIX 6L:
Operating System Requirement
Operating System Filesets for AIX 6L Oracle Installations
32-Bit Kernel Support
Configuring Shell Limits for Oracle RAC Installations

 

4.1 Operating System Requirement

In addition to the supported operating systems listing in the installation guide, AIX 6L, version 6.1,
64-bit kernel is supported with service pack 01 (6100-00-01) or later.

Refer to Oracle Database Installation Guide for AIX 5L Based Systems (64-Bit) for additional information on operating system listings.

4.2 Operating System Filesets for AIX 6L

The following filesets are supported on Oracle Database 10g release 2:

bos.adt.base
bos.adt.lib
bos.adt.libm
bos.perf.libperfstat
bos.perf.perfstat
bos.perf.proctools
xlC.aix61.rte:9.0.0.1
xlC.rte:9.0.0.1

4.3 Oracle Installations

Installing on AIX 6L requires the following installations:

 Oracle patch 6613550
 Oracle Database 10g release 2 (10.2.0.1)
Oracle Database patch set 10g release 2 (10.2.0.3)

4.4 32-Bit Kernel Support

Oracle Database 10g Release 2 (10.2) supports 32-bit kernel for single-instance databases but does not support 32-bit kernel for Oracle Real Application Cluster (RAC).

4.5 Configuring Shell Limits for Oracle RAC Installations

Unlimited shell values must be set for both the Oracle installation owner (oracle) and the root user.
The root user requirement for unlimited is needed because the
Oracle Clusterware daemon (crsd) runs as root.

Configure the shell
limits setting to unlimited either by using the smit utility or by
editing the
/etc/security/limits file. Add the following lines to the limits
file:

time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 4194304
memory(kbytes) unlimited
coredump(blocks) 2097151
nofiles(descriptors) 2000

threads(per process) unlimited
processes(per user) unlimited
( -1 will set to unlimited )
 

This requirement supplements the description of shell limits in “Section 2.4.1,Configure Shell Limits” in Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide for AIX Based Systems 10g Release 2 (10.2).
With Oracle RAC, each instance must have access to the datafiles and recovery files for the Oracle 10g RAC database. Using Automatic Storage Management (ASM) is an easy way to satisfy this requirement.
ASM is an integrated, high-performance database file system and disk manager. ASM is based on the principle that the database should manage storage instead of requiring an administrator to do it.
ASM eliminates the need for you to directly manage potentially thousands of Oracle database files. ASM groups the disks in your storage system into one or more disk groups. You manage a small set of disk groups and ASM automates the placement of the database files within those disk groups.

change new values to network // to add in next boot
/etc/tunables/nextboot  run all command with no -r

no -r -o tcp_sendspace= 262144
no -r -o tcp_recvspace= 262144
no -r -o udp_sendspace= 65536
no -r -o udp_recvspace= 262144
no -r -o sb_max= 1310720
no -r -o rfc1323= 1

DO NOT ADD ALIAS IP TO network card ORACLE will assign automatically.

You must install SDDPCM filesets and enable SDDPCM.. SDDPCM cannot co-exist with SDD.

Disks assigned as follows

The following are mandatory for oracle cluster ware installation and database home installation for RAC setup: OCR disk : Normal Redundancy (OCR mirroring by Oracle Clusterware) If “NORMAL Redundancy” selected,

• /dev/ocr_disk1 ( each 1Gb from DS8000)

• /dev/ocr_disk2 ( from 1GB DS8000x)

Specify the OCR location : this must be a shared location on the shared storage reachable from all nodes. And you must have the read/wright permissions on this shared location from all nodes.

Voting disks : Normal Redundancy (Voting copies managed by Oracle Clusterware)
If “NORMAL Redundancy” selected, specify raw disk location as follow:

• /dev/voting_disk1 ( each 1Gb from DS8000)

• /dev/voting_disk2 ( each 1Gb from DS8000x)

• /dev/voting_disk3 (each 1 Gb from DS8000)

For dbf files

From DS8000

/dev/hddisk6( each 10GB)
/dev/hddisk11( each 10GB)
/dev/hddisk7( each 10GB)
/dev/hddisk12( each 10GB)
/dev/hddisk8( each 10GB)
/dev/hddisk13( each 10GB)

From DS8000x

/dev/hddisk9( each 10GB)
/dev/hddisk14( each 10GB)
/dev/hddisk10( each 10GB)
/dev/hddisk15( each 10GB)

DS8000 and DS8000X are different SAN storages; All disks to be shared among both Nodes.

# ls -al hdisk*

brw——-1 root system 13, 0 Dec 23 22:18 hdisk1

brw——-1 root system 13, 2 Dec 23 22:18 hdisk2

brw——-1 root system 13, 3 Dec 23 22:18hdisk3

brw——-1 root system 13, 4 Dec 23 22:18 hdisk4

brw——-1 root system 13, 5 Dec 23 22:18 hdisk5

if your ocr disk is hdisk3 then major and minor number will be 13,3

mknod /dev/ocr_disk1 c 13 3 (do the same for all OCR and Voting disks)

chdev -l hdisk#no -a pv= clear

chdev -l hdisk#no -a reserve_policy= no_reserve

chdev -l hdisk#no -a reserve_lock= no

set ownerchip to oracle:dba to the /dev/hdisk#no

set read/write permissions to 660 to the /dev/hdisk#no

enable RSH

cd /oracle

vi .rhosts //add other hostname username

chmod 600 .rhosts

cd /etc vi hosts.equiv //add other hostname username

add in /etc/hosts

## public

10.1.115.158 node1
10.1.115.159 node2

## VIP

10.1.115.160 node1-vip
10.1.115.161 node2-vip

## Private

10.2.115.158 node1-priv
10.2.115.159 node2-priv

To establish a default gateway, type:

{node1:root}/ -> route add 0 10.2.115.254

{node2:root}/ -> route add 0 10.2.115.254

The value 0 or the default keyword for the Destination parameter means that
any packets sent to destinations not previously defined and not on a directly
connected network go through the default gateway. The 10.2.115.254 address
is that of the gateway chosen to be the default.

change paging space according to RAM X 2

To create the following groups :

smit group

dba .. Primary group for the oracle user.

oinstall .. The ora inventory group. This group is not mandatory. If it
exists, it will be the group owner of the oracle code files. This group is a
secondary group for oracle user.

To create the users :

smit user

oracle .. Owner of the database.

STEPS done from ORACLE TEAM

Download oracle patchset 10.2.0.4 patch # 6810189

Configured rsh between Node1 & Node2 successfully for clusterware installation.

Uploaded 2DVDS and Oracle patch set to two oracle nodes ( i.e. Node1 & Node2 ) successfully

Installed oracle clusterware ( CRS_HOME i.e. /oracle/products/crs) on Node1 as primary node successfully

Installed oracle ASM ( ASM_HOME i.e /oracle/products/asm) on Node1 as primary node successfully

Executed VIPCA on Node2 for fully configuring RAC nodes.

Installed Oracle HOME ( ORACLE_HOME i.e /oracle/products/10.2.0 on Node1 as primary node successfully .

Created ASM instance between Node1 & Node2 successfully.

Configured ASM disks for ASM , Oracle DB and flash back recovery.

Upgraded CRS_HOME to 10.2.0..4 by applying patch set 6810189 successfully

Upgraded ORACLE_HOME to 10.2.0..4 by applying patch set 6810189 successfully.

Upgraded ASM_HOME to 10.2.0.4 by applying patch set 6810189 successfully

Created “test” RAC database on Node1 node as primary node successfully.

Created ASD disk group for flash back recovery.

Changed newly created RAC database to archive log mode and assigned new ASM disk groups for archive log destination with normal redundancy.

Configured a sample service to test connectivity between nodes.

We performed all sanity checks relating to CRS/ASM/Oracle Home successfully. As of now RAC database is up and running between nodes successfully.

SQL> startup

ORA-27504: IPC error creating OSD context

ORA-27300: OS system dependent operation:sendmsg failed with status: 59

ORA-27301: OS failure message: Message too long

ORA-27302: failure occurred at: sskgxpsnd1

ORA-27303: additional information: MTU verification failed to send msg.

Fix for above issue :

Ora-27302:sskgxpsnd1 – Starting Instance [ID 300956.1]

Applies To

Oracle Server – Enterprise Edition – Version: 9.2.0.1 to 10.1.0.3
AIX5L Based Systems (64-bit)
HP Tru64

Symptoms

When starting an instance on AIX platform we may recieve the following errors in the alert log file:

ORA-00603: ORACLE server session terminated by fatal error

ORA-27504: IPC error creating OSD context

ORA-27300: OS system dependent operation:sendmsg failed with status:59

ORA-27301: OS failure message: Message too long

ORA-27302: failure occurred at: sskgxpsnd1

4 thoughts on “ORACLE 10g RAC(ASM) Install on AIX6.1

  1. Hi,
    Thank you for sharing your experience….

    Regarding the Operationg system requirements for file set, the rte can be later than version 9 xlC.aix61.rte
    xlC.rte

    It would be a little concern if you were actually generating ProC or Oracle C Call Interface programs – but as you only need the rte (runtime environment) I am sure this will be OK for linking within an install.

    Note 282036.1 references the minimum required version of xLC references the ‘minimum’ requirements.

    Regards,
    C.

  2. root is blank

    default:

    fsize = -1
    core = 2097151
    cpu = -1
    data = -1
    rss = -1
    stack = -1
    nofiles = 2000

    oracle:
    fsize = -1
    core = 2097151
    cpu = -1
    data = -1
    rss = -1
    stack = -1
    nofiles = 2000

    set stack to -1 for oracle not in /etc/security/limits but in user enviornment too.

    smitty user
    user oracle
    change / show user enviornment
    set hard stack to -1

  3. Hi,
    Thanks for the gr8 info provided by.

    However I will be glad if you can share the complete step by step doc for 11gR2 installation on AIX 6.1.

    If yoy dont have it please send it for 10gR2 on AIX to my below mail address
    sosforu@gmail.com

Leave a Comment

Your email address will not be published. Required fields are marked *