RAC搭建手册

RAC搭建手册... 1

一．环境介绍... 1

二．配置hostname和hosts文件两台都需要设置... 1

三．关闭防火墙，NTP服务，关闭selinux. 2

四．创建必要的用户、组和目录，并授权... 2

五. 节点配置检查... 3

六．参数文件修改... 3

七．安装所需的依赖包... 4

八．添加软连接... 4

九．配置grid和oracle用户环境变量... 5

十．配置grid用户互信... 5

十一.配置裸盘... 7

十二.grid安装... 8

十三.asm磁盘组创建... 27

十四.安装oracle software. 32

一．环境介绍

scan-ip

172.20.0.174

rac1-public ip

172.20.0.25

rac2-public ip

172.20.0.26

rac1-vip

172.20.0.186

rac2-vip

172.20.0.189

rac1-private ip

192.168.2.112

rac2-private ip

192.168.2.107

二．配置hostname和hosts文件两台都需要设置

172.20.0.25rac1:

[root@localhost ~]# vi /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=rac1

172.20.0.26 rac2

[root@localhost ~]# vi /etc/sysconfig/network

NETWORKING=yes

HOSTNAME=rac2

[root@localhost ~]# vi /etc/hosts

172.20.0.25 rac1

172.20.0.186 rac1-vip

192.168.2.112 rac1-priv

172.20.0.26 rac2

172.20.0.189 rac2-vip

192.168.2.107 rac2-priv

172.20.0.174 scan-ip

三．关闭防火墙，NTP服务，关闭selinux

rac1和rac2都需要关闭

[root@rac1 ~]# vi /etc/sysconfig/selinux

SELINUX=disabled

这里需要重启生效

[root@rac1 ~]# getenforce

disabled

[root@rac1 ~]#/etc/init.d/iptables stop

[root@rac1 ~]# chkconfig iptables off

[root@rac1 ~]# chkconfig ntpd off

四．创建必要的用户、组和目录，并授权

两个节点rac1和rac2都需要创建，这里只演示rac1上面的操作。

[root@rac1 ~]# /usr/sbin/groupadd -g 1000 oinstall

[root@rac1 ~]# /usr/sbin/groupadd -g 1020 asmadmin

[root@rac1 ~]# /usr/sbin/groupadd -g 1021 asmdba

[root@rac1 ~]# /usr/sbin/groupadd -g 1022 asmoper

[root@rac1 ~]# /usr/sbin/groupadd -g 1031 dba

[root@rac1 ~]# /usr/sbin/groupadd -g 1032 oper

[root@rac1 ~]# useradd -d /opt/grid -u 1100 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid

[root@rac1 ~]# useradd -d /opt/oracle -u 1101 -g oinstall -G dba,asmdba,oper oracle

[root@rac1 ~]# passwd oracle

Changing password for user oracle.

New password:

BAD PASSWORD: it is based on a dictionary word

BAD PASSWORD: is too simple

Retype new password:

passwd: all authentication tokens updated successfully.

[root@rac1 ~]# passwd grid

Changing password for user grid.

New password:

BAD PASSWORD: it is too short

BAD PASSWORD: is too simple

Retype new password:

passwd: all authentication tokens updated successfully.

[root@rac1 ~]# mkdir -p /u01/app/11.2.0/grid

[root@rac1 ~]# mkdir -p /u01/app/grid

[root@rac1 ~]# mkdir /u01/app/oracle

[root@rac1 ~]# chown -R grid:oinstall /u01

[root@rac1 ~]# chown oracle:oinstall /u01/app/oracle

[root@rac1 ~]# chmod -R 775 /u01/

[root@rac1 ~]#

五. 节点配置检查

查看内存和swap大小，内存大小：至少2.5GB

root@rac1 ~]# grep MemTotal /proc/meminfo

MemTotal: 8061904 kB

[root@rac1 ~]# grep SwapTotal /proc/meminfo

SwapTotal: 3145720 kB

[root@rac1 ~]#

六．参数文件修改

（1）内核参数设置：

[root@rac1 ~]# vi /etc/sysctl.conf

# Controls the maximum shared segment size, in bytes

kernel.shmmax = 68719476736（已有）物理内存的1/2（M）*1024*1024

# Controls the maximum number of shared memory segments, in pages

kernel.shmall = 4294967296（已有）

#oracle setting

fs.aio-max-nr = 1048576

fs.file-max = 6815744

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default = 262144

net.core.rmem_max = 4194304

net.core.wmem_default = 262144

net.core.wmem_max = 1048576

#/sbin/sysctl -p 执行该命令，使修改生效

[root@rac1 ~]# /sbin/sysctl -p

（2）配置oracle、grid用户的shell限制

[root@rac1 ~]# vi /etc/security/limits.conf
oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

oracle soft stack 10240

grid soft nproc 2047

grid hard nproc 16384

grid soft nofile 1024

grid hard nofile 65536

（3）配置login

[root@rac1 ~]# vi /etc/pam.d/login
session required pam_limits.so

七．安装所需的依赖包

yum -y install binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++

yum -y install glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel

yum -y install libgcc libstdc++ libstdc++-devel make numactl-devel sysstat unixODBC unixODBC-devel

yum install libcap

八．添加软连接

[root@rac1 ~]# cd /lib64

ln -s libcap.so.2 libcap.so.1

九．配置grid和oracle用户环境变量

Oracle_sid需要根据节点不同进行修改

[root@rac1 ~]# su - grid
[grid@rac1 ~]$ vi .bash_profile

export TMP=/tmp

export TMPDIR=$TMP

export ORACLE_SID=+ASM1 # RAC1

export ORACLE_SID=+ASM2 # RAC2

export ORACLE_BASE=/u01/app/grid

export ORACLE_HOME=/u01/app/11.2.0/grid

export PATH=/usr/sbin:$PATH

export PATH=$ORACLE_HOME/bin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

umask 022

需要注意的是ORACLE_UNQNAME是数据库名，创建数据库时指定多个节点是会创建多个实例，ORACLE_SID指的是数据库实例名

[root@rac1 ~]# su - oracle
[oracle@rac1 ~]$ vi .bash_profile

export TMP=/tmp

export TMPDIR=$TMP

export ORACLE_SID=orcl1 # RAC1

export ORACLE_SID=orcl2 # RAC2

export ORACLE_UNQNAME=orcl

export ORACLE_BASE=/u01/app/oracle

export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1

export TNS_ADMIN=$ORACLE_HOME/network/admin

export PATH=/usr/sbin:$PATH

export PATH=$ORACLE_HOME/bin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

$ source .bash_profile使配置文件生效

十．手工配置grid用户互信

配置过程如下：

各节点生成Keys：

配置过程如下：

各节点生成Keys：

[root@rac1 ~]# su - grid

[grid@rac1 ~]$ mkdir ~/.ssh

[grid@rac1 ~]$ chmod 700 ~/.ssh

[grid@rac1 ~]$ ssh-keygen -t rsa

[grid@rac1 ~]$ ssh-keygen -t dsa

[root@rac2 ~]# su - grid

[grid@rac2 ~]$ mkdir ~/.ssh

[grid@rac2 ~]$ chmod 700 ~/.ssh

[grid@rac2 ~]$ ssh-keygen -t rsa

[grid@rac2 ~]$ ssh-keygen -t dsa

在节点1上进行互信配置：

[grid@rac1 ~]$ touch ~/.ssh/authorized_keys

[grid@rac1 ~]$ cd ~/.ssh

[grid@rac1 .ssh]$ ssh rac1 cat ~/.ssh/id_rsa.pub >> authorized_keys

[grid@rac1 .ssh]$ ssh rac2 cat ~/.ssh/id_rsa.pub >> authorized_keys

[grid@rac1 .ssh]$ ssh rac1 cat ~/.ssh/id_dsa.pub >> authorized_keys

[grid@rac1 .ssh]$ ssh rac2 cat ~/.ssh/id_dsa.pub >> authorized_keys

在rac1把存储公钥信息的验证文件传送到rac2上

[grid@rac1 .ssh]$ pwd

/home/grid/.ssh

[grid@rac1 .ssh]$ scp authorized_keys rac2:'pwd'

grid@rac2's password:

authorized_keys 100% 1644 1.6KB/s 00:00

设置验证文件的权限

在每一个节点执行：

$ chmod 600 ~/.ssh/authorized_keys

启用用户一致性

在你要运行OUI的节点以grid用户运行(这里选择rac1)：

[grid@rac1 .ssh]$ exec /usr/bin/ssh-agent $SHELL

[grid@rac1 .ssh]$ ssh-add

Identity added: /home/grid/.ssh/id_rsa (/home/grid/.ssh/id_rsa)

Identity added: /home/grid/.ssh/id_dsa (/home/grid/.ssh/id_dsa)

验证ssh配置是否正确

以grid用户在所有节点分别执行：

ssh rac1 date

ssh rac2 date

ssh rac1-priv date

ssh rac2-priv date

如果不需要输入密码就可以输出时间，说明ssh验证配置成功。必须把以上命令在两个节点都运行，每一个命令在第一次执行的时候需要输入yes。

如果不运行这些命令，即使ssh验证已经配好，安装clusterware的时候也会出现错误：

The specified nodes are not clusterable

因为，配好ssh后，还需要在第一次访问时输入yes，才算是真正的无障碍访问其他服务器。

需要注意的是生成密钥时不设置密码，授权文件权限为600，同时需要两个节点互相ssh通过一次。

十一.配置裸盘

使用asm管理存储需要裸盘，前面配置了共享硬盘到两台主机上。配置裸盘的方式有两种（1）oracleasm添加

（2）/etc/udev/rules.d/60-raw.rules配置文件添加(字符方式帮绑定udev)

采用第二种

fdisk /dev/sdb

Command (m forhelp): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

…….

这里我分了三个区：OCR(投票盘),DATA（数据文件））,FAR（快速恢复区）

最后 w 命令保存更改

partx -a /dev/sdb

分别在两个节点创建裸设备映射（两个节点都要编辑）（两个节点都要编辑）

[root@rac1 rules.d]# vi /etc/udev/rules.d/60-raw.rules

# Enter raw device bindings here.

#

# An example would be:

# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"

# to bind /dev/raw/raw1 to /dev/sda, or

# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"

# to bind /dev/raw/raw2 to the device with major 8, minor 1.

ACTION=="add", KERNEL=="sdb1", RUN+="/bin/raw /dev/raw/raw1 %N"

ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw2 %N"

ACTION=="add", KERNEL=="sdb3", RUN+="/bin/raw /dev/raw/raw3 %N"

KERNEL=="raw[1-3]",OWNER="grid",GROUP="asmadmin" MODE="660"

启动并查看裸盘

[root@rac1 ~]# start_udev

Starting udev: [ OK ]

[root@rac1 ~]# ll /dev/raw/

total 0

crw-rw---- 1 grid asmadmin 162, 1 Apr 13 13:51 raw1

crw-rw---- 1 grid asmadmin 162, 2 Apr 13 13:51 raw2

crw-rw---- 1 grid asmadmin 162, 3 Apr 13 13:51 raw3

crw-rw---- 1 root disk 162, 0 Apr 13 13:51 rawctl

[root@rac1 rules.d]# raw -qa

/dev/raw/raw1: bound to major 8, minor 17

/dev/raw/raw2: bound to major 8, minor 18

/dev/raw/raw3: bound to major 8, minor 19

查看裸设备的主从设备号

[root@rac1 rules.d]# ls -l /dev/raw/

total 0

crw-rw---- 1 grid asmadmin 162, 1 Aug 5 12:44 raw1

crw-rw---- 1 grid asmadmin 162, 2 Aug 5 12:44 raw2

crw-rw---- 1 grid asmadmin 162, 3 Aug 5 12:44 raw3

节点2强制刷新

[root@rac2 rules.d]# partprobe

[root@rac2 ~]# start_udev

Starting udev: [ OK ]

[root@rac1 rules.d]# raw -qa

/dev/raw/raw1: bound to major 8, minor 17

/dev/raw/raw2: bound to major 8, minor 18

/dev/raw/raw3: bound to major 8, minor 19

查看裸设备的主从设备号

[root@rac1 rules.d]# ls -l /dev/raw/

total 0

crw-rw---- 1 grid asmadmin 162, 1 Aug 5 12:44 raw1

crw-rw---- 1 grid asmadmin 162, 2 Aug 5 12:44 raw2

crw-rw---- 1 grid asmadmin 162, 3 Aug 5 12:44 raw3

十二.grid安装

rac1到grid软件目录下执行runcluvfy.sh命令开始预安装检查：

在grid下解压grid，执行预安装检查。

[grid@rac1 ~]$ cd grid/

[grid@rac1 grid]$ ls

install readme.html response rpm runcluvfy.sh runInstaller sshsetup stage welcome.html

[grid@rac1 grid]$ pwd

/opt/grid/grid

[grid@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose

查看cvu报告，修正错误,各项都检查通过即可开始安装grid

登陆到图形化界面

[root@rac1 ~]#xhost +

[root@rac1 ~]#su – grid

[grid@rac1 ~]$cd grid

[grid@rac1 grid]$./runInstaller

已经手动做过互信，这里可以跳过，直接下一步

在两个节点上分别执行这两个脚本

执行脚本时注意顺序，一定是先执行rac1，然后去rac2上执行

执行完之后点击ok

rac1:

[root@rac1 ~]# cd /u01/app/oraInventory/

[root@rac1 oraInventory]# ./orainstRoot.sh

Changing permissions of /u01/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.

The execution of the script is complete.

[root@rac1 oraInventory]# clear

[root@rac1 oraInventory]# cd /u01/app/11.2.0/grid/

[root@rac1 grid]# ./root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to /usr/local/bin ...

Copying oraenv to /usr/local/bin ...

Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

Installing Trace File Analyzer

OLR initialization - successful

root wallet

root wallet cert

root cert export

peer wallet

profile reader wallet

pa wallet

peer wallet keys

pa wallet keys

peer cert request

pa cert request

peer cert

pa cert

peer root cert TP

profile reader root cert TP

pa root cert TP

peer pa cert TP

pa peer cert TP

profile reader pa cert TP

profile reader peer cert TP

peer user cert

pa user cert

Adding Clusterware entries to upstart

CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'

CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'

CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'

CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'

CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded

CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'rac1'

CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'

CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded

CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded

ASM created and started successfully.

Disk Group OCR created successfully.

clscfg: -install mode specified

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

CRS-4256: Updating the profile

Successful addition of voting disk 9ee58ddd21094f61bf43065b4875e9a4.

Successfully replaced voting disk group with +OCR.

CRS-4256: Updating the profile

CRS-4266: Voting file(s) successfully replaced

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE 9ee58ddd21094f61bf43065b4875e9a4 (/dev/raw/raw1) [OCR]

Located 1 voting disk(s).

CRS-2672: Attempting to start 'ora.asm' on 'rac1'

CRS-2676: Start of 'ora.asm' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.OCR.dg' on 'rac1'

CRS-2676: Start of 'ora.OCR.dg' on 'rac1' succeeded

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

rac2

[root@rac2 CVU_11.2.0.4.0_grid]# cd /u01/app/oraInventory/

ContentsXML/ logs/ oraInst.loc orainstRoot.sh

[root@rac2 CVU_11.2.0.4.0_grid]# cd /u01/app/oraInventory/

[root@rac2 oraInventory]# ./orainstRoot.sh

Changing permissions of /u01/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.

The execution of the script is complete.

[root@rac2 oraInventory]# cd ../11.2.0/grid/

[root@rac2 grid]# ./r

racg/ rdbms/ relnotes/ root.sh rootupgrade.sh

[root@rac2 grid]# ./root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to /usr/local/bin ...

Copying oraenv to /usr/local/bin ...

Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

Installing Trace File Analyzer

OLR initialization - successful

Adding Clusterware entries to upstart

CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac1, number 1, and is terminating

An active cluster was found during exclusive startup, restarting to join the cluster

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

点击ok继续安装

十三.asm磁盘组创建

[root@rac1 ~]#su – grid

[grid@rac1 ~]$asmca

十四.安装oracle software

[rooot@rac1 ~]#xhost +

[root@rac1 ~]#su – oracle

[oracle@rac1 ~]$cd database

[oracle@rac1 ~]$./runInstaller

rac1

[root@rac1 ~]# cd /u01/app/oracle/product/11.2.0/db_1/

[root@rac1 db_1]# ./root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

[root@rac1 db_1]#

rac2

[root@rac2 grid]# cd /u01/app/oracle/product/11.2.0/db_1/

[root@rac2 db_1]# ./root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

[root@rac2 db_1]#

十五.DBCA创建实例

在oracle下解压安装包

登陆到图形化界面

[root@rac1 ~]#xhost +

[root@rac1 ~]#su – oracle

[oracle@rac1 ~]$ dbca

安装完成。