楼主: oldunix

原创:构建稳定的Veritas Storage Foundation for Oracle RAC系统(By Oldunix)

[复制链接]
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
11#
 楼主| 发表于 2009-2-15 10:38 | 只看该作者
完成San Foundation for Oracle RAC的安装

在db1和db2上
installp -c -g -X all

shutdown -r

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
12#
 楼主| 发表于 2009-2-15 10:39 | 只看该作者
配置集群
cd  /opt/zqf/dvd1-aix/sfrac
./installsfrac  -rsh -configure


     Veritas Storage Foundation for Oracle RAC 5.0MP3 Configuration Program     

Copyright (c) 2008 Symantec Corporation. All rights reserved.  Symantec, the
Symantec Logo are trademarks or registered trademarks of Symantec
Corporation or its affiliates in the U.S. and other countries. Other names
may be trademarks of their respective owners.

The Licensed Software and Documentation are deemed to be "commercial
computer software" and "commercial computer software documentation" as
defined in FAR Sections 12.212 and DFARS Section 227.7202.

Logs for installsfrac are being created in /var/tmp/installsfrac-fGkmwn.

Enter the system names separated by spaces on which to configure SFRAC: db1 db2


Checking system licensing

Demo SFRAC license registered on db1

Do you want to enter another license key for db1? [y,n,q] (n)

Demo SFRAC license registered on db2

Do you want to enter another license key for db2? [y,n,q] (n)


The SFRAC installer will help you perform the following tasks:
    . ................................................................... .
        * Complete the system checks
        * Configure the SFRAC stack
        * Complete the steps required before an Oracle installation
        * Launch the Oracle Installer from within this SFRAC installer
    . ................................................................... .



Press [Return] to continue:


Main Menu
    . ................................................................... .
        * Keep root shells open on all systems to run ** INSTRUCTIONS ONLY **
                 steps
        * Follow the steps in order
        * Navigation keys for all screens: q to quit, b for previous menu
    . ................................................................... .


     1)  Check systems for SFRAC ** INSTRUCTIONS ONLY **
     2)  Configure SFRAC
     3)  Prepare to install Oracle
     4)  Install or relink Oracle

Select an option [1-4,q,?] 2

Main Menu > Configure SFRAC
    . ................................................................... .
This step will help you configure components of SFRAC stack
    . ................................................................... .


     1)  Configure VCS, CVM and CFS
     2)  Configure I/O fencing ** INSTRUCTIONS ONLY **
     b)  [Go to previous menu]

Select an option [1-2,b,q]1


Main Menu > Configure SFRAC > Configure VCS, CVM and CFS
    . ................................................................... .
After completing VCS, CVM and CFS configuration, the installer will exit.
You will need to restart the SFRAC installer with -configure option to
access the Main Menu.
    . ................................................................... .


Continue with configuration? [y,n,q,b] (n)y


Stopping SFRAC processes. Please wait...
SFRAC processes have been stopped.

Press [Return] to continue:

To configure SFRAC, please answer the sets of questions on the next screen.

When is presented after a question, 'b' may be entered to go back to the
first question of the configuration set.

When [?] is presented after a question, '?' may be entered for help or
additional information about the question.

Following each set of questions, the information you have entered will be
presented for confirmation.  To repeat the set of questions and correct any
previous errors, enter 'n' at the confirmation prompt.

No configuration changes are made to the systems until all configuration
questions are completed and confirmed.

Press [Return] to continue:

To configure VCS for SFRAC the following information is required:

        A unique Cluster name
        A unique Cluster ID number between 0-65535
        Two or more NIC cards per system used for heartbeat links

All systems are being configured to create one cluster

Enter the unique cluster name: [?] tvrz
Enter the unique Cluster ID number between 0-65535: [b,?] 10

    Discovering NICs on db1 ............ discovered en0 en1 en2 en3 en4 en5

To use aggregated interfaces for private heartbeat, enter the name of the
aggregated interface.
To use a NIC for private heartbeat, enter a NIC which is not part of an
aggregated interface.


Enter the NIC for the first private heartbeat link on db1: [b,?] en2
Would you like to configure a second private heartbeat link? [y,n,q,b,?] (y)
Enter the NIC for the second private heartbeat link on db1: [b,?] en4
Would you like to configure a third private heartbeat link? [y,n,q,b,?] (n)
Are you using the same NICs for private heartbeat links on all systems?
[y,n,q,b,?] (y)




Cluster information verification:

        Cluster Name: tvrz
        Cluster ID Number: 10
        Private Heartbeat NICs for db1: link1=en2 link2=en4
        Private Heartbeat NICs for db2: link1=en2 link2=en4

Is this information correct? [y,n,q] (y)

Veritas Storage Foundation for Oracle RAC can be configured to utilize
Symantec Security Services.

Running SFRAC in Secure Mode guarantees that all inter-system communication
is encrypted and that users are verified with security credentials.

When running SFRAC in Secure Mode, NIS and system usernames and passwords
are used to verify identity.  SFRAC usernames and passwords are no longer
utilized when a cluster is running in Secure Mode.

Before configuring a cluster to operate using Symantec Security Services,
another system must already have Symantec Security Services installed and be
operating as a Root Broker.  Refer to the Veritas Cluster Server
Installation Guide for more information on configuring a Symantec Product
Authentication Service Root Broker.

Would you like to configure SFRAC to use Symantec Security Services?
[y,n,q] (n)

The following information is required to add SFRAC users:

        A user name
        A password for the user
        User privileges (Administrator, Operator, or Guest)

Do you want to set the username and/or password for the Admin user

(default username = 'admin', password='password')? [y,n,q] (n)
Do you want to add another user to the cluster? [y,n,q] (y) n
SFRAC User verification:

        User: admin     Privilege: Administrators

        Passwords are not displayed

Is this information correct? [y,n,q] (y)

The following information is required to configure SMTP notification:

        The domain-based hostname of the SMTP server
        The email address of each SMTP recipient
        A minimum severity level of messages to send to each recipient


Do you want to configure SMTP notification? [y,n,q,?] (y) n
The following information is required to configure SNMP notification:

        System names of SNMP consoles to receive VCS trap messages
        SNMP trap daemon port numbers for each console
        A minimum severity level of messages to send to each console


Do you want to configure SNMP notification? [y,n,q,?] (y) n
The following is required to configure the Global Cluster Option:

        A public NIC used by each system in the cluster
        A Virtual IP address and netmask



Do you want to configure the Global Cluster Option? [y,n,q,?] (y)

Do you want to configure the Global Cluster Option? [y,n,q,?] (y)

Active NIC devices discovered on db1: en2 en3 en4
Enter the NIC for Global Cluster Option to use on db1: [b,?] (en2) en3

Is en3 to be the public NIC used by all systems? [y,n,q,b,?] (y)

Enter the Virtual IP address for the Global Cluster Option: [b,?] 192.168.3.185

Enter the netmask for IP 192.168.3.185: [b,?] (255.255.255.0) 255.255.255.0

Global Cluster Option configuration verification:

        NIC: en3
        IP: 192.168.3.185
        Netmask: 255.255.255.0

Is this information correct? [y,n,q] (y)


The following information is required to setup the permission for Veritas
Storage Foundation for Oracle RAC Utility to support Oracle Database:

The Veritas Storage Foundation for Oracle RAC utility for database
administration requires permission changes to allow database administrators
(DBAs) access to the tools. The default settings at installation time for
the /opt/VRTSdbed directory allows only the root user access to the
directory. You need to change permissions to allow a DBA or a group of DBAs
access to the Veritas Storage Foundation for Oracle RAC tools

Do you want to add single user access on db1? [y,n,q,?] (y)
Enter login account name for DBA user: oracle

Do you want to add group access on db1? [y,n,q,?] (y)
Enter group name for DBA users: dba
Are you using the same DBA user/group for all systems? [y,n,q,?] (y)


Veritas Volume Manager default disk group name configuration:


Many Veritas Volume Manager commands affect the contents or configuration of
a disk group. Such commands require that the user specify a disk group.
This is accomplished by using the -g option of a command or setting the
VXVM_DEFAULTDG environment variable.  An alternative to these two methods
for some commands is to configure the name of the default disk group of a
system.

Do you want to set up a default disk group for each system? [y,n,q,?] (y)

Will you specify one disk group name for all eligible systems? [y,n,q,b,?] (y)

Specify a default disk group name for all systems. [?] default_dg


You have decided to specify the default disk groups as follows:

    Host: db1 ...................................... Disk group: default_dg
    Host: db2 ...................................... Disk group: default_dg

Note: If 'nodg' is displayed, then the host will be configured to have no
default disk group.

Is this correct? [y,n,q] (y)

Verifying the install systems fully qualified hostnames.


    Querying fully qualified domain name of host "db1" ............... fail

Press [Return] to continue:
Unable to find Fully Qualifed Host Name for db1!

Enter the fully qualified hostname of "db1" or 'QUIT' to quit install.
Enter the fully qualified host name: db1.hntv.com

    Querying fully qualified domain name of host "db2" ............... fail

Press [Return] to continue:
Unable to find Fully Qualifed Host Name for db2!

Enter the fully qualified hostname of "db2" or 'QUIT' to quit install.
Enter the fully qualified host name: db2.hntv.com


Choose how this install will be managed.

This product can be configured in one of the following two ways:

        * Veritas Storage Foundation Management Server managed host
                 (Recommended)

        * Standalone host


        Pre-requisites for Veritas Storage Foundation Management Server managed
                 host:
                * A Veritas Storage Foundation Management Server (SFM) is configured
                * SFM should be currently running and fully operational.
                * Authentication Broker that is used by SFM should be running.

        The following information would be requested during configuration:
                * The host name of the SFM
                * Previously set SFM agent account password

        There are no pre-requisites for deploying as a standalone host.

Enabling Veritas Storage Foundation Management Server management simplifies
and improves management of the complex data center resources, reducing
planned and unplanned down time.

Enable Veritas Storage Foundation Management Server Management? [y,n,q] (y)n

Verifying the install systems fully qualified hostnames.

    Configuring VEA in STANDALONE mode on "db2" ........................ ok

    Configuring gridnode on "db2" ...................................... ok
    Registering gridnode on "db2" ...................................... ok
    Configuring actionagent on "db2" ................................... ok
    Registering actionagent on "db2" ................................... ok
    Stopping StorageAgent on "db2" ..................................... ok
    Registering StorageAgent on "db2" .................................. ok
    Registering VAILAgent on "db2" ..................................... ok
Do you want to start Veritas Storage Foundation for Oracle RAC processes
now? [y,n,q] (y)
Logs for installsfrac are temporarily being created in
/var/tmp/installsfrac-fGkmwn.


    Starting SFRAC: 12%       ___________________________________________



    Starting SFRAC: 100%                                                   

Startup completed successfully on all systems


Configuring CFS agents:

    Confirming VxVM configuration daemons startup ..... All systems Enabled
    Starting CFS agents ........................................... Started
    Setting default disk group to default_dg on db1 .................. Done
    Setting default disk group to default_dg on db2 .................. Done
Configuration log files, summary file, and response file are saved at:

        /opt/VRTS/install/logs/installsfrac-fGkmwn

I/O Fencing has currently been configured in disabled mode.  Refer to the
Veritas Storage Foundation for Oracle RAC Administrator's Guide for more
information and for instructions on how to enable I/O Fencing.

See the Veritas Storage Foundation for Oracle RAC Installation and
Configuration Guide for more information.

To continue SFRAC configuration, invoke the installer using -configure flag.

[ 本帖最后由 oldunix 于 2009-2-15 10:53 编辑 ]

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
13#
 楼主| 发表于 2009-2-15 10:40 | 只看该作者
配置IO Fencing


# lsattr -El hdisk1
PR_key_value   none                             Persistant Reserve Key Value           True
cache_method   fast_write                       Write Caching method                   False
ieee_volname   600A0B80005062E8000003A249489756 IEEE Unique volume name                False
lun_id         0x0000000000000000               Logical Unit Number                    False
max_transfer   0x100000                         Maximum TRANSFER Size                  True
prefetch_mult  1                                Multiple of blocks to prefetch on read False
pvid           00c0b2825dfcbe920000000000000000 Physical volume identifier             False
q_type         simple                           Queuing Type                           False
queue_depth    10                               Queue Depth                            True
raid_level     1                                RAID Level                             False
reassign_to    120                              Reassign Timeout value                 True
reserve_policy single_path                      Reserve Policy                         True
rw_timeout     30                               Read/Write Timeout value               True
scsi_id        0x10400                          SCSI ID                                False
size           500                              Size in Mbytes                         False
write_cache    yes                              Write Caching enabled                  False
# lsattr -El hdisk8
PR_key_value   none                             Persistant Reserve Key Value           True
cache_method   fast_write                       Write Caching method                   False
ieee_volname   600A0B800048C6EC000002D449489D4A IEEE Unique volume name                False
lun_id         0x0000000000000000               Logical Unit Number                    False
max_transfer   0x100000                         Maximum TRANSFER Size                  True
prefetch_mult  1                                Multiple of blocks to prefetch on read False
pvid           none                             Physical volume identifier             False
q_type         simple                           Queuing Type                           False
queue_depth    10                               Queue Depth                            True
raid_level     1                                RAID Level                             False
reassign_to    120                              Reassign Timeout value                 True
reserve_policy single_path                      Reserve Policy                         True
rw_timeout     30                               Read/Write Timeout value               True
scsi_id        0x10500                          SCSI ID                                False
size           500                              Size in Mbytes                         False
write_cache    yes                              Write Caching enabled                  False
# vxdisk list ds4700-0_0
Device:    ds4700-0_0
devicetag: ds4700-0_0
type:      auto
info:      format=none
flags:     online ready private autoconfig invalid
pubpaths:  block=/dev/vx/dmp/ds4700-0_0 char=/dev/vx/rdmp/ds4700-0_0
guid:      -
udid:      IBM%5F1814%5F600A0B800048C6EC00000000492B47A6%5F600A0B800048C6EC000002D449489D4A
site:      -
Multipathing information:
numpaths:   1
hdisk8  state=enabled
# vxdisk list ds4700-1_0
Device:    ds4700-1_0
devicetag: ds4700-1_0
type:      auto
info:      format=none
flags:     online ready private autoconfig invalid
pubpaths:  block=/dev/vx/dmp/ds4700-1_0 char=/dev/vx/rdmp/ds4700-1_0
guid:      -
udid:      IBM%5F1814%5F600A0B80005062E800000000492B4A74%5F600A0B80005062E8000003A249489756
site:      -
Multipathing information:
numpaths:   1
hdisk1  state=enabled


# vxdisk list ds4700-1_1
Device:    ds4700-1_1
devicetag: ds4700-1_1
type:      auto
info:      format=none
flags:     online ready private autoconfig invalid
pubpaths:  block=/dev/vx/dmp/ds4700-1_1 char=/dev/vx/rdmp/ds4700-1_1
guid:      -
udid:      IBM%5F1814%5F600A0B80005062E800000000492B4A74%5F600A0B80005062E8000003A6494897B4
site:      -
Multipathing information:
numpaths:   1
hdisk2  state=enabled
# lsattr -El hdisk2
PR_key_value   none                             Persistant Reserve Key Value           True
cache_method   fast_write                       Write Caching method                   False
ieee_volname   600A0B80005062E8000003A6494897B4 IEEE Unique volume name                False
lun_id         0x0001000000000000               Logical Unit Number                    False
max_transfer   0x100000                         Maximum TRANSFER Size                  True
prefetch_mult  1                                Multiple of blocks to prefetch on read False
pvid           00c0b2825dfcbf890000000000000000 Physical volume identifier             False
q_type         simple                           Queuing Type                           False
queue_depth    10                               Queue Depth                            True
raid_level     1                                RAID Level                             False
reassign_to    120                              Reassign Timeout value                 True
reserve_policy single_path                      Reserve Policy                         True
rw_timeout     30                               Read/Write Timeout value               True
scsi_id        0x10400                          SCSI ID                                False
size           550                              Size in Mbytes                         False
write_cache    yes                              Write Caching enabled                  False
# vxdisk list ds4700-0_1
Device:    ds4700-0_1
devicetag: ds4700-0_1
type:      auto
info:      format=none
flags:     online ready private autoconfig invalid
pubpaths:  block=/dev/vx/dmp/ds4700-0_1 char=/dev/vx/rdmp/ds4700-0_1
guid:      -
udid:      IBM%5F1814%5F600A0B800048C6EC00000000492B47A6%5F600A0B800048C6EC000002D649489D78
site:      -
Multipathing information:
numpaths:   1
hdisk9  state=enabled
# lsattr -El hdisk9
PR_key_value   none                             Persistant Reserve Key Value           True
cache_method   fast_write                       Write Caching method                   False
ieee_volname   600A0B800048C6EC000002D649489D78 IEEE Unique volume name                False
lun_id         0x0001000000000000               Logical Unit Number                    False
max_transfer   0x100000                         Maximum TRANSFER Size                  True
prefetch_mult  1                                Multiple of blocks to prefetch on read False
pvid           none                             Physical volume identifier             False
q_type         simple                           Queuing Type                           False
queue_depth    10                               Queue Depth                            True
raid_level     1                                RAID Level                             False
reassign_to    120                              Reassign Timeout value                 True
reserve_policy single_path                      Reserve Policy                         True
rw_timeout     30                               Read/Write Timeout value               True
scsi_id        0x10500                          SCSI ID                                False
size           550                              Size in Mbytes                         False
write_cache    yes                              Write Caching enabled                  False


# vxddladm listsupport all
LIBNAME             VID                 PID
=================================================================================================
libvxemc.so         EMC                 SYMMETRIX
libvxibmds4k.so     IBM                 1722, 1724, 3552, 3542, 1742-900, 1742, 3526, 1815, 1814
libvxshark.so       IBM                 2105
libvxibmds6k.so     IBM                 1750
libvxhitachi.so     HITACHI             DF350, DF400, DF400F, DF500, DF500F
libvxxp256.so       HP                  All
libvxvpath.so       IBM                 2105
libvxhds.so         HITACHI             All
libvxpurple.so      SUN                 T300
libvxvscsi.so       AIX                 VDASD
libvxibmds8k.so     IBM                 2107
libvxcscovrts.so    CSCOVRTS            MDS9
libvxhdsalua.so     HITACHI             DF600, DF600-V, DF600F, DF600F-V
libvxhds9980.so     HITACHI             All
libvxhdsusp.so      HITACHI             All
libvxCLARiiON.so    DGC                 All
libvx3par.so        3PARdata            VV
libvxhpalua.so      HP, COMPAQ          HSV101, HSV111 (C)COMPAQ, HSV111, HSV200, HSV210
libvxibmsvc.so      IBM                 2145, 2062
libvxxp12k.so       HP                  All
libvxpp.so          EMC, DGC            All



# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:LVM        -            -            LVM
ds4700-0_0   auto:none       -            -            online invalid
ds4700-0_1   auto:none       -            -            online invalid
ds4700-0_2   auto:none       -            -            online invalid
ds4700-0_3   auto:none       -            -            online invalid
ds4700-0_4   auto:none       -            -            online invalid
ds4700-0_5   auto:none       -            -            online invalid
ds4700-0_6   auto:none       -            -            online invalid
ds4700-1_0   auto:none       -            -            online invalid
ds4700-1_1   auto:none       -            -            online invalid
ds4700-1_2   auto:none       -            -            online invalid
ds4700-1_3   auto:none       -            -            online invalid
ds4700-1_4   auto:none       -            -            online invalid
ds4700-1_5   auto:none       -            -            online invalid
ds4700-1_6   auto:none       -            -            online invalid

初始化VxVM磁盘
vxdisksetup -i ds4700-0_0
vxdisksetup -i ds4700-0_1
vxdisksetup -i ds4700-0_2
vxdisksetup -i ds4700-0_3
vxdisksetup -i ds4700-0_4
vxdisksetup -i ds4700-0_5
vxdisksetup -i ds4700-0_6
vxdisksetup -i ds4700-1_0
vxdisksetup -i ds4700-1_1
vxdisksetup -i ds4700-1_2
vxdisksetup -i ds4700-1_3
vxdisksetup -i ds4700-1_4
vxdisksetup -i ds4700-1_5
vxdisksetup -i ds4700-1_6


# vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
disk_0       auto:LVM        -            -            LVM
ds4700-0_0   auto:cdsdisk    -            -            online
ds4700-0_1   auto:cdsdisk    -            -            online
ds4700-0_2   auto:cdsdisk    -            -            online
ds4700-0_3   auto:cdsdisk    -            -            online
ds4700-0_4   auto:cdsdisk    -            -            online
ds4700-0_5   auto:cdsdisk    -            -            online
ds4700-0_6   auto:cdsdisk    -            -            online
ds4700-1_0   auto:cdsdisk    -            -            online
ds4700-1_1   auto:cdsdisk    -            -            online
ds4700-1_2   auto:cdsdisk    -            -            online
ds4700-1_3   auto:cdsdisk    -            -            online
ds4700-1_4   auto:cdsdisk    -            -            online
ds4700-1_5   auto:cdsdisk    -            -            online
ds4700-1_6   auto:cdsdisk    -            -            online
#

验证是否是同一块磁盘

/dev/rhdisk1
/dev/rhdisk8
/dev/rhdisk2
/dev/rhdisk9

hdisk1
/sbin/vxfenadm -i /dev/rhdisk1

在db1:
# hostname
db1
# /sbin/vxfenadm -i /dev/rhdisk1
Vendor id       : IBM     
Product id      : 1814      FAStT
Revision        : 0916
Serial Number   : 600A0B80005062E8000003A249489756

在db2:

# hostname
db2
# /sbin/vxfenadm -i /dev/rhdisk1
Vendor id       : IBM     
Product id      : 1814      FAStT
Revision        : 0916
Serial Number   : 600A0B80005062E8000003A249489756


使用 vxfentsthdw测试硬盘是否支持I/O Fensing

/opt/VRTSvcs/vxfen/bin/vxfentsthdw -n
(所有的盘都测试)

# /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n

VERITAS vxfentsthdw version 5.0MP3 AIX


The utility vxfentsthdw works on the two nodes of the cluster.
The utility verifies that the shared storage one intends to use is
configured to support I/O fencing.  It issues a series of vxfenadm
commands to setup SCSI-3 registrations on the disk, verifies the
registrations on the disk, and removes the registrations from the disk.


******** WARNING!!!!!!!! ********

THIS UTILITY WILL DESTROY THE DATA ON THE DISK!!

Do you still want to continue : [y/n] (default: n) y
The logfile generated for vxfensthdw is /var/VRTSvcs/log/vxfen/vxfentsthdw.log.417876

Enter the first node of the cluster:
db1
Enter the second node of the cluster:
db2

Enter the disk name to be checked for SCSI-3 PGR on node db1 in the format:
        for dmp: /dev/vx/rdmp/DiskXX
        for raw: /dev/rhdiskXX
Make sure its the same disk as seen by nodes db1 and db2
/dev/rhdisk1

Enter the disk name to be checked for SCSI-3 PGR on node db2 in the format:
        for dmp: /dev/vx/rdmp/DiskXX
        for raw: /dev/rhdiskXX
Make sure its the same disk as seen by nodes db1 and db2
/dev/rhdisk1

***************************************************************************

Testing db1 /dev/rhdisk1 db2 /dev/rhdisk1

Evaluate the disk before testing  ........................ No Pre-existing keys
Register keys on disk /dev/rhdisk1 from node db1 ....................... Passed
Verify registrations for disk /dev/rhdisk1 on node db1 ................. Passed
Read from disk /dev/rhdisk1 on node db1 ................................ Passed
Write to disk /dev/rhdisk1 from node db1 ............................... Passed
Read from disk /dev/rhdisk1 on node db2 ................................ Passed
Write to disk /dev/rhdisk1 from node db2 ............................... Passed
Reserve disk /dev/rhdisk1 from node db1 ................................ Passed
Verify reservation for disk /dev/rhdisk1 on node db1 ................... Passed
Read from disk /dev/rhdisk1 on node db1 ................................ Passed
Read from disk /dev/rhdisk1 on node db2 ................................ Passed
Write to disk /dev/rhdisk1 from node db1 ............................... Passed
Expect no writes for disk /dev/rhdisk1 on node db2 ..................... Passed
RegisterIgnoreKeys on disk /dev/rhdisk1 from node db2 .................. Passed
Verify registrations for disk /dev/rhdisk1 on node db1 ................. Passed
Verify registrations for disk /dev/rhdisk1 on node db2 ................. Passed
Write to disk /dev/rhdisk1 from node db1 ............................... Passed
Write to disk /dev/rhdisk1 from node db2 ............................... Passed
Preempt and abort key KeyA using key KeyB on node db2 .................. Passed
Test to see if I/O on node db1 terminated .............................. Passed
RegisterIgnoreKeys on disk /dev/rhdisk1 from node db1 .................. Passed
Verify registrations for disk /dev/rhdisk1 on node db1 ................. Passed
Preempt key KeyC using key KeyB on node db2 ............................ Passed
Test to see if I/O on node db1 terminated .............................. Passed
Verify registrations for disk /dev/rhdisk1 on node db1 ................. Passed
Verify registrations for disk /dev/rhdisk1 on node db2 ................. Passed
Verify reservation for disk /dev/rhdisk1 on node db1 ................... Passed
Verify reservation for disk /dev/rhdisk1 on node db2 ................... Passed
Remove key KeyB on node db2 ............................................ Passed
Check to verify there are no keys from node db1 ........................ Passed
Check to verify there are no keys from node db2 ........................ Passed
RegisterIgnoreKeys on disk /dev/rhdisk1 from node db1 .................. Passed
Verify registrations for disk /dev/rhdisk1 on node db1 ................. Passed
RegisterIgnoreKeys on disk /dev/rhdisk1 from node db1 .................. Passed
Verify registrations for disk /dev/rhdisk1 on node db1 ................. Passed
Clear PGR on node db1 .................................................. Passed
Check to verify there are no keys from node db1 ........................ Passed

ALL tests on the disk /dev/rhdisk1 have PASSED.
The disk is now ready to be configured for I/O Fencing on node db1.

ALL tests on the disk /dev/rhdisk1 have PASSED.
The disk is now ready to be configured for I/O Fencing on node db2.

Removing test keys and temporary files, if any...
#

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
14#
 楼主| 发表于 2009-2-15 10:42 | 只看该作者
I/O Fensing(继续)

在db1:

vxdg init vxfencoorddg  ds4700-1_0 ds4700-1_1 ds4700-1_2


vxdg -g vxfencoorddg set coordinator=on

vxdg deport vxfencoorddg


vxdg -t import vxfencoorddg
vxdg deport vxfencoorddg

在db2上:


vxdg -t import vxfencoorddg
vxdg deport vxfencoorddg


在db1,db2上:


echo "vxfencoorddg" > /etc/vxfendg

# hostname
db1
# echo "vxfencoorddg" > /etc/vxfendg
# hostname
db2
# echo "vxfencoorddg" > /etc/vxfendg
#

在db1,db2上:

cp /etc/vxfen.d/vxfenmode_scsi3_dmp /etc/vxfenmode


===
注:
■ For DMP configuration:
# cp /etc/vxfen.d/vxfenmode_scsi3_dmp /etc/vxfenmode
■ For raw device configuration:
# cp /etc/vxfen.d/vxfenmode_scsi3_raw /etc/vxfenmode
■ NO IOFENSING:

/etc/vxfen.d
cp /etc/vxfen.d/vxfenmode_disabled /etc/vxfenmode

===

修改集群配置文件,让集群使用I/O fencing

在db1,db2上:

haconf -dump -makero

hastop -all

/etc/init.d/vxfen.rc stop

cd /etc/VRTSvcs/conf/config
cp main.cf main.orig


在db1上:增加标红的行

vi main.cf


# cat main.cf
include "OracleASMTypes.cf"
include "types.cf"
include "CFSTypes.cf"
include "CVMTypes.cf"
include "MultiPrivNIC.cf"
include "OracleTypes.cf"
include "PrivNIC.cf"

cluster tvrz (
        UserNames = { admin = hKLdKFkHLgLLjTLfKI }
        ClusterAddress = "192.168.3.185"
        Administrators = { admin }
        HacliUserLevel = COMMANDROOT
        UseFence = SCSI3
)

system db1 (
        )

system db2 (
        )

group ClusterService (
        SystemList = { db1 = 0, db2 = 1 }
        AutoStartList = { db1, db2 }
        OnlineRetryLimit = 3
        OnlineRetryInterval = 120
        )

        Application wac (
                StartProgram = "/opt/VRTSvcs/bin/wacstart"
                StopProgram = "/opt/VRTSvcs/bin/wacstop"
                MonitorProcesses = { "/opt/VRTSvcs/bin/wac" }
                RestartLimit = 3
                )

        IP gcoip (
                Device = en3
                Address = "192.168.3.185"
                NetMask = "255.255.255.0"
                )

        NIC gconic (
                Device = en3
                )

        gcoip requires gconic
        wac requires gcoip


        // resource dependency tree
        //
        //      group ClusterService
        //      {
        //      Application wac
        //          {
        //          IP gcoip
        //              {
        //              NIC gconic
        //              }
        //          }
        //      }


group cvm (
        SystemList = { db1 = 0, db2 = 1 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { db1, db2 }
        )

        CFSfsckd vxfsckd (
                )

        CVMCluster cvm_clus (
                CVMClustName = hntvrz
                CVMNodeId = { db1 = 0, db2 = 1 }
                CVMTransport = gab
                CVMTimeout = 200
                )

        CVMVxconfigd cvm_vxconfigd (
                Critical = 0
                CVMVxconfigdArgs = { syslog }
                )

        cvm_clus requires cvm_vxconfigd
        vxfsckd requires cvm_clus


        // resource dependency tree
        //
        //      group cvm
        //      {
        //      CFSfsckd vxfsckd
        //          {
        //          CVMCluster cvm_clus
        //              {
        //              CVMVxconfigd cvm_vxconfigd
        //              }
        //          }
        //      }

hacf -verify /etc/VRTSvcs/conf/config

rcp /etc/VRTSvcs/conf/config/main.cf  db2:/etc/VRTSvcs/conf/config

[ 本帖最后由 oldunix 于 2009-2-15 10:54 编辑 ]

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
15#
 楼主| 发表于 2009-2-15 10:43 | 只看该作者
Starting SF Oracle RAC on all nodes

Starting I/O fencing, VCS, CVM, and CFS

To start VCS, CVM, and CFS on a node
1 Start the I/O fencing driver. Run the following command on each node:
# /etc/init.d/vxfen.rc start
The vxfen startup script also invokes the vxfenconfig command, which
configures the vxfen driver to start and use the coordinator disks that are
listed in /etc/vxfentab.
2 With the configuration file in place on each system, start VCS, CVM, and CFS:
# hastart



shutdown -Fr

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
16#
 楼主| 发表于 2009-2-15 10:44 | 只看该作者
准备安装Oracle RAC

cd  /opt/zqf/dvd1-aix/sfrac
./installsfrac  -rsh -configure
                   Veritas Storage Foundation for Oracle RAC 5.0MP3 Configuration Program                  

Copyright (c) 2008 Symantec Corporation. All rights reserved.  Symantec, the Symantec Logo are
trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other
countries. Other names may be trademarks of their respective owners.

The Licensed Software and Documentation are deemed to be "commercial computer software" and "commercial
computer software documentation" as defined in FAR Sections 12.212 and DFARS Section 227.7202.

Logs for installsfrac are being created in /var/tmp/installsfrac-gnQVPM.

SFRAC configuration files exist on this system with the following information:

        Cluster Name: hntvrz
        Cluster ID Number: 10
        Systems: db1 db2
        Service Groups: ClusterService cvm

Do you want to configure SFRAC from these systems? [y,n,q] (y)


Initial system check:

    Checking rsh communication with db2 ....................................................... AIX 5.3
    Checking SFRAC installation on db1 ........................................................ 5.0.3.0
    Checking SFRAC installation on db2 ........................................................ 5.0.3.0


                   Veritas Storage Foundation for Oracle RAC 5.0MP3 Configuration Program                  

Checking system licensing

Demo SFRAC license registered on db1

Do you want to enter another license key for db1? [y,n,q] (n)
Demo SFRAC license registered on db2

Do you want to enter another license key for db2? [y,n,q] (n)

The SFRAC installer will help you perform the following tasks:
    . ............................................................................................... .
        * Complete the system checks
        * Configure the SFRAC stack
        * Complete the steps required before an Oracle installation
        * Launch the Oracle Installer from within this SFRAC installer
    . ............................................................................................... .



Press [Return] to continue:

Main Menu
    . ............................................................................................... .
        * Keep root shells open on all systems to run ** INSTRUCTIONS ONLY ** steps
        * Follow the steps in order
        * Navigation keys for all screens: q to quit, b for previous menu
    . ............................................................................................... .


     1)  Check systems for SFRAC ** INSTRUCTIONS ONLY **
     2)  Configure SFRAC
     3)  Prepare to install Oracle
     4)  Install or relink Oracle

Select an option [1-4,q,?] 3

ain Menu > Prepare to install Oracle
    . ............................................................................................... .
        This step helps you complete system administration tasks before installing the Oracle software.
    . ............................................................................................... .


     1)  [Perform all the following tasks]
     2)  Create userid and groupid for Oracle
     3)  Configure private IP addresses for CRS (Oracle 10g only)
     4)  Configure CRS and UDP private IP addresses for failover (Oracle 11g)
     5)  Create disk groups, volumes and file systems for Oracle ** INSTRUCTIONS ONLY **
     b)  [Go to previous menu]

Select an option [1-5,b,q]

Main Menu > Prepare to install Oracle > Create disk groups, volumes and file systems for Oracle **
INSTRUCTIONS ONLY **
    . ............................................................................................... .
        Before continuing, login to all cluster nodes and complete the following.
        1. Create CVM volumes or CFS directories on shared storage for:
                * Database storage
                * OCR and Vote-device (for 10g and above)
                (can be CVM Volume or CFS file)
        2. Create local or shared file systems for CRS Home and Oracle Home.
        3. Change the permission of all of the above to oracle user and group.
        4. Create VCS resources for CVM Volumes and CFS files. Refer to sample
        configurations in /etc/VRTSvcs/conf/sample_rac/ directory.

        Example for Oracle data disk group, volume and mount creation:
                vxdg -s init oradatadg HDS0_30 IBM0_30 EMC0_30 HDS1_30 #to create shared disk group
                vxassist -g oradatadg make oradatavol 6G layout=striped-mirror #to create volume
                mkdir /oradata #to create a mount point
                mkfs -V vxfs  -o largefiles \
                 /dev/vx/rdsk/oradatadg/oradatavol #to create shared file system
                mount -V vxfs -o cluster,largefiles /dev/vx/dsk/oradatadg/oradatavol \
                 /oradata #to mount shared file system
                chown -R oracleinstall /oradata #to change permissions

............................................................................................... .


Press [Return] to continue:
Main Menu > Prepare to install Oracle > Configure private IP addresses for CRS
    . ................................................................................................... .
        This step will configure private IP addresses using PrivNIC resource of VCS.

        * You must use all LLT links for PrivNIC.
        Non-LLT links can not be used
        * Network 1 of all nodes must be physically connected.
        Similarly Networks 2, 3, and so on
        * IP addresses used here must be manually added to /etc/hosts on all nodes.
    . ................................................................................................... .


Enter the private IP for db1: 10.10.10.181
    Checking 10.10.10.181 in /etc/hosts on db1 ..................................................... exists
    Discovering NICs on db1 ............................................ discovered en0 en1 en2 en3 en4 en5
Enter the NIC 1 for private network for db1 (x if done): en2
Enter the NIC 2 for private network for db1 (x if done): en4
Enter the NIC 3 for private network for db1 (x if done): x
Enter the private IP for db2: 10.10.10.182
    Checking 10.10.10.182 in /etc/hosts on db2 ..................................................... exists
    Discovering NICs on db2 ............................................ discovered en0 en1 en2 en3 en4 en5
Enter the NIC 1 for private network for db2 (x if done): (en2)
Enter the NIC 2 for private network for db2 (x if done): (en4)
Enter the NIC 3 for private network for db2 (x if done): x
Enter the netmask for private network: 255.255.255.0

CRS private IP configuration information verification

        System Name: db2
        CRS IP address: 10.10.10.182
        Interfaces: en2 en4

        System Name: db1
        CRS IP address: 10.10.10.181
        Interfaces: en2 en4

        Netmask: 255.255.255.0

Is this information correct? [y,n,q,b] (y)
    Changing configuration to read-write mode ..................................................... success
    Adding new ora_priv resource .................................................................. success
    Saving configuration .......................................................................... success

Press [Return] to continue:

To verify the private NIC address configuration
1 Access and view the VCS main.cf located in the following directory:
cd  /etc/VRTSvcs/conf/config
2 Verify that the PrivNIC resource, ora_priv, shows up in VCS main.cf:
        PrivNIC ora_priv (
                Critical = 0
                Device @db1 = { en2 = 0, en4 = 1 }
                Device @db2 = { en2 = 0, en4 = 1 }
                Address @db1 = "10.10.10.181"
                Address @db2 = "10.10.10.182"
                NetMask = "255.255.255.0"
                )

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
17#
 楼主| 发表于 2009-2-15 10:45 | 只看该作者
安装oracle

vi /home/oracle/.profile

ORACLE_BASE=/opt/app
export ORACLE_BASE

ORACLE_HOME=/opt/app/crs
#ORACLE_HOME=/opt/app/oracle
#ORACLE_HOME=/opt/app/client
export ORACLE_HOME

#ORACLE_SID=oradb1
#export ORACLE_SID
#ORACLE_SID=oradb2
#export ORACLE_SID


MY_CRS=/opt/app/crs
export MY_CRS
MY_ORACLE=/opt/app/oracle
export MY_ORACLE
MY_CLIENT=/opt/app/client
export MY_CLIENT

PATH=$PATH:$MY_CLIENT/bin:$MY_CRS/bin:$MY_ORACLE/bin
export PATH


vi /home/oracle/.rhosts
db1
db2

创建目录:

mkdir -p /opt/app
chown oracleinstall  /opt/app




创建数据卷
IBM DS4700@1   ds4700-0_?
IBM DS4700@2   ds4700-1_?     mirror

ds4700-0_6  数据
ds4700-1_6  数据  mirror

ds4700-0_2  日志  
ds4700-1_2  日志  mirror

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
18#
 楼主| 发表于 2009-2-15 10:46 | 只看该作者
/usr/sbin/vxassist   -g ora-rac-dg -b make raw_db_300g 15g \
layout=concat-mirror,nolog  nmirror=2 alloc=ora-rac-dg01,ora-rac-dg02\
mirror=ctrl  logtype=dco dcoversion=20 drl=yes

vxdg list
NAME         STATE           ID
ora-rac-dg   enabled,shared,cds   1230115987.20.db1

vxedit -g ora-rac-dg set user=oracle group=dba raw_db_300g


mkdir /u01

mkfs -V vxfs -o largefiles /dev/vx/rdsk/ora-rac-dg/raw_db_300g


mount -V vxfs -o cluster /dev/vx/dsk/ora-rac-dg/raw_db_300g /u01

cd /u01
mkdir arch  oradata crs
chown oracleinstall arch  oradata

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
19#
 楼主| 发表于 2009-2-15 10:47 | 只看该作者
安装CRS

cd  /opt/zqf/dvd1-aix/sfrac
./installsfrac  -rsh -configure


配置agent
group cvm (
        SystemList = { db1 = 0, db2 = 1 }
        AutoFailOver = 0
        Parallel = 1
        AutoStartList = { db1, db2 }
        )
        Application cssd-resource (
                Critical = 0
                StartProgram = "/opt/VRTSvcs/rac/bin/cssd-online"
                StopProgram = "/opt/VRTSvcs/rac/bin/cssd-offline"
                CleanProgram = "/opt/VRTSvcs/rac/bin/cssd-clean"
                MonitorProgram = "/opt/VRTSvcs/rac/bin/cssd-monitor"
                OnlineRetryLimit = 20
                )

rcp /etc/VRTSvcs/conf/config/main.cf  db2:/etc/VRTSvcs/conf/config




安装数据库软件(含companioon)

使用道具 举报

回复
论坛徽章:
24
授权会员
日期:2007-03-03 23:55:31ITPUB十周年纪念徽章
日期:2011-11-01 16:21:15ITPUB季度 技术新星
日期:2011-11-18 16:54:31玉石琵琶
日期:2012-02-21 15:04:38ITPUB 11周年纪念徽章
日期:2012-10-09 18:06:202013年新春福章
日期:2013-02-25 14:51:242014年新春福章
日期:2014-02-18 16:42:02马上有房
日期:2014-02-18 16:42:02
20#
 楼主| 发表于 2009-2-15 10:48 | 只看该作者
创建数据库

创建数据库之前重新连接!


                        Veritas Storage Foundation for Oracle RAC 5.0MP3 Configuration Program                        

Oracle environment information verification

        Oracle Unix User: oracle
        Oracle Unix Group: oinstall
        Oracle Clusterware (CRS) Home: /opt/app/crs
        Oracle Release: 10.2
        Oracle Patch Level: 0.4
        Oracle Base: /opt/app
        Oracle Home: /opt/app/oracle

Is this information correct? [y,n,q] (y) y
    Verifying binaries in /opt/app/oracle on db1 .............................................................. ok
    Verifying binaries in /opt/app/oracle on db2 .............................................................. ok
    Copying SFRAC libskgxn on db1 ............................................................................. ok
    Copying SFRAC libskgxn on db2 ............................................................................. ok
    Copying SFRAC ODM library on db1 .......................................................................... ok
    Copying SFRAC ODM library on db2 .......................................................................... ok
    Copying SFRAC libskgxp on db1 ............................................................................. ok
    Copying SFRAC libskgxp on db2 ............................................................................. ok
    Relinking Oracle on db1 ................................................................................... ok
    Relinking Oracle on db2 ................................................................................... ok

Oracle Relinking is now complete.


Press [Return] to continue:

使用道具 举报

回复

您需要登录后才可以回帖 登录 | 注册

本版积分规则 发表回复

TOP技术积分榜 社区积分榜 徽章 团队 统计 知识索引树 积分竞拍 文本模式 帮助
  ITPUB首页 | ITPUB论坛 | 数据库技术 | 企业信息化 | 开发技术 | 微软技术 | 软件工程与项目管理 | IBM技术园地 | 行业纵向讨论 | IT招聘 | IT文档
  ChinaUnix | ChinaUnix博客 | ChinaUnix论坛
CopyRight 1999-2011 itpub.net All Right Reserved. 北京盛拓优讯信息技术有限公司版权所有 联系我们 未成年人举报专区 
京ICP备16024965号-8  北京市公安局海淀分局网监中心备案编号:11010802021510 广播电视节目制作经营许可证:编号(京)字第1149号
  
快速回复 返回顶部 返回列表