Home

Symantec 7 Computer Drive User Manual

1. commercial computer software documentation as defined in FAR Sections 12 212 and DFARS Section 227 7202 Symantec Corporation 20330 Stevens Creek Blvd Cupertino CA 95014 www symantec com Third party legal notices Third party software may be recommended distributed embedded or bundled with this Veritas product Such third party software is licensed separately by its copyright holder All third party copyrights associated with this product are listed in the accompanying release notes HP UX is a registered trademark of Hewlett Packard Development Company L P Licensing and registration Veritas Storage Foundation Cluster File System is a licensed product See the Veritas Storage Foundation Cluster File System Installation Guide for license installation instructions Technical support For technical assistance visit http support veritas com and select phone or email support Use the Knowledge Base search feature to access resources such as TechNotes product alerts software downloads hardware compatibility lists and our customer email notification service Contents Chapter 1 Installing and configuring the product Hardware OV er view senesne ri e EE eee A IE annals 10 Shared Storage cs sccissisccessedsidssesienecessdesiedesseestdseedesschedesdceesscacesscoutsvasaseideasnaaaes 11 Fibre channel SWitch sise scs cscsscceneerscseseersisies avcesstievivneraeeneeeeienviacn 11 Cluster platforms oo ccccccccs
2. Veritas Storage Foundation Cluster File System Installation Guide HP UX 5 0 ntec ae O symantec Veritas Storage Foundation Cluster File System Installation Guide Copyright 2006 Symantec Corporation All rights reserved SFCES 5 0 Symantec the Symantec logo Veritas and Veritas Storage Foundation Cluster File System are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U S and other countries Other names may be trademarks of their respective owners The product described in this document is distributed under licenses restricting its use copying distribution and decompilation reverse engineering No part of this document may be reproduced in any form by any means without prior written authorization of Symantec Corporation and its licensors if any THIS DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID SYMANTEC CORPORATION SHALL NOT BE LIABLE FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING PERFORMANCE OR USE OF THIS DOCUMENTATION THE INFORMATION CONTAINED IN THIS DOCUMENTATION IS SUBJECT TO CHANGE WITHOUT NOTICE The Licensed Software and Documentation are deemed to be commercial computer software and
3. Y N Suggested solutions check that swagentd is running Check whether there is an entry for the target system in etc hosts If there is no entry then ensure the hosts file is not the primary lookup for the hosts entry Storage Foundation Cluster File System problems If there is a device failure or controller failure to a device the file system may become disabled cluster wide To address the problem unmount file system on all the nodes then run a full fsck When the file system check completes mount all nodes again 65 Storage Foundation Cluster File System problems Unmount failures The umount command can fail if a reference is being held by an NFS server Unshare the mount point and try the unmount again Mount failures Mounting a file system can fail for the following reasons m The file system is not using disk layout Version 6 or 7 m The mount options do not match the options of already mounted nodes m Acluster file system is mounted by default with the qio option enabled if the node has a Quick I O for Databases license installed even if the qio mount option was not explicitly specified If the Quick I O license is not installed a cluster file system is mounted without the qio option enabled So if some nodes in the cluster have a Quick I O license installed and others do not a cluster mount can succeed on some nodes and fail on others due to different mount options To avoid this situation ensure that Quick
4. 33 VCS application failover services CVMVxconfigd cvm_vxconfigd Critical 0 CVMVxconfigdArgs syslog cvm_clus requires cvm_vxconfigd vxfsckd requires cvm_clus resource dependency tree group cvm CFSfsckd vxfsckd CVMCluster Tt cvm_clus CVMVxconfigd cvm_vxconfigd SFCFS HA Only If you configured VCS Cluster Manager Web Console a service group ClusterService was created that includes IP Process and Notifier resources These resources were configured according to information you provided during the installation procedure A resource dependency was also created VCS application failover services If you installed SFCFS HA you can begin implementing the application monitoring failover services provided by the Veritas Cluster Server Information about setting up VCS services is beyond the scope of this document See the Veritas Cluster Server documentation 34 Installing and configuring the product VCS application failover services Upgrading the product If you are running an earlier release of Veritas Storage Foundation Cluster File System you can upgrade your product using the procedures described in this chapter Topics covered in this chapter include Preparing to upgrade the product Upgrade Overview Upgrading from 3 5 to 5 0 Upgrading from 4 1 to 5 0 Upgrading the disk layout versions 36 Upgrading the product
5. a node halts to avoid any chance of network partitioning Reduce the load on the node if this happens frequently A similar situation may occur if the values in the etc 11ttab files on all cluster nodes are not correct or identical
6. mv etc llthosts bak etc llthosts Offline all SFCFS resources on nodes selected in step 2 by running the following commands on one of the cluster nodes hagrp offline service_group sys system01 hagrp offline service_group sys system02 where service_group is the command that displays the service groups by hagrp dep cvm Install SFCFS 5 0 on system01 and system02 and reboot these nodes See Installing the product on page 19 Note Do not configure SFCFS after reboot Shutdown VCS on remaining non upgraded nodes a Run the following commands from one of the non upgraded nodes haconf makerw hagrp unfreeze cvm persistent hagrp unfreeze service group persistent haconf dump makero where service _ groupis the command that displays the service groups by hagrp dep cvm b Run the following commands on all the non upgraded nodes hastop local vxfenconfig U Start vxfen on system01 and system02 vxfen can be started either in disable or enable mode For starting vxfen in disabled mode run the following commands echo vxfen_mode disabled gt etc vxfenmode sbin init d vxfen start See the Veritas Cluster Server Installation Guide for information regarding starting vxfen in enabled mode Upgrading the product 47 Upgrading from 4 1 to 5 0 12 Change the configuration files by running the following commands on one of the upgraded nodes For example system01 opt VRTS bin hasta
7. Preparing to upgrade the product Preparing to upgrade the product Upgrade paths This section prepares you for the Veritas Storage Foundation Cluster File System upgrade Planning the upgrade Complete the following tasks in advance of upgrading m Review the Veritas Storage Foundation Cluster File System Release Notes for any late breaking information on upgrading your system m Be sure that the administrator doing the upgrade has root access and a working knowledge of system administration m Schedule sufficient outage time for the upgrade m Make sure that the prerequisite patches required for SFCFS 5 0 are accessible The upgrade paths for Veritas Storage Foundation Cluster File System are From Upgrade to Tasks Storage Foundation Cluster File System 3 5 formerly known as SANPoint Foundation Suite 3 5 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 3 5 to 5 0 on page 39 Storage Foundation Cluster File System 3 5 Update 1 formerly known as SANPoint Foundation Suite 3 5 Update 1 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 3 5 to 5 0 on page 39 Storage Foundation Cluster File System 3 5 Update 2 formerly known as SANPoint Foundation Suite 3 5 Update 2 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 3 5 to 5 0 on page 39 Upgrading the product Preparing to u
8. The list continues with similar information for system02 not shown and any other systems in the cluster On each system the output should be similar For more information on the hasys display command see the hasys 1M manual page Also refer to the chapter in the Veritas Cluster Server User s Guide Administering VCS From the Command Line System Attribute Value system0 AgentsStopped 0 system0 AvailableCapacity 1 system0 Capacity 1 system0 ConfigBlockCount 54 system0 ConfigCheckSum 29776 Installing and configuring the product Verifying the configuration files System system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 system0 Attribute ConfigDiskState ConfigFile ConfigInfoCnt ConfigModDate CurrentLimits DiskHbStatus DynamicLoad Frozen GUIIPAddr LLTNodelId Limits LoadTimeCounter LoadTimeThreshold LoadWarningLevel MajorVersion MinorVersion Nodeld OnGrpCnt ShutdownTimeout SourceFile SysName SysState SystemLocation SystemOwner TFrozen TRSE UpDownState UseriInt Value CURRENT etc VRTSvces conf config 0 Tues June 25 23 00 00 2006 1890 600 80 60 main cf system01 RUNNING Up Installing and configuring the product 31 Verifying agent configuration System Attribute Va
9. cluster As in previous examples the following removes the system system03 from a three node cluster The procedure can be done from any node remaining in the cluster or from a remote host To remove a node from a cluster 1 Login as superuser on a node other than system03 2 Stop all the cluster components cfscluster stop f system03 3 Open the VCS configuration for writing haconf makerw 58 Adding and removing a node Removing a node from a cluster 4 10 11 Remove system03 from the system list attribute of the CVM and SFCFS service groups hagrp modify service group SystemList delete system03 hagrp modify cvm SystemListf delete system03 where service_group is the command that displays the service groups by hagrp dep cvm If an error message similar to the following is displayed by either of the above commands VCS 10456 Configuration must be ReadWrite hagrp modify delete 0x10f Sysstate RUNNING Channel IPM Flags 0x0 Repeat step 3 and the command that failed in step 4 Write the new VCS configuration to disk haconf dump makero Edit etc 1llthosts on the remaining nodes of the cluster and remove the entry corresponding to the node being removed Edit etc gabtab on the remaining nodes of the cluster and edit the gabconfig command to reflect the correct and new number of nodes in the cluster Login to system03 and remove the following files rm etc vxfenm
10. node to a cluster If you want to add a new node to a multi node cluster first prepare the new system hardware Physically connect the new system to the cluster using private networks and attach to any shared storage Then install the required OS software Install all the prerequisite patches mentioned in Required HP UX patches on page 15 See Cluster platforms on page 11 To add a node to a cluster 1 Log into the new system as superuser 2 Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system 3 Run the following commands to start PFS Portable File System nohup pfs_mountd amp mnohup pfsd amp 4 Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom 5 Add opt VRTS bin to your PATH and opt VRTS man to your MANPATH environment variables See Setting PATH and MANPATH environment variables on page 18 6 Change to the SFCFS directory cd sfcfs 7 Runthe installsfcfs script with installonly option to install all the required SFCFS packages on the new node installsfcfs installonly 8 Enter y to install SFCFS on these systems Do you want to install SFCFS on these systems y n q y 9 Enter the system name of the new node to install SFCFS Enter the system names separted by spaces on which to ins
11. one of the nodes of the cluster and mount the file system locally on this node For example mount it without the ocluster option Enter mount F vxfs block_device_path mnt1 Current disk layout version on a file system can be found using fstyp v char device path grep version awk print 2 On the node selected in step 1 incrementally upgrade the disk layout of this file system to layout Version 6 or layout Version 7 For example if you had a cluster mounted file system of disk layout Version 4 while running with SFCFS 3 5 on HP UX 11i Version 1 after upgrading to SFCFS 5 0 you would need to upgrade the disk layout to version 6 or version 7 incrementally as follows vxupgrade n 5 mnt1 vxupgrade n 6 mnt1 vxupgrade n 7 mnt1 Upgrading the product 51 Upgrading the disk layout versions On the node selected in step 1 after the disk layout has been successfully upgraded unmount the file system umount mnt1 This file system can be mounted on all nodes of the cluster using cfsmount 52 Upgrading the product Upgrading the disk layout versions Chapter Adding and removing a node This chapter provides information on how to add a node to an existing cluster and removing a node from a cluster Topics include m Adding anode to a cluster m Configuring SFCFS and CVM agents on the new node m Removing anode from a cluster Adding and removing a node Adding a node to a cluster Adding a
12. q y 10 11 12 13 14 15 Installing and configuring the product 23 Configuring the Components Answer the prompts to configure VCS for SFCFS You are prompted to configure SFCFS to use Veritas Security Services Would you like to configure SFCFS to use Veritas Security Services y n q n Enter y or n to configure SFCFS to use Veritas Security Services You are prompted to add Storage Foundation Cluster File System users Do you want to set the username and or password for the Admin user default username admin password password y n q n Enter n if you want to decline If you enter y you are prompted to change the password You are prompted to add another user to the cluster Do you want to add another user to the cluster y n q y Enter n if you want to decline enter y if you want to add another user You are prompted to verify the Storage Foundation user Is this information correct y n q y Enter y or n to verify if this information is correct You are prompted to configure the cluster monitor Do you want to configure the Cluster Monitor y n q n Enter y or n to configure the cluster monitor You are prompted to configure SMTP notification Do you want to configure SMTP notification y n q y Enter y or n to configure SMTP notification You are prompted to configure SMTP notification Do you want to configure SNMP notification y n q y Enter y or n to configure SNMP notification
13. remaining non upgraded nodes a Run the following commands from one of the non upgraded nodes haconf makerw hagrp unfreeze cvm persistent hagrp unfreeze service group persistent haconf dump makero where service _ groupis the command that displays the service groups by hagrp dep cvm b Run the following commands on all the non upgraded nodes hastop local Start vxfen on system01 and system02 vxfen can be started either in disable or enable mode For starting vxfen in disabled mode run the following commands echo vxfen_mode disabled gt etc vxfenmode sbin init d vxfen start See the Veritas Cluster Server Installation Guide for information regarding starting vxfen in enabled mode Upgrading the product 41 Upgrading from 3 5 to 5 0 15 Change the configuration files by running the following commands on one of the upgraded nodes say system01 opt VRTS bin hastart opt VRTS bin haconf makerw hagrp unfreeze cvm persistent hagrp unfreeze service group persistent opt VRTS bin hatype add CVMVxconfigd opt VRTS bin hares add cvm_vxconfigd CVMVxconfigd cvm opt VRTS bin hares modify cvm_vxconfigd Enabled 1 opt VRTS bin hares delete qlogckd opt VRTS bin haconf dump makero opt VRTS bin hastop all force If you have configured the VCS Cluster Manager Web Console complete step a through step d to modify the etc VRTSvcs conf config main cf file He Sh SR SE OSE OSE SR OSE H a R
14. the correct update and patches are installed The installation procedure terminates if the correct patches are not found Make sure that EnablevxFS bundle has revision B 11 23 04 or later after installing the latest patches Patches that supersede the ones in this list To verify that you have the latest patches go to the Veritas support website to view the following TechNote http support veritas com docs 281875 Note Most of the above patches are available in the Feature11i bundle The Featurel1i bundle and the EnableVxFS bundle are available from HP software download site http h20293 www2 hp com Search for Veritas 5 0 on this site and follow the instructions Installing and configuring the product 17 Preinstallation Preinstallation Release Notes Read the Release Notes for all products included with this product Portable Document Format pdf versions of the Release Notes are included on the software disc in the storage_foundation_cluster_file_system release_notes directory and on the documentation disc that came with your software Because product Release Notes are not installed by any packages it is recommended that you copy them from the disc to the opt VRTS docs directory on your system so that they are available for future reference Product licensing Product keys are available on the License Key certificate The certificate also includes the number of product licenses purchased A single key en
15. the uninstallsfcfs command to uninstall SFCFS The uninstallsfcfs script uses ssh to communicate with remote nodes as default uninstallsfcfs If you want to use rsh you must specify on the command line uninstallsfcfs rsh Enter the system names to uninstall SFCFS Enter the system names separated by spaces on which to uninstall SFCFS system01 system02 Enter y to uninstall SFCFS Are you sure you want to uninstall SFCFS y n q y Reboot the systems on which SFCFS is uninstalled after successful uninstallation 62 Uninstalling the product Appendix Troubleshooting and recovery Installation issues If you encounter any issues installing SFCFS refer to the following paragraphs for typical problems and their solutions Incorrect permissions for root on remote system The permissions are inappropriate Make sure you have remote root access permission on each system to which you are installing Checking communication with system0l FAILED Remote remsh rcp permissions not available on system01 Correct permissions and continue Continue Y N Suggested solution You need to set up the systems to allow remote access using ssh Or rsh See the Veritas Storage Foundation and High Availability Solutions Getting Started Guide Note Remove remote shell permissions after completing the SFCFS installation and configuration 64 Storage Foundation Cluster File System problems Resource tem
16. to 5 0 using phased or full upgrade procedure Phased upgrade Following procedure assumes a 4 node cluster system01 system02 system03 system04 where system01 and system02 are initially upgraded and rest of the cluster is brought up later To upgrade from 4 1 to 5 0 1 2 3 4 Log in as superuser Select one or more nodes to upgrade say system01 and system02 Insert the appropriate software disc into your system s DVD drive Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom Change to the top level directory on the disc cd dvdrom Install all the prerequisite patches on system01 and system02 See Required HP UX patches on page 15 If this step is being performed in any phase other than the first phase of the upgrade the 11thosts need to be replace to prevent vxfen from starting after reboot by running the following commands a Move etc lithosts to etc llthosts bak on all the nodes to be upgraded 46 Upgrading the product Upgrading from 4 1 to 5 0 10 11 mv etc llthosts etc llthosts bak b Install all the prerequisite patches and reboot the machines c Move etc llthosts to etc 1lthosts bak on all the nodes to be upgraded
17. I O licensing is uniformly applied or be careful to mount the cluster file system with the gio noqio option appropriately specified on each node of the cluster See the mount 1M manual page m A shared CVM volume was not specified m The device is still mounted as a local file system somewhere on the cluster Unmount the device m The fsck or mkfs command is being run on the same volume from another node or the volume is mounted in non cluster mode from another node m The vxfsckd daemon is not running This typically happens only if the CFSfsckd agent was not started correctly m If mount fails with an error message vxfs mount cannot open mnttab etc mnttab is missing or you do not have root privileges m If mount fails with an error message vxfs mount device already mounted The device is in use by mount mkfs or fsck on the same node This error cannot be generated from another node in the cluster 66 Storage Foundation Cluster File System problems m If this error message displays mount slow The node may be in the process of joining the cluster m Ifyou try to mount a file system that is already mounted without o cluster option that is not in shared mode on another cluster node mount F vxfs dev vx dsk share vol01 vol101 The following error message displays vxfs mount dev vx dsk share vol01 is already mounted vol01 is busy allowable number of mount points exceeded or cluster reservation fai
18. VRTSvcesmn Veritas Cluster Server Man Pages VRTSvmdoc Veritas Volume Manager Documentation VRTSvrdoc Veritas Volume Replicator Documentation VRTSfsmnd Veritas File System SDK Manuals Note SFCFS 5 0 operates only on HP UX 11i 64 bit operating system with the September 2004 HP UX 11i Version 2 0 or later All cluster nodes must be running this OS version For cluster file system functionality to work reliably on HP UX you must have the required HP UX patches installed See Required HP UX patches on page 15 Installing and configuring the product 15 Required HP UX patches Required HP UX patches HP UX required patches include the following PHCO_32385 Enables scat 1M PHCO_32387 Enables getext 1M PHCO_32388 Enables set ext 1M PHCO_32389 Enables vxdump 1M PHCO_32390 Enables vxrestore 1M PHCO_32391 Enables vxfsstat 1M PHCO_32392 Enables vxtunefs 1M PHCO_32393 Enables vxupgrade 1M PHCO_32488 Enables LIBC for VxFS 4 1 and later file systems PHCO_32523 Enhancement to quot a 1 for supporting large uids PHCO_32524 Enhancement to edquota for supporting large uids PHCO_32551 Enhancement to quotaon quotaoff for supporting large uids PHCO_32552 Enhancement to repquota for supporting large uids PHCO_32596 Enables df 1M PHCO_32608 Enables bd 1M PHCO_32609 Enables fst yp 1M PHCO_32610 Enables mount 1M PHCO_32611 Fix s_wrapper
19. You are prompted to set up the enclosure based naming scheme Do you want to set up the enclosure based naming scheme y n q n Enter y or n to set up the enclosure based naming scheme You are prompted to set up a VxVM default disk group for each system Do you want to set up a default disk group for each system y n q y Will you specify one disk group name for all eligible systems y n q y Specify a default disk group for all systems sharedg Is this correct y n q y 24 Installing and configuring the product Using the log files 16 Enter y or nif the VxVM default disk group information is correct You are prompted to enable centralized management Enable Centralized Management y n q y n 17 Enter y orn to enable centralized management You are prompted to verify the fully qualified domain name for system01 Is the fully qualified host name system01 domain_name y n q y 18 Enter yorn to verify the fully qualified domain name for system01 You are prompted to start SFCFS process now Do you want to start Veritas Storage Foundation for Cluster File System process now y n q y Using the log files After product installation the installation scripts create three text files in the opt VRTS install logs directory Do not remove the log files until the Veritas products are working properly on your system Technical Support will need these log files for debugging purposes Installation
20. ables product installation on the number and the type of systems for which you purchased the license A key may enable the operation of more products than specified on the certificate but you are legally limited to the number of product licenses purchased The VRTSvlic package executes Veritas product licensing The following commands and their manual pages are available on the system after VRTSvlic is installed vxlicinst installs a license key for a Veritas product vxlicrep displays currently installed licenses vxlictest retrieves features and their descriptions encoded in a license key If you encounter problems while licensing your product visit the Veritas licensing support website at http www veritas com buy vLicense vLicenseHome jhtml Veritas products are installed under the opt directory on the specified host systems Verify that the directory opt exists and has write permissions for root before starting the installation procedure Do not make opt a VxFS file system 18 Installing and configuring the product Preinstallation Also you can get the patches from Hewlett Packard s Patch Database offered under the Maintenance and Support section of the HP Services amp Support IT Resource Center HP s Patch Database provides fast accurate searches for the latest recommended and superseded patches available for Veritas File System or Veritas Volume Manager Setting PATH and MANPATH environment variables The sof
21. annel switch Each node in the cluster must have a Fibre Channel I O channel to access shared storage devices The primary component of the Fibre Channel fabric is the Fibre Channel switch Cluster platforms There are several hardware platforms that can function as nodes in a cluster file system cluster See the Storage Foundation Cluster File System Release Notes Install the HP UX 11i 64 bit operating system with the September 2004 HP UX 11i Version 2 0 or later on each node and install a Fibre Channel host bus adapter to allow connection to the Fibre Channel switch Note For a cluster to work correctly all nodes must have the same time If you are not running the Network Time Protocol NTP daemon make sure the time on all the systems comprising your cluster is synchronized 12 Installing and configuring the product Software components Software components Storage Foundation for Cluster File System is the name of the Veritas Cluster File System product and its supporting software packages Storage Foundation Cluster File System HA SFCFS HA provides support for application failover functionality of Veritas Cluster Server VCS in addition to other SFCFS features Packages installed with SFCFS only The software packages listed below are required for implementing cluster file system functionality They are available on the software disc in the depot directory and are installed on each node in the cluster using the ins
22. ate of OPEN a status for each link of up and an address for each link However in the example above the output shows that for node system02 the private network may have failed or the information in etc 11ttab may be incorrect To obtain information about the ports open for LLT type 11t stat p on any system In the following example 11t stat p is typed on one system in the cluster 4 Log into system01 lltstat p Output resembles LLT port information Port Usage Cookie 0 gab 0x0 opens 01345678 910 11 12 13 connects 0 1 Note The two systems with node ID s 0 and 1 are connected See etc llthosts on page 24 28 Installing and configuring the product Verifying the configuration files Group Membership and Atomic Broadcast configuration files The following files are required by the VCS communication services for Group Membership and Atomic Broadcast GAB etc gabtab After installation the file etc gabtab contains a gabconfig 1M command that configures the GAB driver for use The file etc gabt ab contains a line that resembles sbin gabconfig c n N where the c option configures the driver for use and n N specifies that the cluster will not be formed until at least N systems are ready to form the cluster Nis the number of systems in the cluster Checking Group Membership and Atomic Broadcast operation This section describes how to check GAB operation To check GAB oper
23. ation m Enter the following command on each node in the cluster sbin gabconfig a If GAB is operational the following output displays with GAB port membership information GAB Port Memberships a gen lbbf01 membership 01 b gen lbbf06 membership 01 Port f gen lbbf0f membership 01 h gen lbbf03 membership 01 v gen lbbf0b membership 01 Port w gen lbbf0d membership 01 If GAB is not operational the following output display with no GAB port membership information GAB Port Memberships See the Veritas Cluster Server User s Guide Installing and configuring the product 29 Verifying the configuration files Checking cluster operation This section describes how to check cluster operation To check cluster operation 1 Enter the following command on any system hastatus summary The output for an SFCFS HA installation resembles SYSTEM STATE System State Frozen A system01 RUNNING 0 A system02 RUNNING 0 GROUP STATE Group System Probed AutoDisabled State B cvm system01 Y N ONLINE B cvm system02 Y N OFFLINE Note If the State value is running VCS is successfully installed and running on that node The group state lists the cvm group which is online on system01 and offline on system02 See the hastatus 1M manual page See the Veritas Cluster Server User s Guide 2 Enter the following command on any systems hasys display The example on the next page shows the output of system01
24. ce_group sys system03 hagrp offline service group sys system04 where service_group is the command that displays the service groups by hagrp dep cvm Upgrading the product 49 Upgrading from 4 1 to 5 0 8 Install SFCFS 5 0 and reboot all the nodes See Installing the product on page 19 Note Do not configure SFCFS after reboot 9 Start vxfen on all the nodes vxfen can be started either in disable or enable mode For starting vxfen in disabled mode run the following commands cat vxfen_mode disabled gt etc vxfenmode sbin init d vxfen start See the Veritas Cluster Server Installation Guide for information regarding starting vxfen in enabled mode 10 Change the configuration files by running the following commands from one of the nodes opt VRTS bin hastart opt VRTS bin haconf makerw opt VRTS bin hares delete qlogckd opt VRTS bin haconf dump makero opt VRTS bin hastop all force 11 If you have configured the VCS Cluster Manager Web Console complete step a through step d to modify the etc VRTSvcs conf config main cf file a Remove VRTSweb Process VRTSweb PathName opt VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSweb VERITAS TimeForOnline 5 c Add the NIC resource in the ClusterService group For example where the name of the NIC resource is named csgnic and the p
25. ct Upgrading from 4 1 to 5 0 Full upgrade 16 17 18 19 Configure SFCFS on system01 and system02 See Using the log files Note VCS configuration files are not changed during this configuration Upgrade file systems to proper disk layout version as mentioned in Upgrading the disk layout versions on page 50 Verify that all the file systems are working properly and data is intact See cfsmount 1M Run step 7 step 9 and step 16 on rest of the nodes to be upgraded Following procedure assumes a 4 node cluster system01 system02 system03 system04 where all nodes are simultaneously upgraded from 4 1 to 5 0 1 2 3 Log in as superuser Insert the appropriate software disc into your system s DVD drive Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom Change to the top level directory on the disc cd dvdrom Install all the prerequisite patches on all the nodes See Required HP UX patches on page 15 Offline all SFCFS resources on all nodes by running the following commands on one of the cluster nodes hagrp offline service_group sys system01 hagrp offline service_group sys system02 hagrp offline servi
26. eesseseseeeeseseeeeeees 27 Group Membership and Atomic Broadcast configuration files 29 Checking Group Membership and Atomic Broadcast operation 29 Checking cluster operation 30 Verifying agent configuration 0 0 0 0 132 Synchronizing time on Cluster File Systems 32 Configuring VCS ceccecseesesseseesseeseeeeseseeeeeeees 32 main cf file 33 SFCFS HA Only 34 VCS application failover services ooo esccsscscsssscsessescssescsceecscsceecseescseseeassesees 34 Chapter 2 Chapter 3 Chapter 4 Appendix A Upgrading the product Preparing to upgrade the product cccccecesssscsesseeeseseeceseeecseseeeeseeeeseeeeeesees 36 Planning the upgrade Upgrade paths oe cccceceesssceseeceseteeseseseeseseeeeseees PeTAade Overview aron asna E EE E EOE ERER R Phased upgrade pencran aia E nein ae aiatiaainreils Full upgrade seeren e ar EE EE EE nein AE pgrading from 3 5 to 5 0 Phased upgrade niori E R R Hee JLo UURS Gotear Ce I a EE EEES tee cavendeitltas pgrading from 4 1 to 5 0 Phased Upgrade 0 cccccessesssseseeseseseeeeseseeceseeeeseseeceseeeseeeseseeesseseneeseseeseeeees Fulluperad es cpn E EEEE Ree pgrading the disk layout Versions cceeseseeseceseeseseeeeseseeceseseeeeseeetseeseeaes 50 a E Adding and removing a node Adding a node to a cluster sssricnirerarne in ERA G 54 Configuring SFCFS and CVM agents on the new node eeeeeseseeeeeeeeeees 56 R
27. emove VRTSweb Process VRTSweb PathName opt VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSweb VERITAS TimeForOnline 5 c Add the NIC resource in the ClusterService group For example where the name of the NIC resource is named csgnic and the public NIC device is hmeO add NIC csgnic Device hme0 d Add new dependencies for the new resources in the ClusterService group For example using the names of the VRTSWebApp NotifierMngr IP and NIC resources enter lines that resemble VCSweb requires webip ntfr requires csgnic webip requires csgnic 16 Verify the syntax of the etc VRTSvcs conf config main cf file by running the following commands on system01 cd etc VRTSvcs conf config opt VRTS bin hacf verify 17 Start VCS on all the upgraded nodes Run the following command on system01 and system02 opt VRTS bin hastart 42 Upgrading the product Upgrading from 3 5 to 5 0 Full upgrade 18 19 20 21 Configure SFCFS on system01 and system02 See Using the log files Note VCS configuration files are not changed during this configuration Upgrade file systems to proper disk layout version as mentioned in Upgrading the disk layout versions on page 50 Verify that all the file systems are working properly and data is intact See cfsmount 1M Run step 8 through step 12 st
28. emoving a node from a cluster oo ceeceesesssesesesesssessesssecesesesesesesseesessesesesesesees 57 Uninstalling the product Troubleshooting and recovery Installation issues cannae i E E Ei 63 Incorrect permissions for root on remote system cceeeseseseseteteeees 63 Resource temporarily unavailable 0 0 0 cccesseseseeeeseeseseeeeeseeeeseeeeeeees 64 Inaccessible SYSteM sicie seners ia tea aoia aean onas rris 64 Storage Foundation Cluster File System problems cccccseeseeeeeeeeees 64 Unmount failures o e ecececeececececesesseseseseseseesescscsesesecscseecscseaeseseseeeseacseseaeaens 65 Mount failures i s i ctaiss cance a EEEE EE 65 Command failures oe eeececeseseseseseseceesesesesesesesesecscscseseseseseeeseeseseeeseaeseseesees 66 Performance issues cnnsrn cass ase R E es 66 High availability issues c ce cccescssesesseseseseeseseeceseeeeseseeceseseeeeseeeeseeeesees 67 Installing and configuring the product This chapter describes how to install the Veritas Storage Foundation Cluster File System SFCFS SFCFS requires several Veritas software packages to configure a cluster and to provide messaging services These packages include the Veritas Cluster Server VCS to monitor systems and application services Veritas Low Latency Transport LLT and Veritas Group Membership and Atomic Broadcast GAB for messaging and cluster membership the Veritas Volume Manager VxVM to create the shared volumes necessar
29. ep 14 step 17 step 18 and step 20 on rest of the nodes to be upgraded Following procedure assumes a 4 node cluster system01 system02 system03 system04 where all nodes are simultaneously upgraded from 3 5 to 5 0 1 2 3 Log in as superuser Insert the appropriate software disc into your system s DVD drive Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom Change to the top level directory on the disc cd dvdrom Offline all SFCFS resources on all nodes by running the following commands on one of the cluster nodes hagrp offline service_group sys system01 hagrp offline service_group sys system02 hagrp offline service_group sys system03 hagrp offline service group sys system04 where service_group is the command that displays the service groups by hagrp dep cvm Remove the VRTScavf and VRTSglm 3 5 packages from these nodes and run the following command on all the systems hastop local swremove VRTScavf VRTSglm 10 11 12 Upgrading the product 43 Upgrading from 3 5 to 5 0 Uninstall VCS 3 5 from all the nodes Run the following commands from one of the nodes See the Veritas Cluster Server Installation Guide cd
30. guration of SFCFS components To configure the components 1 2 Log in as superuser Run the installer command to install the SFCFS For example cd cdrom installer From the Installation menu choose the c option for Configuration and select 6 the Veritas Storage Foundation Cluster File System You are prompted to enter one or more system names Enter the system names separted by spaces on which to configure SFCFS system01 system02 Enter one or more system names to configure SFCFS You are prompted to enter Return to continue Press Return to continue Press Return to continue You are prompted to configure I O fencing in enabled mode Will you be configuring I O Fencing in enabled mode y n q y Enter y or n to configure I O fencing in enable mode You are prompted to configure VCS for SFCFS Enter the unique cluster name Enter the unique Cluster ID number between 0 65535 b Enter the NIC for the first private heartbeat link on system01 b Would you like to configure a second private heartbeat link y n q b y Enter the NIC for the second private heartbeat link on hpslia05 b Lang Are you sure you want to use lan0 for the second private heartbeak link y n q b n Do you want to configure an additional low priority heartbeat link y n q b n Are you using the same NICs for private heartbeat links on all systems y n q b y Is this information correct y n
31. irectory on the disc cd dvdrom Offline all SFCFS resources on nodes selected in step 2 by running the following commands on one of the cluster nodes hagrp offline service_group sys system01 hagrp offline service_group sys system02 where service_group is the command that displays the service groups by hagrp dep cvm Remove the VRTScavf and VRTSglm 3 5 packages from these nodes and run the following commands on system01 and system02 hastop local swremove VRTScavf VRTSglm 40 Upgrading the product Upgrading from 3 5 to 5 0 10 11 12 13 14 Uninstall VCS 3 5 from system01 and system02 Run the following commands from one of the nodes See the Veritas Cluster Server Installation Guide cd opt VRTSvcs install uninstallves Note Ignore any errors from the uninstallvcs script and proceed with the uninstall of VCS Also run the following command from rest of the nodes in the cluster on which uninstallation is not performed using uninstallvcs hastart Upgrade the operating system from HP UX 11i Version 1 to HP UX 11i Version 2 on system01 and system02 See the HP UX Operating System documentation Install all the prerequisite patches on system01 and system02 See Required HP UX patches on page 15 Install SFCFS 5 0 on system01 and system02 and reboot these nodes See Installing the product on page 19 Note Do not configure SFCFS after reboot Shutdown VCS on
32. led for the volume Command failures m Manual pages not accessible with the man command Set the MANPATH environment variable as listed under Setting PATH and MANPATH environment variables on page 18 m The mount fsck and mkfs utilities reserve a shared volume They fail on volumes that are in use Be careful when accessing shared volumes with other utilities such as da it is possible for these commands to destroy data on the disk m Running some commands such as vxupgrade n 7 vol02 can generate the following error message vxfs vxupgrade ERROR not primary in a cluster file system This means that you can run this command only on the primary that is the system that mounted this file system first Performance issues Quick 1 0 File system performance is adversely affected if a cluster file system is mounted with the qio option enabled and Quick I O is licensed but the file system is not used for Quick I O files Because qio is enabled by default if you do not intend to use a shared file system for Quick I O explicitly specify the noqio option when mounting 67 Storage Foundation Cluster File System problems High availability issues Network partition jeopardy Network partition or split brain is a condition where a network failure can be misinterpreted as a failure of one or more nodes in a cluster If one system in the cluster incorrectly assumes that another system failed it may restart applications already
33. log file Response file Summary file The installation log file contains commands executed during the procedure the output and any errors generated by the commands This file is for debugging installation problems and can be used by Veritas Technical Services The response file contains the configuration information entered during the procedure The response file can be used for future installations by invoking an installation script with the responsefile option The response file passes arguments to the script to automate the installation This file can be edited to automate installation of additional systems The summary file contains output of the Veritas product installation scripts This file shows which products were installed where the log and response files are for a particular installation and the messages displayed at the end of installation Installing and configuring the product 25 Verifying the configuration files Verifying the configuration files You can inspect the contents of the configuration files that were installed and modified after a successful installation process These files reflect the configuration based on the information you supplied To verify the configuration files 1 Login as superuser to any system in the cluster 2 Setup your environment PATH variable export PATH SPATH sbin usr sbin opt VRTS bin Low Latency Transport configuration files The following files are required by the VCS co
34. lue system01 UserStr Verifying agent configuration This section describes how to verify the agent configuration To verify the agent configuration m Enter the cluster status command from any node in the cluster cfscluster status Output resembles Node system01 Cluster Manager running CVM state running No mount point registered with cluster configuration Node system02 Cluster Manager running CVM state running No mount point registered with cluster configuration Synchronizing time on Cluster File Systems SFCFS requires that the system clocks on all nodes are synchronized using some external component such as the Network Time Protocol NTP daemon If the nodes are not in sync timestamps for change ct ime and modification mt ime may not be consistent with the sequence in which operations actually happened Configuring VCS Configuring VCS means conveying to the VCS engine the definitions of the cluster service groups resources and resource dependencies VCS uses two configuration files in a default configuration m The main cf file defines the entire cluster m The types cf file defines the resource types By default both files reside in the directory etc VRTSvcs conf config Additional files similar to types cf may be present if agents have been added such as Oracletypes cf 32 Installing and configuring the product Configuring VCS In a VCS cluster the first system to be brought
35. mmunication services for Low Latency Transport LLT etc IIthosts The file 11thosts 4 is a database containing one entry per system that links the LLT system ID in the first column with the LLT host name This file is identical on each system in the cluster For example the file etc 11thosts contains entries that resemble 0 system01 1 system02 etc lttab The file 11tt ab 4 contains information that is derived during installation and used by the utility 11t config 1M After installation this file lists the network links that correspond to the specific system For example the file etc 11ttab contains entries that resemble set node system01 set cluster 100 link lanl lan 1 ether link lan2 lan 2 ether The first line identifies the local system name The second line identifies the cluster that is the cluster ID you entered during installation The next two lines beginning with the 1 ink command identify the two network cards used by the LLT protocol See the 11ttab 4 manual page The manual page describes the ordering of the directives in the 11t tab file 26 Installing and configuring the product Verifying the configuration files Checking Low Latency Transport operation Use the 11t stat command to verify that links are active for LLT This command returns information about the links for LLT for the system on which it is typed See the 11tstat 1M manual page In the following example 11
36. nf config main cf file a Remove VRTSweb Process VRTSweb PathName opt VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSweb VERITAS TimeForOnline 5 c Add the NIC resource in the ClusterService group For example where the name of the NIC resource is named csgnic and the public NIC device is hme0 add NIC csgnic Device hme0 d Add new dependencies for the new resources in the ClusterService group For example using the names of the VRTSWebApp NotifierMngr IP and NIC resources enter lines that resemble VCSweb requires webip ntfr requires csgnic webip requires csgnic 14 Verify the syntax of the etc VRTSvcs conf config main cf file by running the following command on system01 cd etc VRTSves conf config opt VRTS bin hacf verify 15 Run the following command on all the nodes to start VCS opt VRTS bin hastart 16 17 18 Upgrading the product 45 Upgrading from 4 1 to 5 0 Configure SFCFS on system01 and system02 See Using the log files Note VCS configuration files are not changed during this configuration Upgrade file systems to proper disk layout version as mentioned in Upgrading the disk layout versions on page 50 Verify that all the file systems are working properly and data is intact See cfsmount 1M Upgrading from 4 1 to 5 0 SFCFS can be upgraded from 4 1
37. nstalled and run on any machine that supports the Java Runtime Environment VEA is required to access the graphical user interface GUI for Veritas Storage Foundation You can use the GUI to administer disks volumes file systems and database functionality on local or remote machines One of the following packages needs to be installed and running on the client m Veritas Enterprise Administrator VRTSobgui This is the client package for HP UX m Veritas Enterprise Administrator for Windows windows VRTSobgui msi This is the client package for Windows Check the Veritas Storage Foundation Release Notes for any patch information before you install VEA Requirements The following are system recommendations for the GUI HP UX 512MB of memory Windows XP NT ME 2000 or 300MHz Pentium with at least 256MB of memory 98 Alternatively remove the file sbin init d vmsa server Prerequisites Each cluster node must be connected to the public network and each must have a unique host name by which it can be addressed on the public network The local node from which you install does not have to be part of the cluster Provide the following information when installing the SFCFS m Thecluster name beginning with a letter a z A Z m A unique ID from 0 65535 for the cluster Within the public subnet a new cluster using a duplicate cluster ID can cause existing clusters to fail m The Storage Foundation Cluster File S
38. ode rm etc llthosts rm etc llttab rm etc gabtab If fencing was enabled on the cluster run the following commands rm etc vxfentab rm etc vxfendg If necessary modify the etc gabtab file No change is required to this file if the sbin gabconfig command has only the argument c although Symantec recommends using the nN option where Nis the number of cluster systems If the command has the form sbin gabconfig c nN where Nis the number of cluster systems then make sure that N is not greater than the actual number of nodes in the cluster or GAB does not automatically seed Modify etc 1ithosts file on each remaining nodes to remove the entry of the leaving node Change to the install directory cd opt VRTS install Adding and removing a node 59 Removing a node from a cluster 12 Fromthe scripts directory run the uninstallsfcfs script and remove SFCFS on system03 uninstallsfcfs If you do not want to remove the Veritas Cluster Server software enter n when prompted to uninstall VCS See the Veritas Cluster Server Installation Guide on the software disc 60 Adding and removing a node Removing a node from a cluster Uninstalling the product If you need to uninstall SFCFS software Use the uninstallsfcfs script To uninstall SFCFS HA 1 Log in as superuser Note Do not use the hastop force command to stop VCS Change directory to opt VRTS install cd opt VRTS install Run
39. online reads the configuration file and creates an internal in memory representation of the configuration Systems brought online after the first system derive their information from systems running in the cluster You must stop the cluster while you are modifying the files from the command line Changes made by editing the configuration files take effect when the cluster is restarted The node on which the changes were made should be the first node to be brought back online main cf file The VCS configuration file main cf is created during the installation procedure After installation the main cf file contains the base definitions of the cluster and its nodes Additionally the file types c listed in the include statement defines the bundled agents for VCS resources See the Veritas Cluster Server User s Guide A typical VCS configuration file for SFCFS file resembles include types cf include CFSTypes cf include CVMTypes cf cluster cfs_cluster UserNames admin bIJbIDiFJeJJhRJAIG Administrators admin HacliUserLevel COMMANDROOT system system0l1 system system02 group cvm SystemList AutoFailOver Parallel 1 AutoStartList system01 system02 system01 0 system02 1 0 CFSfsckd vxfsckd CVMCluster cvm_clus CVMClustName cfscluster CVMNodelId system01 0 system02 1 CVMTransport gab CVMTimeout 200 Installing and configuring the product
40. opt VRTSvcs install uninstallves Note Ignore any errors from the uninstallvcs script and proceed with the uninstall of VCS Upgrade the operating system from HP UX 11i Version 1 to HP UX 11i Version 2 on all the nodes See the HP UX Operating System documentation Install all the prerequisite patches on all the nodes See section Required HP UX patches on page 15 Install SFCFS 5 0 and reboot all the nodes See Installing the product on page 19 Note Do not configure SFCFS after reboot Start vxfen on all the nodes vxfen can be started either in disable or enable mode For starting vxfen in disabled mode run the following commands cat vxfen_mode disabled gt etc vxfenmode sbin init d vxfen start See the Veritas Cluster Server Installation Guide for information regarding starting vxfen in enabled mode 44 Upgrading the product Upgrading from 3 5 to 5 0 13 Change the configuration files by running the following commands from one of the nodes opt VRTS bin hastart opt VRTS bin haconf makerw opt VRTS bin hatype add CVMVxconfigd opt VRTS bin hares add cvm_vxconfigd CVMVxconfigd cvm opt VRTS bin hares modify cvm_vxconfigd Enabled 1 opt VRTS bin hares delete qlogckd opt VRTS bin haconf dump makero opt VRTS bin hastop all force JE Sh Sh SE SE OSE H If you have configured the VCS Cluster Manager Web Console complete step a through step d to modify the etc VRTSvcs co
41. pgrade the product From Upgrade to Tasks Storage Foundation Cluster File System 3 5 Update 3 formerly known as SANPoint Foundation Suite 3 5 Update 3 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 3 5 to 5 0 on page 39 Storage Foundation Cluster File System 4 1 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 4 1 to 5 0 on page 45 Storage Foundation Cluster File System 4 1 MP1 Storage Foundation Cluster File System 5 0 Proceed to Upgrading from 4 1 to 5 0 on page 45 37 38 Upgrading the product Upgrade Overview Upgrade Overview There are two ways to upgrade cluster nodes to the latest version of Storage Foundation Cluster File System phased and full Phased upgrade A phased upgrade minimizes downtime by upgrading portions of the cluster one at atime Although the entire cluster is offline for a shorter period than a full upgrade this method requires command line interaction and some manual configuration Each phase of the phased upgrade should be performed on more than one node of the cluster Cluster will be offline only during step 4 and step 5 below for a phased upgrade Note A phased upgrade should not be performed from one of the nodes in the cluster The stages of the phased upgrade procedure are 1 Select two or more nodes to upgrade 2 Install the new version 3 Shut down VCS on
42. porarily unavailable If the installation fails with the following error message on the console fork failed Resource temporarily unavailable The value of nkthread tunable parameter nay not be large enough The nkthread tunable requires a minimum value of 600 on all systems in the cluster To determine the current value of nkthread enter ketune q nkthread If necessary you can change the value of nkthread using the SAM System Administration Manager interface or by running the kctune command If you change the value of nkthread the kernel must be rebuilt for the new value to take effect It is easier to change the value using SAM because there is an option to process the new kernel immediately See the kct une 1M and sam 1M manual pages for more information on tuning kernel parameters Inaccessible system The system you specified is not accessible This could be for a variety of reasons such as the system name was entered incorrectly or the system is not available over the network Checking communication with system0l1 FAILED System not accessible system01 Suggested solution Verify that you entered the system name correctly use the ping 1M command to verify the accessibility of the host If a system cannot access the software source depot either swagentd is not running on the target system or the swlist command cannot see the source depot Correct etc hosts nsswitch conf and continue from here Continue
43. remaining non upgraded nodes 4 Modify the configuration information in the main cf file 5 Install the new version on each remaining node and reboot them Full upgrade A full upgrade upgrades the product on the entire cluster and the cluster remains offline for the duration of the procedure Minimal command line interaction and some manual configuration are required The stages of the full upgrade procedure are 1 Install the new version on all the nodes 2 Modify the configuration information in the main cf file 3 Bring up the cluster Upgrading the product 39 Upgrading from 3 5 to 5 0 Upgrading from 3 5 to 5 0 SFCFS can be upgraded from 3 5 to 5 0 using phased or full upgrade procedure Phased upgrade Following procedure assumes a 4 node cluster system01 system02 system03 system04 where system01 and system02 are initially upgraded and rest of the cluster is brought up later To upgrade from 3 5 to 5 0 1 2 3 4 Log in as superuser Select one or more nodes to upgrade say system01 and system02 Insert the appropriate software disc into your system s DVD drive Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system Create a directory in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom Change to the top level d
44. ritas Cluster Server Configuration Wizards VRTSvxvm Veritas Volume Manager Binaries VRTSdsa Veritas Datacenter Storage Agent VRTSfspro Veritas File System Management Services Provider VRTSvmpro Veritas Volume Manager Management Services Provider VRTSdcli Veritas Distributed Command Line Interface VRTSalloc Veritas Volume Manager Intelligent Storage Provisioning VRTSvdid Veritas Device Identification API VRTSvsvec Veritas Volume Server and Client Provider VRTSddlpr Veritas Device Discovery Layer Services Provider VRTSvrpro Veritas Volume Replicator Client Extension and Provider for Veritas Enterprise Administrator VRTSvcsvr Veritas Cluster Server Agents for VVR VRTSvrw Veritas Volume Replicator Web Console VRTSvxfs Veritas File System VRTSfsman Veritas File System Manual Pages VRTSfssdk Veritas File System Software Developer Kit VRTSglm Veritas Group Lock Manager VRTScavf Veritas Cluster Server Agents for Cluster File System 14 Installing and configuring the product Software components Optional packages for SFCFS and SFCFS HA VRTScfsdc Veritas Cluster File System Documentation VRTScmccec Veritas Cluster Management Console Cluster Connector VRTScmcs Veritas Cluster Management Console Single Cluster Mode VRTScscm Veritas Cluster Server Cluster Manager VRTScssim Veritas Cluster Server Simulator VRTSfsdoc Veritas File System Documentation VRTSvcsde Veritas Cluster Server Documentation
45. rt opt VRTS bin haconf makerw hagrp unfreeze cvm persistent hagrp unfreeze service group persistent opt VRTS bin hares delete qlogckd opt VRTS bin haconf dump makero opt VRTS bin hastop all force where service _ groupis the command that displays the service groups by hagrp dep cvm SE SR OE OE H 13 If you have configured the VCS Cluster Manager Web Console complete step a through step d to modify the etc VRTSvcs conf config main cf file a Remove VRTSweb Process VRTSweb PathName opt VRTSvcs bin haweb Arguments 10 129 96 64 8181 b Replace it with VRTSWebApp VCSweb Critical 0 AppName vcs InstallDir opt VRTSweb VERITAS TimeForOnline 5 c Add the NIC resource in the ClusterService group For example where the name of the NIC resource is named csgnic and the public NIC device is hme0 add NIC csgnic Device hme0 d Add new dependencies for the new resources in the ClusterService group For example using the names of the VRTSWebApp NotifierMngr IP and NIC resources enter lines that resemble VCSweb requires webip ntfr requires csgnic webip requires csgnic 14 Verify the syntax of the etc VRTSvcs conf config main cf file by running the following commands on system01 cd etc VRTSvcs conf config opt VRTS bin hacf verify 15 Start VCS on all the upgraded nodes Run the following command on system01 and system02 opt VRTS bin hastart 48 Upgrading the produ
46. running on the other system thereby corrupting data CFS tries to prevent this by having redundant heartbeat links At least one link must be active to maintain the integrity of the cluster If all the links go down after the last network link is broken the node can no longer communicate with other nodes in the cluster Thus the cluster is in one of two possible states Either the last network link is broken called a network partition condition or the last network link is okay but the node crashed in which case it is not a network partition problem It is not possible to identify whether it is the first or second state so a kernel message is issued to indicate that a network partition may exist and there is a possibility of data corruption Jeopardy is a condition where a node in the cluster has a problem connecting to other nodes In this situation the link or disk heartbeat may be down so a jeopardy warning may be displayed Specifically this message appears when a node has only one remaining link to the cluster and that link is a network link This is considered a critical event because the node may lose its only remaining connection to the network Caution Do not remove the communication links while shared storage is still connected 68 Storage Foundation Cluster File System problems Low memory Under heavy loads software that manages heartbeat communication links may not be able to allocate kernel memory If this occurs
47. scscscsscscseescsesecsesesecscsececsesecseseecaescsesecseseeeesees 11 Software components ere Re UE N rE TE O 12 Required HP UX patches oi ccccscscsscescssessesecsessesecsecaessessesscaessessceesseesecaesesees 15 Preinistallation 2 c52 3 cecicseses avis icacssessteveaesd arrear eave ncanaesisinavaceseaneieeee 17 Release NOteS 2 25 cose csssceccsestescsedsaciecesasdscsieyedtestvviaeaddessedhceeeswesceddewaceettavesdhiees 17 Product licensing oseesricnne eeteec eecseecacdiceerec eee 17 Setting PATH and MANPATH environment variables eee 18 Secure COMMUMICATION 0 0 eeeececesceseeseeseeceseeseeseeececeseeneeeceaeeeeeeeeeseeaeeetsees 18 Veritas Enterprise Administrator cccccceescesessesscsceseesesecssesseseseeseees 19 PRE FEQUISIECS arnescar a a RA a RAV Tee Sa ES 19 Installing the Product c cccccsessssssesssseseessseessescsceseseseeseeesseseeecseeesaeeeeseeeeesees 20 Configuring the Components ccesssesssesesesssssesesssseecesesesesesesescessseeeseaeseeeaees 23 Using the log files oo ccccssssssssesssssesesesesessesescseseseseseseseseeeeseeeseseseseseeseeecaeeeaes 25 Trstallatiom log Pile ssrisisosiei nri R E ASR 25 Response fil apntsena mknn enen a E 25 Summary file siesena r E 25 Verifying the configuration files 0 0 cesesssesssesesesesessesssssseesescseseseeeseeeeees 26 Low Latency Transport configuration files ceseseseseeeeseneteeeeeees 26 Checking Low Latency Transport operation ccec
48. t has been added usr sbin shutdown R y 0 Start VxVM on the system that has been added vxinstall After starting VxVM proceed to Configuring SFCFS and CVM agents on the new node 56 Adding and removing a node Configuring SFCFS and CVM agents on the new node Configuring SFCFS and CVM agents on the new node You must configure the SFCFS and CVM agents after rebooting the new system To configure SFCFS and CVM agents on the new node 1 Start the VCS server and vxfen on system03 a Usehastart on system03 for starting the VCS server b For starting vx fen in the disable mode run the following commands on system03 echo vxfen_mode disabled gt etc vxfenmode sbin init d vxfen start c For starting vxfen in the enabled mode m Copy the following files from one of the existing cluster nodes to system03 etc vxfenmode etc vxfendg m Run the following command sbin init d vxfen start 2 Check that there are no service groups dependent on CVM such as SFCFS that are still online hagrp dep cvm 3 If there are any dependencies take them offline then take the CVM service group offline hagrp offline cvm sys system01 hagrp offline cvm sys system02 4 Open the VCS configuration for writing haconf makerw 5 Add the new node to the CVM system list and specify a failover priority of zero hagrp modify cvm SystemList add system03 X where Xis one more than the index of the last system in S
49. t stat n is typed on each system in the cluster To check LLT operation 1 Log into system01 lltstat n Output resembles LLT node information Node State Links 0 system01 OPEN 2 1 system02 OPEN 2 2 Log into system02 lltstat n Output resembles LLT node information Node State Links 0 system01 OPEN 2 x 1 system02 OPEN 2 Note Each system has two links and that each system is in the OPEN state An asterisk denotes the system on which the command is typed With LLT configured correctly the output of 11tstat n shows all of the systems in the cluster and two links for each system If the output shows otherwise you can use the verbose option of 11t stat For example type lltstat nvv more ona system to view additional information about LLT In the following example 11t stat nvv more is typed on a system in a two node cluster 3 Log into system01 Lltstat nvv more Output resembles Node State Link Status Address 0 system01 OPEN lanl UP 08 00 20 93 0E 34 lan2 UP 08 00 20 93 0E 34 1 system02 OPEN lanl UP 08 00 20 8F D1 F2 lan2 DOWN 08 00 20 8F D1 F2 2 CONNWAIT lanl DOWN lan2 DOWN Installing and configuring the product 27 Verifying the configuration files 31 CONNWAIT lanl DOWN lan2 DOWN Note The output lists 32 nodes It reports on the two cluster nodes system01 and system02 plus non existent nodes For each correctly configured system the information shows a st
50. tall SFCFS system03 10 Enter a license key for system03 Enter a SFCFS license key for system03 XXXX XXXX XXXX XXXX XXXX X 11 12 13 14 15 16 17 18 Adding and removing a node 55 Adding a node to a cluster Enter y or n for another license key You are prompted to press Return to continue Do you want to enter another license key for system03 y M q 7 n Enter 1 or 2 to be installed on all systems Select the packages to be installed on all systems 1 2 q 2 Press Return to continue Press Return to continue Note Do not reboot the machine now Create the etc 11ttab file the same as it looks on another node in the cluster a Change the set node line to the name of the new node and specify that the LAN ports for the public and private networks are configured the same as on the other cluster nodes set node system03 set cluster 100 link lan2 dev lan 2 ether link lanO dev lan 0 ether b Copy etc 1lithosts from one other node of the cluster and add a new line to it with the name of this node c Copy the same 11thosts file to all nodes of the cluster See etc Ilttab on page 25 Create etc gabtab file the same as it is on another node in the cluster See etc gabtab on page 28 For example sbin gabconfig c n 3 There is no need to reboot the other nodes just update the etc gabtab file on the other nodes in the cluster Reboot the system tha
51. taller installation script The following table shows the package name and contents for each package VRTSperl Veritas Perl 5 8 8 Redistribution VRTSvlic Veritas Licensing VRTSicsco Symantec Common Infrastructure VRTSpbx Symantec Private Branch Exchange VRTSsmf Symantec Service Management Framework VRTSat Symantec Product Authentication Service VRTSobc33 Veritas Enterprise Administrator Core Service VRTSob Veritas Enterprise Administrator Service VRTSobgui Veritas Enterprise Administrator VRTSccg Veritas Enterprise Administrator Central Control Grid VRTSmh Veritas Centralized Management for Storage Foundation Managed Host VRTSaa Veritas Enterprise Administrator Action Agent VRTSspt Veritas Software Support Tools SYMC1lma Symantec License Inventory Agent VRTS11t Veritas Low Latency Transport VRTSgab Veritas Group Membership and Atomic Broadcast VRTSvxfen Veritas I O Fencing Installing and configuring the product 13 Software components VRTSvcs Veritas Cluster Server VRTSacclib Veritas ACC Library VRTSvcsag Veritas Cluster Server Bundled Agents VRTSvcsmg Veritas Cluster Server Message Catalogs VRTSjre Veritas Java Runtime Environment Redistribution VRTSjre15 Veritas Java Runtime Environment Redistribution VRTScutil Veritas Cluster Utilities VRTSweb Veritas Java Web Server VRTScscw Ve
52. to accept vxfs from subtype PHCO_33238 swapon 1M cumulative patch PHCO_34036 LVM commands patch PHCO_34208 SAM cumilative patch CO_34191 Cumulative libc patch SS_32674 Obam patch backend for the SAM patch KL_31500 Sept04 Base Patch KL 32272 Changes to fix intermittent failures in getacl setacl 16 Installing and configuring the product Required HP UX patches PHKL_32430 Changes to separate vxfs symbols from libdebug a so that symbols of VxFS 4 1and later are easily available in q4 p4 PHKL_32431 Changes to disallow mounting of a file system on a vnode having VNOMOUDNDT set Enhancements for supporting quotas on large uids PHKL_33312 LVM Cumulative Patch PHKL_34010 Cumulative VM Patch In addition to the above patches the EnableVxFS bundle needs to be installed before installing the SFCFS 5 0 This bundle is a HP bundle and contains enhancements to various commands to understand the new disk layout Version 6 and later The EnableVxFs bundle contains the following patches FSLibEnh Enhancement to LIBC libraries to understand VxFS disk layout Version 6 and later DiskQuota Enh Enhancements to various quota related commands to support large uids FSCmdsEnh Enhancements to the mount command to support VxFS 5 0 Note Install all the latest required HP UX patches before you install SFCFS You can use the swlist command to determine whether
53. tware and online manual pages for the packages comprising SFCFS are installed in several different directories However there are symbolic links to all commands in the opt VRTS bin directory and symbolic links to all manual pages in opt VRTS man To make all SFCFS commands and manual pages accessible when you do the installation add opt VRTS bin to your PATH and opt VRTS man to your MANPATH environment variables Command line examples in this guide assume these environment variables are set To prevent conflicts with VxFS manual pages previously installed with JFS OnLineJFS 3 5 the VxFS 5 0 manual pages are installed in the opt VRTS vxfs5 0 man directory The opt VRTS vxfs5 0 man directory is automatically added to etc MANPATH when the VxFS 5 0 package is installed Make sure that the opt VRTS man directory or the opt VRTS vxfs5 0 man directory goes before usr share man in you MANPATH environment variable so that the latest version of the VxFS manual pages display Secure communication Establishing a secure communication between nodes is required to install and configure a cluster The ssh command provides a remote communication and a greater level of security than the rsh command See the Veritas Storage Foundation and High Availability Solutions Getting Started Guide Installing and configuring the product 19 Prerequisites Veritas Enterprise Administrator The Veritas Enterprise Administrator VEA client can be i
54. ublic NIC device is hme0 add NIC csgnic Device hme0 d Add new dependencies for the new resources in the ClusterService group For example using the names of the VRTSWebApp NotifierMngr IP and NIC resources enter lines that resemble VCSweb requires webip ntfr requires csgnic webip requires csgnic 50 Upgrading the product Upgrading the disk layout versions 12 13 14 15 16 Verify the syntax of the etc VRTSvcs conf config main cf file by running the following commands on system01 cd etc VRTSves conf config opt VRTS bin hacf verify Run the following command on all the nodes to start VCS opt VRTS bin hastart Configure SFCFS on all the nodes See Using the log files Note VCS configuration files are not changed during this configuration Upgrade file systems to proper disk layout version as mentioned in Upgrading the disk layout versions on page 50 Verify that all the file systems are working properly and data is intact See cfsmount 1M Upgrading the disk layout versions SFCFS 5 0 supports disk layouts Versions 4 5 6 and 7 for locally mounted file systems and disk layouts Versions 6 and 7 for cluster mounted file systems If you have cluster mounted file systems with disk layout versions lower than 6 then after upgrading to SFCFS 5 0 perform the following additional steps to prepare the file system for being mounted on all nodes of the cluster 1 Select
55. y for cluster file systems and the Veritas File System VxFS package Topics covered in this chapter include Hardware overview Software components Preinstallation Prerequisites Installing the product Using the log files Verifying the configuration files Verifying agent configuration Synchronizing time on Cluster File Systems Configuring VCS VCS application failover services 10 Installing and configuring the product Hardware overview Hardware overview VxFS cluster functionality runs optimally on a Fibre Channel fabric Fibre Channel technology provides the fastest most reliable and highest bandwidth connectivity currently available By employing Fibre Channel technology SFCFS can be used in conjunction with the latest Veritas Storage Area Network SAN applications to provide a complete data storage and retrieval solution The figure below shows the configuration of a cluster file system on a Fibre Channel fabric with a disk array Figure 1 1 Four Node SFCFS Cluster Built on Fibre Channel Fabric Installing and configuring the product 11 Hardware overview Shared storage Shared storage can be one or more shared disks or a disk array connected either directly to the nodes of the cluster or through a Fibre Channel Switch Nodes can also have non shared or local devices on a local I O channel It is advisable to have usr var and other system partitions on local devices Fibre ch
56. y in which to mount the software disc and mount the disc using the appropriate drive name For example mkdir p dvdrom usr sbin mount F cdfs dev dsk c3t2d0 dvdrom Change directory to dvdrom cd dvdrom Run the installer command to install SFCFS The installer script uses ssh to communicate with remote nodes as default installer If you want to use rsh you must specify on the command line installer rsh From the Installation menu choose the 1 option for Install and select 6 the Veritas Storage Foundation Cluster File System You are prompted to continue with the installation Enter y to continue with the installation Do you wish to continue with the installation y n q y Enter one or more system names to install SFCFS Enter the system names separted by spaces on which to install SFCFS system01 system02 Enter the license key for system01 Enter a SFCFS license key for system01 Enter y to accept another license key or enter n to proceed Do you want to enter another license key for system02 y n q n Enter 1 2 or 3 to be installed on all systems Select the packages to be installed on all systems 1 3 q 2 Press Return to continue Press Return to continue Reboot all the nodes on which SFCFS is installed and proceed to Configuring the Components 22 Installing and configuring the product Configuring the Components Configuring the Components This sections describes the confi
57. you can type b back to return toa previous section of the installation procedure The back feature of the installation scripts is context sensitive so it returns to the beginning of a grouped section of questions If an installation procedure hangs use Control c to stop and exit the program There is a short delay before the script exits The following sample procedure is based on the installation of a Veritas Storage Foundation Cluster File System HA cluster with two nodes system01 and system02 If you are installing on standalone systems only some steps are unnecessary and these are indicated Default responses are enclosed by parentheses Press Return to accept defaults Note If you have obtained a Veritas product from an electronic download site the single product download files do not contain the installer installation script so you must use the product installation script to install the product For example if you download Veritas Cluster File System use the installsfcfs script instead of the installer script Installing and configuring the product 21 Installing the product To install the product 1 2 10 11 12 13 14 Log in as superuser Insert the appropriate media disc into your system s DVD ROM drive connected to your system Determine the block device file for the DVD drive ioscan fnC disk Make a note of the device file as it applies to your system Create a director
58. ystem is also supported without I O fencing enabled However without I O fencing enabled split brain scenarios can result in data corruption 20 Installing and configuring the product Installing the product m The host names of the cluster nodes m The device names of the network interface cards NICs used for the private networks among nodes m Establishing communication between nodes is required to install Veritas software from a remote system or to install and configure a cluster The node from which the installation utility is run must have permissions to run rsh remote shell or ssh secure shell utilities as root on all cluster nodes or remote systems See Veritas Storage Foundation and High Availability Solutions Getting Started Guide m Symantec recommends configuring the cluster with I O fencing enabled I O fencing requires shared devices to support SCSI 3 Persistent Reservations PR Enabling I O fencing prevents data corruption caused by a split brain scenario The Storage Foundation Cluster File System is supported without I O fencing enabled However without I O fencing enabled split brain scenarios can result in data corruption Installing the product The product installer is the recommended method to license and install the product The installer also enables you to configure the product verify preinstallation requirements and view the product s description At most points during an installation
59. ystem list of CVM service group in etc VRTSvcs conf config main cf 6 Add the new node to the CVM AutoStartList hagrp modify cvm AutoStartList system01 system02 system03 7 Node ID can be obtained from cvMNodetrd of etc VRTSvcs conf config main cf Add the new node system03 and its node ID to the cvm_clust resource hares modify cvm_clus CVMNodelId add system03 2 8 Write the new VCS configuration to disk haconf dump makero Adding and removing a node 57 Removing a node from a cluster 9 Put the CVM resources back online in the following order hagrp online cvm sys system01 hagrp online cvm sys system02 hagrp online cvm sys system03 10 Check the system status to see whether the new node is online hastatus sum SYSTEM STATE gt System A system01 A system02 A system03 GROUP STATE Group System State RUNNING RUNNING RUNNING Frozen 0 0 0 Probed AutoDisabled State B cvm system01 Y B cvm system02 Y B cvm system03 Y ONLINE ONLINE ONLINE 11 Add shared disk groups to the cluster configuration cfsdgadm add cfsdg system03 sw 12 Create a mnt on system03 and run the following commands cfsmntadm modify mnt add system03 rw Refer to cfsmntadm man page for more details 13 Use cfsmount command to cluster mount mnt back on all the nodes cfsmount mnt Removing a node from a cluster This section describes how to remove a node from a

Symantec 7 Computer Drive User Manual

Contents

Download Pdf Manuals

Related Search

Related Contents