Home

Sun Microsystems Sun StorEdge Availability Suite 3.2 User's Manual

1. Error Message From Meaning The volume s is already in use Kernel The data volume in the remote mirror set is already in use as a bitmap volume or a disk queue volume Use a different data volume Too many volumes given for update PITC iiadm ran out of memory Unable to access bitmap PITC During an enable operation iiadm tried to validate the bitmap device but could not get access to it Unable to access master volume PITC During an enable operation iiadm tried to validate the master device but could not get access to it Unable to access set in core PITC During a reset operation iiadm was unable to find the set listed in the kernel Unable to access shadow volume PITC During an enable operation iiadm tried to validate the shadow device but could not get access to it unable to access lt vol gt lt error gt RM The data volume cannot be accessed Verify that the data volume has been entered correctly and that it exists on the system unable to add lt host gt to RM An error has occurred preventing remote mirror from configuration storage lt error gt accessing the configuration storage while trying to update the options field Unable to add interface s to s Kernel The remote mirror software was unable to add host information to its configuration Verify that the system is not running low on memory unable to add lt set gt to RM An error has occurred that prevents the remote mirror configuration storage lt error
2. AaBbCc123 Book titles new words or terms Read Chapter 6 in the User s Guide words to be emphasized These are called class options Replace command line variables A You must be superuser to do this with real names or values To delete a file type rm filename 1 The settings on your browser might differ from these settings Preface vii Related Documentation Application Title Part Number Man pages Latest release information Installation and user System administration Cluster Configuration sndradm iiadm dsstat kstat svadm Sun StorEdge Availability Suite 3 2 Software Release Notes Sun Cluster 3 0 and Sun StorEdge Software Release Note Supplement Sun StorEdge Availability Suite 3 2 Software Installation Guide SunATM 3 0 Installation and User s Guide SunATM 4 0 Installation and User s Guide Sun Gigabit Ethernet FC AL P Combination Adapter Installation Guide Sun Gigabit Ethernet S 2 0 Adapter Installation and User s Guide Sun Gigabit Ethernet P 2 0 Adapter Installation and User s Guide Sun Enterprise 10000 InterDomain Networks User Guide Sun StorEdge Availability Suite 3 2 Remote Mirror Software Administration and Operations Guide Sun StorEdge Availability Suite 3 2 Point In Time Copy Software Administration and Operations Guide Sun Cluster 3 0 and Sun StorEdge Software Integration Guide Sun Enterprise 10000 InterDomain Network Configuration Guide N A
3. Meaning hostname tag exceeds CFG_MAX_BUF Import failed incorrect Solaris release requires lt release gt Instant Image volumes that are not in a device group which has been registered with SunCluster usage of C require Invalid flag s PITC PITC RM PITC Kernel Because CFG_MAX_BUF is 1k this message is not expected to be reported Could not import shadow volume Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_ESHUTDOWN The kernel module is in the process of shutting down the point in time software No new sets can be enabled DSW_EEMPTY One of the volume names is blank File a bug against iiadm DSW_EINUSE One of the volumes is already in use by another set DSW_EOPEN Failed to open one of the volumes DSW_EHDRBMP Could not read bitmap header Bitmap volume might be inaccessible or bad DSW_EOFFLINE One of the volumes is offline and cannot be made part of a set DSW_ERSRVFAIL Could not get access to the underlying volume The Solaris version that the remote mirror software is trying to run on is not supported check_resource_group The set that the software trying to enable has not passed the proper options to the kernel The sync or async value primary or secondary value or enable map set or clear map set value is not valid This is not something the user can contro
4. n cannot proceed gt ole n de n s s is already enabled SS gt Ss not already enabled Set Copy Parameters failed set lt shost gt lt svol gt neither sync nor async set lt shost gt lt svol gt not found in config Shadow and bitmap are the same device Kernel Kernel Kernel Kernel Kernel PITC RM PITC The user requested a forward sync operation for a remote mirror set which needs a reverse sync This occurs when a previous reverse sync does not complete successfully or because the primary volume was damaged and had to be replaced Issue a reverse sync for the set A set cannot contain more than one disk queue To add a new disk queue remove the old disk queue first The disk queue replace command can also be used to accomplish this task The remote mirror software could not determine the size of the secondary volume Verify that the secondary volume is not in an error state and can be accessed The user attempted to enable a remote mirror set using the same secondary volume and secondary host as a remote mirror set that is already enabled Specify a different secondary volume or secondary host for the new set The user has tried to do an operation on a set that is not enabled Verify that the proper set has been specified to sndradm and then verify that the set is enabled using sndradm i iiadm could not modify the copy units and delay values for the specified set P
5. A full copy update sync or reverse sync operation has been requested but the bitmap on the primary host cannot be accessed Verify that the bitmap volume is a valid volume and is not in an error state The user attempted to sync the remote mirror set or put the remote mirror set into logging mode while a previous sync request is being set up If the user issued a second sync request the user must first put the remote mirror set into logging mode and then issue the sync If the user issued a logging request the user must first wait for the sync request to finish setting up and then issue the logging request This stops the sync and places the remote mirror set into logging mode iiadm could not reset the set Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No set was specified to reset DSW_ENOTFOUND The specified set could not be found in the kernel EINVAL Bitmap volume is invalid DSW_ERSRVFAIL The kernel could not access one of the volumes DSW_EHDRBMP Could not set up bitmap header Chapter 3 Error Messages 33 TABLE 3 1 Error Message From Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Meaning Reverse sync needed cannot sync SS SS gt SS s SS SS gt queue s s already has a disk attached has invalid size ole n oe n
6. EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No set was specified to export from DSW_ENOTFOUND The specified set does not exist in the kernel DSW_EDEPENDENCY The set is not independent DSW_ERSRVFAIL Could not access the bitmap header to record the export operation iiadm ran out of memory 28 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Message From Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Meaning Failed to detach overflow volume Failed to move group in kernel failed to update autosync for SNDR set lt shost gt lt svol gt lt file gt contains no matching SNDR sets found matching ndr_ii entry for lt vol gt Group config does not match kernel Group does not exist or has no members Group list access failure group name is longer than lt MAX gt characters hostname is longer than lt MAX gt characters PITC PITC RM RM RM PITC PITC PITC RM RM iiadm had a problem detaching the overflow volume from a set Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No set was specified to detach from DSW_ENOTFOUND The set to detach from does not exist DSW_EODEP
7. MULTICAST IPv6 gt mtu 8252 index 2 inet6 fe80 a00 20ff fe8d f72c 10 Testing with ping Use the ping command to verify that the network interfaces can communicate and that IPv4 or IPv6 addressing is being used Issue this command from the primary host and secondary host to make sure communication is bidirectional This command also determines whether both hosts are using the same IP protocol IPv4 or This example checks the communication on host second atm IPv6 ping s second atm PING second atm 64 bytes 64 bytes 64 bytes 64 bytes from from from from second second second second 56 data atm atm atm atm bytes 192 1 92 192 192 OO WO O 201 2015 201 201 NN N icmp_seq 0 time 1 icmp_seq 1 time 0 icmp_seq 2 time 0 icmp_seq 3 time 0 ms ms ms ms Testing with snoop and atmsnoop Use the snoop or atmsnoop utility to make sure that the software is sending and receiving data during a copy or update operation Chapter 2 Remote Mirror Software Troubleshooting Tips 11 In the first example the command is issued from the primary host nws822 to the secondary host nws350 The network interface is hme0 and the port used by the rdc service is reported nws822 snoop d hme0 port rdc Using device dev hme promiscuous mode nws822 gt nws350 RPC C XID 3565514130 PROG 100143 VERS 4 PROC 8 nws350 gt nws822 RPC R 1 XID 35655
8. Neither of these commands if properly entered overwrites data Chapter 2 Remote Mirror Software Troubleshooting Tips 13 This example shows the newfs N command completing successfully newfs N dev vx rdsk rootdg test0 dev vx rdsk rootdg tony0 2048000 sectors in 1000 cylinders of 32 tracks 64 Sectors 1000 0MB in 63 cyl groups 16 c g 16 00MB g 7680 i g super block backups for fsck F ufs o b at 32 32864 65696 98528 131360 164192 197024 229856 262688 295520 328352 361184 394016 426848 459680 492512 525344 558176 591008 623840 656672 689504 722336 755168 788000 820832 853664 886496 919328 952160 984992 1017824 1048608 1081440 1114272 1147104 1179936 1212768 1245600 1278432 1311264 1344096 1376928 1409760 1442592 1475424 1508256 1541088 1573920 1606752 1639584 1672416 1705248 1738080 1770912 1803744 1836576 1869408 1902240 1935072 1967904 2000736 2033568 This example shows a typical error caused when the secondary host is not enabled or a disk or volume is inaccessible SNDR first atm dev vx rdsk rootdg vol11 dev vx rdsk rootdg bm11 second atm dev vx rdsk rootdg volll dev vx rdsk rootdg bm11 SNDR Error SNDR Could not open file second atm dev vx rdsk rootdg vol11 on remote node Aug 27 14 25 45 ns east 124 rdc NOTICE SNDR Interface 192 9 200 1 lt gt 192 9 200 2 Up Wrong Volume Set Name Specified When you first enable
9. a cluster 19 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Abort failed Allocation of bitmap device s failed Attach failed Bad host specified Bad set specified Bitmap in use PITC Kernel PITC RM PITC iiadn could not abort a copy or update operation on a set Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No set was specified DSW_ENOTFOUND The specified set does not exist Remote mirror could not use the bitmap requested to enable or resume a remote mirror set This could happen for one of these reasons e The bitmap is not accessible Verify that the specified bitmap volume exists and is accessible The volume requested for use as the bitmap is already in use Verify that the volume is not already in use as a remote mirror data volume or bitmap volume or as a point in time copy master volume shadow volume or bitmap volume The overflow volume could not be attached to the specified set Possible errors ENOMEM The kernel module ran out of memory DSW_EEMPTY No overflow volume was specified DSW_EINUSE The overflow volume is already being used by point in time copy software in a different capacity master shadow bitmap DSW_ENOTFOUND The set that the user is trying to attach to does not e
10. attempting to perform the operation The 1 lt hostname gt tag generated for the ctag to use to mark the remote mirror set as local is too long This results if a long host name is returned from the call to uname During an enable operation i iadm discovered that the master volume and the bitmap volume are the same During an enable operation i iadm discovered that the master volume and the shadow volume are the same During an enable operation i iadm discovered that the volume specified as the master is already in use as an overflow volume During an enable operation i iadm discovered that the master volume is a block device and not a character or raw device The master volume must exist in the dev directory tree iiadn ran out of memory Chapter 3 Error Messages 31 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Memory allocation failure PITC iiadm ran out of memory Must be super user to execute Kernel The user issued a remote mirror command but does not have superuser privileges All remote mirror commands require superuser privileges must specify full set details for RM The user attempted to enable a set using the shost svol enable command format All parameters for a set must be specified no matching SNDR sets found in RM The set specified for the command cannot be found in config the current configuration Check the set to ensure that i
11. on page 3 Troubleshooting Checklist This table shows the troubleshooting checklist and related sections TABLE 1 1 Troubleshooting Checklist Step For Instructions 1 Check for installation errors Sun StorEdge Availability Suite 3 2 Software Installation Guide 2 Check that dev ii is created after Sun StorEdge Availability Suite 3 2 Software reboot Installation Guide 3 Check the log file contents Checking Log Files on page 2 Checking Log Files You can check the status of the point in time copy software by examining two system log files m var opt SUNWesm ds log The var opt SUNWesm ds 1log file contains timestamped messages about the point in time copy software including error messages and informational messages For example Mar 05 15 56 16 scm scmadm cach nable succeeded Mar 05 15 56 16 ii iiboot resume cluster tag lt none gt m var adm messages The var adm messages file contains timestamped messages about the point in time copy software including general system error and informational messages For example Mar 5 16 21 24 doubleplay pseudo ID 129642 kern info pseudo device ii0 Mar 5 16 21 24 doubleplay genunix ID 936769 kern info ii0 is pseudo ii 0 Improving Performance If the Sun StorEdge Availability Suite software is used with a filesystem tuning the number of SV threads might produce better performance When a filesystem flushes its cache it generates many pa
12. 121 S 1011 Ack 2878301054 Seq 2333980449 Len 0 Win 41076 12 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 Correcting Common User Errors This section describes user errors encountered often when using the software a Enabled Software on Only One Host on page 13 a Volumes Are Inaccessible on page 13 m Wrong Volume Set Name Specified on page 14 Enabled Software on Only One Host New users sometimes forget to issue the sndradm e enable command on both the primary host and the secondary host Other problems include making a mistake when typing a disk or volume name or attempting to get access to a disk that does not allow access Volumes Are Inaccessible Verify that a volume or disk is accessible Confirm each volume is available on the primary and secondary host by using the dd 1M command to read a volume Issue the following command on the primary and secondary hosts for each primary secondary and bitmap volume dd if volume name of dev null count 10 10 0 records in 10 0 records out The result shows that the command was able to read 10 512 byte records indicating that the volume is accessible m Issue a newfs N command and see if an error results This command displays file system information and does not display an error if the disk or volume is accessible Note Be sure to enter these commands properly including the correct options
13. 2 Point in Time Copy Software Administration and Operations Guide m Sun StorEdge Availability Suite 3 2 Remote Mirror Software Administration and Operations Guide How This Book Is Organized This book includes the following chapters Chapter 1 helps to solve problems associated with the point in time copy software Chapter 2 helps to solve problems associated with the remote mirror software Chapter 3 provides an alphabetical list of error messages from all sources associated with the Sun StorEdge Availability Suite software vi Using UNIX Commands This document might not contain information on basic UNIX commands and procedures such as shutting down the system booting the system and configuring devices See the following for this information a Software documentation that you received with your system Solaris operating environment documentation which is at http docs sun com Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 Shell Prompts Shell Prompt C shell machine name C shell superuser machine name Bourne shell and Korn shell Ss Bourne shell and Korn shell superuser Typographic Conventions Typeface Meaning Examples AaBbCc123 The names of commands files Edit your login file and directories on screen Use 1s a to list all files computer output You have mail AaBbCc123 What you type when contrasted su with on screen computer output password
14. 817 2782 816 5128 817 2783 805 0331 805 6552 806 2385 805 2784 805 2785 806 4131 817 2784 817 2781 816 5127 806 5230 viii Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 Accessing Sun Documentation You can view print or purchase a broad selection of Sun documentation including localized versions at http www sun com documentation Contacting Sun Technical Support If you have technical questions about this product that are not answered in this document go to http www sun com service contacting Sun Welcomes Your Comments Sun is interested in improving its documentation and welcomes your comments and suggestions You can submit your comments by going to http www sun com hwdocs feedback Please include the title and part number of your document with your feedback Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide part number 817 3752 10 Preface ix x Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 CHAPTER 1 Point in Time Copy Software Troubleshooting Tips This chapter describes how to avoid or troubleshoot problems that might occur when using the point in time copy software The chapter includes the following topics Troubleshooting Checklist on page 1 Checking Log Files on page 2 Improving Performance on page 2 Safeguarding the VTOC Information
15. MESURE AUTORISEE PAR LA LOI APPLICABLE Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE A L APTITUDE A UNE UTILISATION PARTICULIERE OU A L ABSENCE DE CONTREFA ON LD Li Ca Adobe PostScript Contents Preface v Point in Time Copy Software Troubleshooting Tips 1 Troubleshooting Checklist 1 Checking Log Files 2 Improving Performance 2 Safeguarding the VTOC Information 3 Remote Mirror Software Troubleshooting Tips 5 Troubleshooting Checklist 6 Troubleshooting Log Files and Services 6 Checking Log Files 7 Checking the etc nsswitch conf File 8 Checking That the rac Service Is Running 8 If the dev rdc Link Is Not Created 9 Checking the Integrity of the Link 10 Testing with ifconfig 11 Testing with ping 11 Testing with snoop and atmsnoop 11 Correcting Common User Errors 13 Enabled Software on Only One Host 13 Volumes Are Inaccessible 13 Wrong Volume Set Name Specified 14 Accommodating Memory Requirements 16 3 Error Messages 19 iv Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 Preface Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide helps users solve common problems that might arise when using the Sun StorEdge Availability Suite 3 2 software Before You Read This Book To use the information in this document you must have thorough knowledge of the topics discussed in these books m Sun StorEdge Availability Suite 3
16. a set the remote mirror software assigns a default volume set name of shost sdev where shost is the secondary host name and sdev is the secondary volume name separated by a colon After enabling the software for a volume set you can use the shost sdev name for a volume set each time you issue an sndradm command instead of specifying the complete primary and secondary host volume and bitmap information for a volume set If you issue an sndradm command without specifying a volume set name the software executes the command on all configured volume sets Make sure that you specify the correct volume set on the command line 14 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 For example this command updates the volume on the secondary host calamari from the primary host volume sndradm un calamari dev vx rdsk rootdg tonyl1 To correctly display the volume set name use the sndradm p command on the primary host See To Find the Volume Set Name on page 16 Using the dsstat Command Incorrectly An administrator might use the dsstat 1M command instead of sndradm p to find the volume set name The output of both commands only appears to be similar dsstat m sndr Name sk rootdg tonyl sk rootdg bmapl Issuing the sndradm p Command on the Secondary Host Incorrectly An administrator might use the sndradm p command to find the volume set name correctly but issue the comman
17. copy operation DSW_E10 The kernel had a problem reading or writing one of the volumes in the set The user has issued a sync for a remote mirror set and the process could not be started for the set The system is probably low on resources Check the amount of memory and threads available The user issued a sync command and the primary host could not reach the secondary host Verify that the link from the primary to the secondary host is operating and that none of the ports used by the remote mirror are blocked by a firewall or in use by another application 24 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Message From Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Meaning Create overflow failed is does not match lt ctagl gt of ctag lt ctag2 gt disk group name volume lt vol gt ctags lt ctagl gt and lt ctag2 gt do not match device name is longer than lt MAX gt characters Disable failed PITC PITC An overflow volume couldn t be initialized Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No overflow volume was specified DSW_EINUSE The volume that was specified is already being used by the point in time copy software in another capacity DSW_EIO The kernel was unable to wr
18. for both the data volume and the bitmap volume 40 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Volumes are not in same disk group PITC iiadm detected that the master shadow and bitmap volumes are not all in the same cluster device group as required by the point in time copy software volume lt vol gt is not part of a RM The volume lt vol gt is not being managed by disk group please specify SunCluster resource ctag Wait failed PITC Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory EINTR The user interrupted the wait process DSW_EEMPTY No set was specified to wait for DSW_ENOTFOUND The specified set could not be found in the kernel DSW_ENOTLOCKED User tried to remove the PIDlock but the set is not locked DSW_EINUSE The user tried to remove the PIDlock but the set is locked by someone else Chapter 3 Error Messages 41 42 Related Error Messages The Solaris configuration administration utility cfgadm reports an error when it is used on systems where the Sun StorEdge Availability Suite software is installed The error occurs because a process does not suspend properly so that the cfgadm operation can proceed The error message has the following
19. format cfgadm Hardware specific failure lt operation gt failed could not suspend user process lt process_id gt You must stop the process manually perform the cfgadm operation and then start the process again Use the following commands to accomplish this 1 Quiesce I O to the sets using the following series of commands etc init d rdc stop etc init d ii stop etc init d sv stop etc init d scm stop 2 Issue the cfgadm command 3 Start I O to the sets using the following series of commands etc init d scm start etc init d sv start etc init d ii start etc init d rde start etc init d rdcfinish start Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003
20. valid If it has not wait for it to complete unsuccessfully before attempting to disable the disk queue disk service lt ctag gt is active on RM The remote mirror set being operated on is not active node lt hostname gt Please re issue on the current node in the cluster the command on that node disk service s is active on node PITC The iiadm command must be issued on the other s Please re issue the command node of the cluster The disk group that the user is on that node attempting to operate on is not active on the node where the iiadm command was issued diskq name is longer than lt MAX gt RM The device specified for the disk queue volume is too characters long for remote mirror to accept disk queue lt diskq2 gt does not match RM The user tried to enable a set into a group that has a lt diskql gt skipping set disk queue but the user specified a disk queue that does not match the group s disk queue diskqueue lt diskq gt is incompatible RM The user tried to enable a set into a group that has a disk queue but the user specified a disk queue that does not match the group s disk queue Disk queue s is already in use Kernel The volume for the disk queue being added to the set or group is already in use as a data volume bitmap volume or disk queue Use a different volume for the disk queue Disk queue s operation not Kernel The user attempted to perform disk queue possible set is in replicating maintenance
21. 14130 Success nws822 gt nws350 TCP D 121 S 1018 Ack 1980057565 Seq 2524537885 Len 0 Win 33304 Options lt nop nop tstamp 1057486 843038 gt nws822 gt nws350 RPC C XID 3565514131 PROG 100143 VERS 4 PROC 8 nws350 gt nws822 RPC R 4 XID 3565514131 Success nws822 gt nws350 TCP D 121 S 1018 Ack 1980057597 Seq 2524538025 Len 0 Win 33304 Options lt nop nop tstamp 1057586 843138 gt nws822 gt nws350 RPC C XID 3565514133 PROG 100143 VERS 4 PROC 8 nws350 gt nws822 RPC R 7 XID 3565514133 Success nws822 gt nws350 TCP D 121 S 1018 Ack 1980057629 Seq 2524538165 Len 0 Win 33304 Options lt nop nop tstamp 1057686 843238 gt nws822 gt nws350 RPC C XID 3565514134 PROG 100143 VERS 4 PROC 8 In the second example the link is ATM so the atmsnoop utility is appropriate etc opt SUNWconn atm bin atmsnoop d ba0 port rde device ba0 Using device dev ba promiscuous mode TRANSMIT VC 32 TCP D 121 S 1011 Syn Seq 2333980324 Len 0 Win 36560 RECEIVE VC 32 TCP D 1011 S 121 Syn Ack 2333980325 Seq 2878301021 Len 0 Win 36512 RANSMIT VC 32 TCP D 121 S 1011 Ack 2878301022 Seq 2333980325 Len 0 Win 41076 RANSMIT VC 32 RPC C XID 1930565346 PROG 100143 VERS 4 PROC 11 RECEIVE VC 32 TCP D 1011 S 121 Ack 2333980449 Seq 2878301022 Len 0 Win 36450 RECEIVE VC 32 RPC R 4 XID 1930565346 Success TRANSMIT VC 32 TCP D
22. C iiadm ran out of memory Out of memory creating lookup table PITC iiadm ran out of memory 32 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Overflow list access failure Overflow volume is already in an InstantImage group overflow volume name must start with dev Overflow volume not in a disk group Overflow volume not in same disk group as shadow set members Recovery bitmaps not allocated Request not serviced s is currently being synced Reset shadow failed PITC PITC PITC PITC PITC Kernel Kernel PITC iiadm could not get a list of overflow volumes from the kernel Possible errors EFAULT The kernel module tried to read out of bound File a bug against iiadm ENOMEM The kernel module ran out of memory During an attach operation i iadm determined that the volume the user specified to attach to a compact dependent set is already in use by another set as a master shadow or bitmap volume The overflow volume must exist in the dev directory tree The user attempted to attach a volume to a set but the set s volumes are part of a cluster device group and the overflow volume is not The user attempted to attach a volume to a set but the set s volumes are in a different cluster device group than the overflow volume
23. ENDENCY The overflow volume is still in use by the set that the user attempted to detach it from DSW_ERSRVFAIL Could not access the bitmap header to record the overflow detach DSW_EHDRBMP Could write the bitmap header to record the overflow detach Could not move the set from one group to another Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY iiadm failed to fill in the group name This is a bug Autosync could not be activated for the set when going from logging mode to replicating mode The configuration file specified with the f switch contains no valid remote mirror sets There is already an ndr_ii entry for this remote mirror set The groups in dscfg are different from the groups in the kernel An invalid group was specified for a group based command For example copy update and abort Could not retrieve the list of sets belonging to a group from the kernel Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory The group name specified is too long for the remote mirror software to accept The host name specified is too long for the remote mirror software Chapter 3 Error Messages 29 TABLE 3 1 Error Message From Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued
24. S KYUN microsystems Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide Sun Microsystems Inc www sun com Part No 817 3752 10 December 2003 Revision 51 Submit comments about this document at http www sun com hwdocs feedback Copyright 2003 Sun Microsystems Inc 4150 Network Circle Santa Clara California 95054 U S A All rights reserved Sun Microsystems Inc has intellectual property rights relating to technology embodied in this product In particular and without limitation these intellectual property rights may include one or more of the U S patents listed at http www sun com patents and one or more additional patents or pending patent applications in the U S and in other countries This document and the product to which it pertains are distributed under licenses restricting their use copying distribution and decompilation No part of the product or of this document may be reproduced in any form by any means without prior written authorization of Sun and its licensors if any Third party software including font technology is copyrighted and licensed from Sun suppliers Parts of the produet may be derived from Berkeley BSD systems licensed from the University of California UNIX is a registered trademark in the U S and in other countries exclusively licensed through X Open Company Ltd Sun Sun Microsystems the Sun logo AnswerBook2 docs sun com Sun StorEdge and Solaris are tr
25. Solaris format 1M utility m Various software components such as dd 1M backup utilities point in time copy software and remote mirror software can copy the VTOC of one volume to another volume if that volume includes cylinder 0 in its mapping a Ifthe VTOC of the source and destination volumes are not identical some type of data loss might occur This data loss might not be detected initially but can be detected later when other utilities are used like sck 1M or when the system is rebooted When first configuring and validating volume replication save copies of all affected devices VTOCs using the prt vtoc 1M utility The fmthard 1M utility can be used to restore them later if necessary m When using volume managers like SVM and VxVM copying between individual volumes created under these volume managers is safe VTOC issues are avoided because the VTOC is excluded from volumes created by these volume managers a When formatting individual partitions on a raw device for all partitions except the backup partition make sure the partitions do not map cylinder 0 which contains the VTOC When using raw partitions as volumes you are the volume manager and you need to exclude the VTOC from partitions that you configure m When formatting the backup partition of a raw device make sure that the physical geometries of the source and destination devices are identical Partition 2 by default maps all cylinders under the backup p
26. ademarks or registered trademarks of Sun Microsystems Inc in the U S and in other countries All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc in the U S and in other countries Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems Inc The Adobe logo is a registered trademark of Adobe Systems Incorporated U S Government Rights Commercial use Government users are subject to the Sun Microsystems Inc standard license agreement and applicable provisions of the FAR and its supplements Products covered by and information contained in this service manual are controlled by U S Export Control laws and may be subject to the export or import laws in other countries Nuclear missile chemical biological weapons or nuclear maritime end uses or end users whether direct or indirect are strictly prohibited Export or reexport to countries subject to U S embargo or to entities identified on U S export exclusion lists including but not limited to the denied persons and specially designated nationals list is strictly prohibited DOCUMENTATION IS PROVIDED AS IS AND ALL EXPRESS OR IMPLIED CONDITIONS REPRESENTATIONS AND WARRANTIES INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE OR NON INFRINGEMENT ARE DISCLAIMED EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID Copyrigh
27. artition If identical device sizing is not possible make sure that the source backup partition is smaller than the destination partition and that the destination partition does not map cylinder 0 Chapter 1 Point in Time Copy Software Troubleshooting Tips 3 4 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 CHAPTER 2 Remote Mirror Software Troubleshooting Tips This section describes how to avoid or troubleshoot any problems might occur when using the remote mirror software The following topics are described a Troubleshooting Checklist on page 6 m Troubleshooting Log Files and Services on page 6 m Checking the Integrity of the Link on page 10 a Correcting Common User Errors on page 13 Note The Sun StorEdge Availability Suite 3 2 Remote Mirror Software Administration and Operations Guide describes the dsstat and scmadm commands These commands are useful for displaying information about remote mirror and point in time copy software volumes Troubleshooting Checklist This table shows the troubleshooting checklist and related sections TABLE 2 1 Troubleshooting Checklist Step For Instructions 1 Check for installation errors 2 Check that dev rdc is created after reboot 3 Check that the sndrd daemon is running 4 Check the log file contents 5 Check that the etc nsswitch conf file is configured correctly 6 Check the in
28. d from the secondary host incorrectly Depending on which host you issue the command from the output differs For example when issued from the primary host the command shows the correct volume set name of calamari dev vx rdsk rootdg tonyl sndradm p dev vx rdsk rootdg tonyl gt calamari dev vx rdsk rootdg tonyl When issued from the secondary host the command shows the incorrect volume set name The name ariell dev vx rdsk rootdg tony0 is the name of the primary host and its volume set sndradm p dev vx rdsk rootdg tony0 lt ariell dev vx rdsk rootdg tony0 Chapter 2 Remote Mirror Software Troubleshooting Tips 15 v To Find the Volume Set Name 1 If you are unsure of the volume set name type the following command from the primary host sndradm p dev vx rdsk rootdg tonyl gt calamari dev vx rdsk rootdg tonyl Running Startup Script Out of Order The scripts to configure the network interface must run before the Availability Suite s startup script If the startup script runs first the following error is reported when the primary host is restarted Error Neither lt phost gt nor lt shost gt is local The scripts are out of order if the configuration file includes the set information but the sndradm i and sndradm P commands do not report the set The startup process runs the startup scripts in alphabetical order The Availability Suite startup script is SOOtrdc Move
29. eaning Enable failed Enable failed can t tidy Enable pending on s gt again later up cfg 2 ss try enabling disk queue on an SNDR secondary is not allowed Export failed Failed to allocate memory lt diskq gt PITC PITC Kernel RM PITC PITC Could not enable volume Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_ESHUTDOWN The kernel module is in the process of shutting down the point in time copy software No new sets can be enabled DSW_EEMPTY One of the volumes names master shadow bitmap is blank File a bug against iiadm DSW_EINUSE One of the volumes master shadow bitmap is already being used by another set DSW_EOPEN Failed to open one of the volumes master shadow or bitmap DSW_EHDRBMP Could not read bitmap header Bitmap volume might be inaccessible or bad DSW_EOFFLINE One of the volumes master shadow bitmap is offline and cannot be made part of a set DSW_ERSRVFAIL Could not get access to the underlying volume master shadow bitmap Could not enable volume and could not remove new entry from configuration file A previous enable operation of a set is still processing when another enable operation is attempted A disk queue can be added only to a set on the primary site Could not export the shadow of the specified set Possible errors
30. econdary volume if it is already in use as a secondary volume by another remote mirror set Cannot enable master volume PITC iiadm attempted to put the master volume under SV control during an enable operation but failed Cannot enable shadow volume PITC iiadm attempted to put the shadow volume under SV control during an enable operation but failed Cannot reverse sync s s lt Kernel A reverse sync has been requested on a set that is in s s set is in queuing mode queuing mode Put the set into logging mode and then the issue the reverse sync command for the set Change request denied don t Kernel A request was sent from one remote mirror host to the understand request version other and the version of the software was not understood by the receiving host Verify that both hosts are running compatible versions of the remote mirror software Change request denied volume Kernel The user asked to sync a remote mirror set and the mirror is up secondary host has refused the sync event changing queue parameters may only RM The queue parameters for the async I O queue both be done on a primary SNDR host memory based and disk based can be changed only on the primary host Cluster list access failure PITC iiadm was unable to retrieve a list of cluster groups from the kernel Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory cluster name is l
31. ed by the point in time copy software bitmap lt vol gt is in use by SCSI RM The volume requested as a remote mirror bitmap Target Emulation volume is already configured for use by STE SCSI Target Emulation Bitmap volume is already an PITC The volume that was specified as the bitmap volume overflow volume during an enable operation is already being used by another set as an overflow volume Bitmap volume is not a character PITC The volume that was specified as the bitmap volume device during an enable operation is a block device and not a character device bitmap volume name must start with PITC The volume that was specified as the bitmap volume dev during an enable or import operation does not start with dev both lt phost gt and lt shost gt are local RM The set specifies the same host as both primary and secondary The primary and secondary hosts must be different Cannot add s s gt s s to Kernel The set being added to a group cannot be added group s Typically this happens because the set being added is of a different type sync or async than the sets already in the group Cannot change disk queue s Must Kernel A disk queue cannot be added to or removed from a be in logging mode set if it is not in logging mode Place the set into logging mode and then attempt to add or remove the disk queue Can not check volume against mount PITC The iiadm operation attempted to determine whether table a volume on t
32. gt software from accessing the configuration storage database when trying to enable lt set gt unable to allocate memory for RM System is running low on available memory cluster tag unable to allocate pair_list array RM System is running out of memory for lt rdc_max_sets gt sets Unable to allocate lt X gt bytes for RM The system is running low on available memory bitmap file lt vol gt unable to ascertain environment PITC iiadm tried to determine whether the host is part of a cluster but was unable to do so unable to ascertain environment RM The remote mirror software was not able to determine if the system is part of a SunCluster configuration unable to clear autosync value in RM Autosync could not be turned off for the remote mirror config for SNDR set lt shost gt lt svol gt set in the configuration file because of an error writing to the configuration database unable to determine disk group name PITC iiadm tried to find out to which cluster device group a for Ss volume belonged but was unable to do so Chapter 3 Error Messages 37 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning unable to determine hostname RM Could not determine the host name of the system lt host gt unable to determine IP addresses RM The IP address for either the primary host or the for either host lt phost gt or host secondary host could not be de
33. he command line belongs to a mounted filesystem The test for this has failed Chapter 3 Error Messages 21 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning cannot find SNDR set RM Remote mirror set cannot be found in the configuration lt shost gt lt svol gt in config database The set is not configured Check the entry for errors Cannot reconfig s s to s s Kernel An operation has been requested that requires the Must be in logging mode remote mirror set to be in logging mode Put the remote mirror set into logging mode and then request the reconfiguration Cannot reconfigure sync async on Kernel The user attempted to reconfigure the mode of a set members of a group that is in a group The set must be removed from the group before its mode can be reconfigured cannot replace disk queue lt diskql gt RM Look at check_diskqueue cfg qvol group_arg with lt diskq2 gt can not start reverse sync as a RM The primary volume has a file system mounted on it file system is mounted on lt vol gt Unmount the file system on the primary volume and then issue the reverse sync command can not start sync as SNDR set RM The remote mirror set must be in logging mode on the lt shost gt lt svol gt is not logging primary host before a sync can be started If the primary site is in an error state fix the error and then place the primary site into logging mode for
34. he configuration database Verify that the set is entered properly and matches a set returned by the sndradm i command Unable to fork PITC iiadm ran out of process space unable to get maxsets value from RM The remote mirror software failed to read the kernel sndr_max_sets value from the usr kernel drv rdc conf file unable to get set status before RM The remote mirror software could not get the kernel reconfig operation configuration Unable to initialize the kernel Kernel The remote mirror software could not initialize a thread set kernel thread Verify that the system is not running low on memory 38 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning unable to obtain unique set id for RM Lookup of the set ID in the configuration database for lt shost gt lt svol gt this set has failed Unable to open bitmap file lt vol gt RM The volume specified for the bitmap could not be opened The volume might not exist or is already in use by another program Unable to open s s Kernel The data volume for the set on the local host cannot be opened by the remote mirror software for one of the following reasons The volume requested does not exist or is inaccessible e The volume is already in use as a remote mirror or point in time copy bitmap Unable to parse config file PITC iiad
35. iiadm tried to perform svadm e on a volume See the ds log file for more information Target of copy update is mounted PITC If the shadow in a master to shadow copy or update unmount it first operation or the master in a shadow to master copy or update operation is mounted it cannot be copied to The bitmap s is already in use Kernel The bitmap requested for the remote mirror set being enabled is already in use as a bitmap for another set Enable the set and specify a different volume for the bitmap The remote state of s s gt Kernel The user attempted to do a sync or reverse sync while s s prevents this operation the secondary volume is mounted First unmount the secondary volume and then issue the sync or reverse sync request The state of s s gt s s Kernel The set the user is attempting to sync is part of an prevents this operation advanced configuration The state of one of the other sets in the configuration prevents this sync from occurring for one of the following reasons e Another set in a one to many configuration is currently doing a reverse sync e A reverse sync is being requested for a set in a one to many configuration and at least one of the other sets is not in logging mode e A set is already syncing 36 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued
36. ing Update failed PITC One or more volumes in a group copy or update command failed Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory EINVAL User is performing a shadow to master copy but two or more shadows are of the same master DSW_E10 The kernel had a problem reading or writing one of the volumes in the set Update of s failed PITC Same as Update failed meaning but for a specific volume Update of config failed PITC During a reset iiadm tried to update the configuration file to indicate that the set is now online but failed to do so Version failed PITC iiadm queried the kernel for the version of the code it was running but failed Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm lt vol gt is already configured as an RM The master shadow or bitmap volume in the ndr_ii SNDR bitmap entry is already configured as a remote mirror bitmap volume lt vol gt is not a character device RM The volume specified is not a character device Volume is not in an InstantImage PITC A volume specified on the command line does not group belong to a point in time copy set Volume is part of an InstantImage PITC The user attempted to initialize an overflow volume group but the volume is in use as a master shadow or bitmap volumes and bitmaps must not match RM The same volume has been specified
37. is 0 0 65535 0 LISTEN rde Far 0 0 65535 0 LISTEN rdg RAR 0 0 65535 0 LISTEN If the dev rdc Link Is Not Created Note Although other applications make entries in the files described in this section you can edit the files to correct these problems Always make a backup copy of a file before editing it The dev rdc pseudo link might not be created for the following reasons m The etc devlink tab file is missing an entry for the dev rdc pseudo link This example shows a valid entry grep rdc etc devlink tab type ddi_pseudo name rdc Chapter 2 Remote Mirror Software Troubleshooting Tips 9 m The etc name_to_ma jor file is missing an entry for the dev rdc pseudo link This example shows a valid entry the number following rdc can be any number grep rdc etc name_to_major EdC 239 m The usr kernel drv rdc conf file is incomplete This example shows a valid entry grep pseudo usr kernel drv rdc conf name rdc parent pseudo Checking the Integrity of the Link After you determine that the rdc service is ready check the integrity of the TCP IP link As part of the installation process you entered the primary and secondary host names and IP addresses of the machines where the software is installed in the etc hosts file Make sure this file contains the same information on the primary and secondary hosts because the software is bidirectional The software uses these ho
38. ite to the volume DSW_ERSRVFAIL The kernel was unable to access the volume The cluster resource tag given for a set is different from the configured cluster resource tag The set specified contains both the syntax C ctagl and C ctag2 This error indicates the values specifed for ctag1 and ctag2 do not match The name for the primary data volume primary bitmap volume secondary data volume or secondary bitmap volume is too long for the remote mirror software iiadm was unable to disable one or more sets Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No set was specified DSW_ENOTFOUND The specified set could not be found in the kernel DSW_EDEPENDENCY An attempt was made to disable an independent set but the set is not yet independent DSW_EOPACKAGE Another package for example RDC told the point in time copy software not to disable the set DSW_ERSRVFAIL The kernel could not access the shadow or bitmap volumes Chapter 3 Error Messages 25 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Disable pending on diskq s try Kernel A request to disable the disk queue is already in again later progress Verify that the previous request has completed successfully If it has this request is no longer
39. l 30 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Message From Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Meaning Join failed List failed local tag lt ltag gt is longer than CFG_MAX_BUF x Master and bitmap are the same device Master and shadow are the same device Master volume is already an overflow volume Master volume is not a character device master volume name must start with dev Memory allocation error PITC PITC RM PITC PITC PITC PITC PITC PITC Could not join shadow volume back to the set Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY A volume was missing on the command line DSW_ENOTFOUND The set could not be found in the kernel DSW_ENOTEXPORTED The set to which the user is trying to join the volume is not in the exported state DSW_EINVALBMP The bitmap volume is not big enough to handle the master volume DSW_ERSRVFAIL The kernel module was unable to access the bitmap volume DSW_EOPEN The kernel module was unable to access the shadow volume Could not get a list of volumes from the kernel Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory when
40. luded in the etc nsswitch conf file ensure that files is placed before nis nisplus ldap dns or any other service the machine is using For example for systems using the NIS naming service the file must include hosts files nis services files nis If you need to edit the etc nsswitch conf A file use a text editor After editing the file shut down and restart your machine etc shutdown y g 0 i 6 Checking That the rdc Service Is Running When the remote mirror software loads it adds an entry into the etc services file for the rdc service Search for an entry that looks like this grep rdc etc services rdc 121 tcp SNDR server daemon Use the rpcinfo and netstat commands to check the service m rpcinfo rpcinfo T tcp hostname 100143 program 100143 version 6 ready and waiting 8 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 where T tcp specifies the transport that the service uses hostname is the name of the machine where the service is running If the service is not running this message is displayed rpcinfo RPC Program not registered If you see this message it is possible that the etc nsswitch conf services entry is incorrectly configured See Checking the etc nsswitch conf File on page 8 netstat This messages shows that the service is running netstat al grep rdc rdc
41. lume is already configured PITC During an enable operation iiadm discovered that the volume specified as the shadow volume is already in use as a shadow for a different master volume Shadow Volume is currently mounted PITC iiadm could not disable the set because the shadow and dependent on the master volume volume is mounted and part of an independent set and the copy has not yet completed Shadow volume is mounted unmount PITC During an enable operation ii adm discovered that the it first volume specified as the shadow volume is currently mounted Shadow volume is not a character PITC During an enable operation ii adm discovered that the device shadow volume is a block device and not a character or a raw device shadow volume name must start with PITC The shadow volume must exist in the dev directory dev tree Shadow volume not in a disk group PITC During an attach operation i iadm determined that the set to which the user is attaching an overflow volume is neither in a cluster device group nor in a local 1 hostname group Size of Primary s s s must be Kernel The user set up a remote mirror set in which the less than or equal to size of secondary volume is smaller than the primary volume Secondary s s s The secondary volume must be equal to or larger than the primary volume but the size is not checked until a sync is issued for the remote mirror set The user must either disable the remote mirror set on the primary ho
42. m attempted to access the configuration file dscfg but could not do so This indicates a problem with the configuration and might require either restoring the configuration file or reinstalling the Sun StorEdge Availability Suite software Unable to read the bitmap file RM Bitmap could not be read correctly read returned lt X gt instead of lt Y gt Unable to register s Kernel The remote mirror software could not use the volume requested Verify that the volume exists is accessible and is not in an error state unable to remove lt set gt from RM An error has occurred that prevents the remote mirror configuration storage lt error gt software from removing the set from the configuration database Unable to set locking on the PITC iiadm needs to lock the configuration file for reading configuration or writing but was unable to do so unable to store unique set id for RM The set ID for the set being enabled could not be lt shost gt lt svol gt added to the configuration database unable to update autosync value in RM Autosync could not be turned on for the remote mirror config for SNDR set lt shost gt lt svol gt set in the configuration database because of an error writing to the configuration database Unexpected return from PITC File a bug for iiadm check_cluster Chapter 3 Error Messages 39 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Mean
43. on a set while the set is replicating mode Disk queue does not exist for set Kernel The user attempted to perform disk queue s S gt s s maintenance on a set that does not have a disk queue disk queue lt diskq gt is incompatible RM The user tried to enable a set into a group that has a with existing queue disk queue but the user specified a disk queue that does not match the group s disk queue disk queue lt diskq gt is not in disk RM The user tried to enable a disk queue that does not group lt ctag gt reside in the same cluster resource group in which the volume and bitmap reside Disk queue operations on Kernel An attempt to enable a sync set with a disk queue or synchronous sets not allowed to add a disk queue to a sync set has been made Sync sets cannot have disk queues attached to them 26 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning disk queue volume lt vol gt must not RM The disk queue volume specified for the match any primary SNDR volume or reconfiguration operation is already in use by the bitmap remote mirror software as a data volume or bitmap volume don t understand shadow type PITC The iiadm e command expected dep or ind DSWIOC_LISTLEN PITC iiadm tried to get a count of the number of sets recognized by the kernel but failed This i
44. onger than lt MAX gt RM The cluster resource tag is too long for the remote characters mirror software to accept config error neither lt hostl gt nor RM The current host is neither the primary nor secondary lt host2 gt is localhost host for the remote mirror set Verify that the hostname of the system has not been changed since the remote mirror set was enabled Chapter 3 Error Messages 23 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Copy failed Could not create rdc_config process Could not open file remote node PITC Kernel Kernel A copy or update operation could not be initiated Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No set was specified on the command line DSW_ENOTFOUND The specified set could not be found in the kernel DSW_ECOPYING A copy is already in progress or a partial copy was aborted and the new copy request does not match the aborted one DSW_EOFFLINE One or more volumes in the set is offline DSW_EINCOMPLETE The shadow is exported or imported DSW_EINUSE The set is PID locked by another process DSW_ERSRVFAIL The kernel was unable to access one or more volumes in the set DSW_EOPACKAGE Another package for example RDC told the point in time copy software not to perform the
45. orEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 CHAPTER 3 Error Messages Error messages during the installation process are described in the Sun StorEdge Availability Suite 3 2 Software Installation Guide Solaris error messages related to the Sun StorEdge Availability Suite software are described in TABLE 3 1 lists Sun StorEdge Availability Suite 3 2 error messages in alphabetical order The error messages come from the following sources a PITC From the point in time copy software This software used to be called Sun StorEdge Instant Image software m RM From the remote mirror software m Kernel From the kernel software Kernel messages might not be printed on the screen but are usually written to var opt SUNWesm ds 1log or to the system console and recorded in var adm messages TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Error Message s is not a From Meaning valid number PITC iiadm was expecting a number to be on the command line This happens when the copy parameters option P is used but the arguments for units and delay are not valid numbers C lt ctag gt does not match disk group PITC iiadm determined that the cluster resource group that contains the volumes does not match the cluster tag given as an argument to the C option C is valid only in a Sun Cluster PITC The user attempted to specify a cluster tag when not in
46. ossible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No set was specified for the parameters DSW_ENOTFOUND The specified set was not found in the kernel EINVAL The delay or units value is out of range The mode of the set specified in the configuration file is incorrect This occurs whenthe user inserts a set into the configuration manually using dscfg with an incorrect mode tag The specified set is not in the current configuration Use sndradm i to verify that the set is in the current configuration During an enable operation i iadm discovered that the shadow volume and the bitmap volume are the same 34 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Shadow group s is suspended PITC The user attempted to perform a copy or update operation on a group with one or more suspended sets The s parameter identifies the first set found in the group that is suspended Shadow group suspended PITC The user attempted to perform a copy or update operation on a suspended set Shadow volume is already an PITC During an enable operation ii adm discovered that the overflow volume volume specified as the shadow volume is already in use as an attached overflow volume Shadow vo
47. rallel write operations The SV s default setting of 32 threads could produce a bottleneck You can increase the number of SV threads The maximum number of threads allowed is 1024 Note Each thread consumes 32k of memory The sv_threads value is in the usr drv conf sv conf file Because the file is read when a module loads changes to the sv_threads value do not take effect until you reboot the system 2 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 Safeguarding the VTOC Information Caution When creating shadow volume sets do not create shadow or bitmap volumes using partitions that include cylinder 0 Data loss might occur The Solaris system administrator must be knowledgable about the virtual table of contents VTOC that is created on raw devices by the Solaris operating system The creation and updating of a physical disk s VTOC is a standard function of the Solaris operating system Software applications like the Sun StorEdge Availability Suite the growth of storage virtualization and the appearance of SAN based controllers have made it easy for an uninformed Solaris system administrator to allow a VTOC to be altered inadvertently Altering the VTOC increases the possibility of data loss Remember these points about the VTOC a A VIOC is a software generated virtual table of contents based on the geometry of a device and written to the first cylinder of that device by the
48. rques d pos es de Sun Microsystems Inc aux Etats Unis et dans d autres pays Toutes les marques SPARC sont utilis es sous licence et sont des marques de fabrique ou des marques d pos es de SPARC International Inc aux Etats Unis et dans d autres pays Les produits protant les marques SPARC sont bas s sur une architecture d velopp e par Sun Microsystems Inc Ce produit est soumis la l gislation am ricaine en mati re de contr le des exportations et peut tre soumis la r glementation en vigueur dans d autres pays dans le domaine des exportations et importations Les utilisations ou utilisateurs finaux pour des armes nucl aires des missiles des armes biologiques et chimiques ou du nucl aire maritime directement ou indirectement sont strictement interdites Les exportations ou r exportations vers les pays sous embargo am ricain ou vers des entit s figurant sur les listes d exclusion d exportation am ricaines y compris mais de mani re non exhaustive la liste de personnes qui font objet d un ordre de ne pas participer d une fa on directe ou indirecte aux exportations des produits ou des services qui sont r gis par la l gislation am ricaine sur le contr le des exportations et la liste de ressortissants sp cifiquement d sign s sont rigoureusement interdites LA DOCUMENTATION EST FOURNIE EN L TAT ET TOUTES AUTRES CONDITIONS DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES DANS LA
49. s a bug in iiadm Dual copy failed offset s Kernel A sync or reverse sync was started but could not complete for one of the following reasons e The user issued a logging request manually causing the sync or reverse sync to abort The network link between the primary and the secondary host failed causing the sync or reverse sync to stop e The primary or secondary volume encountered an error and the remote mirror software was unable to read or write to the volume e In a Sun Cluster environment a failover of the resource group might have been issued causing the sync ot reverse sync to be stopped Duplicate volume specified PITC A command that can take multiple shadow volume names like update or copy detected that a shadow volume was specified more than once either lt phost gt lt pfile gt or RM The command was issued on a system that is not the lt shost gt lt sfile gt is not local primary or secondary host for the specified set Verify the command is being issued on the proper system either lt phost gt lt pvol gt or lt shost gt lt svol gt is not local Empty string Kernel An enable or resume request for a remote mirror set was issued but a required field was not filled in when the request was received in the kernel This is not something the user can control Chapter 3 Error Messages 27 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From M
50. st and decrease the size of the primary volume so that it is equal to or smaller than the secondary volume or disable the remote mirror set on the secondary host and increase the size of the secondary volume so that it is equal to or larger than the primary volume SNDR can t stat lt vol gt RM The volume specified cannot be accessed by the system Chapter 3 Error Messages TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning SNDR set does not have a disk RM Set does not have a disk queue attached when queue attempting either a queue remove operation or a queue replace operation SNDR The volume lt vol2 gt has been RM The user attempted to enable a set in which the configured previously as lt voll gt volume was already enabled but with a different Re enter command with the latter name name SNDR lt voll gt and lt vol2 gt refer RM A physical device cannot be used for more than one to the same device volume in the remote mirror set Stat failed PITC Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No volume name was specified DSW_ENOTFOUND Could not find specified volume in kernel SV disable failed PITC iiadm tried to perform svadm d on a volume See the ds log file for more information SV enable failed PITC
51. sts stalls the primary host from issuing any more requests This hard limit is applied only to the number of possible outstanding requests not the size of their payload For example if the I O consists of 4 Kbyte writes with six groups the total memory requirement could be 4Kbyte x 6 x 64 1536 Kbyte However with an I O size of 1 Mbyte this requirement could rise to 1 Mbyte x 6 x 64 384 Mbyte You need to be aware of this requirement to avoid exhausting the memory of the secondary host The most common symptom of secondary memory exhaustion is the volume sets changing to logging mode The RPC requests fail when memory is low To avoid the problem change the number of asynchronous threads for some of the groups from the default of two to one This forces the earlier version 3 1 behavior with no extra memory requirement on the secondary host The need for extra memory is most common with large I O on a high latency link When selecting groups to use a single thread your best choices are those groups with a good network connection Such groups are most likely to fill the secondary queue and they derive the least benefit from multiple threads Users upgrading from Sun StorEdge Availability Suite 3 1 software must be aware that the default number of asynchronous threads is two not one Out of order requests can occur and extra memory might be required on the secondary host Chapter 2 Remote Mirror Software Troubleshooting Tips 17 18 Sun St
52. sts to transfer data Simple tests to check link integrity include the following m Use the telnet or rlogin commands to connect to the hosts m Use the ifconfig command to check your network interfaces m Use the ping command to make sure packets are being transmitted m Use the snoop or atmsnoop commands to make sure the software is copying data Note The dsstat command displays volume information The sndradm H command displays link I O statistics 10 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 Testing with ifconfig Use the ifconfig command to make sure that the network interface is configured and running correctly This example output shows all the interfaces that are configured and running ifconfig a ba0 flags 100 100 flags 100 hme0O flags 10 100 flags 200 hme0 flags 20 ether inet 1 ether inet6 0843 lt UP BROADCAST RU inet 192 9 201 10 netmask 8 0 20 af 8e d0 0849 lt UP LOOPBACK RU NING MULTICAST IPv4 gt mtu 9180 index 1 f fffff00 broadcast 192 2 201 255 ING MULTICAST IPv4 gt mtu 8232 index 2 27 0 0 1 netmask 000000 00843 lt UP BROADCAST RU inet 192 9 201 124 netmask 8 Or 202802 ETI 2C 0849 lt UP LOOPBACK RU 221 128 00841 lt UP RUNNING MULTICAST IPv6 gt mtu 1500 index 3 ether 8 0 20 8d f7 2c NNING MULTICAST IPv4 gt mtu 1500 index 3 f fffff00 broadcast 192 9 200 255 ING
53. t 2003 Sun Microsystems Inc 4150 Network Circle Santa Clara Californie 95054 Etats Unis Tous droits r serv s Sun Microsystems Inc a les droits de propri t intellectuels relatants a la technologie qui est d crit dans ce document En particulier et sans la limitation ces droits de propri t intellectuels peuvent inclure un ou plus des brevets am ricains num r s a http www sun com patents et un ou les brevets plus suppl mentaires ou les applications de brevet en attente dans les Etats Unis et dans les autres pays Ce produit ou document est prot g par un copyright et distribu avec des licences qui en restreignent l utilisation la copie la distribution et la d compilation Aucune partie de ce produit ou document ne peut tre reproduite sous aucune forme par quelque moyen que ce soit sans l autorisation pr alable et crite de Sun et de ses bailleurs de licence s il y ena Le logiciel d tenu par des tiers et quicomprend la technologie relative aux polices de caract res est prot g par un copyright et licenci par des fournisseurs de Sun Des parties de ce produit pourront tre d riv es des syst mes Berkeley BSD licenci s par l Universit de Californie UNIX est une marque d pos e aux Etats Unis et dans d autres pays et licenci e exclusivement par X Open Company Ltd Sun Sun Microsystems le logo Sun AnswerBook2 docs sun com Sun StorEdge et Solaris sont des marques de fabrique ou des ma
54. t r first atm dev vx rdsk rootdg vol2 dev vx rdsk rootdg bm2 second atm dev vx rdsk rootdg vol2 dev vx rdsk rootdg bm2 Successful Aug 20 19 13 58 sndr sndrboot r first atm dev vx rdsk rootdg vol3 dev vx rdsk rootdg bm3 second atm dev vx rdsk rootdg vol3 dev vx rdsk rootdg bm3 Successful mw var adm messages Make sure that the rdc service is active when the remote mirror software starts or you see the following error messages Completing SNDR startup sndrd Aug 16 08 37 16 sndrd 291 Cannot get address for transport tcp6 host 1 service rdc Aug 16 08 37 16 sndrd 291 Cannot establish RDC service over dev tcp6 transport setup problem Aug 16 08 37 16 sndrd 291 Cannot get address for transport tcp host 1 service rdc Aug 16 08 37 16 sndrd 291 All transports have been closed with errors Exiting Aug 16 08 37 16 sndrd 291 SNDR Fatal server error sndrsyncd done Chapter 2 Remote Mirror Software Troubleshooting Tips 7 Checking the etc nsswitch conf File If entries in the etc nsswitch conf are not configured correctly you might encounter these problems m If the hosts entry is incorrect volume sets not resume after a reboot m If the services entry is incorrect the rdc service might not activate and no data is replicated Note The services port number must be the same between all interconnected remote mirror host systems When the hosts and services entries are inc
55. t matches one of the configured sets Use sndradm i to view the sets configured sets Not a compact dependent shadow PITC The user attempted to attach an overflow volume to a set that is not a compact dependent set Possible errors EFAULT The kernel module tried to read out of bounds File a bug against iiadm ENOMEM The kernel module ran out of memory DSW_EEMPTY No volume name was specified DSW_ENOTFOUND The set could not be found in the kernel Not all Instant Image volumes are PITC All volumes master shadow and bitmap must be in in a disk group the same cluster device group not a valid number nust be a RM The number entered is out of the allowed range or is decimal between 1 and lt max gt not a number Not primary cannot sync s s and Kernel An update full sync or reverse sync command was s s issued on the secondary host These command can be issued only on the primary host Log in to the primary host and issue the request NULL struct knetconfig passed down Kernel The remote mirror software could not get network from user program information for the remote mirror set NULL struct netbuf passed down Kernel The remote mirror software could not get network from user program for s information for the remote mirror set Operation not possible disk queue Kernel Disk queue was not empty when the current operation s is not empty was attempted This usually occurs when the user attempts to disable Out of memory PIT
56. tegrity of the link 7 Check for common errors Sun StorEdge Availability Suite 3 2 Software Installation Guide d Checking That the rdc Service Is Running on page 8 If the dev rdc Link Is Not Created on page 9 Sun StorEdge Availability Suite 3 2 Software Installation Guide Checking Log Files on page 7 Checking the etc nsswitch conf File on page 8 Checking the Integrity of the Link on page 10 Correcting Common User Errors on page 13 Troubleshooting Log Files and Services The remote mirror software is client server software that is bidirectional The primary and secondary hosts each act as a client and server in the protocol Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 Checking Log Files Check the following files to troubleshoot problems m var opt SUNWesm ds log The var opt SUNWesm ds 1log file contains timestamped messages about the software For example Aug 20 19 13 55 scm scmadm cach nable succeeded Aug 20 19 13 55 ii iiboot resume cluster tag lt none gt Aug 20 19 13 58 sndr sndrboot r first atm dev vx rdsk rootdg vol5 dev vx rdask rootdg bm6 second atm dev vx rdsk rootdg vol7 dev vx rdsk rootdg bm7 Successful Aug 20 19 13 58 sndr sndrboot r first atm dev vx rdsk rootdg vol4 dev vx rdsk rootdg bm4 second atm dev vx rdsk rootdg vol4 dev vx rdsk rootdg vol4 Successful Aug 20 19 13 58 sndr sndrboo
57. termined lt shost gt unable to determine IP addresses RM The IP addresses for the host names specified for the for hosts lt host1l gt lt host2 gt primary or secondary hosts could not be determined Verify that the host names are in the etc hosts file unable to determine local hostname PITC iiadm needed to find out the name of the host it is running on but was unable to do so unable to determine network RM Network information for the secondary host could not information for lt host gt be determined Verify the settings in the etc nsswitch conf file unable to determin the current RM Look at rdc_check_release Solaris release lt error gt unable to find disk service RM SunCluster could not find the specified disk service lt ctag gt lt errno gt Unable to find disk service s PITC iiadm could not determine whether a cluster device group is active on the current host Unable to find lt group gt in RM Could not find remote mirror group in configuration configuration storage database while trying to do a diskq operation Unable to find lt shost gt lt svol gt in RM Could not find remote mirror set in configuration configuration storage database while trying to do a diskq operation unable to find SNDR set RM The remote mirror set is not configured while trying to lt shost gt lt svol gt in config set autosync for one of the following reasons The set could not be found in the configuration e The set is not configured in t
58. the network configuration procedures plumb ifconfig and route_add to a file that has a name that comes after SOOtrde in alphabetical order Accommodating Memory Requirements In releases prior to the Sun StorEdge Availability Suite 3 2 software a single asynchronous thread was created for each group of volume sets on the primary host Asynchronous I O requests were placed on an in memory queue and serviced by this single thread Because there was only one thread only one RPC request could be in flight for each group and each request had to be completed before a new request could be issued The version 3 2 software enables the number of asynchronous service threads to be set on a per group basis which enables multiple in flight RPC requests and faster servicing of the asynchronous queue Enabling more than one RPC request creates the possibility that requests arrive out of order with respect to the order that the write operations were issued on the primary host In other words a request might arrive before a previous request has completed its I O 16 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 The order of write operations must be maintained within a group Therefore these out of order requests must be stored in memory on the secondary host until the missing request comes in and completes The secondary host can store up to the hard coded limit of 64 requests per group Exceeding 64 stored reque
59. this set Then reissue the forward sync command can not use current config for RM A single set must be specified for bitmap operations bitmap reconfiguration The default configuration cannot be used for these operations can not use current config for RM An individual set or group must be specified when disk queue operations issuing disk queue operations A single disk queue operation cannot operate on all sets configured or all can not use a config file for disk sets in a file queue operations can not use current config for RM A set must be specified when issuing an enable enable command command An enable command cannot operate on the default configuration Can t export a mounted volume PITC A volume cannot belong to a mounted file system before it is exported Can t get memory for list enquiry PITC iiadm ran out of memory Can t open sort program PITC The iiadm operation is attempting to sort output before displaying it but cannot find the sort utility typically found in usr bin sort 22 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Cannot enable s s gt s s Kernel A set being enabled or resumed has a secondary secondary in use in another set volume that is already in use as a secondary volume for another remote mirror set A volume cannot be enabled as a s
60. xist DSW_EALREADY The set already has an overflow volume attached DSW_EWRONGTYPE The set is not compact dependent DSW_ERSRVFAIL The kernel module could not get access to the overflow volume DSW_EHDRBMP The set s bitmap could not be read A remote mirror command was issued referring to a set in abbreviated format but a mistake was made The format did not specify the set in the format shost svol The volume that was specified as the bitmap volume during an enable operation is already being used by another set 20 Sun StorEdge Availability Suite 3 2 Software Troubleshooting Guide December 2003 TABLE 3 1 Error Messages for the Sun StorEdge Availability Suite 3 2 Software Continued Error Message From Meaning Bitmap reconfig failed s s Kernel A request to reconfigure the bitmap on the local host has failed This can happen for two reasons e The old bitmap cannot be read from to obtain needed information The new bitmap cannot be reserved because the volume is not accessible or is already in use Verify that the new bitmap volume is accessible and is not already in use bitmap lt vol gt is already in use by RM The volume requested as a bitmap for the remote StorEdge Network Data Replicator mirror set is already configured to be used by the remote mirror software bitmap lt vol gt is in use by Instant RM The volume requested as a remote mirror bitmap Image volume is already configured to be a volume us

Sun Microsystems Sun StorEdge Availability Suite 3.2 User's Manual

Contents

Download Pdf Manuals

Related Search

Related Contents