Home

IBM Fibre Channel Planning and Integration: User`s Guide and Service

1. 0000 0042 FSCSI_ERR4 Adapter device driver detected error that indicates adapter software failure Command element included 0000 0043 FSCSI_ERR4 Adapter device driver detected error that indicates system failure Command element included 0000 0044 FSCSI_ERR4 Adapter device driver detected an unknown error status from the adapter driver Command element included 0000 0045 FSCSI_ERR4 Adapter detected an Intermediate Response IOCB entry Command element included 0000 0046 FSCSI_ERR4 Adapter reporting an invalid OCB Command element included 0000 0047 FSCSI_ERR4 Adapter reporting either a LS_RJT or BA_RuT from the device for this IOCB Command element included 0000 0048 FSCSI_ERR2 Adapter reporting a DMA error Command element included 0000 0049 FSCSI_ERR4 Adapter reporting an invalid IOCB local status Command element included 0000 004B FSCSI_ERR4 Adapter reporting an unknown IOCB status status field in IOCB is invalid Command element included 0000 0050 FSCSI_ERR6 Interrupt not received for the failing command The Command and command element are included 0000 0051 FSCSI_ERR6 Could not issue Cancel for a failing command which has not received an interrupt dev_info information is included 0000 0060 FSCSI_ERR6 Call to adapter driver s control entry point failed to accept N_Port Login for a device The return code and command element are included 0000 0061 FSCSI_ERR6 The device and SCSI protocol adapter do
2. When you send comments to IBM you grant IBM a nonexclusive right to use or distribute your comments in any way it believes appropriate without incurring any obligation to you Name Address Company or Organization Phone No Readers Comments We d Like to Hear from You SC23 4329 03 Fold and Tape Please do not staple Fold and Tape NO POSTAGE NECESSARY IF MAILED IN THE UNITED STATES BUSINESS REPLY MAIL FIRST CLASS MAIL PERMIT NO 40 ARMONK NEW YORK a POSTAGE WILL BE PAID BY ADDRESSEE es E Information Development E Department H6DS 905 6C006 11400 Burnet Road Austin TX 78758 3493 Imimimannmm mnnm Fold and Tape Please do not staple Fold and Tape SC23 4329 03 EE EE iiiI iIiAiIiiIiIiiIiIiiIiIiIiIiiIiIiiIiIiAiIiIiiIiIiiIiIiiIiIiiIiIiiIiIiiIiIiiIiIiiIiIiiIiIiiIiIiiIiAi ii Cut or Fold Along Line Cut or Fold Along Line Part Number 00P3117 Printed in U S A SC23 4329 03 1P P N 00P3117
3. Centralized management of distributed storage with the IBM FAStT Storage Manager Software Multi host Attachment The FAStT500 Storage Server is a storage solution for Fibre Channel and Fibre Channel SAN environments This storage system can be attached to AIX systems as well as other open system servers running UNIX Windows NT Solaris HP UX or Novell NetWare It can be shared simultaneously by multi operating systems although some restrictions apply Refer to the specific IBM Installation and Support Guides for more information on hosts with which you want heterogeneous connections Multiple server attachments can be implemented through IBM Fibre Channel switches to one of four mini hubs per FAStT500 controller enclosure The Fibre Array Storage Technology Storage Server supports heterogeneous environments by enabling partitions or zones that enable each host environment to access their individual set of logical volumes independently High availability Design The FAStT500 Storage Server is a 4U rack mountable enclosure or drawer that has dual active RAID controller units providing high throughput and redundancy Both controllers can be simultaneously active to provide seamless failover capability in case of emergency To increase availability each RAID controller supports up to 512 MB of battery backed cache and can perform write back operations Redundant Fibre Channel loops to all FAStT500 EXP500 disk drives which are hot swappable
4. The Logical Block Address LBA of the last block returned by the read capacity command for this device The block size of the device returned by the read capacity command The current block size the device driver is using for this device The block size specified by the configuration method the last time the device was configured These fields are ignored Normally they are set to 0 The Open Count field This 32 bit field indicates the number of times the device has been opened from the closed state This is useful for determining if all the errors are associated with a specific removable media unit Status Validity of 3 with Adapter Status of 1 When a Status Validity of 3 is accompanied by an Adapter Status of 1 an SC_DISK_ERR2 is being reported indicating that the mode data is corrupted When this error occurs the device will be unusable by AIX If the mode data passed to the device driver at configuration time is corrupted the device will fail to configure remain defined and this error will be logged If the device s mode data is corrupted either current or changeable the open will fail and this error will be logged If one of the following is true the device driver determines that the mode data is corrupted A mode page length is greater then 254 The current page length plus the current offset in the mode data is greater than the total length of mode data 92 IBM Fibre Channel Planning and Integration User
5. Assign volumes to be accessible to more than one host system Modify volume assignments e Modify and view communication resource settings such as e mail addresses and telephone numbers e Authorize user access With the ESS Specialist the following information can be viewed e The external connection between a host system and an ESS port e The internal connection of ports to cluster 1 or cluster 2 e How storage space is allocated to volumes ESS Copy Services ESS Expert The Copy Services feature provides a Web based interface for managing Peer to Peer Remote Copy PPRC and FlashCopy functions Copy Services collects information from the storage servers on a single Copy Services server Copy Services is part of the StorWatch Web interface and is accessed from the ESS Specialist main menu The Copy Services panels can be used to view and define Volumes Control Units Paths Tasks and Configurations The StorWatch Enterprise Storage Server Expert ESS Expert is an optional software product that can be purchased to use with the ESS The ESS Expert Web interface provides storage resource management functions for the storage servers such as asset management capacity management and performance management Appendix K IBM 2105 E10 E20 F10 F20 Enterprise Storage Server 133 ESS Expert allows the scheduling of information collection With this information informed decisions can be made regarding volume placement and capacity pla
6. Does the fiber jumper from the AIX System Fibre Channel Adapter provide a complete signal path to the disk storage subsystem tape drive patch panel or other device such as a hub etc to which it is connected NO Correct the fault After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done YES Go to Step 0060 3 Chapter 6 Problem Determination 51 Step 0060 3 Determine that the patch panels and interconnecting trunk or jumpers in this configuration provide a complete end to end signal path Note e lf this path does not include a patch panel skip this step e lf this configuration contains more than one patch panel trunk set use the following procedure to check all of them regardless of whether they exist in the configuration Using an accurate fiber cabling chart perform the following steps 1 Ensure that the correct truck fibers or interconnecting jumper is plugged into the correct clean patch panel connection 2 Ensure that the trunk fibers or interconnecting jumpers deliver the light properly to the patch panel at the other end 3 Ensure that these truck fibers or interconnecting jumper is plugged into the correct clean patch panel connection at the other end 4 Ensure that the fiber jumper connector at this patch panel is clean and correctly connected Do the patch panels and interconnecting trunk or jumper in this conf
7. Point to Point Configurations Hub Configurations Switch Configurations SAN Data Gateway Configurations Contents 56 57 s O7 59 59 61 61 62 63 64 64 65 67 67 69 69 70 70 70 71 71 71 71 43 75 ae sT 82 89 93 95 95 96 96 97 98 98 98 98 99 99 100 105 Vv Appendix E IBM 2103 H07 Fibre Channel Storage Hub 107 IBM 2103 H07 Fibre Channel eae Hub PEENE m E aon amp oe amp amp 2 lOr Configuration Flexibility Hook oe ow eg a mo 4 OF Standard and Extended Distances ae wo ot Se Gee Be TOF 2103 H07 IBM Fibre Channel Storage Hub Hardware ook me oa a w amp a 108 Fibre Channel Storage Hub Reliability and apices eb oe Se oe e a 2 4 108 General Information i y 293 e a 2 106 Fibre Channel Arbitrated Loop Hub fof ges he ae go ek eer le a 2 108 Short wave GBIC ee ee ee 108 Long wave GBIC go ak ow A 108 Publications and Other Sources of Information Gow e amp amp ag ow e S109 Appendix F IBM 2108 G07 SAN Data Gateway T IBM SAN Data Gateway Description ek Ty ed 2 ee ec Using Fibre Channel to Overcome SCSI Channel Limitations yo ee eo ee TT Extended SCSI Channel Distance 2 1 Multiple Configuration Options 2 4901 General Information ooh ao we ow me Ae 8 oe e 12 Publ
8. 677 702 G Microcode Description Error log analysis indicates that an adapter 677 microcode error has occurred 677 703 G 677 Description Error log analysis indicates that an unknown adapter error has occurred Note Reference to MAP 0210 in Table 2 on page 62 refers to the Maintenance Source of SRN Analysis Procedures MAPs in Diagnostic Information for Multiple Bus Systems order number SA38 0509 SRN Src source codes indicate the program or procedure that produced the SRN The source codes are described as follows SRC Aerwoiuriemoeoduwa D Description The SRN is from a steady number in the operator panel display The SRN is from a MAP callout The SRN was due to a missing resource at configuration time The SRN is from a diagnostic test after complete isolation testing The SRN is from a POST Failure The SRN is from a diagnostic test after partial isolation testing The SRN is from the Error Log Analysis program The SRN is from a diagnostic message after a flashing 888 The SRN is from built in ROM diagnostics The SRN is from offline diagnostics Chapter 7 Service Information 63 Failing Function Codes The failing function codes are described as follows SRN Description 447 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus 677 Gigabit Fibre Channel PCI Adapter 221 System or I O Board Microcode Microcode failure Tracing the Gigabit Fibre Channel Adapter You can use the trace facility
9. After you have set all fields to the desired values press Enter to generate the error report 66 IBM Fibre Channel Planning and Integration User s Guide and Service Information 2109 SAN Fibre Channel Switch Service Aid When you are investigating problems associated with the 2109 Switch you can obtain some useful information by logging into its controller using TelNet If you do not know the user ID and password request assistance from the system operator or administrator Table 3 Useful 2109 Commands provides some useful commands Command What it Does aliasShow Print Alias Server Information configShow Print Switch Config Parameters diagHelp Print Diagnostic Help errShow Print Error Log fabricShow Print Fabric Membership Information fanShow Print Fan Status ifmodeShow Print Network Link Operating Mode ifShow Print Network Interface Information login Log in as a New User logout Log out from Remote Session portErrShow Print Port Error Summary portLogShow Print Port Activity Log portShow Print State of specified Port portStatShow Print Hardware Statistics psShow Print Power Supply Status switchShow Print Switch and Port Status tempShow Print Temperature Readings version Print Firmware Version Other Fibre Channel Devices For service information for each of the Fibre Channel devices in the configuration refer to the appropriate appendix for each device
10. Dual fans and power supplies further support continuous operations Extended Distances The SAN Fibre Channel Switch and the FAStT500 mini hub Fibre Channel ports provide greater flexibility and extended distances across Fibre Channel SAN topologies enabling the FAStT500 Storage Server to be configured at distances of up to 500 meters through short wave fiber connections or up to 10 km through long wave fiber connections High Performance The FAStT500 Storage server has dual Fibre Channel RAID Controller units that will support up to two mini hubs each for a total of four Fibre Channel ports in a switched environment Each RAID Controller unit has a 300 MHz AMD K6 processor and from 256 MB to 512 MB of battery backed cache to aid in high performance Dynamic Management The IBM FAStT500 Storage Manager software is a Java based network configurable integrated storage management tool that helps storage administrators configure monitor dynamically change and manage multiple FAStT500 RAID Storage Servers from a single Microsoft Windows 95 or Windows NT workstation Host systems such as AIX systems can be managed from AIX hosts with the IBM FAStT SM7client for AIX software High availability and full redundancy are provided with the host specific Fibre Channel Storage Manager software and the IBM Redundant Disk Array Controller 138 IBM Fibre Channel Planning and Integration User s Guide and Service Information RDAC fail over device drive
11. Tapes Subsystems Hubs Converters and Switches are on order or have been delivered and are featured correctly Prepare a Site Plan Prepare a site plan addressing the following considerations Location and relationship of the physical facilities such as walls doors windows partitions furniture and telephones Location of any telephone lines required for Call Home or remote dial in support for the various hosts and devices being installed Available Ethernet LAN connections and cabling that might be required for control of devices such as Fibre Channel Switches Power requirements for the various devices including any optional uninterruptible power supplies UPS being considered Length of power cables and the location of electrical outlets having the proper voltage amperage capacity and phase Security necessary to protect the installation s physical integrity Proximity of the various hosts and devices Accessibility to a telephone to aid in installation and service Service clearances weights and heat dissipation of the various hosts and devices Temperature and humidity requirements of the various hosts and devices Plan and Diagram All Cable Routing Draw a diagram showing all cables host systems Fibre Channel devices and patch panels This diagram should schematically depict all cabling and should include host device and patch panel locations This diagram will aid installation and be useful for any future
12. Total Capacity Up to 300GB N A N A Up to 600GB N A Native drive data rate 9 MB sec 9 MB sec 9 or 14 MB sec 14 MB sec 14 MB sec Maximum sustained data 27 MB sec 27 MB sec 27 or 34 MB sec 34 MB sec 34 MB sec rate Burst with Ultra SCSI data 40 MB sec 40 MB sec 40 MB sec 40 MB sec 40 MB sec rate Burst with Fibre Channel N A N A N A 100 MB sec 100 MB sec data rate High speed search 5m sec 5m sec 5m sec 5m sec 5m sec Notes 2 Depending on data content with 3 1 LZ1 data compression 1 B models use eight sets of 16 tracks E models use 16 sets of 16 tracks 146 IBM Fibre Channel Planning and Integration User s Guide and Service Information Ultrium 3580 Tape Family This section describes the Ultrium Tape Subsystems The IBM Ultrium Tape Family products adhere to the Linear Tape Open LTO design specifications LTO is an open tape architecture developed by a consortium of three major storage producers including IBM The family consists of the IBM 3580 Ultrium Tape Drive the IBM 3581 Ultrium Tape Autoloader the IBM 3583 Scalable Tape Library and the IBM 3584 Ultrascalable Tape Library The Ultrium Tape Family is intended to provide scalable and flexible tape solutions The building block of the Utrium Tape Family is the IBM 3580 Tape Drive It has a capacity of up to 200 GB with compression When combined with the 3581 Tape Autoloader a compressed data capacity of up to 1 4 TB is realized As storage requirements grow
13. a Examine the value of pssp_level This is the PSSP level installed on the system If the system is not at the required level of 3 1 1 and higher it must be upgraded before proceeding Contact the system administrator for assistance Chapter 3 Fibre Channel Installation 15 b Record the value of lppsource for the system This value is the name of the LPP source directory for device driver installation You will need this name for later steps 5 Determine if the installable image for the device driver is already present in the LPP source directory by typing the following command cd spdata sys1 install 1ppsource substituting the directory name which was recorded in step 4b for 1ppsource a Once in this directory type the following command ls devices pci df1000f If devices pci df1000f7 is returned the device driver image for the Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S is present If devices pci df1000f9 is returned the device driver image for the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus FC 6228 Type 4 W is present If nothing is returned neither image is present b Type the following command ls devices common IBM fc c Repeat Step 5b for all device drivers for Fibre Channel devices you are installing Other image names you may need to perform the Is command for are e devices fcp disk e devices fcp tape e devices fcp disk array If the required device driver images are notin the LP
14. e Provides industry standard Fibre Channel attachment to servers e Extends distances between the server and SCSl attached storage systems up to 500 meters e Provides a full function model for disk and tape storage and a low cost model for tape storage e Uses IBM StorWatch SAN Data Gateway Specialist which simplifies the management of SAN Data Gateways across the enterprise Using Fibre Channel to Overcome SCSI Channel Limitations The experience and expertise in serial loop with Serial Storage Architecture SSA and switched fabric with Enterprise System Connection ESCON fiber technologies can greatly simplify the introduction of SAN Fibre Channel solutions that use open systems Fibre Channel protocols Because Fibre Channel technology offers many performance advantages over parallel SCSI channel technology it is ideal for server and storage consolidation Extended SCSI Channel Distance The SAN Data Gateway can provide Fibre Channel distance extension up to 500 meters between an open system server and a storage system This extended distance capability is ideally suited for disaster recovery implementations such as remote tape vaulting and remote disk mirroring Multiple Configuration Options The SAN Data Gateway is available as a rack mounted unit or as a stand alone tabletop unit The full function Model G07 provides one to three Fibre Channel ports and four Ultra SCSI Differential ports for attachment of disk and tape sto
15. www storage iom com hardsoft tape 3580 index html Appendix N Magstar and Ultrium Tape Subsystems 149 150 IBM Fibre Channel Planning and Integration User s Guide and Service Information Index Numerics 2101 Seascape solutions rack 128 2102 F10 Fibre Channel RAID storage server 127 2103 H07 fibre channel storage hub description 107 hardware 108 2105 E10 E20 F10 F20 enterprise storage server 131 2108 G07 SAN data gateway 111 2109 SAN fibre channel switch service aid 67 2109 F16 SAN Fibre Channel switch 113 2109 S08 S16 SAN fibre channel switch description 113 3552 Fibre Array Storage Technology FAStT500 Storage Server 137 3570 tape subsystem Magstar 143 3575 tape library dataserver Magstar 144 3580 tape subsystem Ultrium 147 3590 tape subsystem Magstar 145 A adapter hardware installing 20 AIX diagnostics for the gigabit fibre channel adapters 61 C check system prerequisites 14 complexity of the fibre channel environment 29 computer I O interface history 1 computer I O interface strategies 1 configurations fibre channel 98 hub 99 hub with four host systems 100 hub with one host system 99 hub with two host systems 100 point to point 99 SAN Data Gateway 105 switch 100 switches in a high availability 104 connection topology fibre channel 2 control workstation creating an installable image 16 D data gateway SAN 111 description EMC symmetrix enterprise storage systems 141 enterprise storage
16. 13 Target Cancel ABORT_MXRI_CN command state D Used for the first 40 bytes of the FCP response IU if valid that is the reply IOCB status field equals 1 or the fcp_status field of the FCP response IU is nonzero Otherwise if this is for a valid command it is the following Byte Description of Contents 0 7 Virtual address of command payload 8 15 Virtual address of response buffer 16 23 Virtual address of BDEs 24 25 Command s preempt count 26 Command s pending_retries count l Used for the address of the failing scsi_info structure Detailed information for the sense data error numbers is shown in Table 5 on page 85 84 IBM Fibre Channel Planning and Integration User s Guide and Service Information Table 5 Error Number Information for the Fibre Channel SCSI Protocol Error Number Error Template Description of Error 0000 0001 FSCSI_ERR8 Command Timeout in SCIOLINQU dev_info information is included 0000 0002 FSCSI_ERR8 Command Timeout in SCIOLSTUNIT dev_info information is included 0000 0003 FSCSI_ERR8 Command Timeout in Test Unit Ready IOCTL dev_info information is included 0000 0004 FSCSIERR8 Command Timeout in Read Block loctl dev_info information is included 0000 0005 FSCSI_ERR6 An unexpected SC_Q_CLR buf with no SCSI command was received in strategy dev_info information is included 0000 0006 FSCSI_ERR6 An unknown IOCB type was received in our in
17. Adapter and all devices in the Fibre Channel configuration e Service manuals for all Fibre Channel devices in the environment For information on these manuals refer to the appropriate appendix for each device Each device s appendix contains a section called Publications and Other Sources of Information This section contains a list of publications and Web sites that provide device specific instructions and information needed for servicing that device e In addition the following publications will be helpful in isolating link failures Link Fault Isolation SY22 9533 8 390 Fiber Optic Links ESCON FICON Coupling Links and Open system Adapters SY27 2597 Chapter 6 Problem Determination 31 Start of PDP PD0010 Start of Call Step 0010 1 Step 0010 2 Start here to troubleshoot the Fibre Channel environment Determine if the Fibre Channel adapter is available in the AIX system Run the following AIX command Isdev C grep fcs The results should be similar to the following assuming three adapters are installed in slots 14 08 21 08 and 2A 08 fcs0 Available 14 08 FC Adapter fcsl Available 21 08 FC Adapter fcs2 Available 2A 08 FC Adapter Is the adapter available NO Go to Step 0020 1 YES Go to Step 0010 2 Determine if the Fibre Channel SCSI I O Controller Protocol Device is available in the AIX system Run the following AIX command Isdev C grep fscsi The results should be similar to the
18. Der Aussteller der Konformitatserklarung ist die IBM Germany Dieses Ger t erf llt die Bedingungen der EN 55022 Klasse B Appendix A Communications Statements 71 72 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix B Notices This information was developed for products and services offered in the U S A The manufacturer may not offer the products services or features discussed in this document in other countries Consult the manufacturer s representative for information on the products and services currently available in your area Any reference to the manufacturer s product program or service is not intended to state or imply that only that product program or service may be used Any functionally equivalent product program or service that does not infringe any intellectual property right of the manufacturer may be used instead However it is the user s responsibility to evaluate and verify the operation of any product program or service The manufacturer may have patents or pending patent applications covering subject matter described in this document The furnishing of this document does not give you any license to these patents You can send license inquiries in writing to the manufacturer The following paragraph does not apply to the United Kingdom or any country where such provisions are inconsistent with local law THIS MANUAL IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KI
19. Each device s appendix contains a section called Publications and Other Sources of Information This section contains a list of publications and Web sites that provide device specific instructions and information needed for installing configuring operating and servicing of that device Chapter 7 Service Information 67 68 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix A Communications Statements The following statement applies to this product The statement for other products intended for use with this product appears in their accompanying documentation Federal Communications Commission FCC Statement Note This equipment has been tested and found to comply with the limits for a Class B digital device pursuant to Part 15 of the FCC Rules These limits are designed to provide reasonable protection against harmful interference in a residential installation This equipment generates uses and can radiate radio frequency energy and if not installed and used in accordance with the instructions may cause harmful interference to radio communications However there is no guarantee that interference will not occur in a particular installation If this equipment does cause harmful interference to radio or television reception which can be determined by turning the equipment off and on the user is encouraged to try to correct the interference by one or more of the following measures e R
20. Extended Distances The SAN Fibre Channel Switch and the Fibre Channel Storage Hub provide greater flexibility and extended distances across Fibre Channel SAN topologies enabling the Fibre Channel RAID Storage Server to be configured at distances of up to 500 meters through short wave fiber connections or up to 10 km through long wave fiber connections High Performance The Fibre Channel RAID Storage Server has dual Fibre Channel ports that provide an aggregate bandwidth of 200 MB sec Dynamic Management The IBM StorWatch Fibre Channel RAID Specialist is a network based integrated storage management tool that helps storage administrators configure monitor dynamically change and manage multiple Fibre Channel RAID Storage Servers from a single Microsoft Windows 95 or Windows NT workstation High availability and full redundancy are provided with the host specific Fibre Channel Storage Manager software which resides on the host system and provides automatic I O path failover when a host adapter or a storage controller fails General Information This section provides general information pertaining to the 2102 F10 Fibre Channel RAID Storage Server Product Information Item Description 2101 100 IBM Seascape Solutions Rack or equivalent 19 inch rack is required 2102 F10 Fibre Channel RAID Storage Server required Dual active RAID controllers with up to 512 MB cache redundant power and cooling units are standard 2102 D00 Expan
21. Fibre Channel Director port a McDATA ES 3016 or ES 3032 Enterprise Fibre Channel Switch port a 2109 S08 S16 or F16 SAN Fibre Channel Switch port or an INRANGE FC 9000 Fibre Channel Director port in a SAN configuration e 2105 E10 E20 F10 and F20 Enterprise Storage Server with Fibre Channel Ports e EMC Symmetrix version 4 8 or higher 3000 5000 and 8000 Enterprise Storage Systems IBM 3552 FibreArray Storage Technology FAStT500 e Magstar 3590 Model E11 and E1A Tape Subsystem or Ultrium 3580 Tape Drives 3583 or 3584 Tape Libraries non cascaded switch Attachment of these tape subsystems to the McDATA ED 5000 ED 6064 ES 3016 or ES 3032 also requires a McDATA ES 1000 between the Director Switch and the tape e 2108 G07 SAN Data Gateway e A port of a 2109 S08 S16 or F16 connected in cascade to another 2109 S08 S16 or F16 only A port of a McDATA ED 5000 ED 6064 ES 3016 or ES 3032 connected in cascade to another McDATA ED 5000 ED 6064 ES 3016 or ES 3032 only A port of an INRANGE FC 9000 connected in cascade to another INRANGE FC 9000 only For detailed information on each of the Fibre Channel devices available refer to the appropriate appendix for each device Each device s appendix contains a section called Publications and other Sources of Information This section contains a list of publications and Web sites that provide device specific instructions and information needed for installing configuring ope
22. Fibre Channel Planning and Integration User s Guide and Service Information 5 Select ADDITIONAL event IDs to Trace 6 In the input field type 3B9 3C4 7 Press Enter to start the trace Note It may be necessary at times to add other event IDs hooks to the trace However it is always best to trace the smallest number of event IDs required for problem determination To stop the trace from the Gigabit Fibre Channel Adapter s SMIT panels after the failure has been reproduced perform the following steps 1 Navigate to the Trace a FC Adapter menu as in steps 1 3 above 2 Select Stop Trace 3 Press Enter to stop the trace To generate a trace report from the Gigabit Fibre Channel Adapter s SMIT panels after the trace has been stopped perform the following steps 1 Type the following smitty fcsa Select FC Adapter Select Trace a FC Adapter Select Generate a Trace Report Press Enter Select 1 filename unless you want to print the report on the pop up window NO oT Em O Fill in fields as desired to control how the report is generated If you want to save the report to a file put a file name in the FILE NAME for Trace report input field Otherwise the default is stdout 8 Press Enter to generate the report Generating an Error Report for the Gigabit Fibre Channel Adapter When you are analyzing a problem in the Fibre Channel environment it is often necessary to examine the AIX Error Log The quickest meth
23. Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with YES Go to Step 0060 1 Chapter 6 Problem Determination 49 Start of PDP PD0060 Fiber Path Failures Step 0060 1 Determine that the fiber jumpers trucks patch panels and any other devices such as hubs in this configuration provide a complete signal path from the AIX System Fibre Channel Adapter to the disk storage subsystem or tape drive The number of possible indications that a problem may exist in the signal path to the disk storage subsystem or tape drive include The AIX error log has entries associated with this adapter or Fibre Channel device whose Probable Causes list includes Cables and Connectors e The Hard Disks cannot be configured made Available and the Disk Storage Subsystem and intervening switches or SAN Data Gateway have been eliminated as the cause of failure e The AIX application reports link or protocol errors associated with the path which includes a specific device and that device and intervening switches or SAN Data Gateway have been eliminated as the cause of failure A Fibre Channel device displays error LEDs for one or more Fibre Channel ports indicating a link problem A Fibre Channel Switch displays link error conditions through its Enterprise Fabric Connectivity Management Termin
24. IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix J IBM 2102 F10 Fibre Channel RAID Storage Server This appendix describes the Fibre Channel RAID Storage Server IBM 2102 F10 Fibre Channel RAID Storage Server Description The 2102 F10 Fibre Channel RAID Storage Server FC RSS is a complete storage subsystem that is housed in one storage rack It offers concurrent multihost system attachment through a Fibre Channel interface supporting AIX systems as well as Windows NT and other open system servers The FC RSS can share the storage and data up to six drawers of disks among multiple servers simultaneously The 2102 F10 Fibre Channel RAID Storage Server has two independent controllers Normally each controller manages approximately half of the attached storage however in the event of most controller failures the FC RSS routes all transactions through the remaining controller The controller module provides management functions for the FC RSS It manages interfaces to host system devices as well as the service interface for example a remote support modem The controller cache acts as an intermediate storage for data being transferred between the host servers and the disk driver module DDM array The controller cache includes battery backup to prevent data loss during power failures The 2102 F10 Fibre Channel RAID Storage Server offers the following Broad range of RAID options RAID
25. IBM Fibre Channel Planning and Integration User s Guide and Service Information Chapter 6 Problem Determination If a problem occurs in the Fibre Channel environment you will need a number of pieces of information to successfully correct the problem This chapter discusses Fibre Channel environment specific problems If problems are experienced with the AIX system see your AIX documentation The Fibre Channel environment can be complex and because of the potential distances between components of the system and the diverse nature of these components additional information will be required to aid in problem determination The information is available from several sources Gigabit Fibre Channel PCI Adapter Service LEDs The Gigabit Fibre Channel PCI Adapter has two LEDs located near the connectors These can be used to determine the state of the adapter For details on these LEDs see Chapter 7 Service Information on page 59 AIX System Problem Determination Information The AIX system provides problem determination information from its operator display codes error logging facilities and application messages For more information on AIX error logs see Appendix C Error Messages on page 75 e Fibre Channel Director Problem Determination Information The Fibre Channel Director provides problem determination information from its operator panel LED indicators on the port cards and the Enterprise Fabric Connectivi
26. ID LUN Dev_info ID LUN cancel_retry count Dev_info ID LUN qstate stop_pending flag dev_queuing flag Dev_info ID LUN Dev_info ID LUN Dev_info ID LUN Dev_info ID LUN need_relogin flag need_reseme_set flag aca_error_state flag Ss SS ea ae a el ee N Oo OF FP WO ND number of active commands for this ID LUN Reply IOCB if valid Otherwise it is the following Word Description of Contents 0 Abort Task Set command s state for this ID LUN 1 Clear ACA command s state for this ID LUN 2 Cancel element ABORT_MXRI_CN state for this ID LUN 3 ABORT MXRI KEY for this ID LUN 4 Cancel element s ABORT_MXRI_CN parameter field for this ID LUN 5 Cancel element s ABORT_MXRI_CN status field of the IOCB for this ID LUN 6 7 Virtual address of this device s head_pending queue Main BDE list for this IOCB if valid Otherwise it is the following Appendix C Error Messages 83 Byte Description of Contents Target_info ID state Target_info ID flags Target_info ID cancel retry count process login retry count Target_info ID relogin retry count YS FTF Fae Fae re wa Target_info ID device_flags field from PRLI Target_info ID Target_info ID stop pending flag FC class used for this Target_info ID Target Reset Command State PDISC command state 0 PLOGI command state ON Oa F WO Nao 11 PRLI command state 12 Cancel ABORT_MXRI_CN command state
27. RS 6000 SP systems with the Gigabit Fibre Channel PCI Adapter or the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus e Intel and UNIX based servers with Fibre Channel adapters e IBM 9672 Parallel Enterprise G5 G6 and zSeries 900 servers with FICON channel cards IBM storage with FICON adapters IBM Magstar 3590 Model A60 Tape Subsystem and IBM Enterprise Storage Server e IBM storage with Fibre Channel adapters IBM Magstar 3590 Tape and IBM Enterprise Storage Server Multiple Directors can be interconnected in non FICON applications FC 9000 Configuration Options The Director provides a wide range of configuration options A single I O Module provides 8 Fibre Channel ports with each port capable of delivering up to 100 MB sec full duplex data transfer Industry leading 12 800 MB sec transmission bandwidth supports full bandwidth 128 port switch performance 123 Configuration options include short wave laser GBICs enabling a transmission distance of up to 500 meters long wave laser GBICs enabling a transmission distance of up to 10 kilometers and extended long wave GBICs enabling a transmission distance of up to 80 kilometers for a maximum of 128 GBICs The FC 9000 Cabinet supports up to 128 ports in a floorspace of 6 square feet Two FC 9000 64 Directors or one FC 9000 128 Director can reside in a single FC 9000 Cabinet The FC 9000 64 Director is scalable with non disruptive upgrades from 24 to 64 ports The FC 9000 128 D
28. SAN Data Gateway e 2102 F10 Fibre Channel RAID Storage Subsystem e 2105 E10 E20 F10 and F20 Enterprise Storage Server with Fibre Channel Ports e EMC Symmetrix Version 4 8 or higher 3000 5000 and 8000 Enterprise Storage Systems e Magstar 3590 Models E11 and E1A Tape Subsystems e Ultrium 3580 Tape Drives 3583 and 3584 Tape Libraries The following SCSI devices are available for connection to a 2108 GO7 SAN Data Gateway SCSI interface in a Fibre Channel SAN configuration e 2105 E10 E20 F10 and F20 Enterprise Storage Server with SCSI interfaces e Magstar MP Multi purpose 3570 Tape Subsystem Models C00 C22 e Magstar MP Multi purpose 3575 Tape Library DataServer Models L06 L32 e Magstar 3590 Tape Subsystem e Ultrium 3580 Tape Drives 3583 and 3584 Tape Libraries The following Fibre Channel devices are available for connection to a 2103 H07 Fibre Channel Hub Port in a SAN configuration e 2103 H07 Hub both long and short wave port connections e 2102 F10 Fibre Channel RAID Storage Subsystem e 2105 E10 E20 F10 and F20 Enterprise Storage Server with Fibre Channel Ports e EMC Symmetrix version 4 8 or higher 3000 5000 and 8000 Enterprise Storage Systems e Magstar 3590 Models E11 and E1A Tape Subsystems e Ultrium 3580 Tape Drives 3583 and 3584 Tape Libraries 95 The following Fibre Channel devices are available for connection to a McDATA ED 5000 Enterprise Fibre Channel Director port a MCDATA ED 6064 Enterprise
29. SP AIX systems see BM RS 6000 SP Planning Volume 1 Hardware and Physical Environment GA22 7280 2 After the installation is complete connect the cable to the adapter Note If you start your system with the wrap plug in the Fibre Channel Adapter you must reboot or run the cfgmgr command after actually cabling the adapter into the Fibre Channel configuration 20 IBM Fibre Channel Planning and Integration User s Guide and Service Information 3 Turn on your system unit If the AIX operating system has not yet been installed you must to do so before verifying the hardware and software installation for this product To verify that the adapter hardware and device driver software are correctly installed follow the procedures outlined in Verifying the Installation Verifying the Installation You can verify the installation following installation or during problem determination by performing one or both of the following procedures e Verifying the Hardware Installation Isdev e Verifying the Software Installation Islpp Verifying the Hardware Installation Verify your system hardware is properly installed as follows 1 If necessary log in as root user 2 Type the following Isdev C grep fcs Press Enter If the Gigabit Fibre Channel Adapter is properly installed and configured the resulting output of the above command indicates it to be Available The following is an example of the data that should display on y
30. Single Mode Fiber 50 Micron Multimode Fiber 62 5 Micron Multimode Fiber Video Coax and Shielded Twisted Pair In all cases the transmission of data is bit by bit serial thus eliminating the skew problems inherent in parallel wire technologies Fibre Channel was designed to possess the following characteristics e Good price performance e Connection versatility with scalable performance e Scalability e Reliability e Very large bandwidth one Gigabit Second e Ability to handle a variety of protocols e High efficiency Fibre Channel Connection Topology In a Fibre Channel environment the terms node port and link are used to describe components of the system Fibre Channel devices are referred to as nodes A node may be a workstation disk drive or other device capable of Fibre Channel connectivity Each node must have at least one port The port provides the actual connection to the outside world The port may for example be an adapter in the workstation The link is the actual transmission medium used to carry the information from one node to another Each port connects to the link with a pair of fibers One fiber connects to the port s receiver and carries information into the port while the other fiber connects to the port s transmitter and carries information out of the port Most cables contain both fibers thus the pair appears as a single entity to a casual observer 2 IBM Fibre Channel Planning and Integration User s G
31. a FLOGI and the retry count has been exceeded on issuing a FLOGI The failing IOCB is included 0000 0014 FCS_ERR4 The IOCB failed with an error indicating Fabric path not available The failing OCB is included Appendix C Error Messages 79 Table 4 Error Number Information for the Fibre Channel Adapter continued Error Number Error Template Description of Error 0000 0015 FCS_ERR5 The IOCB failed with an exchange or sequence error The failing IOCB is included 0000 0016 FCS _ERR6 An IOCB failed with a N_Port Fabric Reject but with an unknown reason code 0000 0017 FCS ERR4 An IOCB failed with an LS_RJT and the retry count has been exceeded including trying a different class when the IOCB was FLOGI The failed IOCB is included 0000 0018 FCS ERR4 An IOCB failed with a BA_RuT The failed IOCB is included 0000 0019 FCS_ERR4 An IOCB failed with remote stop that is ABTS ABTX from another N_Port The failing IOCB is included 0000 001A FCS_ERR2 A DMA error was detected by the adapter on an IOCB The failing IOCB is included 0000 001B FCS_ERR4 An IOCB failed with an error indicating Invalid RPI and the retry and relogin re register count has been exceeded The failing IOCB is included 0000 001C FCS ERR4 A device refused connection 0000 001D FCS_ERR6 An IOCB failed with an data frame sequence corruption reported The failed IOCB is i
32. act on the new information If Soft Zoning is used care must be taken to ensure that the AIX cfgmgr command is run after any zone changes have been made The cfgmgr command is required to ensure that the new zone information is recognized by the server To learn more about how to configure your Fibre Channel director or switch refer to the documentation provided with it Link Distances For long wave Fibre Channel ports the maximum link distance supported is 10 km for both 50 and 62 5 micron optical fibers and for both one and two Gigabit link speeds However for short wave ports the maximum supported link distance varies for fiber size and link speed The following table provides information on maximum supported Fibre Channel link distances for short wave ports Table 6 Fibre Channel Link Distances for Short wave Ports Link Speed Fiber Size 1 Gb 2 Gb 50 micron 500m 300m 62 5 micron 175m 150m While both 50 and 62 5 micron optical fibers are supported in these Fibre Channel configurations mixing the two in any one link is not supported Fibre Channel Boot Fibre Channel Boot FC Boot also known as SAN boot over Fibre Channel allows the AIX operating system to reside on a disk drive of certain Fibre Channel attached storage subsystems pSeries systems which support FC Boot can be booted from this disk drive For FC Boot to function properly only one path to the rootvg device can exist on the system unless s
33. and IBM storage devices such as the IBM Magstar 3590 Model A60 Tape Controller and the IBM Enterprise Storage Server using FICON ports Management Modes In addition to operating modes the McDATA ED 5000 ED 6064 ES 3016 and ES 3032 also have the following management modes that is methods by which the fabric is controlled e Enterprise Fabric Connectivity EFC management mode e Open Systems management mode FICON management mode EFC management mode is the basic method of controlling the fabric and can always be used However Open Systems and FICON management modes are mutually exclusive In EFC management mode the Enterprise Fabric Connectivity Manager EFCM running on the EFC server and connected to the switch via an Ethernet link provides a Java based GUI for managing the fabric Management via an Ethernet link is know as out of band management Open Systems management mode is an optional feature which allows control of the fabric by an application such as Tivoli Storage Network Manager or Veritas SANPoint Control running on an open systems server connected to the switch via one of its Fibre Channel Ports Management through a Fibre Channel port is known as inband management FICON management mode is an optional feature which allows control of the fabric by an application running on an S 390 server connected to the switch via one of its FICON Ports This FICON port is known as a Control Unit Port CUP When the FICON Manag
34. are functioning Location Location code of the FC adapter driver The parent adapter driver instance fcsX will have for its location code the first two values of this three value location code For example if the parent FC adapter driver instance fcs0 has a location code of 20 58 all children will have the location code of 20 58 01 Fibre Channel SCSI Protocol Driver Configuration To configure the Fibre Channel FC SCSI protocol driver do the following 1 Log in as root user 2 To start SMIT type smit Press Enter 3 Select Devices 4 Select the FC SCSI Protocol Driver section FC SCSI Protocol Driver Options display on the screen 5 Select Change Show Device Characteristics A list of configurable FC SCSI Protocol Device options displays as follows a N Change Show Characteristics of a FC SCSI Protocol Device Type or select values in entry fields Press Enter AFTER making all desired changes Entry Fields FC SCSI virtual device fscsid Description FC SCSI I 0 Controller Status Available Location 20 58 01 Adapter SCSI ID 0x1 How this adapter is CONNECTED al FC Class for Fabric 3 Apply change to DATABASE only no A Chapter 4 Using SMIT to Configure the Fibre Channel Adapter 25 Configuration Options The FC SCSI Protocol Device has the FC Class for Fabric configuration option This option is available for AIX 5 1 and later The default is for class 3 operation Class 2 is not currently s
35. as follows 1 Install patch panels trunks and jumper cables 2 Install any new AIX Systems that are to be added 3 Install any Gigabit Fibre Channel Adapters required in the RS 6000 and pSeries Systems 4 Install Fibre Channel Switches Hubs and SAN Data Gateways as applicable 5 Install Disk Storage Subsystems and Tape Subsystems as applicable The remainder of this chapter provides instructions for installing the Gigabit Fibre Channel Adapter and other Fibre Channel devices Installing the Gigabit Fibre Channel Adapter The sequence of steps in the installation depends on whether your system already has the AIX operating system installed If AIX is already installed install the device driver software first and then install the hardware adapter If AIX is not yet installed you need only install the hardware adapter The device driver software will be installed automatically during the installation of the AIX operating system when the adapter hardware is detected To proceed with the installation of the Gigabit Fibre Channel Adapter select the one procedure from the following list that describes the particulars of this installation e If you are installing the Gigabit Fibre Channel Adapter in a system on which the AIX operating system is not yet installed proceed directly to the section Installing thel Adapter Hardware on page 20 e If you are installing the Gigabit Fibre Channel Adapter in an RS 6000 SP system on whi
36. command Note To use the dsh command the Kerberos ticket for the SP system must be current If you are unsure whether the Kerberos ticket is current consult the system administrator for the system on which the installation is to occur To complete the installation of the device drivers on the node s of an SP system perform the following steps 1 Set up for the dsh command that is set up the working collective file in the control workstation Chapter 3 Fibre Channel Installation 13 2 Check prerequisites on the system nodes using the dsh command 3 Create an installable device driver image on the control workstation 4 Install the device drivers on the system nodes using the dsh command If the site setup precludes the use of the dsh command install the device drivers on each node individually To do so telnet to each node and perform the steps in the various procedure sections below omitting the dsh prefix If only a few nodes are involved you can use the dsh command without using the working collective file Use the w option to include the node names within the command For example to determine the AIX level on three nodes run the following command dsh w nodenamel nodename2 nodename3 oslevel where the actual node names are substituted for nodenamel nodename2 and nodename3 Note In the instructions that follow brackets are used to indicate that the user must supply actual values for input or that the sy
37. confirmation is required As in Class 2 frame switching is used rather than a dedicated connection however no confirmation is required IBM Fibre Channel Planning and Integration User s Guide and Service Information Introduction to the Gigabit Fibre Channel Adapters In the Storage Area Network SAN environment the host computers by the nature of their role move to the forefront of importance The following Fibre Channel adapters are available for the RS 6000 and pSeries and provide Fibre Channel connectivity for AIX system hosts e Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S e 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus FC 6228 Type 4 W These Fibre Channel adapter features are available for certain RS 6000 and pSeries models and enable them to perform as initiators in Fibre Channel environments such as SANs and allow users to establish a network of high speed local and remote storage Each adapter generally known as a Host Bus Adapter HBA provides one port for connection into the configuration and provides single initiator capability over an optical fiber link These Gigabit Fibre Channel Adapters connect using either 50 micron or 62 5 micron multi mode optical fiber cables The adapters features include e 1062 Megabits sec Fibre Channel Interface up to 2 Gigabits sec for FC 6228 Type 4 W e PCI Master and Slave DMA capabilities e FC 6227 Type 4 S requires a PCI 2 1 32 bit 33 MHz slot e FC 62
38. drivers being installed such as dsh Islpp 1 devices common IBM fc 2 gt amp 1 more dsh Islpp 1 devices fc disk 2 gt amp 1 more dsh Islpp 1 devices fc disk array 2 gt amp 1 more dsh Islpp 1 devices fc tape 2 gt amp 1 more If the device driver is already installed on the node s contact your service provider to have the adapter s installed There is no need to continue with the device driver installation 3 If the device driver is not installed on the node s proceed to section Installing Device Drivers on the RS 6000 SP System Nodes on page 19 18 IBM Fibre Channel Planning and Integration User s Guide and Service Information Installing Device Drivers on the RS 6000 SP System Nodes Do the following to install device drivers on the SP system nodes 1 Ensure that the LPP source directory is exported to the nodes by typing the following showmount e The result should be similar to the following spdata sys1 install lppsource everyone where ppsource is the name of the directory recorded in step 4b on page 16 2 If the directory is not exported you may temporarily export the directory by typing exportfs i spdata sys1 install 1ppsource substituting the directory name which was recorded in step 4b on page 16 for lppsource 3 Perform an NFS mount of the LPP source directory by typing dsh mount controlwks spdata sysl install lppsource mnt substituting the director
39. dsh installp p acgqXd mnt devices fcp tape 2 gt amp 1 more 6 When device driver installation is complete on the node s contact your service provider to have the adapter s installed Installing the Adapter Hardware To install the Gigabit Fibre Channel Adapter perform the procedure detailed in this section Note If this system already has the AIX operating system installed you should have previously installed the device driver software If this is the case proceed with the hardware installation Otherwise follow the instructions in Installing the Device Driver Software on an RS 6000 and pSeries System on page 12for Installing the Device Driver on an RS 6000 SP System on page 13 Attention Do not remove the Gigabit Fibre Channel Adapter from its anti static package at this time Be sure you have read Handling Static Sensitive Devices on 1 Refer to the documentation that shipped with your system unit to perform the following Attention Before you begin be sure you have read Safety Notices on page ix in the front of this publication or in the documentation that shipped with your system unit e Shut down your system unit e Install the adapter into your system unit in a primary PCI bus slot Note The following publications contain important information regarding adapter placement in your system unit For AIX systems see PCI Adapter Placement Reference SA38 0538 For RS 6000
40. failure of any one unit will not result in loss of the availability of the director Its control processors are redundant dynamically upgradable and designed for concurrent maintenance The ED 5000 has redundant power supplies and a fan assembly with two fans Each fan can be replaced separately The power supply and fan assembly are concurrently 117 maintainable In addition the ED 5000 can accept firmware updates and most configuration commands in the online state thus these operations are not disruptive A single failure on the ED 5000 will at most result in the loss of a path to the fabric of a connected N_Port This can be alleviated by ensuring that each attached node has at least two N_Port connections to the director McDATA ED 6064 Enterprise Fibre Channel Director Description The McDATA ED 6064 Enterprise Fibre Channel Director is a 64 port Fibre Channel Director whose design is based upon the IBM ESCON Director designed and manufactured for IBM by McDATA As such it provides high data availability across a SAN as well as the performance scalability and data integrity required by today s demanding e business and other mission critical applications It provides the capability of attaching FICON systems and devices as well as Fibre Channel systems and devices The McDATA ED 6064 Director requires only 9U of rack space for 64 ports Port density has been been improved by the combination of standard LC small form factor optical
41. grounded to prevent an electrical shock Before installing or removing signal cables ensure that the power cables for the system unit and all attached devices are unplugged When adding or removing any additional devices to or from the system ensure that the power cables for those devices are unplugged before the signal cables are connected If possible disconnect all power cables from the existing system before you add a device Use one hand when possible to connect or disconnect signal cables to prevent a possible shock from touching two surfaces with different electrical potentials During an electrical storm do not connect cables for display stations printers telephones or station protectors for communication lines CAUTION This product is equipped with a three wire power cable and plug for the user s safety Use this power cable with a properly grounded electrical outlet to avoid electrical shock DANGER To prevent electrical shock hazard disconnect the power cable from the electrical outlet before relocating the system CAUTION This unit has more than one power supply cord To reduce the risk of electrical shock disconnect two power supply cords before servicing Handling Static Sensitive Devices Attention Static electricity can damage this device and your system unit To avoid damage keep this device in its static protective bag until you are ready to install it To reduce the possibility
42. information pertaining to the 2109 S08 S16 and F16 SAN Fibre Channel Switches Product Information This information describes the SAN Fibre Channel Switch models Item Description 2109 Model S08 PN 2109S08 SAN Fibre Channel Switch with four short wave GBICs and one to four additional short wave or long wave GBICs 2109 Model S16 PN 2109S16 SAN Fibre Channel Switch with four short wave GBICs standard and one to twelve additional short wave or long wave GBICs 2109 Model F16 PN 2109F 16 SAN Fibre Channel Switch with 16 SFP ports capable of speeds up to two Gigabits per second Feature Codes for SAN Fibre Channel Switch Products The following feature codes are available for the SAN Fibre Channel Switch models S08 and S16 e FC 2010 PN 03K9308 One short wave GBIC e FC 2020 PN 03K9307 One long wave GBIC e FC 6103 PN 09L5403 Additional power supply 114 IBM Fibre Channel Planning and Integration User s Guide and Service Information Publications and Other Sources of Information For further information about these products refer to the following Related General Information and Operation IBM SAN Fibre Channel Switch 2109 Model S08 User s Guide SC26 7349 IBM SAN Fibre Channel Switch 2109 Model S16 User s Guide SC26 7351 e Channel Switch IBM TotalStorage SAN Fibre Channel Switch 2109 Model F16 User s Guide GC26 7439 Planning Installation Configuration and Servicing IBM SAN Fibre Channel Switch 2109 Mo
43. is fully operational Is the SAN Data Gateway operational online and correctly set up NO Refer to the Service Manual for the SAN Data Gateway and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with the next step YES Go to Step 0040 3 Chapter 6 Problem Determination 45 Step 0040 3 Determine that the Fibre Channel Switch is operational online and correctly set up Note If there is no Fibre Channel Switch in the configuration skip this step Go to Perform the following steps 1 2 3 4 Ensure that the Fibre Channel Switch is powered on Ensure that its appropriate Fibre Channel port s are cabled Ensure that its appropriate Fibre Channel port s are enabled Ensure that the Fibre Channel Switch is properly configured that is it is correctly zoned and the applicable ports are not blocked Examine the AIX system s error log for entries associated with this failure whose Probable Causes list includes the Fibre Channel Switch Using the Service Manual for the Fibre Channel Switch verify run diagnostics and so on that the Switch is fully operational Determine if ports on
44. is not ready This is a hardware problem 03 Indicates a media error This is a hardware problem 04 Indicates a hardware error at the device Note If any of these conditions are true and if the PP field equals 5D and the QQ field equals 00 then the drive is about to fail and should be replaced as soon as possible PP SCSI Request Additional Sense Code ASC QQ SCSI Request Additional Sense Code Qualifier ASCQ N The Open Count field This 32 bit field indicates the number of times the device has been opened from the closed state This is useful for determining if all the errors are associated with a specific removable media unit 90 IBM Fibre Channel Planning and Integration User s Guide and Service Information Status Validity of 0 A status validity of 0 indicates that the adapter driver returned an error to the disk driver but did not set the status validity Additional information has been added to this error log to assist in determining the cause of failure Sense Data Layout LLOO CCCC CCCC CCCC CCCC CCCC CCCC CCCC CCCC RRRR RRRR RRRR VVOO 0000 BBBB BBBB DDDD DDDD EEXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXX
45. items received with the hardware feature to ensure all required items are present Preparing for Installation of the Other Fibre Channel Devices To prepare for installing each of the Fibre Channel devices in the configuration refer to the appropriate appendix for each device Each device s appendix contains a section called Publications and other Sources of Information This section contains a list of publications and Web sites that provide device specific instructions and information needed for installing configuring operating and servicing of that device 10 IBM Fibre Channel Planning and Integration User s Guide and Service Information Chapter 3 Fibre Channel Installation This chapter provides information on how to install Fibre Channel adapters and devices Before proceeding ensure that you read and understand the information in Chapter 2 Planning Information on page 9 When you are installing components in the Fibre Channel environment install them in a logical sequence In general the logical sequence is to install components progressively outward from the host In this way facilities are available to test the next sequential component being installed thus providing a building block approach to the installation effort It is normally the best approach to install the fiber cabling first Ata minimum the cabling must be installed as each new device is added The recommended Fibre Channel installation sequence is
46. kilometers or combination Fibre Port Module one long wave port and three short wave ports Port Modules can be combined for a maximum of sixty four ports McDATA ES 3016 and ES 3032 Fabric Switch Description McDATA Fabric Switches are offered in a 16 port model and a 32 port model Each model includes sixteen short wave transceivers for device interconnection at a maximum distance of 500 meters A mixture of short wave and long wave 20 km ports can be configured by adding transceivers General ports G_Port automatically determine the port type when connected to a fabric port F_port or an expansion port E_port These Fabric Switches require only 1U of rack space for 16 ports and just 1 5U for 32 ports The McDATA Fabric Switches provides hot swappable load sharing dual power supplies which allows the switch to remain on line if one supply fails Dual power cords allow attachment to separate power sources for improved availability Hot swappable power and cooling components eliminate downtime for service when replacing a failed component and eliminate the risk of erroneously cabling a replacement switch due to a simple component failure Special tools are not required to replace power supplies and fans Hot pluggable optical transceivers can be replaced without taking the switch offline Firmware upgrades can be downloaded and activated while the fabric switch remains operational Connectivity and Configuration Features McDATA Fabric Swi
47. management request that is not a target reset abort no clear ACA Command is included 0000 001F FSCSILERR8 Data overrun reported in FCP response information unit from FCP device Command element is included 0000 0020 FSCSI_ERR6 Timeout for an unopened device N_Port ID is not valid 0000 0021 FSCSI_ERR6 Timeout for an unopened device N_Port ID LUN is not valid target_info information is included 0000 0022 FSCSILERR6 Timeout for a device with no active commands dev_info information is included 0000 0023 FSCSILERR6 Timeout for command that is not active dev_info information is included 0000 0024 FSCSI_ERR6 The cancel for the SCSI abort due to normal I O timeout failed to be issued dev_info information is included 0000 0025 FSCSI_ERR6 The adapter driver s output routine failed when the protocol driver issue a SCSI abort due to normal I O timeout to it The return code and dev_info information are included 0000 0026 FSCSILERR4 A device cancel timed out There are still commands active at the adapter that were not flushed back dev_info information is included 0000 0027 FSCSI_ERR6 A device cancel timed out and the retry of the cancel failed dev_info information is included 0000 0028 FSCSI_ERR4 A target cancel timed out target_info information is included 0000 0029 FSCSI_ERR4 A relogin issued to the adapter driver s command entry point timed out target_info information is included 0000 002A FSCSI_ERR4 A process login P
48. memory modules where buffers are managed Additional features of the ED 5000 include an operator panel one or two Ethernet interfaces and a maintenance port The ED 5000 supports a large number of features normally associated with Fibre Channel switches The supported features include Buffer to Buffer Credit of 1 to 16 Port Blocking and Zoning and support for both 50 and 62 5 micron fibers The ED 5000 supports connectivity with AIX system adapters and a number of storage devices using Fibre Channel connections as well as the IBM 9672 Parallel Enterprise G5 G6 and zSeries servers and IBM storage devices using FICON connections For a complete list as well as configuration examples refer to Appendix D Fibre Channel Devices and Configurations on page 95 The ED 5000 does not support interconnection with arbitrated loops As a result when attaching devices requiring Loop attachment such as the Magstar 3590 Model E11 and E1A Tape Subsystem the McDATA ES 1000 Switch must also be included in the configuration Director management functions of the ED 5000 are accomplished through a Microsoft Windows NT based server connected to the director s Ethernet port Operations such as Zoning and Port Blocking are handled by a Windows NT based utility product called Enterprise Fabric Connectivity EFC Reliability Availability and Serviceability RAS Features The ED 5000 supports redundancy of most of its functional units such that the
49. models from the same manufacturer rarely employed I O interfacing techniques with any degree of commonality In 1964 IBM introduced the IBM System 360 which incorporated a Channel I O subsystem using a standard parallel I O interface Over the next few years this interface became a de facto industry standard and the idea of a common O interface for computers became the norm In the late 1980 s IBM introduced the Enterprise System Connection ESCON interface to overcome the distance bandwidth and cable bulk deficiencies associated with the parallel I O interface However ESCON like the parallel interface was oriented strictly toward S 390 I O subsystems By the mid 1980 s a variety of desk top computers had been introduced Originally these computers were not interconnected but as their usage proliferated peer to peer networks became common These networks employ technologies such as Ethernet and Token Ring As this segment of the industry has matured a need for high speed dependable versatile connectivity has developed The need to connect large networks of small computers mainframes and storage devices cannot be satisfied by Token Ring Ethernet ESCON or SCSI Fibre Channel was conceived to meet this requirement Computer I O Interface Strategies Historically there have been two diverse computer interface strategies The first known as a Channel usually refers to the input output I O subsystems employed i
50. not support the same FC class The device s FC class mask is included in the rc field target_info information is also included 0000 0062 FSCSI_ERR6 Output routine failed to accept PRLI IOCB for a device during an SCIOLSTART The return code and dev_info information are included 0000 0065 FSCSI_ERR6 Output routine failed to accept SCIOLHALT IOCB for a device The return code and dev_info information are included 0000 0066 FSCSI_ERR6 Output routine failed to accept SCIOLRESET IOCB for a device The return code and dev_info information are included 0000 0067 FSCSI_ERR6 Output routine failed to accept Clear ACA IOCB SC_CLEAR_ACA in strategy for a device The return code and dev_info information are included 0000 0068 FSCSI_ERR6 Output routine failed to accept Abort IOCB SC_Q_CLR in strategy for a device The return code and dev_info information are included Appendix C Error Messages 87 Table 5 Error Number Information for the Fibre Channel SCSI Protocol continued Error Number Error Template Description of Error 0000 0069 FSCSI_ERR6 Output routine failed to accept Clear ACA IOCB SC_Q_RESUME SC_CLEAR_ACA in strategy for a device The return code and dev_info information are included 0000 006A FSCSI_ERR6 Output routine failed to accept normal I O SCSI IOCB for a device The return code and command are included 0000 006B FSCSI_ERR6 Output rou
51. of electrostatic discharge follow the precautions listed below x Limit your movement Movement can cause static electricity to build up around you Handle the device carefully holding it by its edges or its frame Do not touch solder joints pins or exposed printed circuitry Do not leave the device where others can handle and possibly damage the device While the device is still in its anti static package touch it to an unpainted metal part of the system unit for at least two seconds This drains static electricity from the package and from your body Remove the device from its package and install it directly into your system unit without setting it down If it is necessary to set the device down place it on its static protective package If your device is an adapter place it component side up Do not place the device on your system unit cover or on a metal table Take additional care when handling devices during cold weather as heating reduces indoor humidity and increases static electricity IBM Fibre Channel Planning and Integration User s Guide and Service Information About This Book This book provides information on the following e General concepts of Fibre Channel connectivity e Available Fibre Channel Devices e How to integrate the various Fibre Channel components into an efficient Fibre Channel network Details of the Fibre Channel protocol are not addressed but general information is provided regarding s
52. order to maintain the separate independent approval of the manufacturer s adapters it is essential that other optional cards not supplied by the manufacturer do not use main voltages or any other excessive voltages Seek advice from a competent engineer before installing other adapters not supplied by the manufacturer 70 IBM Fibre Channel Planning and Integration User s Guide and Service Information Avis de conformit aux normes du minist re des Communications du Canada Cet appareil num rique de la classe B est conform a la norme NMB 003 du Canada Canadian Department of Communications Compliance Statement This Class B digital apparatus complies with Canadian ICES 003 VCCI Statement IOE ZAIAATRAITRECT _OKECRERER CHATS BRASS ELITI L HHN ET CLOGSICMIEAADBDLaIR 2T AZALIBRANAZLCERSVET VCCI A The following is a summary of the VCCI Japanese statement in the box above This product is a Class B Information Technology Equipment and conforms to the standards set by the Voluntary Control Council for Interference by Information Technology Equipment VCCI This product is aimed to be used in a domestic environment When used near a radio or TV receiver it may become the cause of radio interference Read the instructions for correct handling Radio Protection for Germany Dieses Ger t ist berechtigt in bereinstimmung mit dem deutschen EMVG vom 9 Nov 92 das EG Konformitatszeichen zu f hren
53. problem determination that might be required Preparing for Installation of the Gigabit Fibre Channel PCI Adapter This section outlines the steps involved in preparing to install the Gigabit Fibre Channel PCI Adapter feature 6227 or the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus feature 6228 and the required software components Note The device drivers are included on the AIX installation media If the AIX operating system has not yet been installed install the adapter hardware first In this way the device drivers will be installed automatically with AIX Follow the instructions in Chapter 3 Fibre Channel Installation on page 11 to install the adapter If the AIX operating system is already installed install the device drivers prior to installing the adapter hardware The Gigabit Fibre Channel PCI Adapter requires the AIX operating system to be at level AIX 4 3 2 or higher The 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus requires the AIX operating system to be at level AIX 4 3 3 or higher For RS 6000 SP installations both adapters require the Parallel System Support Program PSSP to be at level 3 1 1 or higher To verify the software levels of your system perform the following steps 1 To check the level of AIX on your system run the following command oslevel 2 Ifyou have an RS 6000 SP system check the level of PSSP by performing step 4 on page 15 3 Return here and continue Inventory the
54. server 131 Fibre Channel RAID storage server 127 fibre channel storage hub 107 functional features 117 Magstar 3590 tape subsystem 145 description continued Magstar MP 3570 tape subsystem 143 Magstar MP 3575 family of tape libraries 144 SAN data gateway 111 SAN fibre channel switch 113 Ultrium 3580 tape subsystem 147 devices and configurations 95 devices fibre channel 95 director fibre channel 117 E EFC enterprise fabric connectivity 117 EMC symmetrix 3000 5000 and 9000 enterprise storage systems 141 enterprise fabric connectivity EFC 117 enterprise storage server 131 copy services 133 description 131 enterprise storage server 131 expert 133 features 132 interfaces 132 error log information for for other fibre channel devices 93 gigabit fibre channel adapter 75 error log information for gigabit fibre channel adapters 75 error log sense information fibre channel adapter 77 SCSI protocol 82 error log sense information SCSI 3 protocol 89 error messages 75 error report for the gigabit fibre channel adapter generating 65 extended SCSI channel distance 111 F failing function codes 64 feature code adapter feature 6227 7 adapter feature 6228 7 fibre channel cable routing 9 connection topology 2 McDATA ED 5000 description 117 director 117 functional features 117 supports redundancy 118 operating other fibre channel devices 27 operation 27 overview 1 151 fibre channel continued port types 5 prepari
55. the closed state This is useful for determining if all the errors are associated with a specific removable media unit Error Log Information for Other Fibre Channel Devices For error log information for each of the other Fibre Channel devices refer to the appropriate appendix for each device Each device s appendix contains a section called Publications and Other Sources of Information This section contains a list of publications and Web sites that provide device specific instructions and information needed for installing configuring operating and servicing of that device Appendix C Error Messages 93 94 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix D Fibre Channel Devices and Configurations This appendix discusses Fibre Channel devices and configurations Fibre Channel Devices The following Fibre Channel devices are available for direct connection to a Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S or a 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus FC 6228 Type 4 W port in a Fibre Channel Storage Area Network SAN configuration e McDATA ED 5000 Enterprise Fibre Channel Director e McDATA ED 6064 Enterprise Fibre Channel Director e McDATA ES 3016 and ES 3032 Enterprise Fibre Channel Switches e 2109 S08 and S16 SAN Fibre Channel Switches e 2109 F16 SAN Fibre Channel Switch e INRANGE FC 9000 Fibre Channel Director e 2103 H07 Fibre Channel Hub e 2108 G07
56. to obtain problem determination information Data obtained by tracing the Gigabit Fibre Channel Adapter device driver provides information about which code paths were used during the failing sequence of Fibre Channel operations This information often enables trained personnel to determine which individual component of the overall Fibre Channel environment was being exercised at the moment of failure thus pinpointing the area for concentrated problem determination procedures You can invoke the trace facility in either of two ways It may be started as an AIX command or it may be started using the Fibre Channel Adapter SMIT panels In either case the trace hooks for the Gigabit Fibre Channel Adapter device driver are 3B9 and 3C4 To start the trace from the AIX command line run the following command trace j 3B9 3C4 a To stop the trace from the AIX command line run the following command trcstop To format the trace from the AIX command line run the following command trerpt This causes the formatted trace information to be displayed on the screen To format the trace from the AIX command line and store it in a file called trace in the tmp directory run the following command trcrpt gt tmp trace To start the trace facility from the Gigabit Fibre Channel Adapters SMIT panels perform the following steps 1 Type the following smitty fcsa 2 Select FC Adapter 3 Select Trace a FC Adapter 4 Select Start Trace 64 IBM
57. transceivers and serial architecture The ED 6064 Director features redundant power supplies and cooling fans along with redundant active components that support automatic failover Other high availability capabilities include hot swapping for all field replaceable units nondisruptive firmware updates and automatic fault detection and isolation In addition call home and e mail capabilities automatically alert support and maintenance personnel to accelerate problem resolution Director High Availability function provides redundancy for all active electronic components helping to enable continuous data access and high performance in the event of a single component failure Together these features help enable nondisruptive maintenance and upgrades without loss of data access Connectivity and Configuration Features 118 The ED 6064 Director provides Fibre Channel connectivity to IBM and non IBM servers and IBM storage including FICON Attachment of IBM 9672 Parallel Enterprise G5 G6 and zSeries servers and IBM storage with FICON adapters such as the IBM Magstar 3590 Model A60 Tape Controller and IBM Enterprise Storage Server e Fibre Channel attachment of IBM and non IBM servers such as IBM pSeries RS 6000 RS 6000 SP Sun Solaris Hewlett Packard HP UX servers IBM xSeries Netfinity and non IBM Intel based servers with Windows NT 2000 e Fibre Channel attachment of IBM and non IBM disk storage such as IBM Enterprise Storag
58. visit the following Web address http www storage ibm com hardsoft products ess ess htm Appendix K IBM 2105 E10 E20 F10 F20 Enterprise Storage Server 135 136 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix L IBM 3552 Fibre Array Storage Technology FAStT500 Storage Server This appendix describes the IBM FAStT500 Storage Server IBM FAStT500 Storage Server Description The IBM 3552 Fibre Array Storage Technology FAStT500 is a complete storage subsystem that is housed in standard 19 inch storage racks It offers concurrent multihost system attachment through a Fibre Channel interface supporting AIX systems as well as Windows NT and other open system servers The FAStT500 can share the storage and data of up to one hundred FC disk drives among multiple servers simultaneously Note The FAStT500 is supported in a switched configuration only See Appendix D Fibre Channel Devices and Configurations on page 95 for supported switches Arbitrated Loop and direct attachment are not supported configurations The FAStT500 RAID Controller provides a fiber to fiber interface from hosts to data storage devices The dual controllers act independently yet offer a redundant path for failover redundancy Normally each controller manages approximately half of the attached storage however in the event of a fabric component failure the FAStT500 routes all transactions through the remainin
59. you can choose the IBM 3583 Ultrium Scalable Tape Library compressed data capacity of up top 14 4 TB or the IBM 3584 UltraScalable Tape Library compressed data capacity of up to 496 2 TB The tape media is interchangeable among all four devices The IBM 3580 3583 and 3584 all feature Ultra2 Wide SCSI Low Voltage Differential LVD or Ultra Wide SCSI High Voltage Differential HVD interfaces and can attach to IBM and non IBM servers workstations and personal computers that support these interfaces As a SCSI attached tape solution the IBM 3580 connects to Fibre Channel server adapters through selected IBM Storage Area Network Data Gateway products It has a sustained data transfer rate of up to 30 MB sec compressed In addition the 3583 and 3584 Libraries offer dual 2 Gigabit Fibre Channel ports for native Fibre Channel attachment to servers and Fibre Channel switches in a Storage Area Network The following models of the 3580 Tape Drive are available and vary according to the type of Small Computer Systems Interface SCSI each uses to communicate with the host e Model L11 uses the Ultra2 Low Voltage Differential Single Ended LVD SE interface e Model H11 uses the Ultra High Voltage Differential HVD DIFF interface The IBM 3580 has an LCD display and indicators for write protection tape in use drive cleaning and data compression These solutions offer highly automated centrally scheduled policy managed backup and archiv
60. 0 1 3 5 and 0 1 also known as RAID 10 e High performance through 200 MB sec bandwidth with two Fibre Channel ports e Flexibility to configure distances up to 10 km e Scalability to accommodate growth starting from 36 GB to over 2 TB e High availability through automatic I O path failover e Concurrent maintenance with hot pluggable components e Easier centralized management of distributed storage with the IBM StorWatch Fibre Channel RAID Specialist Multihost Attachment The Fibre Channel RAID Storage Server is a storage solution for Fibre Channel and Fibre Channel SAN environments This storage system can be attached to AIX systems as well as other open system servers running UNIX Windows NT or Novell NetWare It can be shared simultaneously by two operating systems Multiple server attachments can be implemented through the attachment of the Fibre Channel Storage Hub The Fibre Channel RAID Storage Server supports heterogeneous environments by enabling each RAID controller to work independently 127 High availability design The Fibre Channel RAID Storage Server has dual active RAID controllers that provide high throughput and redundancy Both controllers can be simultaneously active to provide seamless failover capability in case of emergency To increase availability each RAID controller supports up to 256 MB of battery backed cache and can mirror write operations Dual fans and power supplies further support 24x7 operations
61. 0 for IBM AIX HPUX and Sun Solaris Installation and Support Guide GC26 7441 IBM Fibre Channel Hardware Maintenance Manual 19K61230 pdf on CD For additional information on the IBM 3552 Fibre storage Technology FAStT500 Storage Server visit the following Web address http www storage ibm com hardsoft products fast500 fast500 htm 140 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix M EMC Symmetrix 3000 5000 and 8000 Enterprise Storage Systems This appendix describes the EMC Symmetrix 3000 5000 and 8000 Enterprise Storage Systems Symmetrix 3000 5000 and 8000 Enterprise Storage Systems Description The EMC Symmetrix 3000 5000 and 8000 Enterprise Storage Systems are members of a product family of storage servers and attached storage devices The Symmetrix Family provides heterogeneous server connectivity using Fibre Channel as well as Ultra SCSI FWD SCSI channels and Block Multiplexer ESCON and FICON mainframe channels Each channel supports simultaneous peak transfer rates for maximum connectivity and performance Symmetrix supports channel connections to all supported servers concurrently providing maximum information throughput The Symmetrix 3000 family consists of the 3630 3830 and 3930 and is intended for open system connectivity In addition to Fibre Channel the EMC Symmetrix Enterprise Storage Systems can also attach through Fast Wide Differential FWD SCSI
62. 221 447 701 G 447 Description Error log analysis indicates that an error has 221 occurred with the adapter 447 703 G 447 Description Error log analysis indicates that an unknown adapter error has occurred 447 704 G 447 Description Error log analysis indicates that an adapter error has occured 677 101 D 677 Description The Fibre Channel Adapter configuration failed 677 102 D 677 Description The Reset test failed 677 103 D 677 Description The Register test failed 677 104 D 677 Description The SRAM test failed 677 105 D 677 Description The Internal Wrap test failed 677 106 D 677 Description The Gigabaud Link Module GLM failed 62 IBM Fibre Channel Planning and Integration User s Guide and Service Information Table 2 Service Request Numbers for Fibre Channel Adapters continued Service SRN Failing Description and Action Unless otherwise indicated Request Src Function use MAP 0210 See note following the table Number Codes 677 107 D 677 Description The External Wrap test failed 677 201 D 677 Description The Configuration Register test failed 221 677 202 D 677 Description The Interrupt test failed 221 677 203 D 677 Description The PCI Wrap test failed 221 677 204 D 677 Description The DMA test failed 221 677 205 D 677 Description I O error on a read write operation 221 677 701 G 677 Description Error log analysis indicates that an error has 221 occurred with the adapter
63. 28 Type 4 W requires a PCI 2 2 64 bit 66 MHz slot or a PCI 2 1 32 bit 33 MHz slot Short Wave Optics non OFC support e Upgradable Microcode Notes 1 A PCI 2 2 64 Bit 66 MHz slot is recommended for the Type 4 W adapter 2 OFC Open Fiber Control These Gigabit Fibre Channel Adapters support connectivity with a number of Fibre Channel Storage Devices Switches Hubs and other devices in a number of topologies and configurations For a complete list of available devices and example configurations refer to Appendix D Fibre Channel Devices and Configurations on page 95 For more details on the various devices refer to the appropriate appendix for that device Chapter 1 Introduction 7 8 IBM Fibre Channel Planning and Integration User s Guide and Service Information Chapter 2 Planning Information This section provides information on how to plan your Fibre Channel configuration site plan and cabling Planning Your Fibre Channel Configuration Use the information in this section to plan your Fibre Channel configuration site plan and cabling Perform the following steps to configure your Fibre Channel system Use Appendix D Fibre Channel Devices and Configurations on page 95 to guide you in planning your system s configuration Ensure that the Optical Fiber Cabling is properly planned and installed correctly Ensure that all Hosts Fibre Channel Adapters Fibre Channel Disk Subsystems
64. 3580 Ultrium Tape Drive Setup Operator and Service Guide GA32 0415 IBM Magstar 3590 Tape Subsystem Multiplatform Implementation SG24 2594 IBM Magstar 3590 Tape Subsystem Operator s Guide GA32 0330 IBM Magstar MP 3570 Tape Subsystem Operator s Guide C Series Models GA32 0393 IBM Magstar MP 3575 Tape Library Dataserver Operator s Guide GA32 0381 IBM 3580 Ultrium Tape Drive Setup Operator and Service Guide GA32 0415 148 IBM Fibre Channel Planning and Integration User s Guide and Service Information Servicing IBM Magstar 3590 Tape Subsystem Hardware Reference GA32 0331 IBM Magstar MP 3570 Tape Subsystem Hardware Reference C Series Models GA32 0394 IBM Magstar MP 3575 Tape Library Dataserver Hardware Reference GA32 0382 IBM 3580 Ultrium Tape Drive Setup Operator and Service Guide GA32 0415 The following information is available through the Internet e These and other Magstar 3590 Publications http Awww storage ibm com hardsoft tape pubs pubs3590 html e For an introduction to the Magstar 3590 Tape Subsystem see http Awww storage ibm com hardsoft tape 3590 index html e For more information on the Magstar MP 3570 Multi Purpose Tape Subsystem see http www storage ibm com hardsoft tape 3570 index html For more information on the Magstar MP 3575 Multi Purpose Tape Librar Dataserver see e For more information on the 3580 Ultrium Tape Drive visit the following Web address http
65. 5 The Host server attachment can have up to 16 Fibre Channel ports or up to 32 SCSI ports connecting to Fibre Channel SANs through the SAN Data Gateway Enterprise Storage Server 2105 E10 Item Description Disk storage capacity 420 GB to 1 68 TB Maximum storage system capacity 7 28 TB Cache size 6 GB 134 IBM Fibre Channel Planning and Integration User s Guide and Service Information Enterprise Storage Server 2105 E20 Item Description Disk storage capacity 420 GB to 3 36 TB Maximum storage system capacity 11 2 TB Cache size 6 GB Enterprise Storage Server 2105 F10 Item Description Disk storage capacity 420 GB to 1 68 TB Cache size 8 to 32 GB Enterprise Storage Server 2105 F20 Item Description Disk storage capacity 420 GB to 11 2 TB Cache size 8 to 32 GB Publications and Other Sources of Information For further information about the IBM 2105 E10 E20 F10 F20 Enterprise Storage Server refer to the following General Information Planning and Installation e IBM ESS Introduction and Planning Guide 2105 Models E10 E20 F10 and F20 GC26 7294 Configuration e IBM ESS Host System Attachment Guide 2105 Models E10 E20 F10 and F20 SC26 7296 IBM ESS Configuration Planner SC26 7353 Operation IBM ESS User s Guide 2105 Models E10 E20 F10 and F20 SC26 7295 Servicing IBM ESS SCSI Command Reference 2105 Models E10 and E20 SC26 7297 For additional information on supported interfaces and host systems
66. 500 meters with 50 125 um 62 5 125 um multimode fiber 175 meters with 62 5 125 um Description 9 125 um singlemode fiber Up to 10 km with 9 125 um 108 IBM Fibre Channel Planning and Integration User s Guide and Service Information Publications and Other Sources of Information For further information about the IBM 2103 H07 Fibre Channel Storage Hub refer to the following IBM Fibre Channel Storage Hub 2103 Model H07 Installation Service and User s Guide SC26 7289 For more information about the Fibre Channel Storage Hub visit the following Web address http www storage iom com hardsoft products fchub fchub htm Appendix E IBM 2103 H07 Fibre Channel Storage Hub 109 110 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix F IBM 2108 G07 SAN Data Gateway This appendix describes the Storage Area Network SAN Data Gateway IBM SAN Data Gateway Description The 2108 G07 Storage Area Network SAN Data Gateway is a hardware solution that enables the attachment of SCSI storage systems into AIX System Fibre Channel configurations to form Storage Area Networks SANs Fibre Channel SANs can do the following e Interconnect RS 6000 and pSeries systems and other servers with the Enterprise Storage Server and the Magstar tape storage systems e Simplify enterprise storage consolidation e Improve access to data e Enhance disaster recovery The IBM 2108 G07 SAN Data Gateway
67. BRE CHANNEL SWITCH Recommended Actions NO FURTHER ACTION REQUIRED UNLESS PROBLEM PERSISTS CHECK CABLES AND THEIR CONNECTIONS Detail Data SENSE DATA 0000 0000 0000 003D 0200 0008 0000 0000 0000 0000 0000 0007 0000 000A 0000 012C 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0600 0000 0000 0001 0000 0000 0000 0000 0000 07D0 0000 07D0 0000 076C 0000 0064 0000 000F 7400 0000 00A0 2C02 0000 0000 0000 0000 0000 0000 0000 0000 0200 0200 068D 0100 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 Figure 7 Example of a Fibre Channel Adapter Error Log IBM Fibre Channel Planning and Integration User s Guide and Service Information PCI Fibre Channel Adapter Error Log Sense Information The detail sense data log in the FCS_ERR template for PCI FC adapters uses the structure error_log_def defined in the following file src rspc kernext pci fcs efcdd h Sense Data Layout RRRR RRRR EEEE EEEE TTCC RRRR NNNN NNNN NNNN NNNN IIII IIII MMMM MMMM LLLL LULL AAAA AAAA BBBB BBBB DDDD DDDD FFFF FFFF GGGG GGGG HHHH HHHH JJJJ JJJJ KKKK KKKK PPPP PPPP SSRR UUUU VVVV VVVV WWWW WWWW XXXX XXXX YYYY YYYY ZZZZ ZZZZ 2222 2222 3333 3333 4444 4444 QQQQ QQQQ QQQQ QQQQ QQQA QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQA QQQQ QQQQ QQQQ QQQQ QQQ
68. E11 E1A Tape Subsystem The Magstar 3590 Tape Subsystem is a family of magnetic tape drives and controllers This family replaces the 3490 and 3490E Magnetic Tape Subsystems The Magstar 3590 attaches to its own S 390 Control Unit and the 2108 GO7 SAN Data Gateway through SCSI and Ultra SCSI interfaces It attaches to host servers through these interfaces as well as through native Fibre Channel interfaces Attachment to AIX systems as well as other UNIX Windows NT and S 390 class mainframes is supported The Magstar 3590 can be connected into a Fibre Channel Storage Area Network SAN through the IBM 2108 G07 SAN Data Gateway or directly through its native Fibre Channel adapters The Magstar 3590 is available in five models for SCSI and Ultra SCSI attachment Models E11 and E1A also provide Fibre Channel attachment e The Magstar 3590 Models B11 and E11 are rack mounted and incorporate a 10 cartridge Automatic Tape Loader ATL for high capacity unattended operation The Models B11 and E11 can be modified to Models B1A or E1A The Magstar 3590 Models B1A and E1A have no ATL and are designed to be incorporated into the Magstar 3494 tape library e The Magstar Model C12 frame with one to four Model B1A or E1A tape drives provides attachment to the StorageTek 4410 and 9310 ACS Existing Magstar Model B11 and B1A Tape Drives B Models can be field upgraded to the E Models and the E Model drives can both read 128 track and write 256 tra
69. FC Disk Array s HBA Hub HBA Hub Figure 9 Example of a Hub Configuration With One Host System and Four Hubs Hub Configuration With Two Host Systems The following illustration shows an example of two host systems connected through hubs to a Fibre Channel disk array In this example each system is connected to both hubs HBA Hub HBA FC Disk Array s HBA Hub HBA Figure 10 Example of a Hub Configuration With Two Host Systems and Two Hubs Hub Configuration With Four Host Systems The following illustration shows an example of four host systems connected through a single hub to a Fibre Channel disk array This example also illustrates the maximum of seven connections available on a hub HBA HBA FC Disk Hub Array s HBA HBA Figure 11 Example of a Hub configuration With four Host Systems and One Hub Switch Configurations Switches can be included in a Fibre Channel configuration allowing multiple host system HBAs and multiple Fibre Channel devices to be connected in a Fibre Channel fabric topology 100 IBM Fibre Channel Planning and Integration User s Guide and Service Information Switches allow you to set zones of permitted and non permitted port to port connections You can block certain ports from connecting to each other You can use zon
70. Figure 5 Gigabit Fibre Channel PCI Adapter 1 Multimode Fiber SC Connector 2 Data Link Status LEDs 3 Jumper JX1 Pins 1 to 2 only Figure 6 shows the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus 59 JX1 mgg i ee vou U 1 LE 2 O ial O i i rT Ls f par Figure 6 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus 1 Jumper JX1 Pins 1 to 2 only 2 Data Link Status LEDs 3 Multimode Fiber LC Connector 60 IBM Fibre Channel Planning and Integration User s Guide and Service Information Adapter LEDs The adapters have green and yellow LEDs both located near the connectors The LEDs indicate the state of their adapter Table 1 describes the various states of the LEDs For the location of the two LEDs on the Gigabit Fibre Channel PCI Adapter see For the location of the two LEDs on the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus see Figure 6 on page 60 Table 1 LED Status Information Green LED Yellow LED State OFF OFF Wake up failure dead board OFF ON POST failure dead board OFF Slow blink 1HZ Wake up failure OFF Fast blink 4HZ Failure in POST OFF Flashing irregularly POST processing in progress ON OFF Failure while functioning ON ON Failure w
71. IX Diagnostics for the Gigabit Fibre Channel Adapters Service Request Numbers by ea ti F Source of SRN Failing Function Codes Tracing the Gigabit Fibre Channel Adapter Generating an Error Report for the Gigabit Fibre Channel Adapter 2109 SAN Fibre Channel Switch Service Aid eee eee Other Fibre Channel Devices Appendix A Communications Statements Federal Communications Commission FCC Statement European Union EU Statement SS ms ee International Electrotechnical Commission IEC Statement United Kingdom Telecommunications Safety Requirements Avis de conformit aux normes du minist re des Communications du Canada Canadian Department of Communications Compliance Statement VCCI Statement Radio Protection for Germany Appendix B Notices Appendix C Error Messages Error Log Information for Gigabit Fibre Channel Adapters PCI Fibre Channel Adapter Error Log Sense Information PCI Fibre Channel Adapter SCSI Protocol Error Log Sense Information ee PCI Fibre Channel Adapter SCSI 3 Protocol Disk CD ROM Read Write ae Device Error Log Sense Information Error Log Information for Other Fibre Channel Devices Appendix D Fibre Channel Devices and Configurations Fibre Channel Devices Fibre Channel Considerations Switch Cascading Controlling Connectivity Within the Fabric Link Distances a a e E E Fibre Channel Boot Concurrent Storage Subsystem Access Fibre Channel Configurations
72. ND EITHER EXPRESSED OR IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF NON INFRINGEMENT MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE Some states do not allow disclaimer of express or implied warranties in certain transactions therefore this statement may not apply to you This information could include technical inaccuracies or typographical errors Changes are periodically made to the information herein these changes will be incorporated in new editions of the publication The manufacturer may make improvements and or changes in the product s and or the program s described in this publication at any time without notice Information concerning products made by other than the manufacturer was obtained from the suppliers of those products their published announcements or other publicly available sources The manufacturer has not tested those products and cannot confirm the accuracy of performance compatibility or any other claims related to products made by other than the manufacturer Questions on the capabilities of products made by other than the manufacturer should be addressed to the suppliers of those products 73 74 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix C Error Messages This appendix describes error messages encountered in the Fibre Channel environment Error Log Information for Gigabit Fibre Channel Adapters An error log for Gigabit Fibre Cha
73. P source directory proceed to section Creating an Installable Image on the Control Workstation to create the image If the required device driver images are already in the LPP source directory proceed to section Finalizing Preparations for Device Driver Installation on page 17 to finalize preparations for installation Creating an Installable Image on the Control Workstation Do the following to create an installable image on the control workstation Note If your system is partitioned that is there is more than one operating system among the nodes you may only install this adapter in nodes that have AIX 4 3 3 and higher 1 If you are not already logged in to the control workstation as root user do so now You may also need to export the working collective Refer to section Setting Up land Checking System Prerequisites on page 14 2 Insert the installation media into the control workstation s drive 3 Transfer the files to the control workstation s LPP source as follows a Type the following smitty bffcreate 16 IBM Fibre Channel Planning and Integration User s Guide and Service Information Press Enter b Select INPUT device directory for software In the entry field type the following spdata sys1 instal1 1ppsource substituting the directory name which was recorded in step 4b on page 16 for lppsource Press Enter c Move the cursor to the SOFTWARE package to copy option and pres
74. Q QQQA QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQA QQQQ QQQQ QQQQ QQQQ QQQQ QQQA QQAQ QQQQ QQQQ QQQQ QQQQ QQQQ QQQQ cae ata QQQQ QQQQ QQQA QQQQ QQQQ QQQQ QQQQ QQQQ QQQA QQQQ QQQQ QQQQ QQQQ QQQQ The above fields are interpreted as follows Field E T Interpretation Error number The error numbers are described in Table 4 on page 79 Type Possible values are 0 Neither mailbox nor IOCB included 1 Mailbox included in data field 2 IOCB included in data field 3 Adapter register contents included 4 Adapter driver data 5 Command structure data How this adapter is connected Possible values are 0 Unknown how it is connected 1 Adapter is connected point to point 2 Adapter is connected to a switch 3 Adapter is connected to an arbitrated loop N_Port ID of the adapter Reserved for future use Number of IOCBs issued by the adapter driver since it was configured Number of mailboxes issued by the adapter driver since it was configured How often in seconds the adapter driver issues a READ_LNK_STAT mailbox opcode 0x12 Number of times the link has failed Number of times the loss of synchronization has occurred Number of times the loss of signal has occurred Appendix C Error Messages 77 F Number of times a primitive sequence error has occurred G Number of times a transmission character that is not valid has been seen H Number of times a CRC error occurred from a receive frame J Number of t
75. RAID Storage Server 129 130 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix K IBM 2105 E10 E20 F10 F20 Enterprise Storage Server This appendix describes the Enterprise Storage Server ESS IBM Enterprise Storage Server Description The 2105 E10 E20 F10 F20 Enterprise Storage Server ESS is a member of the Seascape product family of storage servers and attached storage devices disk drive modules The ESS provides integrated caching and RAID support for the disk drive modules DDMs While the DDMs themselves are attached through a serial storage architecture SSA interface the ESS attaches to its host s through Fibre Channel FICON ESCON and SCSI 3 Interfaces The ESS can be connected into a Fibre Channel Storage Area Network SAN directly using its Fibre Channel interfaces or by using its SCSI 3 interfaces through the IBM 2108 G07 Storage Area Network Data Gateway The 2105 Models E20 and F20 with the expansion enclosures can provide up to 11 terabytes TB of storage capacity with a choice of 9 1 18 2 or 36 4 gigabyte GB DDMs The E20 and F20 can contain from 16 to 128 384 w expansion enclosure DDMs in increments of 8 The 2105 Models E10 and F10 support a maximum of 64 DDMs and do not support an expansion enclosure The ESS provides up to 16 Fibre Channel ports for directly connecting into a Fibre Channel SAN or from 2 to 16 SCSI 3 adapters each with 2 interfaces fo
76. RLI or a Port Discover PDISC timed out target_info information is included 0000 002B FSCSI_ERR8 Proc level task management function SCIORESET timed out Command is included 0000 002C FSCSI_ERR4 Interrupt level task management function Target Reset timed out Command is included 0000 002D FSCSILERR6 Unknown time out ID 0000 002E FSCSI_ERR4 Data underrun detected by adapter The command is included 0000 0030 FSCSI_ERR2 Async status received from adapter indicating a complete adapter failure 0000 0031 FSCSI_ERR2 Async status received from adapter driver indicating link is down 0000 0032 FSCSI_ERR2 Async status received from adapter driver indicating link is up 0000 0036 FSCSILERR6 Unknown async status returned from adapter driver 86 IBM Fibre Channel Planning and Integration User s Guide and Service Information Table 5 Error Number Information for the Fibre Channel SCSI Protocol continued Error Number Error Template Description of Error 0000 0037 FSCSI_ERR4 Async status received from adapter driver indicating a state change notification was received 0000 0038 FSCSI_ERR2 Unknown async status returned from adapter driver 0000 0040 FSCSI_ERR2 Adapter device driver detected error that indicates HOST IO BUS ERROR Command element included 0000 0041 FSCSI_ERR2 Adapter device driver detected error that indicates adapter hardware failure Command element included
77. RS 6000 server pSeries Fibre Channel Planning and Integration User s Guide and Service Information C23 4329 03 Note Before using this information and the product it supports be sure to read the general information under Product Warranties and Notices included with your system unit Fourth Edition May 2002 Before using this information and the product it supports read the information in Safety Notices on page ixjand Appendix B Notices on page 73 A reader s comment form is provided at the back of this publication If the form has been removed address comments to Information Development Department H6DS 905 6C006 11400 Burnet Road Austin Texas 78758 3493 To send comments electronically use this commercial internet address aix6kpub austin ibm com Any information that you supply may be used without incurring any obligation to you lnternational Business Machines Corporation 1999 2002 All rights reserved Note to U S Government Users Documentation related to restricted rights Use duplication or disclosure is subject to the restrictions set forth in the GSA ADP Schedule Contract with IBM Corp Contents Safety Notices Electrical Safety Handling Static Sensitive Devices About This Book How to Use This Book Highlighting ISO 9000 Online Publications Related Information Ordering Publications Trademarks Chapter 1 Introduction Fibre Channel Overview Co
78. SSNet provides the following Support for multiple ESSs A hub with 16 ports will support 7 ESSs Connectivity between the ESS and the ESS Specialist Web interface Improvements in Web performance compared to other Web browser connections Faster network connectivity and the elimination of network setup problems Ethernet connectivity through an Ethernet hub to the ESSNet An independent platform that facilitates installation and configuration of the ESS Software for maintenance and configuration 132 IBM Fibre Channel Planning and Integration User s Guide and Service Information All data that is sent between the ESS and the Web browser through the ESSNet is encrypted to avoid unauthorized modification of configuration commands Access to the interface is protected by passwords and authorization levels Accessing ESS Specialist and ESS Copy Services The IBM StorWatch Enterprise Storage Server Specialist ESS Specialist and StorWatch Enterprise Storage Server Copy Services ESS Copy Services are accessed from the ESSNet using the browser software ESS Copy Services are accessed from ESS Specialist ESS Specialist The ESS Specialist is a Web based interface that provides the means to configure the ESS From the Web interface the following tasks can be performed e Monitor error logs e View and modify the configuration including Add or delete host systems Configure host ports on the ESS Define disk groups Add volumes
79. T500 is supported in a switched environment only See Appendix D Fibre Channel Devices and Configurations on page 95 for supported switches FC 2010 PN 03K9308 One short wave GBIC FC 2020 PN 03K9307 One long wave GBIC Fibre Channel cable multimode optical 50um 5 meter FC 5825 PN 03K9305 FC 5805 PN 03K9306 Fibre Channel cable multimode optical 50um 25 meter Appendix L IBM 3552 Fibre Array Storage Technology FAStT500 Storage Server 139 Configuration Information The following table provides configuration information for the FAStT500 Table 9 FAStT500 Configuration Information Item Description RAID options RAID 0 1 0 1 3 5 Cache options Up to 512 MB read write cache Host interface Four Fibre Channel ports 100 MB sec supported in a switched only configuration Drive interface Ten bays for slim line hot swappable Fibre Channel disk drives Direct Fibre Channel connection to Controllers via enclosed mini hubs Drives 18 2GB 15 000 RPM 18 2 36 4 and 73 4GB 10 000 RPM Capacity Up to 16 TB maximum Publications and Other Sources of Information For further information about this product refer to the following General Information Fibre Array Storage Technology A FAStT Introduction an IBM Redbook SG24 6246 Planning Installation Configuration and Servicing IBM FAStTStorage Manager Version 7 1
80. TTY Connectivity and Configuration Features Cee ee e ee we 4 119 McDATA ES 1000 Switch Description 2 ee 120 Operating and Management Modes 120 Operating Modes 2 1 ee ee 1221 Management Modes Bobb Sap m ee we amp g Effect of Operating and Management Modes amp oy a nar a ee a Re OS ee nelle Publications and Other Sources of Information 122 vi IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix I INRANGE FC 9000 Fibre Channel Director INRANGE FC 9000 Fibre Channel Director ee FC 9000 Connnectivty a ag FC 9000 Configuration Options FC 9000 Director Management Product and Feature Information Publications and Other Sources of Information Appendix J IBM 2102 F10 Fibre Channel RAID Storage Server IBM 2102 F10 Fibre Channel RAID Storage Server Description Multihost Attachment Be Hp eo Se ok 8 High availability design Extended Distances High Performance Dynamic Management General Information Product Information Configuration Information Publications and Other Sources of Information General Information Planning Installation and Configuration Operation Servicing Appendix K IBM 2105 E10 E20 F10 F20 Enterprise Storage Server IBM Enterprise Storage Server Description food kook ER Enterprise Storage Server Features Enterprise Storage System Web Interfaces Ente
81. Tape Subsystem Introduction and Planning Guide GA32 0329 IBM Magstar 3590 Tape Subsystem Technical Guide SG24 2506 IBM Magstar 3590 Tape Subsystem Multiplatform Implementation SG24 2594 IBM Magstar 3590 Tape Subsystem Operator s Guide GA32 0330 IBM Magstar 3590 Tape Subsystem Hardware Reference GA32 0331 IBM Magstar MP 3570 Tape Subsystem Introduction and Planning Guide C Series Models GA32 0392 IBM Magstar MP 3575 Tape Library Dataserver Introduction and Planning Guide GA32 0380 IBM Magstar MP 3570 Tape Subsystem Operator s Guide C Series Models GA32 0393 IBM Magstar MP 3575 Tape Library Dataserver Operator s Guide GA32 0381 IBM Magstar MP 3570 Tape Subsystem Hardware Reference C Series Models GA32 0394 IBM Magstar MP 3575 Tape Library Dataserver Hardware Reference GA32 0382 IBM Fibre Channel Storage Manager and StorWatch for Fibre Channel Expert SC26 7285 IBM 3580 Ultrium Tape Drive Setup Operator and Service Guide GA32 0415 Preface xiii Ordering Publications Trademarks You can order publications from your sales representative or from your point of sale To order additional copies of this publication contact your sales representative and use order number SC23 4329 The following are trademarks of the International Business Machines Corporation in the United States other countries or both AIX S 390 e logo Seascape Enterprise Storage Server SP Enterprise Systems Connection StorW
82. X XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX NNNN NNNN XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX The above fields are interpreted as follows Field Interpretation LL The length of the failing SCSI command Cc The failing SCSI command The first two digits of this field indicate the SCSI opcode A list of SCSI opcodes and their meanings can be found at the top of the usr include sys scsi_buf h file R Reserved for future use and should be 0 VV The Status Validity field In this case it will be 00 No status valid B The residual byte count of the data that is the number of data bytes that did not get transferred D The original byte count of data that was specified to be transferred by the disk driver If this value equals the residual value then none of the data was transferred EE The errno returned by the adapter driver See usr include sys errno h for a definition of errno values These fields are ignored Normally they are set to 0 N The Open Count field This 32 bit field indicates the number of times the device has been opened from the closed state This is useful for determining if all the errors are associated with a specific removable media unit Status Validity of 3 There are two kinds of errors that may be reported with a Status Validity of 3 The difference between these two errors is determined by the value of the Adapter Status Status Validity of 3 with Adapter Status of 0 When a Status V
83. al e The AIX application reports data or other errors associated with a specific Fibre Channel device and that device has been eliminated as the cause of failure The AIX error log has entries associated with this failure whose Probable Causes list includes a Fibre Channel device and that device has been eliminated as the cause of failure e You were sent to this step from anywhere else in the problem determination procedures or there are other reasons to suspect fiber cabling or connector problems Is there any reason to suspect problems associated with fiber jumpers trucks patch panels or any other devices such as hubs in this configuration NO Go to Siep 00707 YES Go to Step 0060 50 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0060 2 Determine that the fiber jumper from the AIX System Fibre Channel Adapter provides a complete signal path to the disk storage subsystem tape drive patch panel or other device such as a hub to which it is connected Using an accurate fiber cabling chart perform the following steps 1 Ensure that the fiber jumper connector is clean and properly plugged into the Fibre Adapter in the AIX system 2 Ensure that the connector at the other end of this fiber jumper is clean and properly plugged into the patch panel Switch SAN Data Gateway hub disk storage subsystem or tape drive as intended 3 Ensure that the fiber jumper is not defective
84. alidity of 3 is accompanied by an Adapter Status of 0 an SC_DISK_ERR1 is being reported This indicates that the block size of the device does not match the block size that the device driver is using This error is detected on the open sequence from the SCSI read capacity data Appendix C Error Messages 91 Sense Data Layout LLOO FFFF XXXX XXXX XXXX XXXX XXXX XXXX cccc FFFF XXXX XXXX XXXX XXXX XXXX XXXX ccce XXXX XXXX XXXX XXXX XXXX XXXX cccc XXXX XXXX XXXX XXXX XXXX XXXX ccce DDDD XXXX XXXX XXXX XXXX XXXX cccc DDDD XXXX XXXX XXXX XXXX XXXX ccce EEEE XXXX XXXX XXXX XXXX NNNN cccc EEEE XXXX XXXX XXXX XXXX NNNN The above fields are interpreted as follows Field LL Cc VV Interpretation ccce XXXX XXXX XXXX XXXX XXXX XXXX The length of the failing SCSI command The failing SCSI command The first two digits of this field indicate the SCSI RRRR XXXX XXXX XXXX XXXX XXXX XXXX RRRR XXXX XXXX XXXX XXXX XXXX XXXX RRRR XXXX XXXX XXXX XXXX XXXX XXXX vvoo XXXX XXXX XXXX XXXX XXXX XXXX 0000 XXXX XXXX XXXX XXXX XXXX XXXX BBBB XXXX XXXX XXXX XXXX XXXX XXXX BBBB XXXX XXXX XXXX XXXX XXXX XXXX opcode A list of SCSI opcodes and their meanings can be found at the top of the usr include sys scsi_buf h file Reserved for future use and should be 0 The Status Validity field In this case it will be 03 driver status
85. and Ultra SCSI With optional Symmetrix software they also support mainframe connectivity using ESCON FICON and Block Multiplexer channels Depending on the model up to 16 Fibre Channel interfaces are available The Symmetrix 5000 family consists of the 5630 5830 and 5930 and is intended for mainframe connectivity through ESCON and Block Multiplxer channels However by adding optional Symmetrix software they can also support open system connectivity through Fibre Channel FWD SCSI and Ultra SCSI With this software and depending on the model up to 16 Fibre Channel interfaces are available The Symmetrix 8000 family consists of the 8830 8530 and 8230 and provides mainframe connectivity through ESCON and FICON paths as well as Open system connectivity through Fibre Channel connnections e The Model 8830 is a three bay model with a capacity of 579 GB to 69 5 TB It offers 96 internal and 384 external Fibre Channel connections 32 ESCON ports and 16 FICON ports e The Model 8530 is a single bay model with a capacity of 144 GB to 17 4 TB It offers 72 internal and 384 external Fibre Channel connections 24 ESCON ports and 12 FICON ports e The Model 8230 has a form factor of only 11 5 inches 29 21 cm wide It has a capacity of 73 GB to 3 5 TB and offers 24 internal and 288 external Fibre Channel connections 8 ESCON ports and 4 FICON ports Fibre Channel with its 100 MB per second bandwidth supports the development of scalable
86. ape Subsystem The Magstar MP 3570 Tape Subsystem is intended for mid sized storage applications and provides high performance for traditional backup and fast access to mass data Its drives can operate as stand alone systems or can be installed in rack mounted libraries The C00 C22 Models of the Magstar MP 3570 Tape Subsystem are supported for attachment to the IBM 2108 G07 SAN Data Gateway through their SCSI II differential interfaces and thus are easily integrated into a Fibre Channel SAN environment Note For Fibre Channel connection through the 2108 G07 SAN Data Gateway the Magstar MP 3570 C00 C22 must be at drive EC C70654 and microcode level D015_024 or higher Models C00 C22 can use C format and C XL format cartridges and are downward compatible for reading and writing B format cartridges Model C00 C22 features and specifications include e High speed search speed 5 meters sec e Cartridge load ready 6 7 sec e Average search to first byte of data 14 sec C XL 11 sec B amp C e Tape speed of 3 5 meters sec e 7 MB sec 15 MB sec maximum compressed transfer rates e Backup performance of 25 GB hour uncompressed to over 50 GB hour with maximum compression e 7 GB up to 21 GB 8 1 compression capacity with C XL cartridge or 7 GB up to 15 GB 3 1 compression with C cartridge e 8 track heads e 5 MB cache memory buffer 143 IBM Magstar MP Multi Purpose 3575 Tape Library Dataserver The Magstar MP M
87. apter for 64 bit PCI Bus problem Chapter 7 Service Information 61 Service Request Numbers If diagnostics fail the error message will contain one of the following Service Request Numbers SRNs Replace FRU parts in the order listed in Table 2 Table 2 Service Request Numbers for Fibre Channel Adapters Service SRN Failing Description and Action Unless otherwise indicated Request Src Function use MAP 0210 See note following the table Number Codes 447 101 D 447 Description The Fibre Channel Adapter configuration failed 447 102 D 447 Description The Reset test failed 447 103 D 447 Description The Register test failed 447 104 D 447 Description The SRAM test failed 447 105 D 447 Description The Internal Wrap test failed 447 106 D 447 Description The Gigabaud Link Module GLM failed 447 107 D 447 Description The External Wrap test failed 447 108 D 221 Description Enhanced Error Handling Failure on bus 447 109 D 221 Description Enhanced Error Handling Failure on EADs chip 447 110 D 447 Description Enhanced Error Handling Failure on adapter 447 201 D 447 Description The Configuration Register test failed 221 447 202 D 447 Description The Interrupt test failed 221 447 203 D 447 Description The PCI Wrap test failed 221 447 204 D 447 Description The DMA test failed 221 447 205 D 447 Description I O error on a read write operation
88. atch ESCON System 360 FICON System 390 IBM Tivoli Magstar Ultrastar Netfinity xSeries pSeries zSeries RS 6000 Linear Tape Open and Ultrium are trademarks of International Business Machines Corporation Hewlett Packard and Seagate Intel is a registered trademark of the Intel Corporation in the United States other countries or both Java and all Java based trademarks and logos are trademarks or registered trademarks of Sun Microsystems Inc in the United States other countries or both Microsoft Windows and Windows NT are trademarks of Microsoft Corporation in the United States other countries or both UNIX is a registered trademark of The Open Group in the United States and other countries Other company product and service names may be trademarks or service marks of others X V IBM Fibre Channel Planning and Integration User s Guide and Service Information Chapter 1 Introduction Fibre Channel FC is an approach introduced during the late 1990 s to solve the many problems associated with existing computer input output I O interfaces and system connections Fibre Channel Overview This section provides an overview of Fibre Channel and discusses computer interfacing techniques which led to the development of the Fibre Channel Computer I O Interface History Prior to 1964 computers attached their I O using a variety of methods each computer type using its own interface design In fact even computer
89. ation procedure 32 storage area network SAN 111 storage server enterprise 131 switch configuration with four host systems 103 configuration with one host system 101 configuration with two host systems 102 configurations 100 T tracing the gigabit fibre channel adapter 64 U Ultrium 3580 tape subsystem 147 V verifying device driver installation 21 hardware installation 21 software installation 21 the installation 21 Index 153 154 IBM Fibre Channel Planning and Integration User s Guide and Service Information Readers Comments We d Like to Hear from You RS 6000 server pSeries Fibre Channel Planning and Integration User s Guide and Service Information Publication No SC23 4329 03 Overall how satisfied are you with the information in this book Very Satisfied Satisfied Neutral Overall satisfaction o How satisfied are you that the information in this book is Very Satisfied Satisfied Neutral Accurate Complete Easy to find Easy to understand Well organized 000 0 bo Applicable to your tasks Please tell us how we can improve this book Thank you for your responses May we contact you Yes No Dissatisfied Dissatisfied Very Dissatisfied Very Dissatisfied
90. b Note If there is no Fibre Channel Storage Hub in the configuration skip this step Go to Step 0060 7 The number of possible indications that a problem may exist in a Fibre Channel Storage Hub include e The AIX application reports data or other errors associated with a Disk Storage Subsystem connected to the Fibre Channel configuration through a Fibre Channel Storage Hub and you have already eliminated the Disk Storage Subsystem and its devices as the point of failure e The AIX error log has entries associated with this adapter or Fibre Channel SCSI I O Controller Protocol Device whose Probable Causes list includes a device connected through a Fibre Channel Storage Hub and the device has already been eliminated as the point of failure e The Fibre Channel Storage Hub s Port Status LEDs indicate an error Are there errors reported by or associated with a Fibre Channel Storage Hub NO Go to Step 0060 1 YES Refer to the Service Manual for the Fibre Channel Storage Hub and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with Chapter 6 Problem Determination 39 Start of PDP PD0020 Fibre Channe
91. ch AIX is already installed proceed directly to the section Installing the Device Driver on an RS 6000 SP System on page 13 e If you are installing the Gigabit Fibre Channel Adapter in a non SP AIX system on which AIX is already installed proceed directly to the section Installing the Device Driver Software on an RS 6000 and pSeries System on page 12 11 Installing the Device Driver Software on an RS 6000 and pSeries System This section explains how to install device driver software on a RS 6000 and pSeries not an RS 6000 SP system unit Note If the adapter was installed prior to the installation of the AIX operating system do not continue with this section The device driver software was automatically installed with AIX Proceed to Verifying the Installation on page 21 If you are installing the Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S you will be installing device driver devices pci df1000f7 If you are installing the 2 Gigabit Fibre Channel Adapter FC 6228 Type 4 W you will be installing device driver devices pci df1000f9 If the adapter is to be installed in a system that already has AIX installed proceed with the software installation and perform the following steps 1 2 3 11 Ensure that the system unit power is on Log in as root user Insert the media containing the device driver software for example CD ROM into the appropriate media device Type the following sm
92. ck to existing cartridges Magstar 3590 Performance and Capacity Magstar 3590 has leading edge streaming and start stop performance Magstar 3590 provides a maximum uncompressed data transfer rate of up to 14 MB sec The maximum instantaneous data rate is 40 MB sec on an Ultra SCSI interface The Magstar models E11 and E1A can be attached at up to 500 meters using their native Fibre Channel ports This can be increased up to 10 kilometers when attached to the IBM 2109 Switch or the IBM 2103 Hub Magstar 3590 cartridges have a capacity of up to 20 GB Up to 60 GB can be stored by using the 3590 compression technique called LZ1 Magstar 3590 metal particle tape media is housed in a cartridge of the same physical size as 3490 cartridges enabling coexistence in a Magstar 3494 Tape Library together with current media The Magstar 3494 Tape Library with Magstar 3590 drives provides access to as much as 374 TB compressed at 3 1 Servicability Reliability and Data Integrity Magstar 3590 does not require scheduled preventive maintenance Customer service representatives use a built in subsystem panel to perform service functions Resident diagnostics monitor operations to detect potential problems and aid in resolution Appendix N Magstar and Ultrium Tape Subsystems 145 Usability The Magstar 3590 employs several techniques to ensure data integrity including an improved Error Correction Code ECC servo tracks written on tape a bidirectiona
93. dable Storage Unit required 1 to 6 Each unit supports up to 10 Ultrastar hard drives redundant power and cooling units are standard 2103 H07 Fibre Channel Storage Hub optional 7 port FC AL hub with short wave or long wave GBICs included 128 IBM Fibre Channel Planning and Integration User s Guide and Service Information Configuration Information Item Description RAID options RAID 0 1 0 1 3 5 Cache options Up to 512 MB read write cache Host interface Two Fibre Channel channels 100 MB sec Drive interface Six Ultra2 SCSI LVD channels 80 MB sec Drives 9 1 18 2 GB 10 000 RPM 9 1 18 2 34 6 GB 7200 RPM Ultra 2 SCSI LVD Capacity 36 4 GB min 2 2 TB 4 4 TB cascaded maximum Publications and Other Sources of Information For further information about the IBM 2102 F10 Fibre Channel RAID Storage Server refer to the following General Information IBM Fibre Channel RAID Storage Server 2102 Introduction and Planning Guide GC26 7281 Planning Installation and Configuration IBM Fibre Channel RAID Storage Server 2102 Installation and User s Guide for AIX Operating System SC26 7290 Operation IBM Fibre Channel RAID Storage Server and Expandable Storage Unit 2102 Model F10 and Model D00 User s Guide GC26 7288 Servicing IBM Fibre Channel RAID Storage Server and Expandable Storage Unit 2102 Model F10 and Model DOO Service Guide SY27 7604 Appendix J IBM 2102 F10 Fibre Channel
94. del S08 Installation and Service Guide SC26 7350 IBM SAN Fibre Channel Switch 2109 Model S16 Installation and Service Guide SG26 7352 2109 Model F16 Installation and Service Guide SY27 7623 For more information on the 2109 S08 S16 and F16 SAN Fibre Channel Switches visit the following Web address http www storage ibm com hardsoft products fcswitch fcswitch htm Appendix G IBM 2109 S08 S16 and F16 SAN Fibre Channel Switches 115 116 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix H McDATA Enterprise Fibre Channel Directors and Switches This appendix describes the McDATA Enterprise Fibre Channel Directors and switches McDATA ED 5000 Enterprise Fibre Channel Director Description The ED 5000 is a 32 port Fibre Channel Director that provides high availability and enterprise class management for computing environments that require one Gigabit bandwidth performance The ED 5000 is completely applicable to AIX system environments and is particularly suited for AIX system Storage Area Networks SANs Functional Features The ED 5000 supports from 4 to 32 F_Ports that are dynamically configurable and concurrently maintainable in groups of 4 F_Ports The ports are designed to accept Fiber Optic cable connectors The ED 5000 has control processors that provide setup and error recovery support message path controllers to manage the routing of frames through the switch and central
95. e nodenamel N will be the actual node names and should be similar to nodenamel Wed Nov 10 10 37 46 EST 1999 nodename2 Wed Nov 10 10 37 46 EST 1999 nodename3 Wed Nov 10 10 37 47 EST 1999 nodenameN Wed Nov 10 10 37 48 EST 1999 If the list does not include all and_only the required system nodes examine the groupt file you created in step 2b on page 14 to ensure that it contains the correct and complete list 3 Determine if the AIX operating system on each relevant node is at the required level by typing dsh oslevel or without the working collective file dsh w nodenamel nodename2 nodename3 oslevel where the actual node names are substituted for nodenamel 2 and 3 Press Enter See Preparing for Installation of the Gigabit Fibre Channel PCI Adapter on page 10 for AIX level requirements for the adapter you are installing If the nodes are not at a supported AIX level upgrade the AIX level on the nodes Contact the system administrator for assistance 4 Determine if the PSSP level on the system is at the required level by typing usr 1pp ssp bin splstdata G b Press Enter The results are in a form similar to the following node hostname hdw_enet_addr srvr response install_disk last_install_image last_install_time next_install_image lppsource_name pssp_ver 1 eion0l ppd pok i 08005A75A6D4 0 disk hdiskO default Fri_Mar_12_10 41 23 default lppsource pssp_level Perform the following steps
96. e Optional long wave ports enable extended distance configurations of up to 10 kilometers between servers and storage systems e Supports scalable FC AL loop expansion with cascading hubs The 2103 H07 Fibre Channel Storage Hub supports data transfer speeds of up to 100 MB per second Configuration Flexibility The Fibre Channel Storage Hub provides flexible connectivity options for configuring multiple Fibre Channel host and storage server attachments This unmanaged 7 port hub provides a single point solution for Fibre Channel Arbitrated Loop FC AL topologies and supports up to 100 MB per second data transmission speeds between servers and storage systems Hot pluggable ports enable the attachment of new systems without requiring server downtime As a result the Fibre Channel Storage Hub helps expand storage networks with minimal impact to daily operations Standard and Extended Distances Server and storage system connections with distances of up to 10 kilometers can be achieved with the Fibre Channel Storage Hub With four standard short wave gigabit interface converter GBIC ports the Fibre Channel Storage Hub provides attachments of up to 500 meters over fiber optic cables to host system adapters or to another Fibre Channel Storage Hub short wave GBIC port Each hot pluggable GBIC port on the Fibre Channel Storage Hub has one FC AL connection port The Fibre Channel Storage Hub also provides the option of adding up to three shor
97. e Server ESS and Modular Storage Server e Fibre Channel attachment of the IBM SAN Data Gateway with UltraSCSI attachment to Magstar 3590 Linear Tape Open LTO Ultrium tape systems and libraries e The McDATA ES 1000 Loop Switch which provides connectivity and concentration of multiple Magstar and LTO FC AL tape devices to a single Fabric Switch port IBM Fibre Channel Planning and Integration User s Guide and Service Information In addition multiple Directors can be cascaded to create a scalable enterprise SAN backbone that supports consolidated storage management applications such as disk sharing tape pooling and enterprise wide data sharing The Director provides several configuration options Each Fibre Port Module provides four Fibre Channel ports Each port delivers up to 100 MB sec full duplex data transfer The McDATA Viper SAN engine serial cross bar technology delivers scalable 64 port nonblocking performance with up to 6 400 MB sec bandwidth Unique McDATA single stage switching architecture provides consistently low latency across all ports The Full ED 6064 Product Manager simplifies connectivity management Full throughput at extended distances of up to 100 kilometers 60 miles is enabled with long wave optics 60 buffer credits for each port and repeaters Other configuration options include short wave Fibre Port Modules transmission distance up to 500 meters long wave Fibre Port Modules transmission distance up to 20
98. e channel devices 10 preparing for installation of the gigabit fibre channel PCI adapter 10 prerequisites check system 14 problem determination 29 problem determination procedures 31 problem determination start of call 32 publications and other sources of information EMC symmetrix enterprise storage systems 142 enterprise storage server 135 fibre channel RAID storage server 129 fibre channel storage hub 109 Magstar MP 3570 3590 tape subsystems 148 Magstar MP 3570 3590 tape subsystems 112 SAN fibre channel switch 115 publications online xii R RAID storage server 127 reliability availability and serviceability RAS features 117 S safety electrical ix safety notices ix SAN data gateway configurations 105 description 111 feature codes 2203 112 2204 112 2205 112 Storage Area Network Data Gateway 112 SAN storage area network 111 service 2109 SAN fibre channel switch service aid 67 AIX diagnostics for the gigabit fibre channel adapters 61 failing function codes 64 generating an error report for the gigabit fibre channel adapter 65 gigabit fibre channel adapter 59 information 59 service continued service request numbers 62 source of SRN 63 tracing the gigabit fibre channel adapter 64 service request numbers 62 software AIX level 10 check system prerequisites 14 device drivers 10 preparing for installation 10 PSSP level 10 software on the Control Workstation 16 source of SRN 63 start of call problem determin
99. e facilities to protect data in distributed enterprises Appendix N Magstar and Ultrium Tape Subsystems 147 General Information This section provides information pertaining to the IBM 3580 Ultrium Tape Drives Table 12 General Information for the 3580 Ultrium Tape Subsystem Model Number L11 H11 Tape drive type Ultrium Ultrium Capacity per cartridge Up to 200 GB per cartridge 100 GB native compressed Sustained data transfer rate Up to 30 MB sec compressed 15 MB sec native Aggregate sustained data rate Up to 108 GB hour compressed Up to 108 GB hour compressed Media type IBM Ultrium IBM Ultrium Data Cartridge P N 08L9120 P N 08L9120 Cleaning Cartridge P N 08L9124 P N 08L9124 Dimensions 5 75in H x 6 74in W x 13 11in D 146mm x 170mm x 333mm Weight 14 3 Ib 6 6 kg Note Based on 2 1 compression Publications and Other Sources of Information For further information about the Magstar and Ultrium Tape Subsystems refer to the following General Information Planning and Installation Configuration Operation IBM Magstar 3590 Tape Subsystem Introduction and Planning Guide GA32 0329 IBM Magstar 3590 Tape Subsystem Technical Guide SG24 2506 IBM Magstar MP 3570 Tape Subsystem Introduction and Planning Guide C Series Models GA32 0392 IBM Magstar MP 3575 Tape Library Dataserver Introduction and Planning Guide GA32 0380 IBM
100. e fact that the various hosts and devices may be physically separated by considerable distance and located in different rooms on different floors and even in different buildings Nature of Fibre Channel Environment Problems In the complex and diverse Fibre Channel environment a wide variety of problems can be encountered These problems may include but are by no means limited to A Gigabit Fibre Channel PCI Adapter in an AIX system has a hardware defect A Gigabit Fibre Channel PCI Adapter has been incorrectly configured The device driver for a Gigabit Fibre Channel PCI Adapter has been incorrectly installed or is exhibiting incorrect behavior A Fibre Channel SCSI I O Controller Protocol Device is not properly configured A logical hard disk in the AIX system is not properly configured A port adapter in a Fibre Channel switch has a hardware defect A port in a Fibre Channel switch is incorrectly zoned or blocked Ports in a Fibre Channel switch have been soft rezoned and the cfgmgr command has not been run to set up the new configuration parameters Host to switch cabling has been changed or swapped and the cfgmgr AIX command has not been run to update the configuration attributes In this case results of commands such as Isattr EIl will not yield the correct information for attributes such as the scsi_id field A port adapter in a Fibre Channel hub has a hardware defect A Fibre Channel port adapter in a SAN Data Gateway has a
101. e following possible values defined in the scsi_buf Adapter Status structure in the usr include sys scsi_buf h file Value Description 01 Host IO Bus Error This indicates a hardware problem with either the SCSI adapter or the Micro Channel PCI bus 02 Transport Fault This indicates a hardware problem that is probably due to problems with the SCSI transport layer connection cables 03 Command Timeout This indicates that the SCSI command did not complete within the allowed time This usually indicates a hardware problem related to the SCSI transport layer 04 No Device Response This indicates that the SCSI device did not reply to the SCSI command issued This is a hardware problem that can be caused by items such as a bad SCSI device a SCSI device power supply problem or SCSI cabling or termination problems 07 Worldwide Name Change This indicates that the SCSI device at this ID has a different worldwide name Note If any of these conditions are true and if the PP field equals 5D and the QQ field equals 00 then the drive is about to fail and should be replaced as soon as possible R Reserved for future use and should be 0 SCSI Request Sense data These fields will only be valid when the SCSI Status field SS has a value of 02 KK SCSI Request Sense Key Some common values are Value Description 01 Indicates that the device was able to do the IO after completion of error recovery 02 Indicates the device
102. e logical hard disks available NO Go to Step 0040 1 YES Go to Step 0010 4 Chapter 6 Problem Determination 33 Step 0010 4 Determine if the logical tape drives associated with the Fibre Channel adapter are available in the AIX system Note If there are no tape drives in the configuration skip this step Run the following AIX command Isdev Cc tape pg The result should be similar to the following assuming the adapter under test is adapter zero in slot 20 70 rmt1 is one of the tape drives that has been defined and the tape drive is a 3590 rmt1 Available 20 70 01 3590 Tape Drive There should be an entry in the above output for each tape drive defined for this adapter Note See the AIX System Administrator for this installation for assistance in identifying which tape drives have been defined for this adapter Are the appropriate logical tape drives available NO Go to Step 0050 1 YES Go to Step 0010 5 34 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0010 5 Determine if there are errors reported by or associated with a disk storage subsystem The number of possible indications that a problem may exist in a Disk Storage subsystem include e The AIX application reports data or other errors associated with a specific disk device e The Disk Storage Subsystem displays error LEDs for one or more disk devices associated with this adapter e The Disk Storage Subsyste
103. el environment These procedures are intended to help isolate the problem and provide information needed to resolve it It should be noted that because of the complexity of the environment a single Fibre Channel problem can result in a large volume of error reports in the AIX system In such a case it is necessary to carefully analyze these logged errors to find the one which represents the original root cause In addition while Fibre Channel environment problems are often reported by the AIX system indiscriminate replacement of the Gigabit Fibre Channel PCI Adapter is not the recommended problem determination procedure Requirements Before Starting Problem Determination A knowledgeable person is needed to perform the problem determination procedures Someone with skills in Fibre Channel basics AIX operations and RS 6000 hardware Storage Area Network SAN basics Disk Storage Subsystems Tape Subsystems and Fibre Channel Switch basics is required In addition for some configurations a knowledge of SCSI interfaces and the SAN Data Gateway is required Also the following skills and information will likely be required and must be available as required An AIX system administrator An AIX system operator with root user authority A chart showing the Fibre Channel cabling scheme including location of patch panels and trunk cables A list of hardware microcode and device driver levels for the Gigabit Fibre Channel PCI
104. ement feature is enabled S 390 operating mode is the only allowed operating mode Appendix H McDATA Enterprise Fibre Channel Directors and Switches 121 Effect of Operating and Management Modes How these operating and management modes are configured has a direct bearing on fabric behavior in certain key areas The following table describes the fabric behavior associated with each mode Table 7 Mode Effect on Fabric Behavior Operating Modes Open System Open System 390 Open Fabric McDATA Fabric Open System Connectivity Yes Yes Yes FICON Connectivity Yes Yes Yes Hard Zoning No No Yes Soft Zoning by Port No Yes Yes Soft Zoning by WWN Yes Yes Yes Manangement Modes EFC and Open EFC and Open EFC and FICON Systems Systems Publications and Other Sources of Information For further information about the McDATA Fibre Channel Directors and switches refer to the following CD ROMs from McDATA Corporation ED 5000 Director Technical Publications CD ROM P N 622 000001 300 ED 6064 Director Technical Publications CD ROM P N 622 000004 100 ES 3016 Switch Technical Publications CD ROM P N 622 000005 100 ES 3032 Switch Technical Publications CD ROM P N 622 000006 000 ES 1000 Switch Technical Publications CD ROM P N 622 000003 100 Or visit the following Web address http www storage ibm com ibmsan products sanfabric htm 122 IBM Fibre Channel Planning and Integration User s Guide and Ser
105. eorient or relocate the receiving antenna e Increase the separation between the equipment and receiver e Connect the equipment into an outlet on a circuit different from that to which the receiver is connected e Consult an authorized dealer or service representative for help Properly shielded and grounded cables and connectors must be used in order to meet FCC emission limits Proper cables and connectors are available from authorized dealers Neither the provider nor the manufacturer are responsible for any radio or television interference caused by using other than recommended cables and connectors or by unauthorized changes or modifications to this equipment Unauthorized changes or modifications could void the user s authority to operate the equipment This device complies with Part 15 of the FCC Rules Operation is subject to the following two conditions 1 this device may not cause harmful interference and 2 this device must accept any interference received including interference that may cause undesired operation 69 Responsible Party e International Business Machines Corporation e New Orchard Road e Armonk New York 10504 e Telephone 919 543 2193 Tested to Comply lt With FCC Standards FOR HOME OR OFFICE USE European Union EU Statement This product is in conformity with the protection requirements of EU Council Directive 89 336 EEC on the approximation of the laws of the Member States relating to elect
106. esource is highlighted press Enter to select this option The Network Install Operation to Perform panel displays with the reset option highlighted for selection d Move the cursor until the cust option is highlighted Press Enter to select this option The Customize a SPOT panel displays with the Source of Install Images highlighted for selection e Press F4 to display the list of installation images and select the appropriate LPP source directory Move the cursor until the appropriate LPP source directory is displayed For example lppsource_AIX433 resources 1ppsource where lppsource is the name of the directory recorded in step 4b on page 16 Press Enter f Move the cursor to fileset names and for adapter Type 4 S enter devices pci df1l000f7 Or for adapter Type 4 W enter devices pci df1000f9 Press Enter to start the SPOT update This operation can take up to 15 minutes Repeat step 14 for each device you want to install for example devices common ibm fc devices fcp disk devices fcp disk array devices fcp tape 2 Determine if the software is already installed on the node s Type the following commands one line at a time including the quotation marks at the AIX command prompt Press Enter after each line For adapter Type 4 S dsh Islpp 1 devices pci df1000f7 2 gt amp 1 more Or for adapter Type 4 W dsh Islpp 1 devices pci df1l000f9 2 gt amp 1 more And continue for other device
107. following using the same assumptions as in Step 0010 1 above fscsid Available 21 08 01 FC SCSI I 0 Controller Protocol Device fscsil Available 14 08 01 FC SCSI I 0 Controller Protocol Device fscsi2 Available 2A 08 01 FC SCSI I 0 Controller Protocol Device Note The instance number of the FC SCSI I O Controller Protocol Device does not necessarily match its corresponding FC Adapter instance number That is fscsi0 may or may not correspond to fcsO Either condition is valid Is the SCSI I O Controller Protocol Device available NO Go to Step 0030 1 YES Go to Step 0010 3 32 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0010 3 Determine if the logical hard disks hdisks associated with the Fibre Channel adapter are available in the AIX system Note If there are no disk devices in the configuration skip this step Run the following AIX command Isdev C grep hdisk pg The result should be similar to the following assuming the adapter under test is adapter zero in slot 20 70 hdisk4 is one of the hard disks that has been configured and the Disk Subsystem is an EMC Symmetrix Storage System hdisk4 Available 20 70 01 EMC Symmetrix FCP Disk There should be an entry in the above output for each hdisk defined for this adapter Note See the AIX system administrator for this installation for assistance in identifying which hdisks have been assigned to this adapter Are the appropriat
108. g controller The controller module provides management functions for the FAStT500 It manages interfaces to host system devices as well as the service interface for example a remote support modem The controller cache acts as intermediate storage for data being transferred between the host servers and the disk drive module array The controller cache includes battery backup to prevent data loss during power failures Aiding in the FAStT500 s design for high availability and high performance is the FAStT EXP500 Storage Expansion Unit Each 3U EXP500 storage enclosure supports up to ten slim high or half high Fibre Channel hot swappable hard disk drives The EXP500 storage enclosure transfers data to and from the controller enclosure through four short wave GBICs in either a single or dual loop configuration The 3552 Fibre Array Storage Technology Storage Server offers the following Broad range of RAID options RAID 0 1 3 5 and 0 1 also known as RAID 10 e 256 MB battery back up write back cache with an optional maximum of 512 MB e High performance through 200MB sec bandwidth with two Fibre Channel ports e Flexibility to configure distances up to 10 km e Two 2 port mini hubs on both drive and host side with up to four mini hubs per side available e Scalability to accommodate growth with up to 16 TB of storage e High availability through automatic I O path failover Concurrent maintenance with hot pluggable components 137
109. guration Configuration Options i Applying Changes to Database gt Only Information Fields Chapter 5 Fibre Channel Operation Operating Other Fibre Channel Devices Chapter 6 Problem Determination Complexity of the Fibre Channel Environment Nature of Fibre Channel Environment Problems Fibre Channel Environment Problem Determination Procedures Requirements Before Starting Problem Determination Start of PDP PD0010 Start of Call Step 0010 1 Step 0010 2 Step 0010 3 Step 0010 4 Step 0010 5 Step 0010 6 Step 0010 7 Step 0010 8 Step 0010 9 Start of PDP PD0020 Fibre Channel Adapter not Available Step 0020 1 Loe ee E e Sg a gt 4 Step 0020 2 Step 0020 3 Step 0020 4 Step 0020 5 Start of PDP PD0030 Fibre Channel SCSI Vo Controller Protocol Device not Available Step 0030 1 Start of PDP PD0040 Logical Hard Disks Not Available Step 0040 1 sw ke le 8 Step 0040 2 Step 0040 3 Start of PDP PD0050 Logical Tape Drives Not Available Step 0050 1 gt oer X Step 0050 2 Step 0050 3 Start of PDP PD0060 Fiber Path Failures Step 0060 1 Step 0060 2 Step 0060 3 Step 0060 4 Step 0060 5 Step 0060 6 Step 0060 7 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0060 8 Start of PDP PD0070 Other Failures Step 0070 1 Chapter 7 Service Information Gigabit Fibre Channel PCI Adapters Adapter LEDs A
110. hardware defect A SCSI port adapter in a SAN Data Gateway has a hardware defect A port adapter in a Disk Storage Subsystem has a hardware defect A disk drive in a Disk Storage Subsystem has a hardware defect A Fibre Channel jumper cable is defective A Fibre Channel cable connector is not properly seated or is dirty A Fibre Channel trunk has a defective fiber A patch panel connection is defective or incorrectly plugged A host or device has defective logic memory or control circuitry or a defective power or cooling system Optical components somewhere in the environment are defective and are causing intermittent failures As can be seen in the above list problems can be encountered anywhere throughout the Fibre Channel configuration Sometimes the problem is distinctly reported by and at the failing component Often however the AIX system host as the initiator detects and 30 IBM Fibre Channel Planning and Integration User s Guide and Service Information reports the error condition As a result Fibre Channel errors reported by the AIX system must be analyzed carefully to determine the true origin of the failure Note You must not pursue problem determination by Field Replaceable Unit FRU replacement in the AIX system unless the problem is actually isolated to this host component Fibre Channel Environment Problem Determination Procedures This section provides basic problem determination procedures for the Fibre Chann
111. he fault After the problem is resolved you might need to run the cfgmgr AIX command to restore the Fibre Channel environment to a functional state Done YES Go to Step 0060 5 Chapter 6 Problem Determination 53 Step 0060 5 Determine that the fiber jumper and SCSI interface cables plugged into the SAN Data Gateway in this configuration provide a complete signal path through the gateway Note If this path does not include a SAN Data Gateway skip this step Using an accurate fiber cabling chart perform the following steps 1 Ensure that the fiber jumper connector is clean and properly plugged into the correct Fibre Channel port Ensure that the SCSI interface is correctly cabled Ensure that the fiber jumper is not defective Ensure that the SCSI interface cables are not defective Ensure that the SAN Data Gateway does not indicate a port or interface failure a e wN Do the fiber jumper and SCSI interface cables plugged into the SAN Data Gateway in this configuration provide a complete signal path through the gateway NO Correct the fault After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done YES Go to Step 0060 6 54 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0060 6 Determine that the fiber jumpers plugged into the hub in this configuration provide a complete signal path through t
112. he hub Note If this path does not include a hub skip this step Using an accurate fiber cabling chart perform the following steps 1 Ensure that the inbound fiber jumper connector is clean and properly plugged into the correct hub port 2 Ensure that the outbound fiber jumper connector is clean and properly plugged into the correct hub port 3 Ensure that the both inbound and outbound fiber jumpers are not defective 4 Ensure that all other fiber jumpers plugged into ports on this hub have good connections and are not defective 5 Ensure that all open unplugged ports are correctly bypassing the signal Do the fiber jumpers plugged into the hub in this configuration provide a complete signal path through the Hub NO Correct the fault After the problem is resolved you may need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done YES Go to Step 0060 7 Chapter 6 Problem Determination 55 Step 0060 7 Step 0060 8 Determine that the fiber jumpers plugged into any other Fibre Channel device in this configuration provide a complete signal path through the device Note e If this path has no other devices prior to the disk storage subsystem or tape drive skip this step e If this configuration contains more than one device not covered in previous steps use the following procedure to check all of them regardless of whether they exist in the configuration Using a
113. hile functioning ON Slow blink 1HZ Normal inactive ON Flashing irregularly Normal active ON Fast blink 4HZ Normal busy Slow blink OFF Normal link down or not yet started Slow blink Slow blink 1HZ Offline for download Slow blink Fast blink 4HZ Restricted offline mode waiting for restart AIX Diagnostics for the Gigabit Fibre Channel Adapters The Gigabit Fibre Channel Diagnostics are contained on the AIX Diagnostic CD ROM These diagnostics can be run in either standalone mode or under AIX run time diagnostics Attention Halt all applications using the Gigabit Fibre Channel adapter before running diagnostics against that adapter Two levels of diagnostics exist namely standard and advanced Standard diagnostics can be run while the Gigabit Fibre Channel Adapter is cabled into the Fibre channel configuration Advanced diagnostics require the wrap plug More information is available in the Diagnostic Information for Multiple Bus Systems order number SA23 2769 Use Version 4 3 or later If the Gigabit Fibre Channel Adapter diagnostics detect a problem an error code displays either on the screen or on the three digit display on the front of the system unit Table 2 lists possible error codes displayed while running diagnostics on the Gigabit Fibre Channel Adapters The 677 function code indicates a Gigabit Fibre Channel PCI Adapter problem while the 447 function code indicates a 2 Gigabit Fibre Channel Ad
114. hrough its Enterprise Fabric Connectivity Management Terminal The Fibre Channel Switch indicates power or cooling faults The AIX error log has entries associated with this adapter or Fibre Channel SCSI I O Controller Protocol Device whose Probable Causes list includes the Fibre Channel Switch Soft rezoning has not yielded the expected results Run the cfgmgr AIX command to set up the new configuration parameters This will likely resolve the problem Configuration attributes do not reflect the current configuration For example the Isattr El command yields an incorrect scsid_id field Run the AIX cfgmgr command to update configuration attributes This will likely resolve the problem Are there errors reported by or associated with a Fibre Channel Switch NO Go to Step 0010 4 YES Refer to the Service Manual for the Fibre Channel Switch and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the cfgmgr AIX command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with the next step Chapter 6 Problem Determination 37 Step 0010 8 Determine if there are errors reported by or associated with a SAN Data Gateway Note If there is no SAN Data Gateway in the configuration skip this s
115. i df1000f7 com 4 3 3 0 COMMITTED Common PCI FC Adapter Device devices pci df1l000f7 diag 4 3 3 0 COMMITTED PCI FC Adapter Device devices pci dfl000f7 rte 4 3 3 0 COMMITTED PCI FC Adapter Device Software devices pci df1000f7 com 4 3 3 0 COMMITTED Common PCI FC Adapter Device Notes 1 The above data is for the Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S If you are troubleshooting the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus FC 6228 Type 4 W the data displayed will show df 000f9 instead of df1000f7 2 If no data displays on the screen or if some of the above components are missing the device drivers are not properly installed Are the device drivers properly installed NO Refer to Chapter 3 Fibre Channel Installation on page 11 and reinstall the device drivers Done YES Go toSiep 00203 Chapter 6 Problem Determination 41 Step 0020 5 Refer to Chapter 7 Service Information on page 59 and run diagnostics on the Fibre Channel adapter Did the diagnostics fail NO Go to Step 0070 1 YES Follow the correct procedure to have the Gigabit Fibre Channel Adapter replaced Done 42 IBM Fibre Channel Planning and Integration User s Guide and Service Information Start of PDP PD0030 Fibre Channel SCSI I O Controller Protocol Device not Available Step 0030 1 Determine if the device drivers for the Fibre Channel adapter are properly installed on the AIX system Run the following AIX comma
116. ications and Other Sources of Information ge Age x 4 112 General Information Planning and Installation Configuration and Operation 112 EMVICING 2 ya 8 ae Bw Roe m aLa Re o w w a o So eed Appendix G IBM 2109 S08 S16 and F16 SAN Fibre Channel Switches 113 IBM 2109 S08 S16 and F16 SAN Fibre Channel Switch E e s oa ia a S High Performance Scalable Switch Fabric Mae a e o oy S Intelligent Self Management a a aaa aaa a 118 Multiple Configuration Options body Hh oh A ae Gh TS IBM StorWatch SAN Fibre Channel Switch h Specials sb eRe ee a TA General Information of a 8 2 of ae oe So Product Information 4 44 2 amp a a 1 Feature Codes for SAN Fibre Channel Switch Products Lok oh Ayo bh a e MA Publications and Other Sources of Information 115 Related General Information and Operation 115 Planning Installation Configuration and Servicing 115 Appendix H McDATA Enterprise Fibre Channel Directors and Switches 117 McDATA ED 5000 Enterprise Fibre Channel Director Description 117 Functional Features te de ee OR oe T Reliability Availability and Serviceability RAS Features Ron ye YS a woe oe IE McDATA ED 6064 Enterprise Fibre Channel Director Description 118 Connectivity and Configuration Features eR Boe ew eo Gt 4 IS McDATA ES 3016 and ES 3032 Fabric Switch Description Auk ko ome ee amp a
117. ide SC26 7351 IBM Fibre Channel Planning and Integration User s Guide and Service Information IBM SAN Fibre Channel Switch 2109 Model S08 Installation and Service Guide SC26 7350 IBM SAN Fibre Channel Switch 2109 Model S16 Installation and Service Guide SG26 7352 IBM Fibre Channel Storage Hub 2103 Model H07 Installation Service and User s Guide SC26 7289 IBM SAN Data Gateway Installation and User s Guide SC26 7304 IBM SAN Data Gateway Service Guide SC26 7307 IBM Fibre Channel RAID Storage Server 2102 Introduction and Planning Guide GC26 7281 IBM Fibre Channel RAID Storage Server 2102 Installation and User s Guide for AIX Operating System SC26 7290 IBM Fibre Channel RAID Storage Server and Expandable Storage Unit 2102 Model F10 and Model D00 User s Guide GC26 7288 IBM Fibre Channel RAID Storage Server and Expandable Storage Unit 2102 Model F10 and Model D00 Service Guide SY27 7604 Configuring and Implementing the IBM Fibre Channel RAID Storage Server C24 5414 Symmetrix Open System Environment Product Guide Volume I P N 200 999 563 EMC Corporation IBM ESS Introduction and Planning Guide 2105 Models E10 and E20 GC26 7294 IBM ESS Host System Attachment Guide 2105 Models E10 and E20 SC26 7296 IBM ESS Configuration Guide 2105 Models E10 and E20 SC26 7353 IBM ESS User s Guide 2105 Models E10 and E20 SC26 7295 IBM ESS SCSI Command Reference 2105 Models E10 and E20 SC26 7297 IBM Magstar 3590
118. iguration provide a complete end to end signal path NO Correct the fault After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done YES Go to Step 0060 4 52 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0060 4 Determine that the fiber jumpers plugged into the Fibre Channel Switch in this configuration provide a complete signal path through the switch Note If this path does not include a Fibre Channel Switch skip this step Using an accurate fiber cabling chart perform the following steps 1 Ensure that the connectors in both the inbound and outbound ports are clean and properly plugged into the correct ports 2 Ensure that both the inbound and outbound fiber jumpers are not defective 3 Ensure that the Fibre Channel Switch is properly configured and does not indicate any port failures 4 Determine if host to switch cabling has been changed or swapped recently If so run the AIX cfgmgr command to update the configuration attributes If these attributes are not updated results of commands such as the Isattr EI will not yield the correct information for attributes such as the sesi_id field Running the AIX cfgmgr command will likely resolve the problem Do the fiber jumpers plugged into the Fibre Channel Switch in this configuration provide a complete signal path through the switch NO Correct t
119. imes a Primitive Sequence Event Time out has occurred K Number of times an Elastic Buffer overrun has occurred P Number of times a time out has occurred during FC AL arbitration R Reserved S Adapter driver s current init_link_flags setting 6 is old port mode 0 is arbitrated loop mode U Start sequence status V Adapter driver state W Abort IOCB retries without success X Number of times the adapter driver has tried to get the link up without success Y R_A_TOV Resource Allocation time out Delay Value currently used by the adapter driver This value is in milliseconds Z E_D_TOV FC Error Detect Time out Value currently used by the adapter driver This value is in milliseconds 2 ARB_TOV FC AL Arbitration Time out Value currently used by the adapter driver This value is in milliseconds 3 R_T_TOV FC Receiver Transmitter Time out Value currently used by the adapter driver This value is in milliseconds 4 AL_TOV FC AL Time out value currently used by the adapter driver This value is in milliseconds Q The contents of this field depends on the type field as follows e If the type field equals 1 Mailbox included this field will contain the failing mailbox If the type field equals 2 IOCB included this field will contain the failing IOCB e If the type field equals 3 Adapter register contents included the first four bytes will be the adapter s status registers followed by four bytes displaying the PCI c
120. in order to be attached to McDATA Fabric Switches and Enterprise Fibre Channel Directors The McDATA ES 1000 provides this conversion The ES 1000 includes nine short wave laser GBICs for device attachment up to 500 meters from the Switch An optional long wave laser GBIC may be ordered to enable remote attachment of the Switch up to 10 kilometers from a McDATA Switch or Director Up to eight devices can be attached to the Loop Switch H_Ports Hub Ports Storage device data transfers are concentrated onto a single B_Port Bridge Port which is capable of up to 100 MB per second bandwidth The number of devices that can effectively use this attachment is dependent upon specific application characteristics The ES 1000 provides hot swappable load sharing dual power supplies that allow the switch to remain online if one supply fails Dual power cords allow attachment to separate power sources for improved availability Hot swappable power and cooling components eliminate downtime for service when replacing a failed component and eliminate the risk of erroneously cabling a replacement switch due to a simple component failure Special tools are not required to replace power supplies and fans Firmware upgrades can be downloaded to the unit while the loop switch remains operational The ES 1000 distributed switch intelligence provides the fabric services required for attachment of loop devices to a switched fabric IBM Magstar and LTO tape devices automa
121. ing to prevent multiple HBAs in the same RS 6000 and pSeries system from accessing the same LUNs in a disk storage subsystem Refer to the publications for your switch for information on how to configure zones Note Whenever a switch configuration is changed by soft rezoning ports or by host to switch cabling changes the AIX cfgmgr command must be run to set up the new configuration parameters To create a much larger and complex configuration switches can also be interconnected or cascaded Switch Configuration with One Host System The following illustration shows an example of a single host system connected through a switch to a Fibre Channel disk array The host system can have one or more HBAs each connected to the switch Each Fibre Channel device can have from one to four ports also connected to the switch HBA HBA Switch I FCDisk Array s HBA HBA Figure 12 Example of a Switch Configuration With One Host System Attached Appendix D Fibre Channel Devices and Configurations 101 Switch Configuration with Two Host Systems The following illustration Figure 13 shows an example of two host systems connected through a switch to a Fibre Channel disk array HBA HBA Switch FC Disk Array s HBA HBA HBA HBA HBA HBA Figure 13 Example of a Switch Configuration With Two Host Sys
122. irector is scalable with non disruptive upgrades from 48 to 128 ports An Interconnection kit may be used to field upgrade two FC 9000 64 Directors within the same cabinet to the FC 9000 128 Director Field installation of the Interconnection kit requires a brief period of scheduled downtime FC 9000 Director Management The IN VSN Enterprise Manager management system centralizes the management of multiple distributed Directors in an enterprise wide Fibre Channel fabric backbone and continuously monitors their operations Operators can use the IN VSN Enterprise Manager locally or remotely from LAN attached IN VSN Enterprise Manager Clients to manage the Directors One or more Directors can be controlled from the same control node if they are connected through Fibre Channel to the Director that is in turn connected to the IN VSN Management System through the Ethernet The IN VSN Enterprise Manager application runs on the Manager Server The IN VSN Enterprise Manager Server a desktop PC includes a 17 inch monitor internal and external modems for call home and remote diagnostics and an 8 port Ethernet hub with cables for private LAN connections to multiple Director Control Modules The IN VSN Enterprise Manager Client is a Microsoft Windows NT or 2000 based GUI that provides a consistent management interface The Enterprise Manager Client can run locally on the Manager Server or remotely on a LAN attached IN VSN Manager Client and provides a
123. is displayed or some information is missing refer to Chapter 6 Problem Determination on page 29 to determine the cause of the problem Installing Other Fibre Channel Devices 22 To install each of the Fibre Channel devices in the configuration refer to the appropriate appendix for each device Each device s appendix contains a section called Publications and other Sources of Information This section contains a list of publications and Web sites that provide device specific instructions and information needed for installing configuring operating and servicing of that device IBM Fibre Channel Planning and Integration User s Guide and Service Information Chapter 4 Using SMIT to Configure the Fibre Channel Adapter This chapter provides instructions for using SMIT to configure options for the Gigabit Fibre Channel adapter Gigabit Fibre Channel Adapter Configuration Use SMIT to configure options as follows 1 Log in as root user 2 To start SMIT type smit Press Enter 3 Select Devices 4 Select the FC Adapier section FC Adapier Configurable Options display on the screen 5 Select Change Show Characteristics A list of configurable FC Adapter options displays as follows a gt Change Show Characteristics of a FC Adapter Type or select values in entry fields Press Enter AFTER making all desired changes Entry Fields FC Adapter fcs0 Description FC Adapter Status Available Loca
124. itty devinst Press Enter The Install Additional Device Software menu is displayed and the INPUT device directory for software option is highlighted Select or type your input device e Press F4 for a list of devices and select the correct device Press Enter OR e Type the name of the input device you are using in the entry field and press Enter for example CD ROM The Install Additional Device Software menu is expanded and the SOFTWARE to install option is highlighted Press F4 to display the SOFTWARE to install menu Type the following to display the Find window For adapter Type 4 S type the following devices pci df1000f7 Or for adapter Type 4 W type the following devices pci df1000f9 Press Enter The system will find and highlight the device driver software for example for adapter Type 4 S devices pci df1000f7 ALL 12 IBM Fibre Channel Planning and Integration User s Guide and Service Information Or for adapter Type 4 W devices pci df1000f9 ALL 12 Press F7 to select the highlighted device driver software 13 Repeat steps 10 through 12 for devices common IBM fc 14 Repeat steps 10 through 12 for each of any other device drivers you need to install Examples of additional device drivers for other devices you may be installing include e devices fcp disk e devices fcp tape e devices fcp disk array 15 Press Enter The Install Additional Device Software menu displays again The entry data fields a
125. l longitudinal serpentine recording technique and a fourth generation magnetoresistive head that reads and writes 16 data tracks at a time The Magstar 3590 Tape Subsystem incorporates a standard 10 slot Automatic Tape Loader ATL for high capacity stand alone unattended operation The ATL can be used in random access mode as a mini library Cartridges are loaded into and unloaded from the ATL in a convenient portable 10 cartridge magazine The Magstar 3590 Tape Subsystem is supported by the Tivoli Storage Manager client server storage management product The Tivoli Storage Manager TSM uses the full capacity of the Magstar 3590 and supports its ATL in random access mode Magstar 3590 also features an operator service display showing device status activities error conditions and messages General Information This section contains information pertaining to the 3590 Magstar Tape Subsystems Table 11 General Information for the 3590 Magstar Tape Subsystems Model Number B11 BIA C12 E11 E1A LZ1 Compression Standard Standard Standard Standard Standard Recording technique Standard Standard Standard Standard Standard interleaved serpentine longitudinal Number of Tracks 128 128 256 128 or 256 256 Cartridge capacity with 10GB 30GB 10GB 30GB Up to 60GB 20GB 60GB 20GB 60GB compression 10 cartridge magazine 100GB N A N A 200GB N A with compression 300GB 600GB
126. l Adapter not Available Step 0020 1 Step 0020 2 Determine if the Fibre Channel adapter is defined recognized in the AIX system Run the following AIX command Isdev C grep fcs Note If the Gigabit Fibre Channel PCI Adapter is a vendor solution adapter the Isdev command might not recognize the adapter The result should be similar to the following assuming the adapter under test is adapter zero and in slot 20 70 fcs0 Defined 20 70 FC Adapter Is the adapter defined NO Go to Step 0020 2 YES Go to Step 0020 3 Verify that the Fibre Channel adapter is physically installed and properly seated in the AIX system Is the Fibre Channel adapter installed in the AIX system NO Follow the proper procedures for your system to have the Gigabit Fibre Channel PCI Adapter installed Done YES Go to Step 005055 40 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0020 3 Step 0020 4 Attempt to configure the Channel adapter by running Configuration Manager Run the following AIX command cfgmgr After the cfgmgr command has completed run the following AIX command lsdev C grep fcs Is the adapter defined or available now NO Go to Siep 0020 4 YES Done Determine if the device drivers for the Fibre Channel adapter are properly installed on the AIX system Run the following AIX command lslpp 1 grep df1000f The result should be similar to the following devices pc
127. m displays error LEDs for one or more Fibre Channel ports connected in the Fibre Channel path for this adapter e The Disk Storage Subsystem displays error LEDs that indicate power or cooling faults e The AIX error log has entries associated with this adapter or Fibre Channel SCSI I O Controller Protocol Device whose Probable Causes list includes the Device e Configuration attributes do not reflect the current configuration For example the Isattr El command yields an incorrect scsid_id field Run the cfgmgr AIX command to update configuration attributes This will likely resolve the problem Are there errors reported by or associated with a disk storage subsystem NO Go to Step 0010 6 YES Refer to the Service Manual for the disk storage subsystem and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the cfgmgr AIX command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with the next step Chapter 6 Problem Determination 35 Step 0010 6 Determine if there are errors reported by or associated with a tape subsystem The number of possible indications that a problem may exist in a tape subsystem include e The AIX application reports data or other errors associated with a s
128. mputer I O Interface History Computer I O Interface Strategies What is Fibre Channel Fibre Channel Connection Topology Fibre Channel Port Types Fibre Channel Classes of Service Introduction to the Gigabit Fibre Channel Adapters Chapter 2 Planning Information Planning Your Fibre Channel Configuration Prepare a Site Plan Plan and Diagram All Cable Routing Preparing for Installation of the Gigabit Fibre Channel PCI Adapter Preparing for Installation of the Other Fibre Channel Devices Chapter 3 Fibre Channel Installation Installing the Gigabit Fibre Channel Adapter Installing the Device Driver Software on an RS 6000 and pSeries System Installing the Device Driver on an RS 6000 SP System Setting Up and Checking System Prerequisites Creating an Installable Image on the Control Workstation Finalizing Preparations for Device Driver Installation Installing Device Drivers on the RS 6000 SP System Nodes Installing the Adapter Hardware ee oe a we OE Verifying the Installation Verifying the Hardware Installation Verifying the Software Installation Installing Other Fibre Channel Devices Chapter 4 Using SMIT to Configure the Fibre Channel prone Gigabit Fibre Channel Adapter Configuration Configuration Options Loa Xi Xi xii xii xii xii X V XiV NDOINN a 23 23 23 Applying Changes to Database Only Information Fields f Fibre Channel SCSI Protocol Driver Confi
129. n Table 4 Error Number Information for the Fibre Channel Adapter continued Error Number Error Template Description of Error 0000 0031 FCS _ERR6 A command with an unknown adap_opcode was received through the command entry point 0000 0032 FCS_ERR6 A command with an unknown opcode was received through the command entry point 0000 0033 FCS_ERR6 A command with an unknown adap_opcode was received by the start routine 0000 0034 FCS_ERR6 A command with an unknown opcode was received by the start routine 0000 0035 FCS_ERR6 The device Driver ELS command is currently is use 0000 0036 FCS_ERR6 Someone has called the stub function This function is used when a function pointer is not set to any useful function 0000 0037 FCS_ERR6 The adapter driver failed to open the adapter 0000 0038 FCS_ERR6 The adapter driver failed to find a free command to issue reply to an ELS command 0000 0039 FCS_ERR6 The adapter driver could not find an async IOCB command to issue an ELS command for some received event These commands are typically used for sending LOGOs 0000 003A FCS _ERR6 A RECEIVE SEQUENCE IOCB was received but it does not reference a valid host buffer 0000 003B FCS_ERR4 The allowable number of attempts to bring the link up on an open has been exceeded The current link down status indicates Port bypass 0000 003C FCS_ERR4 The allowable number of attempts to bring
130. n accurate fiber cabling chart perform the following steps 1 Ensure that the inbound fiber jumper connector is clean and properly plugged into the correct port 2 Ensure that the outbound fiber jumper connector is clean and properly plugged into the correct port 3 Ensure that the both inbound and outbound fiber jumpers are not defective Do the fiber jumpers plugged into this device provide a complete signal path through this device NO Correct the fault After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done YES Go to Step 0060 8 Determine that the fiber jumper plugged into the disk storage subsystem or tape drive provides a complete signal path to it Using an accurate fiber cabling chart perform the following steps 1 Ensure that the fiber jumper connector is clean and properly plugged into the correct port 2 Ensure that the fiber jumper is not defective Does the fiber jumper plugged into this device provide a complete signal path to it NO Correct the fault After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done YES Go to Siep 0070 56 IBM Fibre Channel Planning and Integration User s Guide and Service Information Start of PDP PD0070 Other Failures Step 0070 1 Perform the following steps Refer to Chapter 7 Se
131. n alternate path The following illustration shows an example of a high availability switch configuration Here the second host provides backup for the primary host The failure of the primary host results in a transfer of the task failover to the second host without loss of access to the data The second switch performs a similar function in providing an alternate path to the data in case of complete failure of one of the switches In addition each host system has two HBAs allowing a single port failure to occur at the host or switch without loss of the path HBA Switch HBA FC Disk X Array s HBA Switch HBA Figure 16 Example of a High Availability Configuration With Two Systems and Two Switches 104 IBM Fibre Channel Planning and Integration User s Guide and Service Information The following illustration Figure 17 shows an example of a high availability cascaded switch configuration In this example four switches are cascaded in such a way that if any one switch port or even one whole switch fails the remaining three switches still form a fully functional fabric HBA Switch Switch FC Disk x Array s HBA ___ Switch Switch Figure 17 Example of a High Availability Configuration With a Single System and Four Cascaded Switches SAN Data Gateway Config
132. n the mainframe environment and is typified by the parallel and ESCON channels previously mentioned Channel design provides high bandwidth security and delivery reliability However its protocol methodology has restricted it to master slave configurations such that the mainframe is the master and initiates all communications The I O devices are relegated to slaves whose only function is to respond by executing the commands from the mainframe With the proliferation of the desk top computer and its introduction into the business and industrial environment it became immediately apparent that peer to peer communications were required Thus the second of the two interface strategies was conceived namely the Network Networks such as Ethernet and Token Ring fill the need for peer to peer communication but tend to fall short in the areas of bandwidth security and delivery reliability Fibre Channel is designed to bring together these two diverse strategies and create a high bandwidth secure reliable delivery medium that lends itself well to both master slave and peer to peer communication configurations What is Fibre Channel Fibre Channel is the name applied to a set of standards developed by the American National Standards Institute ANSI which defines protocols for information transfer While the term Fibre implies optical fiber the ANSI definition allows a variety of both copper and optical transmission media including 9 Micron
133. ncluded 0000 0009 FCS_ERR4 A READ_LA64 completed indicating that an N_Port has done a LIP reset The READ_LA mailbox is included 0000 000A FCS ERR4 An IMPLICIT Logout via a READ_LA64 mailbox was received The READ_LA64 mailbox is included 0000 000B FCS_ERR6 An unknown mailbox type was received by the interrupt handler 0000 000C FCS_ERR6 An unsupported unsolicited IOCB command was received This IOCB is included 0000 000D FCS_ERR2 An IOCB for which there is no detailed error recovery that is XMIT_ELS_REQ64_CX has failed The IOCB is included 0000 000E FCS ERR6 Intermediate response IOCB entry 0000 000F FCS_ERR4 This IOCB failed with either a N_Port Reject or Fabric Reject indicating invalid class and the retry count has been exceeded with different classes attempted on it The failing IOCB is included 0000 0010 FC_ERR4 The IOCB failed with a N_Port Reject or Fabric Reject indicating the destination N_Port is temporarily unavailable and the retry count has been exceeded on this 0000 0011 FCS_ERR4 The IOCB failed with an error indicating the need to be logged in and the retry count has been exceeded and PLOGI attempts on this The failing IOCB is included 0000 0012 FCS ERR4 The IOCB failed with an error indicating the need to do a FLOGI However the failing IOCB was not originated by the adapter driver s startup sequence The failing IOCB is included 0000 0013 FCS ERR4 The IOCB failed with an error indicating the need to do
134. ncluded 0000 001F FCS ERR4 An IOCB failed with an error indicating link down The failed IOCB is included 0000 0020 FCS_ERR4 This IOCB has been aborted too many times 0000 0021 FCS_ERR6 An IOCB failed with a local reject with an undocumented parameter error The failed IOCB is included 0000 0022 FCS_ERR4 An IOCB failed with an F_BSY or P_BSY and the retry count has been exceeded The failed IOCB is included 0000 0023 FCS_ERR6 An IOCB failed with a status that is not documented in the adapter specification The failed IOCB is included 0000 0024 FCS_ERR4 The open sequence has failed due to an IOCB failing The failed IOCB is included 0000 0026 FCS _ERR2 A mailbox has timed out The mailbox is included 0000 0027 FCS _ERR2 An IOCB has timed out The IOCB is included 0000 0028 FCS _ERR2 An unknown event has timed out 0000 0029 FCS_ERR6 A DMA error occurred when doing a D_MAP_LIST 0000 002A FCS _ERR1 Adapter POST failed after reset 0000 002C FCS _ERR1 Adapter POST failed on open 0000 002D FCS _ERR6 A Queue Ring Buffer command has failed The failing IOCB is included 0000 002E FCS_ERR2 An ABORT_MXRI64_CN has timed out 0000 002F FCS_ERR6 An unsolicited IOCB was received on the SCSI response IOCB ring The IOCB is included 0000 0030 FCS_ERR2 An Error Attention has occurred The status register and the PCI command status register are included 80 IBM Fibre Channel Planning and Integration User s Guide and Service Informatio
135. nd lslpp 1 grep df1000f The result should be similar to the following devices pci df1l000f7 com 4 3 3 0 COMMITTED Common PCI FC Adapter Device devices pci df1l000f7 diag 4 3 3 0 COMMITTED PCI FC Adapter Device devices pci df1l000f7 rte 4 3 3 0 COMMITTED PCI FC Adapter Device Software devices pci df1000f7 com 4 3 3 0 COMMITTED Common PCI FC Adapter Device Notes 1 The above data is for the Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S If you are troubleshooting the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus FC 6228 Type 4 W the data displayed will show df1 000f9 instead of df1000f7 2 If no data displays on the screen or if some of the above components are missing the device drivers are not properly installed Are the device drivers properly installed NO Refer to Chapter 3 Fibre Channel Installation on page 11 and reinstall the device drivers Done YES Go to Step 0070 1 Chapter 6 Problem Determination 43 Start of PDP PD0040 Logical Hard Disks Not Available Step 0040 1 44 Determine that the disk storage subsystem is operational online and correctly set up Perform the following steps 1 2 Ensure that the applicable disk storage subsystem is powered on Ensure that its appropriate Fibre Channel or SCSI if connected through a SAN Data Gateway port s are correctly cabled and enabled Ensure that the disk storage subsystem is properly configured that is the correc
136. ng a site plan 9 preparing for installation of other fibre channel devices 10 problem determination procedures 31 strategies 6 whatis 2 fibre channel adapter error messages 75 introduction to 7 preparing for installation 10 fibre channel classes of service 6 fibre channel configurations high availability 104 hub 99 point to point 99 SAN Data Gateway 105 switch 100 fibre channel considerations 96 fibre channel devices 95 fibre channel installation 11 fibre channel introduction 1 fibre channel operation 27 fibre channel overview 1 fibre channel port types 5 fibre channel switch description 113 installation 11 multiple configuration options 113 publications and other information 115 fibre channel topology 4 arbitrated loop 3 arbitrated loop with hub 4 fabric or switched topology 4 point to point 3 G gigabit fibre channel adapter feature 6227 7 feature 6228 7 introduction to 7 servicing 59 H high availability configuration 104 hub configuration with four host systems 100 configuration with one host system 99 configuration with two host systems 100 configurations 99 fibre channel storage 107 information service 59 INRANGE FC 9000 Fibre Channel Director 123 install adapter hardware 20 device driver software on an AIX system 12 device driver software on an SP System 13 device drivers on the SP system nodes 17 19 fibre channel 11 gigabit fibre channel adapter 11 installable image on the Control Workstation 16 se
137. nique AL_PA for each adapter in an Arbitrated Loop No two devices on a loop should have the same Preferred AL_PA The outcome of arbitration would be random and would result in permanent I O errors Every device on a loop must have unique AL_PAs To maximize efficiency hosts must have low AL_PAs for higher loop priorities Targets must have higher AL_PAs for lower priorities INIT Link Flags This option is available for AIX 5 1 and later and is used to select the link protocol to be used The choices are pt2pt Point to Point and al Arbitrated Loop This must be configured for Arbitrated Loop mode Applying Changes to Database Only The default is No for the Apply Changes to Database Only field If the field is No the ODM database is changed and the device is unconfigured and reconfigured If the field is Yes only the ODM database is changed Configuration changes are not applied to the device until it is unconfigured and reconfigured or the system is rebooted 24 IBM Fibre Channel Planning and Integration User s Guide and Service Information Information Fields The rest of the FC Adapter s Change Show Characteristics screen contains the following information Characteristic Definition FC Adapter Parent FC adapter driver instance s identification Description Description of the parent FC adapter driver Status Parent adapter driver s current status Available indicates that the driver and its adapter
138. nnel Adapters contains one of two possible types of information The log will either contain information about an error associated with the Fibre Channel Adapter itself or it will contain information about an error encountered with the Fibre Channel SCSI protocol Both error logs are described in this appendix An example of the structure of the error log is shown in Figure 7 on page 76 75 76 LABEL FCS_ERR4 IDENTIFIER B8113DD1 Date Time Thu Dec 9 18 37 27 Sequence Number 1147199 Machine Id 000023284C00 Node Id sp09 Class H Type TEMP Resource Name fcsO Resource Class adapter Resource Type df1000f7 Location 10 78 VPD Part Number EC Level Serial Number Manufacturer FRU Number Network Address 10000000C920A5E3 ROS Level and ID 029022B0 Device Specific Z0 4002206D Device Specific Z1 10020193 Device Specific Z2 3001506D Device Specific Z3 01000909 Device Specific Z4 FF1011F0 Device Specific Z5 029022BO Device Specific Z6 061122B0 Device Specific Z7 071122B0 Device Specific Z8 10000000C920A5E3 Device Specific Z9 SS2 22 Device Specific ZA S1F2 22 Device Specific ZB utsf222a Description LINK ERROR Probable Causes BUSY ADAPTER HARDWARE OR CABLE DEVICE FIBRE CHANNEL SWITCH OR FC AL HUB Failure Causes CABLES AND CONNECTIONS DEVICE FC AL HUB FI
139. nning as well as isolating I O performance bottlenecks The ESS Service Interface The ESS provides service interface ports for external connection of a service terminal This enables the service provider to perform service on the ESS using a mobile service terminal MoST or equivalent equipment The ESS service interface also provides remote service support with call home capability for directed maintenance for service personnel An analog telephone line must be provided to enable this support The service interface provides an RS232 connection through a modem switch and modem to the analog telephone line This modem and modem switch are required by the first ESS ordered and support up to seven ESSs The cable length from the ESS to the modem switch should be no more than 15 meters 50 feet The ESS provides the following service functions e Continuous self monitoring and initiation of a call call home to service personnel upon occurrence of a failure Remote access to error and problem logs for analysis by service personnel e Remote application of problem fixes by service personnel The ESS is logically configured during installation After the ESS is installed additional configuration can be performed using the ESS Web interfaces This includes modifying the remote service functions General Information This section provides general information pertaining to the 2105 Enterprise Storage Servers Enterprise Storage Server 210
140. o an error reported by the FCS adapter or FCP device 0000 0018 FSCSI_ERR6 Error on issuing a device cancel to the FC adapter driver when trying to retry a SCSI Abort Task Set to a device dev_info information is included 0000 0019 FSCSI_ERR6 Output routine failed to accept a SCSI Abort Task that is being retried from the interrupt level The return code is included dev_info information is included 0000 001A FSCSI_ERR6 Error on issuing a device cancel when trying to retry a Clear ACA to a device dev_info information is included 0000 001B FSCSI_ERR6 Output routine failed to accept a SCSI Clear ACA that is being retried from the interrupt level The return code is included dev_info information is included Appendix C Error Messages 85 Table 5 Error Number Information for the Fibre Channel SCSI Protocol continued Error Number Error Template Description of Error 0000 001C FSCSI_ERR6 This is an unknown task management request that is not a target reset abort or clear ACA received by the interrupt handler Command is included 0000 001D FSCSI_ERR6 A Target reset command that timed out has just been flushed to the FC SCSI Protocol driver However either an Abort Task Set or Clear ACA is still pending and the cancel command to flush them back has failed dev_info information is included 0000 001E FSCSI_ERR6 This is a timeout for unknown task
141. o the following assuming the adapter under test is adapter zero in slot 20 70 rmt1 is one of the tape drives that has been configured and the tape drive is a 3590 rmt1 Defined 20 70 01 3590 Tape Drive There should be an entry in the above output for each tape drive defined for this adapter Are the appropriate logical tape drives defined NO Refer to the Installation Manual for the Tape Drives and using SMIT define the appropriate tape drive s to be associated with the Fibre Channel Adapter After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done YES Go to Step 0050 4 Chapter 6 Problem Determination 47 Step 0050 2 48 Determine that the tape drive s are operational online and correctly set up Perform the 1 2 following steps Ensure that the applicable tape drive s are powered on Ensure that the appropriate SCSI interfaces from the SAN Data Gateway if present are correctly cabled and enabled Ensure that the Fibre Channel interfaces if applicable are correctly cabled and enabled Ensure that the tape drive s are properly configured that is the correct port and device addresses are set up Examine the AIX system s error log for entries associated with this failure whose Probable Causes list includes the tape drives s Using the Service Manual for the tape drives verify that the tape drive s are fully o
142. od to do so is to run the following AIX command errpt a pg This command displays the error log in its entirety one page at a time However the Gigabit Fibre Channel SMIT panels provide a means of generating an error report whose output can be customized and limited in a number of very useful ways This facility also allows the report to be written to a file for future reference To generate an error report using the Gigabit Fibre Channel Adapter SMIT facility perform the following steps 1 Type the following smitty fcsa 2 Select FC Adapter Chapter 7 Service Information 65 3 Select Generate Error Report Select 1 filename unless you want to print the report on the pop up window 5 Select 1 no on the CONCURRENT error reporting pop up window unless you want to perform concurrent error reporting Concurrent error reporting causes errors to be displayed in real time as they are entered into the system error log 6 Fill in fields as desired to control the way the report is generated Refer to Figure 7 on page 76 for an example of an error report and to help you make choices a You can choose SUMMARY or DETAILED error reporting b You can select specific Error CLASSES TYPES LABELS and IDs c You can select specific Resource CLASSES TYPES and NAMES d You can select specific SEQUENCE numbers as well as STARTING and ENDING time intervals e You can enter a FILENAME in which to write the error report 7
143. oftware supporting multiple paths to rootvg is installed To determine if your pSeries system and your Fibre Channel attached storage subsystem support FC boot refer to the documentation for these products Concurrent Storage Subsystem Access The characteristics of a given storage subsystem limit the number of active initiator systems that can concurrently access the same logical unit in the storage subsystem Refer to the documentation for your storage subsystem for further information Fibre Channel Configurations This section discusses Fibre Channel configurations in SAN environments Representative configurations are shown as examples of the wide variety of configurations possible Refer to Fibre Channel Connection Topology on page 2 for descriptions of the basic Fibre Channel topologies employed in these configurations 98 IBM Fibre Channel Planning and Integration User s Guide and Service Information In the following configurations HBA Host Bus Adapter is the Fibre Channel FC adapter in the host system The number of adapters that can be installed in an AIX system is dependent upon the type and model of that system Refer to the PC Adapter Placement Reference SA38 0538 to determine the maximum number of adapters allowed in your AIX system In configurations where multiple HBAs in the same RS 6000 or pSeries system have access to the same Fibre Channel disk storage subsystem each HBA must be assigned its own drive gr
144. oint rejected a relogin operation target_info is included 0000 00CO FSCSILERR2 PRLI failed Command is included 0000 00D0 FSCSI_ERR6 Strategy received a scsi_buf with no scsi command length and it is not a queue error recovery command 0000 00D1 FSCSILERR6 Interrupt handler received an interrupt for an inactive command The command is logged 0000 00F0 FSCSI_ERR6 Strategy received an unsupported scsi_buf dev_info information is included 88 IBM Fibre Channel Planning and Integration User s Guide and Service Information PCI Fibre Channel Adapter SCSI 3 Protocol Disk CD ROM Read Write Optical Device Error Log Sense Information The detail sense data log in the SC_DISK_ERR template for SCSI devices uses the structure scsi_error_log_def defined in the following file usr include sys scsi_buf h Sense Data Layout LLOO DDDD DDDD DDDD DDDD DDDD DDDD XXXX ccce DDDD DDDD DDDD DDDD DDDD DDDD XXXX cccc DDDD DDDD DDDD DDDD DDDD BBBB cccc CCCC CCCC CCCC CCCC CCCC RRRR RRRR RRRR VVSS AARR DDDD KKDD DDDD PPQQ DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD BBBB BBBB BBBB NNNN NNNN XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX The above fields are inte
145. ommand status register If the type field equals 5 command included this will contain the efc_cmd structure defined in the efc h file Detailed information for the sense data error numbers is shown in Table 4 on page 79 78 IBM Fibre Channel Planning and Integration User s Guide and Service Information Table 4 Error Number Information for the Fibre Channel Adapter Error Number Error Template Description of Error 0000 0000 FCS _ERR6 A mailbox completed and the SLIMs in use flag was not set The mailbox is included 0000 0001 FCS_ERR4 A mailbox failed with a link type of error The failed mailbox is included 0000 0002 FCS_ERR2 A mailbox failed with an error indicating problems with IOCB rings The failed mailbox is included 0000 0003 FCS_ERR2 A mailbox failed due to bad responder originator The failed mailbox is included 0000 0004 FCS_ERR2 A mailbox was failed by the adapter indicating no resources are available This will cause the adapter driver to reset the adapter to try to resolve this 0000 0005 FCS ERR2 A mailbox failed due to one of the remaining defined mailbox errors The failed mailbox is included 0000 0006 FCS ERR2 A mailbox failed with an undocumented error code 0000 0007 FCS_ERR4 A config port mailbox failed Mailbox is included 0000 0008 FCS_ERR4 A READ_LA64 mailbox completed but indicated the link is down The READ_LA64 mailbox is i
146. or more logical fabrics A zone can be set up in which all devices within the zone can see and access each other but cannot see or access devices not within the zone In the same way devices outside the zone cannot see or access devices within the zone In this way a specific set of storage subsystem ports can be configured to be accessed by and only by a specific set of system ports In some switches and directors all ports not specifically zoned are placed in a default zone Depending on implementation zoning can be configured using either port addresses or WWYNs and is used to provide e Increased environmental security as needed Optimized SAN resources in response to user demand e Versatility to customize environments as needed Zoning can be established in either of the following ways e Hard zoning is enforced by switch hardware In hard zoning when a zone is defined by the operator a list of permitted connections is established within the switch e Soft zoning is not enforced by switch hardware As in hard zoning a list of permitted connections is established Each server in the configuration is notified of this list at the time that connectivity is first established by the server Enforcement of the zones Appendix D Fibre Channel Devices and Configurations 97 is at the discretion of the server software If one or more soft zones are changed later the server will be notified however the server may or may not automatically
147. orage Area Network SAN environments The 2109 SAN Fibre Channel Switch provides enterprise level scalability and a fault tolerant switch fabric capable of containing a large number of Fibre Channel ports The 2109 SAN Fibre Channel Switch is available in the following models e The S08 provides 8 Fibre Channel ports while the S16 provides 16 ports e The F16 provides 16 ports supporting link speeds of up to two Gigabits per second and which are compatible with small form factor pluggable media SFP e The 2109 allows any to any connectivity that is any of its ports may be cabled to a Fibre Channel system adapter or to a Fibre Channel device In general the SAN Fibre Channel Switch provides Fibre Channel connectivity to e RS 6000 and pSeries AIX systems as well as other UNIX based servers and Intel based servers running Windows NT e Fibre Channel attached disk storage including Fibre Channel RAID Storage Servers Refer to Appendix D Fibre Channel Devices and Configurations on page 95 for a specific list of available attachments High Performance Scalable Switch Fabric Each port of a model S08 or S16 delivers up to 100 MB sec full duplex data transfer Each port of a model of F16 is capable of link speeds up to 200 MB sec and automatically negotiates to the highest common speed of all devices connected to the port Intelligent Self Management The fabric manages the SAN and solves problems For example it automa
148. oups or Logical Units LUNs Refer to the publications for your disk storage subsystem for information on how to assign LUNs Point to Point Configurations Fibre Channel devices such as disk storage subsystems can be directly attached to an AIX system host An example of this type of configuration is shown in the following illustration Figure 8 The host system on the left is shown here with four HBAs connected to a Fibre Channel disk array The host system can have one or more HBAs connecting to ports on the same or different Fibre Channel disk arrays HBA HBA FC Disk Array s HBA HBA Figure 8 Example of a Point to Point Configuration Hub Configurations One or more hubs can be included in a Fibre Channel configuration allowing a host system HBA and one or more Fibre Channel devices to be connected in a Fibre Channel Arbitrated Loop FC AL topology Hubs can be cascaded that is a port of one hub can be connected to a port of another hub Hub Configuration With One Host System The following illustration Figure 9 on page 100 shows an example of a single host system connected through hubs to a Fibre Channel disk array The host system can have one and only one HBA connected to each hub Each Fibre Channel device can have from one to four ports also connected to the hubs Appendix D Fibre Channel Devices and Configurations 99 HBA Hub HBA Hub
149. our screen fcs0 Available 20 60 If no adapter information is displayed or if it is shown as Defined refer to Chapter 6 Problem Determination on page 29 to determine the cause of the problem Verifying the Software Installation Verify the device driver for the Gigabit Fibre Channel Adapter is properly installed as follows 1 If necessary log in as root user 2 For adapter Type 4 S enter lsipp h grep p df1000f7 For adapter Type 4 W enter lsipp h grep p df1000f9 Press Enter If the Gigabit Fibre Channel Adapter device driver software is properly installed the resulting output of the above command will show a list of the applicable images The following is an example of the data that should display on your screen Chapter 3 Fibre Channel Installation 21 devices pci df1000f7 rte 4 3 3 0 COMMIT COMPLETE 10 01 01 11 59 10 devices pci df1000f7 com 4 3 3 0 COMMIT COMPLETE 10 01 01 11 59 10 devices fcp disk array rte 4 3 3 0 COMMIT COMPLETE 10 01 01 11 59 21 devices pci df1000f7 diag 4 3 3 0 COMMIT COMPLETE 10 01 01 11 59 11 devices fcp disk array diag 4 3 3 0 COMMIT COMPLETE 10 01 01 11 59 22 N T4 Note The above data is for the Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S If you installed the device driver for the 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus FC 6228 Type 4 W the data displayed will show df1000f9 instead of af1000f7 If no device driver information
150. out losing any transactions Redundant Array of Independent Disks RAID support is provided The ESS attaches the DDMs in RAID 5 configurations which distributes parity across all DDMs Non RAID disk group support is also provided Arrays may be formatted such that the array is a single DDM The ESS cache provides several features that contribute to performance and reliability including fast read from cache asynchronous write least recently used LRU replacement and internal error recovery The ESS architecture provides data sharing between AIX systems Windows NT based systems and S 390 hosts In addition data availability can be enhanced with the Data Path Optimizer product The product number is 5648 B58 Enterprise Storage System Web Interfaces This section describes the Enterprise Storage System Web Interfaces Enterprise Storage Server Network The Enterprise Storage Server Network ESSNet is a private network resident on a workstation and is a required feature The ESSNet is concurrently installed with the Enterprise Storage Server ESSNet hardware includes a workstation PC and monitor an external Ethernet hub for cable connections from the ESSNet to the Enterprise Storage Server and a modem with modem expander which allows the ESS to communicate with Remote System Support The ESSNet software on the workstation includes Windows NT 4 0 Browser software Microsoft Internet Explorer and the ESSNet application The E
151. pecific tape device e The tape device displays error messages on its LCD Display e The AIX error log has entries associated with this adapter or Fibre Channel SCSI I O Controller Protocol Device whose Probable Causes list includes the device Are there errors reported by or associated with a tape subsystem NO Go oS 00707 YES Refer to the Service Manual for the tape subsystem and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with the next step 36 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0010 7 Determine if there are errors reported by or associated with a Fibre Channel Switch Note If there is no Fibre Channel Switch in the configuration skip this step The number of possible indications that a problem may exist in the Fibre Channel Switch include The AIX application reports link or protocol errors associated with the path which includes the Fibre Channel Switch The Fibre Channel Switch displays error LEDs for one or more Fibre Channel ports connected in the Fibre Channel path for this adapter The Fibre Channel Switch displays error conditions t
152. perational Are the tape drives operational online and correctly set up NO Refer to the Service Manual for the tape drives and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with the next step YES Go to Step 0050 3 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0050 3 Determine that the SAN Data Gateway is operational online and correctly set up Note If there is no SAN Data Gateway in the configuration skip this step Go to Step Perform the following steps 1 Ensure that the SAN Data Gateway is powered on 2 Ensure that its appropriate Fibre Channel port s are cabled correctly 3 Ensure that its appropriate SCSI port s are cabled correctly 4 Using the Service Manual for the SAN Data Gateway verify that the SAN Data Gateway is fully operational Is the SAN Data Gateway operational online and correctly set up NO Refer to the Service Manual for the SAN Data Gateway and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the AIX cfgmgr command to restore the
153. r which resides on the host system and provides automatic I O path failover when a host adapter storage controller or other Fibre Channel component in that path fails General Information This section provides general information pertaining to the FAStT500 Storage Server Product Information The following table provides rack and feature information for the FAStT500 Table 8 FAStT500 Product Information Item Description 9306XXX 9308XXXX IBM Netfinity or NetBay Solutions Rack or equivalent 19 inch rack is required 35521RU Fibre Array Storage Technology enclosure required Dual active RAID controllers with 256MB cache redundant power and cooling units are standard Additional 256MB of cache is optional 35601RU FAStT500 EXP500 Storage Expansion Unit required 1 to 10 Each unit supports up to 10 converged carrier Fibre Channel hard drives redundant power and cooling units are standard FAStT Mini hub The FAStT500 Storage Controller supports up to four host side and four drive side mini hubs Mini hubs support two ports each of either short wave or long wave host side only Fibre Channel GBICs OON6882 FAStT Mini hub The FAStT500 Storage Controller supports up to four host side and four drive side mini hubs Mini hubs support two ports each of either short wave or long wave host side only Fibre Channel GBICs Fibre Channel Switches Minimum of one switch is required The FASt
154. r attachment to hosts or to a 2108 G07 Gateway Each SCSI attachment provides 1 to 15 targets with up to 64 logical units per target The host system can assign Logical Unit Numbers LUNs from 0 to 31 per target With SCSI adapters up to 128 host systems four per SCSI interface can be connected Features are available to enhance function and performance of Models F10 and F20 These features provide FICON and Fibre Channel interoperability and support an increased cache size of up to 32 GB The features are 3012 Enhanced ESCON adapter 3021 Enhanced long wave Fibre Channel FICON adapter 3023 Enhanced short wave Fibre Channel FICON adapter 4005 Increased cache size of up to 32 GB Note Features 3021 and 3023 permit Fibre Channel Boot FC Boot on certain pSeries systems featuring FC Boot support The ESS provides heterogeneous data storage that AIX systems can share with Windows NT System 390 and other UNIX workloads 131 Enterprise Storage Server Features The ESS provides the following characteristics and features FlashCopy provides a fast data duplication capability which helps eliminate the need to stop applications for extended periods of time in order to perform backups and restores Peer to Peer Remote Copy maintains a synchronous copy always up to date with the primary copy of data in a remote location This backup copy of data can be used to quickly recover from a failure in the primary system with
155. r driver This field is used to by the adapter driver to report errors to the protocol driver The valid values are defined in efc h and are Value Description 0x1 The adapter driver is indicating a host I O bus error 0x10 The adapter driver is indicating an adapter hardware failure 0x20 The adapter driver is indicating an adapter microcode failure 0xa0 The adapter has flushed this command This is normally associated with an ABORT_MXRI64_CN IOCB issued by the adapter driver It can also be associated with a RESET_RING mailbox operation done by the adapter driver to flush all IOCBs for a given IOCB ring L LUN ID to which the command was sent 82 IBM Fibre Channel Planning and Integration User s Guide and Service Information 2 lt D MOQ FCP payload control field which has the following bit flag definitions e Bit 0 indicates the FCP command is a write e Bit 1 indicates the FCP command is a read e Bit 9 indicates this is an Abort Task Set e Bit 10 indicates this is a Clear Task Set e Bit 13 indicates this is a Target Reset e Bit 14 indicates this is a Clear ACA The SCSI command payload that failed or command payload at the head of the active queue Data length of SCSI command N Port ID of this adapter N Port ID of the device to which this command was sent Reserved for future use Failed command IOCB if valid Otherwise it is the following Word Description of Contents 0 Dev_info ID LUN 1 Dev_info
156. rage 111 The SAN Data Gateway utilizes Ultra SCSI channel and Fibre Channel bandwidth for attachment of the IBM Enterprise Storage Server and the IBM Magstar 3590 Tape Subsystem General Information The following feature codes are used with the SAN Data Gateway Feature Code Description FC 2204 One Fibre Channel port short wave optical interface FC 2205 Two Fibre Channel ports FC 2303 One additional Fibre Channel port Publications and Other Sources of Information For further information about the IBM 2108 G07 SAN Data Gateway refer to the following General Information Planning and Installation Configuration and Operation IBM SAN Data Gateway Installation and User s Guide SC26 7304 Servicing e IBM SAN Data Gateway Service Guide SC26 7307 For further information visit the following Web address http www storage ibm com hardsoft products sangateway sangateway htm 112 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix G IBM 2109 S08 S16 and F16 SAN Fibre Channel Switches This appendix describes the Storage Area Network Fibre Channel Switch IBM 2109 S08 S16 and F16 SAN Fibre Channel Switch Description The 2109 S08 S16 and F16 SAN Fibre Channel Switches provide Fibre Channel connectivity to AIX systems as well as other X Open UNIX and Windows NT servers They enable connectivity to Fibre Channel attached disk storage systems and are intended to be used in St
157. rated loop or sw if connected to a switch IBM Fibre Channel Planning and Integration User s Guide and Service Information Chapter 5 Fibre Channel Operation This chapter provides information on operating the Fibre Channel devices The operation of the Gigabit Fibre Channel PCI Adapter is not apparent to the system operator The adapter merely acts as a data path between the AIX system and the device to which it is connected The actual I O device may be connected directly to the adapter or it may be connected through a hub switch or SAN Data Gateway Data transfer requests by an AIX application to or from an I O device cause the adapter by way of its device drivers and microcode to perform data and protocol operations on that Fibre Channel path While the AIX system operator may be required to perform specific actions to activate and manage the application no operator actions are required specifically for the Gigabit Fibre Channel PCI Adapter itself Operating Other Fibre Channel Devices To learn more about how to operate each of the Fibre Channel devices in the configuration refer to the appropriate appendix for each device Each device s appendix contains a section called Publications and other Sources of Information This section contains a list of publications and Web sites that provide device specific instructions and information needed for installing configuring operating and servicing of that device 27 28
158. rating and servicing of that device Fibre Channel Considerations This section discusses Fibre Channel considerations for SAN environments Switch Cascading Switch cascading is a configuration method in which the fabric can be greatly enlarged by interconnecting Fibre Channel switches Links within the fabric that is links between two cascaded switches are called hops Both switch ports involved in a cascaded connection must be of the E_Port or G_Port type While cascading enables much larger fabrics it can reduce performance Therefore cascading must be carefully planned It is recommended that the fabric be configured such that no Host Device path include more than four cascaded switches three hops Also a host device path should not contain more than one long wave link When cascading 2109 switches the following 2109 parameters must be set e In Order Delivery IOD set to ON e Dynamic Load Sharing DLS set to OFF 96 IBM Fibre Channel Planning and Integration User s Guide and Service Information Controlling Connectivity Within the Fabric Although ports in a fabric can generally connect to any other port configuration techniques are available to control connectivity within the fabric One technique of controlling connectivity is Port Blocking which allows a port to be configured such that the device attached to it cannot connect into the fabric It is also possible depending on implementation to disallow the connection of t
159. re automatically updated Press Enter to accept the values 16 The ARE YOU SURE window displays Press Enter to continue with the installation 17 The COMMAND STATUS is displayed After the installation process has completed OK will be displayed Scroll to the bottom to view the results to ensure that the installation was successful 18 Remove the installation media from the drive 19 Press F10 to exit SMIT 20 Proceed to Installing the Adapter Hardware on page 20 Installing the Device Driver on an RS 6000 SP System This section explains how to install the device driver on an AIX SP system on which AIX is already installed Installing device drivers in an SP system is complicated by two factors First the SP nodes normally do not have CD ROM or diskette devices Secondly it is often necessary or at least desirable to install the device drivers on multiple nodes To overcome the first problem place the installable images on the SP system s control workstation and then install them on the target nodes from there To overcome the second problem use the dsh distributed shell command which enables the execution of commands on multiple nodes simultaneously This provides significant advantage over using telnet or rlogin to install the device drivers on a node by node basis Refer to the PSSP Command and Technical Reference order number GC23 3900 or PSSP Administration Guide order number GC23 3897 for details on using the dsh
160. romagnetic compatibility The manufacturer cannot accept responsibility for any failure to satisfy the protection requirements resulting from a non recommended modification of the product including the fitting of option cards supplied by third parties Consult with your dealer or sales representative for details on your specific hardware This product has been tested and found to comply with the limits for Class B Information Technology Equipment according to CISPR 22 European Standard EN 55022 The limits for Class B equipment were derived for typical residential environments to provide reasonable protection against interference with licensed communication devices International Electrotechnical Commission IEC Statement This product has been designed and built to comply with IEC Standard 950 United Kingdom Telecommunications Safety Requirements This equipment is manufactured to the International Safety Standard EN60950 and as such is approved in the UK under the General Approval Number NS G 1234 J 100003 for indirect connection to the public telecommunication network The network adapter interfaces housed within this equipment are approved separately each one having its own independent approval number These interface adapters supplied by the manufacturer do not use or contain excessive voltages An excessive voltage is one which exceeds 70 7 V peak ac or 120 V dc They interface with this equipment using Safe Extra Low Voltages only In
161. ror Number Error Template Description of Error 0000 0064 FCS_ERR6 Index beyond end qiocb_free_list Command is included PCI Fibre Channel Adapter SCSI Protocol Error Log Sense Information The detail sense data for the Fibre Channel SCSI Protocol is interpreted as follows Sense Data Layout RRRR RRRR AAAA AAAA BBBB BBBB CCRR RRRR RRRR YYYY LLLL LLLL LLLL LLLL FFFF FFFF QQQQ QQQQ QQQA QQQQ QQQQ QQQQ QQQQ QQQQ GGG GGGG 0000 0000 0000 0000 EEEE EEEE EEEE EEEE VVVV VVVV VVVV VVVV VVVV VVVV VVVV VVVV VVVV VVVV VVVV VVVV VVVV VVWV VVVV VVVV VVVV TTTT TITT TTT TTT TITT TITT TITT TITT TITT TITT TITT TTT TITT TITT TTTT HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH HHHH DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD DDDD IIII IIII IIII III The above fields are interpreted as follows Field Interpretation A Error number determined by the SCSI protocol driver The error numbers are described in Table 5 on page 85 B Return code from either the adapter driver output routine control function or command entry point The possible values for this are defined in the errno h file Cc How this adapter is connected Possible values are 0 Unknown how it is connected 1 Connected point to point 2 Connected to a switch 3 Connected to an arbitrated loop Y Status value of the control element returned by the adapte
162. rpreted as follows Field LL Cc VV SS Interpretation The length of the failing SCSI command The failing SCSI command The first two digits of this field indicate the SCSI opcode A list of SCSI opcodes and their meanings can be found at the top of the usr include sys scsi_buf h file The Status Validity field It has the following possible values Value 00 01 02 03 Description Indicates that no status validity was set by the adapter driver For details of this condition see the section Status Validity of 0 on page 91 Indicates that the SCSI Status field SS is valid Indicates that the Adapter Status field AA is valid Indicates that driver status is valid This is used when the device driver detects special errors not directly related to hardware errors For details of these conditions see the section Status Validity of 3 The SCSI Status field It has the following possible values defined in the scsi_buf structure in the file usr include sys scsi_buf h Value 02 08 18 Description Check condition This indicates the device had an error and additional information is included in the Sense Data fields D PP and QQ The SCSI device is busy This can happen when a device being shared between multiple hosts is involved in an error recovery operation Indicates that the SCSI device is reserved by another host Appendix C Error Messages 89 AA The Adapter Status field It has th
163. rprise Storage Server Network Accessing ESS Specialist and ESS Copy Services ESS Specialist oe 22 amp a ESS Copy Services ESS Expert The ESS Service Interface General Information Enterprise Storage Server 2105 Publications and Other Sources of Information General Information Planning and Installation Configuration Operation Servicing Appendix L IBM 3552 Fibre Array Storage ere i PORO Server IBM FAStT500 Storage Server Description Multi host Attachment aa High availability Design Extended Distances High Performance Dynamic Management Contents gt 123 123 123 123 124 125 125 127 127 127 128 128 128 128 128 128 129 129 129 129 129 129 134 131 132 132 132 133 133 133 133 134 134 134 135 135 135 135 135 137 137 138 138 138 138 138 vii viii General Information Product Information Configuration Information Publications and Other Sources of Information General Information Planning Installation Configuration and Servicing Appendix M EMC Symmetrix 3000 5000 and 8000 Enterprise Storage Systems Symmetrix 3000 5000 and 8000 Enterprise Storage Systems Description Publications and Other Sources of Information sod ee a Appendix N Magstar and Ultrium Tape Subsystems IBM Magstar MP Multi Purpose 3570 Tape Subsystem IBM Mags
164. rt Receive Transmit The most versatile configuration is called Fabric or Switched topology In this topology nodes are cabled to ports on a Fibre Channel Switch see Figure 4 on page 5 IBM Fibre Channel Planning and Integration User s Guide and Service Information a ae Node A Fabric N_Port F_Port F_Port N_Port Transmit Receive Receive Transmit Receive Transmit 5 Transmit Receive x Node B R J Node D N_Port F_Port F_Port N_Port Transmit _ Receive Receive Transmit Receive _ Transmit Transmit Receive Figure 4 Fabric Topology Showing Node A Currently Connected to Node D The characteristics of the switch permit any to any port connection thus any of the connected nodes may be connected to any other connected node This connection is maintained only for the duration of a given transmission and then dropped allowing each of the nodes to connect with other nodes to engage in other transactions The node starting a given transaction is called the initiator and the other node is called the target node The architecture defining the Fabric topology permits the attachment of more than 16 million active ports It is quite common to find installations that incorporate multiple types of topologies For example in Storage Area Networks SANs the disk drives are often configured in groups of arbitra
165. rvice Information on page 59 and start the Device Driver Trace on the AIX system a 6 Done If the problem you are investigating involves an application refer to the documentation for the application and start the application trace if available on the AIX system Reproduce the failing scenario Stop all traces Have the following information available All trace data gathered above Any errpt data in the AIX Error Log Any errors reported by the application Any error data present in any of the Fibre Channel Devices including LED indicators A detailed description of the Fibre Channel cabling scheme Hardware microcode and device driver levels for the Fibre Channel PCI Adapter and all Fibre Channel devices in the failing configuration A detailed description of the error failure or problem Call AIX Support Chapter 6 Problem Determination 57 58 IBM Fibre Channel Planning and Integration User s Guide and Service Information Chapter 7 Service Information This chapter provides information on servicing the Fibre Channel environment Gigabit Fibre Channel PCI Adapters This section has information on servicing your Gigabit Fibre Channel Adapters Figure 5 shows the Gigabit Fibre Channel PCI Adapter O 5 a E O O A Y o _JX1 mj 103 eea ol on mo 3
166. s F4 to select the software to be installed Use the F7 key to select the required device drivers For adapter Type 4 S select devices pci df1000f7 Or for adapter Type 4 W select devices pci df1000f9 d Use the F7 key to select the following device driver devices pci common IBM fc e Repeat Step 3d for any other device drivers that you may need These may include e devices fcp disk e devices fcp disk array e devices fcp tape Press Enter f Move the cursor down to DIRECTORY for storing the software package and enter the appropriate LPP source destination directory spdata sysl install 1ppsource substituting the directory name which was recorded in step 4b on page 16 for 1ppsource Press Enter to begin copying the files Note This step may take several minutes while the directory table of contents is updated Continue to the next section Finalizing Preparations for Device Driver Installation Finalizing Preparations for Device Driver Installation 1 Update the SPOT Shared Product Object Tree a Type the following smitty nim_res_op The Resource Name panel displays with the following highlighted boot resources boot b Move the cursor down until the SPOT resource is highlighted The SPOT resource should look similar to spot_AIX433 resources spot_ ppsource Chapter 3 Fibre Channel Installation 17 where 1ppsource is the directory name recorded in step 4b on page 16 c When the SPOT r
167. s Guide and Service Information Sense Data Layout LLOO XXXX XXXX XXXX XXXX XXXX XXXX XXXX ccce XXXX XXXX XXXX XXXX XXXX XXXX XXXX cccc XXXX XXXX XXXX XXXX XXXX XXXX ccce XXXX XXXX XXXX XXXX XXXX XXXX cccc XXXX XXXX XXXX XXXX XXXX XXXX ccce XXXX XXXX XXXX XXXX XXXX XXXX cccc XXXX XXXX XXXX XXXX XXXX NNNN cccc XXXX XXXX XXXX XXXX XXXX NNNN ccce XXXX XXXX XXXX XXXX XXXX XXXX RRRR XXXX XXXX XXXX XXXX XXXX XXXX RRRR XXXX XXXX XXXX XXXX XXXX XXXX RRRR XXXX XXXX XXXX XXXX XXXX XXXX VVAA XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX The above fields are interpreted as follows Field LL C VV AA Interpretation The length of the failing SCSI command The failing SCSI command The first two digits of this field indicate the SCSI opcode A list of SCSI opcodes and their meanings can be found at the top of the usr include sys scsi_buf h file Reserved for future use and should be 0 The Status Validity field In this case it will be 03 driver status The Adapter Status field When Status Validity is 03 and Adapter Status is 01 this indicates mode data corruption These fields are ignored Normally they are set to 0 The Open Count field This 32 bit field indicates the number of times the device has been opened from
168. stem will supply actual values for output Proceed with the device driver installation starting with section Setting Up and Checking System Prerequisites and perform the following tasks on the control workstation Setting Up and Checking System Prerequisites If you are installing the Gigabit Fibre Channel PCI Adapter FC 6227 Type 4 S you will be installing Device Driver devices pci df1000f7 If you are installing the 2 Gigabit Fibre Channel Adapter FC 6228 Type 4 W you will be installing Device Driver devices pci df1000f9 Do the following to check system prerequisites 1 Log in to the control workstation as root user 2 lf there are several nodes involved and you are going to use the dsh command with a working collective file perform the following steps a Go to a temporary directory by typing cd tmp Press Enter b Create the working collective file containing a list of the relevant nodes on which to perform the update in preparation for the dsh command Type the following commands on the AIX command line substituting actual node names for each nodenameN Press Enter after each line cat gt groupl nodename1 nodename2 nodename3 14 IBM Fibre Channel Planning and Integration User s Guide and Service Information nodenameN CTRL D export WCOLL tmp group1 c Test the working collective file by typing dsh date Press Enter The results should include a line for each node in the list wher
169. storage systems for bandwidth intensive applications 141 In addition to extensive connectivity the Symmetrix 3000 5000 and 8000 Enterprise Storage Systems offer a number of features including Up to 16GB of non volatile cache employing Least Recently Used LRU sequential prefetch and cache searching algorithms Channel Directors which handle I O requests from the hosts Disk Directors which handles access to the disk drives Dynamic Mirror Service Policy for mirrored RAID 1 volumes Remote mirroring Industry standard SCSI Head Disk Assemblies HDAs Non volatile power system Dynamic disk sparing and hardware redundancy Concurrent maintenence capabilities such as nondisruptive component replacement and nondisruptive microcode upgrades Fibre Channel Boot FC Boot on certain pSeries systems featuring FC Boot support Publications and Other Sources of Information For further information about the EMC Symmetrix 3000 and 5000 Enterprise Storage Systems refer to the following 142 Symmetrix Open System Environment Product Guide Volume l 200 999 563 EMC Corporation For more information on the EMC symmetrix enterprise storage systems visit the following Web address http www emc com IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix N Magstar and Ultrium Tape Subsystems This appendix describes the Magstar and Ultrium Tape Subsystems IBM Magstar MP Multi Purpose 3570 T
170. switch view for each Director The FICON Manager Server feature supports in band management of the Director by System Automation for OS 390 on IBM 9672 Parallel Enterprise G5 G6 and zSeries 900 servers This support provides a single point of control for managing connectivity in active I O configurations 124 IBM Fibre Channel Planning and Integration User s Guide and Service Information Product and Feature Information The INRANGE FC 9000 Fibre Channel Director is available as either a 64 port model of a 128 port model Features available for the two Director models are Interconnection Kit for 128 Port Capability interconnects two 64 port directors High Availability Option with N 1 redundant control switch and power modules Short wave GBIC multi mode fiber Long wave GBIC single mode fiber Extended Long wave GBIC 80 kilometers Non high availability Slot Filler I O FIO Module with eight ports FIO Slot Filler Notes 1 Eight GBICs are required for each FIO Module eight short wave eight long wave eight extended long wave or mixture of one or more short wave long wave and or extended long wave GBICs All slots must be filled with FIO modules or fillers Publications and Other Sources of Information For more information on the INRANGE FC 9000 Director visit the folllowing Web Address http www storage iobm com iobmsan products sanfabric htm Appendix INRANGE FC 9000 Fibre Channel Director 125 126
171. t number of LUNs are assigned to the applicable port s Examine the AIX system s error log for entries associated with this failure whose Probable Causes list includes the disk storage subsystem Using the Service Manual for the disk storage subsystem verify run diagnostics and so on that the subsystem is fully operational Is the disk storage subsystem operational online and correctly set up NO Refer to the Service Manual for the disk storage subsystem and continue problem determination procedures using the information provided therein After the problem is resolved you might need run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with the next step YES Go to Siep 00407 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0040 2 Determine that the SAN Data Gateway is operational online and correctly set up Note If there is no SAN Data Gateway in the configuration skip this step Perform the following steps 1 Ensure that the SAN Data Gateway is powered on 2 Ensure that its appropriate Fibre Channel port s are cabled correctly 3 Ensure that its appropriate SCSI port s are cabled correctly 4 Using the Service Manual for the SAN Data Gateway verify that the SAN Data Gateway
172. t wave or long wave optical GBIC ports Long wave GBIC ports are required for distances over 500 meters up to a maximum distance of 10 kilometers 107 2103 H07 IBM Fibre Channel Storage Hub Hardware The standard Fibre Channel Storage Hub configuration is a rack mounted tray that requires 1 U of rack space A single U rack unit is 1 75 or 44 45 mm high Two Fibre Channel Storage Hubs can be housed side by side in an industry standard 19 rack or in an IBM Seascape rack An optional stand alone table top configuration is also available Fibre Channel Storage Hub Reliability and Availability The Fibre Channel Storage Hub is designed for implementing multi node server clusters and storage systems for high availability and disaster recovery solutions In the event of a failure or natural disaster work can be transferred immediately with the appropriate software to a failover location without interruption of operations or loss of data General Information The following information pertains to the SAN Fibre Channel Switches gigabit interface connectors GBIC Fibre Channel Arbitrated Loop Hub Item Ports Media type Standard Optional Short wave GBIC Item Cable type Maximum distance Cable type Maximum distance Long wave GBIC Item Cable type Maximum distance Description 7 Optical GBIC 4 short wave GBICs 3 short wave or long wave GBICs Description 50 125 micron meter um multimode fiber
173. tar MP Multi Purpose 3575 Tape Library Dataserver IBM Magstar 3590 B11 B1A E11 E1A Tape Subsystem Magstar 3590 Performance and Capacity Servicability hain and Data ads Usability a i k General Information Ultrium 3580 Tape Family General Information Publications and Other Sources of Information General Information Planning and Installation Configuration Operation Servicing Index IBM Fibre Channel Planning and Integration User s Guide and Service Information 139 s 139 140 140 140 140 144 144 142 143 143 144 145 145 145 146 146 147 148 148 148 148 148 149 151 Safety Notices A danger notice indicates the presence of a hazard that has the potential of causing death or serious personal injury A caution notice indicates the presence of a hazard that has the potential of causing moderate or minor personal injury For a translation of the safety notices contained in this book see the System Unit Safety Information manual order number SA23 2652 Electrical Safety Observe the following safety instructions any time you are connecting or disconnecting devices attached to the workstation DANGER An electrical outlet that is not correctly wired could place hazardous voltage on metal parts of the system or the devices that attach to the system It is the responsibility of the customer to ensure that the outlet is correctly wired and
174. tches provide Fibre Channel connectivity to e IBM pSeries RS 6000 and RS 6000 SP as well as other UNIX based servers e Intel based servers running Windows NT and 2000 e Disk storage including IBM Enterprise Storage Server and Modular Storage Server e The McDATA ES 1000 Switch which provides connectivity and concentration of multiple Magstar 3590 or Ultrium LTO FC AL tape devices to a single Fabric Switch port e The IBM SAN Data Gateway with UltraSCSI attachment to IBM Magstar 3590 and Ultrium LTO tape systems and libraries Appendix H McDATA Enterprise Fibre Channel Directors and Switches 119 McDATA Fabric Switches and McDATA Enterprise Fabric Connectivity EFC Management software support enable interconnection of multiple Fabric Switches and McDATA Enterprise Fibre Channel Directors for the creation of Enterprise to edge SANs The McDATA Viper SAN engine serial cross bar technology delivers scalable non blocking switch performance Each port delivers up to 100 MB sec full duplex data transfer Full throughput at extended distances up to 100 kilometers 60 miles is enabled with long wave optics 60 buffer credits per port and repeaters McDATA ES 1000 Switch Description The McDATA ES 1000 Loop Switch operates as a managed hub to attach FC AL devices to McDATA switched fabrics Certain Fibre Channel devices such as the IBM Magstar and LTO tape devices require conversion from Fibre Channel loop FL_Port to switched fabric F_Port
175. ted loops with the whole SAN interconnected in a fabric topology The host computers would also be connected within the fabric taking full advantage of the any to any connectivity afforded by the topology Fibre Channel Port Types There are several different types of ports differentiated according to function The more common types are Port Type Description N_Port Used in point to point configurations NL_Port Used in Arbitrated Loops F_Port Used in switched topologies FL_Port Used in switched topologies intersecting with loops Chapter 1 Introduction 5 Fibre Channel Classes of Service 6 Port Type E Port G_Port Description Used in switched topologies to interconnect multiple cascaded switches A port that can be used as either an F_port or an E_Port Fibre Channel Standards define four strategies for communication within the fabric These strategies are known as classes of service and are listed below Class Class 1 Class 2 Class 3 Class 4 Description Used in point to point configurations A dedicated connection between two ports is established and remains in effect throughout the transaction Confirmation of receipt of frames is required of the recipient No dedicated connection is set up but frame switching is used to route the data between nodes Confirmation of receipt of frames is required of the recipient As in Class 2 frame switching is used rather than a dedicated connection however no
176. tems Attached 102 IBM Fibre Channel Planning and Integration User s Guide and Service Information Switch Configuration with Four Host Systems The following illustration Figure 14 shows an example of four host systems connected through a switch to a Fibre Channel disk array HBA HBA HBA HBA HBA HBA HBA HBA Switch FC Disk Array HBA HBA HBA HBA HBA HBA HBA HBA Figure 14 Example of a Switch Configuration With Four Host Systems Attached Appendix D Fibre Channel Devices and Configurations 103 Cascaded Switch Configuration The following illustration Figure 15 shows an example of switch cascading In this example one HBA and one FC Disk port are shown connected through four cascaded switches FC Disk HBA Switch Switch Switch Switch Array s Figure 15 Example of a Configuration With Four Cascaded Switches Switches in a High Availability Configuration Fibre Channel lends itself well to High Availability Cluster Multi Processing HACMP environments Switch configurations employing two or more host systems and two or more switches provide high availability capabilities such that in the event of failure of any one component access to the data remains intact through a
177. tep The number of possible indications that a problem may exist in a SAN Data Gateway include The AIX application reports data or other errors associated with a SCSI Tape Device or SCSI Disk Storage Subsystem connected to the Fibre Channel configuration through a SAN Data Gateway and you have already eliminated the tape or disk device as the point of failure The AIX error log has entries associated with this adapter or Fibre Channel SCSI I O Controller Protocol Device whose Probable Causes list includes a device connected through a SAN Data Gateway and the device has been eliminated as the point of failure e The SAN Data Gateway s Fibre Channel Port SCSI Port or Power Status LEDs indicate a error Are there errors reported by or associated with a SAN Data Gateway NO Go to Step 0010 9 YES Refer to the Service Manual for the SAN Data Gateway and continue problem determination procedures using the information provided therein After the problem is resolved you might need to run the AIX cfgmgr command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue with the next step 38 IBM Fibre Channel Planning and Integration User s Guide and Service Information Step 0010 9 Determine if there are errors reported by or associated with a Fibre Channel Storage Hu
178. terrupt handler 0000 0007 FSCSI_ERR2 A cancel command failed The cancel command is included in the error log 0000 0008 FSCSI_ERR4 Invalid command type for cancel completed Neither device nor target cancel target_info information is included 0000 0009 FSCSI_ERR4 Name Server request for all FCP N_Port IDs SCIOLNMSERV ioctl was rejected by Name Server The Name Server s reason code is included 0000 000A FSCSI_ERR4 Name Server request for the N_Port ID of a Worldwide Name SCIOLQWWN ioctl was rejected by Name Server The Name Server s reason code is included 0000 000B FSCSI_ERR4 Name Server request for a query of the valid FC classes of an N_Port ID was rejected by Name Server The Name Server s reason code is included 0000 000C FSCSILERR4 FC Classes returned for an N_Port ID are not valid with this adapter The class mask of this device is returned 0000 0014 FSCSI_ERR8 Error on either a SCIOHALT or SCIORESET Commamd element is included The error is the due to non zero status returned by FCS adapter driver 0000 0015 FSCSI_ERR8 Error on either a SCIOHALT or SCIORESET Command element is included The error is the due to an error reported by the FCS adapter 0000 0016 FSCSI_ERR8 Error on an interrupt level task management Command element is included The error is the due to non zero status returned by FCS adapter driver 0000 0017 FSCSI_ERR4 Error on an interrupt level task management Command element is included The error is due t
179. the link up on an open has been exceeded 0000 003D FCS ERR4 A link error has been encountered 0000 003E FCS_ERR4 In old Port mode and not allowed to be switched to FC AL mode but a FLOGI has been self initiated 0000 0040 FCS ERR4 A link error has been encountered 0000 0041 FCS ERR4 A READ_LA64 completed indicating that the link is down for this particular READ_LA64 All previous READ_LA64s also failed and this was the last attempt to get link up The link will now be considered down and no further attempts to do a link up will occur until new I O requests are issued to the adapter driver The READ_LA mailbox is included 0000 005B FCS ERR6 A PDISC Port Discover Extended Link Services Command has been received from a device which is not logged in 0000 005D FCS _ERR6 An unknown unsolicited IOCB was received The IOCB is included in the detail data 0000 005E FCS_ERR6 A XMIT Sequence IOCB operation failed This is most likely a Name Server operation 0000 005F FCS ERR4 A microcode error has been encountered 0000 0060 FCS_ERR6 Index beyond end qbuf_free_list OCB and index is included 0000 0061 FCS_ERR6 Index beyond end els_free_list Command is included 0000 0062 FCS_ERR6 Index beyond end qbuf_free_list OCB and index is included 0000 0063 FCS ERR6 Index beyond end qiocb_free_list Command is included Appendix C Error Messages 81 Table 4 Error Number Information for the Fibre Channel Adapter continued Er
180. the switch have been soft rezoned recently If so run the AIX cfgmgr command to set up the new configuration parameters This will likely resolve the problem Determine if host to switch cabling has been changed or swapped recently If so run the AIX cfgmgr command unless you ran it above to update the configuration attributes If these attributes are not updated results of commands such as the Isattr El will not yield the correct information for attributes such as the scsi_id field Running the cfgmgr AIX command will likely resolve the problem Is the Fibre Channel Switch operational online and correctly set up NO Refer to the Service Manual for the Fibre Channel Switch and continue problem determination procedures using the information provided therein After the problem is resolved you might have to run the cfgmgr AIX command to restore the Fibre Channel environment to a functional state Done Note If the problem is not resolved using the Service Manual information for this device return to this problem determination procedure and continue YES Go to Siep 0060 1 46 IBM Fibre Channel Planning and Integration User s Guide and Service Information Start of PDP PD0050 Logical Tape Drives Not Available Step 0050 1 Determine if the logical tape drives associated with the Fibre Channel adapter are defined recognized in the AIX system Run the following AIX command lsdev Cc tape pg The result should be similar t
181. tically identifies and registers hosts and storage devices In addition the fabric isolates problem ports and reroutes traffic onto alternate paths Security and integrity are provided by hardware enforced zoning that controls data access among ports Automatic traffic rerouting occurs as traffic and network conditions change Multiple Configuration Options Rack mounted installation is standard and desktop installation is available as an option For the S08 and S16 models four short wave laser 500 m universal ports are standard In addition a mixture of short wave and long wave 10 km ports can be 113 configured by adding more Gigabit Interface Converters GBICs A second power supply can be added if redundant power is required For the 2109 F16 16 SFP ports and two redundant hot swappable power supplies are standard All three 2109 models have universal ports and automatically determine the port type when connected to a fabric port F_port fabric loop port FL_port or expansion port E_port IBM StorWatch SAN Fibre Channel Switch Specialist The StorWatch SAN Fibre Channel Switch Specialist provides a comprehensive set of management tools that support a Web browser interface for integration into existing enterprise storage management structures The Specialist provides security and data integrity by limiting zoning host system attachment to specific storage systems and devices General Information This section provides general
182. tically initialize and log into the fabric upon cable connection and unit power up Loop switch functions such as port bypass auto LIP upon insertion and hardware validation of newly attached FC AL devices enable tape devices to be added and removed without impacting other device operations Operating and Management Modes This section discusses operating and management modes for McDATA Enterprise Fibre Channel directors and switches as well as the effects of these modes on fabric behavior 120 IBM Fibre Channel Planning and Integration User s Guide and Service Information Operating Modes The McDATA ED 5000 ED 6064 ES 3016 and ES 3032 can be configured to operate in any one of three operating modes These modes are Open Systems Open Fabric mode Open Systems McDATA Fabric mode and S 390 mode Open Systems Open Fabric operating mode refers to connectivity between various UNIX or Intel based servers and storage devices using Fibre Channel ports and interoperability between switches and directors in the fabric all of which need not be manufactured by McDATA Open Systems McDATA Fabric operating mode refers to connectivity between various UNIX or Intel based servers and storage devices using Fibre Channel ports and interoperability between other switches and directors in the fabric all of which must be manufactured by McDATA S 390 operating mode refers to connectivity between Parallel Enterprise G5 G6 and zSeries servers
183. tine failed to accept device Cancel IOCB for a device The return code and dev_info information are included 0000 006C FSCSI_ERR6 Output routine failed to accept target Cancel IOCB for a device The return code and target_info information are included 0000 006D FSCSI_ERR6 Output routine failed to accept process login IOCB for a device The return code and command are included 0000 006E FSCSI_ERR6 Output routine failed to accept target reset IOCB for a device The return code and target_info are included 0000 0070 FSCSI_ERR2 A timeout occurred waiting for link to come up 0000 00A0 FSCSI_ERR4 A command entry point command was returned from the adapter with an error This command was for an IOCTL N_Port Login target_info is included 0000 00A1 FSCSI_ERR4 A command entry point relogin command failed This relogin was issued from the interrupt level as part of error recovery target_info is included 0000 00A2 FSCSI_ERR8 A command entry point relogin command returned successfully but the device at this N_Port ID is different i e a different port name or node name target_info is included 0000 00A3 FSCSILERR4 A Reset Ring on the SCSI IOCB ring failed 0000 00A4 FSCSI_ERR6 Unknown command was sent from the adapter driver to protocol driver 0000 00A5 FSCSILERR6 Send Sequence failed The command is included 0000 00B0 FSCSI_ERR6 Adapter driver s command entry point rejected a reset ring 0000 00B1 FSCSI_ERR4 Adapter driver s command entry p
184. tion 20 58 Maximum number of COMMANDS to queue to the adapter 200 Maximum Transfer Size 0x100000 Preferred AL_PA Ox1 NIT Link flags al Apply change to DATABASE only no XS s Configuration Options The FC Adapter has the following configuration options e Maximum Number of Commands to Queue to the Adapter e Maximum Transfer Size e Preferred AL_PA e INIT Link Flags AIX Version 5 1 only Maximum Number of Commands to Queue to the Adapter Use this option to adapt to various memory system conditions The default is 200 but you can conserve memory at the expense of performance by reducing this number to 23 as little as 20 If you have sufficient memory the maximum is 1024 for a Gigabit Fibre Channel PCI Adapter Type 4 S The maximum is 2048 for a 2 Gigabit Fibre Channel Adapter for 64 bit PCI Bus Type 4 W If the SCSI target device is connected to the host through a SAN Data Gateway SDG then this parameter cannot exceed the SDG limit of 240 SAN Data Gateway is a SCSI to Fibre Channel bridge Maximum Transfer Size The default is 256 kilobytes which is the largest single transfer size For tape transfers of 2 megabytes or more are possible The range is 256 kilobytes to 16 megabytes Preferred AL_PA The preferred AL_PA Arbitrated Loop Physical Address value is the address that the adapter requests at the beginning of every LIP Loop Initialization Process This configuration option must be used to set a u
185. tting up and checking system prerequisites 14 verify adapter hardware 21 verify device driver 21 verify software 21 installation of the gigabit fibre channel PCI adapter 10 installing device drivers on the SP system nodes 19 installing other fibre channel devices 22 installing the adapter hardware 20 installing the gigabit fibre channel adapter 11 introduction fibre channel 1 to problem determination 29 machine type 2101 128 2102 127 2103 107 128 2105 131 2108 111 2109 113 3552 137 3570 143 3575 144 3580 147 3590 145 Magstar 3590 tape subsystem 145 Magstar MP Multi Purpose 3570 tape subsystem 143 Magstar MP Multi Purpose 3575 tape library dataserver 144 McDATA ED 5000 RAS Features 117 fibre channel director 117 McDATA ED 6064 Enterprise Fibre Channel Director 118 McDATA ES 1000 Switch 120 McDATA ES 3016 and ES 3032 Fabric Switch 119 O online publications xii operating other fibre channel devices 27 P PCI fibre channel adapter error log sense information 77 PCI fibre channel adapter SCSI 3 protocol error log sense information 89 152 IBM Fibre Channel Planning and Integration User s Guide and Service Information PCI fibre channel adapter SCSI protocol error log sense information 82 plan and diagram all cable routing 9 planning configuration 9 fibre channel configuration 9 information 9 planning information 9 port types fibre channel 5 preparing asite plan 9 preparing for installation of other fibr
186. ty Management Terminal e Problem Determination Information from other devices Other Fibre Channel devices including disk storage subsystems provide problem determination information in various ways such as status LEDs operator panels and logout information Complexity of the Fibre Channel Environment The Fibre Channel environment can be difficult to troubleshoot A typical Fibre Channel configuration such as a Storage Area Network SAN may contain some or all of the following e One or more hosts perhaps running any of several different operating systems One or more disk storage subsystems with a number perhaps a very large number of disk devices in RAID or non RAID configurations One or more tape subsystems connected by their native Fibre Channel interfaces One or more hubs connecting hosts and disk storage subsystems in loop configurations e One or more Fibre Channel switches connecting the various devices and hosts in a fabric environment One or more SAN Data Gateways allowing the introduction of SCSI attachable Disk Storage Subsystems or Magnetic Tape Subsystems into the Fibre Channel environment 29 A large number of Fibre Channel jumper cables interconnecting the various hosts and devices Fiber trunks carrying data between floors and between buildings Patch panels connecting the various jumper cables and trunk cables Troubleshooting the Fibre Channel environment is further complicated by th
187. uide and Service Information Fibre Channel nodes are interconnected in three general topologies The simplest is known as point to point and consists of two nodes connected exclusively to each other See Node A N_Port N_Port Transmit Receive Receive 4 Transmit Figure 1 Point to Point Topology Another common configuration is known as the arbitrated loop In the loop topology three or more nodes up to a maximum of 127 are connected such that the transmitter receiver cabling forms a closed loop see Figure 2 Node A Node C NL_Port NL_Port Transmit Receive Receive jq _ Transmit Node B Node D NL_Port NL_Port Transmit _ Receive Receive Transmit Figure 2 Arbitrated Loop with Four Nodes Chapter 1 Introduction 3 4 A variation of the loop uses a hub The hub is located in the center of the loop and allows the transmit and receive lines from each port to be run in a single cable The result looks like a star configuration see Figure 3 but operates exactly like the arbitrated loop topology previously described Node A NL_Port Hub Transmit gt Receive l p Node B NL_Port Transmit gt Receive lt NL_Port Receive Transmit Figure 3 Arbitrated Loop Using a Hub Node D NL_Po
188. ulti Purpose 3575 family of tape libraries provide unattended storage for mid range systems and network servers When equipped with Magstar MP 3570 Model C00 C22 Tape Drives the 3575 Library is supported in a Fibre Channel SAN through the 2108 G07 SAN Data Gateway Magstar MP 3575s provide the following features Multi host sharing of homogeneous or heterogeneous systems in user defined logical libraries Magstar MP model C drives are standard on all Magstar MP 3575s Near online data access of up to 6 8 TB 3 1 compression Robot with a dual gripper Bar code reader Support for up to three user defined logical libraries available Two to six drives The Magstar MP 3575 Tape Library Dataserver is available in five models two with expansion capabilities All models come standard with two Magstar MP Model C drives The following table contains information for the five models Table 10 Model Information for the Magstar MP 3575 Tape Library Dataserver Model Number LO6 L12 L18 L24 L32 Cartridge Storage Slots 60 120 180 240 324 Storage Capacity 0 42 1 2 0 84 2 5 1 2 3 7 1 6 5 0 2 2 6 8 Additional Drives 0 2 4 4 4 User defined Logical 1 1or2 1to3 1to3 1to3 Libraries Field Upgradable No To L18 No To L32 No Note Storage Capacity is in TB and 3 1 compression is assumed 144 IBM Fibre Channel Planning and Integration User s Guide and Service Information IBM Magstar 3590 B11 B1A
189. upported Applying Changes to Database Only The default is No for the Apply Changes to Database Only field If the field is No the ODM database is changed and the device is unconfigured and reconfigured If the field is Yes only the ODM database is changed Configuration changes are not applied to the device until it is unconfigured and reconfigured or the system is rebooted Information Fields The rest of the FC SCSI Protocol Device s Change Show Characteristics screen contains the following information 26 Characteristic FC SCSI virtual device Description Status Location Adapter SCSI ID How this adapter is CONNECTED Definition Adapter s identification as a virtual device Description of the virtual device Virtual device s current status Available indicates that the driver and its adapter are functioning Location code of the FC SCSI Protocol Driver All children of this protocol device will have the same value for location The parent adapter driver instance fcsX instance will have for its location code the first two values of this three value location code So for example if Location is 20 58 01 for fscsi0 all children will have the location code of 20 58 01 and the parent FC adapter driver instance for example fcsO will have a location code of 20 58 This is the adapter s SCSI ID identical to the N_Port ID in the Fibre Channel This field indicates either al if connected to an arbit
190. urations SAN Data Gateways can be included in a Fibre Channel configuration allowing SCSI devices to be connected into the Fibre Channel Environment The following illustration shows a single host system with three HBAs connected to a SAN Data Gateway On the right a Tape and Disk Subsystem are shown connected through two SCSI interfaces each The SAN Data Gateway can have up to three Fibre Channel ports each connected to an HBA in one or more hosts It provides four SCSI interfaces for attaching SCSI Tape and Disk Subsystems HBA SCSI HBA SAN Tape s Data Gateway HBA SCSI Disk s Figure 18 Example of a SAN Data Gateway Configuration With One Host System Attached Appendix D Fibre Channel Devices and Configurations 105 106 IBM Fibre Channel Planning and Integration User s Guide and Service Information Appendix E IBM 2103 H07 Fibre Channel Storage Hub This appendix describes the 2103 H07 Fibre Channel Storage Hub IBM 2103 H07 Fibre Channel Storage Hub Description The 2103 H07 Fibre Channel Storage Hub provides Fibre Channel Arbitrated Loop FC AL connectivity in Fibre Channel environments It permits up to seven Fibre Channel host or device ports per hub to be configured in an Arbitrated Loop configuration while providing the following advantages over non hub loops e Provides seven hot pluggable gigabit FC AL ports with automatic bypass of unused ports
191. vice Information Appendix I INRANGE FC 9000 Fibre Channel Director This appendix decribes the INRANGE FC 9000 Fibre Channel Director INRANGE FC 9000 Fibre Channel Director Description The INRANGE FC 9000 is a director class switch that can be used as the backbone of a high speed storage area network SAN It is scalable from 24 to 128 ports and is designed to provide high levels of availability performance and intregrity The Director features N 1 redundancy at all critical points of design automatic internal failover extensive hot swapping nondisruptive firmware updates and automatic fault detection and isolation In addition call home and pager capabilities can automatically alert support and maintenance personnel to accelerate problem resolution The High Availability Option provides redundancy for all electronic and power modules helping to enable continuous data access and high performance in the event of a single component failure Together these capabilities are designed to help provide uninterrupted full bandwidth service without the loss of data access during periods of failure repair maintenance and capacity upgrades Multiple Directors can provide a scalable enterprise SAN backbone that supports consolidated storage management applications such as disk sharing tape pooling and enterprise wide data sharing FC 9000 Connnectivty The Director provides Fibre Channel Connectivity to e RS 6000 pSeries and
192. witched fabric connections and arbitrated loops In addition specifics are provided on various Storage Area Network SAN configurations particularly involving Fibre Channel switches and hubs In this publication AIX system SP system and AIX SP system are used These terms all refer to the AIX operatng system How to Use This Book This guide is intended for operators programmers system administrators and service representatives responsible for the following e Fibre Channel planning and installation e Fibre Channel component integration e Fibre Channel network maintenance e Fibre Channel SAN administration e Fibre Channel network operation and its considerations e Fibre Channel problem determination and service considerations This book supplements the other books shipped with the system unit Fibre Channel Director hubs and storage devices Be sure to observe all the safety precautions outlined in the system unit books when installing cabling and servicing the Fibre Channel Network or any of its components xi Highlighting The following highlighting conventions are used in this book Bold Identifies commands key words files directories and other items whose names are predefined by the system Italics Identifies parameters whose actual names or values are to be supplied by the user Monospace ISO 9000 Identifies examples of specific data values examples of text similar to what you might see displa
193. wo specific ports in the fabric For example the switch port connected to a given port ina system can be configured to expressly forbid connection to the switch port connected to a given port in a storage subsystem In this way any attempt to connect these two ports will be denied This type of configuration customization is enforced by switch hardware Another technique that can be used to control connectivity in the fabric is called Port Binding which allows administrators to logically associate a device with a physical switch port such that no other device can access it This capability provides additional assurances that only authorized devices can access fabric resources Devices are bound to switch ports using the Fibre Channel worldwide name WWN of the physical port on the device The WWN of a port is a 64 bit number and is the unique identifier for each individual Fibre Channel port in existence With Port Binding in effect the switch verifies the supplied WWN against a configured value and allows access to the fabric only if the device has been authorized Devices that attempt to connect on a switch port using an unathorized WWN are prohibited from participating in the fabric and are reported to higher level management applications as potential security violations A more complex and useful technique for controlling connectivity in the fabric is called Zoning This configuration technique allows the SAN fabric to be divided into two
194. y name which was recorded in step 4b on page 16 for 1ppsource and substituting the name of the Control Workstation for controlwks 4 Perform a preview of the files to be installed on the nodes Type the following commands one line at a time including the quotation marks at the AIX command prompt Press Enter after each line For adapter Type 4 S dsh Islpp 1 devices pci df1000f7 2 gt amp 1 more Or for adapter Type 4 W dsh Islpp 1 devices pci df1000f9 2 gt amp 1 more And continue for other device drivers being installed such as dsh Islpp 1 devices common IBM fc 2 gt amp 1 more dsh Islpp 1 devices fc disk 2 gt amp 1 more dsh Islpp 1 devices fc disk array 2 gt amp 1 more dsh Islpp 1 devices fc tape 2 gt amp 1 more Chapter 3 Fibre Channel Installation 19 5 Perform the device driver installation Type the following commands one line at a time including the quotation marks at an AIX command prompt Press Enter after each line For adapter Type 4 S dsh installp p acgqXd mnt devices pci dfl000f7 2 gt amp 1 more Or for adapter Type 4 W dsh installp p acgqXd mnt devices pci df1000f9 2 gt amp 1 more And continue for other device drivers being installed such as dsh installp p acgqXd mnt devices common IBM fc 2 gt amp 1 more dsh installp p acgqXd mnt devices fcp disk 2 gt amp 1 more dsh installp p acgqXd mnt devices fcp disk array 2 gt amp 1 more
195. yed examples of portions of program code similar to what you might write as a programmer messages from the system or information you should actually type ISO 9000 registered quality systems were used in the development and manufacturing of this product Online Publications RS 6000 and pSeries publications are available online To access the online books visit http www rs6000 ibm com resource hardware_docs our Web site at Related Information The following publications provide related information for Fibre Channel configurations and products xii IBM RS 6000 SP Planning Volume 1 Hardware and Physical Envronment GA22 7280 PCI Adapter Placement Reference SA38 0538 Diagnostic Information for Multiple Bus Systems SA38 0509 System Unit Safety Information SA23 2652 Link Fault Isolation SY22 9533 S 390 Fiber Optic Links ESCON FICON Coupling Links and Open system Adapters SY27 2597 PSSP Administration Guide GC23 3897 PSSP Command and Technical Reference GC23 3900 ED 5000 Director Technical Publications CD ROM P N 622 000001 300 ED 6064 Director Technical Publications CD ROM P N 622 000004 100 ES 3016 Switch Technical Publications CD ROM P N 622 000005 100 ES 3032 Switch Technical Publications CD ROM P N 622 000006 000 ES 1000 Switch Technical Publications CD ROM P N 622 000003 100 IBM SAN Fibre Channel Switch 2109 Model S08 User s Guide SC26 7349 IBM SAN Fibre Channel Switch 2109 Model S16 User s Gu

IBM Fibre Channel Planning and Integration: User`s Guide and Service

Contents

Download Pdf Manuals

Related Search

Related Contents