Home

Downloading - ps-2.kev009.com, an archive of old documentation

1. 59 3 1 Overview of the Sun system ee 60 3 2 Sun supported Fibre Channel host adapters 62 3 2 1 Tuning HBA configuration files a 65 3 3 Sun supported Fibre Channel switches and hubs 69 3 3 1 IBM 2109s xx akala ke hunt iste aie a AN daan TEE nG 69 3 3 2 INRANGE FC 9000 2 eee 71 3 3 37MGDATA nanana oh tedden hdres ee a al hintaan 72 3 3 4 Fibre Channel switch installation preparation 74 3 4 Supported storage on the Sun fabric 20000 0c eee eee 74 3 5 Installation planning 0 eee 75 3 5 1 Cabling requirements and considerations 76 3 6 Installation procedure cee eae 76 3 6 1 Check that the system is at a proper level 77 3 6 2 Installing the FC HBA adapter card a eae 77 3 6 3 SWITCH SETUP ise 4s na LANA kaa AIN aii hades Sok 77 3 6 4 Storage setup eens 80 3 6 5 Downloading the Fibre Channel adapter drivers 81 3 6 6 Installing the Fibre Channel adapter driver s 83 3 6 7 Configuring the host device and adapter drivers 86 3 7 Device functionality checks eae 90 3 8 Specific limitations eee 91 Chapter 4 HP 9000 Fibre Channel implementation 93 4 1 lMtrOGUCTION aee suse NGA Ga NEGRA OA aise Sera YNA PANREGLA ATA 94 4 2 Fibre Channel host bus adapters
2. 0 0 cee eee eee 94 402A GAP EG RIBAS AA dais it oo ese Sie the Ba Ges et ele ek ek 94 iv Introducing hosts to the SAN fabric 4 2 2 0therEGHBAS wat pa hn BUDRALNLODY DR RANI Eades 95 4 3 SANAADMNCS a paa Ga eee See ee le ee eee LUN See ea 8 95 4 3 1 General information 0 0 0 0 e ee ee 96 4 3 2 The IBM 2109 family ee ee 97 43 3 INRANGE 0 222 2 3nd Re a BENDA DRA op ne ee ea 97 4 374 MGDATA apa naan Se pA RANT SESE Siete eee Minalin na 98 4 4 SOAGE on Se ad SAna NLA Gee wie tn KAG eels Ae NANA 99 4 5 Installation procedures AA 100 4 5 1 Preparing to install the FC HBA 0 0 100 4 5 2 Installing the FC HBA 0 101 4 5 3 Installing the storage system 0 eee 103 Chapter 5 NUMA Q Fibre Channel implementation 107 5 1 Overview of the NUMA Q system eee 108 5 2 Fibre Channel host bus adapters e eee eee 109 5 2 1 Host adapter placement eee eee 111 5 2 2 Host adapter installation preparation 112 5 3 Fibre Channel switch support 000 e eee eee 112 5 3 1 Fibre Channel switch installation preparation 113 5 4 Supported storage on the NUMA Q fabric 0 005 114 5 5 Installation planning isaurene naria eee 116 5 6 Installation procedure sossar romerna aia a tee 117 5 6 1 Verifying the correct software is installed 117 5 6 2 FO AB
3. Filesets processed 4 of 6 Total time 13 secs installp APPLYING software for printers msg en US rte 4 3 3 6 installp Please mount volume 3 on dey cdb Press the enter key to continue If volume 3 is not available you can enter q to terminate this process BOTTON Figure 2 7 Install additional device software mount additional volume When the status changes from RUNNING to OK scroll to the bottom of the screen as shown in Figure 2 8 to check that installation ended in SUCCESS a Telnet sicily Connect Edit Terminal Help COMMAND STATUS Command J stdout yes stderr no Before command completion additional instructions may appear below MORE 154 devices pci df1066F7 com 4 3 3 USR APPLY SUCCESS devices pci df1066f7 rte 4 3 3 ROOT APPLY SUCCESS devices pci df1066F7 com 4 3 3 ROOT APPLY SUCCESS devices common IBM fc rte 4 3 3 USR APPLY SUCCESS 4 3 3 USR APPLY SUCCESS 4 3 3 USR APPLY SUCCESS printers msg en_US rte X11 msg en_US usm rte installp xx ATTENTION Software changes processed during this session require this system and any of its diskless dataless clients to be rebooted in order for the changes to be made effective F2 Refresh F3 Cancel F6 Command F9 She11 F16 Exit Find Figure 2 8 Install additional device software results Introducing hosts to the SAN fabric m Remove the installation media if you are installing from CD ROM Press F10 to exit SMIT
4. 0 0 2 0 0 ee 13 1 5 5 Installing Emulex Fibre Channel HBAs in Windows 2000 16 1 5 6 Installing Emulex in Windows NT 00 eee eee eee 17 1 5 7 Emulex configuration utiliy 0 0 0 0 eee eee 18 1 5 8 Installing JNI Fibre Channel host bus adapters in Windows 2000 21 1 5 9 Installing JNI Fibre Channel host bus adapters in Windows NT 22 1 5 10 JNI configuration utility 0 2 0 eee eee 22 1 6 Installing the storage system 0 0 0 23 Copyright IBM Corp 2002 iii Chapter 2 eServer pSeries Fibre Channel implementation 33 2 4 Introdut ria ead eae dob aid Se dd ddd eet el as 34 2 2 Fibre Channel host bus adapters 2 00 cee eee 34 2 2 1 IBM FG HBAS iocari saa ana reek na ne Dee Miele aed does 34 222 Other FG HABAS 3 wc ana be bak aed Ge dae ee eed 35 2 3 SAN TADICI nas GANG ete need BAD NG PAGG ea 36 2 3 1 General information 0 0 0 e ee eee 36 2 3 2 The IBM 2109 family 0 2 0 0 eee 37 2 3 3 INRANGE cereti iatan tre Se Bat at de hn en dee lsat ee 38 2 34 MCDATA paa eed ee ee a he eae eat aoe eae 38 2 4 SlOlAGG ss ois nG Aa EAR Mead NAA a st EA eae 40 2 5 Installation procedures 0 0 cece eee 40 2 5 1 Preparations to install the FC HBAS 0 aa 40 2 5 2 Installing the FC HBAS socso enire cikan asea ee 42 2 5 3 Installing the storage systeM 0 cee eee 53 Chapter 3 Sun Fibre Channel implementation
5. 4 Update the drivers you just installed to the latest versions a Start SMIT by typing smitty update all You will see the screen shown in Figure 2 9 F Telnet sicily Connect Edit Terminal Help Update Installed Software to Latest Level Update A11 Type or select a value for the entry field Press Enter AFTER making all desired changes Entry Fields LAINPUT device directory for software F2 Refresh F3 Cancel F4 List F6 Command F7 Edit F8 Image F16 Exit Enter Do Figure 2 9 Update device software install medium b Type in your update directory or device insta11 aix43309 for example and press Enter You see a screen like the one in Figure 2 10 Chapter 2 eServer pSeries Fibre Channel implementation 47 of Telnet sicily Connect Edit Terminal Help Update Installed Software to Latest Level Update A11 Type or select values in entry fields Press Enter AFTER making all desired changes Entry Fields INPUT device directory for software instal1 aix43369 SOFTWARE to update update all PREVIEW only update operation will NOT occur Bo COMMIT software updates yes SAVE replaced files no AUTOMATICALLY install requisite software yes EXTEND file systems if space needed yes VERIFY install and check file sizes no DETAILED output no Process multiple volumes yes e te eee ee F2 Refresh F3 Cancel F4 List F6 Command F7 Edit F8 Image F16 Exit Enter Do Figure 2 10 Update device softwa
6. 7 You may now use the newly added volume and create new partitions on it 32 Introducing hosts to the SAN fabric eServer pSeries Fibre Channel implementation In this chapter we discuss how to successfully connect to a SAN fabric for the IBM eServer pSeries RS 6000 and RS 6000 SP platforms We cover the step by step procedure from start to finish and the supported products which work with the eServer pSeries systems We also point out where the latest firmware and device drivers can be obtained The following topics are discussed Introduction on page 34 Fibre Channel host bus adapters on page 34 SAN fabrics on page 36 Storage on page 40 Installation procedures on page 40 YYYY Y Copyright IBM Corp 2002 33 2 1 Introduction IBM AIX is the primary operating system used on the IBM eServer pSeries and RS 6000 systems There are two versions of AIX currently available AIX V4 3 3 and AIX 5L V5 1 Fibre Channel operations are generally the same on these operating systems and we discuss any differences in their respective sections throughout this chapter The procedures shown in this chapter are also valid for attaching the nodes in an IBM RS 6000 SP system to a Fibre Channel network In this chapter we only cover a single path environment where each fibre device can be accessed via only one path If you have a multi path environment you need to use a storage system specific multi path s
7. Machine Name RFC Type Local 7 DCE Windows NT 7 Transport Type Connection oriented TCP over IP bd Remote Connection requires that the OLview Agent be started on the remote machine Clear Name List Map Network Drive Cancel Figure 1 4 QLview for Fibre utility Set Up Connection screen Chapter 1 Windows Fibre Channel implementation 13 The Map Network Drive button lists the name of all computers currently on the network to assist you in finding the names of the machines In order to get more information related to QLogic Fibre Channel adapter card click Connect as shown in Figure 1 5 QLview for Fibre 10 x File Machine Connection Options amp Help a Local Windows E OLA2200 21 00 00 E0 8B 01 90 EF Machine Local Connect to New Machine Close Figure 1 5 QLview for Fibre utility Main Window 14 Introducing hosts to the SAN fabric We can assume that the QLogic Fibre Channel HBA is successfully installed and connected to the SAN switch or direct to a storage device unless you get the screen shown in Figure 1 6 QLview for Fibre NG Bo 10 x File Machine Connection Options amp Help a Local Windows EE oLA2200 21 00 00 E0 88 01 90 EF Machine Local Connect to New Machine Close Figure 1 6 QLview for Fibre utility In this case the Fibre Channel HBAs is not connected to any storage device or SAN switc
8. scsi2 QLogic QLA2200 PCI to Fibre Channel Host Adapter bus 1 device 4 irg 11 Firmware version 2 01 27 Driver version 4 27b scsi 2 LOOP UP detected scsi 2 Waiting for LIP to complete scsi2 Topology F Port Host Loop address Oxffff scsi 2 Waiting for LIP to complete scsi2 Topology F Port Host Loop address Oxffff root clyde root 15 You can also display information about the FC HBA from proc filesystem as in Example 7 2 Chapter 7 Linux on i386 platform Fibre Channel implementation 149 150 Example 7 2 Checking the QLogic adapter via proc interface root clyde root cat proc scsi qla2x00 2 QLogic PCI to Fibre Channel Host Adapter for ISP2100 ISP2200 ISP2300 Firmware version 2 01 27 Driver version 4 27b Request Queue 0x370f8000 Response Queue 0x37100000 Request Queue count 256 Response Queue count 256 Number of pending commands 0x0 Number of queued commands 0x0 Number of free request entries 253 Number of mailbox timeouts 1 Number of ISP aborts 1 Number of loop resyncs 1 Number of retries for empty slots 0 Number of reqs in retry_q 0 Number of reqs in done_q 0 SCSI Device Information scsi qla0 adapter node 200000e08b0190ef scsi qla0 adapter port 210000e08b0190ef scsi qla0 target 0 50001fe1000ae400 scsi qla0 target 1 50001fe1000ae400 SCSI LUN Information Id Lun root clyde root You can see the WWN of your adapter as the paramete
9. 10 Check the firmware level of your adapter as shown in Example 2 3 Example 2 3 Checking the firmware level Iscfg vl fcs0 fcs0 3A 08 FC Adapter Part Number cc 09P1162 EC LEVEN ic na AA D Serial Number c KT05110105 Manufacturer 0010 FRU NUMDET na waaa 09P1173 Network Address 10000000C9258E54 ROS Level and ID 02903290 Device Specific Z0 4002206D Device Specific Z1 10020193 Device Specific Z2 3001506D Device Specific Z3 02000909 Device Specific Z4 FF101450 Device Specific Z5 02903290 Device Specific Z6 06113290 Device Specific Z7 07113290 Device Specific Z8 2000000009258E54 Device Specific 79 SS3 22A0 Device Specific ZA S1F3 22A0 Introducing hosts to the SAN fabric Device Specific ZB S2F3 22A0 Device Specific YL P1 19 Q1 You can find the firmware level as the value of the ROS Level and ID parameter You can also find the World Wide Name WWN of the adapter as the value of the Network Address parameter You need to know the WWN to configure your storage system 11 1f the firmware level is lower than the latest one upgrade it according to the instructions provided with the firmware package 12 Check that the Fibre Channel adapter is using the correct protocol al or switch as shown in Example 2 4 Example 2 4 Chec
10. 2 Load the Emulex driver root clyde root modprobe 1pfcdd root clyde root Introducing hosts to the SAN fabric 3 Check the kernel messages with the dmesg command root clyde root dmesg Emulex LightPulse FC SCSI IP 4 1le 1pfc 2080374785 045 Vital Product Data Data 82 23 0 36 pfc 2080374785 031 Link Up Event received Data 1110 scsi2 Emulex LPFC DRAGONFLY SCSI on PCI bus 01 device 20 irq 11 Vendor DEC Model HSG80 Rev V85S Type Direct Access ANSI SCSI revision 02 Attached scsi disk sdb at scsi2 channel 0 id O lun 0 SCSI device sdb 17418235 512 byte hdwr sectors 8918 MB sdb root clyde root Note that now we also see the ID of our disk sdb If you already had partitions on the disk they would also be shown 4 Start using the new LUN like any normal SCSI disk We tested the Fibre Channel drive according to the procedure in Example 7 9 Example 7 9 Testing a Fibre Channel disk with Emulex adapter root clyde root fdisk dev sdb Device contains neither a valid DOS partition table nor Sun SGI or OSF disklabel Building a new DOS disklabel Changes will remain in memory only until you decide to write them After that of course the previous content won t be recoverable The number of cylinders for this disk is set to 8504 There is nothing wrong with that but this is larger than 1024 and could in certain setups cause problems with 1 software that runs at boot time e g old versions
11. HP 9000 93 95 99 Hp ux 94 100 l i386 139 IEEE address 159 independently configured 9 infodev 80 INRANGE 6 7 38 71 96 97 102 142 144 150 159 INRANGE FC 9000 8 98 INRANGE FC 9000 40U 144 INRANGE FC 9000 40U cabinet 98 Intel 2 Intel based 140 interactions 7 97 130 143 inter switch links 6 96 143 IN VSN Enterprise Manager 26 102 159 lO cards 108 IP support 4 ISL 6 172 J JNI 2 6 11 22 35 62 81 95 141 JNI 1160 21 JNI driver 81 JNI drivers 142 JNI FC64 1063 63 66 JNI FCE2 6410 95 141 JNI FCE2 6412 95 141 JNI FCE 3210 141 JNI FCE 6410 95 JNI FCE 6460 64 66 77 JNI FCI 1063 63 66 K Kernel 147 155 kernel 108 125 141 142 147 kernel messages 149 152 157 161 kernel source files 147 kernel headers 147 155 kernel source 147 155 KGPSA BC 132 KGPSA CA 132 L L1000 101 LAN 117 large scale applications 5 LEDs 17 Light Pulse 17 18 LiLO 155 limitations 96 link attenuation 7 37 96 130 143 Linux 34 139 146 Linux kernel 142 loadable kernel 148 loadable kernel modules 156 login 136 longwave 6 96 98 112 130 131 143 loop attached devices 73 loop switch 9 loop attached 97 131 LP6000 111 LP7000E 110 LP8000 4 17 62 67 155 LP9000 4 17 LTO 3584 10 LUN 124 125 153 161 LUN masking 7 97 130 143 LUN s 119 LUNs 122 M Magstar 3590 drives 10 mapping storage 23 McDATA 6 8 9 38 72 96 98 102 142 145 150 159 Mc
12. 0 0 0 a 58 Checking the FC HBA 00 cece ete eee 101 Checking the FC HBA parameters 002 cee eee eee 102 Checking the FC HBA connectivity to the switch 103 Creating volume group and filesystem on the new disk 105 Result of set mode diag and wwidmgr show adapter commands 135 Formatting and initializing FC HBA 0 00002 e ee eae 136 Switch connectivity check cee ee 137 Storage connectivity check 0 0 0 eee eee es 138 Decimal volume number for an ESS volume 138 Verifying disk attachment by using hwmgr command 138 Checking the kernel messages from loading of QLogic driver 149 Checking the QLogic adapter via proc interface 150 Checking QLogic adapter switch connectivity 150 Checking the QLogic adapter again via proc interface 152 Checking the kernel messages 0 0 152 Testing a Fibre Channel disk with QLogic adapter 153 Checking the Emulex adapter with the Iputil command 158 Checking the Emulex adapter switch connectivity 159 Testing a Fibre Channel disk with Emulex adapter 161 xi xii Introducing hosts to the SAN fabric Special notices References in this publication to IBM products programs or services do not imply that IBM intends to make these available in all countries in which IBM oper
13. 0x00 Ox04 Ox0C 0x08 0x90 0x00 0x00 OxFD 901004 Ox00000000 OxFD 902004 Chapter 1 Windows Fibre Channel implementation 19 It is recommended to use Light Pulse in order to download the latest version of adapter Firmware Select Firmware Maintenance under the Category list and click the Download button as shown in Figure 1 11 Note Do not upgrade the firmware if it is not suggested by the vendor Check the vendor site for the latest news related to the firmware level Read carefully all instructions before any action is taken LightPulse Utility NT 10 x File Edit Yiew Adapter Help E Adapter O LP8K Category Firmware Maintenance ha TestProgram 1 10 NLPort Loop t W Functional 3 8221 DS3 8241 C Flash ROM Map Y SLI 1 Overl 3 8221 D1D3 8241 RAM Map W SLI 2 Overl 3 82a1 D2D3 8241 Download Delete Figure 1 11 LightPulse Utility Firmware download At this point enter the appropriate firmware destination path and click Open The new software is transferred to the host bus adapter In order to verify that the new firmware level is active expand the Category list and select Adapter Revision Levels as shown in Figure 1 9 on page 19 20 Introducing hosts to the SAN fabric 1 5 8 Installing JNI Fibre Channel host bus adapters in Windows 2000 Microsoft Windows 2000 has a default built in driver the JN 1160 Fibre Channel Adapter which supports FCE 3210 6410 and FCE2 6410 6412 Fibr
14. 1 1 Fibre Channel host bus adapter introduction In the topics that follow we introduce Windows NT and Windows 2000 supported Fibre Channel host bus adapter cards These are manufactured by QLogic Emulex and JNI We also cover their implementation information with respect to the latest drivers and firmwares levels We also detail supported SAN switches and the considerations which should be taken into account in order to establish the connection with Intel based servers running Windows NT or 2000 For more general information or technical information on employment and application of these adapters refer to the appropriate Fibre Channel adapter manufacturer 1 2 Windows supported host bus adapters Windows NT and 2000 environments support many Fibre Channel host bus adapters There are many reasons as to why one would be chosen over another depending on price supported topologies brand loyalty existing adapters already installed and so on We do not recommend one type or another In Table 1 1 we show a list of the latest QLogic Emulex and JNI models which are compatible and supported by Windows NT and 2000 Table 1 1 Windows supported Fibre Channel host bus adapters Manufacturer type ao farmer sree jo 2 Introducing hosts to the SAN fabric Manufacturer type 1 2 1 QLogic Fibre Channel host bus adapter cards There is a wide portfolio of QLogic Fibre Channel host bus adapter cards supported by Windows NT and 2000
15. 5 10 30 60 80 220R 250 420R and 450 with Sbus JNI FC64 1063 and JNI 1963 PCI adapters Additionally there is a JNI 1460 PCI adapter currently under test for PCI use with these platforms gt Sun E3000 6500 Servers with PCI adapters gt Sun E3000 10000 Servers with Sbus adapters The following is a list of Sun operating systems which support Fibre Channel attachment gt Solaris 2 6 also known as SunOS 5 6 gt Solaris 7 also known as SunOS 5 7 gt Solaris 8 also known as SunOS 5 8 Sun supports fabric environments that are both zoned and cascaded For details on these capabilities contact your Sun representative Sun supports the IBM Subsystem Device Driver SDD multipathing driver product for the IBM TotalStorage Enterprise Storage Server ESS or you may use Veritas DMP The IBM SDD comes in two separate varieties for Sun gt Sun host systems which are 32 bit mode and or running Solaris 2 6 gt Sun host systems which are 64 bit mode and running Solaris 7 or 8 Introducing hosts to the SAN fabric With some other fibre devices Sun platforms use Sun s Alternate Pathing AP to provide failover paths to its storage devices Sun has exercised a great deal of partnership to develop a large variety of vendor choices for customers in FC HBAs switches hubs and storage devices which are supported in their SAN environments We cover the ones tested and proofed in IBM environments in detail Required kernel p
16. Checking for previously installed SNIA etc hba conf 85 Chapter 3 Sun Fibre Channel implementation Existing hba conf found Update in progress Existing hba conf saved as hba conf bak 1204 080938 hba conf updated hba conf ready Installation of lt JNIsnia gt was successful Note With some drivers you will be prompted to rebuild the manpages to add the drivers manpage s to the system library This rebuild can take up to as much as ten minutes If you do not want to do this now type n You can run this command later 6 To check that the installation of the driver was truly successful we enter the following command Substitute your drivers name for jni as appropriate pkginfo grep i jni system JNIC146x JNI Fibre Channel SCSI HBA Driver application JNIsnia JNI SNIA Fibre Channel HBA LIBRARY Solaris Before we reboot the system we must configure the necessary parameters for our adapter It is strongly recommended that you do this for all adapters prior to restarting the system 3 6 7 Configuring the host device and adapter drivers 86 At this point we needed to make the necessary changes to the device driver configuration file and the FC HBA driver configuration file to support the adapter as desired and to allow access to the devices To do this we followed these steps 1 Change to the directory by typing cd kernel drv 2 Back up the sd conf file in this subdirectory
17. FCLC Fibre Channel Loop Association an independent working group of the Fibre Channel Association focused on the marketing aspects of the Fibre Channel Loop technology FCP Fibre Channel Protocol the mapping of SCSI 3 operations to Fibre Channel Fiber Optic Refers to the medium and the technology associated with the transmission of information along a glass or plastic wire or fiber Fibre Channel A technology for transmitting data between computer devices at a data rate of up to 4 Gb s It is especially suited for connecting computer servers to shared storage devices and for interconnecting storage controllers and drives FICON Fibre Connection A next generation I O solution for IBM S 390 parallel enterprise server FL_Port Fabric Loop Port the access point of the fabric for physically connecting the user s Node Loop Port NL Port FLOGI See Fabric Log In Frame A linear set of transmitted bits that define the basic transport unit The frame is the most basic element of a message in Fibre Channel communications consisting of a 24 byte header and zero to 2112 bytes of data See also Sequence 166 Introducing hosts to the SAN fabric FSP Fibre Channel Service Protocol The common FC 4 level protocol for all services transparent to the fabric type or topology Full Duplex A mode of communications allowing simultaneous transmission and reception of frames G Port Generic Port a generic switch port that is e
18. In this section we cover the SANblade 2200 shown in Figure 1 1 and the SANblade 2300 family shown in Figure 1 2 Figure 1 1 QLogic SANblade 2200 Series Chapter 1 Windows Fibre Channel implementation 3 nui hb oe a 5 Fri Figure 1 2 QLogic SANblade 2300 Series The following Web site offers more technical information including the latest driver version and whether it is Microsoft certified for use as a storage adapter and cluster device in the Windows NT and Windows 2000 environments http www qlogic com These are some of the supported features that are common for all QLogic FC host bus adapter cards mentioned in Table 1 1 on page 2 Support of F_Port FL_Port and N_Port configurations Extended LUN support IP support FC Tape support Management Server interface support YYYY Y 1 2 2 Emulex Fibre Channel host bus adapter cards Two of the Windows NT and 2000 supported FC host bus adapter cards that belong to the Emulex family are the LP8000 and LP9000 We show the LP9000 in Figure 1 3 4 Introducing hosts to the SAN fabric Figure 1 3 Emulex LP9000 Both are designed for large scale applications The basic technical specifications for the Windows supported models can be found in Table 1 1 on page 2 For more detail we suggest that you visit the Emulex Web site in order to download the latest firmware and driver level for each adapter The URL is http ww
19. as it will be needed to be able to access it using the remote telnet login to complete the configuration as needed This IP address can also be used to access the switch via the Web based GUI session Now we telnet to the switch with the IP address and login using the default login and password login admin password password telnet 9 43 231 91 Trying Connected to 9 43 231 91 Escape character is Fabric OS tm Release a2 4 1c login osl Password osv12109a 0s15 We confirmed these settings by running the following configShow command osv12109a 0s15 configShow Ethernet addresses 0 60 69 12 3f fc Nvram datadata fei 0 0 host usr switch firmware e 9 43 227 19 fffffc00 g 9 43 224 1 u user tn osv12109a diag postDisable 0 fabric domain 1 fabric ops BBCredit 16 fabric ops E_D_TOV 2000 fabric ops R_A_TOV 10000 fabric ops dataFieldSize 2112 fabric ops mode SeqSwitching 0 fabric ops mode fcpProbeDisable 0 fabric ops mode isolate fabric ops mode longDistance fabric ops mode noClassF fabric ops mode pidFormat n n n n n DO O o fabric ops mode sync 0 fabric ops mode tachyonCompat fabric ops mode unicast0nly fabric ops mode useCsCt1 fabric ops mode vcEncode fabric ops vc class 2 2 fabric ops vc class 3 3 fabric ops vc config Oxc0 fabric ops vc linkCtr1 0 fabric ops vc multicast 7 fc4 fcIp address 0 0 0 0 fc4 fcIp mask 0 0 0 0 fcAL fanFrameDisable 0 DO o o Intro
20. cp sd conf sd orig conf 3 Edit the sd conf file to add support for the target and LUN pairs that are configured on the host system The following is what we added for our example vi sd conf sd conf Read only 587 lines 21887 characters Copyright c 1992 by Sun Microsystems Inc ident sd conf1 893 05 03 SMI name sd class scsi class_prop atapi target 0 lun 0 name sd class scsi class_prop atapi Introducing hosts to the SAN fabric target 1 lun 0 name sd class scsi target 2 lun 0 name sd class scsi target 3 lun 0 name sd class scsi target 4 lun 0 name sd class scsi target 5 lun 0 name sd class scsi target 6 lun 0 name sd class scsi target 8 lun 0 name sd class scsi target 9 lun 0 name sd class scsi target 10 lun 0 name sd class scsi target 11 lun 0 name sd class scsi target 12 lun 0 name sd class scsi target 13 lun 0 name sd class scsi target 14 lun 0 name sd class scsi target 15 lun 0 class_prop atapi class_prop atapi Lines added for ESS on JNI 6460 controller name sd class scsi target 0 lun 1 name sd class scsi target 0 lun 2 name sd class scsi target 0 lun 3 name sd class scsi target 0 lun 4 name sd class scsi target 0 lun 5 name sd class scsi target 0 lun 6 name sd class scsi target 0 lun 7 name sd class scsi target 0 lun 8 iwq Note Do not add du
21. port port port port port port port id port sw port 9 port 10 sw port 11 sw port 12 sw port 13 id port 14 id port 15 id id id id id ON QM O Bb WN H On On On On On O o o On On O_ o itsosw3 2 4 Online Principal 3 fffc03 10 00 00 60 69 10 OFF ine F Port Module ine E Port Module Light ine F Port ine E Port Light ine F Port Module Light Light ine L Port Light ine F Port Light itsosw3 admin gt exit Connection closed by foreign host b4 cf 21 00 00 10 00 00 10 00 00 10 00 00 10 00 00 1 public 10 00 00 e0 60 00 60 00 00 8b 02 69 20 c9 23 69 20 c9 22 c9 25 a7 2d ld 4e itsosw1 downstream c3 05 ld 4e itsoswl bf 75 8e 54 The online status of the specific port fabric attachment F Port and FC HBA WWN next to it are the confirmation that the FC HBA has performed a fabric login to the switch An alternative way to access the switch is by using a Web browser such as Internet Explorer Just type the switch IP address and the switch main menu appears as shown in Figure 1 20 Introducing hosts to the SAN fabric E Fabric View Microsoft Internet Explorer File Edit View Favorites Tools Help le 2 2A 8 a a 9 8 gm Back Forward Stop Refresh Home Search Favorites History Mail Print Edit Discuss Address hitp 3 1 38 158
22. root root 3205 2001 04 17 12 43 07 Makefile rw r r root root 10887 2001 04 23 19 31 55 README qla2x00 rw r r root root 25469 2001 03 09 15 58 43 exioct h rw r r root root 4149 2001 03 09 15 58 43 exioctIn h rw r r root root 306034 2001 03 09 15 58 43 g12100 fw h rw r r root root 327593 2001 03 09 15 58 43 q12200_fw h rw r r root root 338250 2001 03 09 15 58 43 q12300_fw h rw r r root root 524896 2001 04 16 18 40 30 qla2x00 c rw r r root root 87171 2001 04 16 16 56 47 qla2x00 h rw r r root root 49375 2001 03 09 15 58 43 qla2x00ioctl c rw r r root root 16993 2001 03 09 15 58 43 qlavendor c rw r r root root 3320 2001 04 23 19 31 56 release txt root clyde qla2x00 Build the qla2x00 0 loadable kernel module root clyde qla2x00 make OSVER 1inux 2 4 cc D KERNEL DMODULE Wall 02 DISP2200 DMODVERSIONS include usr src linux 2 4 include 1inux modversions h I usr src linux 2 4 include I usr src linux 2 4 include drivers scsi Wall Wstrict prototypes fomit frame pointer fno strength reduce pipe m486 malign loops 2 malign jumps 2 malign functions 2 DCONFIG_X86_LOCAL_APIC c 0 qla2x00 0 qla2x00 c root clyde qla2x00 Note This procedure assumes that you are using a single processor system Copy the new module into your modules directory root clyde qla2x00 cp qla2x00 0 lib modules 2 4 2 2 kernel drivers scsi cp overwrite lib modules 2 4 2 2 kernel drivers scsi qla2x00 0
23. see BM SAN Survival Guide Featuring the IBM 2109 SG24 6127 7 3 3 INRANGE The INRANGE switches currently supported by IBM the IBM 2042 family are described in Table 7 4 Table 7 4 The INRANGE switches wast Tons Speed Gomez Rookanoce_ 2042 001 INRANGE FC 9000 64 1 Gb s SC 20U Fibre Channel Director 2042 128 INRANGE 1 Gb s FC 9000 128 Fibre Channel Director In addition IBM also sells the INRANGE FC 9000 40U cabinet 2042 C40 The cabinet is able to hold one 128 port switch or two 64 port switches Features and limitations The INRANGE FC 9000 family of switches support both fabric attached and loop attached devices on any port 144 Introducing hosts to the SAN fabric Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on your requirements For more information see IBM SAN Survival Guide Featuring the INRANGE Portfolio SG24 6150 7 3 4 McDATA The McDATA switches currently supported by IBM the IBM 2031 and 2032 families are described in Table 7 5 Table 7 5 The McDATA switches wos Pore peon Connectors Rarspace 2031 016 McDATA ES 3016 16 1 Gb s LC 1U Fabric Switch 2031 032 McDATA ES 3032 32 1 Gb s LC 1 5U Fabric Switch 2031 L00 McDATA ES 1000 8 1 1 Gb s SC 1U Loop Switch 2032 001 McDATA ED 5000 32 1 Gb s SC 18U Enterprise Fibre Channel Director 2032 064 McDATA ED 6064 1 Gb s Enterprise Fibre Channel Director In add
24. 110 Emulex LP6000 Fibre Channel Adapter 2000 111 IBM 2109 S08 top and S16 0002s 113 Compaq StorageWorks 64 Bit 33MHz PCI to FC HBA 129 64 Bit PCI Fibre Channel Host Adapter 134 Introducing hosts to the SAN fabric Tables 1 1 Windows supported Fibre Channel host bus adapters 2 1 2 Common Fibre Channel cabling configurations 6 1 3 The IBM 2109 family wi menpa an EE eee 7 1 4 The INRANGE switches 0 c eee eae 8 1 5 The McDATA switches 0 000 e ee eee 8 1 6 IBM Fibre Channel attached disk storage systems 10 1 7 IBM Fibre Channel attached tape storage systems 10 2 1 IBM FC HBAs for IBM eServer pSeries and RS 6000 34 2 2 Other FC HBAs available for IBM eServer pSeries and RS 6000 35 2 3 Common Fibre Channel cabling configurations 36 2 4 The IBM 2109 family 00 000 eee 37 2 5 The INRANGE switches 00 000 cece eee 38 2 6 The McDATA switches 00 c eee eee 39 2 7 IBM Fibre Channel attached disk storage systems 40 2 8 IBM Fibre Channel attached tape storage systems 40 3 1 Solaris 2 6 7 and 8 minimum patch levels for Fibre Channel 61 3 2 Solaris kernel parameter settings required 65 3 3 Recommended JNI HBA settings 0 00002 eee eee 66 3 4 JNIFCE
25. 7 3 1 General information 0 0 e eee 143 7 3 2 IBM 2109 family e a te 144 TaS INRANGE m00 ba a wb PG dias Regt eRe haahaha 144 T345 MCDA TA ict sails TA tie NAA LAN NG BAN oe ic Saale aes das aes 145 TA SIOlAGS xg Seats ne eee ene siete le Ses eee i oot 146 7 5 Installation procedures 0 0 0 ee 146 7 5 1 Installing a QLogic FC HBA 2 cee ee 147 7 5 2 Configuring storage for the QLogic FC HBA 151 7 5 3 Installing an Emulex FC HBA 0 000 eee eee 155 7 5 4 Configuring storage for the Emulex FC HBA 160 GlOSS aly Na nde Antes ethene oe ek ne wd eas 163 Related publications 0 0 c eee es 175 IBM REdDOOKS a a eed PPA ete ake eae uke Baer ea os 175 Other TESOUICES 2 472 bela PUNA dee had det Bede GEE Pare de 175 Referenced Web sites 0 0 cece eee eee eee 178 How to get IBM Redbooks 0 0 eee ee 180 IBM Redbooks collections 0 00 c eee eee 180 INGO aesir h a Ba neces ote ag av Roasts ANG ie Ie Bate lve at RNGA 181 Introducing hosts to the SAN fabric Figures 1 1 QLogic SANblade 2200 Series a 3 1 2 QLogic SANblade 2300 Series 00 ee 4 1 3 Emulex LP9000 0 es 5 14 QLview for Fibre utility Set Up Connection screen 13 15 QLview for Fibre utility Main Window 0s 14 1 6 QLview for Fibre utility ee 15 1 7 QLview for Fibre utility adapter
26. CPUs and from a CPU to disk arrays and other peripherals HMMP HyperMedia Management Protocol HMMS HyperMedia Management Schema the definition of an implementation independent extensible common data description schema allowing data from a variety of sources to be described and accessed in real time regardless of the source of the data See also WEBM HMMP HSM Hierarchical Storage Management A software and hardware system that moves files from disk to slower less expensive storage media based on rules and observation of file activity Modern HSM systems move files from magnetic disk to optical disk to magnetic tape HUB A Fibre Channel device that connects nodes into a logical loop by using a physical star topology Hubs will automatically recognize an active node and insert the node into the loop A node that fails or is powered off is automatically removed from the loop HUB Topology see Loop Topology Hunt Group A set of associated Node Ports N_Ports attached to a single node assigned a special identifier that allows any frames containing this identifier to be routed to any available Node Port N Port in the set In Band Signaling This is signaling that is carried in the same channel as the information Also referred to as inband Information Unit A unit of information defined by an FC 4 mapping Information Units are transferred as a Fibre Channel Sequence Intermix A mode of service defined by Fibre Channel t
27. Channel attached disk storage systems Storage system Method of attachment Operating systems IBM 3542 3560 FAStT200 Attach by both fibre switch Supported with Solaris Storage Server with up to five or FC AL 2 6 7 and 8 EXP500 disk expansion unit IBM 3552 3560 FAStT500 Attach by both fibre switch Supported with Solaris with up to 22 EXP500 disk or FC AL 2 6 7 and 8 expansion units IBM 1742 3560 Attach by both fibre switch Supported with Solaris FAStT700 with up to 22 or FC AL 2 6 7 and 8 EXP500 disk expansion units IBM 2106 Modular Storage Attach by both fibre switch Supported with Solaris Server MSS or FC AL 2 6 7 and 8 IBM 7133 Serial Disk Attach by both fibre switch Supported with Solaris System with 7140 SAN or FC AL 2 6 7 and 8 controller 160 IBM 2105 Enterprise Storage Attach by both fibre switch Supported with Solaris Server ESS or FC AL 2 6 7 and 8 Table 3 11 IBM Fibre Channel attached tape storage systems Storage system Method of Attachment Operating systems IBM 3584 Ultra Scalable FC AL only Supported with Solaris 2 6 Tape Library Server LTO 7 and 8 IBM 3590E tape drives with FC AL only Supported with Solaris 2 6 direct fibre interface 7 and 8 3 5 Installation planning Before starting the installation it is important to plan what you intend to do This includes deciding what will be connected and where the equipment will be located Th
28. Connect the adapter to the Fibre Channel network 16 Start up the system and log in as root 17 Load the module as follows root clyde root modprobe 1pfcdd root clyde root 18 Check the kernel messages with the dmesg command root clyde root dmesg Emulex LightPulse FC SCSI IP 4 1le 1pfc 1962934273 045 Vital Product Data Data 82 23 0 36 11pfc 1962934273 031 Link Up Event received Data 1110 Chapter 7 Linux on i386 platform Fibre Channel implementation scsi2 Emulex LPFC DRAGONFLY SCSI on PCI bus 01 device 20 irq 11 root clyde root 19 Check the adapter with the Iputil command as in Example 7 7 Example 7 7 Checking the Emulex adapter with the Iputil command root clyde root Iputil LightPulse Common Utility for Linux Version 1 2a22 5 17 2001 Copyright c 2000 Emulex Network Systems Inc Emulex Fibre Channel Host Adapters Detected 1 Host Adapter O Ipfc 1962934273 is an LP8K V2 Ready Mode MAIN MENU List Adapters Adapter Information Firmware Maintenance Reset Adapter BWN rn 0 Exit Enter choice gt 2 ADAPTER INFORMATION MENU PCI Configuration Parameters Adapter Revision Levels Wakeup Parameters IEEE Address Loop Map Status amp Counters Link Status Configuration Parameters ON MT 0 Bp WON H 0 Return to Main Menu Enter choice gt 4 IEEE Address for Adapter 0 10000000 C923C305 ADAPTER INFORMATION MENU PCI Configuration Parameters Adapter Revision Leve
29. FC 4 level operations with the destination port N_Port Login may be either explicit or implicit Name Server Provides translation from a given node name to one or more associated N_Port identifiers NAS Network Attached Storage a term used to describe a technology where an integrated storage system is attached to a messaging network that uses common communications protocols such as TCP IP NDMP Network Data Management Protocol Network An aggregation of interconnected nodes workstations file servers and or peripherals with its own protocol that supports interaction Network Topology Physical arrangement of nodes and interconnecting communications links in networks based on application requirements and geographical distribution of users NFS Network File System A distributed file system in UNIX developed by Sun Microsystems which allows a set of computers to cooperatively access each other s files in a transparent manner NL_Port Node Loop Port a node port that supports Arbitrated Loop devices NMS Network Management System A system responsible for managing at least part of a network NMSs communicate with agents to help keep track of network statistics and resources Node An entity with one or more N_Ports or NL_Ports Non Blocking A term used to indicate that the capabilities of a switch are such that the total number of available transmission paths is equal to the number of ports Therefore all ports ca
30. Go Links il polled at 44 30 04 7 06 PM polled at 44 30 04 7 06 PM a Fabric Topology Name itsosw1 Name itsosw2 Fabric 05 version a2 4 10 Fabric OS version a2 4 10 EE Domain ID 1 Domain ID 2 77 Name Server Bhemet IP 9 1 38 157 Ehemet IP 9 1 38 158 Ethemet Mask 255 255 254 0 Ethemet Mask 255 255 254 0 og Zone Admin F Cnet IP none FCnet IP none C4 FCnet Mask none FCnet Mask none Gateway IP 9 1 38 1 Gateway IP 9 1 38 1 E3 Summary View WN 10 00 00 60 69 20 1d 4e WN 10 00 00 60 69 20 1d 74 Bts Lageng e Gag 8 Healthy i polled at 4113001 7 06 PM Marginal Name itsosw3 Down Fabric OS version 2241c A Domain ID 3 Unmonitored Bhemet IP 9 1 38 159 Ethemet Mask 255 255 254 0 FCnet IP none F Cnet Mask none Gateway IP 9 1 38 1 WN 10 00 00 60 69 10 64 0f Figure 1 12 IBM 2109 Switch configuration main menu Click the switch to which your server is attached The window shown in Figure 1 13 appears Geo C08 KEI eo COCO Geis COS Cy Ce ee EO Ng OO Ng WWN 10 00 00 60 69 10 64 cf EtherIP 9 1 38 159 Domain ID 3 Ether NM 255 255 254 0 Role Principal FC IP none State Online FC NM none Firmware a24 1c Gateway 9 1 38 1 Serial 10 00 00 60 69 10 64 cf Figure 1 13 IBM 2109 516 Switch Management Application Chapter 1 Windows Fibre Channel implementation 25 Click the switch port where the Wind
31. Interface Converters GBIC and three types of fiber optic cable commonly used in Fibre Channel environments as described in Table 2 3 Table 2 3 Common Fibre Channel cabling configurations GBIC type Cable type Distance 1 Gb s Distance 2 Gb s Shortwave GBICs and 50 micron cable are usually used to connect devices to the fabric since most device adapters only support shortwave GBICs as well as for short inter switch links Longwave GBICs and 9 micron cable are used when required for extended distance 36 Introducing hosts to the SAN fabric Attenuation Fibre Channel is a high speed communication technology Because of this it is very sensitive to link attenuation The attenuation is even more of a critical factor when connection speeds rise from 1 Gb s to 2 Gb s and even higher To limit the effects of attenuation we recommend making sure that You have as few connections as possible on a single link All connectors are clean All connectors are properly seated Only one type of fiber is used on a single link never mix 50 micron and 62 5 micron fiber YYY Yy Zoning We strongly recommend that you use zoning to gt Enhance the security of your Fibre Channel fabric gt Avoid interactions between different hosts gt Secure storage devices that do not support LUN masking like tape drives Since the implementation of zoning and even the terminology used varies between switch manufacturers we do not discuss
32. ML were found Install the base level of device drivers for your FC HBAs a Insert the install media into the CD ROM drive if you are installing from CD ROM b Start SMIT by typing smitty devinst You get the screen shown in Figure 2 2 Introducing hosts to the SAN fabric of Telnet sicily Connect Edit Terminal Help Install Additional Device Software Type or select a value for the entry field Press Enter AFTER making all desired changes Entry Fields LDU ra directory for software F2 Refresh F3 Cancel F4 List Esc 6 Command Esc 7 Edit Esc 8 Image Esc 6 Exit Enter Do Figure 2 2 Install additional device software select install medium c Type in your install device or directory dev cd0 for example and press Enter The next screen is shown in Figure 2 3 F Telnet sicily Connect Edit Terminal Help Install Additional Device Software Type or select values in entry fields Press Enter AFTER making all desired changes Entry Fields INPUT device directory for software dev cd6 SUT SE eel Mevices PREUIEW only install operation will NOT occur no Include corresponding LANGUAGE filesets yes F1 Help F2 Refresh F3 Cancel F4 List Esc 5 Reset Esc 6 Command Esc 7 Edit Esc 8 Image Esc 9 Shell Esc Exit Enter Do Figure 2 3 Install additional device software initial settings d Press F4 to get a list of the software packages Chapter 2 eServer pSeries Fibre Channel implementat
33. Manager application to find the same information For McDATA switches you can find the information in the EFC Manager application 26 Introducing hosts to the SAN fabric Windows 2000 environment As soon as you have connected your server to the storage device via a switch defined and allocated a storage system and created adequate switch zones Windows recognizes the newly added storage device You have to reboot your Windows 2000 or Windows NT in order to use the new storage device Immediately after rebooting follow these steps 1 Select Start Program Administrative Tool and Computer Management 2 Under Storage tree click Disk Management folder 3 Write Signature and Upgrade Disk Wizard window appears as shown in Figure 1 15 Write Signature and Upgrade Disk Wizard Welcome to the Write Signature and Upgrade Disk Wizard This wizard writes signatures on new disks and upgrades empty basic disks to dynamic disks You can use dynamic disks to create software based RAID volumes These volumes can be mirrored as well as striped or spanned across disks You can also expand single disk and spanned volumes without having to restart the computer After you upgrade a disk you cannot use earlier versions of Windows on any volume on that disk To continue click Next Do not show this wizard again T Next gt Cancel Figure 1 15 Write Signature and Upgrade Disk Wizard 4 Click Next C
34. SAN fabric You can find the operating system patches and device drivers required for your adapter at http www software hp com 4 2 2 Other FC HBAs Other FC HBAs available for HP 9000 servers are listed in Table 4 2 These adapters are supported by their manufacturer For more information about these adapters and their limitations see the manufacturer s web site http www emulex com http www jni com All of these adapters are PCl bus adapters and can only be installed on PCl bus based HP 9000 series systems Table 4 2 Other FC HBAs available for HP 9000 systems Pamer Pons comectr speed Woes Pemiertrrooe le Jas EO e e Ea E me vse e Ca es o rare e fe e mee lo fo mres a CC oo 4 3 SAN fabrics The HP 9000 series hosts support two Fibre Channel network topologies gt Fibre Channel Arbitrated Loop FC AL gt Switched fabric Chapter 4 HP 9000 Fibre Channel implementation 95 The HP 9000 series systems have supported FC AL loops for a long time FC AL has however severe limitations in performance and scalability Because of those limitations we only cover switched fabric topology However we also discuss attaching storage devices supporting only FC AL attachment to the fabrics We cover the products from the following three vendors gt IBM 2109 family gt INRANGE gt McDATA 4 3 1 General information There are two types of Gigabit Interface Converters GBIC and three type
35. SCSI 3 proposal that defines a command set for soliciting basic device status temperature fan speed power supply status etc from a storage enclosures Single Mode Fiber In optical fiber technology an optical fiber that is designed for the transmission of a single ray or mode of light as a carrier It is a single light path used for long distance signal transmission See also Multi Mode Fiber SMART Self Monitoring and Reporting Technology SM Single Mode See Single Mode Fiber SMF Single Mode Fiber In optical fiber technology an optical fiber that is designed for the transmission of a single ray or mode of light as a carrier It is a single light path used for long distance signal transmission See also MMF SNIA Storage Networking Industry Association A non profit organization comprised of more than 77 companies and individuals in the storage industry SN Storage Network See also SAN SNMP Simple Network Management Protocol The Internet network management protocol which provides a means to monitor and set network configuration and run time parameters SNMWG Storage Network Management Working Group is chartered to identify define and support open standards needed to address the increased management requirements imposed by storage area network environments SSA Serial Storage Architecture A high speed serial loop based interface developed as a high speed point to point connection for peripherals particula
36. Technical common and that any reader with an intermediate grasp of Support Organization Experts from IBM Customers and Partners from around the world create timely technical information based on realistic SAN and host concepts should be able to apply scenarios Specific recommendations are provided to help you implement IT solutions more effectively in your environment For more information ibm com redbooks SG24 6411 00 ISBN 073842434X
37. The green LED indicates power functions and the yellow LED signifies port activity OF Gd NO HA Normal FC HBA status is indicated by the green LED which blinks slowly The yellow one is OFF since there is no connection to either fabrics or storage devices Chapter 6 Compaq AlphaServer Fibre Channel Implementation 133 Figure 6 2 64 Bit PCI Fibre Channel Host Adapter The Figure 6 2 legend is described in Table 6 4 Table 6 4 64 Bit PCI Fibre Channel HBA description Fibre Channel connector 2 LED Indicates the Embedded Optical Receive port is receiving light POST LEDs indicators 6 Not used 6 From the AlphaServer console type wwidmgr show adapter This is shown in Example 6 1 134 Introducing hosts to the SAN fabric Note On AlphaServer 8x00 4x00 or 1200 models the console must be in diagnostic mode when the wwidmgr command is issued On all other platforms the wwidmgr command can be issued at any time To get into diagnostic mode type set mode diag Example 6 1 Result of set mode diag and wwidmgr show adapter commands P00555set mode diag Console is in diagnostic mode P00 gt gt gt wwidmgr show adapter polling for units on kgpsa0 slot 9 bus 0 hose0 kgpsa0 0 0 9 0 PGAO WWN 2000 0000 c922 69bf polling for units on kgpsal slot 10 bus 0 hosed kgpsa0 0 0 10 0 PGAO WWN 2000 0000 c921 df4b item adapter WWN Cur Topo Next Topo 0 kgpsa0 0 0 10 0 2000 00
38. User Manual 620 000111 ES 3016 Switch Installation and Service Manual 620 000112 ES 3032 Switch Planning Manual 620 0001 18 000 ES 3032 Switch User Manual 620 0001 17 000 ES 3032 Switch Installation and Service Manual 620 0001 16 000 ES 1000 Switch Planning Manual 620 000102 000 ES 1000 Switch User Manual 620 000103 ES 1000 Switch Installation and Service Manual 620 000105 These QLogic publications are also relevant as further information sources gt gt gt gt gt gt gt QLA2200 Hardware Manual FC0151103 00 QLA2200 Hardware Manual FC0151103 00 QLA2100 Software Manual FC0153301 00 QLA2100 Hardware Manual FC0151102 00 QMS V1 Installation Guide FC0051104 00 QLview for Fibre Operations Guide FC0051101 00 QLconfig Operations Guide FC0051 102 00 These Compaq publications are also relevant as further information sources gt gt 64 Bit PCI to Fibre Channel Host Bus Adapter Release Notes for Tru64 UNIX and OpenVMS AV RLLUA TE 64 Bit PCI to Fibre Channel Host Bus Adapter User Guide AA RKPDB TE Related publications 177 Referenced Web sites 178 These Web sites are also relevant as further information sources gt http www storage ibm com ibmsan index htm IBM Enterprise SAN http www pc ibm com ww netfinity san IBM Storage Area Networks Nefinity Servers http www storage ibm com hardsoft products sangateway supserver htm IBM SAN Data Gateway http www stora
39. added should be distributed evenly between the two PCI busses A maximum of four FC HBAs can be installed in a single quad You must not install two FC HBAs to the same fabric in the same quad Chapter 5 NUMA Q Fibre Channel implementation 111 5 2 2 Host adapter installation preparation The firmware to use for the FC HBA is provided on the Dynix ptx Base and Layered Products CD which is provided with the NUMA Q system It is loaded onto the NUMA Q system using the standard layer product installation model defined in the appropriate documentation for the level of operating system you are running The only supported version of the firmware is that provided by NUMA Q as the driver is only proofed with those versions For this reason we recommend that you use these versions only 5 3 Fibre Channel switch support The NUMA Q system is tested and proofed with the switches outlined in Table 5 2 Table 5 2 The NUMA Q supported switch family Speed Connectors Rack space Brocade Silkworm 1000 switch 1 Gb s Brocade Sikworm 2000 oth TO BN TotalStorage SAN Swich S08 8 Tav BM TotalSorage SAN Switch S16 Tbs These are the only switches which are formally supported with the NUMA Q system For host attachment the switch must use the multimode shortwave laser GBICs black in color This allows for distances up to 500 meters For long distances up to 10Km two switches can be used with singlemode longwave GBICs light blue in color con
40. and the following window appears as shown in Figure 1 9 Introducing hosts to the SAN fabric LightPulse Utility NT File Edit View Adapter Help dj ele e E Adapter 0 LP8K Category Adapter Revision Levels hs BIU Revision Sequence Manager Revision Endec Revision Current Operational Firmware Kernel Initial Firmware SLI 1 SLI 2 Max FC PH Version Min FC PH Version IEEE Address Figure 1 9 Light Pulse utility main window 2002506D 00000000 00000000 SLI 2 Overlay LPSK 2 0180 Initial Load 3 8231 LPSK SLI 1 Overlay 3 82a1 LP8K SLI 2 Overlay 3 82a1 43 43 00 00 C9 23 C3 05 By selecting the appropriate adapter and changing the Category section you can obtain much useful information about the driver itself as well as performing other actions on it Scroll up the Category section and select the desired action as shown in Figure 1 10 LightPulse Utility NT E Fie Edit View Adapter Help salas le 2 E Adapter O LP8K Category FCI Registers X Adapter Revision Levels Firmware Maintenance Loop Map Vendor ID PCI Registers Device ID Configuration Data Command Driver Parameters Status Link Statistics Revision Prog If Subclass Base Class Cache Line Size Latency Timer Header Type Built In Self Test Base Address 0 Base Address 1 Base Address 2 D Status and Counters Figure 1 10 LightPulse Utility Category list Ox10DF OxF800 Ox001F 0x0280 0x02
41. before SCSI probes are allowed during startup In Table 3 4 we show the recommended settings for the JNI FCE 6460 Table 3 4 JNI FCE 6460 settings Parameters Recommended FcLoopEnabled 1 Default Initializes on a FC loop 0 Turns off loop protocol for fabric FcFabricEnabled 0 Default Turns off N Port initialization for FC loop 1 For fabric Initializes as an N Port jnic146xn def hba binding null Set to null this is an added parameter that must be added for each adapter in the system where n is the adapter number 66 Introducing hosts to the SAN fabric Parameters Recommended jnic146xn target1_wwnn Set to equal the WWN of the target devices port adapter s that this HBA is able to access where n is the HBAs number in the system jnic146xn targetn lunn hba Set to equal jnic146xn where n is the adapter target lun numbers This entry must be made for each device the adapter is to have access to jnic146xn targetn lunn throttle Equal to 60 where n is the adapter target lun number See Table 3 5 for recommended host bus adapter configuration file parameters for an Emulex LP8000 adapter Table 3 5 Recommended Emulex LP8000 HBA settings Recommended settings 1 Default Automatically assigns SCSI IDs to FCP targets 1 Default Turn on FCP lun queue depth 16 Recommended when there are less then 17 LUNs per adapter Set value 256 Total LUNs per adapter when there are more
42. decide to write them After that of course the previous content won t be recoverable The number of cylinders for this disk is set to 8504 There is nothing wrong with that but this is larger than 1024 and could in certain setups cause problems with 1 software that runs at boot time e g old versions of LILO 2 booting and partitioning software from other OSs e g DOS FDISK OS 2 FDISK Command m for help p Disk dev sdb 64 heads 32 sectors 8504 cylinders Chapter 7 Linux on i386 platform Fibre Channel implementation 153 Units cylinders of 2048 512 bytes Device Boot Start End Blocks Id System Command m for help n Command action e extended p primary partition 1 4 p Partition number 1 4 1 First cylinder 1 8504 default 1 1 Last cylinder or size or sizeM or sizeK 1 8504 default 8504 8504 Command m for help w The partition table has been altered Calling ioct1 to re read partition table WARNING If you have created or modified any DOS 6 x partitions please see the fdisk manual page for additional information Syncing disks root clyde root mkfs dev sdbl mke2fs 1 19 13 Jul 2000 for EXT2 FS 0 5b 95 08 09 Filesystem label OS type Linux Fragment size 4096 log 2 1089152 inodes 2177020 blocks 108851 blocks 5 00 reserved for the super user First data block 0 67 block groups 32768 blocks per group 32768 fragments per group 16256 inodes per group Superblock backups sto
43. drives Since the implementation of zoning and even the terminology used varies between switch manufacturers we do not discuss it 130 Introducing hosts to the SAN fabric 6 3 2 IBM 2109 S08 S16 The current supported models of the IBM 2109 family of switches are described in Table 6 2 Table 6 2 The IBM 2109 family mea Pore Speed coear Rook apace IBM TotalStorage SAN Switch Gi 1 Gb s S08 IBM TotalStorage SAN Switch 1 Gb s S16 Features and limitations The IBM 2109 family of switches support both fabric attached and loop attached devices on any port and switch domain ids 1 239 are supported Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on customer requirements 6 4 Storage The IBM Fibre Channel attached storage systems supported by Compaq AlphaServer running the Tru64 UNIX operating system are listed in Table 6 3 Table 6 3 IBM Fibre Channel attached disk storage systems Storage system HW platforms Operating systems IBM TotalStorage 2106 MSS All AlphaServer models Tru64 UNIX support IBM TotalStorage gt 4 0F 2106 MSS gt 50A Chapter 6 Compaq AlphaServer Fibre Channel Implementation 131 Storage system HW platforms Operating systems IBM TotalStorage 2105 ESS AlphaServer Tru64 UNIX 800 gt 4 0F 4 0G 1200 ASE 1 6 2100 gt 5 0A 4000 4000A TruCluster 5 0A 4100 gt 5 1 8200 8400 TruCluster 5 1 DS1
44. family Speed Connectors Rack space IBM TotalStorage SAN Switch S08 eo 1 Gb s iso fw IBM IBM TotalStorage SAN Switch S16 IBM TotalStorage SAN Switch S16 Switch S16 1 Gb s CU Gb s Features and limitations The IBM 2109 family of switches support both fabric attached and loop attached devices on any port and switch domain IDs 1 239 are supported Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on requirements 1 3 3 INRANGE The INRANGE switches currently sold by IBM the IBM 2042 family are described in Table 1 4 Chapter 1 Windows Fibre Channel implementation 7 Table 1 4 The INRANGE switches Speed Connectors Rack space 2042 001 INRANGE FC 9000 Fibre 64 1 Gb s SC 20U Channel Director 2042 128 INRANGE FC 9000 128 1 Gb s Fibre Channel Director In addition IBM also sells the INRANGE FC 9000 40U cabinet 2042 C40 The cabinet is able to hold one 128 port switch or two 64 port switches Features and limitations The INRANGE FC 9000 family of switches support both fabric attached and loop attached devices on any port Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on requirements 1 3 4 McDATA The McDATA switches currently sold by IBM the IBM 2031 and 2032 families are described in Table 1 5 a cage kad 2031 016 McDATA ES 3016 Fabric 16 1 Gb s LC 1U Swi
45. in their respective sections throughout this chapter The NUMA Q system supports Fibre Channel Point to Point Fibre Channel Arbitrated Loop FC AL or Fibre Channel Switch Switched topologies Only one topology is supported at any one time across the system The topology is selectable via the kernel parameter ff_topology found in the file ustr conf uts io ff ff_space c This parameter must be set for the topology you desire to run For point to point fabric it is recommended that you create a direct FC HBA to storage simple FC AL with only two connections rather than using the point to point topology setting NUMA Q supports fabric levels 1 2 and 3 The NUMA Q system supports class 2 and class 3 Fibre Channel protocol You can also establish a cascaded and zoned fabric environment with the NUMA Q The NUMA Q system does not use the IBM SDD driver product as the Dynix ptx operating systems come with a built in multipath driver This driver will automatically try to use all paths it detects to the fibre devices It also requires that all paths on a common fabric be able to see all of the system s devices equally which are on the fabric For this reason you must take special care when configuring devices on a fabric For additional documentation which would be of great value to you when performing SAN planning and installation on the NUMA Q and product specific implementation documentation for supported direct fibre attached storage
46. of use LIP A Loop Initialization Primitive sequence is a special fibre channel sequence that is used to start loop initialization Allows ports to establish their port addresses Local Area Network LAN A network covering a relatively small geographic area usually not larger than a floor or small building Transmissions within a Local Area Network are mostly digital carrying data among stations at rates usually above one megabit s Login Server Entity within the Fibre Channel fabric that receives and responds to login requests Glossary 167 Loop Circuit A temporary point to point like path that allows bi directional communications between loop capable ports Loop Topology An interconnection structure in which each point has physical links to two neighbors resulting in a closed circuit In a loop topology the available bandwidth is shared LVD Low Voltage Differential Management Agent A process that exchanges a managed node s information with a management station Managed Node A managed node is a computer a storage system a gateway a media device such as a switch or hub a control instrument a software product such as an operating system or an accounting package or a machine ona factory floor such as a robot Managed Object A variable of a managed node This variable contains one piece of information about the node Each node can have several objects Management Station A host system that runs the manageme
47. path redundancy and greater availability If additional FC HBAs were added to the current fabric from this system you must add the WWN to the ESS configuration for these devices Failing to do so would result in the kernel seeing this as an unbalanced fabric access to all the devices would be lost 5 7 Device functionality checks Now we are ready to checkout our disks to see if we can actually write to and read from them We recommend the use of the dd or rdcp for this testing Examples of both are given here dd if unix of dev rdsk sd713 bs 1024 count 25 25 0 records in 25 0 records out dd if dev rdsk sd713 of dev null bs 1024 count 10 10 0 records in 10 0 records out Chapter 5 NUMA Q Fibre Channel implementation 125 5 8 Specific limitations 126 The following is a list of some specific limitations which exist within some NUMA Q environments and not others gt With Dynix ptx V4 4 10 when using the Silkworm 2000 IBM 2109 type switch you must ensure that the following switch parameters are turned on VC Encoded Address Mode Disable Translative Mode When set properly these parameters will show as equal to 1 with a switchConfigure command gt With Dynix ptx V4 4 10 the switch domain ID must be set to O for the first switch In a cascaded environment you can have a second switch with a domain ID of 1 V4 4 10 does not support more than these two domains V4 4 10 does not support
48. that follow we detail the steps that need to be taken to install and successfully connect an HP UX system 4 5 1 Preparing to install the FC HBA There are a number of checks that you need to ensure at a minimum gt Check that you have a free supported PCI slot for the FC HBA gt Check that you have the correct Processor Dependent Code PDC firmware installed if you want to boot from a disk attached to the FC HBA If you need to upgrade the PDC version on your system contact your HP representative gt Check that you have the correct version and latest maintenance level of HP UX installed the minimum required levels are specified in Table 4 9 Table 4 9 Required patches for HP UX versions HP UX version Required patches 10 20 J3630BA FCMS PHSS_ 23581 or later PHKL_17590 PHKL_16751 FCMS PHKL_21381 FCMS PHKL_23626 gt Obtain the latest drivers Download system firmware HP UX fixes and maintenance levels from http www software hp com 100 Introducing hosts to the SAN fabric We use the following hardware and software in our installation HP 9000 model L1000 HP UX 11 0 HP A5158A Fibre Channel adapter Latest device driver updates for the Fibre Channel adapter YYY Yy 4 5 2 Installing the FC HBA Our system has HP UX preloaded at the factory and the Fibre Channel adapter is already installed Because of this we skip steps 1 6 1 Power on the system if it is not already on and log in as user r
49. the vendor s Web site to see if their storage equipment supports the Windows environment 1 5 FC host bus adapter card installation procedure In the following sections we cover in a step by step manner the driver installation procedure for QLogic Emulex and JNI FC host bus adapter cards We use a Netfinity server 5500 running Windows NT SP6 and Windows 2000 Advanced Server SP2 1 5 1 Preparing to install the Fibre Channel HBAs There are a number of common steps that you should do before installation of Fibre Channel HBAs gt Check if you have a free and supported PCI slot for the Fibre Channel HBAs gt Download the latest driver gt Download the latest adapter firmware The latest driver and firmware versions for all adapter types can be found at the following Web sites http www qlogic com http www emulex com http www jni com However any special considerations for specific Fibre Channel HBAs are mentioned in installation sections 1 5 2 Installing QLogic Fibre Channel HBAs family on Windows 2000 The QLogic SANblade 2200 and 2300 family of Fibre Channel HBAs unlike the QLA 2100 model support FC AL as well as Point To Point and Switched Fabric connection This section lists the steps you must follow in order to install the cards and the appropriate driver on Windows 2000 operating systems as well as the configuration utility for specific cards Installation procedure SANblade 2200 and 2300 adapters are p
50. three Fibre Channel network topologies depending on the Fibre Channel adapter gt Fibre Channel Arbitrated Loop FC AL gt Fibre Channel point to point gt Switched fabric FC AL has severe performance and scalability limitations Fibre Channel point to point is usually not feasible either because of the high cost and low number of Fibre Channel ports in any storage system Because of these limitations we only cover switched fabric topology However we also discuss attaching storage devices supporting only FC AL attachment to the fabrics We cover the products from the following three vendors gt IBM 2109 family gt INRANGE gt McDATA 142 Introducing hosts to the SAN fabric 7 3 1 General information There are two types of Gigabit Interface Converters GBIC and three types of fiber cable commonly used in Fibre Channel environments as described in Table 7 2 Table 7 2 Common Fibre Channel cabling configurations GBIC type Cable type Distance 1 Gb s Distance 2 Gb s Shortwave GBICs and 50 micron cable are usually used to connect devices to the fabric since most device adapters only support shortwave GBICs as well as for short inter switch links Longwave GBICs and 9 micron cable are used when required for extended distance Attenuation Fibre Channel is a high speed communication technology Because of this it is very sensitive to link attenuation The attenuation is even more of a critical factor when co
51. we are using As both of those steps are different for different switch manufacturers and different storage subsystems they are not described here Chapter 4 HP 9000 Fibre Channel implementation 103 104 Note The procedure described below is very generic You may have to install additional device drivers depending on your storage system like the Host Attachment Scripts for ESS or the atdd driver for IBM tape drives Refer to the documentation that came with your storage system We have access to a single volume in a storage subsystem We want to configure that volume to our host system The procedure is as follows 1 Scan for new devices ioscan f gt dev null Note The ioscan f command lists all the devices in the system producing a very long listing We are only interested in certain devices and we check for them later 2 Check that the switches switch domains are found ioscan funC fcp Class I H W Path Driver S W State H W Type Description fcp 2 0 4 0 0 1 fcp CLAIMED INTERFACE FCP Domain fcp 1 0 4 0 0 2 fcp CLAIMED INTERFACE FCP Domain Check that the new storage device disk in our case is found ioscan funC disk Class I H W Path Driver S W State 8 H W Type Description disk 0 0 0 1 1 2 0 sdisk CLAIMED DEVICE SEAGATE ST39204LC dev dsk c1t2d0 dev rdsk c1t2d0 disk 1 0 0 2 0 2 0 sdisk CLAIMED DEVICE SEAGATE ST39204LC dev dsk c2t2d0 dev rdsk c2t2d0 disk 2 0 0 2 1 2 0 sdisk CLAIMED DEVI
52. 0 DS20 DS20E ES40 GS60 GS60E GS140 For additional information related to IBM TotalStorage disk storage devices portfolio contact your local IBM Storage Specialist Other storage system vendors may also support their storage systems on Compaq AlphaServer running Tru64 UNIX 6 5 Installation procedures In the following topics we detail the installation procedures 6 5 1 Preparing to install the FC HBA We recommend that at a minimum you check the following gt Check if you have a free and supported PCI slot for the FC HBA 32 or 64 bit PCI bus slot Maximum PCI bus clock rate of 33 MHz gt Tru64 UNIX 4 0F or above operating system level is required gt Ensure that the console firmware level is v6 0 x or later gt Download the latest adapter firmware For KGPSA CA firmware rev 3 81A4 2 01A0 or higher For KGPSA BC firmware rev 3 01 1 31 or higher gt Tru64 patch 399 00 Security SSR50700U must be installed 132 Introducing hosts to the SAN fabric The latest firmware version for the FC HBA can be found at the following Web site http www compaq com products storageworks adapters html 6 5 2 Installing the FC HBA Shut down the system Install the FC HBA in the appropriate PCI slot in the host system Turn on the system Bring host system to a halt condition at the console level Verify the FC HBA installation Observe the LED indicators on the HBA see Figure 6 2 and Table 6 4
53. 0 0x00000070 L mscsi qlcO sd0 sd 0 Oxff000000 L scsi scsibus0 tm0 tm 0 0x00000040 L scsi scsibus0 cd0 cd 0 0x00000050 L scsi scsibus0 pci0 pci 0 0x00000003 L pci quad0 ped pe 0 0x00000003 L pci pci0 pel pe 1 0x00000003 L pci pci0 pe2 pe 2 0x00000003 L pci pci0 pe3 pe 3 0x00000003 L pci pci0 ffl ff 1 0x00000006 SP pci quad0 quad1 quad 1 Oxff000001 L sci sci0 ff2 ff 2 0x00000000 SP pci quad1 fabric3 fabric 3 0x00011c00 SMX fc ff2 sd713 sd 713 Ox2c0118ef SMX fcp fabric3 sd714 sd 714 0x2c0118ef SMX fcp fabric3 sd715 sd 715 Ox2c0118ef SMX fcp fabric3 sd716 sd 716 Ox2c0118ef SMX fcp fabric3 sd713 sd 713 Oxac0119ef SMX fcp fabric3 sd714 sd 714 Oxac0119ef SMX fcp fabric3 sd715 sd 715 Oxac0119ef SMX fcp fabric3 sd716 sd 716 Oxac0119ef SMX fcp fabric3 ff3 ff 3 0x00000006 SP pci quad1 19 Finally we run an etc diskid command to see all the devices and the multiple paths which are available to them Additionally we can see their vendor specific vital product data and the size of the LUN and the detail fabric information on the path diskid cd0 scsibus0 cport 0 target 5 lun O No Pbay on scsibus0 Vendor PLEXTOR product CD ROM PX 32CS revision 1 02 capacity 45 Mbytes sd0 scsibus0 cport 4294967295 target 0 lun O No Pbay on scsibus0 Vendor IBM product DRHS36V revision 0270 serial 680196E81K capacity 35239 Mbytes sd713 fabric3 cport 44 fc id 0x118ef fcp lun 0x1a53 No Pbay on fabric3 Vendor IBM product 2105F20 revision 0 83
54. 00 60 69 20 1d 4e itsoswl downstream port 3 No Module port 4 id No Light port 5 id Online F Port 10 00 00 00 c9 23 c3 05 port 6 id Online E Port 10 00 00 60 69 20 1d 4e itsoswl port 7 id No Light port 8 sw Online F Port 10 00 00 00 c9 22 bf 75 port 9 No Module port 10 sw No Light port 11 sw No Light port 12 sw Online L Port 1 public port 13 id No Light port 14 id Online F Port 10 00 00 00 c9 25 8e 54 port 15 id No Light itsosw3 admin gt exit Connection closed by foreign host root clyde root From Example 7 8 we see that the port we are using port 5 is online it is an F_Port for fabric attachment and the WWN listed is the same that we see on our adapter in Example 7 7 21 Your FC HBA is now correctly installed 7 5 4 Configuring storage for the Emulex FC HBA 160 After the adapter installation we have to create any necessary switch zones and allocate the storage for our host from the storage system we are using As both of those steps are different for different switch manufacturers and different storage subsystems they are not described here We have access to a single volume in a storage subsystem We want to configure that volume to our host system The procedure is as follows 1 Reboot the machine and log in as root Note We also tried to avoid rebooting when adding storage by unloading and reloading the Emulex driver This consistently caused the Linux kernel to panic
55. 00 c921 df4b FABRIC FABRIC 1 kgpsa0 0 0 9 0 2000 0000 c922 69bf FABRIC FABRIC 9999 All of the above The result of the wwidmgr show adapter command provides basic adapter configuration information such as WWN and SAN topology As shown in Example 6 1 Cur Topo Current Topology as well as Next Topo Next Topology value is FABRIC The current topology value can be either FABRIC or LOOP It indicates the mode in which the FC HBA is currently operating during this WWIDMGR session In the next column the Next Topo value represents FC HBA NVRAM setting and three different states are distinguished FABRIC LOOP or UNAVAIL In the example just shown in Example 6 1 fabric Next Topo value indicates how the FC HBA is configured the next time the system is initialized If you have a warning message Nvram Read Failed and Next Topo value is UNAVAIL this means that the FC HBA NVRAM is not formatted To correct this you should initialize and format FC HBA as shown in Example 6 2 by use of the wwidmr set adapter command wwidmgr set adapter item lt itemno gt topo lt fabric loop gt Chapter 6 Compaq AlphaServer Fibre Channel Implementation 135 Example 6 2 Formatting and initializing FC HBA P00 gt gt gt wwidmgr show adapter item adapter WWN Cur Topo Next Topo kgpsa0 0 0 9 0 Nvram read failed 0 kgpsa0 0 0 9 0 2000 0000 c921 df4b FABRIC UNAVAIL kgpsa0 0 0 10 0 Nvram read failed 1 kgpsa0 0 0 10 0 2000 0000 c922 69bf F
56. 01 09 27 12 01 10 SourceBuild Ipfc conf c Chapter 7 Linux on i386 platform Fibre Channel implementation 155 156 rw r r root root 186117 2001 10 01 12 43 37 SourceBuild fcLINUXfcp c r r r root root 3242 2001 09 27 12 01 10 SourceBuild Makefile r r r root root 3242 2001 09 27 12 01 10 SourceBuild Makefile module i386 r r r root root 3280 2001 09 27 12 01 10 SourceBuild Makefile kernel r r r root root 384 2001 09 27 12 01 10 SourceBuild diag_load r r r root root 163 2001 09 27 12 01 10 SourceBuild diag_unload rw r r root root 3627 2001 10 01 17 09 14 SourceBui 1d README rw r r root root 643 2001 09 27 12 01 10 SourceBuild Instal sh rw r r root root 378 2001 09 27 12 01 10 SourceBuild Remove sh rwxr xr x root root 83818 2001 09 27 12 01 10 SourceBuild dfc rw r r root root 13978 2001 09 27 12 01 10 SourceBuild libdfc a rwxr xr x root root 102514 2001 09 27 12 01 10 SourceBuild Iputil rwxr xr x root root 14340 2001 09 27 12 01 10 SourceBuild 1ibHBAAPI so rwxr xr x root root 77946 2001 09 27 12 01 10 SourceBuild 1ibemulexhbaapi so root clyde emulex cd SourceBuild root clyde SourceBuild Change the parameters for the Makefile according to your system If your kernel source resides in any other directory than usr src linux change the BASEINCLUDE variable to point to the correct directory If you are running a single processor kernel r
57. 05 Models E10 E20 F10 and F20 SC26 7297 IBM Enterprise Storage Server System 390 Command Reference 2105 Models E10 E20 F10 and F20 SC26 7298 IBM Storage Solutions Safety Notices GC26 7229 PCI Adapter Placement Reference SA38 0583 Translated External Devices Safety Information SA26 7003 Electrical Safety for IBM Customer Engineers S229 8124 SLIC Router Installation and Users Guide 310 605759 SLIC Manager Installation and User Guide 310 605807 The IBM NUMA Q publications which are also relevant as further information sources are available on the Web at http webdocs numaq ibm com The JNI publications which are also relevant as further information sources are available on the Web at http www jni com Support installguides cfm These McDATA publications are also relevant as further information sources gt gt gt ED 5000 Director Planning Manual 620 005000 Enterprise Fabric Connectivity Manager User Manual 620 005001 ED 5000 Director User Manual 620 005002 ED 5000 Director Service Manual 620 005004 ED 6064 Director Planning Manual 620 000106 100 ED 6064 Director User Manual 620 000107 ED 6064 Director Installation and Service Manual 620 000108 Introducing hosts to the SAN fabric Enterprise Fabric Connectivity Manager User Manual 620 005001 FC 512 Fabricenter Equipment Cabinet Installation and Service Manual 620 000100 ES 3016 Switch Planning Manual 620 0001 10 100 ES 3016 Switch
58. 105 W Windows 1 Windows 2000 2 11 16 Windows boot 16 Windows NT 2 11 12 17 wizard 12 World Wide Name 102 WWN 15 76 102 103 119 122 123 125 135 137 150 151 160 Z zone 74 113 zoned 108 zones 103 151 160 zoning 7 37 74 97 130 143 Index 185 186 Introducing hosts to the SAN fabric Redbooks Introducing Hosts to the SAN Fabric 0 2 spine 0 177x 50 473 90 5249 pages Introducing Hosts ae to the SAN Fabric Ready a switched In this IBM Redbook we aim to be short sharp and to the fabric capable host point and provide an answer There is no marketing no INTERNATIONAL for SAN connectivity sales pitch no extraneous information other than what is TECHNICAL needed to get the job done That job is making hosts Storage SUPPORT Discover the Area Network SAN ready ORGANIZATION simplicity of p a ty EW To this end we show how to make hosts SAN fabric ready and becoming SANitized Se ee ee connect them to the fabric with minimum fuss We cover all A SAN switched fabric capable hosts BUILDING TECHNICAL Verify your SANity in INFORMATION BASED ON a complex world With the vast combination of SAN components system levels BIS Sasi fe hardware and software it is not possible to cover each and every environment that is likely to be encountered in the field IBM Redbooks are developed by However we do intend to cover those elements that are the IBM International
59. 14 SAN switch 15 184 Introducing hosts to the SAN fabric SAN switches 2 SANblade 11 SANblade 2200 3 SANblade 2300 3 SC 94 117 129 scalability 96 SCSI 114 161 SCSI bridge 114 SCSILUNs 150 scsi gla0 adapter port 150 SDD 60 108 SDG 114 seated 96 security 7 97 130 143 Service Support Representative 101 SES 126 shortwave 6 96 98 110 112 128 131 143 Silkworm 112 singlemode 97 98 112 131 single path 34 128 single processor kernel 156 SMIT 42 Solaris 65 Solaris 2 6 60 Solaris7 60 Solaris8 60 SP 33 Sparc5 60 SSR 101 standard layer product installation model 112 statistics 13 storage adapter 4 storage devices 13 storage systems 9 StorageWorks 128 StorWatch 122 Subsystem Device Driver 60 94 128 Sun 450 76 Sun Microsystems 59 Sun Solaris 59 SuSE 140 SuSE 7 2 146 Switch 108 switch 113 127 switch firmware 121 switch information 136 Switched Fabric 5 11 Switched fabric 95 switched fabric 102 switchShow 79 80 136 system requirement 6 T TCP IP 117 technical specifications 5 6 telnet 120 121 topologies 2 108 TotalStorage 10 TotalStorage 2105 ESS 10 TotalStorage 2106 MSS 10 TotalStorage Enterprise Storage Server 146 TotalStorage FAStT200 10 TotalStorage FAStT500 10 TotalStorage FAStT700 10 Tru64 127 Tru64 Unix 128 129 131 132 136 TurboLinux 140 U Ultra2 60 unbalanced fabric 125 V VC Encoded Address Mode 126 version 79 volume 151 volume group
60. 4 18 20 37 Sun Jan 14 18 24 29 Mon Jan 22 15 54 10 PST PST PST PST PST PST PST PST PST PST PST PST PST PST PST PST PST 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 2001 DYNIX ptx R Public Software V4 5 3 PN ptx INSTALL V2 0 4 PN 1003 78071 04 A00 ptx BaseComms V1 2 0 PN 1003 77812 00 A00 ptx LAN V4 7 1 PN 1003 77834 01 A00 ptx TCP IP V4 6 2 PN 1003 77894 02 A00 ptx EFS V1 4 2 PN 1003 77828 02 101 Layered product Install Patch V1 0 2 PN Online Diagnostic Software V2 7 2 PN ptx EES V 1 N 1003 78904 02 A00 ptx XWM V4 N Ji 1003 77900 02 A00 ptx JSE V 1003 79941 00 A00 ptx edb V3 1003 79398 02 A02 ptx NFS V4 7 1 PN 1003 77841 01 A00 ptx Configuration Assistant V1 0 0 PN 2 2 P 6 2 P 0 2 PN 4 2 PN ptx SPDRIVERS V3 2 0 PN 1003 81493 00 A00 CFWDL FW Bundle V1 2 0 PN 1003 81485 00 A00 ptx JSE v1 1 2 PN 1003 74173 03 A00 From this list we can see that the operating system is at a supported level and that the necessary LAN products and CFWDL FW Bundle are installed as well 2 As our system was powered up and running we now shut it down so we can install the add on FC HBAs into the system 5 6 2 FC HBA setup 3 We install the FC HBA into the appropriate quad0 slot6 of PCI bus 1 for our system configuration Remember that this will vary with larger systems 4 After installing the FC HBA we repower the system and
61. 5 Installation procedure on page 76 Device functionality checks on page 90 Specific limitations on page 91 vvvvvrvvy Copyright IBM Corp 2002 59 3 1 Overview of the Sun system 60 There are a great number of Sun platform models which support the use of Fibre Channel to connect their system storage to them For information on which ones support Fibre Channel and in what configuration we recommend you contact your Sun representative or you can visit their Web site at the following URL http www sun com products n solutions The Sun system can also have many different adapters switches and devices which it supports in certain defined configurations These vary from product to product and vendor to vendor We strongly recommend that you contact your Sun representative or visit the varying vendor s Web pages for details on support Usage of their products with the Sun platforms are documented on a by product basis Sun platforms support the standard PCI bus or the Serial Bus Sbus for Fibre Channel host adapter FC HBA support Sun supports Fibre Channel in the Arbitrated Loop FC AL and Fibre Channel Switch switched fabric topologies With the FC AL topology Sun has a maximum limit of 50 LUNs per FC HBA The following Sun platforms are supported by the broadest number of vendors for Fibre Channel support gt Sun Sparc5 and Ultra1 Servers with QLogic QL2100 adapter only gt Sun Ultra2
62. 6460 settings AA 66 3 5 Recommended Emulex LP8000 HBA settings 67 3 6 Recommended QLA2200F HBA settings 68 3 7 The IBM 2109 family 000 ce eee 69 3 8 INRANGE directors 0 0 c eae 71 3 9 McDATA components s ccssar 0 0 e eee tees 72 3 10 IBM Fibre Channel attached disk storage systems 75 3 11 IBM Fibre Channel attached tape storage systems 75 3 12 Common Fibre Channel cabling configurations 76 4 1 Current HP FC HBAs with fabric support 0 000 94 4 2 Other FC HBAs available for HP 9000 systems 95 4 3 Common Fibre Channel cabling configurations 96 4 4 The IBM 2109 family 0 0 00 c eee 97 4 5 The INRANGE switches 0 000 cece eee 98 4 6 The McDATA switches 0 00 eee eee eee 98 4 7 IBM Fibre Channel attached disk storage systems 99 4 8 IBM Fibre Channel attached tape storage systems 100 4 9 Required patches for HP UX versions 020 005 100 5 1 Supported FC HBAS ereina tease dade aa hod an bee bea bad 109 5 2 The NUMA Q supported switch family 00 00005 112 Copyright IBM Corp 2002 ix x 5 3 5 4 5 5 5 6 5 7 5 8 6 1 6 2 6 3 6 4 7 1 7 2 7 3 7 4 7 5 7 6 IBM Fibre Channel attached disk storage systems 114 IBM NUMA Q RPQ Fibre Channel attached disk st
63. 8 and 16 ports switches Chapter 6 Compaq AlphaServer Fibre Channel Implementation 129 6 3 1 General information There are two types of Gigabit Interface Converters GBIC and three types of fiber cable commonly used in Fibre Channel environments FC cabling configurations are described in Table 6 1 Table 6 1 Common Fibre Channel cabling configurations GBIC type Cable type Distance 1 Gb s Distance 2 Gb s Shortwave GBICs and 50 micron cable are usually used to connect devices to the fabric since most device adapters only support shortwave GBICs as well as for short inter switch links Longwave GBICs and 9 micron cable are used when required for extended distance Attenuation Fibre Channel is a high speed communication technology Because of this it is very sensitive to link attenuation The attenuation is even more of a critical factor when connection speeds rise from 1 Gb s to 2 Gb s and even higher To limit the effects of attenuation we recommend making sure that You have as few connections as possible on a single link All connectors are clean All connectors are properly seated Only one type of fiber is used on a single link never mix 50 micron and 62 5 micron fiber YYY Yy Zoning We strongly recommend that you use zoning to gt Enhance the security of your Fibre Channel fabric gt Avoid interactions between different hosts gt Secure storage devices that don t support LUN masking like tape
64. 967 dev sda2 201433 3822 1872 dev sda5 3024016 540128 23302 dev sdb1 8570444 20 81350 root clyde root cp etc hosts test root clyde root 1s 1 etc hosts test hosts rw r r 1 root root 181 Dec rw r r 1 root root 181 Dec root clyde root diff etc hosts test hosts root clyde root The connection is successful 162 Introducing hosts to the SAN fabric p 9200 884736 1605632 formation done le Use Mounted on 08 23 10 3 boot 72 19 usr 68 1 test 3 04 00 etc hosts 3 15 15 test hosts Glossary 8B 10B A data encoding scheme developed by IBM translating byte wide data to an encoded 10 bit format Fibre Channel s FC 1 level defines this as the method to be used to encode and decode data transmissions over the Fibre channel Adapter A hardware unit that aggregates other I O units devices or communications links to a system bus ADSM ADSTAR Distributed Storage Manager Agent 1 In the client server model the part of the system that performs information preparation and exchange on behalf of a client or server application 2 In SNMP the word agent refers to the managed system See also Management Agent AIT Advanced Intelligent Tape A magnetic tape format by Sony that uses 8mm cassettes but is only used in specific drives AL See Arbitrated Loop ANSI American National Standards Institute The primary organization for fostering the development of technology stan
65. A Setups man D na kaa Suede E ote a ned a kabi ana 118 56 3 SWItCh Setup scl NANA ar bn Oh NN dae GAO Avett PARANG 120 DGA Storage Setup sabi Kind pea gee EE KP NA MINA AG 122 5 7 Device functionality checkS 0 0 e eee 125 5 8 Specific limitations ocea nies aiee ee 126 Chapter 6 Compaq AlphaServer Fibre Channel Implementation 127 6 1 Introductions ni ora weeds eG douse AAE a E E R E rk wee RR 128 6 2 Fibre Channel host bus adapters 0 ce eee eee 128 6 2 1 Compaq FG HBAs 26 obese el eee tee eGR eae ghee ka 128 6 3 Supported SAN switches 0 ccc eee 129 6 3 1 General information 0 00 e eee 130 6 3 2 IBM 2109 508 S16 o iertat amaa i eee 131 GA SOraJE o a ea a de ee e i ese 131 6 5 Installation procedures 0 cece eee 132 6 5 1 Preparing to install the FC HBA 20 0 eee eee 132 6 5 2 Installing the FC HBA 0 eee eee 133 6 6 Installing the storage system 0 cee ee 136 Chapter 7 Linux on i386 platform Fibre Channel implementation 139 LAN Introduction 2 0 ce a NAN a ie hae dee hae dah eee eee rare 140 Contents v vi 7 2 Fibre Channel host adapters 00 00 cece eee 141 TAT QLOGIC is ces ho Re eee ee ees Si Pee ee ANA 141 EMU tog setts doy eee E E Rang eas GRAN ON Suse a gyn BF 142 Tale IN ngg Ba HANG th sae alee tins ce eth Wee ee AL we 142 143 SAN fabriGyt eset Pg Maa SREY se ea ee I ed eee Re 142
66. A LIBRARY Solaris sparc v1 0 0 0 0 b 011106 10 Copyright 2001 JNI Corp All Rights Reserved Processing package information Processing system information Verifying disk space requirements Checking for conflicts with packages already installed Checking for setuid setgid programs This package contains scripts which will be executed with super user permission during the process of installing this package Do you want to continue with the installation of lt JNIsnia gt y n y Installing JNI SNIA Fibre Channel HBA LIBRARY Solaris as lt JNIsnia gt Installing part 1 of 1 opt JNIsnia Solaris Common 32bit HbaCommonLib so opt JNIsnia Solaris Common 64bit HbaCommonLib so opt JNIsnia Sol opt JNIsnia Sol aris Common Samp1e JniHbaSample c aris Common Sample Makefile opt JNIsnia Solaris Common Samp1le hbaapi h opt JNIsnia Sol opt JNIsnia Sol opt JNIsnia Sol opt JNIsnia Sol opt JNIsnia Sol opt JNIsnia Sol opt JNIsnia Sol opt JNIsnia Sol aris Common Sample test ib32 aris Common Sample test1 ib64 aris Jni 32bit JniHbaLib so aris Jni 64bit JniHbaLib so aris Jni Sample JniHbaSample c aris Jni Sample Makefile aris Jni Sample hbaapi h aris Jni Sample test1ib32 opt JNIsnia Solaris Jni Sample test1ib64 opt JNIsnia Sol aris SNIA_readme txt opt JNIsnia Solaris hba conf opt JNIsnia Sol aris hbaapi h verifying class lt none gt Executing postinstall script
67. A software program used to contact and obtain data from a server software program on another computer often across a great distance Each client program is designed to work specifically with one or more kinds of server programs and each server requires a specific kind of client program Client Server The relationship between machines in a communications network The client is the requesting machine the server the supplying machine Also used to describe the information management relationship between software components in a processing system Cluster A type of parallel or distributed system that consists of a collection of interconnected whole computers and is used as a single unified computing resource Coaxial Cable A transmission media cable used for high speed transmission It is called coaxial because it includes one physical channel that carries the signal surrounded after a layer of insulation by another concentric physical channel both of which run along the same axis The inner channel carries the signal and the outer channel serves as a ground 164 Introducing hosts to the SAN fabric Controller A component that attaches to the system topology through a channel semantic protocol that includes some form of request response identification CRC Cyclic Redundancy Check An error correcting code used in Fibre Channel DASD Direct Access Storage Device any on line storage device a disc drive or CD ROM DAT Di
68. ABRIC UNAVAIL 9999 All of the above P00 gt gt gt wwidmgr set adapter item 9999 topo fabric P00 gt gt gt wwidmgr show adapter item adapter WWN Cur Topo Next Topo 0 kgpsa0 0 0 9 0 2000 0000 c922 69bf FABRIC FABRIC 1 kgpsa0 0 0 10 0 2000 0000 c922 69bf FABRIC FABRIC 9999 All of the above P00555 6 6 Installing the storage system 136 In this section we describe connection of the Compaq AlphaServer running Tru64 UNIX to a storage device after successful FC HBA installation We connected AlphaServer to a IBM TotalStorage 2105 ESS via IBM 2109 5908 SAN Switch The process of storage definition and allocation as well as switch zones creation are not described here since they differ between manufacturers Read carefully the documentation that come with your storage system and your SAN devices Switch connectivity check In order to check if the path from AlphaServer to switch is established use the telnet command to connect to it As soon as the connection is established you will be prompted to enter a login name and password Administrator privileges are necessary in order to make changes in the switch configuration files The default administrator login name is admin and the password is password At this point you should be familiar with switch interface commands in order to gather switch information or to create a zone For more information about switch commands and their syntax type help from the switch command l
69. CE HP DVD ROM 304 dev dsk c3t2d0 dev rdsk c3t2d0 disk 3 0 4 0 0 1 17 255 0 0 0 sdisk CLAIMED DEVICE DEC HSG80 If the new disk is missing the special files dev dsk and dev rdsk like above create them as follows mksf C disk H 0 4 0 0 1 17 255 0 0 0 mksf C disk H 0 4 0 0 1 17 255 0 0 0 r You need both the block device and the character device Check the storage devices again as follows Introducing hosts to the SAN fabric ioscan funC disk Class I H W Path Driver S W State H W Type Description disk 0 0 0 1 1 2 0 sdisk CLAIMED DEVICE SEAGATE ST39204LC dev dsk c1t2d0 dev rdsk c1t2d0 disk 1 0 0 2 0 2 0 sdisk CLAIMED DEVICE SEAGATE ST39204LC dev dsk c2t2d0 dev rdsk c2t2d0 disk 2 0 0 2 1 2 0 sdisk CLAIMED DEVICE HP DVD ROM 304 dev dsk c3t2d0 dev rdsk c3t2d0 disk 3 0 4 0 0 1 17 255 0 0 0 sdisk CLAIMED DEVICE DEC HSG80 dev dsk c12t0d0 dev rdsk c12t0d0 5 Start using the device like any disk We use the procedure shown below to create a volume group small logical volume and filesystem on the new disk and test it as in Example 4 4 Example 4 4 Creating volume group and filesystem on the new disk mkdir dev vg01 mknod dev vg01 group c 64 0x010000 pvcreate dev rdsk c12t0d0 Physical volume dev rdsk c12t0d0 has been successfully created vgcreate s 16 dev vg01 dev dsk c12t0d0 Volume group dev vg01 has been successfully created Volume Group configuration for dev vg01 has
70. CS arr ket ioe Gea i ea Sed ele BCE DAANG ee GA xi Special notices 6s ech ae en eS Be ed tee Ree te ne Sey xiii IBMitradem rk x 0 08 os 004 4d pane ak me be RAS AWAL bh eee dae xiv Preah a ies A EE A ala Ea NE EEN XV The team that wrote this redbook 0 00 ee eee XV NOUGE pda oar tok ete NANA Maha maga Ak NANG Ng AG PAGA Pak ENG ae LERAAG a xviii Comments welcome 0 0 c eee tees xviii Chapter 1 Windows Fibre Channel implementation 1 1 1 Fibre Channel host bus adapter introduction 000 2 1 2 Windows supported host bus adapters 0 eee ee eee 2 1 2 1 QLogic Fibre Channel host bus adapter cards 3 1 2 2 Emulex Fibre Channel host bus adapter cards 4 1 2 3 JNI Fibre Channel host bus adapter cards 6 1 3 Supported SAN fabric components s sasaaa cece eeeee 6 1 3 1 General information 0 0000 ae 6 1 3 2 The IBM 2109 family eee 7 1 33 INRANGE aiii AG BAN BUGA ee ed GN 7 ASA MDATA amp a tee caty ela See eta sR Spas Be eae Bet 8 TAN SlOrage chai TANA bie al ote a ae KAG neta eae Ma 9 1 5 FC host bus adapter card installation procedure 11 1 5 1 Preparing to install the Fibre Channel HBAS 11 1 5 2 Installing QLogic Fibre Channel HBAs family on Windows 2000 11 1 5 3 Installing QLogic Fibre Channel HBAs family on Windows NT 12 1 5 4 QLogic configuration utility
71. DATA FC 512 145 McDATA FC 512 Cabinet 98 McDATA switch 99 145 memory 108 meshed 114 Microsoft certified 4 module dependencies 149 monolithic kernel 147 multimode 97 98 110 112 128 131 multi path 128 multipath driver 108 multipathing driver 60 multiple protocols 5 multi switch 114 N N_Port 4 102 names 14 Netfinity 11 networking 108 Node WWN 122 NUMA Q 107 113 NUMA Q zone 113 O oil 122 open source 140 operating system level 5 P partitions 153 patch 132 patches 61 77 95 path redundancy 125 paths 108 124 PCI 11 100 111 132 133 PDC 100 performance 96 physical location 116 Index 183 Point to Point 108 Point To Point 11 129 Point to Point 5 point to point 142 Port World Wide Name 102 power 17 price 2 probe 123 processor building blocks 108 Processor Dependent Code 100 protocol 102 pSeries 33 PTTOPT FABRIC 102 Q QLA2200 12 QLA2200F 11 64 68 147 QLA2300 12 QLA2300F 11 QLogic 2 4 11 12 15 62 81 141 147 152 QLogic configuration 13 QLogic driver 82 QLogic Fast Utility 16 QLogic QLA2200F 68 QLogic SANblade 2200 141 QLogic SANblade 2300 141 QLview 13 QLview for Fibre Setup 13 QLview stand alone configuration 13 QLview utility 13 quad 108 110 111 quad0 118 quads 108 R Red Hat 140 Red Hat 7 1 146 Red Hat Linux 147 155 Redbooks Web site 180 Contact us xviii Request for Price Quotation 116 RPQ 116 RS 6000 33 S SAN Data Gateway 1
72. EL 1 DMODULE DMODVERSIONS include usr src linux 2 4 include linux modversions h I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include DLP6000 D LINUX I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include c fcLINUXlan c rm f Ipfc ver cc Wall 02 fomit frame pointer D KERNEL 1 DMODULE DMODVERSIONS include usr src linux 2 4 include linux modversions h I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include DLP6000 D LINUX I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include c Ipfc conf c cp Ipfcdriver Ipfcdriver o ld r o Ipfcdd 2 4 2 2 0 Ipfcdriver o fcLINUXfcp o Ipfc conf o ld r o Ipfndd 2 4 2 2 0 fcLINUXlan o root clyde SourceBuild 9 Install the new module into your modules directory root clyde SourceBuild make install Installing under lib modules 2 4 2 2 kernel drivers root clyde SourceBuild 10 Copy the Emulex tools dfc and Iputil to usr sbin root clyde SourceBuild cp dfc Iputil usr sbin root clyde SourceBuild 11 Add the following line to etc modules conf to configure the new module alias scsi hostadapterl Ipfcdd 12 Rebuild the module dependencies file root clyde root depmod a root clyde root 13 Shut down your system 14 Install the Emulex adapter 15
73. HBAs are shipped with a multimode shortwave GLM Gigabit Link Module installed Some of these adapters work only in certain types of systems and environments See Fibre Channel x430 NUMA Q Service Host Adapter Overview for further details at the Web site http webdocs numaq ibm com When installing the FC HBA s it is important that you first know how many you will be installing With multiple quad systems it is strongly recommended that you install one FC HBA per fabric in each of the quads making up the system this is for performance and redundant paths Therefore for a four quad system with a single fabric you should install four adapters with two fabrics you would install eight and so on In Figure 5 1 we show the LP7000E adapter and in Figure 5 2 we show the LP 6000 adapter Figure 5 1 Emulex LP7000E Fibre Channel Adapter Introducing hosts to the SAN fabric ASSEMBLED IN PUERT decd eS ONN 0 RICO 2yge N 5 y JQA THSOH W W4301Z SHHO HLIM SINdWOD 19N00Hd HISYI D wio reese Clings za Y 1 su lt z is sO Le aT ne Figure 5 2 Emulex LP6000 Fibre Channel Adapter 5 2 1 Host adapter placement The first FC HBA installed in a NUMA Q system should be installed in PCI Bus 1 slot 6 of the boot quad usually quad 0 If a second adapter is to be added to the same quad it should be installed in PCI Bus 0 slot 3 Further FC HBAs being
74. IBM 2109 family of switches are described in Table 3 7 Table 3 7 The IBM 2109 family Speed Connectors Rack space IBM TotalStorage SAN Switch S08 8 1 Gb s sc liu IBM IBM TotalStorage SAN Switch S16 IBM TotalStorage SAN Switch S16 Switch S16 1 Gb s ee Gb s Chapter 3 Sun Fibre Channel implementation 69 The 2109 S08 and 2109 S16 are shown in Figure 3 4 At the time of writing the 2109 F16 is under test with the Sun systems Figure 3 4 IBM 2109 S08 top and S16 Features and limitations Special features of the IBM 2109 switch family are gt Switch domain ids 1 239 are supported gt The IBM 2109 family of switches support both fabric attached and loop attached devices on any port gt Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on customer requirements 70 Introducing hosts to the SAN fabric 3 3 2 INRANGE FC 9000 The INRANGE directors currently sold by IBM the IBM 2042 family are described in Table 3 8 Table 3 8 INRANGE directors Speed Connectors Rack space 2042 001 INRANGE FC 9000 Fibre 64 1 Gb s SC 20U Channel Director 2042 128 INRANGE FC 9000 128 1 Gb s Fibre Channel Director In addition IBM also sells the INRANGE FC 9000 40U cabinet 2042 C40 The cabinet is able to hold one 128 port switch or two 64 port oe In Figure 3 5 we show the INRANGE FC 9000 Figure 3 5 INRANGE FC 9000 switch Cha
75. Introducing Hosts to the SAN Fabric _ Ready a switched fabric capable host for SAN connectivity Discover the simplicity of becoming SANitized Verify your SANity in a complex world Jon Tate Pauli R m Robert Tondini Alexander Watson ibm com redbooks Red b ooks International Technical Support Organization Introducing Hosts to the SAN Fabric February 2002 SG24 6411 00 Take Note Before using this information and the product it supports be sure to read the general information in Special notices on page xiii First Edition February 2002 This edition applies to those IBM SAN software and hardware products currently in the IBM portfolio and described herein Comments may be addressed to IBM Corporation International Technical Support Organization Dept QXXE Building 80 E2 650 Harry Road San Jose California 95120 6099 When you send information to IBM you grant IBM a non exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you Copyright International Business Machines Corporation 2002 All rights reserved Note to U S Government Users Documentation related to restricted rights Use duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp Contents Figures oi iad a key ccm media Vege wet Deane bet a vii VAISS tn a RA NN AAAH wat epo seca ne ix EXAMPI
76. Most vendors will require their host be configured in their own separate zones This is to prevent the risk of any host from impacting the others or their data on the fabric This has been proven to be a good practice with all heterogeneous environments The type of zoning you can use will vary from switch vendor to vendor as does the methods that they use to implement the zoning types on their switches Review closely the details that the vendor provides to ensure that you understand the protection that your zone will be providing you Using cascaded switches Sun systems can be used with cascaded fabric environments as long as they meet the specifications of the switch vendors Again we strongly recommend that you review your planned configuration with the appropriate representatives to ensure compatibility and functionality We especially recommend care when considering mixed heterogeneous switched environments Heterogeneous fabrics have not been fully tested by IBM with Sun platforms 3 4 Supported storage on the Sun fabric 74 The Sun platform supports a great number of storage devices on its fabrics These are both Sun s own StorEdge line as well as a high number of third party storage devices For a complete list of supported devices contact your Sun representative For our purpose we list the IBM supported devices for the Sun platforms in Table 3 10 and Table 3 11 Introducing hosts to the SAN fabric Table 3 10 IBM Fibre
77. P UX supports cascading but with only a maximum of two switches 4 3 2 The IBM 2109 family The current members of the IBM 2109 family of switches are described in Table 4 4 Table 4 4 The IBM 2109 family Speed Connectors Rack space IBM TotalStorage SAN Switch S08 8 1 Gb s sc liu IBM IBM TotalStorage SAN Switch S16 IBM TotalStorage SAN Switch S16 Switch S16 1 Gb s a Gb s Features and limitations The IBM 2109 family of switches support both fabric attached and loop attached devices on any port and switch domain ids 1 239 are supported Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on requirements For more information see BM SAN Survival Guide Featuring the IBM 2109 SG24 6127 4 3 3 INRANGE The INRANGE switches currently supported by IBM the IBM 2042 family are described in Table 4 5 Chapter 4 HP 9000 Fibre Channel implementation 97 Table 4 5 The INRANGE switches Tuoast Pons spd Connon Rack pare_ 2042 001 INRANGE FC 9000 64 1 Gb s SC 20U Fibre Channel Director 2042 128 INRANGE 1 Gb s FC 9000 128 Fibre Channel Director In addition IBM also sells the INRANGE FC 9000 40U cabinet 2042 C40 The cabinet is able to hold one 128 port switch or two 64 port switches Features and limitations The INRANGE FC 9000 family of switches support both fabric attached and loop attached devices on any port Any port can be in
78. R ON Select Display a list of known drivers for this device so that can choose a specific driver 16 Introducing hosts to the SAN fabric 6 Click Next 7 Click Have Disk and insert the containing driver to the appropriate drive then type the location of the driver 8 Click OK 9 Select the Emulex Fibre Channel card and then click Next Windows 2000 indicates that the wizard is ready to install the device 10 Click Yes to restart the system The adapter contains two green and one yellow LEDs Immediately after successful installation of the adapter and the appropriate driver the green LED in the middle is ON indicating that the adapter has power As soon as the adapter is connected to a SAN switch or to a storage device directly both green LED s are ON as well as the yellow one which signifies port activity This is shown in Figure 1 8 on page 18 1 5 6 Installing Emulex in Windows NT The appropriate driver for both LP8000 and LP9000 adapters family are installed by default during your installation of Windows NT as well as the Light Pulse utility see Emulex configuration utility on page 18 You should update the existing driver to the latest version which can be found at http www emulex com We use the 4 53a5 Emulex driver version Windows NT Service Pack 3 is the minimum operating system requirement Installation procedure 1 Shut down the system 2 Install the Emulex Fibre Channel HBAs in the host s
79. SAP Certified Technical Consultant UNIX Oracle and UNIX DB2 UDB Introducing hosts to the SAN fabric Robert Tondini is a Storage Specialist in IBM Croatia He has 7 years of experience in S 390 platform initially as a VM VSE system programmer and later on migrating to the OS 390 environment as a DB2 system administrator He has been employed at IBM for 1 year His areas of expertise include IBM disk and tape storage subsystem solutions Alexander Watson is a System Engineer with Remotely Delivered Services RDS Beaverton Support Center BSC Beaverton Oregon USA He has 28 years of experience in the computer service field spanning from mainframe to personal systems He currently specializes in the IBM NUMA Q and pSeries Open Systems SAN and storage networking support His areas of expertise include hardware support I O device networking mixed open systems SAN environments RPQ storage and ESS support on open systems platforms Thanks to the following people for their contributions to this project Will Carney Emma Jacobs Yvonne Lyon Barry Mellish Deanna Polm Sokkieng Wang International Technical Support Organization San Jose Center Christopher Dace IBM SSG Scott Drummond IBM SSG Glenda Fuller IBM SSG Alejandro Halili IBM SSG Richard Heffel IBM SSG Michael Janini IBM SSG Preface xvii Notice This publication is intended to help systems and storage administrators install IBM SAN portfolio equipme
80. See N_Port Login Point to Point Topology An interconnection structure in which each point has physical links to only one neighbor resulting in a closed circuit In point to point topology the available bandwidth is dedicated Port The hardware entity within a node that performs data communications over the Fibre Channel Glossary 169 Port Bypass Circuit A circuit used in hubs and disk enclosures to automatically open or close the loop to add or remove nodes on the loop Private NL_Port An NL_Port which does not attempt login with the fabric and only communicates with other NL Ports on the same loop Protocol A data transmission convention encompassing timing control formatting and data representation Public NL_Port An NL_Port that attempts login with the fabric and can observe the rules of either public or private loop behavior A public NL_Port may communicate with both private and public NL_Ports Quality of Service QoS A set of communications characteristics required by an application Each QoS defines a specific transmission priority level of route reliability and security level RAID Redundant Array of Inexpensive or Independent Disks A method of configuring multiple disk drives in a storage subsystem for high availability and high performance Raid 0 Level O RAID support Striping no redundancy Raid 1 Level 1 RAID support mirroring complete redundancy Raid 5 Level 5 RAID support Striping with p
81. Server 114 115 2108 114 2109 6 7 38 69 96 97 102 131 142 144 146 150 159 2109 S08 97 129 2109 S16 129 3590E 115 6227 35 41 44 6228 35 41 44 64 Bit PCI 128 7556 109 7597 109 7598 109 7599 109 A A5158A 94 A6684A 94 A6685A 94 Access Restricted 122 adapter 11 adapter configuration 135 adapter configuration settings 16 adapter status 22 AIX 34 alarm notification 13 AlphaServer 127 129 131 132 134 136 137 Alternate Pathing 61 Arbitrated Loop 108 Arbitrated Loop 5 129 attenuation 7 76 96 116 130 143 availability 125 Copyright IBM Corp 2002 B bandwidth 129 BIOS 15 boot 16 118 boot probe 118 brand loyalty 2 bridge port 9 39 99 146 bridging 172 C cable lengths 116 Caldera 140 cascaded 74 108 114 cascading 97 CFWDL FW Bundle 118 class2 108 class3 108 clean 96 cluster device 4 Compaq 64 bit PCI 128 Compaq AlphaServer 127 128 Compaq KGPSA 128 configShow 78 configuration files 136 configuration information 13 connections 96 130 connectivity check 136 connectors 7 130 correct software 117 D device adapters 6 diagnostics 13 dirt 122 Disable Translative Mode 126 disk storage systems 146 domain 126 domain ids 131 domains 104 driver 11 81 driver configuration file 86 driver installation 11 driver level 5 181 driver package 155 driver support 109 driver utility 18 driver version 4 15 drivers 6 95 139 dust 122 Dynix pt
82. State switchRole switchDomain switchId switchWwn switchBeacon port 0 id port port port port port port port port port port 10 sw port 11 sw port 12 sw port 13 id port 14 id port 15 id id id id id id SW NN ODON Bb WN H wo itsosw3 2 4 Online Principal 3 fffc03 10 00 00 60 69 10 OFF Online F Port o Module Online E Port Module Light o Light Online E Port o Light Online F Port o Module o Light o Light Online L Port o Light Online F Port o Light o o itsosw3 admin gt quit Connection closed b4 cf 21 00 00 e0 8b 02 a7 2d 10 00 00 60 69 20 1d 4e itsoswl downstream 10 00 00 60 69 20 1d 4e itsoswl 10 00 00 00 c9 22 bf 75 1 public 10 00 00 00 c9 25 8e 54 From Example 2 5 we can see that the port we are using port 14 is online It is an F_Port for fabric attachment and the WWN listed is the same as that we see on our adapter as shown in Example 2 3 on page 50 14 Your adapter is now correctly installed Introducing hosts to the SAN fabric 2 5 3 Installing the storage system After the adapter installation we have to create any required switch zones and allocate the storage for our host from the storage system we are using As both of those steps are different for different switch manufacturers and different storage subsystems they are not described here Note The procedure described below is very generic You may have to install
83. Storage Area Network SG24 5758 Implementing an Open IBM SAN SG24 6116 Introduction to Storage Area Network SAN SG24 5470 vvvvvvy With the vast combination of SAN components system levels hardware and software it is not possible to cover each and every environment that is likely to be encountered in the field However we do intend to cover those elements that are common and that any reader with an intermediate grasp of SAN and host concepts should be able to apply The team that wrote this redbook This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization San Jose Center Copyright IBM Corp 2002 XV xvi NG The team from left to right Jon Robert Al Pauli Jon Tate is a Project Leader for SAN TotalStorage Solutions at the International Technical Support Organization San Jose Center Before joining the ITSO in 1999 he worked in the IBM Technical Support Center providing level 2 support for IBM storage products Jon has 16 years of experience in storage software and management services and support and is an IBM SAN Certified Specialist Pauli R m is an IT Specialist in IBM Finland He has 9 years of experience in RS 6000 AIX and Linux His areas of expertise include RS 6000 and RS 6000 SP systems AIX HACMP open systems storage solutions and SAP R 3 Basis He is an IBM Certified Advanced Technical Expert RS 6000 AIX V4 and a
84. Table 1 6 and Table 1 7 Chapter 1 Windows Fibre Channel implementation 9 Table 1 6 IBM Fibre Channel attached disk storage systems Storage system Operating systems IBM TotalStorage FAStT200 Windows NT 4 0 gt Service Pack 6a or greater Windows 2000 gt Service Pack 1 or greater IBM TotalStorage FAStT500 Windows NT 4 0 Server and Enterprise edition gt Service Pack 6a or greater Windows 2000 Server and Advanced server gt Service Pack 2 or greater IBM TotalStorage FAStT700 Windows NT 4 0 Server and Enterprise edition gt Service Pack 6a or greater Windows 2000 Server and Advanced server gt Service Pack 2 or greater IBM TotalStorage 2106 MSS Windows NT 4 0 Server and Enterprise edition Windows 2000 Server and Advanced server IBM TotalStorage 2105 ESS Windows NT 4 0 Server and Enterprise edition gt Windows 2000 Server and Advanced server Table 1 7 IBM Fibre Channel attached tape storage systems LTO 3584 Windows NT Server 4 0 gt service Pack 6a or later Windows 2000 gt Build 2195 or later Magstar 3590 drives Windows NT Server 4 0 gt service Pack 6a or later Windows 2000 Build 2195 or later For additional information related to IBM TotalStorage disk and tape storage devices portfolio contact your local IBM Storage Specialist 10 Introducing hosts to the SAN fabric Windows 2000 and Windows NT may also support storage systems produced by other vendors but you should check
85. Table 3 2 for the adapters which have been proofed by IBM For other adapters check with your Sun representative or the adapter vendor Table 3 2 Solaris kernel parameter settings required kernel parameter JNI adapters Emulex adapter QLogic adapter settings settings settings Er CC maxphys 8388608 maxphys 8388608 maxphys 8388608 maxphys 8388608 sd max throttle 256 highest LUN count per controller Chapter 3 Sun Fibre Channel implementation 65 In addition Fibre Channel host bus adapter configuration files are tunable for host system reliability and performance http www storage ibm com hardsoft products ess supserver htm See Table 3 3 for the recommended configuration settings for the host bus adapter for a JNI FC64 1063 and a JNI FCI 1063 Table 3 3 Recommended JNI HBA settings fca nport 0 Default Initializes on a loop 1 Recommended for fabric Initializes as an N Port public loop 0 Default Recommended Initializes according to which fca nport is set as disabled ip disable 0 Default IP side of the driver is enabled 1 Recommended for fabric The IP side of the adapters is completely disabled failover See the following Web site for details on how to set the parameter setting http www storage ibm com hardsoft products ess su pserver htm busy retry delay 500 Recommended Delay between retries after device returns a busy response for a command scsi probe delay 5000 Recommended Delay
86. We use the following in our installation IBM Netfinity 5500 Red Hat Linux 7 1 with kernel sources and kernel includes installed Kernel 2 4 2 QLogic QLA2200F FC HBA QLogic driver 4 27beta YYYY Y Red Hat Linux already contains modules for QLogic adapters but because it is an old version we rebuild a new one from the sources Our installation procedure is as follows 1 Start up your system and log in as root 2 Check that the kernel source files and headers are installed as follows root clyde root rpm q a grep kernel kernel headers 2 4 2 2 kernel 2 4 2 2 kernel source 2 4 2 2 root clyde root You need to have both kernel headers and kernel source installed If either of these packages is missing you have to install it before proceeding 3 Add the following line to the file etc lilo conf append max_scsi_luns 128 If you already have an append line in etc lilo conf add the max_scsi_luns parameter to that line 4 Reinstall LiLO by the following command root clyde root sbin lilo Added linux root clyde root Chapter 7 Linux on i386 platform Fibre Channel implementation 147 148 8 9 Create a directory for the driver package and unpack it there as follows root clyde root mkdir usr src qla2x00 root clyde root cd usr src qla2x00 root clyde qla2x00 tar xvvfz tmp qla2x00src v4 1 27Beta tgz rw r r root root 1924 2001 04 10 10 13 42 AddKernel txt rw r r
87. _Light port 3 sw No_Light port 4 No Module port 5 No Module port 6 No Module port 7 No Module port 8 No Module port 9 No Module port 10 No Module port 11 No Module port 12 sw No Light port 13 sw No Light port 14 sw No Light port 15 sw No Light 3 6 4 Storage setup In this section we explain the steps we took to prepare and connect the storage device which in this example is an ESS 1 We need to ensure that our storage device is ready to be connected to our Sun system In our example we are using a group of LUNs from an IBM ESS So we must now define the FC HBA to the ESS using the StorWatch manager As we are sharing the ESS between multiple hosts and the Sun the ESS ports will be configured for Access Restricted and specific LUN assignments will be used To do this we must map the desired LUNs to the FC HBA s WWN We obtain the Node WWN of the FC HBA from the infodev command we ran in step 3 above 2 We now need to connect the cables from the ESS to the switch again being careful not to contaminate them with dirt or oil from our hands We seat them securely into the GLM of the ESS adapter and the GBIC of the switch port Another check using switchShow gives us the following osv12109a 0s15 switchShow switchName osv12109a switchType 2 4 switchState Online switchRole Principal switchDomain 1 switchId fffc01 switchWwn 10 00 00 60 69 12 3f fc switchBeac
88. a window displays that shows the progress of the download 10 When the download completes click Close Chapter 3 Sun Fibre Channel implementation 81 82 Note If you are installing a different JNI adapter the process is the same except that you would pick the appropriate adapters s driver file to download from the Web page Downloading the Emulex adapter driver Following is the step by step process to follow to download the Emulex driver to your system for installation 1 Plug in and restart your host system and login as root 2 Enter the following to create a temporary directory mkdir emulextemp 3 Move to the temporary directory with the following command cd emulextemp 4 Goto the following Web site http www emulex com ts dds html 5 From the Models selection box select LP8000 or whatever your adapter type may be When you click on your model you will be automatically taken to the relevant documentation and driver page for downloading 6 Scroll down to the section titled Drivers for Solaris and select the PCI Version download 7 From the Solaris Driver Version screen select either the Intel or Sparc download version depending on your platform you will be running it on 8 From the Open it Save to disk menu select Save to disk and save the file to your emulextemp directory 9 The file will save as a tar file and will need to be extracted by doing the following tar xvf Ipfc sparc t
89. able 11 08 00 2 0 16 Bit LVD SCSI Disk Drive hdiskl Available 11 08 00 4 0 16 Bit LVD SCSI Disk Drive hdisk2 Available 3A 08 01 Other FC SCSI Disk Drive Chapter 2 eServer pSeries Fibre Channel implementation 57 58 4 Start using the device like any disk We use the procedure shown in Example 2 6 to create a volume group and filesystem on the new disk and test it Example 2 6 Creating and testing a filesystem mkvg s 32 y myfcvg hdisk2 0516 1254 mkvg Changing the PVID in the ODM myfcvg crfs v jfs g myfcvg m fctest A yes a size 1048576 Based on the parameters chosen the new fctest JFS file system is limited to a maximum size of 134217728 512 byte blocks New File System size is 1048576 mount fctest cp etc hosts fctest hosts diff etc hosts fctest hosts The connection is successful Introducing hosts to the SAN fabric Sun Fibre Channel implementation In this chapter we discuss how to successfully set up a SAN fabric for Sun Microsystems running Sun Solaris V2 6 7 and 8 We cover the step by step procedure from start to finish and the supported products which will work with these Sun systems The following topics are discussed Overview of the Sun system on page 60 Sun supported Fibre Channel host adapters on page 62 Sun supported Fibre Channel switches and hubs on page 69 Supported storage on the Sun fabric on page 74 Installation planning on page 7
90. additional device drivers depending on your storage system like the Host Attachment Scripts for ESS or the Atape driver for tapes Refer to the documentation that came with your storage system We have access to a single volume in a storage subsystem We want to configure that volume to our host system The procedure is as follows 1 Try to run cfgmgr to scan for devices cfgmgr 2 If you get an error message like the one below you have to install additional device drivers cfgmgr cfgmgr 0514 621 WARNING The following device packages are required for device support but are not currently installed devices fcp disk The procedure to install the new drivers is as follows a Start SMIT by typing smitty cfgmgr You get the screen shown in Figure 2 13 Chapter 2 eServer pSeries Fibre Channel implementation 53 of Telnet sicily Connect Edit Terminal Help Install Configure Devices Added After IPL Type or select values in entry fields Press Enter AFTER making all desired changes Entry Fields NOTE A selection of none configures devices added after IPL without the installation of software INPUT device directory for software Hone F2 Refresh F3 Cancel F4 List Esc 6 Command Esc 7 Edit Esc 8 Image Esc Exit Enter Do Figure 2 13 Install storage drivers select install medium b Choose or type in your install device or directory dev cd0 for example c Accept the choices by pressing E
91. allation in progress 45 2 7 Install additional device software mount additional volume 46 2 8 Install additional device software results 46 2 9 Update device software install medium 2 47 2 10 Update device software installation settings 48 2 11 Update device software installation in progress 48 2 12 Update device software results eee 49 2 13 Install storage drivers select install medium 54 2 14 Install storage drivers installation in progress 54 2 15 Install storage drivers results aaa 55 Copyright IBM Corp 2002 vii viii 2 16 2 17 2 18 2 19 3 1 3 2 3 3 3 4 3 5 3 6 5 1 5 2 6 1 6 2 Update storage drivers select install medium 56 Update storage drivers installation settings 56 Update storage drivers installation in progress 57 Update storage software results a 57 Emulex LP8000 adapter 0 cece eee eee 63 JNI FC64 1063 adapter 0 ee eee 64 QLogic QLA2200F adapter 0000 eee ee 65 IBM 2109 508 top and S16 2 0 00 eee 70 INRANGE FC 9000 switch 0 0 0000 e eee eee 71 McDATA cabinet left and 6064 rack mountable switch 73 Emulex LP7000E Fibre Channel Adapter
92. ar Downloading the QLogic adapter driver The following is the step by step process to follow to download the QLogic driver to your system for installation 1 Plug in and restart your host system and login as root 2 Enter the following to create a temporary directory mkdir qlogictemp 3 Move to the temporary directory with the following command cd qlogictemp Introducing hosts to the SAN fabric 4 Go to the following Web site http www qlogic com support home_support asp From the HBA selection box select the product QLA2200F or whatever your adapter type may be When you click on your model you will be automatically taken to the relevant documentation and driver page for downloading 6 Scroll down to the section titled Drivers and select Solaris 7 From the Description section select Installation and download the readme file to your system for reference From the Download section select download to download the driver to your system and save it to your qlogictemp directory The file will save as a compressed tar file and will need to be uncompressed and untarred into the directory for installation uncompress qla2200 Z tar xvf qla2200 tar Note If we had downloaded the file to another platform we would need to move it to our system for installation normally with something like ftp Additionally some vendors package their driver files in tar files and may also compress th
93. arity Repeater A device that receives a signal on an electromagnetic or optical transmission medium amplifies the signal and then retransmits it along the next leg of the medium Responder A Fibre Channel term referring to the answering device Router 1 A device that can decide which of several paths network traffic will follow based on some optimal metric Routers forward packets from one network to another based on network layer information 2 A dedicated computer hardware and or software package which manages the connection between two or more networks See also Bridge Bridge Router 170 Introducing hosts to the SAN fabric SAF TE SCSI Accessed Fault Tolerant Enclosures SAN A Storage Area Network SAN is a dedicated centrally managed secure information infrastructure which enables any to any interconnection of servers and storage systems SAN System Area Network term originally used to describe a particular symmetric multiprocessing SMP architecture in which a switched interconnect is used in place of a shared bus Server Area Network refers to a switched interconnect between multiple SMPs SC Connector A fiber optic connector standardized by ANSI TIA EIA 568A for use in structured wiring installations Scalability The ability of a computer application or product hardware or software to continue to function well as it or its context is changed in size or volume For example the ability to retain perf
94. atches for Fibre Channel support In the Table 3 1 we show the minimum required kernel patches for running Fibre Channel with Solaris 2 6 7 and 8 Ensure that you have these patches installed prior to trying to run a Fibre Channel environment Table 3 1 Solaris 2 6 7 and 8 minimum patch levels for Fibre Channel CA 105181 23 kernel update 106541 12 kernel update 108528 03 kernel update 105356 16 sd ssd drivers 106924 06 isp driver 109524 02 ssd driver 105580 16 glm driver 106925 04 glm driver 109657 01 isp driver 105600 19 105600 19 isp driver 105600 19 isp driver 107147 08 107147 08 pci driver 107147 08 pci driver 108974 03 108974 03 sd uata drivers uata drivers Not applicable 107458 10 dad sd ssd aappear applicable uata drivers Chapter 3 Sun Fibre Channel implementation 61 Note Care should be taken when you this command Frequently patches are obsoleted by newer versions of the kernel update which means you will no longer see the original patch number listed as an active patch at all Such is the case with 105600 19 for the ISP driver as it was rolled up into version 105181 29 of the kernel update on this system 3 2 Sun supported Fibre Channel host adapters 62 Sun supports a great number of different vendor FC HBAs for their products in both the 1 Gb s and 2 Gb s models An important point to consider here when selecting the adapter you will use is whether your other device v
95. ates Any reference to an IBM product program or service is not intended to state or imply that only IBM s product program or service may be used Any functionally equivalent program that does not infringe any of IBM s intellectual property rights may be used instead of the IBM product program or service Information in this book was developed in conjunction with use of the equipment specified and is limited in application to those specific hardware and software products and levels IBM may have patents or pending patent applications covering subject matter in this document The furnishing of this document does not give you any license to these patents You can send license inquiries in writing to the IBM Director of Licensing IBM Corporation North Castle Drive Armonk NY 10504 1785 Licensees of this program who wish to have information about it for the purpose of enabling i the exchange of information between independently created programs and other programs including this one and ii the mutual use of the information which has been exchanged should contact IBM Corporation Dept 600A Mail Drop 1329 Somers NY 10589 USA Such information may be available subject to appropriate terms and conditions including in some cases payment of a fee The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS The use of this information or the implementation of any of these te
96. been saved in etc lvmconf vg01 conf Ivcreate L 32 dev vg01 Logical volume dev vg01 vol1 has been successfully created with character device dev vg01 rlvol1 Logical volume dev vg01 vo11 has been successfully extended Volume Group configuration for dev vg01 has been saved in etc lvmconf vg01 conf newfs dev vg01 rlvol1 newfs etc default fs is used for determining the file system type version 3 layout 32768 sectors 32768 blocks of size 1024 log size 1024 blocks unlimited inodes 32768 data blocks 31672 free data blocks 1 allocation units of 32768 blocks 32768 data blocks first allocation unit starts at block 0 overhead per allocation unit is 0 blocks mkdir testfilesys mount dev vg01 lvoll testfilesys df k testfilesys testfilesys dev vg01 1vol1 ye 30797 total allocated Kb 29688 free allocated Kb 1109 used allocated Kb 3 4 allocation used cp etc hosts testfilesys Is 1 etc hosts testfilesys hosts rw r r 1 root sys 578 Sep 26 15 27 etc hosts rw r r 1 root sys 578 Nov 28 16 44 testfilesys hosts diff etc hosts testfilesys hosts The connection is successful Chapter 4 HP 9000 Fibre Channel implementation 105 106 Introducing hosts to the SAN fabric NUMA Q Fibre Channel implementation In this chapter we discuss how to successfully connect IBM NUMA Q platforms running Dynix ptx to the SAN fabric We cover the step by step procedure from start to finish and the supported pro
97. boot it up At this time the boot probe has discovered our new FC HBA and we can confirm it is present and gather data on it by using the following commands etc dumpconf grep ff quad0 quad asy0 asy asyl asy mdc0 mdc ff0 ff sd0 sd ffl ff quad1 quad ff2 ff fabric3 fabric ff3 ff 118 Introducing hosts to the SAN fabric FH HDD Hr O o Oxff000000 L sci sci0 Oxff000000 L eisa quad0 Oxff000001 L eisa quad0 Oxff000000 L eisa quad0 0x00000000 SP pci quad0 Oxff000000 L scsi scsibus0 0x00000006 SP pci quad0 Oxff000001 L sci sci0 0x00000000 SP pci quad1 0x00011c00 SMX fc ff2 0x00000006 SP pci quad1 5 Now we run devct1 N command to assign the temporary devices to their permanent names devctl N devctl Assigned ff2 gt ff2 6 Next we gather data from it using the infodev a command This command provides us with the WWN which we will need when configuring the LUNs on the ESS to be assigned to the NUMA Q system etc infodev a ff2 Vendor ID Emulex Product ID Firefly Revision Level SuperFly 4 Device Class ctlr Node WWN 10 00 00 00 c9 21 58 59 Port WWN 10 00 00 00 c9 21 58 59 Located on quadl 7 Finally we check to ensure the correct version of FC HBA firmware is loaded on the adapter If this is not the correct version we would use the same cfwd1 command with the download option to load the correct firmware file See the cfwdl manpage for details on the command usage usr service bi
98. bric k You get the ARE YOU SURE window press Enter to accept it You get the COMMAND STATUS window as shown in Figure 2 18 F Telnet sicily Connect Edit Terminal Help COMMAND STATUS Command EM stdout no stderr no Before command completion additional instructions may appear below Figure 2 18 Update storage drivers installation in progress When the status changes from RUNNING to OK scroll to the bottom of the screen as shown in Figure 2 19 to check that installation ended in SUCCESS of Telnet sicily Connect Edit Terminal Help COMMAND STATUS Command J stdout yes stderr no Before command completion additional instructions may appear below devices fcp disk rte 3 3 75 USR COMMIT SUCCESS devices scsi scarray rte 3 3 50 USR COMMIT SUCCESS devices scsi scarray rte 3 3 50 ROOT COMMIT SUCCESS devices fcp disk array rte 3 3 75 USR COMMIT SUCCESS devices fcp disk array rte 3 3 75 ROOT COMMIT SUCCESS devices fcp disk array diag 4 3 3 56 USR COMMIT SUCCESS installp xx ATTENTION Software changes processed during this session require this system and any of its diskless dataless clients to be rebooted in order for the changes to be made effective F2 Refresh F3 Cancel Esc 6 Command Esc 9 Shell Esc 6 Exit Find Figure 2 19 Update storage software results m Press F10 to exit SMIT 3 Check that the new storage device disk in our case is found Isdev Ccdisk hdisk0 Avail
99. bs 1024 count 25 records in records out if dev rdsk c2t1d0s2 of dev null bs 1024 count 10 records in records out 25 0 25 0 dd 10 0 10 0 Introducing hosts to the SAN fabric 3 8 Specific limitations The following is a list of some specific limitations which exist within some Sun environments and not others gt Some of the newer FC HBAs use the above mentioned WWN entry for the device ports to control their fabric probe This feature is not done by all adapters gt The FC AL environment cannot exceed 50 devices per loop For more details on these and other considerations that may exist in other environments contact your Sun representative for help Chapter 3 Sun Fibre Channel implementation 91 92 Introducing hosts to the SAN fabric HP 9000 Fibre Channel implementation In this chapter we discuss how to successfully set up a SAN fabric for the HP 9000 series platforms We cover the step by step procedure from start to finish and the supported products which work with the HP 9000 systems We also point out where the latest firmware and device drivers can be obtained In this chapter the following topics are discussed Introduction on page 94 Fibre Channel host bus adapters on page 94 SAN fabrics on page 95 Storage on page 99 Installation procedures on page 100 YYYY Y Copyright IBM Corp 2002 93 4 1 Introduction HP UX is the operating system used on the HP 9000 syste
100. check if the SAN switch is ready Assume that the switch is already initialized and configured That means that it is configured in the local network with an IP address Find out the switch IP address and see if it is reachable from your Windows 2000 or Windows NT environment by typing the ping lt switch_ip_address gt command using the Command Prompt interface If you get a reply from the switch you can access it using the telnet session as well as a Web browser If you want to access the switch through a telnet session as soon as the connection is established you will be prompted to enter a login name and password Administrator privileges are necessary in order to make some changes in the switch configuration files The default administrator login name is admin and the password is password At that point you should be familiar with switch interface commands in order to gather switch information or to create a zone For more information about switch commands and their syntax type help from the switch command line To check if your FC HBA is properly connected to a switch port type the swi tchshow command as shown in Example 1 1 Chapter 1 Windows Fibre Channel implementation 23 24 Example 1 1 Switch connectivity check Fabric OS tm login admin Password Release a2 4 1c itsosw3 admin gt switchShow switchName switchType switchState switchRole switchDomain switchId switchWwn switchBeacon port 0 id
101. ched disk storage systems Storage system Supported distributions IBM TotalStorage Enterprise Red Hat 7 1 Storage Server SuSE 7 2 IBM TotalStorage FAStT700 Red Hat 7 1 Only supports IBM 2109 switch Storage Server fabrics IBM TotalStorage FAStT500 Red Hat 7 1 Storage Server IBM TotalStorage FAStT200 Red Hat 7 1 Only supports IBM 2109 switch Storage Server fabrics Other storage system vendors may also support their storage systems on Linux systems 7 5 Installation procedures Installing new devices in a Linux system can be quite different from installing the devices in any other operating system The device drivers are often delivered as source files written usually in the C programming language and have to be compiled before they can be used 146 Introducing hosts to the SAN fabric There are two ways to install a device driver in Linux Device drivers can either be compiled directly into the kernel monolithic kernel or they can be compiled as loadable kernel modules Most device drivers are compatible with both types of installation We chose to install the drivers as modules since that gives us much more flexibility We choose Red Hat Linux 7 1 as the starting point of our installations because that distribution is currently supported by the widest range of FC HBAs and storage devices The installation procedure is however similar in most other Linux distributions 7 5 1 Installing a QLogic FC HBA
102. ching Hub ports that can only be configured with shortwave multimode GBIC or copper interface gt One bridge port for connecting the loop switch to the fabric that can be configured with a shortwave multimode or longwave singlemode GBIC The McDATA switches support fabrics of up to 31 switches only For more information see IBM SAN Survival Guide Featuring the McDATA Portfolio SG24 6149 4 4 Storage The IBM Fibre Channel attached storage systems supported on HP 9000 series machines are described in Table 4 7 and Table 4 8 Table 4 7 IBM Fibre Channel attached disk storage systems IBM TotalStorage Enterprise Storage Only supports HP UX 11 0 Server IBM TotalStorage FAStT500 Storage Server IBM TotalStorage FAStT200 Storage Only supports IBM 2109 switch fabrics Server IBM TotalStorage SAN Controller 160 Only supports IBM 2109 switch fabrics with IBM 7133 disk subsystems Chapter 4 HP 9000 Fibre Channel implementation 99 Table 4 8 IBM Fibre Channel attached tape storage systems Storage system IBM TotalStorage Enterprise Tape Requires feature 3510 or 9510 on the drive System 3590 Only supports IBM 2109 switch fabrics Cascaded switches are not supported IBM 3584 UltraScalable Tape Library Requires feature 1456 or 1466 on the drive Only supports IBM 2109 switch fabrics Other storage system vendors also support their storage systems on HP 9000 series systems 4 5 Installation procedures In the topics
103. chniques is a customer responsibility and depends on the customer s ability to evaluate and integrate them into the customer s operational environment While each item may have been reviewed by IBM for accuracy in a specific situation there is no guarantee that the same or similar results will be obtained elsewhere Customers attempting to adapt these techniques to their own environments do so at their own risk Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites Copyright IBM Corp 2002 xiii IBM trademarks The following terms are trademarks of the International Business Machines Corporation in the United States and or other countries e logo FICON RS 6000 Redbooks logo IBM 390 Magstar SANergy AIX Netfinity SP AIX 5L Notes SP2 DB2 NUMA Q StorWatch DYNIX OS 2 System 390 DYNIX ptx OS 390 Tivoli Enterprise Storage Perform TotalStorage Server pSeries Wave ESCON Redbooks Other company trademarks xiv The following terms are trademarks of other companies C bus is a trademark of Corollary Inc in the United States and or other countries Java and all Java based trademarks and logos are trademarks or registered trademarks of Sun Microsystems Inc in the United States and or other countries Microsoft Windows Windows NT and the Windows logo ar
104. d Chapter 3 Sun Fibre Channel implementation 89 9 Acheck which can be used with all adapters is the switch monitor tool With the IBM 2109 we used a switchShow command to show whether the adapter is logged in to the switch or not This is also useful in isolating the problem osv12109a 0s15 switchShow osv12109a switchName switchType switchState switchRole switchDomain switchId switchWwn switchBeacon 0 port port port port port port port port port port port port port port port port wo 10 11 12 13 14 15 O NQM 0 Bp WON H SW SW id SW SW id 2 4 Online Principal 1 fffc01 10 00 00 60 69 12 3f fc OFF No Light No Light Online No Light No Module No Module Online No Module No Module No Module No Light No Light No Light No Light No Light No Light F Port 20 00 00 01 73 00 2f 6e F Port 50 05 07 63 00 c0 8b 2b For INRANGE switches you should use the IN VSN Enterprise Manager application to find the same information For McDATA switches you can find the information in the EFC Manager application Both of these applications are graphical user interface GUI based for ease of use 3 7 Device functionality checks 90 Now we are ready to checkout our disks to see if we can actually write to and read from them We recommend the use of the dd or rdcp commands for this testing An example of dd is shown here dd if unix of dev rdsk c2t1d0s2
105. dards in the United States The ANSI family of Fibre Channel documents provide the standards basis for the Fibre Channel architecture and technology See FC PH Arbitration The process of selecting one respondent from a collection of several candidates that request service concurrently Arbitrated Loop A Fibre Channel interconnection technology that allows up to 126 participating node ports and one participating fabric port to communicate ATL Automated Tape Library Large scale tape storage system which uses multiple tape drives and mechanisms to address 50 or more cassettes Copyright IBM Corp 2002 ATM Asynchronous Transfer Mode A type of packet switching that transmits fixed length units of data Backup A copy of computer data that is used to recreate data that has been lost mislaid corrupted or erased The act of creating a copy of computer data that can be used to recreate data that has been lost mislaid corrupted or erased Bandwidth Measure of the information capacity of a transmission channel Bridge 1 A component used to attach more than one I O unit to a port 2 A data communications device that connects two or more networks and forwards packets between them The bridge may use similar or dissimilar media and signaling systems It operates at the data link level of the OSI model Bridges read and filter data packets and frames Bridge Router A device that can provide the functions of a bridge ro
106. dependently configured with a shortwave multimode or longwave singlemode GBIC depending on requirements For more information see IBM SAN Survival Guide Featuring the INRANGE Portfolio SG24 6150 4 3 4 McDATA The McDATA switches currently supported by IBM the IBM 2031 and 2032 families are described in Table 4 6 Table 4 6 The McDATA switches 2031 016 McDATA ES 3016 16 1 Gb s LC 1U Fabric Switch 2031 032 McDATA ES 3032 32 1 Gb s LC 1 5U Fabric Switch 2031 L00 McDATA ES 1000 8 1 1 Gb s SC 1U Loop Switch 2032 001 McDATA ED 5000 32 1 Gb s SC 18U Enterprise Fibre Channel Director 2032 064 McDATA ED 6064 1 Gb s Enterprise Fibre Channel Director In addition IBM also sells the McDATA FC 512 Cabinet 2032 C36 98 Introducing hosts to the SAN fabric Features and limitations The only McDATA switch that supports loop attached devices tape drives for example is the McDATA ES 1000 Loop Switch The ports in the 2032 001 and 2032 064 can be added in cards The cards are not compatible between the models Each card contains four ports There are three different cards available gt Four shortwave multimode ports gt Four longwave singlemode ports gt Three shortwave ports and one longwave port On the 2031 016 and 2031 032 switches any port can be independently configured with a shortwave multimode or longwave singlemode GBIC The 2031 L00 switch has two different types of ports gt Eight swit
107. devices see the following Web site http webdocs numaq ibm com From this location select Fibre Channel SAN Disk Systems or Tape Systems as appropriate Introducing hosts to the SAN fabric 5 2 Fibre Channel host bus adapters With all the Dynix ptx operating systems there is driver support incorporated into the base operating system In Table 5 1 we list all of the Fibre Channel host bus adapters FC HBA which are supported by the NUMA Q system Table 5 1 Supported FC HBAs FC HBA type IBM Part Usage comments EmulexLP7000E 7556 9P8178 This adapter supports all of nicknamed the NUMA Q supported Superfly storage subsystems and quad types It does not support any FC AL hub connections Emulex LP6000 7599 9P8183 This is the most recent V3 non OFC model of this adapter and nicknamed is used with NUMA Q Firefly quads Q06 through Q11 and Mquad Q01 through Q05 based systems Some of the newer storage subsystems are not supported with this adapter It does not support any FC AL hub connections Emulex LP6000 7598 Used only in quad based systems Did not support switched environments Supported FC AL hubs Some of the newer storage subsystems are not supported with this adapter Emulex LP6000 7597 Used only with NUMA Q quads Q06 through Q11 Some of the newer storage subsystems are not supported with this adapter Chapter 5 NUMA Q Fibre Channel implementation 109 110 All of these FC
108. ducing hosts to the SAN fabric fcAL openSendCLS 0 fcAL useAl tBBCredit 0 1cdContrast 128 1cdContrast orange 208 licenseKey zdeSSRIZy2efTTy SbRcyRQSdcTcSp0y route delayReroute 0 route embeddedPortBcast 1 route stickyRoutes 1 rpc rapid 1 rpc rstatd 0 rpc rusersd 0 shell delete 0 shell quiet 0 shell timeout 0 syslog errDisplayLevel 5 syslog errsOnConsole 1 thresh alarmsfilter 1 thresh thad 1 xlativeModeDisable 0 Displaying enabled disabled portlog events Type lt CR gt to continue Q lt CR gt to stop q We now need to ensure we have the proper switch firmware This we do by running the version command on the switch telnet session osv12109a 0s15 version Kernel 534 Fabric OS a2 2 1la Made on Tue May 8 15 44 06 PDT 2001 Flash Tue May 8 15 45 24 PDT 2001 BootProm Thu Jun 17 15 20 39 PDT 1999 We now need to connect the cables ensuring that we are careful not to touch the ends and making sure they are free of any dirt or dust We seat them securely into the GLM of the FC HBA and the GBIC of the switch port By issuing the switchShow command we can see these switch details osv12109a 0s15 switchShow switchName SNJ2109DD2 switchType 2 4 switchState Online switchRole Principal switchDomain 1 switchId fffc01 switchWwn 10 00 00 60 69 12 3f fc switchBeacon OFF port 0 sw No Light port 1 sw No Light Chapter 3 Sun Fibre Channel implementation 79 port 2 sw No
109. ducts which will work with the NUMA Q system The following topics are discussed Overview of the NUMA Q system on page 108 Fibre Channel host bus adapters on page 109 Fibre Channel switch support on page 112 Supported storage on the NUMA Q fabric on page 114 Installation planning on page 116 Installation procedure on page 117 Device functionality checks on page 125 Specific limitations on page 126 vvvvvvvy Copyright IBM Corp 2002 107 5 1 Overview of the NUMA Q system 108 The NUMA Q system is composed of four processor building blocks called quads Each quad has two separate PCI busses for installing IO cards You can interconnect the quads to each other to create a single larger system When you connect the quads together in this manner they run on a single instance of the operating system and share their memory storage and networking capabilities The NUMA Q system runs with only the Dynix ptx operating system A single instance of the operating system is shared across all of the quads configured in a single system node There are three levels of Dynix ptx that can be installed on the NUMA Q system These are Dynix ptx V4 4 10 Dynix ptx V4 5 3 or Dynix ptx V4 6 1 Fibre Channel operations are generally the same on all of these operating systems with only some differences in the level of feature support in some areas These limitation differences will be discussed
110. e 4 3 For INRANGE switches you should use the IN VSN Enterprise Manager application to find the same information For McDATA switches you can find the information in the EFC Manager application 102 Introducing hosts to the SAN fabric Example 4 3 Checking the FC HBA connectivity to the switch telnet 9 1 38 58 Trying Connected to 9 1 38 158 Escape character is Fabric OS tm login admin Password Release a2 4 1c itsosw2 admin gt switchShow switchName switchType switchState switchRole switchDomain switchId switchWwn switchBeacon id id id id id id port 0 port port port port port port port 7 Mm 0 BB WN re itsosw2 3 4 Online Subordinate 2 fffc02 10 00 00 60 69 20 OFF Online No Light No Module No Light Online Online No Module Online E Port F Port F Port F Port itsosw2 admin gt quit Connection closed ld 74 10 00 00 60 69 20 1d 4e itsoswl upstream 50 00 1f e1 00 0a e4 02 10 00 00 00 c9 20 d2 1f 50 06 0b 00 00 07 81 de From Example 4 3 we see that the port we are using port 7 is online it is an F_Port for fabric attachment and the WWN listed is the same that we see on our adapter in Example 4 2 12 Your adapter is now correctly installed 4 5 3 Installing the storage system After the adapter installation we have to create any necessary switch zones and allocate the storage for our host from the storage system
111. e Channel HBAs However it is highly recommended to update your current installed driver to the latest version Installation procedure 1 Shut down the system 2 Install JNI adapter card in the host system 3 Turn on the system 4 Windows 2000 automatically detects a new added Fibre Channel HBAs and installs the default JNI driver already built in Windows itself 5 Right click My Computer icon and select Properties 6 Select Hardware tab and click Device Manager 7 Double click the SCSI and RAID controllers to expand the view 8 Double click JNI 1160 Fibre Channel Adapter and the click the Driver tab 9 Click Update Driver The Upgrade Device Driver Wizard appears 10 Click Next to start the update operation 11 Select Display a list of the known drivers for this device and then click Next 12 Click Have Disk 13 Enter the path to the Driver s location and click OK 14 Select appropriate JNI FCE adapter and click Next Windows 2000 indicates that the wizard is ready to install the device 15 Click Next 16 1f the Digital Signature Not Found dialog box appears click Yes 17 Click Finish 18 Click Yes to restart the system Note To see storage targets under Windows 2000 you must install the EZ Fibre Utility in order to map the target storage device see JNI configuration utility on page 22 Chapter 1 Windows Fibre Channel implementation 21 1 5 9 Installing JNI Fibre Channel host bus adapters in Windo
112. e Compaq AlphaServer console if required by the host to place the console in diagnostic mode Chapter 6 Compag AlphaServer Fibre Channel Implementation 137 138 4 Type wwidmgr show adapter command to confirm storage attachment as shown in Example 6 4 Example 6 4 Storage connectivity check P00555set mode diag Console is in diagnostic mode P00 gt gt gt wwidmgr show wwid 0 UDID 1 WWID 01000010 6000 1fe1 4942 4d20 0000 0000 28b1 5660 ev none 1 UDID 1 WWID 01000010 6000 1fe1 4942 4d20 0000 0000 2881 5660 ev none 2 UDID 1 WWID 01000010 6000 1fe1 4942 4d20 0000 0000 2821 5660 ev none P00555 Example 6 4 shows the information about the newly added ESS volumes that you can see at the AlphaServer console Note that the UDID for each volume appears as 1 signifying that UDID is undefined The third and fourth quartet of the UDID number is 4942 4d20 and represents a hex string that identifies an ESS volume The underscore in Example 6 5 highlights an example of a hex string that identifies the decimal volume number of the ESS volume Example 6 5 Decimal volume number for an ESS volume 0 UDID 1 WWID 01000010 6000 1fe1 4942 4d20 0000 0000 28b1 5660 ev none The last five characters of UDID number are the hex representation of the ESS volume serial number In order to verify the attachment of the new added volumes use the hwmgr command as shown in Example 6 6 Example 6 6 Verifying disk attachme
113. e Fibre Channel host bus adapters supported on Linux by their manufacturers are listed in Table 7 1 There are also storage system specific limitations on which adapters are supported Refer to the documentation for your storage system Table 7 1 FC HBAs supported in Linux mirceso iii presen frees se Pe frees fe JNI FCE2 6412 7 2 1 QLogic The current QLogic driver for Linux supports Linux kernel versions 2 2 16 and 2 4 x The complete driver is distributed as source code You can find more information on QLogic adapters as well as drivers for them at http www qlogic com Chapter 7 Linux on i386 platform Fibre Channel implementation 141 7 2 2 Emulex 7 2 3 JNI The current Emulex driver for Linux supports Linux kernel versions 2 2 19 2 4 2 2 4 3 and 2 4 4 It also supports both FCP and IP protocols on a single Fibre Channel adapter simultaneously We do not discuss the IP functionality You can find more information on Emulex adapters as well as drivers for them at http www emulex com The current JNI drivers for Linux supports only Linux kernel versions 2 2 12 and 2 2 14 Since the drivers are only distributed in binary format it is not possible to get the current drivers to work on current 2 4 x kernels For that reason we do not test installation of JNI adapters You can find more information on JNI adapters as well as drivers for them at http www jni com 7 3 SAN fabric Linux supports
114. e INRANGE Portfolio SG24 6150 Designing an IBM Storage Area Network SG24 5758 Implementing an Open IBM SAN SG24 6116 Introduction to Storage Area Network SAN SG24 5470 IBM Storage Solutions for Server Consolidation SG24 5355 Implementing the Enterprise Storage Server in Your Environment SG24 5420 Implementing Linux with IBM Disk Storage SG24 6261 Storage Area Networks Tape Future In Fabrics SG24 5474 IBM Enterprise Storage Server SG24 5465 Other resources These publications are also relevant as further information sources gt Building Storage Networks ISBN 0072120509 These IBM publications are also relevant as further information sources gt ESS Web Interface User s Guide for ESS Specialist and ESS Copy Services SC26 7346 IBM Storage Area Network Data Gateway Installation and User s Guide SC26 7304 Copyright IBM Corp 2002 175 176 gt gt IBM Enterprise Storage Server Configuration Planner SC26 7353 IBM Enterprise Storage Server Quick Configuration Guide SC26 7354 IBM SAN Fibre Channel Managed Hub 3534 Service Guide SY27 7616 IBM Enterprise Storage Server Introduction and Planning Guide 2105 Models E10 E20 F10 and F20 GC26 7294 IBM Enterprise Storage Server User s Guide 2105 Models E10 E20 F10 and F20 SC26 7295 IBM Enterprise Storage Server Host Systems Attachment Guide 2105 Models E10 E20 F10 and F20 SC26 7296 IBM Enterprise Storage Server SCSI Command Reference 21
115. e example installation which follows will be small but your installation may be quite large Items to consider are the location of your system the storage that is being connected and the switch es which you are connecting them to It is best to map out the entire layout prior to starting any work You will want to know the power requirements along with distance and routing of all fiber optic cables you will use Chapter 3 Sun Fibre Channel implementation 75 3 5 1 Cabling requirements and considerations Cables must be purchased to connect these all together or possibly an installer will need to be called especially if running between floors of a building Ready made up cable lengths are available from some vendors for most common installations With fibre cables a point to remember is that attenuation is your worst enemy it can cause your entire network to be highly unstable or not to function at all Watching your number of connections ensuring clean ends and proper seating is very critical to your success Do not try to stretch the maximum distances of the signal In Table 3 12 we show the cable types and their maximum limits Table 3 12 Common Fibre Channel cabling configurations GBIC type Cable type Distance 1 Gb s Distance 2 Gb s Ensure that any cables you buy or have made are constructed with the correct connector ends to match with your switch ports and FC HBA either SC or LC 3 6 Installation procedu
116. e trademarks of Microsoft Corporation in the United States and or other countries PC Direct is a trademark of Ziff Communications Company in the United States and or other countries and is used by IBM Corporation under license ActionMedia LANDesk MMX Pentium and ProShare are trademarks of Intel Corporation in the United States and or other countries UNIX is a registered trademark in the United States and other countries licensed exclusively through The Open Group SET SET Secure Electronic Transaction and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC Other company product and service names may be trademarks or service marks of others Introducing hosts to the SAN fabric Preface In this IBM Redbook we aim to be short sharp and to the point and provide an answer There is no marketing no sales pitch no extraneous information other than what is needed to get the job done That job is making hosts Storage Area Network SAN ready To this end we show how to make hosts SAN fabric ready and connect them to the fabric with minimum fuss We cover all SAN switched fabric capable hosts For background reading on SAN we suggest these books as relevant IBM SAN Survival Guide SG24 6143 IBM SAN Survival Guide Featuring the IBM 2109 SG24 6127 IBM SAN Survival Guide Featuring the McDATA Portfolio SG24 6149 IBM SAN Survival Guide Featuring the INRANGE Portfolio SG24 6150 Designing an IBM
117. ection scheme that allows multiple Fibre Channel ports to communicate For example point to point Arbitrated Loop and switched fabric are all Fibre Channel topologies T_Port An ISL port more commonly known as an E_Port referred to as a Trunk port and used by INRANGE TL_Port A private to public bridging of switches or directors referred to as Translative Loop Twinax A transmission media cable consisting of two insulated central conducting leads of coaxial cable Twisted Pair A transmission media cable consisting of two insulated copper wires twisted around each other to reduce the induction thus interference from one wire to another The twists or lays are varied in length to reduce the potential for signal interference between pairs Several sets of twisted pair wires may be enclosed in a single cable This is the most common type of transmission media ULP Upper Level Protocols UTC Under The Covers a term used to characterize a subsystem in which a small number of hard drives are mounted inside a higher function unit The power and cooling are obtained from the system unit Connection is by parallel copper ribbon cable or pluggable backplane using IDE or SCSI protocols UTP Unshielded Twisted Pair Virtual Circuit A unidirectional path between two communicating N_Ports that permits fractional bandwidth WAN Wide Area Network A network which encompasses inter connectivity between devices over a wide geo
118. elect Start Settings and Control Panel 5 Double click the SCSI Adapters icon The SCSI Adapter window displays 6 Select the Drivers tab and then click Add 7 Click Have Disk 8 Enter the path to the Driver s location and click OK 9 Select the driver and select OK to install necessary files 10 Click Yes to restart the system Introducing hosts to the SAN fabric 1 5 4 QLogic configuration utility QLogic provides a few utilities which can help in configuring and monitoring Fibre Channel adapter cards We cover QLview which provides configuration information statistics diagnostics and alarm notification for adapter as well as for attached storage devices You can download it from the following Web site http www qlogic com In this section we discuss the QLview stand alone configuration used for monitoring QLogic adapters locally The installation procedure is straightforward QLview for Fibre Setup is a self extracting program that installs the QLview for Fibre application and related software Just double click the QLview Fibre Setup icon and follow the wizard s instructions More detail with respect to this can be found at the QLogic Web site After a successful installation run the QLview utility and the following screen appears as shown in Figure 1 4 Set Up New Connection 2 x Enter a machine name or select a name from the Machine Name list For RPC type use DCE for Windows and ONC for Novell Netware
119. em These need to be uncompressed and untarred prior to installation Commands for these steps are uncompress filename tar Z tar xvf filename tar The result is the pure driver files you will need to install possibly in a sub directory structure We are now ready to install the driver on our system 3 6 6 Installi ng the Fibre Channel adapter driver s In this section we are installing the FC HBA driver for the adapter The steps here a re for the JNI 6460 but are generally the same for all the driver vendors To install the Fibre Channel adapter drivers we performed the following steps 1 2 If you have not already done so login as root Change directory to the jnitemp or whatever temp directory you used for your adapter s driver Create a subdirectory tree under opt Drivers called JNI JNIC146x or whatever name is appropriate for your adapter s driver Chapter 3 Sun Fibre Channel implementation 83 84 mkdir opt Drivers JNI mkdir opt Drivers JNI JNIC146x Copy the device driver file to the newly created opt Drivers JNI JNIC146x directory cp jnitemp JNIC146x pkg opt Drivers JNI JNIC146x Now we install the driver with the following command and answer the script questions as needed pkgadd d opt Drivers JNI JNIC146x JNIC146x pkg Note An installation script displays that prompts you to answer a number of questions For each question enter the appropriate
120. embedded multimode shortwave optical interface Introducing hosts to the SAN fabric Soeur camel pala 100 ar thee AG lv N KIN S ih A f 5 u Le oo o we oa y i koi S Ee a Es nr o 0000 Eo ii tc a Figure 6 1 Compaq StorageWorks 64 Bit 33MHz PCI to FC HBA The Compag FC HBA supports gt Link bandwidth providing a total throughput of 2 Gb s gt Point To Point Arbritated Loop and switch fabric FC connections gt SC type shortwave connection The number of FC HBAs per server varies and it depends on the AlphaServer model as well as on FC network topology FC AL or switch fabric All the related details can be found in the manual 64 Bit PCI to Fibre Channel Host Bus Adapter Release Notes for Tru64 UNIX and OpenVMS AV RLLUA TE which can be found at the following Web site http www compaq com products storageworks adapters html The FC HBA drive is already built in Tru64 UNIX operating systems and supported by Console software It is recommended to download the latest firmware level in order to ensure that you are at the most current maintenance level 6 3 Supported SAN switches The Compaq AlphaServer systems running the Tru64 UNIX operating system support only a few SAN switch models which we cover in this section gt IBM 2109 S08 gt IBM 2109 S16 gt Corresponding Brocade 8 and 16 ports switches gt Corresponding Compaq
121. embeddedPortBcast 1 route stickyRoutes 1 rpc rapid 1 rpc rstatd 0 rpc rusersd 0 shell delete 0 shell quiet 0 shell timeout 0 syslog errDisplayLevel 5 syslog errsOnConsole 1 thresh alarmsfilter 1 thresh thad 1 xlativeModeDisable 0 Displaying enabled disabled portlog events Type lt CR gt to continue Q lt CR gt to stop q 12 We now need to check to ensure we are loaded with the proper switch firmware This we do by running the version command on the switch telnet session SNJ2109DD2 admin gt version Kernel 5 3 1 Fabric OS a2 2 1la Made on Tue May 8 15 44 06 PDT 2001 Flash Tue May 8 15 45 24 PDT 2001 BootProm Thu Jun 17 15 20 39 PDT 1999 Chapter 5 NUMA Q Fibre Channel implementation 121 13 We now need to connect the cables ensuring that we are careful not to touch the ends and making sure they are clean of any dirt or dust We seat them securely into the GLM of the FC HBA and the GBIC of the switch port By issuing the switchShow command we can see these switch details SNJ2109DD2 admin5 switchShow switchName SNJ2109DD2 switchType 2 4 switchState Online switchRole Principal switchDomain 1 switchId fffc01 switchWwn 10 00 00 60 69 12 3f fc switchBeacon OFF port 0 sw No Light port 1 sw No Light port 2 id No Light port 3 sw No Light port 4 No Module port 5 No Module port 6 No Module port 7 No Module port 8 No Module port 9 No Module por
122. emove the line D__SMP__ 1 from the CFLAGS variable Check the parameter Ipfc_topology in the file Ipfc conf c and change it if necessary Note By default the value of Ipfc topology is set to 4 That means that the adapter will only try loop mode login and not point to point mode We change the value to 6 meaning that the adapter will try first point to point mode and then loop mode if point to point mode fails Build the 1pfcdd and 1pfndd loadable kernel modules root clyde SourceBuild make build Build Environment root usr src linux 2 4 cc D GENKSYMS D KERNEL 1 DMODULE DMODVERSIONS include usr src linux 2 4 include linux modversions h I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include DLP6000 D LINUX I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include E fcLINUXfcp c gt Ipfc verl cat Ipfc verl sbin genksyms k 2 2 5 gt Ipfc ver cc Wall 02 fomit frame pointer D KERNEL 1 DMODULE DMODVERSIONS include usr src linux 2 4 include linux modversions h I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include DLP6000 D LINUX I include I usr src linux 2 4 drivers scsi I usr src linux 2 4 include scsi I usr src linux 2 4 include c fcLINUXfcp c rm f Ipfc verl Introducing hosts to the SAN fabric cc Wall 02 fomit frame pointer D KERN
123. endors support your FC HBA A large selection of adapters are supported by some vendors with only a few being documented as supported by all We have listed the vendors and their respective Web site URLs gt QLogic Corporation http www qlogic com products fc_san_hostadapers asp gt Emulex Corporation http www emulex com products fc index htm gt JNI Corporation http www jni com Products hba cfm gt Antares Microsystems http www antares com fibre_channel fibre_channel htm gt Genroco Inc http www genroco com products fcnet index html gt ATTO Technology Inc http www attotech com fcha html At the time of writing IBM supports the following JNI QLogic and Emulex FC HBAs with which IBM has tested our products with the Sun platforms gt Emulex LP8000 PCI adapter This requires the latest firmware and driver from Emulex In Figure 3 1 we show this adapter It supports Open Boot software on Solaris All downloads including documentation are available at the following URL http www emulex com ts fc docs frame8kd htm Introducing hosts to the SAN fabric 12 RERRDR 13 HANAN Pd a D 4 a N Ri N Busjanem uous paw 901 H1 MA NG901 0S panung jeondo jeuas Ok Figure 3 1 Emulex LP8000 adapter gt JNI FCI 1063 PCI adapter This adapter requires the latest driver from JNI There is no firmware for this adapter The driver download file is available at the fol
124. er pSeries Fibre Channel implementation 55 56 of Telnet sicily Connect Edit Terminal Help Update Installed Software to Latest Level Update All Type or select a value for the entry field Press Enter AFTER making all desired changes Entry Fields LAINPUT device directory for software F2 Refresh F3 Cancel F4 List Esc 6 Command Esc 7 Edit Esc 8 Image Esc Exit Enter Do Figure 2 16 Update storage drivers select install medium i Type in your update directory or device insta11 aix43309 for example and press Enter You see a screen like the one in Figure 2 17 oh Telnet sicily Connect Edit Terminal Help Update Installed Software to Latest Level Update A11 Type or select values in entry fields Press Enter AFTER making all desired changes Entry Fields INPUT device directory for software install1 aix43369 SOFTWARE to update update_all PREVIEW only update operation will NOT occur fo COMMIT software updates yes SAVE replaced files no AUTOMATICALLY install requisite software yes EXTEND file systems if space needed yes VERIFY install and check file sizes no DETAILED output no Process multiple volumes yes e bh be be be be F1 Help F2 Refresh F3 Cancel F4 List Esc 5 Reset Esc 6 Command Esc 7 Edit Esc 8 Image Esc 9 Shell Esc 6 Exit Enter Do Figure 2 17 Update storage drivers installation settings j Accept the screen by pressing Enter Introducing hosts to the SAN fa
125. fiber optic cable commonly used in Fibre Channel environments as described in Table 1 2 Table 1 2 Common Fibre Channel cabling configurations GBIC type Cable type Distance 1 Gb s Distance 2 Gb s Shortwave GBICs and 50 micron cable are usually used to connect devices to the fabric since most device adapters only support shortwave GBICs as well as for short inter switch links ISLs Longwave GBICs and 9 micron cable are used when required for extended distance 6 Introducing hosts to the SAN fabric Attenuation Fibre Channel is a high speed communication technology Because of this it is very sensitive to link attenuation The attenuation is even more of a critical factor when connection speeds rise from 1 Gb s to 2 Gb s and even higher To limit the effects of attenuation we recommend that you Have as few connections as possible on a single link Make sure that all connectors are clean Make sure that all connectors are properly seated Make sure that you only use one type of fiber on a single link never mix 50 micron and 62 5 micron fiber YYY Yy Zoning We strongly recommend that you use zoning to gt Enhance the security of your Fibre Channel fabric gt Avoid interactions between different hosts gt Secure the storage devices that do not support LUN masking like tape drives 1 3 2 The IBM 2109 family Current members of the IBM 2109 family of switches are described in Table 1 3 Table 1 3 The IBM 2109
126. ge ibm com hardsoft products tape ro3superserver htm IBM SAN Data Gateway Router http www storage ibm com hardsoft products fcss fcss htm IBM Fibre Channel RAID Storage Server http www storage ibm com hardsoft products ess ess htm Enterprise Storage Server http www 1 ibm com servers eserver pseries library hardware_docs index html IBM eServer pSeries and RS 6000 Hardware Documentation http www techsupport services ibm com server fixes view pSeries IBM eServer pSeries and RS 6000 Support http www techsolutions hp com Hewlett Packard Support http www software hp com Hewlett Packard Software Depot http www redhat com Red Hat Inc http www suse com SuSE Inc http www caldera com Caldera International Inc http www turbolinux com Turbolinux Inc http www cdp com Columbia Data Products Inc http www emulex com Emulex Corporation http www fibrechannel com Fibre Channel Industry Association Introducing hosts to the SAN fabric http www jni com JNI Corporation http www mcdata com McDATA Corporation http www pathlight com Pathlight http www qlogic com QLogic Corporation http www sanergy com Tivoli SANergy http www snia org Storage Networking Industry Association http wwwhttp www tivoli com Tivoli http www tll org Technical Committee T11 http www vicom com Vicom Systems http www vixel com Vixel http www scsita org SCSI Trade Ass
127. ght rays or modes concurrently each at a slightly different reflection angle within the optical core Multi Mode fiber transmission is used for relatively short distances because the modes tend to disperse over longer distances See also Single Mode Fiber SMF Multicast Sending a copy of the same transmission from a single source device to multiple destination devices on a fabric This includes sending to all N_Ports on a fabric broadcast or to only a subset of the N_Ports on a fabric multicast Multi Mode Fiber MMF In optical fiber technology an optical fiber that is designed to carry multiple light rays or modes concurrently each at a slightly different reflection angle within the optical core Multi Mode fiber transmission is used for relatively short distances because the modes tend to disperse over longer distances See also Single Mode Fiber Multiplex The ability to intersperse data from multiple sources and destinations onto a single transmission medium Refers to delivering a single transmission to multiple destination Node Ports N_Ports N_Port Node Port A Fibre Channel defined hardware entity at the end of a link which provides the mechanisms necessary to transport information units to or from another node N_Port Login N_Port Login PLOGI allows two N_Ports to establish a session and exchange identities and service parameters It is performed following completion of the fabric login process and prior to the
128. gital Audio Tape A tape media technology designed for very high quality audio recording and data backup DAT cartridges look like audio cassettes and are often used in mechanical auto loaders typically a DAT cartridge provides 2GB of storage But new DAT systems have much larger capacities Data Sharing A SAN solution in which files on a storage device are shared between multiple hosts Datagram Refers to the Class 3 Fibre Channel Service that allows data to be sent rapidly to multiple devices attached to the fabric with no confirmation of delivery dB Decibel a ratio measurement distinguishing the percentage of signal attenuation between the input and output power Attenuation loss is expressed as dB km Disk Mirroring A fault tolerant technique that writes data simultaneously to two hard disks using the same hard disk controller Disk Pooling A SAN solution in which disk storage resources are pooled across multiple hosts rather than be dedicated to a specific host DLT Digital Linear Tape A magnetic tape technology originally developed by Digital Equipment Corporation DEC and now sold by Quantum DLT cartridges provide storage capacities from 10 to 35GB E Port Expansion Port a port on a switch used to link multiple switches together into a Fibre Channel switch fabric ECL Emitter Coupled Logic The type of transmitter used to drive copper media such as Twinax Shielded Twisted Pair or Coax Enterprise Net
129. graphic area A wide area network may be privately owned or rented but the term usually connotes the inclusion of public shared networks WDM Wave Division Multiplexing A technology that puts data from different sources together on an optical fiber with each signal carried on its own separate light wavelength Using WDM up to 80 and theoretically more separate wavelengths or channels of data can be multiplexed into a stream of light transmitted on a single optical fiber WEBM Web Based Enterprise Management A consortium working on the development of a series of standards to enable active management and monitoring of network based elements Zoning In Fibre Channel environments the grouping together of multiple ports to form a virtual private storage network Ports that are members of a group or zone can communicate with each other but are isolated from ports in other zones Glossary 173 174 Introducing hosts to the SAN fabric Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook IBM Redbooks For information on ordering these publications see How to get IBM Redbooks on page 180 gt gt gt IBM SAN Survival Guide SG24 6143 IBM SAN Survival Guide Featuring the IBM 2109 SG24 6127 IBM SAN Survival Guide Featuring the McDATA Portfolio SG24 6149 IBM SAN Survival Guide Featuring th
130. gured on the ESS for access to the NUMA Q system We are also using Dynix ptx V4 5 3 to perform this installation Any special instructions which may differ from our performed steps for the other supported releases of the operating system are noted as we progress To start with it is assumed that this is not a new installation and that the Fibre Channel we are installing is an add on to the platform hardware The system is powered up and running the operating system of choice It is also assumed that the system has been configured for LAN and TCP IP support and has the supported firmware package installed Now we are ready to start installing our fabric 5 6 1 Verifying the correct software is installed 1 Before we start our install we make one last check to ensure the proper software is really on the system This we do by running cat etc versionlog Sun Jan 14 17 42 33 PST 2001 DYNIX ptx R Base V4 5 3 PN 1003 77788 03_A00 Sun Jan 14 17 42 45 PST 2001 DYNIX ptx R Encryption V4 5 3 PN 1003 77790 03_A00 Chapter 5 NUMA Q Fibre Channel implementation 117 Sun Jan 14 17 45 51 1003 77789 03 A00 Sun Jan 14 17 45 58 Sun Jan 14 17 46 32 Sun Jan 14 17 47 48 Sun Jan 14 17 50 57 Sun Jan 14 17 53 00 Sun Jan 14 17 53 02 1003 78507 02 A00 Sun Jan 14 17 54 08 1003 76392 02 A00 Sun Jan 14 17 54 27 Sun Jan 14 18 05 59 Sun Jan 14 18 15 56 Sun Jan 14 18 16 28 Sun Jan 14 18 18 33 Sun Jan 14 18 19 50 1003 79203 00 A02 Sun Jan 1
131. gwave singlemode GBIC depending on customer requirements For more information see IBM SAN Survival Guide Featuring the INRANGE Portfolio SG24 6150 2 3 4 McDATA The McDATA switches currently supported by IBM the IBM 2031 and 2032 families are described in Table 2 6 38 Introducing hosts to the SAN fabric Table 2 6 The McDATA switches moan TPons Sreed_ connectors Rackapace 2031 016 McDATA ES 3016 16 1 Gb s LC 1U Fabric Switch 2031 032 McDATA ES 3032 32 1 Gb s LC 1 5U Fabric Switch 2031 L00 McDATA ES 1000 8 1 1 Gb s SC 1U Loop Switch 2032 001 McDATA ED 5000 32 1 Gb s SC 18U Enterprise Fibre Channel Director 2032 064 McDATA ED 6064 1 Gb s Enterprise Fibre Channel Director In addition IBM also sells the McDATA FC 512 Cabinet 2032 C36 Features and limitations The only McDATA switch that supports loop attached devices tape drives for example is the McDATA ES 1000 Loop Switch The ports in the 2032 001 and 2032 064 can be added in cards The cards are not compatible between the models Each card contains four ports There are three different cards available gt Four shortwave multimode ports gt Four longwave singlemode ports gt Three shortwave ports and one longwave port On the 2031 016 and 2031 032 switches any port can be independently configured with a shortwave multimode or longwave singlemode GBIC The 2031 L00 switch has two different types of ports gt Eight switch
132. h Select a specific QLogic adapter On the right side of the window you can find the adapter WWN as well as BIOS firmware and driver version See Figure 1 7 Chapter 1 Windows Fibre Channel implementation 15 QLview for Fibre E 5 x File Machine Connection Options amp Help a Local Windows Pt OLA2200 21 00 00 E0 8B 01 90 EF Local Serial Number A61328 8 00 08 W2K IP PCI Slot Number Information amp Statistics Diagnostics amp Utilities Figure 1 7 QlLview for Fibre utility adapter information In addition to the QLview for Fibre Utility there is also the QLogic Fast Utility which is used for changing adapter configuration settings For instance if you want to boot from an FC disk drive attached to the adapter use the QLogic Fast Utility in order to enable it You can invoke it by pressing the Alt and Q keys when requested during Windows boot We recommended that you carefully read the QLogic Fast Utility user guide before changing adapter settings 1 5 5 Installing Emulex Fibre Channel HBAs in Windows 2000 This section lists the steps you have to follow in order to install the card and appropriate driver on Windows 2000 operating systems Windows 2000 Service Pack 2 is recommended Installation procedure 1 Shut down the system Install the Emulex Fibre Channel HBAs in the host system Turn on the system A window displays Found New Hardware Wizard Click Next a
133. h card contains four ports There are three different cards available Four shortwave multimode ports Four longwave singlemode ports Three shortwave ports and one longwave port On the 2031 016 and 2031 032 switches any port can be independently configured with a shortwave multimode or longwave singlemode GBIC The 2031 L00 switch has two different types of ports Eight switching Hub ports that can only be configured with shortwave multimode GBIC or copper interface One bridge port for connecting the loop switch to the fabric that can be configured with a shortwave multimode or longwave singlemode GBIC The McDATA switches support fabrics of up to 31 switches only Chapter 3 Sun Fibre Channel implementation 73 3 3 4 Fibre Channel switch installation preparation If you are installing the Sun system on a new unused Fibre Channel switch you should set up the fabric as a single zone which allows you to familiarize yourself with the SAN environment before expanding it If however you are adding the Sun to an already existing fabric you may need to configure your switch to work with multiple hosts and different vendors and operating systems For this more planning is required Using zones with the switches When you are sharing a switch with other hosts systems in a heterogeneous environment you will need to know what the requirements of all the host vendors are with respect to sharing
134. hapter 1 Windows Fibre Channel implementation 27 28 5 Select a newly added disk in order to write a signature as shown in Figure 1 16 Write Signature and Upgrade Disk Wizard Select Disk to Write Signature Choose the disks on which you want to write a signature Select the disks on which you want to write a signature Disk 1 lt Back Cancel Figure 1 16 Select Disk 6 Click Next Introducing hosts to the SAN fabric 7 Click Finish to complete the Write Signature task as shown in Figure 1 17 Write Signature and Upgrade Disk Wizard J x Completing the Write Signature and Upgrade Disk Wizard You have successfully completed the Write Signature and Upgrade Disk Wizard You have selected these settings Write a signature on the following disks Disk 1 Upgrade the following disks None To close this wizard click Finish lt Back Cancel Figure 1 17 Completing the Write Signature task Chapter 1 Windows Fibre Channel implementation 29 8 At that point Windows 2000 recognizes the newly added storage devices and it is able to perform a volume format as shown in Figure 1 18 ERC yrr Action View e gt amle 2 e a Trea Computer Management Local Partition i System Tools Partition fa Event Viewer Partition F Partition System Information 5G Partition amp Performance Logs and A Partiti artition J J Shared Folders Partition A Dev
135. hat reserves the full Fibre Channel bandwidth for a dedicated Class 1 connection but also allows connection less Class 2 traffic to share the link if the bandwidth is available 1 O Input output IP Internet Protocol IPI Intelligent Peripheral Interface Isochronous Transmission Data transmission which supports network wide timing requirements A typical application for isochronous transmission is a broadcast environment which needs information to be delivered at a predictable time JBOD Just a bunch of disks Jukebox A device that holds multiple optical disks and one or more disk drives and can swap disks in and out of the drive as needed L_Port Loop Port A node or fabric port capable of performing Arbitrated Loop functions and protocols NL_Ports and FL_Ports are loop capable ports LAN See Local Area Network A network covering a relatively small geographic area usually not larger than a floor or small building Transmissions within a Local Area Network are mostly digital carrying data among stations at rates usually above one megabit s Latency A measurement of the time it takes to send a frame between two locations LC Lucent Connector A registered trademark of Lucent Technologies Link A connection between two Fibre Channel ports consisting of a transmit fibre and a receive fibre Link_Control_Facility A termination card that handles the logical and physical control of the Fibre Channel link for each mode
136. her and V4 6 1 the FC HBA V4 4 4 and higher EMC CLARiiON DAE Direct FC AL direct attached Supported with Attach Enclosure JBOD from the FC HBA only DYNIX ptx V4 5 1 and Just a Bunch Of Disk higher and V4 6 1 V4 4 7 and higher 114 Introducing hosts to the SAN fabric In addition to the above standard supported list of disk devices there is also a list of Request for Price Quotation RPQ disk devices that are available These are listed in Table 5 4 Table 5 4 IBM NUMA Q RPQ Fibre Channel attached disk storage systems Storage system Method of attachment Operating systems IBM 2105 Enterprise Storage Switched or FC AL DYNIX ptx V4 4 7 8 9 Server ESS direct attached from the FC HBA HDS Freedom 9960 Switched or FC AL direct Supported with attached from the FC HBA DYNIX ptx V4 5 2 and higher and V4 6 1 V4 4 7 and higher HDS Freedom 9200 Series Switched or FC AL direct Supported with attached from the FC HBA DYNIX ptx V4 5 2 and higher and V4 6 1 V4 4 7 and higher HDS 7700E Switched or FC AL direct Supported with attached from the FC HBA DYNIX ptx V4 5 1 and higher and V4 6 1 V4 4 4 and higher HDS 5800 Series Switched or FC AL direct Supported with attached from the FC HBA DYNIX ptx V4 5 1 and higher and V4 6 1 V4 4 4 and higher HDS 5700F Switched or FC AL direct Supported with attached from the FC HBA DYNIX ptx V4 5 1 and higher and V4 6 1 V4 4 4 and higher Table 5 5 describe
137. ice Manager Partition 4G Local Users and Groups 089 Storage J Disk Management Disk Defragmenter amp Logical Drives Removable Storage H Services and Applications muni Figure 1 18 Disk Management disk 1 unallocated and unformatted Windows NT environment These are the steps for the Windows NT environment Select Start Program Administrative Tool and Disk Administrator and the information appears as shown in Figure 1 19 Disk Administrator Ki N Disk Administrator has determined that the configuration of one of more disks has been altered since Disk Administrator was last run System configuration will be automatically updated to reflect these changes when you next opt to save changes when exiting Disk Administrator Figure 1 19 Disk Administrator change warning 2 Click OK 30 Introducing hosts to the SAN fabric 3 Windows NT reco Figure 1 20 4 At this No hh volume ass gnizes the newly added storage device as shown in lo Ss r more partitions on the newly added n Fi ba aid pr Mii 5 Your newly added volume has a status as shown in Figure 1 22 a Disk Administrator Eee Figure 1 22 After creation of partitions 6 Prior to the format volume task you should commit all changes which have been made as shown in Figure 1 23 d Disk Administrator EZET Commit Changes Now B Figure 1 23 Commit changes
138. ine To check if your FC HBA is properly connected to switch port type the swi tchShow command as shown in Example 6 3 Introducing hosts to the SAN fabric Example 6 3 Switch connectivity check telnet 9 1 Trying Connected to 38 58 9 1 38 158 Escape character is Fabric OS tm login admin Password Release a2 4 1c itsosw2 admin gt switchShow switchName switchType switchState switchRole switchDomain switchId switchWwn switchBeacon id id id id id id port 0 port port port port port port port 7 Mm 0 BB WN re itsosw2 3 4 Online Subordinate 2 fffc02 10 00 00 60 69 20 OFF Online No Light No Module No Light Online Online No Module Online E Port F Port F Port F Port itsosw2 admin gt quit Connection closed ld 74 10 00 00 60 69 20 1d 4e itsoswl upstream 50 00 1f e1 00 0a e4 02 10 00 00 00 c9 20 d2 1f 50 06 0b 00 00 07 81 de The online status of the specific port fabric attachment F_Port and FC HBA WWN next to it are the confirmation that the FC HBA has performed a fabric login to the switch Storage connectivity check At that point it is assumed that volumes on the disk subsystem are created and assigned to AlphaServer The following procedures describe how to confirm storage connectivity 1 Restart the host 2 3 Bring the host system to a halt condition at the console level Type set mode diag at th
139. information 16 1 8 Emulex LP8000 Fibre Channel HBAs Status LEDs 18 1 9 Light Pulse utility main window ce eee eee 19 1 10 LightPulse Utility Category list 0 0 2 0 e ee eee 19 1 11 LightPulse Utility Firmware download 2 0 055 20 1 12 IBM 2109 Switch configuration main menu 255 25 1 13 IBM 2109 S16 Switch Management Application 25 1 14 IBM 2109 S16 Port Information 26 1 15 Write Signature and Upgrade Disk Wizard 27 1 16 Select DEK Lana al Beate Meme LAT daa eee Se Sank ees 28 1 17 Completing the Write Signature task aaa 29 1 18 Disk Management disk 1 unallocated and unformatted 30 1 19 Disk Administrator change warning eee eee eee 30 1 20 New device recognized 0 ee ees 31 1 21 Creating partitions 2 2 eee 31 1 22 After creation of partitions 0 cee ee 32 1 23 Commit changes RE Ea R E a a EEA eee 32 2 1 IBM eServer pSeries Support eee eee 41 2 2 Install additional device software select install medium 43 2 3 Install additional device software initial settings 43 2 4 Install additional device software choose software package 44 2 5 Install additional device software installation settings 45 2 6 Install additional device software inst
140. ing Hub ports that can only be configured with shortwave multimode GBIC or copper interface gt One bridge port for connecting the loop switch to the fabric that can be configured with a shortwave multimode or longwave singlemode GBIC The McDATA switches support fabrics of up to 31 switches only For more information see IBM SAN Survival Guide Featuring the McDATA Portfolio SG24 6149 Chapter 2 eServer pSeries Fibre Channel implementation 39 2 4 Storage The IBM Fibre Channel attached storage systems supported on eServer pSeries and RS 6000 machines are described in Table 2 7 and Table 2 8 Table 2 7 IBM Fibre Channel attached disk storage systems IBM TotalStorage Enterprise Storage Server IBM TotalStorage FAStT500 Storage Server IBM TotalStorage SAN Controller 160 Only supports IBM 2109 switch fabrics with IBM 7133 disk subsystems Table 2 8 IBM Fibre Channel attached tape storage systems Storage system IBM TotalStorage Enterprise Tape Requires feature 3510 or 9510 on the drive System 3590 IBM 3584 UltraScalable Tape Library Requires feature 1456 or 1466 on the drive Only supports IBM 2109 switch fabrics IBM 3583 Ultrium Scalable Tape Library with SAN Data Gateway Module Feature 8005 Tape drives attached via IBM 2108 G07 IBM Storage Area Network Data Gateway or IBM 2108 R03 IBM Storage Area Network Data Gateway Router Other storage system vendors may also support their storage systems on eSe
141. ing the physical characteristics of the interface and media FC 1 Middle level of the Fibre Channel Physical standard defining the 8B 10B encoding decoding and transmission protocol FC 2 Highest level of the Fibre Channel Physical standard defining the rules for signaling protocol and describing transfer of frame sequence and exchanges FC 3 The hierarchical level in the Fibre Channel standard that provides common services such as striping definition FC 4 The hierarchical level in the Fibre Channel standard that specifies the mapping of upper layer protocols to levels below FCA Fiber Channel Association FC AL Fibre Channel Arbitrated Loop A reference to the Fibre Channel Arbitrated Loop standard a shared gigabit media for up to 127 nodes one of which may be attached to a switch fabric See also Arbitrated Loop FC CT Fibre Channel common transport protocol FC FG Fibre Channel Fabric Generic A reference to the document ANSI X3 289 1996 which defines the concepts behavior and characteristics of the Fibre Channel Fabric along with suggested partitioning of the 24 bit address space to facilitate the routing of frames FC FP Fibre Channel HIPPI Framing Protocol A reference to the document ANSI X3 254 1994 defining how the HIPPI framing protocol is transported via the fibre channel FC GS Fibre Channel Generic Services A reference to the document ANSI X3 289 1996 describing a common transport protocol used
142. ion 43 44 You see a list of software packages similar to Figure 2 4 of Telnet sicily Connect Edit Terminal Help Install Additional Device Software Ty1 qqqqqq99999999999999999999999999999999999999999999999999999999999999999G9GK00k Prx SOFTWA 1 x RE to instal Move cursor to desired item and press F7 Use arrow keys to scroll ONE OR MORE items can be selected Press Enter AFTER making all selections Already installed No license password required x x x x x x x x x x x x x x x MORE 759 x x F1 Help F2 Refresh F3 Cancel Fix F7 Select F8 Image F16 Exit Esx Enter Do Find n Find Next F9mgqqqqqqqqqq99q9q9q999999999999999999999999999q99999999999999999999q9999999999 Figure 2 4 Install additional device software choose software package e Search the correct software package by typing for search and then the name of the software e devices pci df1000f7 for the Gigabit Fibre Channel Adapter f c 6227 e devices pci df1000f9 for the 2 Gigabit Fibre Channel Adapter f c 6228 You should end up on a line similar to the following devices pci df1000f7 ALL f Press F7 to select the software package and press Enter g You get taken back to the previous screen with the entry data field for the software automatically updated as shown in Figure 2 5 Introducing hosts to the SAN fabric of Telnet sicily Connect Edit Terminal Help Install Additional Device Software Type or select values in e
143. it across multiple levels STP Shielded Twisted Pair Storage Media The physical device itself onto which data is recorded Magnetic tape optical disks floppy disks are all storage media Switch A component with multiple entry exit points ports that provides dynamic connection between any two of these points Switch Topology An interconnection structure in which any entry point can be dynamically connected to any exit point In a switch topology the available bandwidth is scalable T11 A technical committee of the National Committee for Information Technology Standards titled T11 I O Interfaces It is tasked with developing standards for moving data in and out of computers Tape Backup Making magnetic tape copies of hard disk and optical disc files for disaster recovery Tape Pooling A SAN solution in which tape resources are pooled and shared across multiple hosts rather than being dedicated to a specific host TCP Transmission Control Protocol a reliable full duplex connection oriented end to end transport protocol running on top of IP TCP IP Transmission Control Protocol Internet Protocol a set of communications protocols that support peer to peer connectivity functions for both local and wide area networks 172 Introducing hosts to the SAN fabric Time Server A Fibre Channel defined service function that allows for the management of all timers used within a Fibre Channel system Topology An interconn
144. it in any detail 2 3 2 The IBM 2109 family The current members of the IBM 2109 family of switches are described in Table 2 4 ne kaa aa IBM TotalStorage SAN Switch S08 eo 1 Gb s se liu Table 2 4 The IBM 2109 family IBM IBM TotalStorage SAN Switch S16 IBM TotalStorage SAN Switch S16 Switch S16 1 Gb s a Gb s Chapter 2 eServer pSeries Fibre Channel implementation 37 Features and limitations The IBM 2109 family of switches support both fabric attached and loop attached devices on any port and switch domain ids 1 239 are supported Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on customer requirements For more information see BM SAN Survival Guide Featuring the IBM 2109 SG24 6127 2 3 3 INRANGE The INRANGE switches currently supported by IBM the IBM 2042 family are described in Table 2 5 Table 2 5 The INRANGE switches woes Tr apn Comes kame 2042 001 INRANGE FC 9000 64 1 Gb s SC 20U Fibre Channel Director 2042 128 INRANGE 1 Gb s FC 9000 128 Fibre Channel Director In addition IBM also sells the INRANGE FC 9000 40U cabinet 2042 C40 The cabinet is able to hold one 128 port switch or two 64 port switches Features and limitations The INRANGE FC 9000 family of switches support both fabric attached and loop attached devices on any port Any port can be independently configured with a shortwave multimode or lon
145. ither a Fabric Port F Port or an Expansion Port E Port The function is automatically determined during login Gateway A node on a network that interconnects two otherwise incompatible networks Gb s Gigabits per second Also sometimes referred to as Gbps In computing terms it is approximately 1 000 000 000 bits per second Most precisely it is 1 073 741 824 1024 x 1024 x 1024 bits per second GB s Gigabytes per second Also sometimes referred to as GBps In computing terms it is approximately 1 000 000 000 bytes per second Most precisely it is 1 073 741 824 1024 x 1024 x 1024 bytes per second GBIC GigaBit Interface Converter Industry standard transceivers for connection of Fibre Channel nodes to arbitrated loop hubs and fabric switches Gigabit One billion bits or one thousand megabits GLM Gigabit Link Module a generic Fibre Channel transceiver unit that integrates the key functions necessary for installation of a Fibre channel media interface on most systems Half Duplex A mode of communications allowing either transmission or reception of frames at any point in time but not both other than link control frames which are always permitted Hardware The mechanical magnetic and electronic components of a system e g computers telephone switches terminals and the like HBA Host Bus Adapter HIPPI High Performance Parallel Interface An ANSI standard defining a channel that transfers data between
146. ition IBM also sells the McDATA FC 512 Cabinet 2032 036 Features and limitations The only McDATA switch that supports loop attached devices tape drives for example is the McDATA ES 1000 Loop Switch The ports in the 2032 001 and 2032 064 can be added in cards The cards are not compatible between the models Each card contains four ports There are three different cards available gt Four shortwave multimode ports gt Four longwave singlemode ports gt Three shortwave ports and one longwave port On the 2031 016 and 2031 032 switches any port can be independently configured with a shortwave multimode or longwave singlemode GBIC Chapter 7 Linux on i386 platform Fibre Channel implementation 145 The 2031 L00 switch has two different types of ports gt Eight switching Hub ports that can only be configured with shortwave multimode GBIC or copper interface gt One bridge port for connecting the loop switch to the fabric that can be configured with a shortwave multimode or longwave singlemode GBIC The McDATA switches support fabrics of up to 31 switches only For more information see IBM SAN Survival Guide Featuring the McDATA Portfolio SG24 6149 7 4 Storage The IBM Fibre Channel attached disk storage systems supported on Linux systems are described in Table 7 6 There is no support available for IBM Fibre Channel attached tape drives on Linux at the moment Table 7 6 IBM Fibre Channel atta
147. king the protocol Isattr El fscsi0 scsi id 0x31e00 Adapter SCSI ID False attach switch How this adapter is CONNECTED False sw_fc_class 3 FC Class for Fabric True You can find the protocol used as the value of the attach parameter You can also see whereabouts in the switch fabric our card is attached to by looking at the hexadecimal scsi_id parameter From the rightmost byte the first two numbers are always OO for a switch attached adapter the third hexadecimal number e tells the port of the switch board the fourth hexadecimal number 1 the number of the switch board within the switch and the remaining one or two hexadecimal numbers 3 the domain id of the switch 13 Optionally check to see that the port is correctly configured on the switch The procedure you need to use is different for each switch vendor We show the procedure for the IBM 2109 family of switches in Example 2 5 For INRANGE switches you should use the IN VSN Enterprise Manager application to find the same information For McDATA switches you can find the information in the EFC Manager application Chapter 2 eServer pSeries Fibre Channel implementation 51 52 Example 2 5 Checking the FC HBA connectivity to the switch telnet 9 1 Trying Connected to 38 159 9 1 38 159 Escape character is Fabric OS tm Release a2 4 1c login admin Password itsosw3 admin gt switchShow switchName switchType switch
148. l parts of IBM eServer product family Unlike other operating systems there is no single version of Linux available Several individuals and companies have created distributions usually containing large number of freely available and in some cases even commercial applications in addition to the base Linux system These distributions are usually distributed on CDs DVDs or over the Internet Some of the major distributions available include Caldera Red Hat SuSE TurboLinux YYY Yy Because of the number of different distributions available storage system vendors cannot test and support every possible configuration Refer to the documentation of your storage system for details on what is supported The hardware support for Linux was originally based on the work done by independent developers working at their own pace and time While the quality of the drivers was generally excellent the lack of support from the hardware manufacturers meant that it was often hard to find drivers for the very latest hardware Some hardware manufacturers even decided not to reveal any technical information about their products making the development of device drivers impossible In the last few years however many hardware manufacturers have started to realize the market potential Linux has and are developing device drivers to enable their hardware to be used in Linux systems Introducing hosts to the SAN fabric 7 2 Fibre Channel host adapters Th
149. ller will be called in if running between floors of a building or over long distances Ready made cable lengths are available for order for most general installations Table 5 7 shows the cable lengths and ordering information for these cables Table 5 7 Ready made fibre optic duplex cables Length Multimode style model Singlemode style model number number 35 meters CBL 3500 F3 CBL 3500 F2 70 meters CBL 7000 F3 CBL 7000 F2 With fibre cables remember that attenuation is your enemy it can cause your entire network to be highly unstable or not to function at all Watching your number of connections ensuring clean ends and proper seating is critical to your success Do not try to stretch the maximum distances of the signal 116 Introducing hosts to the SAN fabric See Table 5 8 for types and maximum limits Table 5 8 Common Fibre Channel cabling configurations GBIC type Cable type Distance 1 Gb s Distance 2 Gb s Ensure that any cables you buy or have made are constructed with the correct connector ends to match your switch ports and FC HBAs All NUMA Q 1Gb s cabling uses the ISC type connectors Note NUMA Q does not support the 2 Gb s Fibre Channel speed 5 6 Installation procedure In this section we run through a simple fabric installation and show the success of the work We are using a two quad NUMA Q system with four FC HBAs installed connected to an IBM ESS using two FC host adapters We have four LUNs confi
150. lowing URL http www jni com Drivers drivers 3 cfm ID 8 JNI FC64 1063 Sbus adapter This adapter requires the latest driver from JNI There is no firmware for this adapter which is shown in Figure 3 2 The driver download file is available at the following URL http www jni com Drivers drivers 3 cfm 1D 10 Chapter 3 Sun Fibre Channel implementation 63 64 Figure 3 2 JNI FC64 1063 adapter gt JNI FCE 6460 PCI adapter This is the latest JNI adapter and is currently being tested by IBM This adapter requires the latest driver from JNI currently v5 1 There is no firmware for this adapter Unlike other adapters this driver requires that the WWN of the targets be defined prior to it probing the fabric for them gt QLogic QLA2200F PCI adapter This adapter requires the latest firmware and driver from QLogic Figure 3 3 is a picture of the this adapter All downloads including documentation are available at the following URL http www qlogic com support product_resources asp id 113 Introducing hosts to the SAN fabric Figure 3 3 QLogic QLA2200F adapter Note QLogic QLA2200F requires driver V3 07 for support 3 2 1 Tuning HBA configuration files With the Solaris operating system there are a number of parameters which must be tuned for proper operation with Fibre Channel usage These parameters are FC HBA dependent but reside in the standard Solaris kernel These parameters are outlined in
151. ls Wakeup Parameters IEEE Address BWN HK 158 Introducing hosts to the SAN fabric Loop Map Status amp Counters Link Status Configuration Parameters ON QM 0 0 Return to Main Menu Enter choice gt 0 MAIN MENU List Adapters Adapter Information Firmware Maintenance Reset Adapter BWN rn 0 Exit Enter choice gt 0 root clyde root You will need the IEEE address to configure your storage system 20 Optionally check to see that the port is correctly configured on the switch The procedure you need to use is different for each switch vendor We show the procedure for the IBM 2109 family of switches in Example 7 8 For INRANGE switches you should use the IN VSN Enterprise Manager application to find the same information For McDATA switches you can find the information in the EFC Manager application Example 7 8 Checking the Emulex adapter switch connectivity root clyde root telnet 9 1 38 159 Trying 9 1 38 159 Connected to 9 1 38 159 Escape character is Fabric OS tm Release a2 4 1c login admin Password itsosw3 admin gt switchShow switchName itsosw3 switchType 2 4 switchState Online switchRole Principal switchDomain 3 switchId fffc03 Chapter 7 Linux on i386 platform Fibre Channel implementation 159 switchWwn 10 00 00 60 69 10 64 cf switchBeacon OFF port 0 id Online F Port 21 00 00 e0 8b 02 a7 2d port 1 No Module port 2 id Online E Port 10 00
152. lug and play devices that are automatically detected by Windows 2000 Perform the following steps in order to install QLA2200F or QLA2300F drivers 1 Shut down the system Chapter 1 Windows Fibre Channel implementation 11 2 Install the QLogic adapter card in the host system 3 Turn on the system 4 Windows 2000 detects QLA2200 or QLA2300 and starts Found New Hardware Wizard 5 Select Search for a suitable driver for my device recommended and click Next 6 Select SCSI and RAID controllers in the Hardware Types list then click Next 7 Click Have Disk The Install From Disk dialog box appears 8 Insert the disk containing the QLogic driver into an appropriate drive then type the location of the driver 9 Click OK 10 Select the desired QLogic Fibre Channel HBAs then click Next Windows 2000 indicates that the wizard is ready to install the device 11 Click Next 12 If the Digital Signature Not Found dialog box appears click Yes 13 Click Finish 14 Remove the disk that contains the QLogic driver from the drive then click Yes to restart the system 1 5 3 Installing QLogic Fibre Channel HBAs family on Windows NT 12 This section lists the steps you have to follow in order to install card and appropriate driver on Windows NT operating systems Installation procedure 1 Shut down the system 2 Install the QLogic adapter card in the host system 3 Turn on the system 4 On the Windows NT taskbar s
153. ms There are three versions of HP UX currently available HP UX v10 20 HP UX v11 0 and HP UX v11i Fibre Channel operations are generally the same on these operating systems and we discuss any known differences in their respective sections throughout this chapter In this chapter we only cover a single path environment where each fibre device can be accessed via only one path If you have a multi path environment you need to use either the HP UX PV Links feature or a storage system specific multi path software for example IBM Subsystem Device Driver for ESS 4 2 Fibre Channel host bus adapters In this topic we detail Fibre Channel host bus adapters FC HBA that can be implemented 4 2 1 HP FC HBAs There are three Fibre Channel adapters supporting switched fabrics that are currently available from HP The adapters are described in Table 4 1 Each of these adapters supports only certain HP 9000 series models Table 4 1 Current HP FC HBAs with fabric support A5158A PCI Superdome V Class 11 0 and later N Class L Class A500 A400 rx9610 rx4610 A6684A HSC eff D Class R Class A180 11i 11 0 10 20 All of these adapters use SC connectors and have the speed of 1 Gb s There are also several older adapters that do not support switched fabrics We do not cover those adapters You can find more information on these adapters and the servers supporting them at http techsolutions hp com 94 Introducing hosts to the
154. n have simultaneous access through the switch Non L_Port A Node or Fabric port that is not capable of performing the Arbitrated Loop functions and protocols N_Ports and F_Ports are not loop capable ports Operation A term defined in FC 2 that refers to one of the Fibre Channel building blocks composed of one or more possibly concurrent exchanges Optical Disk A storage device that is written and read by laser light Optical Fiber A medium and the technology associated with the transmission of information as light pulses along a glass or plastic wire or fiber Ordered Set A Fibre Channel term referring to four 10 bit characters a combination of data and special characters providing low level link functions such as frame demarcation and signaling between two ends of a link Originator A Fibre Channel term referring to the initiating device Out of Band Signaling This is signaling that is separated from the channel carrying the information Also referred to as outband Peripheral Any computer device that is not part of the essential computer the processor memory and data paths but is situated relatively close by A near synonym is input output I O device Petard A device that is small and sometimes explosive PLDA Private Loop Direct Attach A technical report which defines a subset of the relevant standards suitable for the operation of peripheral devices such as disks and tapes on a private loop PLOGI
155. n cfwdl a d ff2 cfwdl Versions for device dev ff ff2 Chipset Revision Levels BIU Revision 0x4 Sequence Manager Revision 0x10020193 Endec Revision 0x3001506d Firmware Revisions Levels Operational Firmware Revision 0x2232000 2 23 0 Diagnostic Firmware Revision 0x1212000 1 21 0 FC PH Version Supported Highest FC PH Version Supported 4 3 Lowest FC PH Version Supported 4 3 cfwdl Flash program list versions for dev ff ff2 Diagnostic boot firmware version 1 21 Test firmware version 1 20 Functional firmware version 2 23 SLI 1 firmware version 2 23 SLI 2 firmware version 2 23 Device Type is SuperFly LP7000 Chapter 5 NUMA Q Fibre Channel implementation 119 5 6 3 Switch setup 120 8 Using the switch s local control panel we set the switch to its default settings and password and also set the TCP IP address as needed to be able to access it using the remote telnet login to complete the configuration as needed This IP address can also be used to access the switch via the Web based GUI session Now we telnet to the switch with the IP address and login using the default login and password login admin password password telnet 9 43 227 19 Trying Connected to 9 43 227 19 Escape character is Fabric OS tm Release a2 4 1c login admin Password SNJ2109DD2 admi n gt 10 We also set the switch for a unique name correct date and time and change 11 the following tw
156. necting them together In Figure 5 3 we show the IBM 2109 S08 and S16 switches 112 Introducing hosts to the SAN fabric Figure 5 3 IBM 2109 S08 top and S16 5 3 1 Fibre Channel switch installation preparation If you are installing the NUMA Q on a new unused Fibre Channel switch you should set up the fabric as a single zone and familiarize yourself with a simple network before expanding it If however you are adding the NUMA Q to an already existing fabric you may need to configure the switch to work with multiple hosts different vendors and operating systems For this more planning is required Using zones with the switches When you are sharing a switch with other hosts systems in a heterogeneous environment the NUMA Q requires its own zone to be established The NUMA Q cannot share a zone with any other hosts in a heterogeneous fabric It can share the switch as long as each host is zoned separately It is strongly recommended that these zones be established using hardware port number zoning for the host ports to provide increased security for the host attachments The process for establishing zones is covered in detail in SAN Zoning Concepts at the following Web site http webdocs numaq ibm com Chapter 5 NUMA Q Fibre Channel implementation 113 Using cascaded switches The NUMA Q system can also be connected to a cascaded or meshed fabric Some limitations exist with certain releases of the operating system in
157. nnection speeds rise from 1 Gb s to 2 Gb s and even higher To limit the effects of attenuation we recommend making sure that You have as few connections as possible on a single link All connectors are clean All connectors are properly seated Only one type of fiber is used on a single link never mix 50 micron and 62 5 micron fiber YYY Yy Zoning We strongly recommend that you use zoning to gt Enhance the security of your Fibre Channel fabric gt Avoid interactions between different hosts gt Secure storage devices that do not support LUN masking like tape drives Since the implementation of zoning and even the terminology used varies between switch manufacturers we do not discuss it in any detail Chapter 7 Linux on i386 platform Fibre Channel implementation 143 7 3 2 IBM 2109 family The current members of the IBM 2109 family of switches are described in Table 7 3 Table 7 3 The IBM 2109 family Speed Connectors Rack space IBM TotalStorage SAN Switch S08 Ve il 1 Gb s Ss fu IBM IBM TotalStorage SAN Switch S16 IBM TotalStorage SAN Switch S16 Switch S16 1 Gb s a Gb s Features and limitations The IBM 2109 family of switches support both fabric attached and loop attached devices on any port and switch domain ids 1 239 are supported Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on your requirements For more information
158. nt The information in this publication is not intended as the specification of any programming interfaces that are provided by any of the SAN hardware and software components contained herein See the PUBLICATIONS section of the IBM Programming Announcement for the SAN hardware and software components contained herein for more information about what publications are considered to be product documentation Comments welcome Your comments are important to us We want our Redbooks to be as helpful as possible Send us your comments about this or other Redbooks in one of the following ways gt Use the online Contact us review redbook form found at ibm com redbooks gt Send your comments in an Internet note to redbook us ibm com gt Mail your comments to the address on page ii xviii Introducing hosts to the SAN fabric Windows Fibre Channel implementation In this chapter we describe the steps involved in implementing Fibre Channel host bus adapter cards in an Intel based server running Windows NT or Windows 2000 operating system and its connection to the SAN The following topics are discussed Fibre Channel host bus adapter introduction on page 2 Windows supported host bus adapters on page 2 Supported SAN fabric components on page 6 Storage on page 9 FC host bus adapter card installation procedure on page 11 Installing the storage system on page 23 vvvvvy Copyright IBM Corp 2002 1
159. nt by using hwmgr command hwmgr view dev cat disk HWID Device Name Mfg Model Location 54 dev disk floppy0c 3 55in floppy fdi0 unit 0 60 dev disk dsklc DEC RZ2DD LS C DEC bus 2 targ 0 1un 0 63 dev disk cdrom0c COMPAQ CDR 8435 bus 5 targ 0 1un 0 66 dev disk dsk5c IBM 2105F20 bus 0 targ 253 1un 0 67 dev disk dsk6c IBM 2105F20 bus 0 targ 253 1un 1 68 dev disk dsk7c IBM 2105F20 bus 0 targ 253 1un 2 The disk attachment is successful Introducing hosts to the SAN fabric Linux on i386 platform Fibre Channel implementation In this chapter we discuss how to successfully set up a SAN fabric for a Linux server running on i386 platform We cover the step by step procedure from start to finish and the supported products which work with Linux We also point out where the latest firmware and device drivers can be obtained The following topics are discussed Introduction on page 140 Fibre Channel host adapters on page 141 SAN fabric on page 142 Storage on page 146 Installation procedures on page 146 YYYY Y Copyright IBM Corp 2002 139 7 1 Introduction 140 Linux is a free open source operating system that has been gaining popularity in the last few years especially on the Intel based platforms Linux is used extensively as a platform for Internet based services In addition to the Intel based platforms Linux has been ported to several architectures including members from al
160. nt software Mb s Megabits per second Also sometimes referred to as Mbps In computing terms it is approximately 1 000 000 bits per second Most precisely it is 1 048 576 1024 x 1024 bits per second MB s Megabytes per second Also sometimes referred to as MBps In computing terms it is approximately 1 000 000 bytes per second Most precisely it is 1 048 576 1024 x 1024 bits per second Meter 39 37 inches or just slightly larger than a yard 36 inches Media Plural of medium The physical environment through which transmission signals pass Common media include copper and fiber optic cable Media Access Rules MAR MIA Media Interface Adapter MIAs enable optic based adapters to interface to 168 Introducing hosts to the SAN fabric copper based devices including adapters hubs and switches MIB Management Information Block A formal description of a set of network objects that can be managed using the Simple Network Management Protocol SNMP The format of the MIB is defined as part of SNMP and is a hierarchical structure of information relevant to a specific device defined in object oriented terminology as a collection of objects relations and operations among objects Mirroring The process of writing data to two separate physical devices simultaneously MM Multi Mode See Multi Mode Fiber MMF See Multi Mode Fiber In optical fiber technology an optical fiber that is designed to carry multiple li
161. nter d You get the COMMAND STATUS window as shown in Figure 2 14 ol Telnet sicily BEI Connect Edit Terminal Help COMMAND STATUS Command Gime stdout no stderr no Before command completion additional instructions may appear below Figure 2 14 Install storage drivers installation in progress e When the status changes from RUNNING to OK scroll to the bottom of the screen as shown in Figure 2 15 to check that installation ended successfully 54 Introducing hosts to the SAN fabric of Telnet sicily Connect Edit Terminal Help COMMAND STATUS Command J stdout yes stderr no Before command completion additional instructions may appear below MORE 62 lt lt End of copyright notice for devices fcp disk gt gt Finished processing all filesets Total time 11 secs installp xx ATTENTION Software changes processed during this session require this system and any of its diskless dataless clients to be rebooted in order for the changes to be made effective bosboot Boot image is 8485 512 byte blocks F2 Refresh F3 Cancel Esc 6 Command Esc 9 Shell Esc 6 Exit Find Figure 2 15 Install storage drivers results f Remove the installation media if you are installing from CD ROM g Press F10 to exit SMIT h To update the drivers you just installed to the latest versions start SMIT by typing smitty update all You see the screen shown in Figure 2 16 Chapter 2 eServ
162. ntry fields Press Enter AFTER making all desired changes x Move cursor to desired item and press F7 Use arrow Entry Fields INPUT device directory for software dev cd8 ka Qevices pci dF1666F7 b PREUIEW only install operation will NOT occur no Include corresponding LANGUAGE filesets yes F2 Refresh F3 Cancel F4 List F6 Command F7 Edit F8 Image F16 Exit Enter Do Figure 2 5 Install additional device software installation settings h Accept the choices by pressing Enter i You get the ARE YOU SURE window press Enter to accept it j You get the COMMAND STATUS window as shown in Figure 2 6 of Telnet sicily x Connect Edit Terminal Help COMMAND STATUS Command EM stdout no stderr no Before command completion additional instructions may appear below Figure 2 6 Install additional device software installation in progress k The system may ask you to mount another CD ROM of the installation package like that shown in Figure 2 7 Chapter 2 eServer pSeries Fibre Channel implementation 45 46 of Telnet sicily Connect Edit Terminal Help COMMAND STATUS Command stdout yes stderr no Before command completion additional instructions may appear below MORE 89 US Government Users Restricted Rights Use duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp m u K End of copyright notice for devices common IBM fc gt gt
163. o each HBA Simplifies configuring JNI driver parameters Detects configuration errors Saves and restores adapter configuration information YYYY Y Introducing hosts to the SAN fabric In addition to the aforementioned functions the EZ Fibre Configuration Utility is also used for mapping storage devices When ready to connect Fibre Channel HBAs to a storage device directly or via fabrics both Windows 2000 and Windows NT users should use the following mapping storage device procedure 1 In the EZ Fibre Configuration Utility Main Menu right click the adapter and select Clear LLZ Targets Then click Yes With the adapter selected click LUN Level Zoning Click the Mapped column and select the target LUN you want to be mapped Click Commit Changes at the bottom of the screen a BA 9 N Restart the system 1 6 Installing the storage system In this section we describe a Windows based server connection to a storage device after successful Fibre Channel HBAs installation We connected a Netfinity 5500 to an IBM TotalStorage 2106 Modular Storage Server via an IBM 2109 S16 SAN Switch Procedures for storage definition and allocation as well as switch zones creation are not described here since they differ among various manufacturers Read carefully any documentation that comes with your storage system IBM 2109 S16 connectivity check Before you try to allocate and define the storage device in Windows 2000 or Windows NT you should
164. o settings as required for the NUMA Q The fabric parameter route delayReroute must be disabled The fabric parameter route stickyRoutes must be enabled If any additional help is needed with these settings refer to Fibre Channel IBM 2109 or SilkWorm 2000 Configuration at url http webdocs numaq ibm com We confirmed these settings by running the following configShow command SNJ2109DD2 admin gt configShow Ethernet addresses 0 60 69 12 3f fc Nvram datadata fei 0 0 host usr switch firmware e 9 43 227 19 fffffc00 g 9 43 224 1 u user tn SNJ2109DD2 diag postDisable 0 fabric domain 1 fabric ops BBCredit 16 fabric ops E D TOV 2000 fabric ops R_A_TOV 10000 fabric ops dataFieldSize 2112 fabric ops mode SeqSwitching 0 fabric ops mode fcpProbeDisable 0 fabric ops mode isolate 0 fabric ops mode longDistance 0 Introducing hosts to the SAN fabric fabric ops mode noClassF 0 fabric ops mode pidFormat 0 fabric ops mode sync 0 fabric ops mode tachyonCompat 0 fabric ops mode unicast0nly 0 fabric ops mode useCsCtl 0 fabric ops mode vcEncode 0 fabric ops vc class 2 2 fabric ops vc class 3 3 fabric ops vc config Oxc0 fabric ops vc linkCtr1 0 fabric ops vc multicast 7 fc4 fcIp address 0 0 0 0 fc4 fcIp mask 0 0 0 0 fcAL fanFrameDisable 0 fcAL openSendCLS 0 fcAL useAltBBCredit 0 1cdContrast 128 1cdContrast orange 208 licenseKey zdeSSR9zy2efTTy SbRcyRQSdcTcSpO0y route delayReroute 0 route
165. ociation http www futureio org InfiniBand SM Trade Association http www nsic org National Storage Industry Consortium http www ietf org Internet Engineering Task Force http www ansi org American National Standards Institute http www standards ieee org Institute of Electrical and Electronics Engineers http www pc ibm com us US Personal Systems Group http www compaq com products storageworks adapters html Compaq Storage Host Bus Adapters Related publications 179 How to get IBM Redbooks Search for additional Redbooks or redpieces view download or order hardcopy from the Redbooks Web site ibm com redbooks Also download additional materials code samples or diskette CD ROM images from this Redbooks site Redpieces are Redbooks in progress not all Redbooks become redpieces and sometimes just a few chapters will be published this way The intent is to get the information out much quicker than the formal publishing process allows IBM Redbooks collections Redbooks are also available on CD ROMs Click the CD ROMs button on the Redbooks Web site for information about all the CD ROMs offered as well as updates and formats 180 Introducing hosts to the SAN fabric Index Numerics 2031 8 98 145 2031 016 9 99 145 2031 032 9 99 145 2031 LOO 9 99 145 146 2032 8 98 145 2032 001 99 145 2032 064 99 145 2032 C36 98 145 2042 97 144 2042 C40 98 144 2105 Enterprise Storage
166. oes not support a wide range of Fibre Channel host bus adapters and we recommend that you order the latest or most appropriate one through the Compaq sales channels In the topics that follow we introduce a Compaq 64 bit PCI Fibre Channel host bus adapter for Tru64 UNIX in a step by step installation procedure In addition to this we cover the supported SAN switches and the considerations which should be taken into account in order to establish the connection with AlphaServer running Tru64 UNIX We only discuss a single path environment where each fibre device can be accessed via only one path If you want to implement a multi path environment you have to use a storage specific multi path software such as the IBM Subsystem Device Driver for ESS disk subsystem For more general information or technical information on the employment and application of adapters refer to the Compaq Web site http www compaq com 6 2 Fibre Channel host bus adapters In the topics that follow we describe the Fibre Channel host bus adapters 6 2 1 Compaq FC HBA 128 The StorageWorks 64 Bit PCI to Fibre Channel Host Adapter is one of the FC HBAs available from Compaq and supported by AlphaServer running Tru64 UNIX It belongs to a Compaq KGPSA family and it can be ordered through P N 168794 B21 KGPSA CA It replaces the Compaq PCI to Fibre Channel host adapter KGPSA BC P N 380574 001 As shown in Figure 6 1 this is a single port FC HBA with an
167. of LILO 2 booting and partitioning software from other OSs e g DOS FDISK OS 2 FDISK Command m for help p Disk dev sdb 255 heads 64 sectors 1084 cylinders Units cylinders of 16065 512 bytes Device Boot Start End Blocks Id System Command m for help n Command action e extended p primary partition 1 4 p Partition number 1 4 1 First cylinder 1 8504 default 1 1 Last cylinder or tsize or sizeM or sizeK 1 1084 default 1084 1084 Chapter 7 Linux on i386 platform Fibre Channel implementation 161 Command m for help w The partition table has been altered Calling ioct1 to re read partition table WARNING If you have created or modified any DOS 6 x partitions please see the fdisk manual page for additional information Syncing disks root clyde root mkfs dev sdbl mke2fs 1 19 13 Jul 2000 for EXT2 FS 0 5b 95 08 09 Filesystem label OS type Linux Fragment size 4096 log 2 1089152 inodes 2177020 blocks 108851 blocks 5 00 reserved for the super user First data block 0 67 block groups 32768 blocks per group 32768 fragments per grou 16256 inodes per group Superblock backups stored on blocks 32768 98304 163840 229376 294912 81 Writing inode tables done Writing superblocks and filesystem accounting in root clyde root mkdir test root clyde root mount dev sdb1 test root clyde root df k Filesystem 1k blocks Used Availab dev sda7 402885 85376 2
168. oftware for example IBM Subsystem Device Driver for ESS Linux is also available for selected models of eServer pSeries and RS 6000 systems but since there are no drivers for the Fibre Channel adapters available we do not discuss it 2 2 Fibre Channel host bus adapters In this topic we will discuss the Fibre Channel host bus adapters FC HBAs applicable to the aforementioned products and operating systems 2 2 1 IBM FC HBAs There are two Fibre Channel host bus adapters available from IBM for the eServer pSeries and RS 6000 families as specified in Table 2 1 Table 2 1 IBM FC HBAs for IBM eServer pSeries and RS 6000 Gigabit Fibre Channel Adapter 6227 1 Gb s eS 2 Gigabit Fibre Channel Adapter 628 1 or 2 Gb s The device drivers for both of these adapters are included in the AIX install media The 2 Gigabit adapter is the one we currently recommend Both adapters have a single Fibre Channel port and support attaching both disk and tape devices 34 Introducing hosts to the SAN fabric Note IBM RS 6000 model S70 does not support f c 6228 On those systems you must use f c 6227 instead There are system specific limitations on the number of host bus adapters you can install in a host The limitations as well as the recommended slots are defined in PCI Adapter Placement Reference SA38 0583 delivered with your system The most recent version can also be found at http www 1 ibm com servers eserver pseries library ha
169. om proc filesystem as in Example 7 4 Example 7 4 Checking the QLogic adapter again via proc interface root clyde proc cat proc scsi qla2x00 2 QLogic PCI to Fibre Channel Host Adapter for ISP2100 ISP2200 ISP2300 Firmware version 2 01 27 Driver version 4 27b Request Queue 0x370f8000 Response Queue 0x37040000 Request Queue count 256 Response Queue count 256 Number of pending commands 0x0 Number of queued commands 0x0 Number of free request entries 245 Number of mailbox timeouts 0 Number of ISP aborts 0 Number of loop resyncs 0 Number of retries for empty slots 0 Number of reqs in retry_q 0 Number of reqs in done q 0 SCSI Device Information scsi qla0 adapter node 200000e08b0190ef scsi qla0 adapter port 210000e08b0190ef scsi qla0 target 0 50001fe1000ae400 scsi qla0 target 1 50001fe1000ae400 SCSI LUN Information Id Lun 0 0 Total reqs 5 Pending reqs 0 Queued reqs 0 root clyde proc Note that now you also see a LUN behind the FC HBA 3 Check the kernel messages with the dmesg command as in Example 7 5 Example 7 5 Checking the kernel messages root clyde root dmesg qla2x00 detect found an HBA qla2x00 VID 1077 DID 2200 SSVID 1077 SSDID 2 scsi Found a QLA2200 bus 1 device 0x4 irq 11 iobase 0x5200 scsi 2 Configure NVRAM parameters Introducing hosts to the SAN fabric scsi 2 Verifying loaded RISC code scsi 2 Verifying chip sc
170. on OFF port 0 sw No Light port 1 sw No Light port 2 id No Light port 3 sw No Light port 4 No Module port 5 No Module 80 Introducing hosts to the SAN fabric port 6 id Online F Port 50 05 07 63 00 c0 8b 2b port 7 No Module port 8 No Module port 9 No Module port 10 sw No_Light port 11 sw No Light port 12 sw No Light port 13 sw No_Light port 14 sw No Light port 15 id No Light 3 6 5 Downloading the Fibre Channel adapter drivers We show how to download the JNI Emulex and QLogic drivers in the following topics Downloading the JNI adapter driver In this section we explain how to download the JNI 6460 adapter driver 1 2 Plug in and restart your host system and login as root Enter the following to create a temporary directory mkdir jnitemp Move to the temporary directory with the following command cd jnitemp Go to the following Web site http www jni com 5 From the navigation menu at the top of the page click Drivers 6 From the menu for Locate Driver by Product click FCE 6460 7 From the menu for FCE 6460 find the section for Solaris and copy or print the readme txt file for reference while installing Now from the menu for FCE 6460 and the section for Solaris select JNIC146x pkg and click to download In the Save As dialog box we directed it to our temporary folder jnitemp and left the filename as JNIC146x and clicked Save When you click Save
171. oot Check that you have the correct HP UX maintenance level installed Install the base level of device drivers for your FC HBA Update the drivers you just installed to the latest versions Shut down the system oa fF ON Install the host adapter card in a supported slot Depending on the model of your system the adapter is installed by HP s Service Support Representative SSR or you need to install it yourself You can find the installation instructions in the documentation provided with your system Note Some HP 9000 models support hot pluggable adapters If your system is one of those models you have the option of installing the adapter without shutting down the system If you want to make use of the hot plug feature follow the directions in the installation manual delivered with your system 7 Connect the adapter to your Fibre Channel fabric 8 Power on the system and log in as user root 9 Check that the adapter is correctly configured as in Example 4 1 Example 4 1 Checking the FC HBA ioscan funC fc Class I H W Path Driver S W State H W Type Description fc 0 0 4 0 0 td CLAIMED INTERFACE HP Tachyon TL TS Fibre Channel Mass Storage Adapter dev td0 Chapter 4 HP 9000 Fibre Channel implementation 101 You get a list of the Fibre Channel cards installed in the system The adapter is correctly configured by the operating system if the S W State is CLAIMED 10 Check the parameters of your adapter as in E
172. orage systems 115 IBM NUMA Q standard Fibre Channel attached tape storage systems115 IBM NUMA Q RPQ Fibre Channel attached tape storage systems 116 Ready made fibre optic duplex cables 116 Common Fibre Channel cabling configurations 117 Common Fibre Channel cabling configurations 130 The IBM 2109 family 0 0000 cee 131 IBM Fibre Channel attached disk storage systems 131 64 Bit PCI Fibre Channel HBA description 134 FC HBAs supported in Linux 141 Common Fibre Channel cabling configurations 143 The IBM 2109 family si sages ee ees Rea ON 4a a bean NG 144 The INRANGE switches 1 0 0 0 00 0 c eee eee 144 The McDATA switches 0 0 00 cece ee 145 IBM Fibre Channel attached disk storage systems 146 Introducing hosts to the SAN fabric Examples 1 1 2 1 2 2 2 3 2 4 2 5 2 6 4 1 4 2 4 3 4 4 6 1 6 2 6 3 6 4 6 5 6 6 7 1 7 2 7 3 7 4 7 5 7 6 7 7 7 8 7 9 Copyright IBM Corp 2002 Switch connectivity check 0 00 cee eee 24 Checking the AIX maintenance level aa 42 Checking the Fibre Channel adapter 00 eee eee 50 Checking the firmware level 1 0 0 0 0 00 eee 50 Checking the protocol 0 0 eee eae 51 Checking the FC HBA connectivity to the switch 52 Creating and testing a filesystem 1
173. ormance levels when adding additional processors memory and or storage SCSI Small Computer System Interface A set of evolving ANSI standard electronic interfaces that allow personal computers to communicate with peripheral hardware such as disk drives tape drives CD_ROM drives printers and scanners faster and more flexibly than previous interfaces The table below identifies the major characteristics of the different SCSI version SCSI Ver sion pear A SCSI Ultra 20 16 40 7 12 SCSI 2 40 16 15 12 SCSI 3 SCSI 3 consists of a set of primary commands and additional specialized command sets to meet the needs of specific device types The SCSI 3 command sets are used not only for the SCSI 3 parallel interface but for additional parallel and serial protocols including Fibre Channel Serial Bus Protocol used with IEEE 1394 Firewire physical protocol and the Serial Storage Protocol SSP SCSI FCP The term used to refer to the ANSI Fibre Channel Protocol for SCSI document X3 269 199x that describes the FC 4 protocol mappings and the definition of how the SCSI protocol and command set are transported using a Fibre Channel interface Sequence A series of frames strung together in numbered order which can be transmitted over a Fibre Channel connection as a single operation See also Exchange SERDES Serializer Deserializer Server A computer which is dedicated to one task SES SCSI Enclosure Services ANSI
174. ows server is connected and the basic port information is displayed as shown in Figure 1 14 E8 Port Information for itsosw3 Mm Ei o 1 2 3 6 7 8 3 10 11 12 13 14 15 PortWwWN 20 05 00 60 69 10 64 cf Port Status Online Port Module sw Serial id GBIC PortType F Port PortStats caic Loop 4 Byte Word Transmitted 2320628405 Short Frames z 4 Byte Word Received 2585472714 Long Frames 0 Frames Transmitted 12791969 Bad End of Frames 5 Frames Received 5607088 Encd Errs Outside Frames 650 C2 Frames Received o C3 Frames Discarded o C3 Frames Received 5607088 LIP Ins o Link Control Frames Received 0 LIP Outs o Moast Frames Received o Last LIP Received 00 00 Mcast Timeouts o Frames Rejected o Mcast Frames Transmitted o Frames Busied o Time R_RDY Priority 3784263721 Link Failure 57 Time BB_Credit Zero 31414 Loss of Sync 208 Ened Errs Inside Frames 4 Loss of Signal 1899 Frames with CRC Errs 2 Figure 1 14 IBM 2109 516 Port Information This is the easiest way to find out all the relevant switch information and to configure your own zone before you access the storage device from a Windows environment Additionally you do not have to know the switch login name and password in order to check if a switch is correctly connected to the server The switch connectivity check procedure is different for each switch vendor For INRANGE switches you should use the IN VSN Enterprise
175. pletion option For other vendors you must contact your Sun representative for assistance 68 Introducing hosts to the SAN fabric 3 3 Sun supported Fibre Channel switches and hubs In this section we discuss the different Fibre Channel switches which are available for use with the Sun platforms in the SAN environment Again Sun has partnered with a number of vendors to create quite a list of supported products which we have listed here with their vendor s URLs gt StorEdge Network FC Switch 8 and 16 Sun Microsystems http www sun com storage fcswitch san_8 16 datasheet pdf gt Silkworm Series Brocade Communications Systems Inc http www brocade com products gt ED5000 2032 001 ED6064 2032 064 ES3016 2032 016 ES3032 2032 032 McDATA Corporation www mcdata com IBM suptdev gt FC 9000 Series INRANGE Corporation http www inrange com products san fc9000 gt SANbox SANbox2 Series QLogic Corporation Previously Ancor Communications Inc http www qlogic com products fc_san_switchs asp gt 8100 switch 1000 2000 hubs Vixel Corporation http www vixel com products IBM proofed switches with Sun systems IBM has proofed the Sun systems on the IBM 2109 INRANGE 9000 McDATA ED5000 series and will be testing it with the McDATA ED6064 The following brief descriptions provide a brief introduction to them For greater detail contact the specific switch vendor 3 3 1 IBM 2109 Current members of the
176. plicate target and LUN pairs Chapter 3 Sun Fibre Channel implementation 87 88 4 Now we edit the driver conf file for our driver to have the specific parameter settings that are needed to work properly with our Sun system In our example we edited the jnic146x conf file Refer to the vendor s documentation for your adapters settings Note In Table 3 3 on page 66 Table 3 5 on page 67 and Table 3 6 on page 68 we show the recommended parameters for the IBM tested adapters vi jnicl46x conf and add the following FcFabricEnabled 0 FcFabricEnabled 1 jnicl46x0 def_hba_binding null for the first adapter jnicl46x1 target1_wwnn 5005076300c08b2b from the ESS host port we will access the devices through jnicl146x1 target0O_lunl_hba jnicl46x1 jnicl46x1 target0 lunl throttle 605 jnicl46x1 target0 lun2 hba jnic146x1 5 jnicl46x1 target0 lun2 throttle 605 jnicl46x1 target0 lun3 hba jnic146x1 jnicl46x1 target0 lun3 throttle 605 jnicl46x1 target0 lun4 hba jnicl146x1 jnicl46x1 target0 lun4 throttle 605 jnicl46x1 target0 lunb hba jnicl46x1 jnicl46x1 target0 lun5 throttle 605 jnicl46x1 target0 lunb hba jnic146x1 jnicl46x1 target0_lun6_throttle 60 jnic146x1 target0O_lun7_hba jnicl46x1 jnicl46x1 target0 lun7 throttle 605 wq We now have the drivers ready to be reconfigured into the kernel But before we do this we want to connect the system to our switch that we will use Now we are read
177. port 12 sw Online L Port 1 public port 13 id No Light port 14 id Online F Port 10 00 00 00 c9 25 8e 54 port 15 id No Light itsosw3 admin gt exit Connection closed by foreign host root clyde root From Example 7 3 we see that the port we are using port 5 is online it is an F_Port for fabric attachment and the WWN listed is the same that we see on our adapter in Example 7 2 17 f you want your FC HBA to be activated automatically in system startup add the following lines to etc rc d rc sysinit before the Add raid devices comment Add Fibre Channel devices modprobe scsi hostadapterl 18 Your FC HBA is now correctly installed 7 5 2 Configuring storage for the QLogic FC HBA After the adapter installation we have to create any necessary switch zones and allocate the storage for our host from the storage system we are using As both of those steps are different for different switch manufacturers and different storage subsystems they are not described here We have access to a single volume in a storage subsystem We want to configure that volume to our host system The procedure is as follows Chapter 7 Linux on i386 platform Fibre Channel implementation 151 152 1 Remove and reinstall the QLogic driver as follows root clyde root rmmod qla2x00 root clyde root modprobe qla2x00 root clyde root Alternatively you can reboot the machine 2 Check the current information on the FC HBA fr
178. pter 3 Sun Fibre Channel implementation 71 Features and limitations Special features of the INRANGE FC 9000 are gt The INRANGE FC 9000 family of switches support both fabric attached and loop attached devices on any port gt Any port can be independently configured with a shortwave multimode or longwave singlemode GBIC depending on customer requirements 3 3 3 McDATA The McDATA components currently sold by IBM the IBM 2031 and 2032 families are described in Table 3 9 Table 3 9 McDATA components Ca 2031 016 McDATA ES 3016 16 1 Gb s LC 1U Fabric Switch 2031 032 McDATA ES 3032 32 1 Gb s LC 1 5U Fabric Switch 2031 L00 McDATA ES 1000 8 1 1 Gb s SC 1U Loop Switch 2032 001 McDATA ED 5000 32 1 Gb s SC 18U Director with 32 ports 2032 064 McDATA ED 6064 64 1 Gb s LC 9U Enterprise Fibre Channel Director with 64 ports Note McDATA ED5000 is not supported with QLogic QLA2100F FC HBA In addition IBM also sells the McDATA FC 512 Cabinet 2032 C36 which can house up to four ED 6064 rack units Both of these are shown in Figure 3 6 72 Introducing hosts to the SAN fabric Figure 3 6 McDATA cabinet left and 6064 rack mountable switch Features and limitations Special features and limitation of the McDATA switches are gt The only McDATA switch that supports loop attached devices tape drives for example is the McDATA ES 1000 Loop Switch The ports in the 2032 064 can be added in cards Eac
179. r scsi qla0 adapter port You will need that parameter to configure your storage system Note also that you do not see any SCSI LUNs yet 16 Optionally check to see that the port is correctly configured on the switch The procedure you need to use is different for each switch vendor We show the procedure for the IBM 2109 family of switches in Example 7 3 For INRANGE switches you should use the IN VSN Enterprise Manager application to find the same information For McDATA switches you can find the information in the EFC Manager application Example 7 3 Checking QLogic adapter switch connectivity root clyde root telnet 9 1 38 159 Trying 9 1 38 159 Connected to 9 1 38 159 Escape character is Fabric OS tm Release a2 4 1c login admin Password Introducing hosts to the SAN fabric itsosw3 admin gt switchShow switchName itsosw3 switchType 2 4 switchState Online switchRole Principal switchDomain 3 switchId fffc03 switchWwn 10 00 00 60 69 10 64 cf switchBeacon OFF port 0 id Online F Port 21 00 00 e0 8b 02 a7 2d port 1 No Module port 2 id Online E Port 10 00 00 60 69 20 1d 4e itsoswl downstream port 3 No Module port 4 id No Light port 5 id Online F Port 21 00 00 e0 8b 01 90 ef port 6 id Online E Port 10 00 00 60 69 20 1d 4e itsoswl port 7 id No Light port 8 sw Online F Port 10 00 00 00 c9 22 bf 75 port 9 No Module port 10 sw No Light port 11 sw No Light
180. rdware_docs index html You can download adapter firmware AIX fixes and maintenance levels from http techsupport services ibm com server fixes view pSeries You can also order the fixes on a CD at the same address 2 2 2 Other FC HBAs Other FC HBAs available for IBM eServer pSeries and RS 6000 servers are listed in Table 2 2 Table 2 2 Other FC HBAs available for IBM eServer pSeries and RS 6000 Chapter 2 eServer pSeries Fibre Channel implementation 35 These adapters are supported by their manufacturer For more information about the adapters and their limitations see the manufacturer s Web site http www emulex com http www jni com 2 3 SAN fabrics The IBM eServer pSeries and RS 6000 hosts support two Fibre Channel network topologies gt Fibre Channel Arbitrated Loop FC AL gt Switched fabric The IBM eServer pSeries and RS 6000 systems support no more than one host in an FC AL loop This means that FC AL is only useful for connecting the host directly to a storage device such as the IBM Enterprise Storage Server ESS FC AL also has severe performance and scalability limitations Because of these limitations we only cover switched fabric topology However we also discuss attaching storage devices supporting only FC AL attachment to the fabrics We cover products from the following three vendors gt IBM 2109 gt INRANGE gt McDATA 2 3 1 General information There are two types of Gigabit
181. re installation settings c Accept the screen by pressing Enter d You get the ARE YOU SURE window press Enter to accept it Next you get the COMMAND STATUS window as shown in Figure 2 11 Telnet sicily Connect Edit Terminal Help COMMAND STATUS Command EM stdout no stderr no Before command completion additional instructions may appear below Figure 2 11 Update device software installation in progress e When the status changes from RUNNING to OK scroll to the bottom of the screen as shown in Figure 2 12 to check that installation ended in SUCCESS 48 Introducing hosts to the SAN fabric of Telnet sicily Connect Edit Terminal Help COMMAND STATUS Command J stdout yes stderr no Before command completion additional instructions may appear below MORE 179 devices pci df1066f7 rte 3 3 75 USR APPLY SUCCESS devices common IBM fc rte 3 3 75 USR APPLY SUCCESS devices common IBM fc rte 3 3 75 USR COMMIT SUCCESS devices pci df1066F7 com 3 3 76 USR COMMIT SUCCESS devices pci df1066f7 rte 3 3 75 USR COMMIT SUCCESS devices pci df1066F7 diag 3 3 76 USR COMMIT SUCCESS installp xx ATTENTION Software changes processed during this session require this system and any of its diskless dataless clients to be rebooted in order for the changes to be made effective F2 Refresh F3 Cancel F6 Command F9 She11 F16 Exit Find Figure 2 12 Update device software results f Pre
182. re In this section we run through a simple fabric installation and show the success of the work We are using a Sun 450 system with JNI s FCE 6460 PCI FC HBAs installed connected to an IBM ESS using one FC host adapter We have LUNs configured on the ESS for access by the Sun system We are also using Solaris V2 6 to perform our installation Any special instructions which may differ from our performed steps for the other supported releases of the operating system are noted as we go To start with it is assumed that this not a new system installation and that the Fibre Channel we are installing is an add on to the platform hardware Also the ESS has been setup and configured to provide the LUNs to our system once we are ready to provide it with the correct WWNs The system is powered up and running the operating system of choice It is also assumed that the system has been configured for LAN and TCP IP support Now we are ready to start installing our fabric 76 Introducing hosts to the SAN fabric 3 6 1 Check that the system is at a proper level Before we start our installation we first make some checks to ensure that the system is running the version of operating system we desire the correct patches and does not already have the driver loaded This we do by running the following commands 1 To check the operating system level we run uname a SunOS sol e 5 6 Generic 105181 29 sun4u sparc SUNW Ultra 4 To ensure that we ha
183. red on blocks 32768 98304 163840 229376 294912 819200 884736 1605632 Writing inode tables done Writing superblocks and filesystem accounting information done root clyde root mkdir test root clyde root mount dev sdb1 test root clyde root df k Filesystem 1k blocks Used Available Use Mounted on dev sda7 402885 84463 297621 23 dev sda2 201433 3822 187210 34 boot dev sda5 3024016 538392 2332008 19 usr dev sdb1 8571328 20 8135904 1 test root clyde root cp etc hosts test root clyde root 1s 1 etc hosts test hosts rw r r 1 root root 181 Dec 3 04 00 etc hosts rw r r 1 root root 181 Dec 3 15 15 test hosts root clyde root diff etc hosts test hosts root clyde root The connection is successful 154 Introducing hosts to the SAN fabric 7 5 3 Installing an Emulex FC HBA We used the following in our installation YYYY Y IBM Netfinity 5500 Red Hat Linux 7 1 with kernel sources and kernel includes installed Kernel 2 4 2 Emulex LP8000 FC HBA Emulex driver SCSI IP v4 11e Our installation procedure is as follows 1 2 Start up your system and log in as root Check that the kernel source files and headers are installed as follows root clyde root rpm q a grep kernel kernel headers 2 4 2 2 kernel 2 4 2 2 kernel source 2 4 2 2 root clyde root You need to have both kernel headers and kernel source installed If either of these packages is mi
184. response or press Enter to each question to accept the default setting The following packages are available 1 JNIC146x JNI Fibre Channel SCSI HBA Driver sparc v5 1 01110800 2 JNIsnia JNI SNIA Fibre Channel HBA LIBRARY Solaris sparc v1 0 0 0 0 b 011106 10 Select package s you wish to process or all to processall packages default all q Processing package instance lt JNIC146x gt from lt opt Drivers JNI FCE 6460 v5 1 JNIC146x pkg gt JNI Fibre Channel SCSI HBA Driver sparc v5 1 01110800 Copyright c 1999 2001 JNI Corp All Rights Reserved Processing package information Processing system information Verifying disk space requirements Checking for conflicts with packages already installed Checking for setuid setgid programs This package contains scripts which will be executed with super user permission during the process of installing this package Do you want to continue with the installation of lt JNIC146x gt y n y Installing JNI Fibre Channel SCSI HBA Driver as lt JNIC146x gt Introducing hosts to the SAN fabric Installing part 1 of 1 kernel drv jnicl46x kernel drv jnic146x conf kerne1 drv sparcv9 jnic146x verifying class lt none gt Executing postinstall script Installation of lt JNIC146x gt was successful Processing package instance lt JNIsnia gt from lt opt Drivers JNI FCE 6460 v5 1 JNIC146x pkg gt JNI SNIA Fibre Channel HB
185. rly high speed storage arrays RAID and CD ROM storage by IBM Star The physical configuration used with hubs in which each user is connected by communications links radiating out of a central hub that handles all communications StorWatch Expert These are StorWatch applications that employ a 3 tiered architecture that includes a management interface a StorWatch manager and agents that run on the storage resource s being managed Expert products employ a StorWatch data base that can be used for saving key management data e g capacity or performance metrics Expert products use the agents as well as analysis of storage data saved in the data base to perform higher value functions including reporting of capacity performance etc over time trends configuration of multiple devices based on policies monitoring of capacity and performance automated responses to events or conditions and storage related data mining Glossary 171 StorWatch Specialist A StorWatch interface for managing an individual fibre Channel device ora limited number of like devices that can be viewed as a single group StorWatch specialists typically provide simple point in time management functions such as configuration reporting on asset and status information simple device and event monitoring and perhaps some service utilities Striping A method for achieving higher bandwidth using multiple N_Ports in parallel to transmit a single information un
186. rt 10 00 00 00 c9 22 a2 8d port 11 sw Online F Port 10 00 00 00 c9 28 05 da port 12 id Online F Port 10 00 00 00 c9 21 58 59 port 13 sw No Light port 14 sw No Light port 15 id No Light o OANA 0 Bb WN FE 1 1 Koj I 1 16 After connecting the cables we are ready to probe for the LUNs This is done by using the devct1 c command on the NUMA Q system which we will point at the FC HBA ff2 devctl c ff2 devct l Found fabric3 sd713 sd714 sd715 sd716 This probe should find the fabric and all of the LUN devices that we assigned to the FC HBA WWN on the ESS We see that these devices are all assigned names with a sign attached to them This is a temporary assigned name which we now need to make permanent 17 Now we again run devct1 N command to assign the temporary devices to their permanent names devctl N devctl Assigned sd713 gt sd713 devctl Assigned sd714 gt sd714 devctl Assigned sd715 gt sd715 devctl Assigned sd716 gt sd716 18 We run etc dumpconf d command to see all the devices now found on the system and their parent tree dumpconf d NAME CFGTYPE DEVNUM UNIT FLAGS OnBUS OnDEVICE quad0 quad 0 Oxff000000 L sci sci0 Chapter 5 NUMA Q Fibre Channel implementation 123 asy0 asy 0 Oxff000000 L eisa quad0 asyl asy 1 Oxff000001 L eisa quad0 mdc0 mdc 0 Oxff000000 L eisa quad0 ff0 ff 0 0x00000000 SP pci quad0 qlc0 qlc 0 0x00000001 L pci quad0 scsibus0 scsibus
187. rver pSeries and RS 6000 systems 2 5 Installation procedures In this topic we discuss the installation procedures 2 5 1 Preparations to install the FC HBAs These are the preparations necessary in order to install the FC HBAs 40 Introducing hosts to the SAN fabric gt Check that you have a free supported PCI slot for the FC HBAs gt Check that you have the correct version and latest maintenance level of AIX installed At the time of writing the latest maintenance level is 4 3 3 0 9 gt Obtain the latest drivers For the Gigabit Fibre Channel Adapter f c 6227 you need the packages devices pci df1000f7 For the 2 Gigabit Fibre Channel Adapter f c 6228 you need the packages devices pci df1000f9 gt Obtain the latest adapter firmware Download adapter firmware AIX fixes and maintenance levels from http techsupport services ibm com server fixes view pSeries The page opened by this URL is shown in Figure 2 1 Fixes Server Support Netscape i 3 N ol x File Edit View Go Communicator Help Ci 3 4 a ad d d ai ww Back Forward Reload Home Search Netscape Print Security Shop Stop calf Bookmarks 2 Location http techsupport services ibm com server fixes view pSeries X E What s Related Home Products amp services Support amp downloads My account Select a country IBM server pSeries Support lt Products amp services Servers Solutions pSeries Fixes Choose
188. s of fiber cable commonly used in Fibre Channel environments as described in Table 4 3 Table 4 3 Common Fibre Channel cabling configurations GBIC type Cable type Distance 1 Gb s Distance 2 Gb s Shortwave GBICs and 50 micron cable are usually used to connect devices to the fabric since most device adapters only support shortwave GBICs as well as for short inter switch links Longwave GBICs and 9 micron cable are used when required for extended distance Attenuation Fibre Channel is a high speed communication technology Because of this it is very sensitive to link attenuation The attenuation is even more of a critical factor when in the future connection speeds rise from 1 Gb s to 2 Gb s and even higher To limit the effects of attenuation we recommend making sure that You have as few connections as possible on a single link All connectors are clean All connectors are properly seated Only one type of fiber is used on a single link never mix 50 micron and 62 5 micron fiber YYY Yy 96 Introducing hosts to the SAN fabric Zoning We strongly recommend that you use zoning to gt Enhance the security of your Fibre Channel fabric gt Avoid interactions between different hosts gt Secure storage devices that do not support LUN masking like tape drives Since the implementation of zoning and even the terminology used varies between switch manufacturers we do not discuss it in any detail Cascading H
189. s the standard supported direct fibre tape device that is available for NUMA Q with details Table 5 5 IBM NUMA Q standard Fibre Channel attached tape storage systems Storage system Method of attachment Operating systems IBM 3590E tape drives with FC AL attached from Supported with DYNIX ptx direct fibre interface either the FC HBA V4 5 2 and greater directly ora FL port on a Brocade switch Chapter 5 NUMA Q Fibre Channel implementation 115 In addition to the above standard supported tape device there is also a table of the Request for Price Quotation RPQ tape device that is available See Table 5 6 for details Table 5 6 IBM NUMA Q RPQ Fibre Channel attached tape storage systems Storage system Method of attachment Operating systems STK 9840 tape drives with FC AL attached from Supported with DYNIX ptx direct fibre interface either the FC HBA V4 5 2 and greater directly ora FL portona Brocade switch 5 5 Installation planning Before starting the installation it is important to plan what you intend to do This includes deciding what will be connected and where they will be located Our example install will be small although your installation may need to be quite a lot larger Items to consider will be the physical location of your system the storage that is being connected and the switch or switches which you are connecting them to Cables must be purchased to connect these all together or possibly an insta
190. serial 31A20313 capacity 3814 Mbytes sd713 fabric3 cport 172 fc id 0x119ef fcp lun 0x1a53 No Pbay on fabric3 Vendor IBM product 2105F20 revision 0 83 serial 31A20313 capacity 3814 Mbytes sd714 fabric3 cport 44 fc id 0x118ef fcp lun 0x1b53 No Pbay on fabric3 124 Introducing hosts to the SAN fabric Vendor IBM product 2105F20 revision 0 83 serial 31B20313 capacity 3814 Mbytes sd714 fabric3 cport 172 fc id 0x119ef fcp lun 0x1b53 No Pbay on fabric3 Vendor IBM product 2105F20 revision 0 83 serial 31B20313 capacity 3814 Mbytes sd715 fabric3 cport 44 fc id 0x118ef fcp lun 0x1c53 No Pbay on fabric3 Vendor IBM product 2105F20 revision 0 83 serial 31C20313 capacity 3814 Mbytes sd715 fabric3 cport 172 fc id 0x119ef fcp lun 0x1c53 No Pbay on fabric3 Vendor IBM product 2105F20 revision 0 83 serial 31C20313 capacity 3814 Mbytes sd716 fabric3 cport 44 fc id 0x118ef fcp lun 0x1d53 No Pbay on fabric3 Vendor IBM product 2105F20 revision 0 83 serial 31D20313 capacity 3814 Mbytes sd716 fabric3 cport 172 fc id 0x119ef fcp lun 0x1d53 No Pbay on fabric3 Vendor IBM product 2105F20 revision 0 83 serial 31D20313 capacity 3814 Mbytes As you can see there are two entries for each ESS LUN which was assigned to the NUMA Q system This is due to the two host ports that were connected to the switch from the ESS If additional fabrics were added these devices could be mapped to them also giving you even higher
191. si 2 Waiting for LIP to complete scsi 2 LOOP UP detected scsi 2 Waiting for LIP to complete scsi2 Topology F Port Host Loop address Oxffff scsi 2 Waiting for LIP to complete scsi2 Topology F Port Host Loop address Oxffff scsi qla0 adapter node 200000e08b0190ef scsi qla0 adapter port 210000e08b0190ef scsi qla0 target 0 50001fe1000ae400 scsi qla0 target 1 50001fe1000ae400 scsi2 QLogic QLA2200 PCI to Fibre Channel Host Adapter bus 1 device 4 irg 11 Firmware version 2 01 27 Driver version 4 27b Vendor DEC Model HSG80 Rev V85S Type Direct Access ANSI SCSI revision 02 Vendor DEC Model HSG80 Rev V85S Type Processor ANSI SCSI revision 02 scsi 2 0 0 0 Enabled tagged queuing queue depth 16 scsi 2 0 1 0 Enabled tagged queuing queue depth 16 Attached scsi disk sdb at scsi2 channel 0 id 0 lun 0 SCSI device sdb 17418235 512 byte hdwr sectors 8918 MB sdb root clyde root Note that now we also see the ID of our disk sdb If you already had partitions on the disk they would also be shown 4 Start using the new LUN like any normal SCSI disk We tested the Fibre Channel drive according to the procedure in Example 7 6 Example 7 6 Testing a Fibre Channel disk with QLogic adapter root clyde root fdisk dev sdb Device contains neither a valid DOS partition table nor Sun SGI or OSF disklabel Building a new DOS disklabel Changes will remain in memory only until you
192. ss F10 to exit SMIT 5 Shut down the system 6 Install the host bus adapter card in a supported slot Depending on the model of your system the adapter is installed by IBM s Service Support Representative SSR or you may need to install it yourself You can find the installation instructions in the documentation provided with your system Note Some eServer pSeries and RS 6000 models support hot pluggable adapters If your system is one of those models you have the option of installing the adapter without shutting down the system If you want to make use of the hot pluggable feature follow the directions in the installation manual delivered with your system 7 Connect the adapter to your Fibre Channel fabric Chapter 2 eServer pSeries Fibre Channel implementation 49 50 Note The Fibre Channel device drivers choose the network topology when the adapter is configured If your adapter is not connected to a Fibre Channel fabric it will be configured as FC AL attached You can correct the problem by unconfiguring and reconfiguring the adapter as follows rmdev R1 fcs0 fscsi0 Defined fcs0 Defined cfgmgr 8 Power on the system and log in as user root 9 Check that the adapter is correctly configured as shown in Example 2 2 You get a line for each of the Fibre Channel cards installed in the system Example 2 2 Checking the Fibre Channel adapter Isdev C grep fcs fcs0 Available 3A 08 FC Adapter
193. ssing you have to install it before proceeding Add the following line to the file etc lilo conf append max_scsi_luns 128 If you already have an append line in etc lilo conf add the max scsi luns parameter to that line Reinstall LiLO by the following command root clyde root sbin lilo Added linux root clyde root Create a directory for the driver package and unpack it there as follows root clyde root mkdir usr src emulex root clyde root cd usr src emulex root clyde emulex tar xvvf tmp Ipfc i386 tar drwxr xr x root root 0 2001 10 01 17 09 14 SourceBuild drwxr xr x root root 0 2001 09 27 12 01 10 SourceBuild include rw r r root root 27645 2001 09 27 12 01 10 SourceBuild include fc_os h rw r r root root 5662 2001 09 27 12 01 10 SourceBuild include fc_ertn h rw r r root root 8548 2001 09 27 12 01 10 SourceBuild include dfc h rw r r root root 75966 2001 09 27 12 01 10 SourceBuild include fc h rw r r root root 95934 2001 09 27 12 01 10 SourceBuild include fc_hw h rw r r root root 12439 2001 09 27 12 01 10 SourceBuild include fcdds h rw r r root root 16251 2001 09 27 12 01 10 SourceBuild include fcdiag h rw r r root root 18532 2001 09 27 12 01 10 SourceBuild include fcfgparm h rw r r root root 15720 2001 09 27 12 01 10 SourceBuild include fc_crtn h rw r r root root 177693 2001 09 27 12 01 10 SourceBuild Ipfcdriver rw r r root root 16993 20
194. t 10 No Module port 11 No Module port 12 id Online F Port 10 00 00 00 c9 21 58 59 port 13 sw No Light port 14 sw No Light port 15 id No Light 5 6 4 Storage setup 14 Now we need to ensure our storage device is ready to be connected to our NUMA Q system In our example we are using a group of LUNs from an IBM ESS So we must now define the FC HBA to the ESS using the StorWatch manager As we are sharing the ESS between multiple hosts and the NUMA Q the ESS ports will be configured for Access Restricted and specific LUN assignments will be used To do this we must map the desired LUNs to the FC HBAs WWN We get the Node WWN of the FC HBA from the infodev command we ran in step 3 above 15 We now need to connect the cables form the ESS to the switch again being careful not to contaminate them with dirt or oil from our hands We seat them securely into the GLM of the ESS adapter and the GBIC of the switch port Another check using switchShow gives us the following SNJ2109DD2 admin gt switchShow 122 Introducing hosts to the SAN fabric switchName SNJ2109DD2 switchType 2 4 switchState Online switchRole Principal switchDomain 1 switchId fffc0l switchWwn 10 00 00 60 69 12 3f fc switchBeacon OFF port 0 sw No Light port sw No Light port id No Light port sw No Light port No Module port No Module port No Module port No Module port No Module port No Module port 10 sw Online F Po
195. tch 2031 032 McDATA ES 3032 Fabric 32 1 Gb s LC 1 5U Switch Table 1 5 The McDATA switches 2031 L00 McDATA ES 1000 Loop 8 1 1 Gb s SC 1U Switch 2032 001 McDATA ED 5000 32 1 Gb s SC 18U Enterprise Fibre Channel Director 2032 064 McDATA ED 6064 1 Gb s Enterprise Fibre Channel Director In addition IBM also sells the McDATA FC 512 Cabinet 2032 C36 8 Introducing hosts to the SAN fabric Features and limitations The only McDATA switch that supports loop attached devices tape drives for example is the McDATA ES 1000 Loop Switch The ports in the 2032 064 can be added in cards Each card contains four ports There are three different cards available gt Four shortwave multimode ports gt Four longwave singlemode ports gt Three shortwave ports and one longwave port On the 2031 016 and 2031 032 switches any port can be independently configured with a shortwave multimode or longwave singlemode GBIC The 2031 L00 switch has two different types of ports gt Eight switching hub ports that can only be configured with shortwave multimode GBIC or copper interface gt One bridge port for connecting the loop switch to the fabric that can be configured with a shortwave multimode or longwave singlemode GBIC The McDATA switches support fabrics of up to 31 switches 1 4 Storage The IBM Fibre Channel attached storage systems supported by the Windows 2000 and Windows NT operating systems are listed in
196. than 16 LUNs per adapter If your configuration includes more than one LP8000 adapter per server calculate the lun queue depth value using the adapter with the most LUNs attached no device delay 15 Recommended Delay to failback and I O network on 0 Default Recommended for fabric Do not turn on IP networking 1 Turn on IP networking scan down 2 Recommended Use inverted ALPA map and cause target assignment in private loop topology 2 Recommended for fabric Point to point mode only 4 Recommended for non fabric Arbitrated loop mode only zone rscn 0 Default 1 Recommended for fabric check name server for RSCNs Chapter 3 Sun Fibre Channel implementation 67 In Table 3 6 we show the recommended configuration file parameters for the host bus adapters for the QLogic QLA2200F adapter Table 3 6 Recommended QLA2200F HBA settings Parameters Recommended settings hba0 max frame length hba0 max iocb allocation hba0 execution throttle hba0 login timeout hba0 login retry count hba0 fabric retry count hba0 enable adapter hard loop hba0 adapter hard loop hba0 enable 64bit addressing hba0 enable LIP reset hba0 enable LIP full login hba0 enable target reset 0 Non clustered 1 Clustered hbaO reset delay hba0 port down retry count hba0 link down error hba0 loop down timeout hba0 connection options 1 Fabric connection 2 Direct connection hba0 device configuration mode hba0 fc tape hba0 command com
197. the var messages file pg var adm messages messages Dec 4 08 09 10 trel FCE 6460 N messages Dec 3b32 messages Dec 4 08 09 10 trel 4 08 09 10 trel messages Dec 4 08 09 10 trel Rev 100 Vend 1242 Dev 14 messages Dec 4 08 09 10 trel WWPN 2000000173002 f6e messages Dec 4 08 09 10 trel speed messages Dec Loop port messages Dec messages Dec 4 08 09 10 trel 4 08 09 11 trel 4 08 09 11 trel 4 08 09 16 trel 4 08 09 16 trel 4 08 09 21 trel messages Dec messages Dec messages Dec detected JNI Corp All Rights Reserved ane 60 ane ane ane ane ane ane ane ane unix unix unix unix unix unix unix unix unix unix unix unix jnicl46x0 Hba JNI FCR Model jnicl46x0 FCode Version 3 6 2 Channel 1 1242 Sub 6460 jnic146x0 jnic146x0 IRQ 1 SubVend WWNN jnic146x0 1000000173002 f6e jnicl46x0 Auto negotiating link jnicl46x0 Configured as Private JNIC v5 1 01110800 Copyright c 1995 2001 jnic146x0 jnic146x0 PCI device JNI FCR 1 jnicl46x 0 JNI FCRO is pci 1f 2000 JNI FCR 1 jnicl46x0 Link Failure event If you see these entries then there is a good chance that you have something incorrect in your configuration file for the adapter Normally a good candidate is the WWN entry for the device Review your settings carefully If you make changes to any of the conf files you must restart again with the reboot r comman
198. the use of SES SCSI Enclosure Services gt With Dynix ptx V4 5 x and V4 6 x the switch domain ID is limited by the switch type These versions of Dynix ptx do support the SES functions With Silkworm 1000 type switches domain IDs of 0 31 can be used With the Silkworm 2000 IBM 2109 type switches domain IDs of 1 239 can be used To avoid conflict with SES default LUNO do not use the domain ID O gt When using Silkworm 2000 IBM 2109 type switch in a mixed cascaded environment with a Silkworm 1000 switch you must ensure that it is configured to run in compatibility mode For more details on these and other considerations see the documentation on guidelines for FC Switches under Fibre Channel at http webdocs numaq ibm com Introducing hosts to the SAN fabric Compag AlphaServer Fibre Channel Implementation In this chapter we describe the steps involved in implementing Fibre Channel host adapter cards in a Compaq AlphaServer running Tru64 UNIX operating system and its connection to the appropriate SAN switch The following topics are discussed Introduction on page 128 Fibre Channel host bus adapters on page 128 Supported SAN switches on page 129 Storage on page 131 Installation procedures on page 132 Installing the storage system on page 136 vvvvvy Copyright IBM Corp 2002 127 6 1 Introduction Tru64 UNIX running on Compaq AlphaServer d
199. this area See Section 5 8 Specific limitations on page 126 to ensure your planned environment will work with the version of Dynix ptx you are using Further details on establishing cascaded fabrics is covered in detail in the multi switch sections of SAN x430 NUMA Q Configurations at http webdocs numaq ibm com 5 4 Supported storage on the NUMA Q fabric The NUMA Q system supports a great number of storage devices on its fabric It also has its own fibre to SCSI bridge IBM feature code 3019 allowing it to connect a number of SCSI devices as well For some SCSI tape support the SAN Data Gateway SDG IBM 2108 is used Table 5 3 is a list of the standard supported direct fibre disk devices that are available for NUMA Q Table 5 3 IBM Fibre Channel attached disk storage systems Storage system Method of attachment Operating systems IBM 3542 3560 FAStT200 Attach by both fibre switch Supported with Storage Server withup totwo or FC AL DYNIX ptx V4 4 8 and EXP500 disk expansion unit higher V4 5 2 and higher and V4 6 1 IBM 2105 Enterprise Storage Switched or FC AL direct Supported with Server ESS attached from the FC DYNIX ptx V4 5 2 and HBA maximum of 128 higher and V4 6 1 LUNs per LSS supported V4 4 10 see note with DYNIX ptx V4 4 10 below EMC Symmetrix Storage Switched or FC AL direct Supported with Subsystems Switched or attached from the FC HBA DYNIX ptx V4 5 2 and FC AL direct attached from hig
200. to communicate with the server functions a full X500 based directory service mapping of the Simple Network Management Protocol SNMP directly to the Fibre Channel a time server and an alias server FC LE Fibre Channel Link Encapsulation A reference to the document ANSI X3 287 1996 which defines how IEEE 802 2 Logical Link Control LLC information is transported via the Fibre Channel FC PH A reference to the Fibre Channel Physical and Signaling standard ANSI X3 230 containing the definition of the three lower levels FC 0 FC 1 and FC 2 of the Fibre Channel FC PLDA Fibre Channel Private Loop Direct Attach See PLDA FC SB Fibre Channel Single Byte Command Code Set A reference to the document ANSI Glossary 165 X 271 1996 which defines how the ESCON command set protocol is transported using the fibre channel FC SW Fibre Channel Switch Fabric A reference to the ANSI standard under development that further defines the fabric behavior described in FC FG and defines the communications between different fabric elements required for those elements to coordinate their operations and management address assignment FC Storage Director See SAN Storage Director FCA Fibre Channel Association a Fibre Channel industry association that works to promote awareness and understanding of the Fibre Channel technology and its application and provides a means for implementers to support the standards committee activities
201. uter or both concurrently A bridge router can route one or more protocols such as TCP IP and bridge all other traffic See also Bridge Router Broadcast Sending a transmission to all N_Ports on a fabric Channel A point to point link the main task of which is to transport data from one point to another Channel I O A form of I O where request and response correlation is maintained through some form of source destination and request identification CIFS Common Internet File System Class of Service A Fibre Channel frame delivery scheme exhibiting a specified set of delivery characteristics and attributes 163 Class 1 A class of service providing dedicated connection between two ports with confirmed delivery or notification of non deliverability Class 2 A class of service providing a frame switching service between two ports with confirmed delivery or notification of non deliverability Class 3 A class of service providing frame switching datagram service between two ports or a multicast service between a multicast originator and one or more multicast recipients Class 4 A class of service providing a fractional bandwidth virtual circuit between two ports with confirmed delivery or notification of non deliverability Class 6 A class of service providing a multicast connection between a multicast originator and one or more multicast recipients with confirmed delivery or notification of non deliverability Client
202. ve the necessary patches loaded we use the following command with the appropriate patch number showrev p grep 105181 perform this command for each patch number Result goes here showrev p grep 105356 Result goes here showrev p grep 105580 Result goes here showrev p grep 105600 made obsolete in our case by 105181 29 Result goes here To check for the driver s existence we run pkginfo grep i jni At this point the system is ready for the adapter to be installed 3 6 2 Installing the FC HBA adapter card In this section we explain the steps we performed to install the JNI FCE 6460 PCI adapter card These steps are the same for installing all PCI based adapters 1 Turn off and unplug the computer 2 Remove the computer case 3 4 Insert the host adapter board into the empty PCI bus slot Press firmly until Remove the blank panel from an empty PCI bus slot seated 5 Secure the mounting bracket for the adapter to the case with the panel screw 6 Replace the computer case by tightening the screws on the case or use the clamp to secure the cover 3 6 3 Switch setup In this section we explain the steps we took to install the switch and prepare it to be connected Chapter 3 Sun Fibre Channel implementation 77 78 1 Using the switch s local control panel we set the switch to its default settings and password and also set the TCP IP address 9 43 231 91
203. w emulex com Following are the prerequisites related to the operating system level for the Emulex FC adapters mentioned gt Windows NT 4 0 Service Pack 3 service Pack 6 recommended gt Windows 2000 Service Pack 2 recommended but not necessary These are the basic features in common for all the Emulex Fibre Channel HBAs listed in Table 1 1 on page 2 gt Point to Point Arbitrated Loop and Switched Fabric topologies are supported gt FC tape support gt Concurrent use of multiple protocols SCSI and IP Chapter 1 Windows Fibre Channel implementation 5 1 2 3 JNI Fibre Channel host bus adapter cards JNI offers four FC host cards supported in Windows NT and 2000 environments The minimum system requirement related to Windows NT is service Pack 4 Additional technical specifications as well as the latest drivers and firmware levels can be found at http www jni com The JNI adapters listed in Table 1 1 on page 2 support the Point to Point Arbitrated Loop and Switched Fabric topologies Note FC tape devices are currently not supported by all JNI Fibre Channel HBAs mentioned in the Table 1 1 on page 2 1 3 Supported SAN fabric components The SAN fabric components that are supported in the Windows NT and 2000 environment are covered in the following sections gt IBM 2109 portfolio gt INRANGE gt McDATA 1 3 1 General information There are two types of Gigabit Interface Converters GBIC and three types of
204. work A geographically dispersed network under the auspices of one organization Entity In general a real or existing thing from the Latin ens or being which makes the distinction between a thing s existence and it qualities In programming engineering and probably many other contexts the word is used to identify units whether concrete things or abstract ideas that have no ready name or label ESCON Enterprise System Connection Exchange A group of sequences which share a unique identifier All sequences within a given exchange use the same protocol Frames from multiple sequences can be multiplexed to prevent a single exchange from consuming all the bandwidth See also Sequence F_Node Fabric Node a fabric attached node F_Port Fabric Port a port used to attach a Node Port N_Port to a switch fabric Fabric Fibre Channel employs a fabric to connect devices A fabric can be as simple as a single cable connecting two devices The term is most often used to describe a more complex network utilizing hubs switches and gateways Fabric Login Fabric Login FLOGI is used by an N_Port to determine if a fabric is present and if so to initiate a session with the fabric by exchanging service parameters with the fabric Fabric Login is performed by an N_Port following link initialization and before communication with other N_Ports is attempted FC Fibre Channel FC 0 Lowest level of the Fibre Channel Physical standard cover
205. ws NT Windows NT 4 0 SERVICE PACK 4 is required when installing FCE 3210 and FCE 6410 For FCE2 6410 and FCE2 6412 JNI recommends that you install Service Pack 5 or greater to ensure proper operation of your system Installation procedure 1 Shut down the system 2 Install the adapter card in the host system 3 Turn on the system 4 On the Windows NT taskbar select Start Settings and Control Panel 5 Double click the SCSI Adapter icon The SCSI Adapter window displays 6 Select the Drivers tab and then click Add 7 Click Have Disk 8 Enter the path to the Driver s location and click OK 9 Select the driver and select OK to install necessary files 1 0 Click Yes to restart the system Note To see storage targets under Windows NT you must install the EZ Fibre Utility in order to map the target storage device see 1 5 10 JNI configuration utility on page 22 1 5 10 JNI configuration utility 22 In order to configure and monitor adapter status we recommend that you install the EZ Fibre Configuration Utility for Windows NT and Windows 2000 You can download it from http www jni com It is found under the drivers section The minimum Windows NT requirement for EZ Fibre Configuration Utility implementation is Service Pack 6 The EZ Fibre Configuration utility provides following features Displays the JNI adapters installed in a system Displays the Fibre Channel targets and devices currently visible t
206. x 107 108 Dynix ptx V4 4 10 108 126 Dynix ptx V4 5 3 108 117 Dynix ptx V4 5 x 126 Dynix ptx V4 6 1 108 E E3000 60 EFC Manager 26 102 Emulex 2 4 5 11 16 35 62 81 82 95 109 141 155 Emulex driver 17 82 Emulex LP6000 109 Emulex LP8000 67 95 141 Emulex LP8000DC 95 141 Emulex LP850 141 Emulex LP9000 95 141 Emulex LP9002DC 95 141 Emulex LP9002L 95 141 Emulex LP9402DC 141 Emulex LP952L 141 Emulex tools 157 enemy 116 Enterprise Storage Server 60 ES 1000 9 99 145 eServer 33 140 ESS 119 122 125 128 138 existing adapters 2 extended distance 6 130 Extended LUN support 4 EZ Fibre Configuration Utility 22 F F_Port 4 103 137 151 160 fabric 96 107 110 123 129 142 fabric attachment 160 fabric information 124 fabric installation 117 fabric levels 108 fabric attached 97 131 FAStT200 114 146 FAStT500 146 182 Introducing hosts to the SAN fabric FAStT700 146 FC AL 11 36 60 95 96 142 fiber 7 130 fiber optic cable 6 Fibre Channel adapter manufacturer 2 Fibre Channel Arbitrated Loop 36 filesystem 105 firmware 5 6 11 15 112 129 139 firmware package 117 FL_Port 4 G GBIC 6 36 96 97 122 130 143 Gigabit Interface Converters 6 36 96 130 143 Gigabit Link Module 110 GLM 110 122 H HBA configuration files 65 headers 147 heterogeneous 113 high speed communication 7 96 130 143 host adapter placement 111 host bus adapter 1 hot pluggable adapters 101
207. xample 4 2 Example 4 2 Checking the FC HBA parameters fcmsutil dev td0 Vendor ID is 0x00103c Device ID is 0x001028 TL Chip Revision No is 2 3 PCI Sub system Vendor ID is 0x00103c PCI Sub system ID is 0x000006 Topology PTTOPT_FABRIC Local N Port id is 0x021700 N_Port Node World Wide Name 0x50060b00000781df N Port Port World Wide Name 0x50060b00000781de Driver state ONLINE Hardware Path is 0 4 0 0 Number of Assisted I0s 1188 Number of Active Login Sessions 0 You can find the World Wide Name WWN of the adapter as the value of the N_Port Port World Wide Name parameter You need to know the WWN to configure your storage system You can find the protocol the adapter is using as the Topology parameter It should be PTTOPT_FABRIC for switched fabric attachment You can also see where in the switch fabric our card is attached to by looking at the hexadecimal Local N_Port_id is parameter In our case it is 0x021700 From the leftmost byte byte 1 02 is the domain id of the switch this adapter is connected to the third byte is 17 where 1 is the switch board within the switch and 7 is the port number within the switch board Byte 3 is always 00 for a switch attached adapter 11 Optionally check to see that the port is correctly configured on the switch The procedure you need to use is different for each switch vendor We show the procedure for the IBM 2109 family of switches in Exampl
208. y root clyde qla2x00 Shut down your system Install the QLogic adapter 10 Connect the adapter to the Fibre Channel network 11 Start up the system and log in as root Introducing hosts to the SAN fabric Note During the startup Linux may notice that you have installed the adapter and want to configure it Let Linux configure the adapter by selecting Configure 12 Rebuild the module dependencies file root clyde root depmod a root clyde root 13 Load the module as follows root clyde root modprobe qla2x00 root clyde root The command may take some time to complete 14 Check the kernel messages with dmesg command as in Example 7 1 Example 7 1 Checking the kernel messages from loading of QLogic driver root clyde root dmesg qla2x00 detect found an HBA qla2x00 VID 1077 DID 2200 SSVID 1077 SSDID 2 scsi Found a QLA2200 bus 1 device 0x4 irq 11 iobase 0x5200 scsi 2 Configure NVRAM parameters scsi 2 Verifying loaded RISC code scsi 2 Verifying chip scsi 2 LIP reset occurred scsi 2 Waiting for LIP to complete scsi 2 LOOP UP detected spurious 8259A interrupt IR07 qla2100 Performing ISP error recovery ha f70e0078 scsi 2 LIP reset occurred scsi 2 Waiting for LIP to complete scsi 2 Waiting for LIP to complete qla2100 configure hba Get host loop ID failed scsi qla0 adapter node 200000e08b0190ef scsi qla0 adapter port 210000e08b0190ef
209. y to shut down and restart the system We do this with the following command reboot r This will cause the kernel to reconfigure during the reboot When reboot completes we should be able to see our devices by simply running the following command format Searching for disks done AVAILABLE DISK SELECTIONS 0 c0t0d0 lt SUN9 0G cyl 4924 alt 2 hd 27 sec 1335 pci 1f 4000 scsi 3 sd 0 0 Introducing hosts to the SAN fabric 8 c2t1d0 lt IBM 2105F20 0 38 cyl c2tldl lt IBM 2105F20 0 38 cyl c2t1d2 lt IBM 2105F20 0 38 cyl c2tld3 lt IBM 2105F20 0 38 cyl c2t1d4 lt IBM 2105F20 0 38 cy c2tld5 lt IBM 2105F20 0 38 cyl c2tld6 lt IBM 2105F20 0 38 cyl pci 1f 4000 scsi 3 sd 2 0 pci 1f 4000 scsi 3 sd 3 0 pc1 1f 2000 JNI FCR 1 sd01 0 m pci 1f 2000 JNI FCR 1 sd 1 pc1 1f 2000 JNI FCR 1 sd01 2 pc1 1f 2000 JNI FCR 1 sd01 3 pci 1f 2000 JNI FCR 1 sd 1 4 pci 1f 2000 JNI FCR 1 sd 1 5 pc1 1f 2000 JNI FCR 1 sd01 6 15256 15256 15256 15256 15256 15256 15256 c0t2d0 lt SUN18G cyl 7506 alt 2 hd 19 sec 2485 C0t3d0 lt SUN18G cyl 7506 alt 2 hd 19 sec 2485 alt 2 hd alt 2 hd alt 2 hd alt 2 hd alt 2 hd alt 2 hd alt 2 hd tre SOL6 tre so17 30 sec 645 30 sec 645 30 sec 645 30 sec 645 30 sec 645 30 sec 645 30 sec 645 If devices do not get recognized with the JNI 6460 adapter check for the following in the boot messages of
210. your server z Server Support Planning Problem Solving Hard c e Fixes Library xes for AIX V4 environment Education 3 Fixes for Linux Services Sitemap Service bulletins and securities advisories a 3 Preventive maintenance News Search technical databases Events Troubleshooting to buy 3 Tips and howto About IBM Privacy Legal Contact Document Done SS Sse ig a Figure 2 1 IBM eServer pSeries Support Chapter 2 eServer pSeries Fibre Channel implementation 41 We use the following hardware and software in our installation gt gt gt IBM RS 6000 model F80 with AIX 4 3 3 0 9 installed IBM Gigabit Fibre Channel Adapter f c 6227 AIX 4 3 3 maintenance level 9 packages in directory install aix43309 Latest device driver updates for the Fibre Channel adapter also located in directory install aix43309 devices pci df1000f7 diag 4 3 3 76 devices pci df1000f7 com 4 3 3 76 bos rte lvm 4 3 3 76 Latest microcode for the Fibre Channel adapter version 02903290 3 22A0 2 5 2 Installing the FC HBAs The following instructions explain how to install the FC HBAs 42 1 2 Power on the system if it is not already on and log in as user root Check that you have the correct AIX maintenance level installed as shown in Example 2 1 Example 2 1 Checking the AIX maintenance level instfix i grep 4330 09 AIX ML 3 A11 filesets for 4330 09 AIX
211. ystem 3 Turn on the system 4 On the Windows NT taskbar select Start Settings and Control Panel 5 Double click the SCSI Adapters icon The SCSI Adapter window displays 6 Select the Drivers tab and then click Add 7 Click Have Disk 8 Enter the path to the Driver s location and click OK 9 Select the driver and select OK to install necessary files 10 Click Yes to restart the system Chapter 1 Windows Fibre Channel implementation 17 The adapter contains two green LEDs and one yellow LED Immediately after successful installation of the adapter and the appropriate driver the green LED in the middle is ON indicating that the adapter has power As soon as the adapter is connected to the SAN switch or to a storage device directly both green LEDs are ON as well as the yellow one which signifies port activity as shown in Figure 1 8 lo gt gt E mbedded optical port j LED j a E Figure 1 8 Emulex LP8000 Fibre Channel HBAs Status LEDs 1 5 7 Emulex configuration utility 18 During the Emulex Fibre Channel HBAs driver installation Light Pulse utility installation occurs Light Pulse is a driver utility for Windows NT and Windows 2000 which allows you to modify the registry and upgrade adapter firmware In addition to this you can view adapter information and status In order to run the Light Pulse utility click the Start button and click Run In the Open section write down Lputilnt

Downloading - ps-2.kev009.com, an archive of old documentation

Contents

Download Pdf Manuals

Related Search

Related Contents