Home

EDM01-31v3 DAG_5.4SGA

1. Before configuring the DAG card Before configuring the FPGA you should ensure that dagmem has been run and memory allocated to each installed DAG card e dagload has been run so that all DAG drivers have been installed Refer to the Installing the drivers section for the required Operating system in EDM04 01 DAG Software Installation Guide for the further details 2008 Endace Technology Ltd Confidential Version 3 November 2008 13 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Firmware images The following lists the features available on each firmware image available on this DAG card FPGA image 0C3 OC 12 10 100 1000 Enhanced Raw Software version string STM 1 Ethernet packet SONET processing capture dag54sga48pci ipf fbmr4tl stm4 bit dag54sga48pci ipf fbmr4tl stm4 dag54sga48cp ipf fbmr4tl stm4 bit dag54sga48cp ipf fbmr4tl stm4 dag54sga48pci ipf fbmr4tl stml6 bit dag54sga48pci ipf fbmr4tl1 Sieme TREN dag54sga48cp ipf fbmr4tl stml6 bit dag54sga48cp ipf fbmr4tl stml6 dagb54sga48pci ipf fbmr4tl eth bit dag54sga48pci ipf fbmr4tl eth dag54sga48cp ipf fbmr4tl1 eth bit dag54sga48cp ipf fbmr4tl eth dag54sga48pci raw sdh stm4 bit dag54sga48pci raw sdh stm4 dag54sga48pci raw sdh stml6 bit dag54sga48pci_raw sdh stml16 The software version strings are displayed in the dagconfig output and when using the dagrom x command They inclu
2. This record format is for the Ethernet links 802 3 incorporating filter results The record format is the same type as the Type 2 TYPE_ETH page 63 record with the exception that the Ictr field is reassigned as color Requires Endace Coprocessor and appropriate firmware The TYPE_COLOR_ETH variable length record is shown below _byte3 byte2 bytet _byted Extension header s Protocol header Padding Payload The following is a description of the TYPE_COLOR_ETH record format NW eee eee offset 1 byte Pad 1 byte Payload bytes of record The color field is a hardware generated tag indicating the result of a filtering or classification operation This field is divided into the following Bit Description fo Set if the record should have been sent to receive stream 0 Set if the record should have been sent to receive stream 2 2 15 A14 bit unsigned integer that corresponds to the filter rule this packet matched Number of bytes not captured from the start of the frame This is typically used to skip link layer headers when they are not required in order to save bandwidth and space Note This field is currently not implemented contents should be disregarded The Ethernet frame begins immediately after the pad byte so that the layer 3 IP header is 32 bit aligned This is typically used to skip link layer headers when they are not required in order to save bandwidth and space Payl
3. 2008 Endace Technology Ltd Confidential Version 3 November 2008 27 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide 28 relative Sets TR TERF to Relative Timed Release mode In Relative Timed Release Mode each packet is transmitted according to its timestamp relative to the first packet in the transmit stream This is shown in the following example where Packet A will be transmitted immediately Packet B will be transmitted one second later and Packet C will be transmitted one second after that Packet A Timestamp 2006 07 24 03 45 55 9720326 UTC Packet B Timestamp 2006 07 24 03 45 56 9720326 UTC Packet C Timestamp 2006 07 24 03 45 57 9720326 UTC If for any reason the transmission of Packet B is delayed the module will still attempt to transmit Packet C two seconds after Packet A It will do this even if it means reducing the gap between Packet B and Packet C When using multiple output ports in this mode the timing for each port is independent This is shown in the following example where Packet A and Packet B are transmitted at the same time even though their timestamps are one second apart Note The relative placement of the packets is accurate to 100ns Packet A Port A Timestamp 2006 07 24 03 45 55 9720326 UTC Packet B Port B Timestamp 2006 07 24 03 45 56 9720326 UTC Example dagconfig relative no delay Disables Relative Timed Release mode No Delay Mode In No Delay Mode
4. rer rtr treten attore eee P EXER does o serere area 54 Connecting the Time Distribution Server esses nennen nennen enne 54 Testing the Signal sisisi 54 Single Card No Referente M 55 Two Cards No References P O 56 uud M 56 Synchronizing with Each Other tret tetti tnnt rk tee na iE oon 56 Synchronizing with EFlost eset eter to tritt th I e EEEE E EEEE e E a 57 Gonnector Pin o ts 5 ite eacus dee fte E este ere itr t eia EET HE uie D fe ria Pee ere EST 58 COVELVICW e E 58 Pin Assignments suc eee eeu hide enie e e te cru e d oie 58 Data Formats 59 OVerView isle E EIL EI M Vnd EM D DN td ca DM LM 59 Generic ERE Header oce eR RO DRE ER RNEPRRD OR E ERE RIED REDDE REM 60 ERE T TYPE POS HDLC 22e aeter oerte ev o e eto tenti erobert i SS 62 ERP2CDYPESEITHES a enero anos HOD RR OR USO RU RENI IRE NE 63 ERE 102 TYPE COEOR HDEG POSDs a tton door teettete e ret estoit ete nr ER uns 64 ERFIR TYPE COLOR ETH re ense eee REO Re RO REPRE HM 65 BERE T5 LYPE DSM COLOR HDEG POS tereti eost tee ee eto tente ertet tein hoe 66 ERE6 TYPE DSM COEOR ETEL intet ee RERO RI EU REIR REP REESE E Re ER Ie Hie 67 ERE T9 TYRE COLOR HASH POS ettet a E te ber ee tto DERE ERENE EE yey 68 BRE20 TYPE COLOR HASA BIBLE
5. Example dagconfig overlap dagconfig nooverlap Note This option is only applicable on firmware images containing TX PCIx Describes the following information about the DAG card e Bus speed e buffer size Example PCI Burst Manager 133MHz buffer_size 128 pmin nopmin pmincheck Enables pmin or disables nopmin the discard of packets smaller than the pre defined minimum size Example dagconfig pmin dagconfig nopmin pos Set framer into Packet over SONET PoS mode Example dagconfig pos pscramble nopscramble Enables Disables the scrambling mode on the PoS Mapper Demapper Example dagconfig pscramble dagconfig nopscramble raw Set framer to capture Raw SONET frames See also Scramble mode for Raw SONET capture page 26 Example dagconfig default raw reset noreset Resets the images to the default state Clears the Port drop counters All settings done before are kept in place Example dagconfig reset dagconfig noreset 2008 Endace Technology Ltd Confidential Version 3 November 2008 25 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide 26 rx and tx Streams Indicates the number of rx and tx streams are available on the DAG card Not configurable Stream information relates to the setting of mem page 24 rxonly Configures the memory hole to only receive Example dagconfig rxonly rxtx Enables both transmit and receive and splits the memory hole for rx and tx This allocates 16MB
6. Number of bytes not captured from the start of the frame This is typically used to skip link layer headers when they are not required in order to save bandwidth and space Note This field is currently not implemented contents should be disregarded The Ethernet frame begins immediately after the pad byte so that the layer 3 IP header is 32 bit aligned This is typically used to skip link layer headers when they are not required in order to save bandwidth and space Payload rlen ERF header 16 bytes Extension headers optional Padding 2 bytes 2008 Endace Technology Ltd Confidential Version 3 November 2008 67 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide ERF 19 TYPE COLOR HASH POS Type Bit7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 19 Short description TYPE COLOR HASH POS Long description Type 19 Colored PoS HDLC record with Hash load balancing Use This record format is for data links incorporating filter results The record format is the same type as the Type 1 POS HDLC page 62 record but with IPF color and hash value instead of the loss counter field The TYPE COLOR HASH POS record is shown below byte3 byte bytet byte yee tas en ERF header Extension header s Protocol header Padding Payload The following is a description of the TYPE_COLOR_HASH_POS record format Hed Deseiption 0
7. Specify the image number to write or to program the card 0 3 0 factory image 1 user image 1 2 user image 2 3 user image 3 7 5G2 G4 only swid write lt swid gt Write given SWID The key must be supplied with the m option requires a valid running XScale ROM Image 3 7T 3 7D 3 8S and 7 15 only r reprogram Reprogram ROM may imply erase and write reset method reprogram method Specify the method to reprogram the card 1 Ringo 2 George 3 Dave S swid rom writ lt swid gt Write given SWID to ROM The key must be supplied with the m option t Swid read bytes bytes Read bytes of SWID requires a valid running XScale ROM image 3 7T only u swid eras Erase SWID from ROM unknown Force loading firmware Dangerous v verbose Increase verbosity V version Display version information w write Write ROM implies erase Default is read write out filename Write the contents of the ROM to a file x list revisions Display Xilinx revision strings the default if no arguments are given y verify Verify write to ROM Z Zero Zero ROM Default is read All commands apply to the current image portion of the ROM unless one of the options a A c is specified Note Not all commands are supported by all DAG cards 2008 Endace Technology Ltd Confidential
8. Variable length varlen pen capture This is not a configurable option Only the first 9616 bytes of GPPO 4 Y each packet will be captured varlen slen 9616 align64 4 Records will be generated with 64 bit Port A drop count 0 alignment This is not a configurable option Port B drop count 0 lt Number of packets dropped during the current capture sesion on this port Stream 0 stream drop count 12172 Stream 2 stream drop count 19114 Stream 4 stream drop count 19115 Stream 6 stream drop count 30214 Allows each memory PCI Burst Manager hole to operate Total memory in MB available independently 133 MHz Y for allocation to this card buffer size 512 rx streams 4 tx streams 1 drop nooverlap Memory Streams mem I24UEGiJ2A4SOPEZASUISOAEU trie pond txetesm e n MB TERF terf strip32 rx error a tx error b off time mode relative Te olay poy lotta ue fime mode Packets with RX errors are either discarded off notxrxerror or transmitted txrxerror Raw SONET 0C3 12 Firmware dag54sga48pci raw sdh stm4 v2 1 4vlx40ff1148 2008 10 7 16 12 37 user Copro dag54sga48cp ipf fbmr4tl stm4 cp v2 1 4vl1x40ff1148 2008 9 25 17 45 56 user Serial 0 Sets eql or unsets noeql equipment loopback Sets fcl or unsets nofcl facility loopback Note eql loops back to the PC bus ote fcl loops back to the line Port A laser nophy bist enable noeql nofcl phy tx clock off phy
9. A entire rom Entire ROM Default is current half only Ds swid rom check Check if there is a SWID on the ROM c cpu region region Access CPU region c copro b boot k kernel f filesystem continue Continue on erase error d device device DAG device to use e erase Erase ROM Default is read F disable cfi fast Disable fast program option for CFI mode f file filename File to be read when programming ROM There are multiple FGPA images per DAG card covering the different versions ERF TERF DSM etc force Force loading firmware Dangerous g rom number rom Access specified ROM controller Default is 0 h help usage This page i halt ixp Halt the embedded IXP Processor DAG 7 18 only image table fpga image table fpga gt Specify the Power On image selection table FPGA number image table imag image table image Specify the Power On image selection table Image number j swid rom check key key Check the ROM SWID key with the one supplied l1 hold bus Hold PBI bus from XScale DAG 3 7T only m swid key key Hexadecimal key for writing the Software ID aka SWID o swid rom read Read SWID from ROM p program current Program current User 1 Xilinx image into FPGA q image number image number
10. EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Version History Version Date Reason 1 June 2008 First release Based on 5 0SG2A Added TR TERF information Added ERF Extension header information 2 August 2008 Released Nonvarlen dagconfig option not recommended for use on this DAG card Added information TR TERF Head of line blocking known issue 3 November 2008 Updated Buffer_size and mem dagconfig tokens and associated cross references Updated front matter Update dagconfig options table Added new dagrom options Supported OS information now in release notes Added external power supply information Added card description to the overview Added Enhanced packet processing information Removed TCP IP filtering information Added Raw SONET capture information stotus Demon O Preliminary The products described in this technical document are in development and have yet to complete final production quality assurance Released The products described in this technical document have completed development and final production quality assurance 2008 Endace Technology Ltd Confidential Version 3 November 2008 75
11. Number of bytes that were not captured from the start of the frame This is typically used to skip link layer headers when they are not required in order to save bandwidth and space This field is currently not implemented contents should be disregarded The Color Ethernet frame begins immediately after the pad byte so that the layer 3 IP header is 32 bit aligned This is typically used to skip link layer headers when they are not required in order to save bandwidth and space Payload rlen ERF header 16 bytes Extension headers optional Protocol header 2 bytes 2008 Endace Technology Ltd Confidential Version 3 November 2008 69 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide ERF 24 TYPE_RAW_LINK Type Bit 7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 24 Short description TYPE RAW LINK Long description Type 24 Raw link data typically SONET or SDH Frame Use Used in Raw SONET SDH capture Used with Extension Header 5 The TYPE RAW LINK record is shown below Key _ ERF header Extension header s Protocol header mn Padding n Payload The following is a description of the TYPE RAW LINK record format Hed fi eseription Payload Payload rlen ERF header 16 bytes Extension headers optional bytes of record 70 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 4
12. 4 The DAG card can also output a synchronization pulse for use when synchronizing two DAG cards i e without a GPS input The synchronization pulse is output on the Out PPS channel pins 1 and 2 To connect two DAG cards use a DUCK crossover cable page 58 to connect the two time synchronization sockets Note The DAG card is supplied with a 4 pin to RJ45 adapter DUCK Crossover cable To synchronize two DAG cards together use a cable with RJ45 plugs and the following wiring TX PPS 1 3 RX PPS TX PPS 2 6 RX PPS RX PPS 3 1 TX PPS RX SERIAL 4 7 TX SERIAL RX SERIAL 5 8 TX SERIAL RX PPS 6 2 TX PPS TX SERIAL 7 4 RX SERIAL TX SERIAL 8 5 RX SERIAL Note This wiring is the same as an Ethernet crossover cable Gigabit crossover All four pairs crossed 4 Pin to RJ45 adapter The 4 pin to RJ45 adaptor pin assignments are shown below no meo 2 1 msow 5 mm mm 3 58 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Data Formats Overview The DAG Card produces trace files in its own native format called ERF Extensible Record Format The ERF type depends upon the type of connection you are using to capture data The DAG 5 4SGA 48 supports the following ERF Types ERF Type Description 1 TYP
13. Colored Ethernet variable length record with Hash load balancing ETH 21 TYPE INFINIBAND Infiniband Variable Length Record 22 TYPE IPVA4 IPV4 Variable Length Record 23 TYPE IPV6 IPV6 Variable Length Record 24 TYPE RAW LINK Raw link data typically SONET or SDH Frame 32 47 Reserved for CoProcess Development Kit CDK Users and Internal use 48 TYPE PAD Pad Record type 2008 Endace Technology Ltd Confidential Version 3 November 2008 61 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide ERF 1 TYPE POS HDLC Type Bit7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 1 Short description TYPE_POS_HDLC Long description Type 1 PoS HDLC Record Use This record format is for HDLC data links For example Packet over SONET Point to Point Protocol PPP over SONET e Frame Relay MTP2 S57 The TYPE_POS HDLC record is shown below oyes bed oner oreo me 3 ERF header Extension header s Protocol header Padding Payload The following is a description of the TYPE POS HDLC record format Hed o o Demipion HDLC Header Protocol Header Length may vary depending on protocol typically 4 bytes 4 bytes Payload bytes of record Payload rlen ERF header 16 bytes Extension headers optional Protocol header 4 bytes 62 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_
14. To prevent the DAG card clock time stamps drifting against UTC the master DAG card can be synchronized to the host computer s clock which in turn utilizes NTP This provides a master signal to the slave DAG card Configure one DAG card to synchronize to the computer clock and output a RS 422 synchronization signal to the second DAG card as follows dagclock d0 none overin overout Output muxin over muxout over status Synchronised Threshold 11921ns Failures 0 Resyncs 0 error Freq 691ppb Phase 394ns Worst Freq 147ppb Worst Phase 424ns crystal Actual 49999354Hz Synthesized 16777216Hz input Total 87464 Bad 0 Singles Missed 0 Longest Sequence Missed 0 start Wed Apr 27 14 27 41 2007 host Thu Apr 28 14 59 14 2007 dag Thu Apr 28 14 59 14 2007 Note The slave DAG card configuration is not shown the default configuration is sufficient 2008 Endace Technology Ltd Confidential Version 3 November 2008 57 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Connector Pin outs Overview DAG cards have a 4 pin connector for time synchronization The connector has one bi directional RS422 differential circuit A The Pulse Per Second PPS signal is carried on circuit A Pin Assignments The 4 pin connector pin assignments and plugs and sockets are shown below PPS Out PPS Out PPS In PPS In Ep ESI Bon ie Normally you connect the GPS input to the PPS A channel input pins 3 and
15. Version 3 November 2008 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide To view the FPGA image revision strings type the following dagrom d0 x where 0 is the device number of the DAG card you wish to capture data from user dag54sga48pci ipf fbmr4tl stml16 v2 1 4v1x40ff1148 2008 11 24 10 33 24 active factory dag54sga48pci ipf eth v2 1 4vl1x40ff1148 2008 6 23 8 41 29 Card Serial 528 user dag54sga48cp ipf fbmr4ti stml16 v2 1 4vl1x40ff1148 2008 9 25 17 37 36 active factory dag54sga48cp ipf eth v2 1 4vl1x40ff1148 2008 4 4 13 12 39 Loading new firmware images onto a DAG Card New DAG card FPGA images are released regularly by Endace as part of software packages They can be downloaded from the Endace website at https www endace com support Endace recommends you use the dagrom r command when loading images from the computer to the ROM on the DAG card The r option invokes a comparison of images on the computer and in the DAG card Newer versions are automatically loaded onto the DAG card and programmed into the FPGA See dagrom page 16 This eliminates unnecessary reprogramming of the ROM and extends its life Preparing the DAG card for use Before configuring the DAG 5 4SGA 48 card you must run the following dagconfig command to set the default parameters in the DAG card This ensures the DAG 5 4SGA 48 card functions correctly once you begin capturing data Note Ensure you run this command each time the F
16. cards DUCK Configuration The DUCK DAG Universal Clock Kit is designed to reduce time variance between sets of DAG cards or between DAG cards and coordinated universal time UTC You can obtain an accurate time reference by connecting an external clock to the DAG card using the time synchronization connector Alternatively you can use the host computer s clock in software as a reference source without any additional hardware Each DAG card can also output a clock signal for use by other DAG cards Common Synchronization The DAG card time synchronization connector supports a Pulse Per Second PPS input signal using RS 422 signaling levels Common synchronization sources include GPS or CDMA cellular telephone time receivers Endace also provides the TDS 2 Time Distribution Server modules and TDS 6 expansion units that enable you to connect multiple DAG cards to a single GPS or CDMA unit For more information please refer to the Endace website at https www endace com support or the EDM05 01 Time Distribution Server User Guide 2008 Endace Technology Ltd Confidential Version 3 November 2008 47 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Network Time Protocol 48 NTP Network Time Protocol can be used to synchronize a computer clock to a network based reference When the NTP daemon starts it exchanges packets with network time servers to establish the correct time If the computer clock is significantly differe
17. causing the transmission of packets on all other ports to also be delayed This type of problem is often inherent in systems where one transmit buffer is supplying multiple output ports When transmitting across multiple ports Endace recommends that packets for all port be interleaved into the transmit buffer in the appropriate time order i e with strictly increasing timestamps This minimizes the chance of head of line blocking occurring Should head of line blocking occur it will not stop packet transmission rather the inter packet timing of the transmit stream on some ports will not faithfully replicate the time stamps If guaranteed inter packet timing is required Endace recommends that TR TERF be enabled on only one interface at a time Applicable to Multi port DAG cards only 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Synchronizing Clock Time Overview DAG cards have sophisticated time synchronization capabilities This allows for high quality timestamps and optional synchronization to an external time standard The core of the DAG synchronization capability is known as the DAG Universal Clock Kit DUCK A clock in each DAG card runs independently from the computer clock The DAG card s clock is initialized using the computer clock and then free runs using a crystal oscillator Each DAG card s clock can vary relative to a computer clock or other DAG
18. d0 rvp f dag54sga48pci ipf sdh stm4 bit To program the Co Processor type the following dagrom d0 rvp cc f dag54sga48cp ipf sdh stm4 bit SONET 0C48 STM 16 To program the FPGA use one of the following dagrom d0 rvp f dagb54sga48pci ipf sdh stml 6 bit To program the Co Processor type the following dagrom d0 rvp cc f dag54sga48cp ipf sdh stml6 bit Ethernet To program the FPGA type the following dagrom d0 rvp f dagb54sga48pci ipf eth bit To program the Co Processor type the following dagrom d0 rvp cc f dag54sga48cp ipf eth bit where 0 is the device number of the DAG card you wish to capture data from The filename of the FPGA image may differ from the above depending on the version required For details on using the Co Processor see emd04 26 Raw SONET 0C3 0C12 STM 1 STM 4 To program the FPGA type the following dagrom d0 rvp f dag54sga48pci raw sdh stm4 bit Raw SONET 0C48 STM 16 To program the FPGA type the following dagrom d0 rvp f dag54sga48pci raw sdh stml6 bit 2008 Endace Technology Ltd Confidential Version 3 November 2008 15 EDMO01 31v3 DAG_5 4SGA 48 Card User Guide 16 dagrom dagrom is a software utility that enables you to configure the FPGA on Endace DAG cards The following is a list of options available in dagrom Option Description a alternate half Use alternate stable half Default is current half Factory User
19. e The color field is a hardware generated tag indicating the result of a filtering or classification operation This field is divided into the following et Deep 00000 Hash Value IPF Color HDLC header Protocol header Length may vary depending on protocol 4 bytes Payload Payload 7 rlen ERF header 16 bytes Extension headers optional bytes of record Protocol header 4 bytes 68 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide ERF 20 TYPE_COLOR_HASH_ETH Type Bit 7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 20 Short description TYPE COLOR HASH ETH Long description Type 20 Colored Ethernet variable length record with hash load balancing Use This record is like Type 2 TYPE ETH page 63 but with IPF color and hash value instead of the loss counter field The TYPE COLOR HASH ETH record is shown below type 20 flags rlen ERF header Extension header s Protocol header Padding Payload The following is a description of the TYPE COLOR HASH ETH record format Fed scription Offset 1 byte Pad 1 byte Payload bytes of record The color field is a hardware generated tag indicating the result of a filtering or classification operation This field is divided into the following Ug Hash Value IPF Color
20. on how to use dagbits see dagbits page 39 Note dagbits decodes and displays ERF header fields and packet contents are displayed as a Hex dump only To decode higher level protocols Endace recommends using a third party application see Using third party applications page 43 Examples Test live traffic on dag0 stream 0 for 60 seconds running the lctr flags and fcs tests dagbits vvc d0 0 s60 lctr flags fcs To read a trace log file using dagbits dagbits vvc print f logname log less To check for errors in the trace dagbits vvc ltcr flags fcs f logname log If dagbits reaches the end of the traffic and prints its report then the ERF records were valid dagbits dagbits is a software utility used to view and test ERF records dagbits can receive data from either e directly from the DAG card using the a option or e a ERF data file created by dagsnap The following is a list options available in dagbits Options Description 0 ERF records contain no Link Layer CRCs 16 ERF records contain 16 bit Link Layer CRCs PoS 32 ERF records contain 32 bit Link Layer CRCs PoS and Ethernet a Set legacy format to ATM this is the default b Treat ERF timestamps as big endian c Print real time progress reports as dagbits captures traffic This is a useful indicator that a test is running correctly C NUM Sets CRC correction factor for BTX test 0 16 or 32 bits d DEVI
21. the input and output types v verbose Increase output verbosity V version Select variable length output ERF only Display version information y lt DLT gt This sets the pcap data link type to be used for BPF filtering b and for pcap output Previously only one DLT was mapped to each ERF type Supported DLT types case insensitive EN10MB Ethernet DOCSIS Ethernet CHDLC HDLC PPP SERIAL HDLC MTT2 HDLC ATM_RFC1483 ATM AAL5 SUNATM ATM AAL5 Note Not all options are applicable to all DAG cards 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Using third party applications Once the captured data is in Pcap format you can use third party applications to examine and process the data The third party applications include e Wireshark Tshark formerly Ethereal Tethereal e TCPDump e Libpcap e SNORT e Winpcap etc Note Wireshark can also read ERF formatted data Transmitting captured data Configuration The DAG 5 4SGA 48 card is able to transmit as well as receive packets and can capture received traffic while transmitting This allows you to use capture tools such as dagsnap dagconvert and dagbits while dagflood is sending packets To configure the DAG 5 4SGA 48 card for transmission you must allocate some memory to a transmit stream By default 16 MB of memory is allocated to the tx stream and the remain
22. to not strip the 32 bit CRC and configure the MAC to add a hardware calculated CRC This has the effect of reducing the burden of calculating a CRC in the software Example dagconfig noterf strip tx crc 44 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide Situation 2 The ERF stream was generated with a CRC but the CRC must be correct on the retransmitted packet In this case you should configure TR TERF to strip the 32 bit CRC and configure the MAC to add a hardware calculated CRC as in Configuration 1 above This means even if the CRC is incorrect in the received ERF stream a correct CRC will be generated when the packet is retransmitted Example dagconfig terf strip32 tx crc Situation 3 The ERF stream was generated with a CRC and the CRC must be retransmitted exactly even if it is incorrect In this case you should configure TR TERF not to strip the CRC and configure the MAC not to add a hardware calculated CRC This allows received packets with bad CRCs to be retransmitted which may be a useful tool for testing or diagnostic purposes Example dagconfig noterf strip notx crc Retransmitting Errored Packets In some circumstances it may not be desirable to retransmit packets which have been incorrectly received for any reason To allow for this TR TERF can be optionally configured not to retransmit any packets marked with the rxerror receive error flag U
23. value 190 TX_bytes Port 1 value 1947 TX Error Port 1 value 0 Number of packets received from the line 4 with errors typically CRC errors Value of Number of payload bytes received from the line the counter Number of packets recevied from the line Ropresenia thel por SONET Number of counters in this block Number of counter blocks on the card Type of counters Number of blocks 1 Palas Total number of counters 12 E Nb of counter s in Block ID Sonet Frame Tx Rx 12 RX Frame Port 0 value 157 RX Byte Port 0 value 18743 RX err Port 0 value 0 TX frames Port 0 value 136 TX bytes Port 0 value 19123 TX Error Port 0 value 0 RX Frame Port 1 value 260 RX Byte Port 1 value 3846 RXXerr Port 1 value 0 TX frames Port 1 value 245 TX bytes Port i 1 value 37892 TX Error Bone value Number of packets received from the line with errors typically CRC errors Value of Number of payload bytes received from the line the counter Number of packets received from the line Pac ghar soy vs thoport 2008 Endace Technology Ltd Confidential Version 3 November 2008 33 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Using your DAG card to capture data Introduction This chapter describes how to complete the following operations for a DAG card e Basic data capture page 35 For Enhanced packet processing see EDMO04 26 Enhanced Packet Processing e Viewing captured data
24. when the absolute Phase error page 52 is above the Threshold value for 10 consecutive seconds Threshold This is the value above which the DAG card port is considered Not Synchronized The Threshold value changes depending on the type of input time synchronization The defaults are e 596 for RS422 synchronization e 12000 for host synchronization Unix e 50000 for host synchronization Windows This value can be adjusted using the dagclock 1 option Resyncs This is a count of the number of times the DAG card Phase error has exceeded 1 second See Error Dagclock page 52 If the DAG card is Not Synchronized for more than 10 seconds the DAG card automatically runs the following command to update the time on the DAG card dagclock d0 set Where 0 is the device number Example status Synchronised Threshold 596ns Failures 0 Resyncs 0 Error This line reports on the synthesized frequency of the DAG card output scription An estimate of the synthesized frequency error over the last second in parts per billion The difference between the DAG card s clock and the reference clock at the last time pulse Worst Freq Highest absolute value of the Frequency error since statistic collection began Reset to zero when statistics are reset see Dagclock Statistics reset page 51 Worst Phase Highest absolute value of the Phase error since statistic collection began Reset to zero when statistics are reset see Dagclock Statist
25. 2 cuan t et D ERU REDE RCRUM 69 ERE2 TYPE RAW EINK enine E eO DERE NR EE WT estet UMS 70 Extension Headers EET i 2d REED dede t tei pere tet pod esg he 71 Troubleshooting 73 Reporting Problems etre i e e RE i a oevsodosgossatsspuenbiacnstodbasnovonenensoeatads 73 Version History 75 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Introduction Overview The Endace DAG 5 4SGA 48 card provides the means to transfer data at the full speed of the network into the memory of the host computer with zero packet loss in even worst case conditions Further unlike a Network Interface Card NIC Endace products actively manage the movement of network data into memory while only consuming a minimal amount of the host computer s resources The full attention of the CPU remains focused on the analysis of incoming data without a constant stream of interruptions as new packets arrive from the network For a busy network link this feature has a turbo charging effect similar to that of adding a second CPU to the system The DAG 5 4SGA 48 card is a two port PCIx card that allows capture and transmission of data It is designed for capture and transmission of OC3 OC12 OC48 SONET Raw OC3 12 48c or 10 100 1000 Ethernet network data The PCIx interface enables a capture rate of up to 7 6Gbps This card has a built in Co Processor used for Enhanced packet processing Card Features The f
26. 48 has an additional power supply socket This must be connected to a 12 V power supply using the supplied or equivalent cabling 2008 Endace Technology Ltd Confidential Version 3 November 2008 3 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Battery removal don t do it Removing the battery from a DAG card voids your warranty Removing the battery from a DAG card will cause the loss of encryption key used to decode the DAG card s firmware Once the encryption key is lost the DAG card must be returned to Endace for reprogramming The battery in this product is expected to last a minimum of 10 years Caution Risk of explosion if the battery is replaced by an incorrect type Dispose of used batteries carefully Card Architecture Serial SONET SDH or Ethernet data is received by the two optical interfaces and fed immediately into two physical layer chips The SONET SDH or Ethernet payload is extracted within the FPGA The main FPGA also contains the DAG Universal Clock Kit DUCK timestamp engine which provides high resolution per packet timestamps which can be accurately synchronized The close association of these two components means that packets or cells can be time stamped very accurately The time stamped packet records are stored in the FIFO which interfaces to the PCIx bus via the FPGA For further information on DUCK and time synchronizing please see Synchronizing Clock Time page 47 later in this User Guide All pa
27. 8_Card_User_Guide Extension Headers EH Introduction The addition of an Extension Header into the ERF record allows extra data relating to the packet to be transported to the host The extension header allows certain features to be added independently of ERF types for example features shared by different ERF records do not have to be implemented separately This results in automatic support across ERF types Bit 7 of the ERF type field is used to indicate that Extension Header s are present If set to 1 Extension Headers are present The Extension Header type field indicates the type and format of the Extension Header It also indicates whether further Extension Headers are present If bit 7 of the Extension Header is set to 1 further Extension Headers exist in the record The Extension Headers are 8 bytes in length The following diagram shows presence of an Extension Header in addition to the ERF record byte3 byte2 byte1 byted The following diagram shows presence of two Extension Headers with Bit 7 of the first Extension Header set to 1 byte3 byte2 byte1 byted 2008 Endace Technology Ltd Confidential Version 3 November 2008 71 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Troubleshooting Reporting Problems If you have problems with a DAG 5 4SGA 48 card or Endace supplied software which you are unable to resolve please contact Endace Customer Support at support endace com Supplying as much inform
28. 8pci ipf fbmr4ti stml6 v2 1 4vlx40ff1148 2008 11 24 10 33 24 user Copro dag54sga48cp ipf fbmr4tl stml6 v2 1 4vlx40ff1148 2008 9 25 17 37 36 user Serial 89564 Sets eq or unsets noeql equipment loopback Sets fcl or unsets nofcl facility loopback Note eql loops back to the PC bus ote fcl loops back to the line Port A nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate C phy ref select 0 master enablea config 0 nodiscard data laser nocre sonet scramble pscramble oc48 pos Generate SONET tx clock internally Enables enable lt port gt or disables master or from rx clock slave disable lt port gt port for capture Port B nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate C phy ref select 0 master enableb config 0 nodiscard data laser nocrc sonet scramble pscramble oc48 pos Indicates card is in POS mode Indicates framer receive mode GPPO0 varlen slen 9616 SISSHBH Records will be generated with 64 bit Port A drop count 1 alignment This is not a configurable option Port B drop count 6 Number of packets dropped during the current capture sesion on this port Stream 0 stream drop count 0 Stream 2 stream drop count 0 Stream 4 stream drop count 0 Stream 6 stream drop count 0 Allows each memory PCI Burst Manager jna ag Total memory in MB available 133MHz for allocation to this card buffer size 512 r
29. CE Use the DAG device referred to by DEVICE Supported syntax includes 0 dag1 device DEVICE and dev dag3 to refer to DAG cards and 0 2 dagl 1 and dev dag2 0 to refer to specific streams on cards D NUM Introduces a NUM nanosecond delay between processing each record e Set legacy format to Ethernet default ATM E NUM Halt operation after a maximum of NUM errors This option prevents dagbits from creating extremely large output files when being redirected to a file f FILE Read captured data from FILE h Display usage help information Use API interface for live DAG API capture Possible options are 0 DAG 2 4 legacy API interface dag offset 3 1 DAG 2 5 API interface dag advance stream 3 2 DAG 2 5 API interface dag rx stream next record 3 Assume the ERF contains color information in the pad and offset bytes for Ethernet ERFs or HDLC header bytes for PoS ERFs and display this information as a packet classification and destination memory buffer j NUM Set the threshold for the jitter test to NUM microseconds m NUM Print the first NUM errored records only and then continue to count errors silently for the duration of the session 2008 Endace Technology Ltd Confidential Version 3 November 2008 39 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide n NUM Expected number of packets to receive
30. DEVICE Supported syntax includes 0 dag1 and dev dag3 to refer to DAG cards and 0 2 dag1 0 and dev dag2 0 to refer to specific streams on cards hy 2 help usag Display usage help information J Maximize disk writing performance by only writing data to disk in chunks This option may not be available on all operating systems m NUM Write at most NUM megabytes of data per call to the DAG API default is 4 MiB o FILE Write the captured packets to FILE in ERF format default is standard fname FILE output s NUM runtime NUM Run for NUM seconds then exit v verbose Increase output verbosity When dagsnap is run with this command three columns of data are reported every second These columns contain 1 The cumulative total of data written out from the DAG card 2 The buffer occupancy Small values indicate no packet loss 3 The rate at which data is currently being written V version Display version information w wait SEC Delay wait in seconds before capture and after 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Capturing data at high speed As the DAG 5 4SGA 48 card captures packets from the network link it writes a record for each packet into a large buffer in the host computer s main memory To avoid packet loss the user application reading the re
31. De endace accelerated DAG 5 4SGA 48 Card User Guide EDM01 31 EN EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Protection Against Harmful Interference When present on equipment this manual pertains to the statement This device complies with part 15 of the FCC rules specifies the equipment has been tested and found to comply with the limits for a Class A digital device pursuant to Part 15 of the Federal Communications Commission FCC Rules These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment This equipment generates uses and can radiate radio frequency energy and if not installed and used in accordance with the instruction manual may cause harmful interference to radio communications Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at their own expense Extra Components and Materials The product that this manual pertains to may include extra components and materials that are not essential to its basic operation but are necessary to ensure compliance to the product standards required by the United States Federal Communications Commission and the European EMC Directive Modification or removal of these components and or materials is liable to cause non compliance to these standards and in doing so invalidate the user s ri
32. E_HDLC_POS PoS HDLC Variable Length Record 2 TYPE ETH Ethernet Variable Length Record 10 TYPE COLOR HDLC POS PoS HDLC Variable Length Record 11 TYPE COLOR ETH Colour Ethernet Variable Length Record 15 TYPE DSM COLOR HDLC POS PoS HDLC Variable Length Record with DSM 16 TYPE DSM COLOR ETH Ethernet Variable Length Record with DSM 19 TYPE COLOR HASH POS PoS Variable Length Record with Hash Load Balancing 20 TYPE COLOR HASH ETH Ethernet Variable Length Record with Hash Load Balancing 24 TYPE RAW LINK Raw link data typically SONET or SDH Frame The ERF file contains a series of ERF records with each record describing one packet ERF files consists only of ERF records there is no file header or trailer This allows for simple concatenation and splitting of files to be performed on ERF record boundaries The addition of an Extension Header into the ERF record allows extra data relating to the packet to be transported to the host Raw Link Extra information for ERF 24 TYPE RAW LINK page 70 records For information on other ERF types please refer to EDM11 01 ERF types 2008 Endace Technology Ltd Confidential Version 3 November 2008 59 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Generic ERF Header All ERF records share some common fields Timestamps are in little endian Pentium native byte order All other fields are in big endian network byte order All payload data is captured as a b
33. PGA is reprogrammed dagconfig d0 default where 0 is the device number of the DAG card you wish to capture data from The current DAG 5 4SGA 48 configuration displays and the firmware is verified as correctly loaded See dagconfig page 30 for more information 2008 Endace Technology Ltd Confidential Version 3 November 2008 17 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Configuring the DAG card 18 Display Current Configuration Once you have loaded the FPGA image you should run the dagconfig tool without arguments to display the current card configuration and verify the firmware has been loaded correctly To display the default configuration for the first card use dagconfig d0 default where 0 is the device number of the DAG card you wish to capture data from A description of available tokens follows Note Not all tokens displayed in the following diagram SONET 0C3 12 STM 1 STM 4 Firmware dag54sga48pci ipf fbmr4tl stm4 v2 1 4vlx40ff1148 2008 11 11 10 48 2 user Copro dag54sga48cp ipf fbmr4tl stm4 v2 1 4vlx40ff1148 2008 9 25 17 45 56 user Serial 89564 Sets eq or unsets noeql equipment loopback Sets fcl or unsets nofcl facility loopback Note egl loops back to the PCJ bus ote fcl loops back to the line Port A nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate C phy ref select 0 master enablea config 0 nodiscard data laser nocrc sonet scramble pscramble ocl2
34. Returns an error if the actual number is different P Set legacy format to PoS default ATM P PARAMS DAG 3 5S capture parameters q Quiet This instructs dagbits to suppress summary information when terminating Error messages are not affected by this option r NUM Set legacy format record lengths to NUM R NUM When used in conjunction with the rlen test indicates the RLEN of ERF records to match against NUM S Check for strictly monotonic increasing timestamps rather than monotonic non decreasing Affects the behavior of the mono test With strict checking it is an error for consecutive timestamps to be equal they must always increase S NUM Terminate dagbits after NUM seconds of capture This option only makes sense when capturing packets from a DAG card i e when used in conjunction with the d flag t NUM Terminate dagbits if any ERF record type does not match NUM U NUM Process at most NUM records in one pass This option enables the user to reduce the performance of dagbits for various purposes See also D v Increase verbosity of dagbits This option increase the amount of data CMM displayed when printing an ERF record due to the print test or errors in other verbose tests v will print payload contents vv will print payload contents and an accompanying ASCII dump of the packet payload V version Display version information wW Instruct dagbits to treat all warn
35. Sequence Missed 1 Start Host DAG output Description This is the time statistics collection started See Dagclock Statistics reset page 51 Current Host computer time DAG The DAG card time at the last time pulse If the DAG card has never been synchronized the following displays No active input free running Example start Thu Apr 28 13 32 45 2007 host Thu Apr 28 14 35 35 2007 dag Thu Apr 28 14 35 35 2007 2008 Endace Technology Ltd Confidential Version 3 November 2008 53 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Card with Reference 54 Overview To obtain the best timestamp accuracy you should connect the DAG card to an external clock reference such as a GPS or CDMA time receiver To use an external clock reference source the host computer s clock must be accurate to UTC to within one second This is used to initialize the DUCK When the external time reference source is connected to the DAG card time synchronization connector the DAG card automatically synchronizes to a valid signal Pulse Signal from External Source The DAG time synchronization connector supports an RS 422 PPS signal from an external source This is derived directly from an external reference source or distributed through the Endace TDS 2 Time Distribution Server module which allows two DAG cards to use a single receiver It is also possible for more than two DAG cards to use a single receiver by dai
36. Set the Health threshold in nanoseconds default is 596ns v Increase output verbosity FN Display version information X Clear clock statistics clearstats Command Description default Set the dagclock input and output to RS422 in and none out none Clears the input and output settings rs422in Sets the dagclock input to RS422 hostin Sets the dagclock input to Host unused overin Sets the dagclock input to Internal input auxin Sets the dagclock input to Auxiliary input unused rs4220ut Sets the dagclock output to repeat the RS422 input signal loop Output the selected input hostout Sets the dagclock output to host unused overout Internal output master card set Sets the DAG card s clock to computer clock time and clears clock statistics The DAG card takes approximately 20 to 30 seconds to re synchronize reset Full clock reset Load time from computer set RS422in none out Clears clock statistics The DAG card takes approximately 20 to 30 seconds to re synchronize Note By default all DAG cards listen for synchronization signals on their RS 422 port and do not output any signal to that port 2008 Endace Technology Ltd Confidential Version 3 November 2008 Dagclock Statistics reset Statistics are reset to zero when the following occur Loading a DAG driver Loading firmware EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide dagclock witha x option dagcl
37. User_Guide ERF 2 TYPE_ETH Type Bit 7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 2 Short description TYPE_ETH Long description Type 2 Ethernet Record Use This record format is for Ethernet 802 3 data links The TYPE_ETH record is shown below ERF header Extension header s Protocol header Padding Payload The following is a description of the TYPE_ETH record format Red fi eseription 0 Offset Number of bytes not captured from start of frame Typically used to skip link layer 1 byte headers when not required in order to save bandwidth and space Note This field is currently not implemented contents should be disregarded Pad The Ethernet frame begins immediately after the pad byte so that the layer 3 IP header 1 byte is 32 bit aligned This is typically used to skip link layer headers when they are not required in order to save bandwidth and space Payload Payload rlen ERF header 16 bytes Extension headers optional Padding 2 bytes of record bytes 2008 Endace Technology Ltd Confidential Version 3 November 2008 63 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide ERF 10 TYPE COLOR HDLC POS Type Bit7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 10 Short description TYPE COLOR HDLC POS Long description Type 10 Colored PoS HDLC Record Use This record format is for data links inc
38. a time synchronization signal programmed 7 SFP ports Le Should always be On when the card is powered u Blue LED Green LED Green LED P synchronization signal Yellow LED Tuus T PPS Out The card is In addition there is a 4 pin socket located beside the port connectors which is available for connection to an external time synchronization source Caution Connect only a PPS input to the 4 pin time synchronization socket Connecting anything else to this socket may damage to the DAG card External power supply The DAG 5 4SGA 48 Card has an additional power supply socket This must be connected to a 12 V power supply using the supplied cable Caution Prolonged operation of the DAG 5 4SGA 48 card without this extra power supply connected could damage parts of the card The external power supply connector is a 2 5mm Pitch 4 way Flat 3 5 Floppy Drive Power Connector A 3 to 5 1 4 floppy disk drive cable is supplied with the card to connect to the server s power supply This must be connected to a 12 V power supply Some servers are equipped with floppy drive power supply cables that plug directly into this connector Otherwise use the Endace supplied floppy drive power supply cable to connect the DAG card to the power supply The current draw through this extra power connector is up to Caution Prolonged operation of the DAG 5 4SGA 48 card without this extra power supply connected could damage parts of th
39. acket that is larger than the s1en value will be truncated to that size Any packet that is smaller than the s1en value will be captured at its actual size therefore producing a shorter record which saves bandwidth and storage space Example The example below shows a configuration for variable length full packet capture dagconfig d0 varlen In fixed length novarzien mode the card will capture all packets at the same length Any packet that is longer than the s1en value will be truncated to that size in the same way as for varlen capture However any packet that is shorter than the s1en value will be captured at its full size and then padded out to the size of the s1en value This means that in novarien mode you should avoid large s1en values because short packets arriving will produce records with a large amount of padding which wastes bandwidth and storage space Note Using the novarlen option on DAG cards with an on board Co Processor accelerated cards is not recommended It may cause excessive loss of packets Example The example below shows a configuration for fixed length packet capture that will produce a 64 byte record dagconfig d0 align64 novarlen slen 40 nonvarlen is not recommended for use on this DAG card Version information Details the following information about the connected DAG card e Firmware image programmed in the FPGA e The DAG card serial number e The MAC address s of the DAG card s ports ethernet card
40. all packets are transmitted at the maximum rate permitted by the line interface using only minimum gaps between packets Example dagconfig nodelay tx crc Enables the crc fcs in the transmitted packets CRC FCS Ethernet only Example dagconfig tx crc dagconfig notx crc txonly Configures the memory hole to only transmit Note Only displayed if the DAG card supports transmit i e has a terf image Example dagconfig txonly Note This option is only applicable on firmware images containing TX txrxerror Indicates how to process packets that are transmitted with an RX error flag Packets are either discarded notxrxerror on or transmitted txrxerror Note Only displayed if the DAG card supports transmit i e has a terf image Example dagconfig txrxerror dagconfig notxrxerror 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide varlen novarlen The DAG 5 4SGA 48 card is able to capture packets in two ways They are e Variable length capture varien e Fixed length capture novarien not support on some firmware images In variable length varlen mode the DAG card will capture the whole packet providing its size is less than the s1en value Therefore to use this capture mode effectively you should set the s1en value to the largest number of bytes that a captured packet is likely to contain For more information on snaplength see slen page 27 Any p
41. are printout level 0 Note 1 indicates the condition is present on the link o indicates the condition is not present on the link SONET Port bl error count b2 error count b3 error count rei error count A 0 0 0 0 B 0 0 0 0 Condition Description bl error count B1 error count b2 error count B2 error count b3 error count B3 error count rei error count Remote Error Indicator The above extended statistics are also applicable to the DAG 5 4SGA 48 Ethernet image 32 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Universal counters The counters contain details of the number of frames and any errors The counters are latch and clear so values indicate the amount of data since the last time the counters were read dagconfig d0 For more information see dagconfig page 30 Example outputs are shown below Ethernet Number of counters in this block Number of counter blocks on the card Type of counters Number of blocks 1 URDU Total number of counters 3 4 Nb of counter s in Block ID Eth Frame Rx 3 RX Frame Port 0 value 64 RK Byte E Port 0 value 9412 RX err Port 0 value 0 TX_frames Port 0 value 63 TX_bytes Port 0 value 10321 TX Error Port 0 value 0 RX_Frame Port 1 value 134 RX_Byte Port 1 value 19768 RX_err Port 1 value 0 TX_frames Port 1
42. ation as possible enables Endace Customer Support to be more effective in their response to you The exact information available to you for troubleshooting and analysis may be limited by nature of the problem The following items may assist in a quick resolution DAG 5 4SGA 48 card s model and serial number Host computer type and configuration Host computer operating system version DAG software version package in use Any compiler errors or warnings when building DAG driver or tools For Linux and FreeBSD messages generated when DAG device driver is loaded These can be collected from command dmesg or from log file var log syslog Output of daginf Firmware versions from dagrom x Physical layer status reported by dagconfig Link statistics reported by dagconfig si Statistics either depending on the DAG card e Extended statistics reported by dagconfig ei e Universal statistics reported by dagconfig ui Network link configuration from the router where available Contents of any scripts in use Complete output of session where error occurred including any error messages from DAG tools The typescript Unix utility may be useful for recording this information A small section of captured packet trace illustrating the problem If you have just rebooted and the system can not see any DAG cards you need to load the DAG drivers Run dagload 2008 Endace Technology Ltd Confidential Version 3 November 2008 73
43. cket records are written to host computer memory during capture operations The diagram below shows the DAG 5 4SGA 48 card s major components and flow of data 18Mb QDR Co Pro FPGA PCIx Interface 4 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Line Types It is important that you understand the physical characteristics of the network to which you want to connect If your configuration settings do not match your network the DAG 5 4SGA 48 card will not function as expected Note If you are unsure about which of the options listed below to apply to your network please contact your Network Administrator for further information Supported Types The line types supported by the DAG 5 4SGA 48 card are described below Line Types Description OC 3c STM 1c Optical Carrier Specification with line speed of up to 155 52Mbps concatenated OC 12c STM 4c Optical Carrier Specification with line speed of up to 622 08Mbps concatenated OC 48c STM 16c Optical Carrier Specification with line speed of up to 2488 32Mbps concatenated Ethernet 10Base T 10Mbps over two pairs of twisted telephone cable Ethernet 100Base TX 100Mbps over two pairs of shielded or unshielded twisted Cat5 copper cable Ethernet 1000Base T 1000Mbps over four pairs of balanced Cat5 or Cat6 copper cable Ethernet 1000Base LX 1000Mbps over single mode or multimo
44. cord such as dagsnap must be able to read records out of the buffer as fast or faster than they arrive If not the buffer will eventually fill and packet records will be lost If the user process is writing records to hard disk it may be necessary to use a faster disk or disk array If records are being processed in real time a faster host CPU may be required In Linux and Free BSD when the computer buffer fills the following message displays on the computer screen kernel dagN pbm safety net reached OxNNNNNNNN The same message is also printed to log var 109 messages file In addition when the computer buffer fills the Data Capture LED on the card will flash or flicker or may go OFF completely In Windows no screen message displays to indicate when the buffer is full Please contact Endace Customer Support at support endace com for further information on detecting buffer overflow and packet loss in Windows Detecting Packet Losses Once the buffer fills any new packets arriving will be discarded by the DAG 5 4SGA 48 card until some data is read out of the buffer to create free space You can detect any such losses by observing the Loss Counter 1ctr field of the Extensible Record Format ERF See Data Formats page 59 later in this User Guide for more information on the Endace ERF record format Increasing Buffer Size You can increase the size of the host computer buffer to enable it to cope with bursts of high t
45. de a version number and creation date 14 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Setting up the FPGA All DAG cards have at least one Field Programmable Gate Array FPGA The FPGA contains the firmware for the DAG card The firmware defines how the DAG card operates when capturing data and contains the specific configuration Note Some DAG cards have multiple FPGA s For each FPGA there are two firmware images e a factory image contains fixed basic functionality for operating the DAG card e auser image contains an upgradable version of the DAG card firmware Additional functionality for the DAG card is available via the user image Different user images may be available with different functionality i e TERF DSM etc Firmware images are loaded into DAG card flash ROM in the factory The image is programmed into the FPGA each time the DAG card is powered up The user image can then be programmed into the FPGA either manually or via a script Programming the FPGA Before configuring the DAG card for capture you must load and program the DAG card with the appropriate FPGA image Note For information about the dagrom options see dagrom page 16 Before you can configure the DAG 5 4SGA 48 card for capture you must load the appropriate FPGA image using dagrom as follows SONET 0C3 0C12 STM 1 STM 4 To program the FPGA type the following dagrom
46. de fiber optic cable with long wavelength laser driver 1310nm Ethernet 1000Base SX 1000Mbps over single mode or multimode fiber optic cable with short wavelength laser driver 850nm Note For detailed information regarding Ethernet line types and speeds please refer to IEEE Standard 802 3 available from the IEEE website at www ieee org 2008 Endace Technology Ltd Confidential Version 3 November 2008 5 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Extended Functions Enhanced packet processing With the enhanced packet processing capability the following types of enhanced capabilities can be achieved Filtering The goal is to save bus bandwidth and reduce the amount of traffic the application must process by filtering out unneeded traffic before it is written into the application s stream buffer Filtering requires identifying the desired fields in the packet s protocol headers performing a comparison function against these fields to identify i e classify traffic streams and dropping the unneeded traffic while letting the desired streams pass through to the stream buffer Hash Load Balancing HLB The goal of Hash Load Balancing is to receive a mixed stream of packets and spread the traffic evenly across several stream buffers This is typically used to optimize the parallel processing done by multi core servers since each core will only see a subset of the full traffic load You normally want to distributing traffic equal
47. der is allocated to the rx stream For information on setting the Memory allocation see mem page 24 Note You can not change the stream memory allocations while packet capture or transmission is in progress Explicit Packet Transmission The operating system does not recognize the DAG 5 4SGA 48 card as a network interface and will not respond to ARP ping or router discovery protocols The DAG 5 4SGA 48 card will only transmit packets that are explicitly provided by the user This allows you to use the DAG 5 4SGA 48 card as a simple traffic load generator You can also use it to retransmit previously recorded packet traces The packet trace is transmitted as fast as possible The packet timing of the original trace file is not reproduced 2008 Endace Technology Ltd Confidential Version 3 November 2008 43 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Trace Files You can use dagflood to transmit ERF format trace files providing the files contain only ERF records of the type matching the current link configuration When you use DAG cards with multiple ports ensure all ports referred to by the Trace file are active This ensures the dagflood traffic is not blocked when trying to delivering data to an inactive port Check the interface status output for the DAG card and ensure the link status for all required destination ports is active See Viewing the DAG card statistics page 31 For further information on using dagflood pl
48. e beet External power supply ettet etn tenete teet a Pluggable Optical Transceiver sss uas Optical modules ee p a tere gere tus ie acp Power Mp t m M Splitter Losses i e ese ine Pp c rtc a ape ee ee iai uci Configuring the DAG card Introd ctlon i EIER ES Extended statistics ccccccccccscesssccssscesssecssccesseecssecesseeessecesseeesseeeees Universal counters nennen ener enne Using your DAG card to capture data lieisviolstoistor t Basic data capture cider reete ree edet Starting a capture session sssssssseeeee eee GEPcuCIUM Capturing data at high speed sssssssssssssssss Capturing Raw SONET frames sss eee 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG 5 495GA 48 Card User Guide ts 4 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Viewing Captured data csiccissercccdncrsceencesscesscuntensseesseesheosevssasevsvesnsnstonsbevadeashevtuenevtubsvesesodivesdvebeseohsuptonenevdceadventstned 39 ASD ItS T IN 39 Converting captured data eei tete ttt bote tite e ie Eo Ene Fea Eo EIE uode cbe ope seine tuae ota 41 Dagen vert E 42 Using third party applications terne repentino
49. e card External power supply connector 10 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide Pluggable Optical Transceivers Overview The DAG 5 4SGA 48 card SFP optical transceivers consist of two parts as follows e Mechanical chassis attached to the circuit board e Transceiver unit which may be inserted into the chassis You can connect the transceiver to the network via LC style optical connectors For further information on pluggable optical transceivers please refer to the Endace website at http www endace com Caution To prevent damage to the DAG card ensure you select the correct transceiver type for the optical parameters of the network to which you want to connect Optical modules The optical power range depends on the particular SFP module that is fitted to the DAG card However Endace recommends you use the 1310nm short range modules which are shipped with the card Optical power is measured in dBm This is decibels relative to 1 mW where 10 dB is equivalent to a factor of 10 in power A negative optical power value indicates power that is less than 1 mW The most sensitive devices can work at power levels down as low as 30dBm or 1uW If there s doubt check the card receiver ports light levels are correct using an optical power meter Cover card transmit ports with LC style plugs to prevent dust and mechanical hazards damaging optics if no
50. e contact Endace Customer Support at support endace com 2 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Card Description The DAG 5 4SGA 48 has two optical Small Form factor SFP pluggable interfaces The two transceivers can be operated simultaneously allowing a single card to monitor one or both directions of a full duplex link It is capable of full rate capture or transmit However when maximally configured with two interfaces the PCIx bus limits the combined receive and transmit data bandwidth to 7 6 Gigabits per second The DAG 5 4SGA 48 Card has an additional power supply socket This must be connected to a 12 V power supply using the supplied cable The DAG 5 4SGA 48 card is shown below with pluggable transceivers fitted Co Processor FPGA TCAM Heat sink a Si SFP interface 3 Power supply cable 4 pin connector for supplied time synchronization The key features of the cards are User configurable dual port network monitoring interface card for OC3 12 48 and 1 Gigabit Ethernets networks Two Small Form Factor SFP pluggable optical transceivers Transmit and receive capable Header only or variable length capture Integrated Co Processor with Enhanced packet processing Capture of Raw SONET OC3c OC12c OC48c STM 1c STM 4c STM 16c frames Conditioned clock for PPS input or local synchronization The DAG 5 4SGA
51. ease refer to the EDM04 03 dagflood User Manual available from Endace Customer Support at https www endace com support In addition the length of the ERF records to be transmitted must be a multiple of 64 bits You can configure this when capturing packets for later transmission by setting 64 bit alignment using the dagconfig align64 command If packets have been captured without using the align 4 option you can convert the trace files so that they can be transmitted by using dagconvert page 42 as shown below dagconvert v i tracefile erf o tracefile erf A8 Alternatively if you are unsure if a trace file is 64 bit aligned you can test the file using dagbits page 39 as shown below dagbits v align64 f tracefile erf If you do not have any ERF trace files available you can use daggen to generate trace files containing simple traffic patterns This allows the DAG 5 4SGA 48 card to be used as a test traffic generator For further information on using daggen please refer to the EDM04 06 Daggen User Guide available from Endace Customer Support at https www endace com support TR TERF Selectable CRC Length TR TERF can be optionally configured to strip the CRC of an incoming ERF stream to allow the retransmitted stream to calculate and add a CRC There are three basic situations where this option may be useful Situation 1 The ERF stream was generated without a CRC In this case you should configure TR TERF
52. ed during current capture session on the individual stream Resets to 0 when a capture is started enable disable Sets whether this DAG card captures data on the defined port a or b Note DAG ports are enabled by default You do not need to use dagconfig to enable the card in order to begin capture unless you have previously disabled it Example dagconfig d0 enablea dagconfig d0 disablea dagconfig d0 enableb dagconfig d0 disableb where 0 is the device number of the DAG card you wish to capture data from Note On some firmware images changes to this option may not take effect egl noeql Sets or unsets equipment loopback For testing set to eq1 mode and normal operation set to noeqi mode Note eq1 mode loops transmit data from the host back to the PCIx bus Example dagconfig eql dagconfig noeql 2008 Endace Technology Ltd Confidential Version 3 November 2008 23 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide 24 fcl nofcl Note Sets or unsets Facility loop back For testing set to tci mode and normal operation set to nofcl mode FCL retransmits the data received and also send it to the host Example dagconfig fcl dagconfig nofcl laser nolaser Enables disables the transmit laser on for the optical transceivers Example dagconfig laser dagconfig nolaser master slave Defines whether the data transmitted is clocked with the recovered clock from the input slave or if it uses the synthesized reference clock o
53. en ERF header Extension headers optional Protocol header Padding Extension header types Number Type Description 0 TYPE LEGACY Old style record 1 TYPE HDLC POS Packet over SONET SDH frames using either PPP or CISCO HDLC framing 2 TYPE ETH Ethernet 3 TYPE ATM ATM cell 4 TYPE AAL5 reassembled AAL5 frame 5 TYPE MC HDLC Multi channel HDLC frame 6 TYPE MC RAW Multi channel Raw time slot link data 7 TYPE MC ATM Multi channel ATM Cell 8 TYPE MC RAW Multi channel Raw link data CHANNEL 9 TYPE MC AAL5 Multi channel AAL5 frame 10 TYPE COLOR HDLC HDLC format like TYPE HDLC POS but with the LCNTR field POS reassigned as COLOR 11 TYPE COLOR ETH Ethernet format like TYPE ETH but with the LCNTR field reassigned as COLOR 12 TYPE MC AAI2 Multi channel AAL2 frame 13 TYPE IP COUNTER IP Counter ERF Record 14 TYPE TCP FLOW TCP Flow Counter ERF Record COUNTER 15 TYPE DSM COLOR HDLC format like TYPE HDLC POS but with the LCNTR field HDLC POS reassigned as DSM COLOR 16 TYPE_DSM_COLOR_ Ethernet format like TYPE_ETH but with the LCNTR field ETH reassigned as DSM COLOR 17 TYPE_COLOR_MC_ Multi channel HDLC like TYPE MC HDLC but with the LCNTR HDLC POS field reassigned as COLOUR 18 TYPE AAL2 Reassembled AAL2 Frame Record 19 TYPE COLOR HASH Colored PoS HDLC record with Hash load balancing POS 20 TYPE COLOR HASH
54. ent of the ERF to NUM bytes ERF only b EXPRESSION Specify a tcpdump 1 style BPF expression to be applied to the packets c 0 16 32 Specify the size in bits of the frame checksum FCS pcap 3 only d DEVICE Use the DAG device referred to by DEVICE Supported syntax includes 0 device DEVICE dagl and dev dag3 to refer to DAG cards and 0 2 dag1 1 and dev dag2 0 to refer to specific streams on cards f FILTERS A comma delimited list of filters to be applied to the data Supported filters are e rx Filter out rx errors link layer e ds Filter out ds errors framing e trunc Filter out truncated packets e a b c d Filter on indicated port interface s Select fixed length output ERF only G NUM Set the GMT offset to NUM seconds pcap 3 only h help usage Display usage help information i FILES Name s of the input file s If more than one filename is given the ERF records from the files will be merged in timestamp order to the output o FILE Name of the output file r NUM Rotate the output file after NUM bytes Add k kilobytes m megabytes g gigabytes and t terabytes suffixes s NUM Set the snap length to NUM bytes t NUM Capture from the DAG card for NUM seconds T atm dagl erf eth pcap pos erf pcaplprt Input and output types See the DESCRIPTION section above for more information about
55. external GPS reference clock and the computer clock is synchronized to a local NTP server 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide Timestamps ERF files contain a hardware generated timestamp of each packet s arrival The format of this timestamp is a single little endian 64 bit fixed point number representing the number of seconds since midnight on the 1 January 1970 The high 32 bits contain the integer number of seconds while the lower 32 bits contain the binary fraction of the second This allows an ultimate resolution of 2 seconds or approximately 233 picoseconds Different DAG cards have different actual resolutions This is accommodated by the lower most bits that are not active being set to zero In this way the interpretation of the timestamp does not need to change when higher resolution clock hardware is available The DAG 5 4SGA 48 implements the 27 most significant bits which provides a time resolution of 7 5 nanoseconds The ERF timestamp allows you to find the difference between two timestamps using a single 64 bit subtraction You do not need to check for overflows between the two halves of the structure as you would need to do when comparing Unix time structures Example Below is example code showing how a 64 bit ERF timestamp erfts can be converted into a struct timeval representation tv unsigned long long lts struct timeval tv l
56. ght to operate this equipment in a Class A industrial environment Disclaimer Whilst every effort has been made to ensure accuracy neither Endace Technology Limited nor any employee of the company shall be liable on any ground whatsoever to any party in respect of decisions or actions they may make as a result of using this information Endace Technology Limited has taken great effort to verify the accuracy of this manual but nothing herein should be construed as a warranty and Endace shall not be liable for technical or editorial errors or omissions contained herein In accordance with the Endace Technology Limited policy of continuing development the information contained herein is subject to change without notice Website http www endace com Copyright 2008 Endace Technology Ltd All rights reserved No part of this publication may be reproduced stored in a retrieval system or transmitted in any form or by any means electronic mechanical photocopying recording or otherwise without the prior written permission of the Endace Technology Limited Endace the Endace logo Endace Accelerated DAG NinjaBox and NinjaProbe are trademarks or registered trademarks in New Zealand or other countries of Endace Technology Limited Applied Watch and the Applied Watch logo are registered trademarks of Applied Watch Technologies LLC in the USA All other product or service names are the property of their respective owners Product and co
57. gy Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide Viewing the DAG card status Interface Status When you have configured the card according to your specific requirements you can view the interface statistics to check the status of each of the links using dagconfig d0 s For more information see dagconfig page 30 There are multiple printout levels The statistics displayed below are printout level 0 Example outputs are shown below Note 1 indicates the condition is present on the link o indicates the condition is not present on the link SONET Port lock lof lop los A 1 0 0 0 B T 0 0 0 A definition of each of the SONET status conditions up to printout level 2 is described below Condition Description Ais Alarm indication signal indicates a sonet sdh remote aps error Bl A b1 error has occurred B2 A b2 error has occurred B3 A b3 error has occurred Crc error Indicates a crc error has occurred Fifo empty Indicates a fifo empty error has occurred Fifo full Indicates a fifo full error has occurred Fifo overflow Indicates a fifo overflow error has occurred Lock Card is locked to the incoming signal and can capture data Lof Loss of frame Indicates oof has been asserted for more than 3 ms Lop Loss of pointer Los Loss of signal Indicates there is either no signal at the receiver or the optical
58. ics reset page 51 Example error Freq 30ppb Phase 60ns Worst Freq 75ppb Worst Phase 104ns 52 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide Crystal This line reports on the DAG card crystal oscillator ww um The DAG card s crystal frequency calculated based on the reference clock Synthesized The target time stamping frequency Different for each DAG card type Example crystal Actual 100000028Hz Synthesized 67108864Hz Input This line reports on the time pulses received by the DAG card Description The total number of time pulses received Reset to zero when statistics are reset see Dagclock Statistics reset page 51 The number of time pulses that were rejected considered Bad by the DAG card Reset to zero when statistics are reset see Dagclock Statistics reset page 51 Time pulses are considered Bad if they were not received 1 second approximately after the last time pulse These may be caused by noise Singles missed The number of times a single time pulse failed to be received by the DAG card i e a two second gap Reset to zero when statistics are reset see Dagclock Statistics reset page 51 This displays the longest time gap in seconds between a pair of time pulses Reset to Missed zero when statistics are reset see Dagclock Statistics reset page 51 Example input Total 3765 Bad 0 Singles Missed 5 Longest
59. ings as errors W NUM When used in conjunction with the wlen test the wire length of ERF records must be exactly NUM bytes Z Stop when no traffic is received for one second dagbits takes several options that serve as parameters to particular tests Available tests include monotonic time stamp increment and frame checksum FCS aka CRC validation See the dagbits help for further details 40 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Converting captured data dagconvert is the software utility that converts captured data from ERF format to Pcap and other formats Once in non ERF format the data can be read using third party applications page 43 dagconvert can also be used to capture data directly into pcap format Examples To read from DAG card 0 and save to a file in ERF format dagconvert d0 o outfile erf To read from DAG card 0 and save to a file in pcap format dagconvert d0 T dag pcap o outfile pcap To convert a file from ERF format to pcap format dagconvert T erf pcap i infile erf o outfile pcap To convert a file from pcap format to ERF format ensuring the ERF records are 64 bit aligned and therefore suitable for transmission using dagflood dagconvert T pcap erf A 8 i infile pcap o outfile64 erf To capture from DAG Card 0 using a BPF filter dagconvert d0 o outfile erf b host 192 168 0 1 and tcp port 80 To capture fr
60. ity and length of input pulse 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Single Card No Reference When a single DAG card is used with no external reference the DAG card can be synchronized to the host computer clock Most computer clocks are not very accurate by themselves but the DUCK drifts smoothly at the same rate as the computer clock The synchronization achieved with this method is not as accurate as using an external reference source such as GPS The DUCK clock is synchronized to a computer clock by setting input synchronization selector to overflow as follows dagclock d0 none overin Output muxin overin muxout none status Synchronised Threshold 11921ns Failures 0 Resyncs 0 error Freq 1836ppb Phase 605ns Worst Freq 147ppb Worst Phase 324ns crystal Actual 49999347Hz Synthesized 16777216Hz input Total 87039 Bad 0 Singles Missed 0 Longest Sequence Missed 0 start Wed Apr 27 14 27 41 2007 host Thu Apr 28 14 38 20 2007 dag Thu Apr 28 14 38 20 2007 2008 Endace Technology Ltd Confidential Version 3 November 2008 55 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Two Cards No Reference 56 Overview If you are using two DAG cards in a single host computer with no reference clock you must synchronize the DAG cards using the same method if you wish to compare the timestamps between the two DAG cards You may wish to do this for example if
61. kill rxclock reset phy rate 0 phy ref select 0 slave enablea config 0 nodiscard data nocrc sonet scramble pscramble oc3 raw Port B laser nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate 0 phy ref select 0 slave enableb config 0 nodiscard data nocrc sonet scramble pscramble oc3 raw Generate SONET tx clock internally Enables enable lt port gt or disables Sets scramble or unsets master or from rx clock slave disable lt port gt port for capture noscramble SONET scrambling GPPO varlen slen 10240 align64 lt Records will be generated with 64 bit Port A drop count 0 alignment This is not a configurable option Port B drop count 0 lt wNumber of packets dropped during the current capture sesion on this port Stream 0 stream drop count 0 Allows each memory hole PCI Burst Manager to operate independently 133MHz buffer size 512 rx streams 1 tx streams 0 drop nooverlap Memory Streamsa wor currently allocated in MB mem 512 0 lt to the rx and tx stream s 20 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Raw SONET 0C48 Firmware dag54sga48pci raw sdh stml6 pci v2 4 4vlx40ff1148 2008 11 10 14 33 14 user Copro dag54sga48cp ipf fbmr4tl stm4 cp v2 1 4vlx40ff1148 2008 9 25 17 45 56 user Serial 0 Sets eql or unsets noeql equipment loopback Sets fcl or unsets nofcl facility loopback Note eql lo
62. ly or roughly equally across the stream buffers while preserving flow coherence meaning all packets associated with a single TCP IP session must always go to the same stream buffer Hashing is the tool used to classify the flow coherent streams and mathematically assign a bin number to each packet Bin numbers are then associated with specific stream buffers to achieve the desired load balancing A full discussion of how hash functions work is beyond the scope of this document but in essence the Hash Load Balancing performs packet classification and steering similar to filtering and steering Classify and Colorize The goal is to have the hardware perform the first level of traffic classification and then write the classification result commonly called the packet s colour into a field in the extended ERF record Later when processing the packet the application can accelerate its processing by doing a simple lookup on the packet s colour and jump directly to the appropriate processing function Classify and Steer Similar to filtering but the goal is not to eliminate i e drop traffic but rather spread traffic across multiple stream buffers in order to take better advantage of multi core processors For example all traffic to and from a specific IP address could be sent to one stream buffer while all other traffic sent to another Duplication For this document packet duplication is defined as writing a single packet into more than one st
63. mpany names used are for identification purposes only and such use does not imply any agreement between Endace and any named company or any sponsorship or endorsement by any named company Use of the Endace products described in this document is subject to the Endace Terms of Trade and the Endace End User License Agreement EULA 2008 Endace Technology Ltd Confidential Version 3 November 2008 Contents Introduction QV OL VIEW Card Features ce eet tede tenet tine arid Purpose of this User Guide serene System Requirements netter reiten etri tete eerta Geena m Operating System citi anperiiae aeta reperi teens Other Syste S jueces tert tett ts iren HUE RON Te NA ERS Card Descriptio a u ee etit teret me EEE tar esa Battery removal don t do it sse Card Architecture e sciet sotto te perc tete tont eret een Line VY Supported Typessoscnsi nitent epe net genet dta Extended Pnetlons e erret peer eee tpe et e Enhanced packet processing sse Timed Release TERF TR TERE sese Raw SONET capture nn iti ttp trt tre tig le trees Installation Introduction e eerte erp tp QU epe ecl EM be OE DAG Software package ttt etnies to tiene ni Inserting the DAG Card eie rrt tere tere nt regit Port Connectots iiiie er steep e titre F
64. n ERF header 16 bytes Extension headers optional Protocol header 4 bytes Payload bytes of record 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide ERF 16 TYPE_DSM_COLOR_ETH Type Bit 7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 16 Short description TYPE_DSM_COLOR_ETH Long description Type 16 DSM Color Ethernet Record Use This record format is for Ethernet 802 3 data links incorporating filter results The record format is the same type as the Type 2 TYPE_ETH page 63 record with the exception that the Ictr field reassigned as DSM type color The TYPE_DSM_COLOR_ETH record is shown below _byte3 byte2 byter byteo type 16 color E Extension header s Protocol header BN Padding Payload The following is a description of the TYPE_DSM_COLOR_ETH record format Hed scription 00000 Offset 1 byte Pad 1 byte Payload bytes of record The color field is a hardware generated tag indicating the result of a filtering or classification operation This field is divided into the following Bit Description 0 5 Receive stream number 0 63 6 13 Filter match bits bit6 filter0 bit7 filter1 and so on 14 hlIb0 CRC calculation output bit 15 hlb1 parity calculation output bit
65. n internal error generated inside the card annotator Not present on the wire Reserved Reserved Record length in bytes Total length of the record transferred over the PCIx bus to storage The timestamp of the next ERF record starts exactly rlen bytes after the start of the timestamp of the current ERF record Depending upon the ERF type this 16 bit field is either a loss counter of color field The loss counter records the number of packets lost between the DAG card and the stream buffer due to overloading on the PCIx bus The loss is recorded between the current record and the previous record captured on the same stream interface The color field is explained under the appropriate type details 60 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide Wire length Packet length on the wire including some protocol overhead The exact interpretation of this quantity depends on physical medium This may contain padding extension Extension headers in an ERF record allow extra data relating to each packet to be transported headers to the host Extension header s are present if bit 7 of the type field is 1 If bit7 is 0 no extension headers are present ensures backwards compatibility Note There can be more than one Extension header attached to a ERF record Payload Payload is the actual data in the record It can be calculated by either e Payload rl
66. n the card Master Setting required for SONET images only mem You can split the DAG card s allocated memory between the receive and transmit stream buffers to suit your own requirements The split is displayed as a ratio as shown below mem X Y where x is the memory allocated in MB to the rx stream y is the memory allocated in MB to the tx stream If there are multiple rx or tx streams memory can be allocated to each stream mem X Y X Y X Y Buffer size page 22 and rx and tx Streams page 26 are related to mem Example You can split 128MB of memory evenly between the tx and rx streams using dagconfig d0 mem 64 64 Note You can not change the stream memory allocations while packet capture or transmission is in progress nic nonic Note From DAG software 3 1 0 onwards nic nonic is replaced by auto neg noauto neg Both options are valid and still can be used auto neg noauto neg is not supported by some older cards See auto neg noauto neg page 22 oc3 Set framer to OC3 receive mode Example dagconfig oc3 0c12 Set framer to OC12 receive mode Example dagconfig ocl2 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide 0c48 Set framer to OC48 receive mode Example dagconfig oc48 overlap nooverlap Configures the rx and tx memory hole to be overlapped This enables in line forwarding without copying the data across the memory holes
67. nt the NTP can adjust the computer clock in a single large step Over time NTP adjusts the rate of computer clock to minimize the offset from its reference It can take several days for NTP to fully synchronize the computer clock The DAG card clock is initialized from the computer s clock rather than from the NTP Using NTP to synchronize the computer s clock ensures the DAG card clock remains accurate DAG cards can also be synchronized to external references such as GPS or to the computer clock directly In both cases the computer clock time is loaded onto the DAG clock when the DAG card is started dagload dagreset dagrom p When clock synchronization is enabled the DAG card time is compared to the computer time once per second regardless of the synchronization source If the times differ by more than 1 second the DAG card clock is reloaded from the computer clock and synchronization is restarted For this reason the computer clock should be maintained with better than 1 second accuracy If the DAG card clock is synchronized to the computer clock then small step adjustments of the computer clock by the NTP daemon can cause the DAG driver to emit warning messages to the console and system log files if the adjustment exceeds the warning threshold These messages are intended to allow the user to monitor the quality of the clock synchronization over time The best synchronization is achieved when the DAG card is synchronized to an
68. oad rlen ERF header 16 bytes Extension headers optional Padding 2 bytes 2008 Endace Technology Ltd Confidential Version 3 November 2008 65 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide 66 ERF 15 TYPE_DSM_COLOR_HDLC_POS Type Bit 7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 15 Short description TYPE_DSM_COLOR_HDLC_POS Long description Type 15 DSM Color HDLC PoS Record Use This record format is for HDLC data links incorporating filter results The record format is the same type as the Type 10 TYPE COLOR HDLC POS page 64 record with the exception that the Ictr field is reassigned as DSM type color The TYPE DSM COLOR HDLC POS variable length record is shown below Wes ERF header Extension header s Protocol header Padding Payload The following is a description of the TYPE DSM COLOR HDLC POS record format Hed fi escription 0 The color field is a hardware generated tag indicating the result of a filtering or classification operation This field is divided into the following Bits Description 0 5 Receive stream number 0 63 6 13 Filter match bits bit6 filter0 bit7 filter1 and so on 14 hlIb0 CRC calculation output bit hlb1 parity calculation output bit HDLC header Protocol header Length may vary depending on protocol 4 bytes Payload rle
69. ock with a set or reset command Example To view the default dagclock configuration dagclock d0 The following is the output from DAG card that has its clock reference connected The clock statistics have been reset since the card was last synchronized Note Values will differ for each DAG card type muxin rs422 muxout none status error crystal input Total 3765 start Thu Apr 28 host Thu Apr 28 dag Thu Apr 28 Note Bad 0 13 32 1272353 14 35 Synchronised Threshold 596ns Failures 0 Resyncs 0 Freq 30ppb Phase 60ns Worst Freq 75ppb Worst Phase 104ns Actual 100000028Hz Synthesized 67108864Hz Singles Missed 5 Longest Sequence Missed 1 45 2007 35 2007 35 2007 For a description of the dagclock output see Dagclock output explained page 52 2008 Endace Technology Ltd Confidential Version 3 November 2008 51 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Dagclock output explained Muxin Lists the dagclock time input source for this DAG card The options are Rs422in Hostin Overin or Auxin Example muxin rs422 Muxout Lists the dagclock time output source for this DAG card The options are Rs422out Hostout Overout or Loop Example muxout none Status This line reports on the status of the DAG card output i eseription Synchronised This indicates whether this DAG card is synchronized to the time source listed Muxin Not synchronised The DAG card becomes Not Synchronized
70. of memory to each transmit stream and divides the remaining memory between the receive streams Example dagconfig rxtx Note This option is only applicable on firmware images containing TX scramble noscramble Enables Disables the scrambling mode on the SONET Framer Deframer Example dagconfig scramble dagconfig noscramble Scramble mode for Raw SONET capture When using Raw SONET capture images the DAG card firmware supports both scramble and noscramble modes Scramble mode configures the hardware to descramble the SONET frame before writing it to application memory Scramble mode is recommended due to the high processor load that software based descrambling requires If operation in noscramble mode is required we recommend you monitor the packet frame loss counter If the application is unable to keep up the memory buffer will eventually overflow and the DAG card will begin discarding frames sdh Set framer into SDH mode Example dagconfig sdh 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide slen Before you begin to capture data you can set the size that you want the captured packets to be You can do this using the dagconfig tool to define the packet snaplength sien Note The snaplength value must be a multiple of 8 and in the range 48 to 9600 per card inclusive By default s1en which is the portion of the packet that you want to capture is se
71. ollowing features are available on this DAG card Note Different firmware images may be required Not all features are available on each firmware image For further information on which feature is available in what firmware image see Firmware images page 14 e OC3c OC 12c OC48c POS e STM 1c STM 4c STM 16c SDH POS e 10 100 1000 Ethernet e Co Processor with Enhanced packet processing with four stream buffers e Raw SONET capture at OC3c OC12c OC48c STM 1c STM 4c STM 16c e Timed Release TERF TR TERF 2008 Endace Technology Ltd Confidential Version 3 November 2008 1 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Purpose of this User Guide The purpose of this User Guide is to provide you with an understanding of the DAG 5 4SGA 48 card architecture functionality and to guide you through the following e Installing the card and associated software and firmware e Configuring the card for your specific network requirements e Running a data capture session e Synchronizing clock time e Data formats You can also find additional information relating to functions and features of the DAG 5 4SGA 48 card in the following documents which are available from the Support section of the Endace website at http www endace com e EDMO04 01 DAG Software Installation Guide e EDM04 03 dagflood User Manual e EDMO04 06 Daggen User Guide e EDMO04 19 DAG Programming Guide e EDMO04 26 Enhanced Packet Processing e EDMO5 01 Time Dist
72. ology Ltd Confidential Version 3 November 2008 Gain direct access to the overhead bytes of each frame Examine and debug SONET SDH systems at the section line and path layers Monitor payload pointer adjustments Implement software demapping of the payload SPE Synchronized Payload envelope Implement software demultiplexing and demapping of lower order VTs Implement software demapping of PDH Implement software ATM or HDLC demapping from concatenated interfaces EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Installation Introduction The DAG 5 4SGA 48 card can be installed in any free 3 3V PCIx slot Better performance is obtained if installed on a 133 Mhz slot Although the DAG driver supports up to sixteen DAG cards in one system limitations on the number of available slots in most computers means that it is not usually possible to support more than 3 to 4 DAG cards in the same computer DAG Software package The latest DAG Software package must be installed before you install the DAG 5 4SGA 48 card itself See EDM04 01 DAG Software Installation Guide which is included on the CD shipped with the DAG 5 4SGA 48 card Inserting the DAG Card Caution 8 9 It is very important to protect both the computer and the DAG 5 45GA 48 card from damage by electro static discharge ESD Failure to do so could cause damage to components and subsequently cause the card to partially or completely fail Turn power to the compu
73. om DAG card 0 using ERF filtering dagconvert d0 o outfile erf f rx a To capture from DAG card 0 to a series of files of size 128 MB dagconvert d0 o outfile erf r 128m The first file created is labeled out 11e0000 ert once the file size reaches 128MB a second file is created The second is labeled ouc i1e0001 ert etc 2008 Endace Technology Ltd Confidential Version 3 November 2008 41 EDMO01 31v3 DAG_5 4SGA 48 Card User Guide 42 Dagconvert dagconvert is a software utility for converting data to various file formats Supported formats are File format Description dag Read ERF records directly from DAG card input only erf ERF Extensible Record Format file input and output atm Legacy ATM files input only eth Legacy Ethernet files input only pos Legacy PoS files input only null Produces no input or output pcap pcap 3 format file input or output prt ASCII text packet dump output only Data can be input from a file or captured from a DAG card dagconvert can be used for converting data captured from a DAG card to pcap format This allows the trace file to be used with tools that support the pcap file format Also the reverse is possible where data can be converted to ERF format for use in other dag utilities The following is a list of options available in dagconvert Options Description A NUM Set the record alignm
74. ops back to the PCI bus ote fcl loops back to the line Port A laser nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate 0 phy ref select 0 slave enablea config 0 nodiscard data nocrc sonet noscramble nopscramble oc48 raw Port B laser nophy bist enable noegl nofcl phy tx clock off phy kill rxclock reset phy rate 0 phy ref select 0 slave enableb config 0 nodiscard data nocrc sonet noscramble nopscramble oc48 raw Generate SONET tx clock internally Enables enable lt port gt or disables Sets scramble or unsets master or from rx clock slave disable lt port gt port for capture noscramble SONET scrambling GPPO varlen slen 10240 align64 lt Records will be generated with 64 bit Port A drop count 0 alignment This is not a configurable option Port B drop count 0 lt Number of packets dropped during the current capture sesion on this port Stream 0 stream drop count 0 Allows each memory hole PCI Burst Manager to operate independently 133MHz buffer size 512 rx streams 1 tx streams 0 drop nooverlap Memory Streamsa non currently allocated in MB mem 512 0 lt to the rx and tx stream s 2008 Endace Technology Ltd Confidential Version 3 November 2008 21 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide dagconfig tokens explained 10 100 1000 Set one of the following modes of operation e Set 10BaseT mode 10Mbps e Set 100BaseTX mode 100Mbps e Set 1000BaseTX mode 1000Mbp
75. or e Change the splitter ratio if power is too high or too low Splitter Losses Splitters have the insertion losses either marked on their packaging or described in their accompanying documentation General guidelines are e A50 50 splitter will have an insertion loss of between 3 dB and 4 dB on each output e 90 10 splitter will have losses of about 10 dB in the high loss output and lt 2 dB in the low loss output Note Endace recommends that you do not use a combination of single mode and multi mode fibers and optics modules on the same link as the quality of the received signal cannot be guaranteed If you have no choice but to mix single mode and multi mode you should be aware that a single mode input connected to a multi mode fiber will have some attenuation but may still be acceptable However a multi mode input connected to a single mode fiber will likely have large and unpredictable losses 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Configuring the DAG card Introduction Configuring the DAG 5 45GA 48 card ready for capturing data requires the following steps e Setting up the FPGA page 15 e Preparing the DAG card for use page 17 e Configuring the DAG Card page 18 e Viewing the DAG card statistics page 31 Once the DAG 5 4SGA 48 is configured you can start capturing data see Using your DAG card to capture data page 35 for details on capturing data
76. orporating filter results The record format is the same type as the Type 1 POS HDLC page 62 record with the exception that the Ictr field is reassigned as color Requires Endace Coprocessor and appropriate firmware The TYPE COLOR HDLC POS record is shown below bytes byte2 byte1 byte d ype 10 ERF header Extension header s Protocol header Padding Payload The following is a description of the TYPE_COLOR_HDLC_POS record format Hed f scription The color field is a hardware generated tag indicating the result of a filtering or classification operation This field is divided into the following Bit Description DEM Set if the record should have been sent to receive stream 0 Set if the record should have been sent to receive stream 2 2 15 A 14 bit unsigned integer that corresponds to the filter rule this packet matched HDLC header Protocol header Length may vary depending on protocol 4 bytes Payload 7 rlen ERF header 16 bytes Extension headers optional bytes of record Protocol header 4 bytes 64 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide ERF 11 TYPE_COLOR_ETH Type Bit 7 1 Extension header present See Extension Headers page 71 Bits 6 0 Type 11 Short description TYPE_COLOR_ETH Long description Type 11 Colored Ethernet Record Use
77. page 39 e Converting captured data page 41 e Using third party applications page 43 e Transmitting captured data page 43 Basic data capture dagsnap is a software utility used to write to disk the raw data captured from a DAG card Data is collected in Extensible Record Format ERF which can then be viewed using dagbits or converted to other formats using dagconvert When capturing high speed data Endace recommends you use dagsnap see Capturing data at high speed page 37 For further information on the software utilities see e dagsnap page 36 e dagbits page 39 e dagconvert page 42 Starting a capture session To start the capture session type the following at the prompt dagsnap d0 v o tracefil where 0 is the device number of the DAG card you wish to capture data from Note You can use the v option to provide user information during a capture session although you may want to omit it for automated trace runs By default dagsnap runs indefinitely To stop use CTRL C You can also configure dagsnap to run for a fixed time period then exit 2008 Endace Technology Ltd Confidential Version 3 November 2008 35 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide 36 dagsnap dagsnap is a data capture software utility The following is a list of the options available in dagsnap Option Description d DEVICE device DEVICE Use the DAG device referred to by
78. plays the MAN pages The information displayed is dynamically based on the DAG card and does not work correctly when there is no DAG card in the system Note There are a few commands that display even though they are not applicable i interval seconds Interval to repeat in seconds m hmon Outputs the hardware monitor information n voltages Outputs the DAG card voltage monitor information S setattribute setattribute Sets individual attributes by attribute name Use in conjunction with the porta or portb options to get individual only multi port cards s statistics Outputs the statistics for the DAG card Verbosity levels from 0 basic default to 3 full T tree Outputs the supported Configuration and Status attributes and components with the description and name Using the v2 verbosity level also outputs all components and attribute codes Verbosity levels from 0 basic default to 3 full t txstats Outputs the transmit statistics for the DAG card Where applicable u ucounters Outputs the universal counters for the DAG card Where applicable v verbose level Sets the verbosity level from 0 basic to 3 full V version Display the DAG card version information Note For cards with more than 2 ports you can select the required port using portnumber or portletter 30 2008 Endace Technolo
79. pos Generate SONET tx clock internally Enables enable lt port gt or disables master or from rx clock slave disable lt port gt port for capture Port B nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate C phy ref select 0 master enableb config 0 nodiscard data laser nocrc sonet scramble pscramble oc12 pos Indicates card is in POS mode Tiodiinten framer receive mode GPPO0 varlen slen 9616 align64 4 Records will be generated with 64 bit Port A drop count 1 alignment This is not a configurable option Port B drop count 6 4 Number of packets dropped during the current capture sesion on this port Stream 0 stream drop count 0 Stream 2 stream drop count 0 Stream 4 stream drop count 0 Stream 6 stream drop count 0 Allows each memory PCI Burst Manager hole to operate Total memory in MB available independently 133MHz for allocation to this card buffer size 512 rx streams 4 tx streams 1 drop nooverlap Memory Streams mem i16 19124 16 124 0 124 0 1240 te tne c and tx elon ne TERF terf strip32 rx error a Off q Packets with RX errors are either discarded rx error b GFE notxrxerror or transmitted txrxerror time mode relative 4 TRTERF Mode Either no time mode no delay or relative 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide SONET 0C48 STM 16 Firmware dag54sga4
80. raffic load on the network link For information on increasing the buffer size see buffer size page 22 2008 Endace Technology Ltd Confidential Version 3 November 2008 37 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Capturing Raw SONET frames When capturing Raw SONET SDH each frame is encapsulated in one or more ERF records The ERF record type use is ERF 24 TYPE_RAW_LINK which is used in conjunction with extension header EH 5 RAW_LINK For details on the ERF records used see Data Formats page 59 For DAG cards capturing at OC3 OC 12 STM 1 STM 4 the interfaces generate one ERF record per frame The SONET SDH frame is aligned with record number 0 starting with the first A1 byte of the section overhead Single record frames are denoted within the ERF extension header information via the More Fragmentation bit and the Sequence Number field as follows For DAG cards capturing at OC48 STM 16 the interfaces generates four ERF records per frame The SONET SDH frame is aligned with record number 0 starting with the first A1 byte of the section overhead Single record frames are denoted within the ERF extension header information via the More Fragmentation bit and the Sequence Number field as follows 38 2008 Endace Technology Ltd Confidential Version 3 November 2008 Viewing captured data EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Data captured in ERF format can be viewed with dagbits For further details
81. ream buffer It does not mean writing the packet twice into the same stream buffer Packet duplication can simplify software architectures where different types of processing needs to be done on the same packet For example all traffic can be sent to stream buffer 0 for use by a capture to disk applications while only a subset of traffic is sent to stream buffer 2 for handling by a traffic analysis application For further details see EDM04 26 Enhanced Packet Processing Timed Release TERF TR TERF The Timed Release TERF TR TERF module is a option that enables you to transmit an ERF stream while reproducing the timestamps of the packets within that stream It is able to transmit on all channels TR TERF has two modes of operation They are No Delay Mode and Relative Timed Release Mode 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide Raw SONET capture Raw SONET capture mode is used to directly capture the entire SONET STS N frame or STM N frame and pass it onto an application for processing The frames are passed to the application within ERF records The processing performed on the SONET SDH frames is descrambling and standard physical layer error monitoring As with all DAG cards the physical layer status information is retrieved via the DAG Configuration and Status API Raw SONET capture mode can be used by a software application to 2008 Endace Techn
82. ribution Server User Guide e EDMI11 01 ERF types e PN01 13 DAG Card Quick Start Guide This User Guide and the EDM04 01 DAG Software Installation Guide are also available in PDF format on the installation CD shipped with your DAG 5 4SGA 48 card System Requirements General The minimum system requirements for the DAG 5 45GA 48 card are e Acomputer with at least a Intel Xeon 1 8GHz or faster and a minimum of 1GB RAM e Atleast one free PCIx slot supporting 133MHz operation e Software distribution requires 60MB free space e For details of the supported operating systems see one of the following documents e EDMO04 01 DAG Software Installation Guide e Current release notes See the Documentation CD or the Endace support website at https www endace com support Requires 12V external power via a 2 5mm Pitch 4 way Flat 3 5 Floppy Drive Power Connector A 3 1 2 to 5 1 4 floppy disk drive cable is supplied with the card to connect to the server s power supply This must be connected to a 12 V power supply Operating System This document assumes you are installing the DAG 5 4SGA 48 card in a computer which already has an operating system installed To install refer to EDM04 01 DAG Software Installation Guide All related documentation is included on the CD shipped with the DAG 5 4SGA 48 card Other Systems For advice on using an operating system that is substantially different from any of those specified above pleas
83. rte koe toto EEE EEE NESE ob abe tenuta 43 Transmitting captured data i f eb tieni eei rie a irte dires 43 Config rationic s sie pendere ne en ee coves ie ete ibo e ordeo 43 Explicit Packet Transmission aui boe ito eee bete eto ee i it ode tesa 43 Tr ce Files 553558 esee eon eie etd eie ote e eth 44 asi 44 Selectable CRE Length ertt ee tdeo etm e oe re ede ide rts 44 Retransmitting Errored Packets uico seem mein ree re desit odia 45 Usage INOI6S iss s EEEE E EEE EEE een ilc ee en be eeu eom bie etes 45 TRSEERE Kriown Issues 4er tei iiie eee t eoe eene ehe 46 Synchronizing Clock Time 47 OVeLVIOW 1a ese Dr eeemat oer ie stu st Meise Le vei reae EDEN Reo E ep DESEE PEDIDOS 47 DUCK Configurations assetto ede i REOR EN DEP AE R A E ORA 47 Common Synchronization sie t PR OO TERI C OH ERR EO ERU RE AS Ife ERE AA 47 Network Time Protocol C M 48 PIMOS CAINS PR 49 cdd c 49 Da Sele E 50 Dagclock Statistics reset scsi css cscetesssssotsvevaressicvasauts consnssosensusvasevevevaseviencaguntosencuasentveunvtesdeetsvevesevecevavebevesnecstonss 51 Dagclock output explained eret toti etie ra tope EHE HER ce eere oe hoe koe Eee phot 52 Card MH SII E M 54 OVENI W e E 54 Pulse Signal from External Source
84. s Example dagconfig 10 dagconfig 100 dagconfig 1000 align64 Sets the packets to be all generated as multiples of 8 bytes 64 bit aligned align64 before being received by the host Not a configurable option auto_neg noauto_neg Note From DAG software 3 1 0 onwards nic nonic is replaced by auto_neg noauto_neg Both options are valid and still can be used auto_neg noauto_neg is not supported by some older cards For Ethernet only The DAG 5 4SGA 48 card operates in either auto neg Or noauto neg mode In auto_neg mode you must connect the DAG 5 4SGA 48 card directly to a Ethernet switch or card with a full duplex cable in which case the DAG 5 4SGA 48 card will perform Ethernet auto negotiation The noauto_neg mode is intended for use with optical fiber splitters In this mode you must connect the receive socket of the DAG port to the output of an optical splitter inserted into a network link between two other devices The transmit socket of the DAG should be unconnected In noauto_neg mode Ethernet auto negotiation is not performed This allows you to use one splitter on each DAG receive port to monitor each direction of a full duplex Ethernet link Example dagconfig auto_neg dagconfig noauto_neg buffer_size The buffer size nMB indicates that a total of n MB of memory have been allocated to the DAG card in total Memory allocation occurs when the dagmem driver is loaded at boot time See EDM04 01 DAG Software Ins
85. s only 2008 Endace Technology Ltd Confidential Version 3 November 2008 29 EDMO01 31v3 DAG_5 4SGA 48 Card User Guide dagconfig options dagconfig is a software utility used to configure and display statistics By default all commands unless otherwise defined run on device 0 a0 Commands only apply to one DAG card The following is a list options available in dagconfig Not all options listed are applicable to all cards Options Description l1 porta Port A only default all Multi port cards only 2 portb Port B only default all Multi port cards only 3 portc Port C only default all Four port cards only 4 portd Port D only default all Four port cards only eporte Lo pOrtp As above for extra ports on the 3 7T DAG card c concfg lt conncfg gt Connection configuration Used by the DAG 7 15 only C counters Outputs the counters Verbosity levels from 0 basic default to 3 full d device device DAG device to use Default is d0 e extended Displays the current extended statistics non boolean and image dependant Verbosity levels from 0 basic default to 3 full Note Some images may not contain extended statistics G getattribute lt getattribute gt Gets individual attributes by attribute name Use in conjunction with the porta or portb options to get individual only multi port cards h help Dis
86. sage Notes The following points should be noted when using TR TERF e When using dagflood to transmit an ERF stream to the card you should set the 1 flag maximum data burst length to a value greater than the default of 1MB During testing Endace found a value of 16MB 16777216 to be effective This reduces the possibility of a buffer under run occurring if insufficient data is committed in a burst and the dagflood process is not scheduled by the OS to run in a timely manner e For best accuracy when testing you should ensure both the sending and receiving cards are synchronized to the same time source 2008 Endace Technology Ltd Confidential Version 3 November 2008 45 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide 46 TR TERF Known Issues TR TERF is less accurate when using multiple interfaces If a large packet is transmitted to the card on a particular interface the transmit buffers for the other interfaces may experience a buffer under run This is because data is only transferred to one interface at anyone time record by record as the input ERF stream is interleaved between interfaces This may be especially noticeable where very small packets are transmitted on one interface and very large packets transmitted on another interface TR TERF Head of line blocking may occur when performing timed transmission of packets across multiple interfaces Head of line blocking is defined as a delay transmitting a packet on one port
87. signal strength is too low to be recognized Oof Out of frame indicates the section overhead processor is not locked to the SONET stream PORT Indicates the port the status conditions apply to Rdi error Remote Defect Indicator When this condition exists the link will still carry data Rei error Remote error indicator Txlockerror Lock error on transmit 2008 Endace Technology Ltd Confidential Version 3 November 2008 31 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Ethernet Port lock los lof lop A 1 0 0 0 B 0 1 1 0 A definition of each of the ETHERNET status conditions up to printout level 2 is described below Condition Description Link The link is fully validated Lock Card is locked to the incoming signal and can capture data Lof Loss of frame Indicates oof has been asserted for more than 3 secs Lop Loss of pointer Los Loss of signal Indicates there is either no signal at the receiver or the optical signal strength is too low to be recognized Peer link The peer link is up Remote fault Remote error indicator Sfp detect SFP link detected Tx lock error Lock error on transmit Extended statistics Extended statistics are also available The following example shows the extended statistics dagconfig e For more information see dagconfig page 30 There are multiple printout levels The statistics displayed below
88. sy chaining TDS 6 expansion modules to the TDS 2 module Each TDS 6 module provides outputs for an additional 6 DAG cards Synchronize to an external source as follows dagclock d0 Output muxin rs422 muxout none status Synchronised Threshold 596ns Failures 0 Resyncs 0 error Freq 30ppb Phase 15ns Worst Freq 238ppb Worst Phase 326ns crystal Actual 100000023Hz Synthesized 67108864Hz input Total 225 Bad 0 Singles Missed 1 Longest Sequence Missed 1 start Thu Apr 28 14 55 20 2007 host Thu Apr 28 14 59 06 2007 dag Thu Apr 28 14 59 06 2007 Connecting the Time Distribution Server You can connect the TDS 2 module to the DAG card using DUCK crossover cable page 58 Note A 4 pin to RJ45 adapter may be required The TDS may be located up to 600m 2000ft from the DAG card depending upon the quality of the cable used possible interference sources and other environmental factors Please refer to the EDM05 01 Time Distribution Server User Guide for more in formation Caution Never connect a DAG card and or the TDS 2 module to active Ethernet equipment or telephone equipment Testing the Signal For Linux and FreeBSD when a synchronization source is connected the driver outputs messages to the console log file var log messages To test the signal is being received correctly and has the correct polarity use the dagpps tool as follows dagpps d0 dagpps measures the input state many times over several seconds displaying the polar
89. t in use Receive Characteristics Transmit Characteristics Wavelength Sensitivity Wave Tx Power nm dBm length dBm Product Number Network Support Min nm 850 OC 3 STM 1 OC 12 TXR OC3 OC12 MM STM 4 Multimode 770 860 20 OC 3 STM 1 OC 12 TXR OC3 OC12 SM STM 4 Single mode 1260 1600 2 TXR OC48 SM OC 48 STM 16 Single mode 1270 1600 las jo 9 TXR 10005X 1000BASE SX Multimode 770 s60 20 fo sso o lss 3 5 8 15 5 3 2 TXR 1000LX 1000BASE LX Single mode 1270 1600 3 2008 Endace Technology Ltd Confidential Version 3 November 2008 11 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide 12 Power Input Note The optical power input to the DAG card must be within the receiver s dynamic range See the previous table for details If it is slightly outside of this range it will cause an increased bit error rate If it is significantly outside of this range the system will not be able to lock onto the signal When power is above the upper limit the optical receiver saturates and fails to function When power is below the lower limit the bit error rate increases until the device is unable to obtain lock and fails In extreme cases excess power can damage the receiver When you set up the DAG card you should measure the optical power at the receiver and ensure that it is within the specified power range If it is not adjust the input power as follows e Insert an optical attenuator if power is too high
90. t to 48 per card This means that only the first 48 bytes of each packet will be captured If for example you want to capture only the IP header of each packet you may want to set the length to a different value Alternatively if you want to ensure you capture the whole packet you can set the length to a larger value Example Setting up a DAG 5 4SGA 48 card with a snap length of 200 bytes dagconfig d0 slen 200 Note The ERF header is not included in the s1en value Therefore a s1en of 48 will produce a 64 byte capture record made up of 48 bytes plus the number of bytes in the ERF header sonet Set framer into SONET mode Example dagconfig sonet Stream drop count Details the number of packets dropped during current capture session on this stream Example Stream 0 stream drop count 0 Stream 2 stream drop count 0 terf strip16 terf strip32 noterf strip Strips the CRC value 16 or 32 bits from the packet or sends packet as is notert strip The TERF line in the current configuration indicates the current Terf option Note Only displayed if the DAG card supports transmit i e has a terf image Example dagconfig terf stripl6 dagconfig terf strip32 dagconfig noterf strip time mode Sets the time release mode nodelay sends packets out with no delay Shows as no time mode relative mode sends out packets in the same time spacing as they where received Applicable to TR TERF Example dagconfig nodelay dagconfig relative
91. tallation Guide for details on how to allocate memory crc16 crc32 nocrc Sets the Packet over SONET PoS checking to None 16 or 32 bits SONET only Example dagconfig nocrc dagconfig crcl6 dagconfig crc32 22 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDM01 31v3 DAG_5 4SGA 48 Card User Guide default The default command initializes the DAG card configuration and sets all settings to default values The command also resets the DAG card configuration back to its default state Note When you run dagconfig d0 default the dagclock inputs and outputs are also reset to defaults Example dagconfig d0 default drop nodrop Determines if the DAG card s memory holes are de coupled In drop mode the memory holes are de coupled If the data rate on one memory hole slows the data rate on any other memory holes is not affected The dropping of packets occurs at the individual stream s burst manager In nodrop mode the memory holes are coupled The speed of the slowest memory hole determines the overall speed of the data rate The dropping of packets occurs at the port Example dagconfig drop dagconfig nodrop drop count Note The drop nodrop setting for the DAG card determines where the packets are being dropped Port GPP no drop The number of packets dropped during current capture session on this port Resets to 0 when dagconfig reset is run Stream Burst manager drop The number of packets dropp
92. ter OFF Remove the PCIx bus slot screw and cover Using an approved ESD protection device attach the end with the strap to your wrist and pull or clip firmly so there is firm contact with your wrist Securely attach the clip on the other end of the strap to a solid metal area on the computer chassis as shown below Anti static wrist strag Clip secured to chassis Insert the DAG 5 4SGA 48 card into PCIx bus slot ensuring it is firmly seated If this DAG card requires an external power supply complete the following steps a Connectthe supplied or equivalent power cable to the external power connector on the DAG card b Connect the cable to the appropriate power connector on your server s power supply unit Check the free end of the card fits securely into the card end bracket that supports the weight of the card Secure the card with the bus slot cover screw Turn power to the computer ON 10 Ensure the blue FPGA successfully programmed LED on the DAG card illuminates 2008 Endace Technology Ltd Confidential Version 3 November 2008 9 EDM01 31v3 DAG_5 4SGA 48_Card_User_Guide Port Connectors The DAG 5 4SGA 48 card has two industry standard Small Form factor Pluggable SFP port connectors each consisting of an optical fiber transmitter and receiver as shown below Port A 4 pin socket Data capture in has signal for time Sync PPS In The Card progress Yellow LED creak is receiving a time T se
93. the two DAG cards monitor different directions of a single full duplex link You can synchronize the DAG cards in two ways e One DAG card can be a clock master for the second This is useful if you want both DAG cards to be accurately synchronized with each other but not so for absolute time of packet time stamps or e One DAG card can synchronize to the host and also act as a master for the second DAG card Synchronizing with Each Other Although the master DAG card s clock drifts against UTC the DAG cards will be locked together This is achieved by connecting the time synchronization connectors of both DAG cards using a DUCK crossover cable page 58 Note A 4 pin to RJ45 Adapter may be required Configure one of the DAG cards as the master so that the other defaults to being a slave as follows dagclock d0 none overout Output muxin none muxout over status Not Synchronised Threshold 596ns Failures 0 Resyncs 0 error Freq Oppb Phase Ons Worst Freq 213ppb Worst Phase 251ns crystal Actual 100000000Hz Synthesized 67108864Hz input Total 0 Bad 0 Singles Missed 0 Longest Sequence Missed 0 start Thu Apr 28 14 48 34 2007 host Thu Apr 28 14 48 34 2007 dag No active input Free running Note The slave DAG card configuration is not shown as the default configuration will work 2008 Endace Technology Ltd Confidential Version 3 November 2008 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Synchronizing with Host
94. ts erfts tv tv sec lts gt gt 32 lts lts amp OxffffffffULL 1000 1000 lts lts amp Ox80000000ULL lt lt 1 rounding tv tv usec lts gt gt 32 if tv tv_usec gt 1000000 tv tv usec 1000000 tv tv sec 1 2008 Endace Technology Ltd Confidential Version 3 November 2008 49 EDMO1 31v3 DAG_5 4SGA 48_Card_User_Guide Dagclock 50 The DUCK is very flexible and can be used with or without an external time reference It can accept synchronization from one of several input sources and also be made to drive its synchronization output from one of several sources Synchronization settings are controlled by the dagclock utility Note You should only run dagclock after you have loaded the appropriate FPGA images If at any stage you reload the FPGA images you must rerun dagclock to restore the configuration Note when you run dagconfig d0 default the dagclock inputs and outputs are also reset to defaults A description of each argument is shown below Option Description d DEVICE Use the DAG device referred to by DEVICE Supported syntax includes 0 device DEVICE dag1 and dev dagj to refer to DAG cards h Display the information on this page help usag k Wait for DUCK synchronization before exiting sync K NUM Set the synchronization timeout in seconds default is 60 seconds l NUM
95. x streams 4 tx streams 1 drop nooverlap Memory Streams mem i16 9124 16 124 0 124 0 124 0 to the x and te elream e in MB TERF tert tripaz Packets with RX errors are either discarded rx_error_a off notxrxerror or transmitted txrxerror rx error b off time mode FNAEENE IR TER Meda pher no ime mode 2008 Endace Technology Ltd Confidential Version 3 November 2008 19 EDMO01 31v3 DAG_5 4SGA 48_Card_User_Guide Ethernet Firmware edag54sga48pci ipf fbmr4tl eth pci v2 2 4v1x40ff1148 2008 10 14 17 0 47 user Copro edag54sga48cp ipf fbmr4tl eth cp v2 1 4vlx40ff1148 2008 9 25 17 34 29 user Serial 45879 MAC Address A 00 0e a7 aa bb cc MAC Address B 00 0e a7 aa bb cd Sets eql or unsets noeq equipment loopback Sets fcl or unsets nofcl facility loopback Note eql loops back to the bus ote fcl loops back to the line Port A nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate 1 phy ref select 3 master enablea config 0 nodiscard data nolaser auto neg tx crc crc32 pmincheck enablea 100 Generate SONET tx clock internally Enables enable lt port gt or disables master or from rx clock slave disable lt port gt port for capture Port B nophy bist enable noeql nofcl phy tx clock off phy kill rxclock reset phy rate 1 phy ref select 3 master enableb config 0 nodiscard data nolaser auto neg tx crc crc32 pmincheck enableb 1000 Indicates the Ethernet mode
96. yte stream in network order no byte or re ordering is applied The generic ERF header is shown below bytes byte2 byte1 byteo ERF header Extension header s Protocol header Padding Payload The fields are described below EISE The time of arrival of the cell an ERF 64 bit timestamp UE 7 Extension header present Extension header type See table below This byte is divided into several fields as follows Bits Description 1 0 Binary enumeration of capture interface 11 Interface 3 or D 10 Interface 2 or C 01 Interface 1 or B 00 Interface 0 or A Cards with more than four interfaces typically use Multichannel ERF types type 5 to 9 12 and 17 which provide a separate larger interface field Varying length record When set packets shorter than the snap length are not padded and rlen resembles wlen When clear longer packets are snapped off at snap length and shorter packets are padded up to the snap length rlen resembles snap length Setting novarlen and slen greater than 256 bytes is wasteful of bandwidth Truncated record insufficient buffer space e wlen is still correct for the packet on the wire e rlen is still correct for the resulting record But rlen is shorter than expected from snap length or wlen values Note truncation is depreciated and this bit is unlikely to be set in an ERF record RX error An error in the received data Present on the wire DS error A

EDM01-31v3 DAG_5.4SGA

Contents

Download Pdf Manuals

Related Search

Related Contents