Home

PLBv46 Endpoint Bridge for PCI Express in a ML505

1. b E Type Len H Req ID H Tag H Last DW BE H First DW BE H Bus Num H Dev Num H Func Num H Reg Num Data 4 Bytes H Comp ID H Status z M TEZE Type Len H Req ID H Tag H Last DW BE H First DYV BE H Bus Num H Dew Num H Func Num H Reg Num Data 4 Bytes H Comp ID H Status Type Len H Req ID H Tag H Last DW BE H First DW BE H Bus Num H Dev Num H Func Num H Reg Num Data 4 Bytes H Comp ID H Status y Type Len H Req ID H Tag H Last DV BE H First DW BE H Bus Num H Dev Num H Func Num H Reg Num Data 4 Bytes H Comp ID H Status B E w E2 Type Len H Req ID H Tag H Last DW BE H First DY BE H Bus Num H Dev Num H Func Num H Reg Num Data 4 Bytes H _ Comp ID H Status sees ae ere bi Programming Interface T 1 Byte 7 00 7 Identifie bits Sib Gass Code te w ee bit 16 i Base Class Code 1 Byte 05 Memory controller Broadly RES X1030_15_040908 Figure 15 Results from Catalyst Configuration XAPP1030 v1 0 1 May 6 2008 www xilinx com 15 Catalyst Testing lt XILINX Figure 16 shows an excerpt of the Exerciser cfg_x1 sdc file The file contains the stimuli TLPs While it is generally easier to read and edit the TLPs using the Catalyst Display Viewer the text file is readable and editable and more details are provided than c
2. Output Design Netlist iscratch designs mI505_mb_plbv46_pcie Browse Output Directory ich designs ml505 mb_plbv46_pcie_cs in Browse ent adel Device Settings Device Family Vines st Yj Use SRL16s v Use RPMs lt Previous Next gt X1030_72_040908 Figure 72 Opening plbv46_pcie cdc XAPP1030 v1 0 1 May 6 2008 www xilinx com 67 Using ChipScope with the PLBv46 Endpoint Bridge XILINX 5 The plbv46_pcie cdc provides a good starting point for analyzing designs In most analyses additional nets are needed Figure 73 shows the GUI for making net connections Click Next four times to move to the Modify Connections window Select Modify Connections The Filter Pattern is used to find net s As an example of using the Filter Pattern enter ack in the dialog box to locate acknowledge signals such as SI_AddrAck In the Net Selections area select either Clock Trigger or Data Signals Select the net and click Make Connections Correct Clock Trigger and or Data signals displayed in red o x Select Net Structure Nets Net Selections pcie_bridge wrapper Trigger Signals Data Signals PCle_Bridge comp_master_bridge RxSFIFO_64 RxSFIFO fifo_7O0x5 12 Clock Signals PCle_Bridge comp_master_bridge TxSFIFO CompFiIFO_64 dpram dpram_7 PCle_Bridge comp_plbw46_master USE_SYNC_RD_LL
3. IslslelsilLels 2 Trigger Setup Waveform Signals DEV 4 UNIT 0 Data Port M_ABus M_wrDBus SiI_rdDBus 0 Sl_rearbitrate 1 JPCIE_LUSER_C 2 SI_wrDAck 3 M_rdBurst 4 SlwrBTerm 5 PLB_MVrDAck 6 M_request 7 Si_rdComp 8 PLB_PAValid 9 IP2INTC_Irpt 10 5l_wrComp 11 DataPort 11 12 iM_wrBurst 13 5l _addrAck 14 PLB_MAddrAc _ Waveform DEV 4 MyDevice4 XC5VLX50T UNIT 0 MyiLAO0 ILA Waiting for upload Device 4 Unit 0 Waiting for trigger Sample Buffer has 100 samples 9 X1030_76_040908 Figure 76 ChipScope with Trigger Armed 13 Run XMD or GDB to trigger patterns which cause ChipScope to display waveform output For example set the trigger to SI_addrAck arm the trigger and run xmd tcl xmd_commands dma tcl at the command prompt This produces signal activity in the Analyzer waveform viewer XAPP1030 v1 0 1 May 6 2008 www xilinx com 71 Using ChipScope with the PLBv46 Endpoint Bridge lt XILINX shows the bus signals generated in Step 10 To share the results with remote colleagues save the results in the waveform window as a ChipScope Pro Analyzer m1505_mb_plbv46_pcie File View JTAG Chain Device TriggerSetup Waveform Window Help Ell T ie ole elilele a 7 4 Project miS505_mb_plbv46_ _ waveform DEV 4 MyDevice4 MC5VLX501 UNIT 0 MyILAQ ILA DEV 2 MyDevice2 l
4. Figure 26 MRd32 Performance Results Length 3 XAPP1030 v1 0 1 May 6 2008 www xilinx com 24 Using Catalyst to test PCle Performance XILINX Root Complex to Endpoint Write Transactions Figure 27 shows a write transaction The length field is set to O20H or 128 bytes The data written is an Upcount pattern The Continuous radio button is selected Catalyst Enterprises Inc SPX Analyzer Exerciser Software rc2ep_wr64 File Edit View Configuration Tools Project Setup Window Help 0 548082 0u al gt m m B2 e SAE OX OlPe SOe S e Link Chart Average Payload fo x Link Chart Throughput uje 9D ss J fv21aae Payload 128 00 Bytes aaa a a BB Payload Throughputrt85 3 MB Se6 J 2212 Throughput247 1 MB Se6 rc2ep_wr64 Exerciser Program Performance Items Link Settings Setting T Global Loop 1 a BIK x No Loop C Count Continuous Packet Number Req ID H Last DW BE H First DW BE H E 0 MEM WRITE Addr H OQOOUDU0U006B0000000 Analyzer 1 000 85000159 USB Exerciser Run Analyzer Run A X1030_27_040908 Figure 27 MWr64 Performance Results Length 20 XAPP1030 v1 0 1 May 6 2008 www xilinx com 25 Endpoint to Root Complex Transactions XILINX Figure 28 shows the performance results from running a continuous MWr82 transaction The data and payload throughput are 241 3 MB s and 185 3 MB s Catalyst Enterprises Inc SPX
5. Select the bit file download bit as shown in Figure 59 Look in implementation ct BSlaveSerial z cache rs232_uart_1_wrapper aiSelectMAP x cdock_generator_0_wrapper xps_bram_if_cntlr_1_bram_wrapper My Recent ddr2_sdram_wrapper xps_bram_if_cntlr_1_wrapper Documents debug_module_wrapper xps_central_dma_0_wrapper B SystemACE 3 E dmb_entlr_wrapper i 5 dimb_wrapper PROM File For dmb Bj Desktop ilmb_cntlr_wrapper E download_cclktemp bit jilmb_wrapper system bit oo lleds_sbit_wrapper R aiD Scurene imb _bram_wrapper E mb_plb_wrapper 3 E microblaze _0_wrapper F Opcie _bridge_wrapper IMPACT Processes M Camer 2 pcie_diff_clk_wrapper Available Operations ar y pi proc_sys_reset_O_wrapper My Network File name download bd Places Files of type FPGA Bit Files bit X Cancel Operations E Boundary Scan R PROM File Formatter Output Format mes Swap Bits true LoadDirection UP PROM Basename m1505_mb_plbv46_pcie File Location H designs m1505_mb plbv46_pcie implementation Auto Selcet false Number of Data Stream 1 Number of PROMs 1 PROM Name xcf32p PROM Size 4194304 Bytes END of Report BATCH CMD setCurrentDeviceChain index 0 BATCH CMD setattribute design attr name value 0 lt il L Output Error J Waring _ Transcript Configuration Platform Cable USB 6 MHz usb hs X1030_59_040908 Figure
6. Click on BARO and use the edit ConfReg dialog box to change the value of BARO to xE000000C as shown in Figure 63 Click Write ConfReg and then Refresh Dump The new value of BARO is displayed ea m direct select bus dev o func 4 az El host CPU 0 00 0 0 01 0 1 00 0 0 27 0 0 28 0 0 28 4 3 00 0 0 28 8 4 00 0 0 29 0 Le arer e s 02922 0 29 3 0 29 7 0 30 0 0 31 0 0 31 1 0 31 2 0 31 3 0 gt 1 1 O gt 2 2 0 gt 3 3 O gt 4 4 0 gt 5 5 Host PCI Bridge I PCI PCI Br VGA PC Compati o Multimedia 80 PCI PCI Br PCI PCI Br Other Memory C PCI PCI Br Ethernet Netwe Universal Host Cor Universal Host Cor Universal Host Cor Universal Host Cor o serial bus Dev Subtractive PCI ISA Bridge De o Mass Storage Cif o Mass Storage Cc SMBus Serial Bus 3 0 0 Other Memory Controller VID xlOEE Xilinx Corp DID xO0505 no device name found no SubVID x0000 SubID xo0000 rev x00 16 C 64 edit ConfReg Nr of ConfRegs Ixzhoooooc hex use BIOS int refresh dump itype l xs Write ComtReg sites congas refr after wr Config Space Dump DID VID Stat Cmd BaseClass SubClass I BIST Header LatTimer BAR O mem pref 64b BAR 1 BAR Z BAR 3 BAR 4 BAR 5 Cardbus CIS Ptr SubID SubVendorID Exp_ROM_ BAR reserved reserved maxLat minGnt IntPir 0000 000c gt X1030_63_040908
7. Figure 63 Defining BARO in PCltree XAPP1030 v1 0 1 May 6 2008 www xilinx com 57 PCltree Testing gt XILINX Figure 64 is XMD output which shows that BARO has been written as 0xE000000C The XMD mrd command also shows that the data in the initial 8 addresses in XPS BRAM is 0x00000000 As noted earlier the XMD displays data in Big Endian formant while the x86 displays data in Little Endian format ec Command Prompt xmd 85002014 alallala 85C02018 68080808 85C8281C 1515 5 5 5 5 5 RMD mrd x85cH2606 8 85C62088 EE16 5 5 85C02004 66661608 85C62088 66668405 85C0200C i 5 5 515 515 5 GCOBHGER 98080808 98080808 515 5 15151515 5 RMD mrd x8AE16086 8 SAE1 6006 98080008 8AE10004 5 5 5 515 515 5 6 5 5 515 515 5 5 5151515 515 5 6 515 515 515 5 96080808 68680808 515 515 5 5 5 5 x1030_64_040908 Figure 64 XMD showing the Configuration Space Header XPS BRAM XAPP1030 v1 0 1 May 6 2008 www xilinx com 58 PCltree Testing XILINX Figure 65 shows the memory test for PCI tree To run the memory test click on Mem Test at the lower left of the BAR Space GUI Check Auto Read Memory at the top of the BAR Space GUI to display memory values in the left side of the display To edit a memory location highlight the location to be edited and enter the value in the Edit memory dialog box Click on Write Memory To view the results click on the Ref
8. Link Chart Average Payload OX Link Chart Throughput fg Data Throughpute1 6 MB Sec J 2verage Payload 61 54 Bytes BR Payload Throughput38 8 MB Se0 per_ep2rc_x1 Exerciser Program Performance Items Link Settings Setting Items Group 0 Link Usage DLink Utilization O Transmission Efficiency r Report Direction Link Aggregate I Upstream Transmit XAPP1030 v1 0 1 May 6 2008 e J Throughput BA Psi I Downstream Receive werage Payloa C Number of Packets O Latency Request On Upstream Transmit Analyzer 1 000E85000159 USB Exerciser Stop Analyzer Run X1030_32_040908 Figure 32 EP to RC Performance Test Using C Code www xilinx com 30 LeCroy Testing XILINX LeCroy Testing Use the LeCroy ML505 test setup shown in Figure 33 to verify the PLBv46 Endpoint Bridge using the LeCroy tester as root complex including configuration and data transactions The ML505 is inserted into the host emulator The m1505_ mb plbv46_pcie lecroy directory contains the stimuli files which use peg as the filename extension This section discusses the procedures used in setting up the LeCroy including defining the Recording and Generation Options Root Complex to Endpoint transactions are discussed followed by a section on Endpoint to Root Complex transactions Ya LeCroy PETracer TM PCI Express Protocol Analyzer Eile Setup Record Generate View To
9. Note The Reverse Bus Order operation is useful for analyzing buses in Analyzer Si ChipScope Pro Analyzer ml1505_mb_plbv46_pcie File View JTAG Chain Device TriggerSetup Waveform Window Help gt mit SRI eleja DEV 4 MyDevice4 cov System Monitor Cons Project ml505_mb_plbv46_ DEV 2 MyDevice2 XC9504 DEV 3 MyDevice3 Syster UNIT O MyILAO ILA Trigger Setup Waveform 3 4 7 E Trigger Setup DEV 4 MyDevice4 XC5VLX50T UNIT 0 MyiLAO ILA o Match Unit Function Radix Counter o C MO TriggerPortd gt OOCK Bin jisabled Trigger Condition Name Trigger Condition Equation Active TriggerConditionO MO Signals DEV 4 UNIT 0 Data Port Sal JM_ABus iM_wrDBus SI_rdDBus CH 0 SI_rearbitrate CH 1 PCIE_USER_ CH 2 Sl_wrDAck CH 3 M_rdBurst CH 4 Sl _wrBTerm CH 5 PLB_MVW rDAck CH 6 M_request CH 7 Sl_rdComp CH 8 PLB_PAValid CH 9 IP2INTC_Irpt CH 10 SIwrComp CH 11 DataPort 11 CH 12 M_wrBurst CH 13 Sl_addrAck CH 14 PLB_MAddrA CH 15 DataPort 1 5 CH 16 SI_rdBTerm CH 17 PLB_MRearbi CH 18 SI_rdDAck CH 19 Sl_wait CH 20 PLB_MRdDA CH 21 M_RNW CH 22 SI_rdDBus lt 0 CH 23 SI_rdDBus lt 1 gt Window Windows 1 Depth 1024 v Position 0 All Data Bus Signal M _AB
10. 000 1111 0000 Packet 25 FE ee 1st BE Last BE DEE i i 60000000 Packet 25 CornpleterlD 000 00 0 000 00 0 sc 1 dword vies MRd 32 1st BE Last BE i 00 00000 000 00 0 sooo0000 1111 oo00 CompleteriD Data 5 10 01010 000 00 0 000 00 0 12345678 PETrainer ML SN 1102 f Link State InitFC State SEH SS BSa Complete B B i B pal B t Packet _ Packet _ EE _ Packet _ _Packet _ Packet Fa _Packet _ Packet _ Packet _ ae ae Ki Search Fwd Figure 43 BARO Test Results XAPP1030 v1 0 1 May 6 2008 www xilinx com 41 LeCroy Testing XAPP1030 v1 0 1 May 6 2008 XILINX Figure 44 shows the verification of the Endpoint to Root Complex PCle transactions using XMD In the system mhs the PLBv46 Endpoint Bridge generic C_PCIBAR2IPIFBARO is 0x8AE10000 the location of XPS BRAM This shows that the 0x12346578 written by the LeCroy Root Complex MWr64 TLP is resident in XPS BRAM ex Command Prompt xmd WMD mrd x8aece1GG00 8 SAE16006 SAE1 004 SAE1 G08 SAE1G GC 12345678 99880008 66660608 66060680 lalalalalalala 666606080 66660606 9o880068 Figure 44 XMD Verification of BARO Tests www xilinx com X1030_44_ 040908 42 LeCroy Testing lt XILINX Figure 45 shows an excerpt of a peg file The peg file used as stimuli in LeCroy transactions is readable and editable In the
11. Endpoint to Endpoint to root complex transactions are tested using XMD commands and C code Two Root Com plex software projects pcie_dma and pcie_mch_dma generate Direct Memory Access DMA T ti transactions which create PCle traffic This code provides an interface to the user which allows ransactions the selection of the number of loops to run and the seed The code generates and verifies pseudo random traffic patterns on the PCle link The pcie dma c code uses one DMA channel The pcie_mch_dma c code allows specification of 1 3 DMA channels XAPP1030 v1 0 1 May 6 2008 www xilinx com 5 Endpoint to Root Complex Transactions XILINX The PLBv46 Endpoint Bridge Configuration Space Header CSH must be written for the code to run correctly The Catalyst and LeCroy scripts cfg_x1 sdc and cfg_x1 peg set up the configuration space header of the PLBv46 Endpoint Bridge The Catalyst PCI Express Bus Protocol Exerciser Analyzer has memory located at address 0x00000000 In the reference systems the PLBv46 Endpoint Bridge generic C_IPIFBAR2PCIBAR_0 is set to 0x00000000 This is different from the Base System Builder BSB generated value for C_IPIFBAR2PCIBAR_O Figure 5 shows the selection of the pcie_dma software project Aa Xilinx Platform Studio fhomeflesters designsimls05_mb_plbv46_pciefsystem xmp System Assembly View1 x File Edit View Project Hardware Software Device Configura
12. FDFFFFFF 64660808 FBFFFFFF 66660608 F9FFFFFF X1030_67_040908 Figure 67 XMD Verification of PCltree Write Operation In the next two figures XMD is used to write XPS BRAM which is then read by PCltree XAPP1030 v1 0 1 May 6 2008 www xilinx com 61 PCltree Testing gt XILINX Figure 68 shows the writing and reading of 0x12345678 to the first four locations in XPS BRAM e Command Prompt xmd lof x SAE16018 o8HH0008 SAE1 1C 51515 5 151515 5 RMD mrd x8AE16086 8 66660808 FFFFFFFF 02000000 FDFFFFFF 64680808 FBFFFFFF 66660608 F9FFFFFF AMD mur x8AELAOHO G x12345678 4 RMD mrd x8AE16088 8 SAE1 60060 12345678 SAE16004 12345678 SAE1 608 12345678 12345678 64660808 FBFFFFFF 66 466608 F9FFFFFF X1030_68_040908 Figure 68 Writing XPS BRAM using XMD XAPP1030 v1 0 1 May 6 2008 www xilinx com 62 Memory Endpoint Test lt XILINX Figure 69 shows a PCltree read of XPS BRAM The first four locations are read as 0x1234567 BAR space 78563412 78563412 78563412 78563412 oooo00004 FFFFFFFB oooo0006 FFFFFFF9 00000008 FFFFFFF ooooo008 FFFFFFFS oooooo00c FFFFFFF3 oooo000E FFFFFFF1 00000010 FFFFFFEF ooo00012 FFFFFFED 00000014 FFFFFFEB 00000016 FFFFFFE9 000000lg FFFFFFE 000000lA lt x00000000 gt x00000004 gt lt x00000008 gt x0000000C gt x00000010 gt lt x00000014 gt x00000018 gt lt x0000001C gt lt x000000Z0 gt x0000
13. connect mb rs p o o o o o o o o DMAS DMAS ts ou while for set mdm tfile m tfile m tfile 1 i 1 open dma txt 0x85C001E0 0x80200000 0x80200030 0x80200004 0x80200008 0x8020000C 0x20000000 0x20002000 R mrd 0x80200014 1 R BUSY 0x40000000 DMA Status Register i lt 1000 incr if DMASR S DMASR BUSY puts Soutfile puts Soutfile puts Soutfile mwr mrd mrd close Soutfile exit XAPP1030 v1 0 1 May 6 2008 0x80200010 0x20000000 0x20002000 Ww 1 0x003F0107 0x00000004 0x00000003 0xC0000004 0x20000000 0x20002000 0x12345678 100 0x0 100 SDMASR i 64 X1030_30_040908 Figure 30 dma tcl www xilinx com 28 Endpoint to Root Complex Transactions XILINX Figure 31 shows the Catalyst SPX4 Analyzer Exerciser output after running the ep2rc_ spf performance analyzer project The payload throughput depends on various factors such as the size of the transfer if print statements are included in the source code and if the verification is included in the source code For this run all print statements are removed there is no verification and length is set to 20 The transfer is from XPS BRAM to Catalyst memory across the PCle link The data throughput is 19 0 MB s and the payload throughput is 8 3 MB s Catalyst Enterprises Inc SPX Analyzer Exerc
14. signals in Inserter and generate the ICON and ILA cores 5 From ML505_mb_plbv46_pcie implementation copy the file displayed in the Inserter Output Design Netlist window usually implementation system2 ngo to implementation system ngc 6 In XPS run Hardware Generate Bitstream Inserting ChipScope in the PLBv46 Endpoint Bridge The m1505 mb plbv46 pcie chipscope plbv46_ pcie cdc file is used to insert a ChipScope ILA core into the pcie_bridge_wrapper core Do the following steps to insert a core and analyze PLBv46 Endpoint Bridge signals with ChipScope 1 Invoke XPS Run Hardware Generate Netlist 2 Copy chipscope plbv46 pcie cdc file to the project area usually either one directory above the chipscope directory or the implementation directory 3 Run Start gt Programs ChipScope Pro gt ChipScope Inserter 4 From ChipScope Inserter run File Open plbv46_pcie cdc XAPP1030 v1 0 1 May 6 2008 www xilinx com 66 Using ChipScope with the PLBv46 Endpoint Bridge XILINX Figure 72 shows the ChipScope Inserter setup GUI after File Open plbv46_pcie cdc T ChipScope Pro Core Inserter plbv46_pcie cdc File Edit Insert Help Dee eo gt mCore Utilization Core resource estimation is currently not available for this architecture Select Device Options Design Files Input Design Netlist scratch designs mI505_mb_plbv46_pcie Browse
15. 10 The data and payload throughput are 135 1 MB s and 133 5 MB s Catalyst Enterprises Inc SPX Analyzer Exerciser Software rc2ep_rd64 File Edit view Configuration Tools Project Setup Window Help D 54808 u alm m BL Le HABE CK Ole e POee Sia s L E Link Chart Average Payload j B x Link Chart Throughput 8 J Dt Throughputt135 1 MB Sec J Svetage Payload 65 15 Bytes Sa By Pavioad Throughput 133 5 MB Sec 90 80 70 60 50 40 30 20 rc2ep_rd64 Global Loop C NoLoop Count Continuous Analyzer 1 000E85000159 USB Figure 24 MRd64 Performance Results Length 10 XAPP1030 v1 0 1 May 6 2008 www xilinx com 22 Using Catalyst to test PCle Performance XILINX Figure 25 shows the performance results of a MRd64 TLP of length 100 The data and payload throughput are 135 2 MB s and 133 6 MB s Catalyst Enterprises Inc SPX Analyzer Exerciser Software rc2ep_rd64 BEE File Edit View Configuration Tools Project Setup Window Help O eB8S ul e gt m m BE P eln AACD KIOPE Link Chart Average Payload Bel EE E paid au o0 jE J 222 Thouohputi35 2 MB Seo J fvetage Payload65 18 Bytes JR Payload Throughput 133 6 MB Sec re2ep_rd64 DER Exerciser Program Performance Items Link Settings Setting Global Loop C NoLoop Continuous Req ID H Last DW
16. Analyzer Exerciter Software Link Chart Average Payload File View Configuration Tools Project Setup Window Help 22u88 u a JASE t wWhRE OR OP E SOe Bs ei Link Chart Throughput J 2282 Throughputi2a1 3 MB Sec J fverage Payload 128 00 Bytes a ee ee I Payload Throughput 185 3 MB Sec f ax Global Loop C NoLoop C Count Continuous BP PE Last OW BEH First WBE H Addr H Data H enw 020 M E AT X1030_28_040908 Figure 28 MWr32 Performance Results Length 20 Endpoint to This section measures the performance of Endpoint to Root Complex transactions The stimuli Root Complex for these transactions are generated using the Xilinx XPS Central DMA Controller DMAC in Transactions the system mhs The functionality of the DMA controller is discussed earlier in this application note The DMA transaction is from the address specified in the DMAC Source Address register to the address specified in the DMAC Destination Address register The length of the DMA transaction is specified by the value in the DMAC Length register XAPP1030 v1 0 1 May 6 2008 www xilinx com 26 Endpoint to Root Complex Transactions XILINX Prior to generating the stimuli the performance test is set up Figure 29 shows the importing of the performance test setup file catalyst pcie_dma spf The throughput measurements are aggregate Catalyst Enterprises Inc SPX Analyzer Exerciser Software pcie_dma
17. BE H First DW BE H Addr H Analyzer 1 000E85000159 USB Exerciser Run Analyzer Run 4 X1030_29_030408 Figure 25 MRd64 Performance Results Length 100 Because of a Catalyst limitation performance measurements can not be done for TLPs longer than 400 bytes XAPP1030 v1 0 1 May 6 2008 www xilinx com 23 Using Catalyst to test PCle Performance XILINX Figure 26 shows the performance of MRd32 transactions of length 3 The data and payload throughput are 114 6 MB s and 15 4 MB s Catalyst Enterprises Inc SPX Analyzer Exerciser Software PerformanceAnalyzer3 Carat ysr DER Fie Edit View Configuration Tools Project Setup Window Help 0 c585 oua m a Beee naine nxore vee amn OU Link Chart Average Payload D X Link Chart Throughput ox uje a gt Efe ijuje lao fr 100 ld J Data Throughputt114 5 MB Seo Bz Average Payload 6 00 Bytes Payload Throughput 15 4 MB Sec PerformanceAnalyzer3 DER Exerciser Program Performance Items Link Settings Setting EA x K sym H Seat Fmt H TypecH TC TOC EPC attr Len H Req ID H Tag H Last DW BE H First DW BE H Addr H Digest H LCRC H K Sym H w Analyzer 1 000E85000159 UsB Exerciser Stop Analyzer Stop A X1030_26_042408 Global Loop lr NoLoop Count p Continuous
18. Enterprises Inc SPX Analyzer Exerciser Software rc2ep_rd64 0 548 uwa gt B S A eln A AEDS E ene aI Exerciser Program Performance Items Link Settings Setting Global Loop 1 H B x C NoLoop Count Continuous Fw oo Analyzer 1 000E85000159 USB X1030_23_040908 Figure 23 Defining MRd64 Performance Stimuli The next figures show the performance results of MRd64 transactions varying the length of the TLP The single continuously transmitted TLP stimuli just defined is shown in the pane at the bottom of the figure The left pane is a Link Chart which provides the average payload size The right pane is a Link Chart which provides the data throughput and the payload throughput In the Performance Items tab Link Usage Number of Packets and Latency are unchecked Under Report Directions Aggregate is checked In the following tests Data Throughput is the overall bus traffic of all non idle packets divided by the update interval Payload Throughput is the payload data of TLPs divided by the update interval The update interval defined in the Settings tab for the performance measurements in this document is 1 second The MRd performance is the round trip time including the MRd and Completiion with Data packets XAPP1030 v1 0 1 May 6 2008 www xilinx com 21 Using Catalyst to test PCle Performance XILINX Figure 24 shows the performance results of a MRd64 TLP of length
19. PCle_Bridge in the System Assembly View to invoke the PLBv46 _PCle generics editor The generics shown in Figure 2 are used to configure the PLBv46 Endpoint Bridge The Xilinx Device ID 0x0505 and Vendor ID 0x10EE are displayed in many of the PCle tests done in this application note PCle_Bridge plbv46_pcie_v1_00_a HDL Toggle Names Datasheet All Buses C_IPIFBAR_NUM C_INCLUDE_BAROFFSET_REG C_PCIBAR_NUM C_NO_OF_LANES C_DEVICE_ID oxosos C_VENDOR_ID foxtore C_CLASS_CODE oxossooo C_REV_ID oxoo C_SUBSYSTEM_ID oxoo00 C_SUBSYSTEM_VENDOR_ID oxo000 C_COMP_TIMEOQUT C_MPLB_AWIDTH C_MPLB_DWIDTH C_MPLB_SMALLEST_SLAVE X1030_02_040908 Figure 2 PLBv46 Endpoint Bridge Parameters Implementation Results The resource utilization in the reference design is shown in Table 2 Table 2 Design Resource Utilization Resources Used Available Utilization Slice Registers 11984 28800 41 Slice LUTs 12247 28800 42 DCM_ADV 2 12 12 Block RAM 56 60 93 XAPP1030 v1 0 1 May 6 2008 www xilinx com 3 ML505 Setup XILINX ML505 Setup Figure 3 shows the ML505 Embedded Development Platform The ML505 has a x1 PCle connector on one edge of the printed circuit board R omm ER x Z w 4 X W E x gt XC5VLX50T X1030_03_040908 Figure 3 ML505 PCI PCle Evaluation Platform Executing the The sequence of
20. PLBv46 Endpoint Bridge configuration space and memory in a PC environment This is the least expensive and easiest way to learn about a PCle hardware test environment The use of the ChipScope tool in debugging PLBv46 Endpoint Bridge issues is described Included The reference system for the PLBv46 Endpoint Bridge in the ML505 Embedded Development System Platform is available at http www xilinx com support documentation application_notes xapp1030 zip The reference system in the xapp1030 zip file is described on page 2 Introduction The PLBv46 Endpoint Bridge is a PCle endpoint instantiated in a Xilinx FPGA which communicates with a root complex The reference systems are tested using commercial test equipment from LeCroy and Catalyst LeCroy and Catalysts are two Analyzers Exercisers used to verify PCle systems The Catalyst and LeCroy testers allow generation analysis capture and triggering of Translation Layer Data Link Layer and Physical Layer packets The reference systems are also tested in two test environments which are inexpensive and PC based The PLBv46 Endpoint Bridge is tested using the LeCroy and Catalyst testers as root complex The ML505 Evaluation Board is inserted into the LeCroy or Catalyst PCle slots for testing Sample Catalyst scripts are provided in the m1505 mb plbv46 pcie catalyst directory Sample Lecroy scripts are provided in the m1505_mb plbv46 pcie lecroy directory The tests for the PLBv46 Endpoint B
21. an inexpensive method of verifying PLBv46 Endpoint Bridge functionality PCltree and the Memory Endpoint Test run on PCs Figure 55 shows the ML505 in a Dell 390 PC The PC runs Windows XP and has the ISE EDK and PCltree software installed The PC PCle integrated circuits act as root complex The Dell 390 has a x1 connector for PCle slot 1 and a x8 connector for PCle slot 4 In the Dell 390 only 4 of the 8 lanes of the x8 connector are active The ML505 is powered from the ML505 power supply The ML505 is inserted in PCle slot 1 The USB Platform Cable is connected to the ML505 JTAG port for Impact XMD and ChipScope operations A Serial Communication Cable is connected to communicate to a communication terminal X1030_55_040908 Figure 55 PC Test Environment The power up sequence of the PC affects the PCle scan In order for BIOS to recognize the drivers and PCle BARs at power up the FPGA bit file should be loaded prior to PC power up It is possible to configure the FPGA after PC power up using JTAG mode but a warm restart is usually required to get a PCI scan to work A warm restart is a PC Shutdown with Restart Xilinx recommends writing the ML505 XCF32P PROM with the contents of the MCS file so that configuration occurs at power up Xilinx recommends the use of the Master SelectMap configuration mode Since it is faster than Master Serial mode the ML505 is more likely to be configured at the time of the PCle scan Configuring
22. sets the source address destination address and DMA length The pcie_dma code is used for DMA operations between user defined source and destination addresses Figure 6 shows the parameters in pcie_dma c which are edited to test PCI transactions between different memory regions The elf for pcie_dma c runs on the MicroBlaze processor in the xc5vIx50t FPGA on the ML505 pcie_mch_dma The pcie_mch_dma project runs multi channel Direct Memory Access DMA operations The user sets the source address destination address and DMA length for each channel The pcie_mch_dma code is used for DMA operations between user defined source and destination addresses As with the pcie_dma code the parameters in pcie_mch_dma c which can be edited to test PCI transactions between different memory regions are DMAChannel BAR The elf for pcie_mch_dma c provided in ready_for_download as pcie_mch_dma elf runs on the MicroBlaze processor in the xc5vIx50t FPGA on the ML505 DMA Transactions As examples of specifying the source and destination addresses in DMA transactions the source address may be an address in the ML505 XPS BRAM and the destination address a Catalyst memory across the PCle link Another option is source address in Catalyst memory to a second location in Catalyst memory define MEM_0O_BASEADDR 0x8AE10000 define MEM_1_BASEADDR 0x20000000 DMALength 1024 X1030_06_040908 Figure 6 Defining Source and Destination Addresses Length
23. steps to test the PLBv46 Endpoint Bridge reference system differs depending Reference on whether endpoint to root complex transactions or root complex to endpoint transactions are System run For endpoint to root complex transactions the steps must be run in the order below For root complex to endpoint transactions the steps are the same but there is no elf to download Change directories to the ready_for_download directory 1 Use iMPACT to download the bitstream impact batch xapp1030 cmd 2 Invoke XMD and connect to the MicroBlaze processor xmd XAPP1030 v1 0 1 May 6 2008 www xilinx com 4 Testing the PLBv46 Endpoint Bridge gt XILINX connect mb mdm rst 3 Download the executable dow pcie dma elf 4 Write to the Bridge Control Register to enable Bus Master and the BARs mwr 0x85C001E0 0x003F0107 5 Use the Catalyst to write the PLBv46 Endpoint Bridge Configuration Space Header File gt Open catalyst cfg x1l sdc 6 From Catalyst click Run 7 From the XMD prompt run con Testing the The system including the interface to the LeCroy Catalyst test equipment is shown in Figure 4 PLBv46 The root complex is the Catalyst or LeCroy test equipment and the endpoint is the PLBv46 i Endpoint Bridge in the ML505 reference system Endpoint Bridge S Analyzer Catalyst or LeCroy Exerciser Root Complex Catalyst or LeCroy X1030_04_040908 Figure 4 PLBv46 Endpoint Bridge System Identifying Root Complex Endpoint
24. the Catalyst Link Settings Select the Platform mode hidden behind the Link Status pane Click on the Link Status button to invoke the Link Status pane displayed The figure shows a Link Width 1 so the link is up and trained as x1 Catalyst Enterprises Inc SPX Analyzer Exerciser Software cfg_x1 Fie Edit View Contig ati 2 Setup Wingos Hen O sA S u a PEE m BS Le HABE OX OF E POR Hla Exerciser Program Capture Trigger On Link Settings Settings ion Tools Project Seti Physical Layer Settings SP Reference Clock Link Width Slow Speed System Clock 1 1 25 Gbps s 5 Cx C x4 Cxe ia 5 32 Slow Speed System Clock 1 25 2 5 Gbps c Full Speed System Clock 3 3 supports Spread Spectrum Clocking Link Settings 3 Transmitter p n Scramble Bypass Polat Physical Data Link Layers Status Lane 0 Physical Layer Status LTSSM Lo Link Number 0x00 Link Width 1 15 14 1312 11 109 a 7 6 5 amp 3 lt 2 SO Lane Polarity 4 4 4 Data Link Layer Status DLCMSM ACTIVE Update Check All Check All Platform Mode Transmitter Device Clear All Clear All Cc sll C End Point Mode Receiver Device ersal I Auto Polarity I Lane Reversal I Auto Polarity Advanced Options I Define different patterns for pre trigger and post trigger data captures Easy switch to Advanced mode x1030_0
25. 0024 gt lt x000000Z8 gt lt x000000Z2C gt x00000030 gt lt x00000034 gt lt x00000038 gt x0000003C gt lt x00000040 gt x00000044 gt x00000048 gt lt x0000004C gt lt x000000S0 gt x000000S54 gt lt x000000S8 gt x000000SC gt x00000060 gt lt x00000064 gt x00000068 gt V auto read memory Memory Space typeZ prefetchable base 0000000 range f fff0000 64 KByte 64bit BARs not supported yet edit memory Data toggle ae Write Memory comt ee verif loop on off a refresh view after write mem copy source destination mem copy select view range KB range 0 63 FFFFFFES lt x0000006C gt oooo001c lt x00000070 gt FFFFFFES lt x00000074 gt ooooo01E lt x00000078 gt FFFFFFElL lt x0000007C gt o00000z0 x00000080 gt Display range men test load tite save file 128 Bytes 1024 Bytes X1030_69_040908 Figure 69 PCltree Read of XPS BRAM Memory Endpoint Test The Memory Endpoint Test MET is run on a PC with the ML505 inserted into a PCle slot MET provides a simple method of writing and reading memory Like PCltree the ML505 memory written read is the BRAM and or DDR2 defined in the system mhs and addressed with the PLBv46 Endpoint Bridge C_PCIBAR2IPIFBAR_ generics The MET requires the installation of the Xilinx Virtex 5 PCle Endpoint Driver The Xilinx application note XAPP1022 Using the Memory Endpoint Test MET Drive
26. 1024 Bytes Figure 65 Running PCltree Memory Test X1030_65_040908 XAPP1030 v1 0 1 May 6 2008 www xilinx com 59 PCltree Testing XILINX Figure 66 shows the results of running the memory test The leftmost column shows the count pattern used for data The count increments for even addresses and decrements on odd addresses With the PCltree read of BARO the data is the count value specified in the PCltree memory test The results No Errors are provided Pcilree BAR space oooo0000 FFFFFFFF oooo0002 FFFFFFFD oooo0004 FFFFFFFB oooo0006 FFFFFFFS oooo0008 FFFFFFF oooo000a FFFFFFFS oooo000c FFFFFFF3 Oo00000E FFFFFFF1 00000010 FFFFFFEF ooo00012 FFFFFFED oooo0014 FFFFFFEB oooo0016 FFFFFFES oo000018 FFFFFFE oo00001A FFFFFFES ooooo001c FFFFFFE3 oooo001E FFFFFFE1 oooo00z0 lt x00000000 gt lt x00000004 gt lt x00000008 gt lt x0000000C gt lt x00000010 gt lt x00000014 gt lt x00000018 gt lt x0000001C gt x000000Z0 gt x000000Z24 gt x000000Z28 gt x0000002C gt x00000030 gt x00000034 gt lt x00000038 gt lt x0000003C gt lt x00000040 gt lt x00000044 gt lt x00000048 gt lt x0000004C gt lt x000000S0 gt x000000S4 gt x000000S8 gt x000000SC gt x00000060 gt x00000064 gt x00000068 gt x0000006C gt lt x00000070 gt lt x00000074 gt lt x00000078 gt lt x0000007C gt lt x00000080 gt m select view range V auto re
27. 1_030408 Apply Settings Bun Triggered Exerciser Z X1030_13_040908 Figure 13 Catalyst Link Settings XAPP1030 v1 0 1 May 6 2008 www xilinx com 13 Catalyst Testing XILINX Figure 14 is a graphical view of the stimuli for configuring the PLBv46 Endpoint Bridge including BAR 0 The m1505_mb_plbv46_pcie catalyst directory contains the cfg_x1 sdc stimuli file The cfg_x1 sdc project is loaded using the File gt Open pull down menu The sdc files are readable text files which contains the transactions used as stimuli In cfg_x1 sdc the Device ID Vendor ID is read The Command Status register is written and read The Revision ID and Class Code register is read In the figure the Name column provides the type of transaction and the Reg Num column specifies the register in the Configuration Space Header BARPO is written and read BARO is a 64 bit BAR with the lower 32 bits defined at Configuration Space Header CSH Register Number 4 and the higher 32 bits defined at CSH Register Number 5 Packets 10 and 11 are Configuration Writes and packets 12 and 13 are Configuration Reads In the Data field in packet 10 the endianess of the data written is swapped Catalyst Enterprises Inc SPX Analyzer Exerciser Software cfg_x1 File Edit View Configuration Tools Project Setup Window Help O cha S ulalereic gt m m BL ee WRB OR OlHSE POR BB Exerciser Program Capture Trigger On Link Settings S
28. 2 carat YST k Fie Edt ols Pr j elp T 0 826 ujal gt m m BL E t nA AEDO E Exerciser Program Performance Items Link Settings Setting Items Group Report Direction Link Usage x DLink Utiization IM Link Aggregate O Transmission Efficiency I Upstream Transmit Throughput Average Payload C Number of Packets 1 Latency F Downstream Receive I pcie_dma2 Check All Uncheck All B examples mur ey pcie_dma Note Link Aggregate E direction does not File name Files of type PerformanceAnalyzer Files spf Apply Settings Start Analyzer 1 000E85000159 USB Exerciser Stop Analyzer X1030_29 040908 Figure 29 Importing Performance Test Setup XAPP1030 v1 0 1 May 6 2008 www xilinx com 27 Endpoint to Root Complex Transactions lt XILINX To generate stimuli either C code or an XMD script is used to write the DMAC registers Figure 30 shows an XMD script to generate stimuli Using XMD scripts and commands allows the relatively quick verification that the operation is functioning correctly After running a DMA transaction a mrd command can verify that the data at the source and destination addresses are equivalent XMD commands may be too slow to give maximum performance results The DMA Status Register is monitored to determine if the DMAC is Busy When it is not busy a DMA transaction is initiated by a write to the DMAC Length register set outfile
29. 46_pcie sig x t lt Dutput Eror l Waning PROM File Generation Target Xilinx PROM 14 052 352 Bits used File mIS05_mb_plbv46_pcie in Location H designs ml505_mb_plbv46_pcie implementation o REES X1030_60_040908 Figure 60 Selecting Generate File The recommended configuration mode is Master SelectMap which is specified when the configuration Mode Switch SW3 should be set to MO 0 ON M1 0 M2 1 Use Impact to download the mcs file into the ML505 XCF32 PROM Select the XCF32P left click to invoke a menu and select Program Under the Programming Properties menu check Parallel Mode under PROM Specific Properties Insert the ML505 into the PCle slot and power on the PC Verify that the DONE LED is lit PCltree Testing PCltree is shareware available from http www pcitree de It runs on Windows XP PCltree can be used for either PCI or PCle tests In the tests described in this section the ML505 Embedded Development Platform is inserted into a Dell 390 x1 slot for the ml505_mb_plbv46_pcie project XAPP1030 v1 0 1 May 6 2008 www xilinx com 54 PCltree Testing XILINX Invoke XMD and enable the master and BARs by writing to the Bridge Control Register mwr 0x85C001E0 0x003F0107 Figure 61 shows the XMD output when the PLBv46 Endpoint Bridge configuration space header registers are read At power up the Device ID is 0x0505 and the Vendor ID is Ox10EE BARO is 0x0000000C The va
30. 59 Specifying the Bit File XAPP1030 v1 0 1 May 6 2008 www xilinx com 53 PCltree Testing XILINX Figure 60 shows the generated mcs file Users generating th ePROM file for the first time should reference pages 101 107 of UG201 v1 4 Virtex 5 FPGA ML555 Development Kit for PCI and PCI Express Designs E iMPACT PROM File Formatter DER T File Edit View Operaties Window Help DBH XAXA xN al H 2iBoundary Scan Fa SlaveSerial TalSelectMAP FaDesktop Configuration Fa Direct SPI Configuration xef32p E SystemACE 41 88 Full xeSvbcS0t B PROM File Formatter download bit ble Operations are ate File Operations Ep Boundary Scan E PROM File Formatter BATCH CMD setAttribute configdevice attr path value H designs ml505_mb_plbv46_pcie BATCH CMD setAttribute configdevice attr name value m1505_ mb plbov46_pcie g Total configuration bit size 14052352 hits Total configuration byte size 1756544 bytes BATCH CMD setCurrentDesign version 0 BATCH CMD generate Swap bit can only be disabled in Hex file format only Oxlacd8O 1756544 bytes loaded up from 0x0 Using user specified prom size of 4096K Writing file H designs m1505_mb plbv46_pcie implementation m1505_ mb plbv46_pcie mes Writing file H designs m1505_mb_plbv46_pcie implementation m1505_mb_ plbv46_pcie prm Writing file H designs m1505_ mb plbv46_pcie implementation m1505 mb plbv
31. 6 2008 www xilinx com 35 LeCroy Testing gt XILINX Figure 38 shows the use of XMD to enable the Bridge Control Register The BCR enables the Bus Master and the Base Address Registers BARs c Command Prompt xmd x MO Of PG BPBAKDOINCE scccneccececene a A a No of Read Addr Data Watchpoints No of Write Addr Data Watchpoints Instruction Cache Support on Instruction Cache Base Address brag 3151515151515 Instruction Cache High Address Ox9f FFE LEE Data Cache SUppOStcc ccc ece case cee s Data Cache Base Address Data Cache High Address Exceptions Support PU AOU NO ME as cle e oseiniae o crele EA T O Hard Divider Support Hard Multiplier Support X Barrel Shifter Support MSR clr set Instruction Support Compare Instruction Support MDM UART Target mb target id server for mb target Cid gt at TCP port no 1234 System reset successfully AMD mur x85C6G1e8 Ox483F6187 AMD v X1030_38_040908 Figure 38 Using XMD Commands to Write the Bridge Control Register After generation and recording options are specified and the BCR is written the link must be trained The Link State is displayed at the bottom of the PETracer GUI Prior to training the Link State is displayed as Detect Quiet as shown at the bottom of Figure 37 After training the Link State is displayed as LO
32. Application Note Embedded Processing Reference System PLBv46 Endpoint Bridge for PCI Express in a ML505 Embedded Development Platform Author Lester Sanders XILINX XAPP1030 v1 0 1 May 6 2008 Abstract This reference system demonstrates the functionality of the PLBv46 Endpoint Bridge for PCI Express used in the Xilinx ML505 Embedded Development Platform The PLBv46 Endpoint Bridge uses the Xilinx Endpoint core for PCI Express in the Virtex 5 XC5VLX50T FPGA The PLBv46 Bus is an IBM CoreConnect bus used for connecting IBM PPC405 and PPC440 and the MicroBlaze microprocessors to Xilinx IP cores A variety of tests generate and analyze PCle traffic for hardware validation of the PLBv46 Endpoint Bridge PCle transactions are generated and analyzed by Catalyst and LeCroy test equipment For endpoint to root complex transactions the pcie_dma software application generates DMA transactions which move data over the PCle link For root complex to endpoint transactions Catalyst and LeCroy scripts generate PCle traffic A Catalyst script which configures the PLBv46 Endpoint Bridge and performs memory write read transactions is discussed The steps to use Catalyst to measure PCle performance are given and performance results are provided The principal function of the section on performance measurements is to show how performance measurements are made Two stand alone tools PCltree and Memory Endpoint Test are used to write and read
33. Digits FF PROGRESS END End Operation Bicet hoa Elapsed time 2 sec PROM File Name Untitled Rat i i ff BATCH CHD identifyMPM EH Amet g if j omy X1030_56_040908 Figure 56 ML505 Boundary Scan Chain XAPP1030 v1 0 1 May 6 2008 www xilinx com 50 Testing with a PC XAPP1030 v1 0 1 May 6 2008 Provide the PROM file name as shown in Figure 57 iMPACT Prepare PROM Files want to target a Xilinx PROM Generic Parallel PROM 31d Party SPI PROM PROM Supporting Multiple Design Versions PROM File Format MCS OTEK UFP C format EX0 BIN Isc HEX Swap Bits XILINX Spartan3E MultiBoot Checksum Fill Value 2 Hex Digits FF PROM File Name ml505_mb_plbv46_pciel X1030_37_022808 Location H designs ml505_mb_plbv46_pcie implementation lt a X1030_57_040908 Figure 57 Defining the PROM File www xilinx com 51 Testing with a PC XILINX Specify the XCF32P PROM as shown in Figure 58 iMPACT Specify Xilinx PROM Device C Auto Select PROM C Enable Revisioning Number of Revisions C Enable Compression Select a PROM bits xcf EA Position Part Name 0 xcf32p Delete All C Add Data Files X1030_58_040908 Figure 58 Specifying the XCF32P PROM XAPP1030 v1 0 1 May 6 2008 www xilinx com 52 Testing with a PC XILINX
34. INK I_LLINK_RD_BKEND P SladdrAck PCle_Bridge comp_plbw46_master USE_SYNC_WR_LLINK _WR_LL_BACKEND SLrdDAck PCle_Bridge comp_slave_bridge comp_rx_fifo GEN_64 COMP_RX_RAM dpr ST am PCle_Bridge comp_slave_bridge GEN_TX_64_FIFO comp_tx_tifo fifo_ 72x51 z PLB_RNW PCle_Bridge comp_tlif RxFIFO_64 rx_afifo fifo_ 70x32 d SLrdDBus lt 0 gt gt PCle_Bridge comp_tlif TxFIFO_64 tx_afifo fifo_ 70x16 i6 __YSLrdDBus lt 1 gt S_LrdDBus lt 2 gt g 4 SLrdDBus lt 3 gt I gt i Sl_rdDBus lt 4 gt SLrdDBus lt 5 gt Net Name Pattern Filter 4 SLrdDBus lt 6 gt SLrdDBus lt 7 gt Net Name pees se eee 3 Sl_rdDBus lt 8 gt P2INTC_Irpt ee P M SlrdDBus lt 9 gt MSl_request p p P 3 SLrdDBus lt 10 gt SPLB_Rst p F SlLrdDBus lt 11 gt PaPe Sl_rdDBus lt 12 gt SLrdDBus lt 13 gt SLrdDBus lt 14 gt SLrdDBus lt 15 gt SlLrdDBus lt 16 gt SLrdDBus lt 17 gt SL_rdDBus lt 18 gt SLrdDBus lt 19 gt PLB_wrBurst Sl_addrAck Sl_rdBTerm MPLB_Rst Make Connections Move Ne ieee p p E Remove Connections Move Ne X1030_73_040908 Figure 73 inserter Data Signals 6 Click Insert to insert the core into pcie_bridge_wrappe
35. Lwr Adar MEE LCRC Time Stamp fo 0o o0 OxDF6ACEB1 0000 000 000 000 s PETrainer ML SN 1102 Link State InitFC State t SE Ae amp 5 Jat B5 Lo Complete Ready x1030_47_040908 Figure 47 EP to RC MRd32 Test Stimuli 1 DW 44 LeCroy Testing XILINX Figure 48 shows results from running the EP to RC memory read The peg is loaded Start recording by clicking on the Sun icon in the menu bar Click the Traffic Light icon Generate a 1 read using XMD mrd 0x20000000 1 Click the Black Square to stop recording and view the results gt LeCroy PETracer TM PCI Express Protocol Analyzer C Program Files CATC PETracer data1 157 pex DEAR EF Eile Setup Record Generate Report Search View Tools Window Help lax OLLI OFTERE EAE JE ED 32 00 00000 000 00 0 o 00000000 ENN Last BE ET Time Delta Time Stamp 0000 OxAOBIEEFT 1004s 0033 288 160 804 s z LI CompleterID 10 01010 000 00 0 o 000 01 0 BCM Byte Cnt Lwr Addr LCRC SC 0 0 0x00 12345678 0x25586D9F 0033 288 161 808 s PETrainer ML SN 1102 p Link State InitFC State BERR S amp B o Complete Ready Search x1030_48_040908 Figure 48 EP to RC MRd32 Test Results 1 DW Figure 49 shows the ep2rc_mrd32_4dw peg for a four doubleword Endpoint to Root Complex MRd32 S LeCroy PETracer TM PCI Express Protocol Analyzer H lecroy ep2rc_mrd32_4dw peg DER pl File Setup Record Generate Report Search View Tools
36. MENT OR FITNESS FOR A PARTICULAR PURPOSE IN NO EVENT WILL XILINX BE LIABLE FOR ANY LOSS OF DATA LOST PROFITS OR FOR ANY SPECIAL INCIDENTAL CONSEQUENTIAL OR INDIRECT DAMAGES ARISING FROM YOUR USE OF THIS APPLICATION NOTE XAPP1030 v1 0 1 May 6 2008 www xilinx com 75
37. MRd32 TLPs S LeCroy PETracer TM PCI Express Protocol Analyzer H lecroy rc 2ep_cfg_wr_rd_bar0 peg ite File Setup Record Generate Report Search View Tools Window Help Snp de eo R DER laix Sa Lae CfgvVvi0 Data 10 00100 000 00 0 000 00 0 0x010 aE 00000060 GE a 3DW header no 3DW header no data Cpl CpID Cfg ENE 1st BE BEJE g 10 ar 000 00 0 000 00 0 e 0x014 EHA 00000000 Coats freee ma 3DW header no 3DW header no data Cpl CpID 0o00 1st BE Last BE GETS pooooo0 Mem 2 00 ooooooo PETrainer ML SN 1102 Link State 45 5 4 25 Ready InitFC State Complete Search Fwd X1030_42_040908 Figure 42 Configuring and Testing BARO XAPP1030 v1 0 1 May 6 2008 www xilinx com 40 LeCroy Testing XILINX Figure 43 shows the results after running rc2ep_cfg_wr_rd_bar0 peg Packet 9 is a MWr32 of 0x12345678 to address 0x0000000060000000 The address is translated using the generic C_PCIBAR2IPIFBAR_0 to XPS BRAM at 0x8AE10000 In packet 12 the data value 0x12345678 is returned in the CpID packet The status fields indicate Successful Completion SC Ee File Setup Record Generate Report Search View Tools Window Help BD eM elm A Ft RAZ eo RRS Packet 5 CompleterID 000 00 0 000 00 0 Packet ANN 2 5 1st BE 000 00 0 CompleterlD Base Address Register 1 000 00 0 ox00000000 1st BE Last BE
38. PETrainer Exerciser Reference This reference system includes the MicroBlaze Processor MPMC XPS BRAM XPS INTC System XPS GPIO XPS UART Lite XPS Central DMA and PLBv46 Endpoint Bridge Both the Specifics processor and the bus run at a frequency of 125 MHz The MicroBlaze processor uses 2 KB for the instruction cache I cache and 4 KB for the data cache D cache MPMC runs at a frequency of 125 MHz and is set up for three ports Figure 1 is the block diagram of the reference system MicroBlaze Processor X1030_01_040908 Figure 1 Block Diagram of Reference System Table 1 provides the address map of the system Table 1 Reference System Address Map Peripheral Instance Base Address High Address MDM debug_module 0x84400000 Ox8440FFFF XPS INTC xps_intc_O 0x81800000 Ox8180FFFF XPS GPIO xps_gpio_0O 0x81400000 Ox8140FFFF XPS BRAM CNTLR xps_bram_if_cntlr_1 0x8AE10000 Ox8AE1FFFF XPS Central DMA xps_cdma_0O 0x80200000 Ox8020FFFF PLBv46 Endpoint plobv46_pcie_0 0x85C00000 Ox85COFFFF Bridge XAPP1030 v1 0 1 May 6 2008 www xilinx com Implementation Results Table 1 Reference System Address Map lt XILINX Peripheral Instance Base Address High Address XPS Uartlite RS232 0x84000000 0x8400FFFF LMB Cntlr ilmb_cntlr 0x00000000 0x00001FFF LMB Cnitir dimb_cnilr 0x00000000 Ox00001FFF MPMC DDR2_SDRAM_32Mx32 0x90000000 OxOFFFFFFF In XPS double click on
39. Report Search view Tools Window Help sae D CEE QQ SRR 2 ee obe F CfgRdO ist BE J oomoo oo0 00 0 o ist BE Last BE 01 00000 ooo oo 0 o ist BE Last BE SEEE 11 00000 ooo o0 0 0 12345678 esl ne yen 01 00000 000 00 0 0 AAAAAAAA F0000100 al By Bl o c AddressHi Oxaaaaaaaa AddressLo OxFo000100 Script commands z Payload 0x12345678 Packet Idle Packet TLP TLPType MRd64 Link Length 1 Config FirstDwBe Oxf Wait LastDwBe 0x0 Template AddressHi Oxaaaaaaaa Include Loop AddressLo OxF0000100 Repeat Branch LI Proc Link State IFC State Complete Search Fwd X1030_41_040908 Figure 41 RC to EP Write Read Test The next figures show BARO configuration packets followed by write then read operations on BARO XAPP1030 v1 0 1 May 6 2008 www xilinx com 39 LeCroy Testing XILINX Figure 42 shows the configuration of BARO and the read write and read transactions The address of BARO is OxO000000060000000 Packet 0 is a CfgWr of the lower order address and packet 2 is a CfgWr of the higher order address Packets 4 and 5 use CfgRd TLPs to verify the configuration writes Packets 6 7 and 8 are MRd32 MWr32 and MRd32 TLPs used to read and write BARO memory Double click on the Data field in packet 7 to display the 1234678 value The endianess of the address in the CfgWr0 TLP differs from the endianess of the address in the MWr32 and
40. To initiate training click on the Connect icon To disable a trained link click on the Disconnect icon XAPP1030 v1 0 1 May 6 2008 www xilinx com 36 LeCroy Testing 7 XILINX Figure 39 shows LeCroy ML505 PLBv46 Endpoint Bridge link is trained with the LTFSM in LO If the clocking and resets are correct link training occurs in less than one second If link training is unsuccessful the LTFSM cycles through training states S LeCroy PETracer TM PCI Express Protocol Analyzer H lecroy cfg_x1 peg BAL FI File Setup Record Generate Report Search view Tools Window Help 8 x HE eH o 2 F We CK PRE ER a Koa Idle Time Stamp Ox2AC19647 0 000ns 0000 000 000 000s IE E ke Mlo o Ke KRKE MBS The following scripts perform configuration reads and writes to configure space Script commands Packet H template TLP Idle i Link Name MyCfigirite Template name Config TlpType CfgUrO Write device Configuration Space Wait FirstDwBe OxF First DW Byte Enables Template Length 1 1 DWORD Include Loop Payload OxFFFFFFFF Repeat Branch Proc AddressSpace a template TLP n 4 gt cfg_x1 peg LU Link State InitFC State i g Ta E i 8 ies amp E Lo Complete Traffic generation finished Ready ili i Search Fwd X1030_39_040908 Figure 39 LeCroy After Link Trained XAPP1030 v1 0 1 May 6 2008 www xilinx com 37 LeCroy Tes
41. Window Help lal x sae tei o n AG st RQB SRR Dae he TEE CompleteriD o 10 01010 000 00 0 o 000 01 0 Lwr Addr f Data LCRC o o oo OxF6A3A9DA PETrainer ML SN 1102 3 Link State InitFC State n5 66 S4 lu Complete Ready Search X1030_49_ 040908 Figure 49 EP to RC MRd32 Test Stimuli 4 DW XAPP1030 v1 0 1 May 6 2008 www xilinx com 45 LeCroy Testing XILINX Figure 50 shows results from running the XMD command below mrd 0x20000000 4 S LeCroy PETracer TM PCI Express Protocol Analyzer C Program Files CATC PETracer data1 157 pex DER EF File Setup Record Generate Report Search Yiew Tools Window Help 8 x 2 BE p HE ont yp RART ORR 2 Me De whee TLP CompleterID 10 01010 000 00 0 0 000 01 0 SC 0 Byte Cnt Lwr Addr f Data o oo Packet PANS TP RTE aa i 00 00000 000 00 0 0 o0000000 1111 0000 3 Di 35 a 00 00000 000 00 0 0 0o00000C 1111 0000 Packet Z tea 32 ee FE Las SE e x1 36 00 00000 000 00 0 0 0000000C 1111 0000 A Link State InitFC State Complete Search Fwd X1030_50_040908 Figure 50 EP to RC MRd32 Test Results 4 DW Endpoint to Root Complex Write Transactions Figure 51 shows the peg for the EP to RC MWr82 As with EP to RC memory reads start recording by clicking on the Sun icon and then click on the traffic light wait TLP TLPType MWr32 X1030_51_040908 Fig
42. _Bridge 1_wrBTerm PCle_Bridge PLB_rdBurst PCIe_Bridge PLB_PAValid 900 0 900 FFFFFFFF 00000000 pi 20000000 UU l AANA WUN jl AMN mMM MA AA ANN Upload X1030_78_040908 Figure 78 System Debugging Using ChipScope XAPP1030 v1 0 1 May 6 2008 www xilinx com 73 Reference Design Matrix XILINX Reference The reference design matrix is shown in Table 4 Design Matrix Table 4 Reference Design Matrix General Developer Name Xilinx Target devices stepping level ES production speed grades Virtex 5 XC5VLX50T Production Silicon Source code provided No Source code format VHDL Design uses code IP from an existing reference design application No note 3rd party or CORE Generator software Simulation Functional simulation performed No Timing simulation performed No Testbench used for functional simulations provided No Testbench format N A Simulator software used version i e ISE software Mentor N A Cadence other SPICE IBIS simulations No Implementation Synthesis software XST EDK Software EDK10 1i Implementation software tools used versions ISE10 1i Static timing analysis performed Yes Hardware Verific
43. _central_dma_O M_wrD CH 71 dxps_central_dma_O M_wrDI CH 72 xps_central_dma_O M_wrDB _ CH 73 xps_central_dma_O M_wrDB CH 74 fxps_central_dma_O M_wrDI CH 75 kps_central_dma_O M_wrD CH 76 PCle_Bridge S _wrDAck CH 77 PCle_Bridge PLB_MVVrBTern CH 78 PCle_Bridge SI_wrBTerm CH 79 PCle_Bridge PLB_rdBurst CH 80 PCle_Bridge PLB_PAValid CH 81 PCle_Bridge SI_wrComp CH 82 PCle_Bridge PLB_RNW CH 83 PCle_Bridge PLB_MAddrAck CH 84 PCle_Bridge PLB_wrBurst CH 85 PCle_Bridge Sl_addrAck CH 86 PCle_Bridge S _wait CH 87 JPCle_Bridge PLB_ABus lt 0 gt CH 88 PCle_Bridge PLB_ABus lt 1 gt v T MMAND upload UU INFO Device 0 Unit 0 Waiting for core to be armed Waveform DEV 0 MyDevice0 XC5VLX507 UNIT 0 MyILA0 ILA Bus Signal x o 400 360 320 280 240 200 160 120 80 40 0 40 80 120 160 200 240 ADDR2 SDRAM 32Mx32 SPLB0_PLB ABUS DDR2_SDRAM_32Mx32 SPLBO_ 1_rdDAck PCIe_Bridge PLB_ABus xps_central_dma_0 M_wrDBus xps_central_dma_0 M_ABus xps_central_dma_0 M_request xps_central_dma_0 MPLB_MRdDAck xps_central_dma_0 1_rdDAck xps_central_dma_0 SPLB_RNW xps_central_dma_0 MPLB_MWrBTerm xps_central_dma_0 MPLB_MWrDAck xps_central_dma_0 SPLB_PAValid xps_central_dma_0 M_wrBurst xps_central_dma_0 31_addrack xps_central_dma_0 MPLB_MAddrAck xps_central_dma_0 xps_central_dma xps_central_dwa_0 xps_central_dma PCIe_Bridge 1_wrDAck PCIe_Bridge PLB_MWrBTerm PCIe
44. aab24 AAPAHAHA 166106066 HEG66636 86666412 646666061 HOHOHOHO bHH60008 MEM 32 Hex 66666646 gt X1030_71_040908 Figure 71 Running the Memory Endpoint Test XAPP1030 v1 0 1 May 6 2008 www xilinx com 65 Using ChipScope with the PLBv46 Endpoint Bridge XILINX Using ChipScope with the PLBv46 Endpoint Bridge ChipScope is used to debug hardware problems Debugging is done at either the system or PLBv46 Endpoint Bridge level To analyze PLBv46 Endpoint Bridge internal signals insert the ChipScope cores into implementation pcie_bridge_wrapper ngc To analyze signals involving multiple cores insert the ChipScope cores into system ngc The flow for using the two debugging methods differs Below an outline of the steps for debugging at the system level is provided This is followed by a detailed list of steps for debugging at the core level Inserting ChipScope at the System Level The following steps insert the ChipScope cores into the system 1 In XPS select Hardware Generate Netlist 2 From the command prompt in the implementation directory run ngcbuild i system ngc system2 ngc 3 Copy the chipscope m1505 mb plbv46 pcie cdc file to the project area usually either one directory above the chipscope directory or the implementation directory 4 Invoke ChipScope Inserter To specify the input in the Input Design Netlist window browse to the system2 ngc file created in step 2 Define the Clock Trigger and Data
45. ad memory Memory Space type prefetchable base e0000000 range ffff0000 64 KByte 64bit BARs not supported yet edit memory Data f_i toggle M count verify mem copy destination KB range 0 63 4 T x1030_01_030408 MB range 0 O a I Display range mem test load file save file E 128 Bytes 1024 Bytes X1030_66_040908 Figure 66 PCltree Memory Test Results The ML505 memory written read is the BRAM and or DDR2 defined in the system mhs and addressed with the PLBv46 Endpoint Bridge C_PCIBAR2IPIFBAR_ generics In this reference system two PLBv46 Endpoint Bridge BARs are active The C_PCIBAR2IPIFBAR_0O generic points to the ML505 BRAM located at Ox8AE10000 After writing the ML505 BRAM using PCI tree Edit Memory XMD can be used to verify BRAM or DDR2 if the BAR is enabled from the PLBv46 side XAPP1030 v1 0 1 May 6 2008 www xilinx com 60 PCltree Testing gt XILINX Figure 67 shows XMD verification that the XPS BRAM contains the data written by PCltree using XMD commands ec Command Prompt xmd lof x 85062014 alallala 85C02018 98080008 85C8201C 015 51515 51515 AMD mrd x8AE16086 8 615 15 15 51515 alallala 98080008 5 515 515 515 5 alallala 68680008 15 51515 515 5 015 5 5 15 5 5 5 RMD mrd G x8AE16086 8 SAE1 6006 64080008 SAE16004 FFFFFFFF SAE1 608 02000000
46. alyst Enterprises Inc SPX Analyzer Exerciser Software wr_rd_x1 File Edit View Configuration Tools Project Setup Window Help D Saaga a BDSP p nA ACTROS Exerciser Program Capture Trigger On Link Settings Settings Global Loop 1 HEH 33 x NoLoop C Count Continuous sai me 0 MEM WRITE 64 4DWheader with data 020 Req ID H Last DW BE H First DW BE H Addr H Data H RandomData M Seq H Fmt H EEE eo OH MEM READ 64 4DWheader nodata 001 o Last DW BE H First DW BE H Addr H Analyzer 1 000E85000159 US6 Triggered Ex X1030_18_040908 Figure 18 wr_rd_x1 TLP stimuli XAPP1030 v1 0 1 May 6 2008 www xilinx com 17 Catalyst Testing XILINX Figure 19 shows the results after running a version of wr_rd_x1 sdc in which a random pattern of OxOAADC5B9F1B0DC839 is transmitted Catalyst Enterprises Inc SPX Analyzer Exerciser Software wr_rd_x1 Bl File Edit View Configuration Tools Filtering Report Window Help D Suaa l g See B 8 PL a AA El Sle yeaS wu T Ae z1 AK ro SKIP Tez Len H Req ID H Tag H Last DW BE H a 00060000000 Data 128 Bytes H Len H Req ID H Tag H Last DW BE H Data 0 Bytes H Comp ID H Status X1030_19_040908 Figure 19 Catalyst wr_rd_x1 Results Figure 20 shows the use of XMD to read ML505 DDR2
47. an be efficiently presented in a Display Viewer The figure shows the content of a single Configuration Read TLP Reserved 1 Sequence N Packet Type Config Read TO Framing Symboll FB QO umber 000 u Format Type 04 TC 0 Reserved 4 T SO EP QO Attribute Reserved 5 Length Reserved 2 7 Requester _ QO 0 n Reserved 3 o o 001 ID 0000 Tag 00 Last_DW_BE Bus Number NO E RF First _DW_ BE 00 Function N Reserved 6 Reserved 7 TLP Digest LCRC Loop Count Iterate Af Delay Coun Trigger So ZData Trigger Ou Trigger Ou Register Address Disparity 10000000000000000001 Symbol View Device Number 00 umber 0 QO 000 QO AVN 2AC19647 Framing Symbol2 FD Loop Type No Loop wn ter Trigger No bs MO urce cution Immediate Ex Error No Collapse tput No tput_Type Pulse Global Loop X1030_16_040908 Figure 16 sdc_example XAPP1030 v1 0 1 May 6 2008 xilinx com www 16 Catalyst Testing XILINX As Root Complex the Catalyst Exerciser performs memory writes and memory reads to the ML505 memory The ML505 reference design contains XPS BRAM and a Multiport
48. and d 40 causes the values of 40 current memory locations to be displayed The values displayed 00000000 FFFFFFFF 00000002 FFFFFFFD arethe same as the values displayed by PCltree in Figure 23 because this test was run shortly after the PCltree tests The location command 10 moves the address to location OxOO000000 All addresses are offset addresses from the BAR start address The set command s 12345678 is a memory write to the current address In the figure after the write of 0x12345678 the address pointer is move back to location 0x00000000 I 0 and the contents of the memory is re displayed using d 40 The 0x12345678 value just written at location 0x00000000 is displayed amp Command Prompt met log 505 log of x H met gt met log 505 log MET v2 4 s request Target report OK 6x1 Vendor 6x1iGee MemAddress 6xe 8600008 MemS ize 6x1 6606 IloAddress 6x Io ize 6x IsPciExpress TRUE Sorry will not overwrite logfile Interactive mode MEM 32 Hex G6666666 gt d 46 POHOHOHA 864c1254 b616a964 BEHHHHHH 44616266 H6G60616 66681666 66168964 666168604 466166068 OG660826 8256821f 2aiaah24 AHOHOHO 166160606 H6666636 86666412 64666661 HAHAHHOHO HANAHANA MEM 32 Hex 66666646 gt 1 AxA MEM 32 Hex G6666666 gt s 12345678 MEM 32 Hex 6006606 gt 1 AxA MEM 32 Hex G66666606 gt d 46 PAONAHAHA 12345678 b616a964 HAHAHAHA 66616266 H6666616 86681666 66168964 66616864 66616666 HEG60826 B256821f 2al
49. ation Hardware verified Yes Hardware platform used for verification ML505 XAPP1030 v1 0 1 May 6 2008 www xilinx com 74 References References Revision History Notice of Disclaimer XILINX 1 UG197 Virtex 5 Integrated Endpoint Block for PCI Express Designs User Guide 2 UG201 Virtex 5 FPGA ML555 Development Kit for PCI and PCI Express Designs User Guide v1 4 March 10 2008 3 XAPP1022 Using the Memory Endpoint Driver MET with the Programmed Input Output Example Design for PCI Express Endpoint Cores 4 LeCroy PCI Express Multi Lane Exerciser User Manual Version 5 0 SpekChek User Manual Version 6 5 Catalyst PCI Express Bus Protocol Analyzer Exerciser User s Guide The following table shows the revision history for this document Date Version Revision 04 25 08 1 0 Initial release 5 6 08 1 0 1 Made minor non technical edits Xilinx is disclosing this Application Note to you AS IS with no warranty of any kind This Application Note is one possible implementation of this feature application or standard and is subject to change without further notice from Xilinx You are responsible for obtaining any rights you may require in connection with your use or implementation of this Application Note XILINX MAKES NO REPRESENTATIONS OR WARRANTIES WHETHER EXPRESS OR IMPLIED STATUTORY OR OTHERWISE INCLUDING WITHOUT LIMITATION IMPLIED WARRANTIES OF MERCHANTABILITY NONINFRINGE
50. ction dS LeCroy PETracer TM PCI Express Protocol Analyzer C Program Files CATC PETracer data1157 pex IE Fie Setup Record Generate Report Search view Tools Window Help v ww je 2m 3 w RaT ERRE 000 00 0 00000000 TO chs et aah Ho CD 1st BE Last BE 1111 0000 12345678 000 00 0 00000004 1st BE Last BE 1111 0000 12345678 b Mem 000 00 0 00000008 00000000C 1111 0000 1st BE 1111 0000 ist BE Last BE fj Last BE 12345678 J T Mem 10 00000 00000010 1st BE Last BE 1111 0000 Data 12345678 R x1 R x1 D w g w Mem Muner 10 00000 Mem 10 00000 000 00 0 000 00 0 000 00 0 00000014 00000018 1st BE Last BE 1111 0000 1st BE 1111 0000 Last BE fj Data 12345678 12345678 Ro x1 a lt Par m z a mea b a MVVr 32 10 00000 0 0 0 0 0 0 0 000 00 0 0000001C 1st BE Last BE 1111 0000 1 dword PETrainer ML SN 1102 n 5653 Ready Link State s w Complete ___InitFC State _ gt Figure 54 EP to RC Write Results 8 DW Search Fwd X1030_54_040908 XAPP1030 v1 0 1 May 6 2008 www xilinx com 48 Testing with a PC Testing with a PC XILINX Using a Personal Computer PC as Root Complex is
51. d the Trigger Position is set at 90 post triggering The x1 Lane Width is selected Recording Options General Recording Rules Recording Type Snapshot Manual Trigger Event Trigger r Buffer Size 32 000 MB Misc l Beep When Trigger Occurs Save External Interface Signals V Preserve TC to VC mapping across the recordings Default TC to VC mapping Trace Filename amp Path C Program Files CATC PE Tracer data pex Switch to Simple Mode Target Analyzer PETracer Edge PETracer ML PETracer ML 2 units PETracer EML PETracer Summit Trigger Position 90 post triggering I Save As Multisegment Trace mee Options Name Default Save Save As Default Load Browse Link x1 x2 x4 x8 oe C Port 1 of PETracer ML SN 1157 Inhibit Channel J Reverse Lanes Invert Polarity e a A e e oe ore Fone Papas BIE AES GS ae lis iis Port 2 of PETracer ML SN 1157 Inhibit Channel J Reverse Lanes Invert Polarity OF 1 2 B45 8 Fe fe file ae ga ete ie Me hs Use External Reference Clack I Disable Descrambling IV Auto Configure Lane Polarity F Swap Recording Channels I Base Spec Rev 1 0 Compatibility Mode X1030_36_040908 Figure 36 Setting Recording Options XAPP1030 v1 0 1 May 6 2008 www xilinx com 34 LeCroy Testing XILINX Figure 37 shows using File gt Open to open a LeCroy stimuli peg file The LeCroy PETracer software provi
52. des the interface to the PETracer Analyzer and PETrainer Exerciser To run an analysis click on the Record icon the Sun in the menu bar Click the Traffic Light icon at the bottom left of the GUI After the status bar indicates Traffic Finished click the Stop icon black filled square next to the Sun This causes results to be shown in the Display area Results files have a pex extension Like peg files pex files can be opened using File gt Open Ya LeCroy PETracer TM PCI Express Protocol Analyzer Eile Setup Record Generate View Tools Help e HE gt HE o nt w p Look in gt lecroy ex 1DW_completion drp_wait_mwr32 2DW_MRd ep2rc_1 Elow mrd Blep2rc_mrd32_1dw 4k_out_of_order_cpld pte E16 Dw _Mrde4 Blep2rc_wait_mwr32 Bels2bit_dma_read_write_burst Blep_to_re 1_DW_Mrde cfg_rc2ep_rd_wr ep_to_rc_2 ep_to_rc_4 cfg_x4 ep_to_rc_6 Elcompleter abort Bl Ic_rcZep_wr_rd config i ET My Documents Elconfig_3dw flm32_dma_BARn_to_BRAM_burst_RCB BElconfiguration_Only m32_dma_BARn_to_BRAM_single Beldrp_re_to_ s4 m64_rdwr_singles_bursts a x My Network File name Jan of Places Files of type All PE Tracer Files pex pem peraw peg PETrainer ML SN 1102 Link State InitFC State 19 318 BRHF g S S etecrauet Not initialized Ready Figure 37 Opening a LeCroy peg file X1030_37_040908 XAPP1030 v1 0 1 May
53. ed Memory Controller MPMC interface to DDR2 Figure 17 shows the memory addressing for Root Complex Catalyst to Endpoint ML505 transactions The memory addressed is controlled by the BAR value written and by the C_PCIBAR2IPIFBAR_ generic s In the reference design BARO is written as Ox0000000060000000 C_PCIBAR2IPIFBAR_0 addresses XPS BRAM at 0x8AE10000 and C_PCIBAR2IPIFBAR_1 addresses DDR2 at location 0x90000000 ML505 XPS BRAM PLBv46 PCle ee Catalyst PCle x1 C_PCIBAR2IPIFBARO OX8AE10000 C_PCIBAR2IPIFBAR1 0X90000000 MPMC DDR2 Root Complex 0x90000000 End Point X1030_17_040908 Figure 17 Catalyst Root Complex Figure 18 shows the write then read TLPs in the wr_rd_x1 sdc file In the figure Packet 0 is a MWr64 to address 0x0000000060000000 of 128 bytes The Data Field allows the user to specify data as Upcount Walking Bit or Random pattern or a user defined pattern such as 0x12345678 can be entered As exercises in learning to use the PLBv46 Endpoint Bridge the data can be varied and the memory written read can be changed from XPS BRAM to DDR2 The Length field is O20H which is 32 doublewords DWs or 128 bytes Packet 1 is a MRd64 of address Ox0000000060000000 used to verify the written data The MRd64 TLP address endianess differs from the CfgWr address endiness used when the BAR was written with a CfgWr in Figure 14 Bit Order and Endianess can be defined by right clicking a field to invoke a pop up menu Cat
54. elect File Open Database to open the vcd file in the Cadence Design System Inc Simvision design tool After running ChipScope it is sometimes necessary to revise the Trigger or Data nets or both used in a debug operation Saving Inserter and Analyzer projects simplifies this procedure The saved project can be re opened in Inserter and edits can be made XAPP1030 v1 0 1 May 6 2008 com www xilinx 72 Using ChipScope with the PLBv46 Endpoint Bridge XILINX Figure 78 is the waveform output of a ChipScope inserted into the reference system when running the endpoint to root complex performance tests Memory XPS Central DMA and PLBv46 Endpoint Bridge transactions are monitored simultaneously The trigger is PCle_bridge comp_slave_bridge sig_request_complete The ml505_mb_plbv46_pcie_scs cdc is included in the chipscope directory S ChipScope Pro Analyzer 1_ep2rc_mpmc_2000 File View JTAG Chain Device TriggerSetup Waveform Window Help re m T Q S Project 1_ep2rc_mpmc_2000 JTAG Chain DEV 0 MyDeviced KC5VLX50T System Monitor Console g UNIT O MyILAO ILA Trigger Setup Waveform Listing Bus Plot Signals DEV 0 UNIT 0 CH 02 IXpS CENTA UNTA UN WG CH 63 kps_central_dma_O M_wrDB gt CH 64 kps_central_dma_O M_wrD CH 65 kps_central_drma_O M_wrD CH 66 xps_central_dma_O M_wrD CH 67 kps_central_dma_O M_wrD CH 68 kps_central_dma_O M_wrD CH 69 kps_central_dma_O M_wrD CH 70 xps
55. erSetup orm Window Help el Tait Project mI505_mb_plbv46_ SER Match Unit Function Counter C MO TriggerPortd El Trigger Setup DEV 4 MyDevice4 XC5VLX50T UNIT 0 MyiLAO ILA disabled Ly iSl_raDAck Ly M_RNW C M request PLB_MRdDAck Ly Sl_addrAck LY PLB_MAddrAck Ly PLB_MWrDAck C PLB_PAValid add Del Type Windows 1 Depth 1024 w Position 100 All Data Active Trigger Condition Name Trigger Condition Equation TriggerConditionO MO Storage Qualification av TET z YEIFA INFO Found 1 Core Unit in the JTAG device Chain COMMAND save project C Xilinx ChipScope Pro 9 2i binintiml505 mb_plbv46_pcie cpj Writing project file CXilin4ChipScope_Pro_9_2ibinintimi505_mb_plby46_pcie cpj Figure 75 ChipScope Analyzer Trigger Setup DONE X1030_75_040908 XAPP1030 v1 0 1 May 6 2008 www xilinx com 70 Using ChipScope with the PLBv46 Endpoint Bridge lt XILINX 12 Arm the trigger by selecting Trigger Setup gt Arm or clicking on the Arm icon as shown in Figure 76 a t Project mi505_mb_plbv46_ DEV 2 MyDevice2 XC950 DEV 3 MyDevice3 Syste 9 DEV 4 MyDeviced CAV System Monitor Cons UNIT 0 MyILAO ILA i ChipScope Pro Analyzer ml505_mb_plbv46_pcie File View JTAG Chain Device TriggerSetup Waveform Window Help
56. ettings r Global Loop NoLoop Count E C Continuous ReaD H Last OW BE CONFGWRTETO ooi OOO ON Req ID H Last DW BE H First DWV BE H Bus Num H Dev Num H Func Num H Reg Num H oo o E Req ID H Last DW BE H 0000 First DY BE H Bus Num H Dev Num H Func Num H Reg Num H Ooo o oo E J Define different pattems for pre trigger and post trigger data captures Easy switch to Advanced mode Apply Settings Run Analyzer 1 000E85000159 LISB Triggered Exerciser Sti X1030_14_040908 Figure 14 Catalyst Configuration Stimuli XAPP1030 v1 0 1 May 6 2008 www xilinx com 14 Catalyst Testing XILINX Figure 15 shows the Analyzer output after running cfg_x1 The results are contained in the cfg_x1 ssf file Registers in the Configuration Space Header are displayed in packet 0 using Vendor ID and Device ID symbolic names with Xilinx 0x10EE and 0x0505 values The Command Status Register is read The SC in the status field indicates successful completion of the transaction In the figure the Revision ID and Class Code Register field is expanded to provide a readable table of the values in the Data field u x 3 Fie Edit View Configuration Tools Filtering Report Window Help a O s28S mja ler lt is gt m B PLS eA BE BBP OH e SOR Oi B 8 amp s e VaR A elma else zm ox Co if om E
57. figure templates are defined for Configuration Write and Configuration Read TLPs The Configuration Write template is called in the repeat loop to write FFFFFFFFs to the six Configuration Space Header BARs The peg files inm1505 mb plbv46 pcie lecroy can be used to test the PLBv46 Endpoint Bridge on the ML505 template TLP Name MyCfgWrite Template name TlpType CfgWr0 Write device Configuration Space FirstDwBe OxF First DW Byte Length 1 1 DWORD Payload template TLP Enables OxFFFFFFFF Name MyCfgRead Template name TlpType CfgRd0 Read device Configuration Space FirstDwBe Length 1 1 DWORD OxF Enumerate all 6 Bas repeat Begin Write OxFFFFFFFF into Bas MyCfgWrite 0x10 packet Register First DW Byte Enables Address Count 6 Counter registers i i 4 Wait for completion received TLP TLPType wait Read Bas Cpl Register Address register packet MyCfgRead 0x10 i 4 Wait for completion received TLP TLPType wait repeat E Cp1D Address register X1030_45_040908 Figure 45 peg Example XAPP1030 v1 0 1 May 6 2008 www xilinx com 43 LeCroy Testing XAPP1030 v1 0 1 May 6 2008 www xilinx com lt XILINX Endpoint to Root Complex Transactions In Endpoint to Root C
58. in pcie_dma c The XMD scripts and C code generate DMA operations to transfer data between different ML505 and Catalyst memory regions DMA transactions are generated by writing to the Control Source Address Destination Address and Length registers of the DMA controller Table 3 provides the register locations for the XPS Central DMA In the reference design C_BASEADDR is set to 0x80200000 Table 3 XPS Central DMA Registers DMA Register Address Control Register C_BASEADDR 0x04 Source Address Register C_BASEADDR 0x08 Destination Address Register C_BASEADDR 0x0C Length Register C_BASEADDR 0x10 XAPP1030 v1 0 1 May 6 2008 www xilinx com 7 Endpoint to Root Complex Transactions gt XILINX The pcie_dma c code consists of the four functions in the functional diagram in Figure 7 The Barberpole Region function provides a rotating data pattern in the memory located at the source address The Zero Region function sets the memory located at the destination address to all zeroes The DMA Region function generates a DMA transaction of data located at the source address to the memory at the destination address Following the DMA transfer the Verify function verifies that data at the source and destination address are equal Barberpole Region Zero Region DMA Region Verify X1030_07_040908 Figure 7 Functional diagram of pcie_dma c Figure 8 show the communication terminal output when running
59. iser Software per_ep2rc_x1 File Edit View 0 sc58S One Configuration Tools Project Setup Window Help gt Em B G 2N tn A AELK OP Link Chart Average Payload uje e J Data Throughput19 0 MB Sec J Payload Throughput 3 MB Sec oor J fv2129 Payload a0 05 Bytes per_ep2rc_x1 Exerciser Program Performance Items Link Settings Setting Items Group r Report Direction D1 Link Usage Link Utilization Cink fAaereaata O Transmission Efficiency F Upstream Transmit Throughput Average Payload C Number of Packets C Latency F Downstream Receive Exerciser Stop Analyzer Run X1030_31_040908 Analyzer 1 000E85000159 U5B Figure 31 EP to RC Performance Test Using XMD XAPP1030 v1 0 1 May 6 2008 www xilinx com 29 Endpoint to Root Complex Transactions XILINX Figure 32 shows the performance of an Endpoint to Root Complex transaction using C code pcie_dma_0 c to generate stimuli with the length 200 The data throughput is 61 8 MB s and the payload throughput is 36 8 MB s In this test the Source Address is XPS BRAM which is 0x8AE10000 and the Destination Address is 0x20000000 which translates to Catalyst memory across the PCle link Catalyst Enterprises Inc SPX Analyzer Exerciser Software per_ep2rc_x1 File Edit View Configuration Tools Project Setup vindow Help 0 2408 H amp gt m m B2 2e WRB OK OR e VOee B
60. iser used to analyze and or exercise PCI Express data transactions The SPX4 Analyzer consists of the SPX4 card and Analyzer software The Analyzer allows capture and trigger on Transaction and Data Link Layer Packets Physical Layer Ordered Sets and all bus conditions The Exerciser generates bus traffic while operating as either a root complex or endpoint device PXP 100a PCI Express DVT Platform Figure 9 Catalyst Test Setup www xilinx com X1030_09_040908 Catalyst Testing XILINX Figure 10 is a photograph of the Catalyst setup The ML505 is inserted into the PCle slot The Platform Cable USB cable is connected to the ML505 to use Impact XMD and GDB A USB cable connects the PC based Catalyst software to the SPX4 Analyzer ory Tinh some jae i ar XP 100 PCI Express DVT Platform wwwig tcatalyst com XILINX Platform Cable USE X1030_10_040908 Figure 10 Photo of Catalyst PCI Express Test Equipment In addition to using the Catalyst Bus Protocol Analyzer Exerciser software as discussed extensively in this application note the Catalyst SoekChekTM PCI Express Compliance Suite has been run with this reference design to verify that the PLBv46 Endpoint Bridge meets PCI SIG compliance tests The SpekChek tests are defined in the SpekChek User Manual Version 6 5 After downloading the bit file into the ML505 FPGA using Impact the PLBv46 Endpoint Bridge Bridge Control Register BCR is written as shown in Fig
61. lues are displayed in Big Endian format EE100505 The address value at offset 10 contains the addressing size and prefetchability fields MSR clr set Instruction Support on Compare Instruction Support Connected to MicroBlaze mdm target id 8 Starting GDB server for mdm target Cid gt at TCP port no 1234 Note mbconnect command is Deprecated Use connect mb command RMDZz rst System reset successfully RMD mur x85c Aie Ox863f6107 XMDz mrd x85cO61e8 1 85C 61 EG 663 F167 AMD mrd x85c82086 8 EE100505 666614608 66668405 5 5 151515 515 5 6CHBH808 6 5 151515 515 5 615 5 15 51515 85C6201C 15 5 5 5 5 5 5 AMDz X1030_61_040908 Figure 61 XMD Read of PLBv46 Endpoint Bridge Registers XAPP1030 v1 0 1 May 6 2008 www xilinx com 55 PCltree Testing XILINX After invoking PCltree and running a scan Figure 62 shows the ML505 PLBv46 Endpoint Bridge detected as Other Memory Controller with Bus Number 3 Device Number 0 Function Number 0 or BDF 3 0 0 The Xilinx Vendor ID and Device ID are displayed In its Configuration Space Header BARO has a value of 0x0000000C The address 0xE000000 in this example varies for different BIOS setups Pcilree direct select show INT routing bus dev func g ed 3 0 per 0 show Mem Map 320 0 host CPU Other Memory Controller 0 00 0 Host PCI Bridge Ij GID xl0EE Xilinx Corp 0 01 0 O gt 1 l PCI PCI B
62. me of the output ssf file Catalyst Enterprises Inc SPX Analyzer Exerciser Software cfg_x1 File Edit View Configuration Tools Project Setup Window Help D saaa 2 uw ea e F eg B E E t n k AE gt E Exerciser Program cga Trigger On Link Settings Settings Trace Memory Status Trig Point PositionInMemoy Lijit tt ttt ttt sy ig 1 99 Entire Memory Up to 512 MB Output File Name H catalyst cfg_x1 ssf Browse Partial Memory I Auto Run Note cfg_x1 sde Project Name Note Creation Date 16 09 38 on Thursday Jun 14 07 Last Modified 15 43 17 on Tuesday Feb 26 08 Last Run 15 41 14 on Tuesday Feb 26 08 I Define different patterns for pre trigger and post trigger data captures DROP e 4 Project Tree a Capture Everything Exclude Idles S ie Trigger On P TLP Any Type Fmt Any Format Type Any Type Requester ID Xxx H Traffic Class Any TC Payload XXX H Start at Xx H Address XXXXXKXKKKKKK H Enhanced XO Direction Any Direction a 4 Settings Trig Position in Memory 9 Capture Memory Space 100 KB Output Sample File H catalyst ctg_x1 ssf Analyzer Simulation Mode lt il Not Triggered 74 X1030_12_040908 Analyzer Simulation Mode Figure 12 Capture Settings XAPP1030 v1 0 1 May 6 2008 www xilinx com 12 Catalyst Testing XILINX Figure 13 shows the setup of
63. memory to provide a second verification that the wr_rd_x1 sdc script functioned as intended The data read in XMD should be the same as the data in the Analyzer waveform display c Command Prompt xmd Hard Divider Support Hard Multiplier Support Barrel Shifter Support MSR clr set Instruction Support Compare Instruction Support Connected to MicroBlaze mdm target id Starting GDB server for mdm target Cid Deprecated Note mbconnect command is AMD rst System reset successfully SMD mur x85c801e0 4x863F6167 RMD mrd x8AE16000 8 SAE1 6606 9AADCSBA SAE1 6604 FiB DC8 SAE1 G08 DSFBB 787 SAE1G 0C F2F4BFAC SAE1 6616 B2BEAFBE 96EDB1iAS B8AS8483 8DE6 DAC4 a gt at TCP port no 1234 Use connect mb command X1030_20_040908 Figure 20 Verifying Root Complex to Endpoint Transactions with XMD XAPP1030 v1 0 1 May 6 2008 www xilinx com 18 Using Catalyst to test PCle Performance XILINX Using Catalyst Catalyst is used for performance testing This section provides performance tests for Root to test PCle Complex to Endpoint transactions first for read transactions and then for write transactions The test setup is defined and then performance results are given for various lengths for 32 and Performance 64 bit transactions Figure 21 shows the physical link setup for the performance test For the ml505_mb_plbv46_pcie project change the Physical Layer Se
64. neration Options The LeCroy ML test equipment is selected Link Width is specified as x1 Select Host as the Interposer Ya LeCroy PETracer TM PCI Express Protocol Analyzer Ele Setup Record Generate View Tools Help S OE WH o t 4 pi Generation Options General Link Integrity Flow Control Transactions m Target r Link Width PETrainer ML xl C PETrainer EML G C Summit 22 16 Tii Rx T Disable Scrambling I Disable Descrambling I Reverse Lanes I Reverse Lanes Invert Polarity Invert Polarity Et PA SE Ja uie PRE I0 A2 ji M ea Ja ate er Sie ie mE Skew 2 ae eS be ae a PRPPPPEE en 8 9 10 11 12 13 14 15 c PEPEPEPE ube r Analyzer control JT Automatically detect Link Configuration I Start recording when generation starts I Base Spec Rev 1 0 Compatibility Mode I Stop recording when generation stops I Use External Reference Clock Save SaveAsDefaut Load Cancel PE Trainer ML SN 1102 Link State IniFC State S nDl S S SSF deter duet Not initialized Ready 4 X1030_35_040908 Figure 35 Setting Generation Options XAPP1030 v1 0 1 May 6 2008 www xilinx com 33 LeCroy Testing XILINX Figure 36 shows the menu for setting Recording Options after selecting Setup gt Recording Options The Simple Mode is used An Event Trigger is selected The Buffer Size is specified as 32 MB an
65. ols Help By Veil o m t 4 Look in fe lecroy gt e Ri ow _Mrd 4 Beldrp_weait_cpld 1D W_completion drp_wait_mwr32 MyRecent l2ow_mrd ep2rc_t Documents Elow mrd lepzrc_mrd32_1dw B Fl 4k_out_of_order_cpld pe el i6_pw_mrde ep2rc_wait_mwr32 Desktop Bels2bit_dma_read_write_burst Blep_to_re Belefa_rc2ep_rd_wr Blep_to_rc_2 ara cfg_x4 ep_to_rc_6 Belcompleter abort Belic_re2ep_wr_rd z El config m32_dma_BARn_to_BRAM_burst l config_adw m32_dma_BARn_to_BRAM_burst_RCB BElconfiguration_Only lm32_dma_BARn_to_BRAM_single Eldrp_rc_to_ep EElm 4_rdwr_singles_bursts My Documents My Computer GR K My Network File name Places Files of type PETrainer ML SN 1102 Link State InitFC State 35h jus gaat S E Detect Quiet Not initialed Ready Search Fwd X1030_33_040908 Figure 33 LeCroy Test Setup XAPP1030 v1 0 1 May 6 2008 www xilinx com 31 LeCroy Testing XILINX Figure 34 is a photograph of the LeCroy test setup The ML505 is inserted into the LeCroy Host Emulator The Platform Cable USB Programming cable is connected to the ML505 JTAG connector Kaocel Ok Taam CATE Probo Armali SHS bem X1030_34_040908 Figure 34 LeCroy Test Equipment XAPP1030 v1 0 1 May 6 2008 www xilinx com 32 LeCroy Testing XILINX Figure 35 shows the menu for setting Generation Options after selecting Setup gt Ge
66. omplex transactions the read and write operations originate from the ML505 and target the LeCroy The LeCroy model used in this application note the PCI Express Multi Lane ML Exerciser Analyzer does not have target memory For read operations the peg files are written to respond with read data Invoke PETracer and run File Open lecroy ep2rc_mrd32_1dw Endpoint to Root Complex transactions are generated with XMD commands or C code Since the MWr and MRd TLPs originate from the ML505 the LeCroy peg files cause the LeCroy to wait for the TLP s from the ML505 Figure 46 shows the peg for the EP to RC MRd32 The LeCroy waits for the MRd32 packet from the ML505 When the MRd32 packet is received the LeCroy returns a Completion with Data CpID packet with a 0x12345678 payload wait TLP TLPType MRd32 Packet TLP TLPType CpID CompleterID 0 1 0 Length 1 ByteCount 0 LowerAddr 0x00 Payload 0x12345678 X1030_46_040908 Figure 46 ep2re_mrd32 Figure 47 defines the functionality of the LeCroy Root Complex when receiving a MRd32 transaction from the PLBv46 Endpoint Bridge endpoint on the ML505 gt LeCroy PETracer TM PCI Express Protocol Analyzer H lecroy ep2rc_mrd32_1dw peg E Eile Setup Record Generate Report Search View Tools Window Help SHE PRM o n A BAA E ORR 2 Mw Time Stamp TA 3DW header no data MRa Mvvr 0000 000 000 000 s EEE 25 TLP CompleterlD bal o 10 01010 000 00 0 O 000 01 0
67. oot Complex to Endpoint Read Operations Figure 22 shows the opening of the rc2ep_rd64 performance project Performance projects use the spf extension Catalyst Enterprises Inc SPX Analyzer Exerciser Software File View Configuration Tools Project Setup Help O c586 u a Look in ja catalyst ce B ey perf_ep2rc ey PerformanceAnalyzer1 icl rc2ep_rd64_10 PerformanceAnalyzer2 Ith rc2ep_wr isl Performanceanalyzer_x1_ep2rc ll re2ep_wre4 PerformanceAnalyzer_x4 el re2ep_wre4_10 ey rcZep_rd32 gj ae Y File name rc2ep_rd64 Files of type Performancesnalyzer Files spf z Cancel A X1030_22_040908 Figure 22 Opening a Catalyst Performance Test The four tabs used in performance projects are the Exercise Program Performance Items Link Settings and Settings In Performance Items the type of performance tests run are defined The PCle traffic used in the performance measurement is defined in the Exercise Program XAPP1030 v1 0 1 May 6 2008 www xilinx com 20 Using Catalyst to test PCle Performance XILINX Figure 23 shows a single TLP used in the performance measurements of Rd64 transactions of length 003 Click the TLP button below Performance Items to add the TLP to the Exercise Program Using the pop up menu select Memory gt Read Request 64 bits Fill out the address and Len fields Select the Continuous radio button so that the TLP is continuously transmitted Catalyst
68. r DID x0505 no device name found no 1 00 0 VGA PC Compati SubVID x0000 0 27 0 o Multimedia 80t SubID x00000 0 28 0 O gt Z 2 PCI PCI Br rev x00 0 28 4 0 gt 3 3 PCI PCI Br edit ConfReg Nr of ConfRegs Other Memory l hex 16 C 64 O gt 4 4 PCI PCI Br i SE Ethernet Netwe use BIOS int Universal Host Cor Write ConfReg reire O Universal Host Cor dump Universal Host Cor refr after ur Universal Host Cor Config Space Dump type 1 xs 2 DID VID Stat Cmd PCI ISA Bridge De BaseClass SubClass I o Mass Storage Ci BIST Header LatTimer o Mass Storage Cc BAR mem pref 64bi SMBus Serial Bus BAR BAR BAR BAR BAR Cardbus CIS Ptr SubID SubVendorID Exp _ROM_BAR reserved reserved maxLat minGnt IntPir o serial bus Dev O gt S 5S Subtractive 0 1 2 3 4 5 rescan write to reset X1030_62_040908 Figure 62 PCltree Scan To edit the registers in the Configuration Space Header CSH highlight the register in the CSH to edit and provide a value in the Edit Config Reg dialog box As an example select the Command Status Register write xFFFFFFFF in the Edit Config Reg dialog box click WriteConfReg and then click Refresh Dump to see the new value of the Command Status Register CSR displayed The CSR value is not OxFFFFFFFF because some bits in the CSR are reserved XAPP1030 v1 0 1 May 6 2008 www xilinx com 56 PCltree Testing XILINX
69. r View icon Pcilree BAR space oooo0000 oooo00000 ooooo000 ooooo0000 ooooo000 ooooo000 oooo0000 ooooo0000 00000000 oooo0000 ooooo000 oooo00000 00000000 o0o0o00000 oooo0000 ooooo000 oooo0000 ooooo0000 ooooo000 ooooo0000 ooooo000 ooooo000 oooo0000 ooooo000 00000000 oooo0000 ooooo000 oooo00000 ooooo000 00000000 oooo0000 ooooo000 ooooo0000 lt x00000000 gt x00000004 gt lt x00000008 gt x0000000C gt lt x00000010 gt lt x00000014 gt x00000018 gt lt x0000001C gt lt x000000Z20 gt lt x00000024 gt lt x000000Z28 gt x0000002C gt lt x00000030 gt x00000034 gt x00000038 gt lt x0000003C gt lt x00000040 gt x00000044 gt lt x00000048 gt x0000004C gt lt x00000050 gt x000000S84 gt x000000S8 gt lt x000000SC gt x00000060 gt x00000064 gt lt x00000068 gt x0000006C gt lt x00000070 gt x00000074 gt lt x00000078 gt lt x0000007C gt x00000080 gt amp save tile Iv auto read memory Memory Space typezZ prefetchable 20000000 ffffoo000 base range 64 KByte 64bit BARs not supported yet 1 medit memory Data Write Memory toggle loop on off M count I verify refresh view after write refr view mem copy source destination men copy select view range KB range 0 63 0 xil MB range 0 QO ea Display range 128 Bytes
70. r ngo In the m1505_ mb plbv46_pcie implementation directory copy pcie bridge wrapper ngo to pcie bridge wrapper ngc 8 In XPS run Hardware Generate Bitstream and Device Configuration Download Bitstream Do not rerun Hardware Generate Netlist as this overwrites the implementation pcie bridge wrapper ngc produced by the step above Verify that the file size of the pcie_bridge_wrapper ngc with the inserted core is significantly larger than the original version 9 Invoke ChipScope Pro Analyzer by selecting Start Programs gt ChipScope Pro gt ChipScope Pro Analyzer Click on the Chain icon located at the top left of Analyzer s GUI Verify that the message in the transcript window indicates that an ICON is found XAPP1030 v1 0 1 May 6 2008 www xilinx com 68 Using ChipScope with the PLBv46 Endpoint Bridge lt XILINX 10 The ChipScope Analyzer waveform viewer displays signals named DATA To replace the DATA signal names with the familiar signal names specified in ChipScope Inserter select File Import and browse to plbv46_pcie cdc in the dialog box The Analyzer waveform viewer is more readable when buses rather than discrete signals are displayed Select the SI_rdDBus lt gt signals click the right mouse button and select Add to Bus New Bus With SI_rdDBus in the waveform viewer select and delete the discrete SI_rdDBus lt gt signals The signals are displayed as buses in Figure 74
71. r with the Programmed Input Output PIO Example Design for PCI Express Endpoint Cores provides instructions on setting up and running the MET XAPP1022 uses the PCle Endpoint Block Plus core driven by the PIO interface This section uses MET to write and read ML505 memory using the PLBv46 Endpoint Bridge Pages 6 11 of XAPP1022 provide instructions for installing the Xilinx Virtex 5 PCle Endpoint Driver XAPP1030 v1 0 1 May 6 2008 www xilinx com 63 Memory Endpoint Test XAPP1030 v1 0 1 May 6 2008 XILINX Figure 70 shows the invocation of the Memory Endpoint Test The values for the Device Number Vendor Number and the address indicate that the PLBv46 Endpoint Bridge on the ML505 is detected e Command Prompt met H designs n1565_mb_plbv46_pcie gt cd met H met gt met MET v2 4 R Status request ital Target report MemAddress O xe 8000000 Mem ize 6x1 6666 IoAddress 6xG6 IoS ize 6x IsPciExpress TRUE Interactive mode MEM 32 Hex 666666000 gt X1030_70_040908 Figure 70 Invoking the Memory Endpoint Test Pages 11 15 of XAPP1022 provide detailed instructions on using the MET to test transfers to PLBv46 Endpoint Bridge memory www xilinx com 64 Memory Endpoint Test XILINX Figure 71 shows basic read and write operations using the MET In the figure the Display d Location I and Set s instructions illustrate basic memory read and write transactions The comm
72. ridge which do not require LeCroy or Catalyst test equipment are the PCIE Configuration Verification PCIE CV PCltree and the Memory EndPoint Test MET tests These are run using the ml505_mb_plbv46_pcie project These tests are quick to setup and costs nothing other than a PC with PCle slots For these tests the 2008 Xilinx Inc All rights reserved XILINX the Xilinx logo and other designated brands included herein are trademarks of Xilinx Inc All other trademarks are the property of their respective owners XAPP1030 v1 0 1 May 6 2008 www xilinx com 1 XILINX Hardware and Software Requirements ml505 Embedded Development Platform is inserted into the x1 PCle slot of a PC Dell 390 The PC based PCltree and or MET software are installed The PCltree Bus Viewer www pcitree de and the Xilinx MET tests allow the user to write and read ML505 memory with any pattern with different lengths PCltree and the MET do not provide the capability to analyze PCle traffic Hardware and The hardware and software requirements for this reference system are Software e Xilinx ML505 Rev A board Requirements e Xilinx Platform USB or Parallel IV programming cable e Serial communication cable and serial communication utility TeraTerm e Xilinx Platform Studio 10 1i e Xilinx Integrated Software Environment ISE 10 1 e Xilinx ChipScope Pro 10 1i e Catalyst SPX Series PCI Express Bus Protocol Analyzer Exerciser e LeCroy PETracer Analyzer
73. t C950l4 DEV 3 MyDevice3 Syste DEV 4 MyDevice4 KCS5VI System Monitor Cons UNIT O MyILAG ILA Trigger Setup Waveform TE ar Signals DEV 4 UNIT 0 Data Port jM_ABus o iM_wrDBus SI_rdDBus ha 14 ChipScope results are analyzed in the waveform window as shown in Figure 77 This figure ER Dd Bus Signal 0 15 10 x 0 15 10 5 0 CH 0 Sl_rearbitrate CH 1 JPCIE_USER_C CH 2 Sl_wrDAck CH 3 M_rdBurst CH 4 SI_wrBTerm CH 5 PLB_MVVrDAck CH 6 M_request CH 7 Sl_rdComp CH 8 PLB_PAValid CH 9 IP2INTC_Irpt CH 10 SI_wrComp CH 11 DataPort 11 CH 12 M_wrBurst CH 13 Sl_addrAck CH 14 PLB_MAddrAc CH 15 DataPort 15 _ y M ABus MM wrDBus S1_rdDBus 3l_rearbitrat4 PCIE_USER_CLK 1_weDAck M_rdBurst 51_wrBTerm PLB_MWrDAck M_request 31_rdComp PLB_PAValid IPZINTC_Irpt 51_wrComp DataPort 1l 10 15 20 25 30 35 40 45 50 55 ooo 00000000 ooo 00000000 000 00000000 B l m l ol o alal n X 100 A X 0 0 Upload DONE X1030_77_040908 Figure 77 ChipScope Pro Analyzer Triggered Value Change Dump vcd file The vcd files can be translated and viewed in most simulators The vcd2wlf translator in ModelSim reads a vcd file and generates a waveform log file wif file for viewing in the ModelSim waveform viewer S
74. the ML505 vix50t when used in a PC PCle Slot Them1505 mb plbv46_ pcie ready for _download m1505 mb plbv46_ pcie mcsis the configuration file for this reference design Because in PC based application it can save so much time to configure from the PROM the next figures provide the steps for creating a mcs for the ML505 XAPP1030 v1 0 1 May 6 2008 www xilinx com 49 Testing with a PC XILINX Figure 56 shows the ML505 Boundary Scan chain The first XCF32P is used to configure the FPGA Right clicking on the XCF32P invokes the Prepare PROM GUI iMPACT Boundary Scan Operations Output Debug Window Help PBR SRI i RHO ww E pa Boundary Scan FaSlaveSerial TaSelectMAP FEiDesktop Configuration FaDirect SPI Configuration E SystemACe xet32p xef32p xc95144x1 xccace xc5vix50t E PROM File Formatter file file file bypass file E iMPACT Prepare PROM Files want to target a Modes Xilinx PROM iMPACT Processes Generic Parallel PROM Available Operations are 31d Party SPI PROM PROM Supporting Multiple Design Versions Spartan3E MultiBoot b Get Device ID gt Get Device Checksum anii PROM File Format mcs OTEK UFP T fomat Reading J 39 rttf xcfp data xc EX0 BIN O Isc INFO iMPACT 501 1 Added De Operations HEX Swap Bits Manufacturer s ID INFO iMPACT 501 1 ksum Fill Value 2 Hex
75. the pcie_dma executable elf XAPP1030 v1 0 1 May 6 2008 www xilinx com 8 Catalyst Testing Catalyst Testing XAPP1030 v1 0 1 May 6 2008 t HyperTerminal DMASrc 20000000 DMA Finished DHASrc 2000002c DMA Finished DMASrc 2000005c DHA Finished 20000090 DMA Finished DMASrc 200000c8 DMA Finished DMASrc 20000104 DMA Finished DMASrc 20000144 DHA Finished DMASrc 20000144 DMA Finished DMASrc 20000148 DMA Finished 20000150 DMA Finished 2000015c DMA Finished DMASrc 2000016c DHA Finished Dest Dest Dest Dest Dest Dest Dest Dest Dest Dest Dest Dest 20002000 2000202c 2000205c 20002090 200020c8 20002104 20002144 20002144 20002148 20002158 2000215c 2000216c DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words DMAlength words XILINX 0000000c 0000000d 0000000e Q000000F 00000010 00000000 00000001 90000002 00000003 00000004 00000005 Connected 14 16 32 Auto detect PC Catalyst Software EDK ISE 9600 8 N 1 Figure 8 pcie_dma c output Figure 9 shows a functional diagram of the Catalyst test setup SPx4 Slot PCle Slot X1030_08_040908 This section discusses testing using Catalyst Enterprises SPX Series PCI Express Analyzer Exerciser system The SPX is a serial bus Analyzer Exerc
76. ting lt XILINX Root Complex to Endpoint Transactions As Root Complex the LeCroy Trainer generates memory writes and memory reads to the ML505 memory The ML505 reference design contains XPS BRAM and an MPMC interface to DDR2 Figure 40 shows the memory addressing for Root Complex Catalyst to Endpoint ML505 transactions The memory addressed is controlled by the BAR value written and by the C_PCIBAR2IPIFBAR_ generics In the reference design PCI BARO is written as 0x0000000060000000 C_PCIBAR2IPIFBAR_0O addresses XPS BRAM at 0x8AE10000 and C_PCIBAR2IPIFBAR_1 addresses DDR2 at location 0x90000000 XAPP1030 v1 0 1 May 6 2008 ML505 XPS BRAM PLBv46 PCle ee LeCroy PCle x1 C_PCIBAR2IPIFBARO OX8AE10000 C_PCIBAR2IPIFBAR1 0X90000000 MPMC DDR2 Root Complex 0x90000000 End Point X1030_40_040908 Figure 40 LeCroy ML505 Memory Addressing www xilinx com 38 LeCroy Testing XILINX The display area shows the TLPs defined in the peg file Figure 41 shows an excerpt from the rc2ep_wr_rd peg file The rc2ep_wr_rd peg shown is writes FFFFFFFFs to the six BAR registers in the Configuration Space Header CSH This is done using the Repeat construct The first register written is BARO located at offset 0x10 After writing and reading the CSH packets 32 34 are MRd64 MW64 0x12345678 and MRd64 gt LeCroy PETracer TM PCI Express Protocol Analyzer H lecroy rc2ep_wr_rd peg le Sie Record Generate
77. tion Debug Simulation Window Help 18 x JDP able glaa yx e 8Ang it a Slax AS mln ela Kile aannam E E Bus Interfaces Ports Addresses Project Applications IP Catalog Software Projects microbla microblaze 7 00 a Add Software Application Project Imb_v10 1 00 a E Default microblaze_O_bootloop dimt Imb_v10 1 00 a Default microblaze_0_xmdstub j li plb_v46 1 00 a Project pcie_mch_dma i imb_cntlr Imb_bram_if_cntlr 2 10 a Processor microblaze_O H cnt Imb_bram_if_cntlr 2 10 a Executable home lesters designs ml505 xps_bram_if_cntir_1 xps_bram_if_cntlr 1 00 a Compiler Options mb_bram bram_block 1 00 a Sources T xps_bram_if_cntlr_1_bram bram_block 1 00 a Headers i LEDs_8Bit xps_gpio 1 00 a xps_intc_0 xps_intc 1 00 a Processor microblaze_O DDR2_SDRAM mpmc 3 00 a je Executable home lesters designs ml505 i PCle_Bridge plbv46_pcie 1 00 a Compiler Options debug_module mdm 1 00 a Sources RS232_Uart_1 xps_uartlite 1 00 a Headers PCle_Diff_Clk util_ds_buf 1 00 a P proc_sys_reset_O proc_sys_reset 2 00 a xps_central_dma_O xps_central_dma 1 00 a gt clock_generator_O clock_generator 1 00 a X1030_05_040908 Figure 5 Selecting the pcie_dma Software Project XAPP1030 v1 0 1 May 6 2008 www xilinx com 6 Endpoint to Root Complex Transactions XILINX pcie_dma The pcie_dma project runs Direct Memory Access DMA operations The user
78. ttings Link Width to x1 Catalyst Enterprises Inc SPX Analyzer Exerciser Software rc2ep_rd64 BE x O sBUS Ota ire Be e o nA AEDO FE ene Exerciser Program Performance Items Link Settings Setting Physical Layer Settings SPX Reference Clock Slow Speed System Clock 1 1 25 Gbps Physical Data Link Layers Status gt lt g Clock 1 25 2 5 Gbps Clock Physical Layer Status Spectrum Clocking Trans LTSSM LO Spread Spectrum Scramble Bypass Lane 0 Link Number 0x00 Link Width 1 aie e Salis jot eE all AMS ef oe ais EY a yep Lane Polarity 4 4 4 ilities Level ack Transmission hger Out Data Link Layer Status DLCMSM ACTIVE Layer MV Enable LISSM LTSSM Behavior Platform Mode Transmitter Device ear All C End Point Mode Receiver Device Advanced Options Link Status Analyzer 1 000E85000159 USB Check All Check All Check All Clear All Clear All Clear All ie IT Auto Polati ja E X1030_21_040908 Figure 21 Performance Test Physical Settings Root Complex to Endpoint Performance Tests To setup the performance test the ML505 is inserted into the Catalyst The bitstream is downloaded into the FPGA Use XMD to write 0x003F0107 to the PLBv46 Endpoint Bridge Bridge Control Register to enable the Bus Master and BARs XAPP1030 v1 0 1 May 6 2008 www xilinx com 19 Using Catalyst to test PCle Performance XILINX R
79. ure 11 The BCR enables the PCle Bus Master and the Base Address Registers BARs This step must be done prior to using the Catalyst software XAPP1030 v1 0 1 May 6 2008 www xilinx com 10 Catalyst Testing g XILINX amp Command Prompt xmd INo of PC Breakpoints eee l No of Read Addr Data Watchpoints No of Write Addr Data Watchpoints Instruction Cache Support Instruction Cache Base Addre 6x 6660080 Instruction Cache High Addre Ox9ffFFF FF Data Cache Support 0n Data Cache Base Address 8x9 6666608 Data Cache High Address Ox9f fF FFL EE Exceptions Support off FPU Support of f Hard Divider Support off Hard Multiplier Suppo on Mul32 gt Barrel Shifter Support P of f MSR clr set Instruction Support Compare Instruction Support MDM UART Target mb target id Starting GDB server for mb target Cid gt at TCP port no 1234 XMDZ rst System reset successfully AMD mwr x85COG1e8 Ox403f6187 aMDx X1030_11_040908 Figure 11 Writing the Bridge Control Register XAPP1030 v1 0 1 May 6 2008 www xilinx com 11 Catalyst Testing XILINX Five tabs are used to setup the Catalyst PCle Bus Protocol Analyzer Exerciser Figure 12 shows Catalyst Capture settings The option selected is to Capture Everything except Idles In the Trigger On tab select Pattern and Trigger on TLP Any Type Select Any Direction In the Settings tab specify the na
80. ure 51 ep2rc_wait_mwr32 peg XAPP1030 v1 0 1 May 6 2008 www xilinx com 46 LeCroy Testing XILINX Figure 52 shows LeCroy Root Complex setup for analyzing an Endpoint to Root Complex MWr32 operation S LeCroy PETracer TM PCI Express Protocol Analyzer H lecroy ep2rc_wait_mwr32 peg I eile Setup Record Generate Report Search View Tools Window Help Sab wi es hw TARE EOE De PETrainer ML SN 1102 Link State InitFC State S h5 SS Saw Complete Ready Ln 1 Coli Search Fwd X1030_52_040908 Figure 52 EP to RC Write Operation The xmd command below generates the stimuli for the PLBv46 Endpoint Bridge to transmit the TLP mwr 0x20000000 0x12345678 Figure 53 shows the Analyzer output for an EP to Root Complex Memory Write of 0x12345678 File Setup Record Generate Report Search Yiew Tools Window Help 5 BY wile mA Ft R SB e O RR AA 000 00 0 o 90000000 xt LCRC 0000 _ 12345678 Ox11200AFA_ 0017 593 332 988 s PETrainer ML SN 1102 7 Link State InitFC State Bunn S 5 ES Lo Complete Ready Search F X1030_53_040908 Figure 53 EP to RC Write Results The write operation is easily varied using XMD The XMD command below writes eight locations mwr 0x20000000 0x12345678 8 XAPP1030 v1 0 1 May 6 2008 www xilinx com 47 LeCroy Testing XILINX Figure 54 shows the results from running the eight doubleword Endpoint to Root Complex write transa
81. us gt M wrDBus S1_raDBus 31_rearbitratq PCIE_USER_CLK 51_wrDAck M_xrdBurst 351_wrBTerm PLB_MWxrDAck M_request 51_rdComp PLB_PAValid INFO Found 1 Core Unit in the JTAG device Chain COMMAND save_project C Xilinx ChipScope_Pro_9_2i bin ntim1505_mb_plbv46_pcie cpj Writing project file C Xilim ChipScope_Pro_9_2i bin ntimI505_mb_plbv46_pcie cpj DONE X1030_74_040908 Figure 74 ChipScope Pro Analyzer Waveform XAPP1030 v1 0 1 May 6 2008 www xilinx com 69 Using ChipScope with the PLBv46 Endpoint Bridge lt XILINX 11 Set the trigger in the Trigger Setup window as shown in Figure 75 The trigger used depends on the problem being debugged Simple triggers are PA_Valid SI_AddrAck Sl_wrComp l ChipScope Pro Analyzer m1505_mb_plbv46_pcie gt Gs IL2T re cee Signals DEV 4 UNIT O Data Port IM_ABus JM_wrDBus Sl_rdDBus 0 Sl_rearbitrate 1 IPCIE_USER_C gt 2 5 _wrDAck 3 JM_rdBurst 4 SlwrBTerm 5 PLE_MWrD ek 6 M_request 7 Sl_rdComp 8 PLB_PAValid 9 sIP2INTC_Irpt 210 SI_wrComp 211 DataPort 11 12 M_wrBurst 13 Sl_addrAck 14 PLB_MAddrAc DEV 2 MyDevice2 xcasoja DEV 3 MyDevice3 System 9 DEV 4 MyDevice4 XCS5VI System Monitor Cons UNIT 0 MyILAOD ILA Trigger Setup Waveform ile View JTAG Chain Device Trigg

PLBv46 Endpoint Bridge for PCI Express in a ML505

Contents

Download Pdf Manuals

Related Search

Related Contents