Home

An FPGA-based Trigger Processor for a Measurement of Deeply

Contents

1. host to board memory bridge sg_dma controller BRAM wishbone TLP manager PARI configmem bridge DMA engine Fast registers registers Static registers PCle interface block Figure 6 3 Internal structure of the PCIe interface block 6 1 1 4 PCIe Endpoint Block A PCIe endpoint interface is included in the Base design to enable a high bandwidth communi cation between the CPU and the FPGA on the TIGER board It is based on an open source PCIe DMA controller pcie sg dma available at OpenCores 164 which was extended to provide some project specific features The main elements of the PCIe interface block see Fig 6 3 are e a Xilinx integrated PCIe IP core configured as an endpoint with a Gen x1 link and three base address registers BARO BAR2 the controller containing a TLP manager to receive and send transaction layer packets and the DMA and PIO engines anda memory bridge to direct the read and write transactions to the appropriate targets Three target address ranges are located at the slave side of the memory bridge each related to a specific memory type The register space BARO contains system registers status control and DMA related registers as well as a number of user definable registers The memory space BARI implements a
2. 6 1 2 TIGER Trigger Processor Design o o o eee 6 1 3 TIGER Readout Concentrator Design o o 6 2 Operating System and Device Drivers o oo 6 2 1 Kontron Drivers 6 2 2 CPLD Driver 6 2 3 PCle Driver 6 2 4 JTAG Driver and ChipScope Server LL 6 2 5 AMD Driver and OpenCL SDK LL 6 3 Monitoring and Control Tools 7 Commissioning 7 1 CAMERA Installation 7 1 1 Laser Calibration Runs 83 83 83 87 87 89 89 89 91 92 93 94 97 97 97 98 100 101 101 101 105 105 106 107 107 109 111 112 113 114 116 119 120 121 121 121 123 124 124 viii Contents 7 1 2 TIGER Latency Adjustment 2 2 2 oo oo nennen 132 71 3 The Start Counter 24 we a ad a 134 1 2 Physics Data sii Bee ud AA Bale eise quoe 135 7 2 1 First Proton Signals ud 24 5 se 22 a E du A Ea 135 7 2 2 Comparison of two CAMERA Calibration Methods 136 17 2 3 M on Runs uc er Uy RU M ede i rU ER 139 8 Summary 143 A TIGER Board Layout 145 B FPGA and CPLD Pin Plan 149 C Connector Pin Out 157 C 1 VXSBackplaneConnectors leen 157 6 2 VEIDGEGOnneCtOE sli Br Exe Ra deu el eU RUE o 161 D CAMERA Photos 163 Bibliography 167 A poet once said The whole universe is in a glass of wine We will probably never know in what sense he meant that for poets do not write to be understood But it is true that
3. Qu Ebo Fro R23 RR ten Iram SEI 16 LETY a g TE 962 EE tD 95 29 Ma dia ME el m Ta vo co0000000000 2 88 ss ecscecccccco EE n age se E Riso ang zo na Bl EA BD riti BM qu R166 WI hg R109 R107 RITO R108 me m B mos RIO Riog Wig EH oma 5 me mo BS Riog SE i ms 09 agg R100 mag 9 EXE fo ES ass pon Re yy Mme M LE x ke nas M om gum musa EE E mp 9 omo MM Lo ape Ew Ns io car RIS R76 e kw MP ve Sa ne Ro Pow I ow M R61 a R62 Lol B mr B gu Me m Es Rss Rss da Em fa ES AT HE rm 3 mr CU fase EE R297 EE o s E cio gr nm ano m m mm an C135 C180 R354 x E M R277 rar E wn tom ET R276 di n ve VCC_GPU s RIS n n6 ee 00 eo 00000 Figure A 3 TIGER PCB bottom view 148 A TIGER Board Layout Multilayer TIGER board HIILFA Material mm Stack Up File Assembly 0 060 Prepreg DRI 0 100 Prepreg Via 200 ym Pad 500 ym 0 025 Copper plated 0 017 Copper 0 100 Prepreg 0 017 Copper 0 017 Copper 0 100 Prepreg B 9 Z O M 0 060 Prepreg 0 017 Copper 0 100 FR4 0 017 Copper 0 050 HF Prepreg 0 017 Copper 0 100 FR4 0 017 Copper 0 060 Prepreg n io E 02 ci eo 9
4. o oo e eee eee The Spectromeler vaa a Bon a ee A 3 3 1 Tracking Detectors creere srine ha Brake ae UR eA 3 3 2 Calorimeters 5i o X RURSUS UR d URS 3 3 3 Muonldentification o e EE E E 3 3 4 RICHT A A A A RA AAA The Trigser System o fa aet ao ee ee gg OR d The Data Acquisition System lees 3 5 1 The GANDALF Framework o e eee eee 4 CAMERA Proton Trigger 4 1 4 2 Detector Principles 2 24 2008 2 sa a VUES S RU 4 1 1 Time of Flight Measurement 2 2 2 on onen nen 4 1 2 Energy Loss Measurement LL Trigger Concept Jw n do qux uen IRR os eli ee ee T TUE Ges o QD DU VW w Ww 10 12 13 14 16 19 19 22 22 22 26 26 26 27 29 31 32 34 vi Contents 42 1 Design Objectives a 2 oru EU e AIR BURN eee Ged 41 4 2 2 Trigger Conditions llle 41 4 2 2 1 Geometric Coincidence ooo ooo o 41 4 2 2 2 Time Calibration o o 42 4 2 2 3 Cuts on Energy Deposition o 43 4 2 2 4 Correlation with the Beam o ooo oo 45 4 2 3 Further Requirements ooo e eee eee ee eee 46 43 Electronics Framework 2 2 2 2 2 moon nennen 46 4 3 1 VME64x VXS Crate Con onen 47 4 3 2 Backplane Link iu dne uns pr van m er MR a 48 5 The TIGER Module 53 5 l Mainboard i duae RUE REA S PR SE eei lade 54 bl Virtex 6 FPGA se AERE E Not aee te eh Ss 54
5. The ILA integrated logic analyzer core is the main part of the ChipScope logic It pro vides a customizable number of data and trigger inputs in order to connect the signals to be investigated Flexible match units are available for elaborate triggering based on conditions which are configured during runtime to detect the events of interest Upon a trigger event the ILA captures data frames of variable length and stores them using on chip Block RAM The VIO virtual input output core is used to monitor and drive internal FPGA signals Both synchronous and asynchronous inputs and outputs are available The ICON integrated controller core establishes the communication between the afore mentioned cores and the host computer via a JTAG download cable It can connect up to 15 instances of ILA and VIO cores To analyze the TIGER FPGA firmware ChipScope cores have been inserted into the Base design which may be connected to the signals of interest The design is synthesized and implemented as usual either with the ISE tool chain or the PlanAhead design flow The FPGA is initialized 6 1 FPGA Firmware 113 with the resulting bitstream and hooked up to a Platform Cable USB via the JTAG boundary scan port cf 5 1 5 3 The ChipScope Pro Analyzer tool is a graphical user interface to communicate with the ChipScope cores in the target design It is used to define the trigger settings and the sampling parameters of the logic analyzer core
6. 37 General purpose computing on graphics processing units 38 open drain signal 39The specification allows any voltage between 7 V and 20 V and a current of up to 10 A 88 5 The TIGER Module Table 5 10 PCIe signal group of the MXM interface Signal Name I O Type Description connected to PEX_TX 7 0 I Diff TX lanes from the upstream PCIe device PEX TX 7 0 2 connected to the Virtex 6 GTX transceivers PEX_RX 7 0 O Diff RX lanes to the upstream PCle device PEX_RX 7 0 connected to the Virtex 6 GTX transceivers PEX_REFCLK I Diff PCle reference clock PEX_REFCLK from the ICS9DB102 clock fan out PEX_RST I CMOS PCIe system reset connected to the CPLD PEX CLK REQ O OD PCIe clock request connected to clock fan out via CPLD PEX_STD_SW I OD selects voltage swing of differential PCIe signals 0 full swing level 1 or NC reduced swing level connected to the CPLD Table 5 11 Power and thermal management signal group of the MXM interface Signal Name I O Type Description connected to SMB_CLK I O OD SMBus clock data for access to the thermal sensor SMB_DAT connected to the SMBus of the CPU via bus switch TH_OVERT O OD thermal shutdown request due to over temperature connected to the CPLD which will instantly power down the MXM module when this signal is asserted TH_ALERT I O OD thermal interrupt request connected to the CPLD but
7. TIGER TIGER Uni Freiburg Figure 5 34 The TIGER module features four Figure 5 35 A dual VHDCI connector provides LEDs and an OLED display on its front panel 32 LVDS inputs on its left hand side and 32 LVDS The meaning of the LEDs is explained in Tab outputs on its right hand side In addition two 5 17 The display is used to show general status IVTTL outputs are available using LEMO con information nectors 104 5 The TIGER Module 6 Firmware and Software The hardware of the TIGER module has been detailed in the previous chapter But since an FPGA based electronic device like this is not operational without the adequate firmware the three firmware designs which are available to date will be outlined in the following section The so called Base design contains all the necessary building blocks to implement the board in terfaces and to control the internal clock generation and the start up of the FPGA Two further designs have been derived from the Base design in order to implement the specific behavior of the Trigger Processor and the Readout Concentrator applications In the subsequent sections the software to control the TIGER board is briefly explained Starting from a description ofthe operating system and device drivers running on the COM Express CPU 6 2 the development of various configuration and monitoring tools is highlighted 6 3 6 1 FPGAFirmware The FPGA firmware for the TIGER board is written in
8. 0 100 Prepreg 0 017 Copper 0 100 FR4 0 017 Copper 0 050 HF Prepreg 0 017 Copper 0 100 FR4 0 017 Copper 0 060 Prepreg ep lt 9 Z pod O O Multilayer Stack Up Modules are protected by patent 0 100 Prepreg 0 017 Copper 0 017 Copper 0 100 Prepreg 0 017 Copper 0 025 Copper plated Bare Board 53 mm 1 71 mm Immersion Tin 62 mm 1 81 mm including soldermask Hot Air 64 mm 1 84 mm and copperplating Immersion Gold 1 61 mm 1 80 mm Drawing 18 05 2008 Wi Thickness 2008 by ILFA Feinstleitertechnik GmbH Figure A 4 Cross section of the TIGER multi layer PCB B FPGA and CPLD Pin Plan The FPGA I O signals of the TIGER module are listed in tables B 1 to B 3 together with the pin numbers and signal standards Signals of the LVDS VXS backplane interface are listed in appendix C 1 in conjunction with the connector pin out Signals of the IVDS VHDCI connector are listed in appendix C 2 in conjunction with the con nector pin out The CPLD I O signals ofthe TIGER module are listed in table B 4 together with the pin numbers and signal standards 150 B FPGA and CPLD Pin Plan FPGA Pin Plan Signal Pin Standard pcie x8 tx n 0 P4 MGT pcie x8 tx n l R2 MGT pcie x8 tx n 2 T4 MGT pcie_x8_tx_n 3 U2 MGT Signal Pin Standard pcie_x8_tx_n 4 W2 MGT aurora mgtclk n E9 MGTCLK pcie x8 tx n 5 AA2 MGT aurora mgtclk p E10 MGTCLK pcie x8 tx n 6 AC2 MGT aurora rx
9. The TIGER CPLD is connected to the LPC bus and it is located in the system s I O map at address 800h A direct access to the CPLD registers is possible with low level port input and output functions like inb and outb O however the usage of these functions in user space requires I O permissions and the user would have to deal with the actual hardware register contents Therefore a simple ioctl char driver 174 175 was written which creates a device file at dev cp1d0 to ease the communication with the CPLD The defined ioctl commands are listed in Tab 6 4 6 2 3 PCIe Driver Communication with the PCIe interface of the Virtex 6 FPGA is accomplished using the Linux pciDriver 176 and the MPRACE library 177 The Linux pciDriver is a generic driver for PCI 5Programmable Logic Device 122 6 Firmware and Software Table 6 4 TIGER CPLD ioctl commands ioctl command type description CPLD GET STATUS _IOR returns the status register for the interpretation of the data fields see Tab 6 5 CPLD SET FM _IOW setsthe flash mode of the CPLD interface cf 6 1 1 1 CPLD SET CA _IOW sets the CA signals of the SelectMap interface CPLD SET CD _IOW sets the CD signals of the SelectMap interface the CCLK is automatically toggled by the CPLD after this register has been written CPLD RESET FPGA _IO resets the FPGA by pulling down the Program B pin CPLD INDICATE PDOWN _IO notifies the power sequencing state machine ofthe CPLD
10. o co 10 Energy loss in Ring A MeV 10 20 30 40 50 60 70 80 90 Energy loss in Ring B MeV Figure 4 7 Simulated energy loss of the DVCS proton in the inner A and outer 5 scintillators of the CAMERA detector 78 p 89 Another possibility to show the energy loss ofthe protons in the CAMERA detector is to plot the energy loss in the inner scintillators versus the energy loss in the outer scintillators Fig 4 7 resulting in a representation of the proton band which is not depending on f The branch above the knee consists of the protons which are stopped in B while the lower branch contains the protons which are fast enough to leave the detector 4 2 2 4 Correlation with the Beam While leaving the target the recoil protons can lose a large amount of energy Especially the slow protons f lt 0 4 are decelerated significantly by the liquid hydrogen and the material surrounding the target i e the Kapton cell the super insulation foil and the carbon tube 79 For this reason and also due to the moderate time resolution of the A counters itis not possible to obtain a precise vertex time from the proton alone To optimize the timing ofthe trigger signal which is required to have a constant time offset with respect to the incident beam particle additional information from another detector is needed The scintillating fiber SciFi stations FIO1 and FIO2 located upstream of the target can provide a precise timing ofthe b
11. An FPGA based Trigger Processor for a Measurement of Deeply Virtual Compton Scattering at the COMPASS II Experiment Sebastian Schopferer UNI Fakult t f r Mathematik und Physik Albert Ludwigs Universit t Freiburg FREIBURG An FPGA based Trigger Processor for a Measurement of Deeply Virtual Compton Scattering at the COMPASS II Experiment Dissertation Zur Erlangung des Doktorgrades der Fakult t f r Mathematik und Physik der Albert Ludwigs Universit t Freiburg im Breisgau vorgelegt von Sebastian Schopferer aus Lahr Schwarzwald Freiburg Oktober 2013 Dekan Prof Dr Michael R Zicka Leiter der Arbeit Prof Dr Horst Fischer Referent Prof Dr Horst Fischer Korreferent Prof Dr Karl Jakobs Tag der Verk ndigung des Pr fungsergebnisses 16 12 2013 Teile dieser Arbeit wurden in folgenden Fachzeitschriften ver ffentlicht M Alekseev et al Physics Letters B 693 3 227 235 2010 doi 10 1016 j physletb 2010 08 034 C Adolph et al Phys Rev D 87 5 052018 2013 doi 10 1103 PhysRevD 87 052018 S Bartknecht et al Nucl Instr and Meth A 623 1 507 509 2010 doi 10 1016 j nima 2010 03 052 S Bartknecht et al IEEE Trans Nucl Sci 58 4 1456 1459 2011 doi 10 1109 TNS 2011 2142195 J Bieling et al Nucl Instr and Meth A 672 0 13 20 2012 doi 10 1016 j nima 2011 12 104 M B chele et al Physics Procedia 37 0 1827 1834 2012 doi 10 10
12. The availability of the CPLD interface signals on the SelectMap buses is indicated by the CA 22 21 gt flags see Tab 6 1 To coordinate the usage of the SelectMap signals the CPLD firmware implements a register called flash mode which is set by means of the CPLD driver cf 6 2 2 to one of the following values FM STARTUP flash is enabled for automatic FPGA configuration default FM SELECTMAP flash is disabled SelectMap signals are driven by the CPLD FM FLASHPROG flash is enabled to be reprogrammed by the CPLD FM NORMAL flash is disabled CA and CD signals are used for FPGA communication The CPLD interface uses the CD bus to announce the presence of the GANDALF boards in the VXS crate to the FPGA This information is generated by the CPLD based on the VXS handshake signals cf 5 3 1 2 and evaluated by the FPGA in order to decide whether the output buffers of the VXS ports are to be enabled or tri stated The CA bus is used to transmit various flags between the CPLD and the FPGA CA lt 22 8 gt contains flags from the CPLD to the FPGA while CA lt 7 0 gt transmits flags in the opposite direction The assignment of the flags is shown in Tab 6 1 6 1 1 2 Clock Management Reset Logic and VXS I O Buffers As seen in the Base design top level schematic three VHDL blocks are directly connected to the CPLD interface module The first one is the clock management block which is responsible for the generation and
13. The impact parameter b is measured relative to the center of momentum of the nucleon R which is defined as the sum over the transverse positions r of all partons weighted with their fractional momentum x R Y xir 2 35 1 4 8 For x 1 the center of momentum is mainly defined by the active quark and the width of the distribution gy x b1 2 f qf x b1 necessarily tends to zero Fig 2 9 On the other hand partons with small x can be found also farther away from the nucleon s center of momentum 2 4 4 GPDsand the Nucleon Spin Structure One of the goals of the COMPASS experiment is the investigation of the nucleon spin structure Up to now the contribution of the orbital angular momenta Ly to the nucleon spin 1 7 Vp Jg 2 36 f 1 gt Jf 9 Ags Ady Ly 2 37 is mainly unknown since there was no process to measure the orbital angular momenta of the partons directly In 1997 Xiangdong Ji 21 published his sum rule which relates the second moments of the GPDs to the total angular momentum of the quarks 1 Jp liim f dx x Hf D Ef 6 0 2 38 2 t20J 1 14 2 Theoretical Motivation N N 1 P o Y Figure 2 10 Leading order processes for lepto production of real photons a deeply virtual Compton scattering b and c Bethe Heitler process Using Ji s sum rule it is possible for the first time to determine the orbital angular momenta of the quarks For this purpose it is necessary to
14. f dx Hf x amp t G 0 2 32 1 1 f dx Ef x amp t G 0 2 33 1 The functions F 0 El 0 AO and Gl are the contributions of the quark flavor f to the Dirac Pauli axial and pseudoscalar form factors For the GPDs there are higher order corrections in as so they are like the usual PDFs de pending on Q But F and F 1 are independent from Q since they are defined as matrix 12 2 Theoretical Motivation valence quark longitud NE a b x 0 003 x 0 03 Figure 2 8 Impact parameter dependent PDFs for nucleon tomography a For a fixed x qf x b1 describes the distribution of the transverse distance b b of partons carrying the fraction x of the nucleons longitudinal momentum P from the center of momentum of the nucleon R b Transverse spatial parton distribution in the nucleon at certain parton longitudinal momentum fractions x 27 p 10 elements of a conserved current Hence the Q dependency of the GPDs has to disappear in the first moments 2 30 and 2 31 By the integration over x in 2 30 2 33 the reference to the infinite momentum frame in which is defined is lost Consequently Lorentz invariance requires these integrals to be inde pendent from 2 4 3 Impact Parameter Dependent Parton Distributions In the limit of 0 where the parton carries the same longitudinal momentum fraction x in initial and final state the GPDs can be i
15. 147 148 149 150 151 152 153 154 155 156 157 158 I STRAZNICKY 2012 Ruggedization of MXM Graphics Modules In High Performance Extreme Computing HPEC 2012 IEEE Conference on p 1 MXM SIG 2008 MXM Version 3 0 System Design Guide Revision 0 3 WOLF Industrial Systems MXM Products http wolf ca products mxm Khronos Group 2012 The OpenCL Specification Version 1 2 http www khronos org PCI SIG 2003 PCI Express Base Specification Revision 1 0a R BUDRUK D ANDERSON T SHANLEY 2004 PCI Express System Architecture Mind Share Inc 1st edn PCI SIG 2006 PCI Express Base Specification Revision 2 0 Intel Corp 2011 PHY Interface for the PCI Express SATA and USB 3 0 Architectures Ver sion 4 0 PLDA 2013 PCI Express XpressRICHS Reference Manual Version 1 5 2 Mentor Graphics ModelSim SE 64 10 0b http www mentor com products fpga simulation modelsim PLDA 2012 PCI Express Bus Functional Model 3 Reference Manual Version 1 4 0 Xilinx Inc 2007 Hot Swapping Virtex II Virtex II Pro Virtex 4 and Virtex 5 Devices XAPP251 Xilinx Inc 2012 Virtex 6 FPGA Data Sheet DC and Switching Characteristics DS152 Xilinx Inc Will driving the I Os of an unpowered bank cause damage to the part http www xilinx com support answers 13428 htm F HERRMANN 2011 GANDALF Framework User Guide Version 1 1 http wwwhad physik uni freiburg de gandalf pa
16. 4 5 6 7 8 9 10 11 12 J D BJORKEN E A PASCHOS 1969 Inelastic Electron Proton and y Proton Scattering and the Structure of the Nucleon Phys Rev 185 1975 R P FEYNMAN 1969 Very High Energy Collisions of Hadrons Phys Rev Lett 23 24 1415 EUROPEAN MUON COLLABORATION J ASHMAN ET AL 1988 A measurement of the spin asymmetry and determination of the structure function gi in deep inelastic muon proton scattering Physics Letters B 206 2 364 SPIN MUON COLLABORATION B ADEVA ET AL 1998 Spin asymmetries Aj and structure functions g of the proton and the deuteron from polarized high energy muon scattering Phys Rev D 58 112001 P ANTHONY R ARNOLD ET AL 1999 Measurement of the deuteron spin structure func tion g x for 1 GeVIo lt Q lt 40 GeV o Physics Letters B 463 2 4 339 HERMES COLLABORATION A AIRAPETIAN ET AL 2007 Precise determination of the spin structure function g of the proton deuteron and neutron Phys Rev D 75 012007 COMPASS COLLABORATION C ADOLPH ET AL 2013 Leading and next to leading order gluon polarization in the nucleon and longitudinal double spin asymmetries from open charm muoproduction Phys Rev D 87 052018 M GELL MANN 1964 A schematic model of baryons and mesons Physics Letters 8 3 214 R L JAFFE A MANOHAR 1990 The g problem Deep inelastic electron scattering and the spin of the
17. 64 hosts eight 12 bit 500 MS s analog to digital converters for continuous digitization of up to eight detector channels Two ADCs at a time can optionally be combined to sample the same signal in time interleaved mode resulting in a 1 GS s digitization of four analog channels per mezzanine card The raw data stream is forwarded to the DSP FPGA on the mainboard for real time pulse shape analysis and feature extraction This operation mode is called the GANDALF transient analyzer 65 66 The digital I O mezzanine card 67 is capable of reading out 64 detector channels ac cepting signals in the differential LVDS or LVPECL standard as they are usually provided by pre amplifier and discriminator modules The acquired signals are directly fed into the FPGA where the user can select from various processing functionalities which have been implemented so far a time to digital converter called M1 TDC 68 69 a scaler 70 a combination of both 71 or a mean timer with subsequent coincidence logic 72 20 More information about the VMEbus architecture can be found in section 4 3 1 21Daughtercards which are mounted on top of and parallel to the mainboard 22 Field Programmable Gate Array 3 5 The Data Acquisition System 35 The ARWEN optical mezzanine card 73 is available for receiving data from digital front end cards which are typically mounted directly on the detector The connection is estab lished via optical fibers using a
18. Additionally each macrocell contains a storage element which can be configured as either D or T flip flop or transparent latch The model XC2C512 7FTG256C used on the TIGER board features 32 function blocks with a total of 512 macrocells and 212 I O pins in four banks Compared to an FPGA a CPLD is rather limited in terms of logic capabilities and speed but it has the advantage of internal non volatile NV configuration memory allowing it to operate directly after power up The time needed for reading the configuration from NV memory during start up is below 400 us 105 and for firmware updates the NV memory is in system programmable using a JTAG interface A CPLD is ideally suited for many low level board management tasks due to its instant on feature On the TIGER module it is controlling several vital processes both during start up and during operation Due to its tight connection with multiple onboard devices and its diverse purposes it is reasonable not to detail all the CPLD functions at this point but rather list refer ences to the sections where these functions will be described in conjunction with the respective components power up and power down sequencing the various power converters are enabled and disabled in a safe manner triggered by the switch handles on the front panel see 5 1 6 e FPGA configuration after switching on the module the FPGA is initialized with the ap plication specific firmware using one of the m
19. COMPASS experiment reference clock But all have in common the requirement for low jitter clock signals for best possible performance The clock generation and distribution network shown in Fig 5 10 has been carefully designed to meet these demands 5 1 3 1 PCI Express Reference Clock The PCIe data transfer is based on high speed serial point to point links using 8b 10b encod ing More details about the PCI Express architecture can be found in 5 2 6 No clock signal is transmitted on the link instead the receiver recovers the RX clock from the bit transitions of the data stream using a PLL In addition a 100 MHz reference clock is supplied to each PCIe device to allow for the generation of all required internal clocks The PCIe hierarchy on the TIGER board is quite simple It contains the CPU which forms the root complex the FPGA acting as endpoint switch and finally the GPU as another endpoint Hence the reference clock which is provided by the COM Express CPU module has to be dis tributed to the two other members of the PCIe structure This is done with a 1 2 fan out buffer ICS9DB102 from IDT 98 a PLL based zero delay buffer compliant to PCIe Genl and Gen2 clocking requirements The PLL bandwidth is selectable 0 5 MHz or 2 5 MHz to minimize jitter peaking in the PLLs of the downstream devices The HCSL outputs are connected via AC coupling capacitors to the reference clock inputs of the MXM socket and the FPGA s GTX 14Serial Pres
20. M38 M39 VXSPORT_17 lt 7 gt P5 G4 H4 AR27 AT27 VXSPORT 18 0 P2 A13 B13 AR28 AP28 VXSPORT 18 1 P2 C14 D14 AN28 AM27 VXSPORT_18 lt 2 gt P2 A15 B15 AU28 AV28 VXSPORT_18 lt 3 gt P2 C16 D16 AU29 AV29 VXSPORT 18 4 P2 E13 F13 BB29 BB28 VXSPORT_18 lt 5 gt P2 G14 H14 AW28 AY28 VXSPORT 18 6 P2 E15 F15 AN26 AP27 VXSPORT 18 7 P2 G16 H16 AK28 AK29 VXS_sp_rx lt 0 gt P2 15 J5 AU33 AU32 VXS sp rx 1 P2 K6 L6 AP31 AN31 VXS sp rx 2 P2 I7 J7 AL27 AM28 VXS sp rx 3 P2 K8 L8 AR34 AP33 VXS sp tx 0 P2 1 jl AN33 AN34 VXS_sp_tx lt 1 gt P2 K2 L2 AM33 AM32 VXS sp tx 2 P2 I3 J3 AP32 AR32 VXS sp tx 3 P2 K4 L4 Table C 1 LVDS VXS Backplane Interface Signals C 2 VHDCI Connector 161 C 2 VHDCI Connector The pin assignment of the IVDS connectors on the TIGER front panel is listed in the following table The letter L or R in the column Conn denotes the connector on the left hand side or on the right hand side respectively FPGA Pins Signal Name Connector Pins Conn P18 P17 LVDS_IN lt 0 gt L 1 35 G16 FI6 LVDS_IN lt 1 gt L 2 36 C15 D15 IVDS IN 2 L 3 37 D18 C18 LVDS_IN lt 3 gt L 4 38 D13 E13 LVDS_IN lt 4 gt L 5 39 M14 N14 LVDS_IN lt 5 gt L 6 40 H14 G13 LVDS_IN lt 6 gt L 7 4l J16 H16 IVDS IN 7 L 8 42 D17 EI LVDS_IN lt 8 gt L 9 43 E15 FI5 LVDS_IN lt 9 gt L 10 44 A2 Jll IVDS_IN lt 10 gt L 11 45 K14 L14 IVDS IN 11 L 12 46 K12 Lll IVDS IN 12 L 13 47 JA
21. hole of about 50x 50 cm The hole of ECALO matches the outer acceptance of ECAL1 ECALO is built from modules sized 12 x 12 cm which requires 300 modules for the final construction However during the 2012 DVCS test run a smaller prototype 132 x 108 cm with 56 modules was used 48 For the construction of ECALO a new type of sampling calorimeter module Fig 3 8 has been developed 49 This was necessary for two reasons On the one hand due to space constraints the total length of the module is limited On the other hand the close proximity to the spec trometer magnet SM1 required a design that is insensitive to magnetic fields The calorimeter part of the Shashlyk module is built from 109 sampling sandwiches with an edge length of 12 cm Each sandwich consists of a 0 8 mm thick lead plate and a polystyrene based scintillator layer with a thickness of 1 5 mm The scintillator is segmented into 9 tiles of 4x4 cm each The edges ofthe tiles are painted white to form 9 light isolated towers which are separately read out with wavelength shifting WLS fibers The total length of the calorimeter stack is 25 cm which corresponds to 15 radiation lengths In order to be insensitive to magnetic fields no PMTs are used for photon detection Instead multi pixel avalanche photo diodes MAPD from Zecotek Photonics 50 were chosen to per form this task The WLS fibers are coupled to a MAPD 3N photo diode separately for each of the read
22. i2c_ck_s0 R3 LVCMOS33 i2c_dat_s0 P4 LVCMOS33 ics_clkreq0_B E6 LVCMOS33 ics clkreq1 B D6 IVCMOS33 INIT B M8 IVCMOS25 ldo en E10 LVCMOS33 ldo pg E9 IVCMOS33 led cpldiG B H3 IVCMOS33 led cpld1R B H5 IVCMOSS33 led cpld2G B B7 LVCMOS33 led_cpld2R_B B8 LVCMOS33 led gpl B K5 IVCMOS33 led gp2 B K2 IVCMOS33 led gp3 B L3 IVCMOS33 led gp4 B M5 LVCMOS33 Ipc_ad lt 0 gt R4 LVCMOS33 Ipc_ad lt 1 gt N4 LVCMOS33 Ipc_ad lt 2 gt T2 LVCMOS33 Ipc_ad lt 3 gt Tl LVCMOS33 Ipc_clk M3 LVCMOS33 Ipc_drq0_B N3 LVCMOS33 Ipc frame B M4 LVCMOS33 Ipc serirq P1 LVCMOS33 Itm pg E2 IVCMOS33 ltm run El LVCMOS33 MO T5 LVCMOS33 MI T4 LVCMOS33 M2 T3 IVCMOS33 main en A6 IVCMOS33 main pg AS IVCMOS33 pex clk req B Fl LVCMOS33 pex_rst_B F2 IVCMOS33 pex std sw B B2 IVCMOS33 pmbus cntl A4 LVCMOS33 PROGRAM_B L16 IVCMOS25 RDWR_B N9 IVCMOS25 SI2_INTR N5 LVCMOS33 SI_DEC R1 IVCMOS33 SI INC R2 IVCMOS33 SI LOL P2 IVCMOS33 Signal Pin Standard SI LOS N1 LVCMOS33 SI RST B N2 IVCMOS33 sus s3 B C7 IVCMOS33 sus s5 B G5 IVCMOS33 sus stat B H4 IVCMOS33 switchl C5 LVCMOS33 switch2 A7 IVCMOS33 TMC LOCK P7 IVCMOS33 TMC RATE M6 IVCMOS33 ucd gpil E7 IVCMOS33 ucd gpi2 B1 LVCMOS33 ucd_gpol E3 LVCMOS33 ucd_gpo2 C6 LVCMOS33 ucd_gpo3 A2 LVCMOS33 ucd_gpo4 B4 IVCMOS33 ucd seq 2 B6 IVCMOS33 ucd tms pgood D4 IVCMOS33 VXS ga0 B D3 IVCMOS33 VXS pen B H14 IVCMOS33 VXS pp scl 1 F12 LIVCMOS33 VXS pp scl 2 B15 LVCMOS33 VXS_pp_scl lt 3 gt E12 LVCMOS33 VXS_pp
23. like resistors for termination or capacitors for AC coupling Both differential and single ended connections of this type are entered very efficiently using the spreadsheet ed itor Fig 5 21 Schematic drawings of the devices and connections captured with this method are created automatically for documentation purposes Fig 5 22 The traditional schematic editor on the other hand is used for entering power and analog blocks of the design by placing device symbols on a canvas and drawing wires between them 25Electronic Design Automation 5 1 Mainboard 77 i30 xc6vlx365tff1759 U8 Functions Expand All Pins Y Show Differential Pairs V Show vectors 9 Pin Name PinNumber PinType Signal Termination S i DP MGTREFCLKO 117 G10 G9 Input E DP MGTREFCLK1 117 E10 E9 Input aurora magtclk DPSeriesCap motrefclkip 117 E10 Input aurora_matclk matrefclkin_117 E9 Input aurora mgtclk gt E DP MGTRXO 117 H7 H8 Input aurora rx 0 e matrxp0_117 H Input aurora_rx lt 0 gt o matrxn0_117 H8 Input aurora rx 0 DP MGTRX1 117 GS G6 Input aurora_rx lt 1 gt DP MGTRX2 117 F7 F8 Input aurora_rx lt 2 gt DP_MGTRX3_117 ES E6 Input aurora rx 3 DP MGTTXO 117 31 32 Output aurora_tx lt 0 gt DP_MGTTX1_117 H3 H4 Output aurora_tx lt 1 gt DP_MGTTX2_117 G1 G2 Output aurora_tx lt 2 gt DP_MGTTX3_117 F3 F4 Output aurora_tx lt 3 gt Figure 5 21 The FPGA connectivity is captured using the Allegro System Architect spread
24. not calibrated for the plots in the right column the TO calibration constants are applied 1 and 2 show the time of flight for each possible A B counter combination 3 and 4 show the mean time of the hits in the counters 5 and 6 show the z position of these hits 132 7 Commissioning Table 7 3 1 calibration offsets for the CAMERA detector obtained from the laser run 108912 i A up A down B up B down 0 32 609 30 454 1 845 2 778 1 33 894 30 848 1 187 1 760 2 32 953 30 671 2 058 0 883 3 34 309 31 030 3 696 0 561 4 36 096 32 965 3 171 4 119 5 36 152 32 368 0 532 3 708 6 36 222 33 175 4 997 5 865 7 36 773 33 554 4 521 5 180 8 34 187 29 454 2 477 4 160 9 35 570 32 045 4 584 1 758 10 35 621 32 373 4 036 4 190 11 34 894 31 941 2 000 2 000 12 36 631 31 969 1 235 3 160 13 36 814 32 330 3 259 2 106 14 36 980 34 009 0 430 3 366 15 37 344 32 291 3 267 3 727 16 35 902 31 675 5 711 4 936 17 35 359 33 398 3 409 3 155 18 36 386 34 540 3 402 6 587 19 36 835 33 252 5 184 4 959 20 36 208 30 595 3 313 2 431 21 35 209 32 034 2 605 2 809 22 35 124 32 158 2 820 2 885 23 35 645 32 075 1 229 0 214 by the shift crew to observe the behavior of the CAMERA detector during the data taking An example is given in Fig 7 3 for the run 108898 pion beam The time of flight distribution for each A B combination is shown in the topmost plots The distributions of the mean
25. pair of hits in the scintillator slats Due to the azimuthal rotation of the two detector barrels by 7 5 against each other to increase the angular resolution two A B combinations are possible for every element see Fig 4 4 The coincidence logic therefore has to check a total number of 48 combinations between A and B counters Ci Aj Bi A Bj 4 10 C Co Coa 4 11 for i 0 1 23 and j i 1 mod 24 Here X amp Y denotes the coincidence between two hits in the elements X and Y and C is the logical disjunction of all 48 possible coincidences The conditions under which two hits are meant to be in coincidence are defined as follows 77 p 55 e Az gt 6 scattering in forward direction e fmin Mor S fmax reasonable time of flight 42 4 CAMERA Proton Trigger direction ofbeam SIIL target Tre SE trigger Ring B Figure 4 4 Coincidence logic between the inner and the outer ring of the CAMERA detector For ev ery inner counter A there are two possible outer counters B and B 1 which have to be checked for corresponding hits Since protons can only be scattered in forward direction the lower limit for Az is principally 0 but to accommodate the finite position resolution of the counters an additional margin 6 is allowed The limits for the time of flight are chosen according to the CAMERA geometry and the detectable 6 range To be on the safe side this can be e g
26. 1 1 and is therefore preferred for the offline analysis On the other hand the calibration con stants obtained from the laser data allow to almost instantly see CAMERA proton signals in the online monitoring software Energy loss vs for plots like in Fig 7 6 are available for all counters of the CAMERA detector enabling the shift crew to verify the correct operation of the detector during the data taking Although time of flight corrections and energy calibrations are not applied in the online monitoring plots the proton band is clearly visible and the distri bution is comparable to the results obtained from the Monte Carlo simulations Fig 4 6 The horizontal band below the proton band at high pror is caused by the background from 6 ray electrons For the offline analysis some more calibration steps are required 136 7 Commissioning 4000 Ring A Ring B 3500 3000 2500 2000 1500 Ampup Ampgown ADC digits 1000 500 ULT TTTTTTTTT TTTTTTTTTTTTTTTTTTTTTTTT 0 6 0 8 1 0 Bror Bor o o N 4 e A Figure 7 6 Energy loss of particles crossing the CAMERA scintillators Ag and Bo versus their velocity Por Time of flight corrections and energy calibrations are not applied The plot is generated with the COMPASS online monitoring software COOOL for a run with 77 beam A time of flight correction is performed for each of the 48 scintillator combinations A Bj and Aj B i y Various me
27. 1 3 is performed to align the phase of the derived clocks with the reference clock In practice one of the Si5326A clock outputs is programmed identically to the ADC sampling clock on the GANDALF modules 505 44 MHZ to obtain the time unit which is necessary to interpret the time stamps of the digitized PMT pulses that are transferred via the VXS bus For this reason the TCS related part of the TIGER clock network is an exact replication of the correspondent GANDALF part 5 1 3 3 Free Running Clocks A Silicon Labs Si5338 102 clock generator chip is used on the TIGER module to produce four additional clock signals which are made available to the FPGA This I C programmable de vice is capable of generating arbitrary frequencies up to 350 MHz either locked to an external reference clock or in free running mode Its two stage synthesis architecture consists of a high frequency VCO 2 2 to 2 84 GHz and a PLL with fractional N feedback and output dividers From the wide range of supported single ended and differential output signal standards LVDS was chosen for the connection to the Virtex 6 FPGA On the TIGER board the Si5338 is operated in combination with a 25 MHz reference crystal in free running mode to generate the frequencies listed in table 5 2 The DDR3 clock and the User clock are fed into the FPGAS logic fabric while the Aurora and SFP transceiver clock signals are connected via AC coupling capacitors to the reference clock input
28. 70 dB up to 5 GHz Diameter 5mm 3 16mm Attenuation 0 42 dB m 600 MHz 0 66 dB m 600 MHz Table 7 2 List of electronic devices for the CAMERA readout Quantity Device type and purpose 1 VME64x VXS crate WIENER UEV 6021 with power supply UEP 6021 and fan tray UEL 6020 backplane Hartmann B18118233I 1 VME CPU MEN A20 to control the GANDALF modules 6 GANDALF transient analyzers 1 GS s 2 V dynamic range for the readout of the Ring A PMTs 6 GANDALF transient analyzers 1 GS s 4 V dynamic range for the readout of the Ring B PMTs 2 GANDALF transient analyzers 1 GS s 2 V dynamic range for the readout of the SciFi dynode signals 2 TIGER modules for trigger processor and readout concentrator tasks 1 HV power supply CAEN SY1527 for the CAMERA PMTs 1 Gbit Ethernet switch HP V1810 24G with SX LC Mini GBIC provides network connections for VME CPU fan tray TIGER mod ules HV supply 1 Readout PC with S LINK spill buffer card LDC43_1 in DAQ room 7 1 CAMERA Installation 129 7 1 1 Laser Calibration Runs The CAMERA scintillator tiles are equipped with a laser pulser for calibration purposes Short light pulses are coupled via optical fibers into the counters at their central z positions Just like the light from particles crossing the scintillators also the laser light is propagated to the PMTs at the upstream and downstream ends of the counters
29. GPDs Table 2 1 Spin 1 2 GPDs unpolarized polarized nucleon helicity conservation HIE Als nucleon helicity flip EPE Es 10 2 Theoretical Motivation I k Ik y q hard leptonic part N P te a N P Figure 2 6 Handbag diagram of the DVCS process IN I Ny The bubble represents the inner structure of the nucleon which here is described by GPDs Indicated in brackets are the four momenta of the respective particles The kinematic variables t A and are explained in 2 4 1 2 4 1 Kinematic Variables To describe the GPDs some additional kinematic variables are introduced The Mandelstam variable t P P 4 2 24 specifies the transferred four momentum between the initial and final state of the nucleon In the Bjorken limit the dimension less variable skewness 2 25 characterizes the direction of the momentum transfer t relative to the direction of momentum of the nucleon in the infinite momentum frame For 0 they are orthogonal for 4 0 the momentum transfer has a component parallel to the virtual photon The momentum fraction of the interacting quark in the initial and in the final state is given by the sum x and the difference x respectively Please note that x is the not measurable mean value of the quark s momentum fractions before and after the scattering process and may not be identified with the Bjorken xg GPDs are defined
30. H6 AP36 AP35 VXSPORT_4 lt 6 gt P3 E7 F7 AR35 AT35 VXSPORT_4 lt 7 gt P3 G8 H8 ACA ADAI VXSPORT_5 lt 0 gt P4 I5 J5 AB37 AB38 VXSPORT_5 lt 1 gt P4 K6 L6 AD36 AD35 VXSPORT 5 2 P4 17 J7 AE37 AD37 VXSPORT_5 lt 3 gt P4 K8 L8 AB39 AA40 VXSPORT_5 lt 4 gt P4 Il Jl AB32 AB33 VXSPORT 5 5 P4 K2 L2 AA42 AB42 VXSPORT_5 lt 6 gt P4 I3 J3 AA41 AB41 VXSPORT_5 lt 7 gt P4 K4 L4 AVA AUAI VXSPORT_6 lt 0 gt P3 H3 J13 AY42 BA42 VXSPORT_6 lt 1 gt P3 K14 Ll4 BA40 AY40 VXSPORT 6 2 P3 ns J15 BA41 BB41 VXSPORT_6 lt 3 gt P3 K16 L16 AP42 AR42 VXSPORT_6 lt 4 gt P3 I9 J9 AV4A0 AW40 VXSPORT_6 lt 5 gt P3 K10 L10 AT42 AU42 VXSPORT_6 lt 6 gt P3 11 J11 AT40 AU39 VXSPORT 6 7 P3 K12 L12 AC36 AB36 VXSPORT_7 lt 0 gt P4 A5 B5 AC40 AD40 VXSPORT_7 lt 1 gt P4 C6 D6 AE38 AD38 VXSPORT 7 2 P4 AT B7 AD42 AE42 VXSPORT 7 3 P4 C8 D8 AC35 AB34 VXSPORT_7 lt 4 gt P4 E5 F5 AC38 AC39 VXSPORT 7 5 P4 G6 H6 AC34 AC33 VXSPORT_7 lt 6 gt P4 E7 F7 AE40 AE39 VXSPORT_7 lt 7 gt P4 G8 H8 AR40 AT41 VXSPORT_8 lt 0 gt P3 A9 B9 AN41 AP41 VXSPORT_8 lt 1 gt P3 C10 DIO AN38 AP38 VXSPORT_8 lt 2 gt P3 All Bll AR39 AT39 VXSPORT_8 lt 3 gt P3 C12 D12 AM37 AM36 VXSPORT_8 lt 4 gt P3 E9 F9 AL37 AM38 VXSPORT_8 lt 5 gt P3 G10 H10 AN40 AP40 VXSPORT_8 lt 6 gt P3 Ell Fl AWA2 AWAI VXSPORT_8 lt 7 gt P3 G12 H12 AA35 Y35 VXSPORT_9 lt 0 gt P4 Al B1 W37 Y37 VXSPORT_9 lt 1 gt P4 C2 D2 Y40 Y39 VXSPORT_9 lt 2 gt P4 A3 B3 AA36 AA37 VXSPORT
31. PWR_EN must be deasserted before removing power System reset may be deasserted at the earliest 200 us after PWR_GOOD indicates stable power and 100 us after a valid PCIe reference clock is provided 134 p 39 The power up and reset sequence is controlled by the CPLD on the TIGER board in order to assure the specified timings and dependencies 5 2 5 AMD Radeon GPU Card Although the MXM 3 0 specification was released some years ago it is still hard to find MXM modules with state of the art GPUs on the market Only recently thanks to the increasing in terest in GPU computing for medical scientific and military applications the situation seems to improve slowly For the TIGER project an AMD based MXM card from WOLF 140 has been selected It features a Radeon E6760 embedded GPU running with up to 600 MHz and 1 GB of GDDR5 memory connected via a 128 bit 800 MHz interface The GPU contains 480 shader processing units providing a peak performance of 576 GFLOP s single precision float ing point operations The AMD Radeon E6760 embedded GPU supports OpenCL 141 for GPGPU computing OpenCL is a programming framework to perform general purpose computations on hetero geneous systems i e systems consisting of multi core CPUs GPUs and possibly also other processors like DSPs or FPGAs The C based language is used to write kernels which are exe cuted on the cores of the underlying hardware using the advantages of parallel computing and dyn
32. This poses a problem for the fast addition of wide binary numbers since the carry bit has to propagate through all stages until it is avail able in the most significant bit position Carry lookahead adders overcome this limitation by pre calculating whether a stage is going to propagate an incoming carry bit and using fast ded icated paths to transmit the carry information Each slice contains a four bit carry chain made of multiplexers and XOR gates running upward in the FPGA with direct carry in and carry out connections to the slices above and below The other inputs and outputs of the slices are con nected via a switch matrix to the general routing matrix of the FPGA Fig 5 4 The next higher hierarchical unit in the Virtex architecture is the configurable logic block CLB containing two slices In roughly half of the CLBs one of the slices is of the SLICEM type The LUTs in these slices can optionally be used as distributed 64 bit RAM or as 32 bit shift registers 6 advanced RISC Machine 5 1 Mainboard 57 LUT DD 06 a Do Inputs O5 L DMUX FF LAT D Qr DQ DX gt CE F7BMUX FF CLK dr ii LUT C 6 Inputs D 06 r De O5 DO CMUX FF LAT D Qr ca CX D CE CLK SR LUT B 6 Inputs D 06 DB O5 gt BMUX FF LAT D QDD BQ EXE F7AMUX CE CLK S
33. added by the physical layer At the receiving device the TLP is disassembled stepwise while moving upwards in the protocol stack DLLPs are exchanged between the data link layers of the two link partners for link management purposes e g flow control credit up dates and TLP acknowledgement DLLPs are always related to a direct connection between two devices they do not contain routing information and do not pass through switches Fi nally PLPs which are also called ordered sets in the specification are used during link initial ization and training and for clock tolerance compensation by regularly inserting SKIP ordered sets Like DLLPs the PLPs also do not contain routing information and do not pass through switches On top of the protocol stack the software layer contains the user application which generates outgoing and processes incoming requests and completions The transaction layer constructs out bound TLPs and puts them to the TX buffer and it pulls in bound TLPs from the RX buffer checks for errors based on the ECRC field and forwards the packets to the intended destination which can be either the software layer another port if the device is a switch or the configu ration management block Although not strictly part of the transaction layer according to the specification the management of the configuration space registers is often implemented at this level in PCIe IP cores to relieve the user of the burden to handle configuration
34. and Verification of a High Performance Electronic Readout Framework for High Energy Physics Ph D thesis ALU Freiburg Physikalisches Institut Albert Ludwigs Universit t Freiburg Bibliography 171 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 S SCHOPFERER 2009 Entwicklung eines hochaufl senden Transientenrekorders Diploma thesis ALU Freiburg Physikalisches Institut S BARTKNECHT H FISCHER ET AL 2010 Development of a 1GS s high resolution sam pling ADC system Nucl Inst amp Meth A 623 1 507 S BARTKNECHT H FISCHER ET AL 2011 Development and Performance Verification of the GANDALF High Resolution Transient Recorder System IEEE Trans Nucl Sci 58 4 1456 M BUCHELE H FISCHER ET AL 2012 The GANDALF 128 Channel Time to Digital Converter Physics Procedia 37 0 1827 M B CHELE 2012 Entwicklung eines FPGA basierten 128 Kanal Time to Digital Con verter f r Teilchenphysik Experimente Diploma thesis ALU Freiburg Physikalisches In stitut M B CHELE H FISCHER ET AL 2012 A 128 channel Time to Digital Converter TDC inside a Virtex 5 FPGA on the GANDALF module Journal of Instrumentation 7 03 C03008 C MICHALSKI 2013 Entwicklung eines Echtzeit Strahlprofil Monitoring Systems f r das COMPASS II Experiment Diploma thesis ALU Freiburg Physikalisches Institut T BAUMANN M B CHELE ET AL 2013
35. and interpreted as TCS commands which are 20 bits wide plus an optional 6 bit 110 6 Firmware and Software CLK Si2 505 44 MHz gt phase offset ok EN Counter gt ref clock ref clock 38 88 MHz Figure 6 2 Schematic diagram of the method to determine the alignment of the 505 44 MHz Si5326A output clock with respect to the 38 88 MHz COMPASS reference clock wide checksum The most common ones are the broadcast commands BC1 and BC2 162 p 18 BC1 is used to transmit the Begin of Spilland End of Spill signals together with the spill number BC2 contains the event type and event number Based on this information a so called event label is generated and written to a FIFO from where it is read out by the user application at a later time As mentioned earlier the Si5326A clock multiplier chip is utilized to reduce the jitter of the 155 52 MHz TCS clock and to generate some related clock frequencies for use in the TIGER FPGA These clocks are required to show a fixed phase relationship to the COMPASS reference clock Due to the operation principle of the Si5326A chip the phase offset between the input clock and the output clock is not defined per se This may be illustrated with the help of Fig 5 11 showing that the output clocks are obtained by dividing the high frequency clock of the digitally controlled oscillator fpco 5GHz This will result in a random phase offset depend ing on which clock edge
36. appendix D 128 7 Commissioning Twelve GANDALF transient analyzer modules with eight channels each were placed in the VXS crate in order to read out the 96 CAMERA channels together with two TIGER modules for the trigger processing and readout concentration tasks For readout tests in the clean area an au tonomous DAQ system consisting of a single PC and the TIGER internal TCS controller cf 6 1 3 was used Since there are several devices at the CAMERA detector which require a net work connection an Ethernet switch with optical uplink has been mounted next to the readout crate A list of all devices involved in the CAMERA readout is given in Tab 7 2 All in all the number of required connections which have to be established at the moment of installing the CAMERA detector in the experimental area is reduced to a minimum Besides a power connec tion and a multi fiber cable which is used to transmit Ethernet S Link and TCS signals only a coax cable is needed to transmit the proton trigger signal to the COMPASS trigger barrack which is located at the downstream end of the spectrometer Table 7 1 Parameters of the signal cables for the CAMERA detector Halogen free coax cables from Huber Suhner 182 are used PMT panel panel GANDALF Length 10m 1m Connectors BNC plug BNC jack BNC plug SMC jack Type H S Enviroflex 400 H S Enviroflex 316D Impedance 500 Signal delay 4 71 ns m Screening double braid gt
37. based on the trigger primitives received from the GANDALF transient analyzers and buffering the trigger attempt in the release logic cf 6 1 2 until a configurable time span has been elapsed since the event This latency setting is consequently adjusted in order to overlap the TIGER proton trigger signal with the other triggers In practice a latency scan was performed using a pion beam to bring the CAMERA trigger in coincidence with the beam trigger The latency of the TIGER release logic can be set in steps of 1 ns The effective trigger latency which includes some additional cable delay has been measured for different latency settings and is shown in Fig 7 4 The measurements a b and c are taken with latency settings of 500 ns 1500 ns and 700 ns respectively The result of a latency the exact value is 2 fapc 0 989ns 134 7 Commissioning scan is plotted in d where the latency setting was increased in steps of 200 ns starting at 800 ns The rectangular distribution arises from the fact that the TIGER trigger processor re leases triggers with a granularity of 1 ns 77 p 71 The non flat shape of the plateau is caused by the measurement setup which was used to determine the latency The trigger output signal of the TIGER module was connected to an additional GANDALF channel in order to obtain a time stamp ofthe trigger signal The latency was calculated as the difference between the time stamp of the trigger pulse an
38. bitstream files to the Flash is drawn from the CPU via the LPC bus to the CPLD which can likewise access the NOR flash interface and execute the write commands 5 1 6 Power Distribution System A stable and low noise power distribution system has been designed for the TIGER module taking into account the specific requirements of all the onboard components The primary power input of the board is defined by the VXS standard which specifies a 6 pin backplane connector providing a voltage of 5 V with a current of up to 30 A All other necessary voltages are generated by onboard power regulators 5 1 6 1 Hot Swap Capability An overview of the available power rails in Fig 5 17 shows that the power distribution net work consists of two branches which are turned on independently by dedicated hot swap controllers The standby branch is enabled as soon as the two VME injector ejector handles on the board s front panel are closed using a Texas Instruments TI TPS2420 hot swap con troller with an integrated MOSFET The rail VCC5V0_S5 is used for the standby power supply of the CPU module and two more rails 1 8 V and 3 3 V are generated by low dropout LDO linear regulators for powering the CPLD and Platform Flash XL devices and the controller for the switched mode DC DC converters cf 5 1 6 2 The power rails of the standby branch are indicated by the suffix S5 referring to the ACPI Soft Off power state The TPS2420 senses the load c
39. block diagram of the TIGER Base design implemented in the Virtex 6 FPGA The interface signals are indicated by black arrows and the clock signals are drawn in blue color input to the implementation tools which perform the steps translate map place and route in order to generate the physical layout The result is finally stored in a bitstream file to configure the device 6 1 1 TIGER Base Design A top level block diagram of the Base design is shown in Fig 6 1 The purpose of the Base de sign is to provide the user defined application with all the necessary interfaces global clocks and control signals The goal is to relieve the firmware designer who will write the actual user application of the burden to implement all the interface and board management logic as well as to ensure the compatibility between different user designs in terms of the communication protocols with the outside world Therefore several VHDL components are instantiated in the Base design next to the application module whose functionalities will be described in the fol lowing All clock inputs as well as I O signals which enter the Virtex 6 FPGA are connected to the top level entity as shown in the schematic drawing The Base designis developed in the form of a Xilinx ISE PlanAhead 106 project which includes a number of source files specifically atop level VHDL file containing the FPGA top level module 6 1 FPGA Firmware 107 VHDL files for the Ba
40. constant latency serial protocol for the up link allowing to synchronously transmit a reference clock and trigger signals to the front end cards and a 3 Gbit s high speed serial down link to receive detector data which has been dig itized on the front end cards A TDC front end card utilizing this optical connection is currently developed for the readout of the prospective THGEM chambers for the RICH 1 detector CAMERA Readout For the readout of the CAMERA detector 12 GANDALF transient analyzer modules are em ployed to digitize the 96 PMT channels with 1 GS s resolution An online pulse feature extrac tion based on a digital constant fraction discrimination dCFD is performed by the on board FPGAs yielding a significant reduction of the data volume which has to be transferred to the DAQ After the signal processing each PMT pulse is represented by a set of three values namely a time stamp an amplitude and an integral For a small fraction of all events the full wave form of the pulse is also sent to the DAQ which enables a monitoring of the feature extraction algorithm Thanks to the interpolation of the dCFD method 74 the timing resolution of the transient analyzers is much better than the 1 ns sampling but the actual value depends on the signal amplitude The digitized signal is represented by discrete samples with 4096 possible ampli tude values due to the 12 bit ADC Hence it is obvious that the interpolation is more precise the
41. controlled by the PCA9546A bus switch Sub bus Address Device Comment 58h Digital potentiometer COMe internal use only A0h Kontron JIDA EEPROM COMe internal use only 78h MI12832DO OLED display DOh Si5326A clock multiplier chip E2h Si5338 clock generator chip EEh PCA9546A 4 channel bus switch 0 98h MXM card GPU temperature sensor 1 A4h DDR3 memory DDR3 SO DIMM SPD 2 A2h SFP 1 module diagnostic interface 3 A2h SFP 2 module diagnostic interface 86 5 The TIGER Module LCLK og er ca OP ea USE Fg ES ESSE Smp LFRAMES Lf LAD 3 0 Figure 5 28 Typical I O read cycle on the LPC bus Additional wait states could be inserted during the SYNC phase by the peripheral until it is ready to send data The last START field in the diagram is already the beginning of the next transfer cycle An LPC interface requires only 7 signals a 33 MHz PCI clock LCLK a reset signal LRESET a control line LFRAME to indicate the start of a new cycle and a 4 bit wide multiplexed com mand address data bus LAD 3 0 The data transfer via the LPC bus takes place in so called cycles During a cycle the LAD 3 0 lines carry several fields each with a length of one or mul tiple nibbles cf Tab 5 9 Depending on the amount of data which has to be transferred a cycle takes a variable number of clock periods until itis completed The timing of an exemplary cycle is shown in Fig 5 28 Since the
42. distinct firmware configurations The TIGER trigger processor implemented the CAMERA trigger based on the trigger primitives received from the GANDALF transient analyzer modules It was utilized to generate first level triggers during the pion beam runs and the laser runs for calibration purposes During the muon runs this trigger was used in tagging mode to prove the trigger concept in a dedicated offline analysis The TIGER readout concentrator was employed to multiplex the event data of all 12 GANDALF modules into a single S LINK data stream for transmission to the DAQ system The TCS signal was also received and distributed within the VXS crate by this module Based on the TIGER first level trigger an online calibration of the CAMERA detector was per formed with a laser pulser system in order to obtain calibration constants for on the fly ap plications The derived calibration constants are in good agreement with the values obtained from a precise offline calibration using physics events The TIGER trigger processor made use of the online calibration constants during the treatment of the incoming trigger primitives The analysis of the data that has been acquired in the 2012 DVCS test run is currently in full swing and first results may be expected soon The performance of the CAMERA detector and its readout chain is evaluated thoroughly and improvements will be made where necessary The goal is to be well prepared for the forthcoming D
43. distribution of several clock signals for use throughout the FPGA design A 108 6 Firmware and Software Table 6 1 Assignment of the CPLD interface flags to the CA lt gt and CD lt gt bus bits Bits which are not listed here are available for future use Busrange Name and description of the flag CPLD to FPGA CD lt 17 0 gt status of the GANDALF boards CD lt i gt 1 VXS port i 1 is ready CA lt 22 21 gt 10 constant indicates that CA and CD signals are valid FM NORMAL CA lt 20 gt TigerFunction switch slot position which defines the module function Trigger or Readout CA 19 COMe_cb_resettt carrier board reset signal asserted by CPU at boot time CA lt 18 gt OtherTigerReady status of the other TIGER board in the crate CA lt 17 gt UserReset1 optional reset signal software controlled via CPLD driver CA lt 10 gt tcs 101 TCS receiver loss of lock from CLC016 CA lt 9 gt si lol Si5326A loss of lock CA lt 8 gt si_los Si5326A loss of signal FPGA to CPLD CA lt 7 gt tcs_rate rate select pin for the TCS receiver CA lt 6 gt si_inc Si5326A input to output skew increment pin CA lt 5 gt si_dec Si5326A input to output skew decrement pin CA lt 2 gt c2_startup_rst startup reset for clock domain 2 CA lt 1 gt ci startup rst startup reset for clock domain 1 CA lt O gt binfile mismatch indicates that the loaded design type does not match the module function
44. dual port Block RAM with a size of 4096 x 64 bits and BAR2 contains two FIFO data ports one in read and one in write direction FIFO control and status registers are located in BARO Analogous to the three address spaces the PCIe interface block exposes three types of communication ports to the user application user registers a configuration memory and a FIFO interface User registers are available in the TIGER Base design in two different forms the so called Static registers and Fast registers The Static registers are 32 bit registers that can be accessed by PCIe PIO read and write operations There are read only registers pcie rx regs i and write only registers pcie tx regs i The Static registers may be used in the design for arbitrary purposes However the register pcie tx regs 1 has a special function it controls the Fast registers which are 1 bit write only registers FastRegister i The Fast registers are a con venient way to manipulate control signals in the FPGA design They can be either set to 1 or 112 6 Firmware and Software 0 or they can be toggled for one period of the PCIe transaction clock A typical application for Fast registers is to enable disable or trigger a certain process in the FPGA design The configuration memory Configmem is located in the dual port BRAM at BARI One port of the BRAM is connected to the PCIe memory bridge enabling read and write access by the CPU The second port is co
45. for x 1 1 with negative values describing the momentum fractions for anti quarks The dependence of the GPDs on x and is mostly unknown today Fig 2 7 shows a model calculation for H x t 0 2 4 2 Relation of the GPDs to Known Distributions In the forward limit t 0 and 0 2 26 2 4 Generalized Parton Distributions 11 J Antiquark Distribution q x SS N SN RE NS SS AN N RR wee ATI RETIRE LER N i AZ ER LEEK N ER Lg N D N Me Figure 2 7 Model calculation for H x 0 from 25 The thick red line at 0 corresponds to the normal PDFs The outer region x gt is accessible via the DVCS process the four momentum and helicity of the nucleon remain unchanged during the scattering pro cess Hence the helicity conserving GPDs are related to the corresponding PDFs 26 forx gt 0 H x0 0 2q CO A x 0 0 AqrGO 2 27 H8 x 0 0 xg x H x 0 0 xAg x 2 28 forx lt 0 Hf x 0 0 qp x HA x 0 0 Agr 3 2 29 The helicity inverting GPDs are not related to known PDFs since the nucleon helicity flip re quires a transfer of orbital angular momentum which is possible only with finite transverse momentum transfer Furthermore there is a relation between the first moments of the GPDs and the elastic form factors of the nucleon 21 p 613 1 f dx Hf x amp t FI 1 2 30 1 1 f dx Ef x t FI 0 2 31 1 1
46. functions for read and write access to the three BARs of the PCIe endpoint interface The registers BARO can only be accessed by PIO operations while the Configmem BAR1 and the FIFOs BAR2 can be accessed by both PIO and DMA operations Two important extensions have been made to the TIGER class in order to allow an online FPGA reconfiguration A helper class TgCPLD was created to manage the access to the CPLD registers via the device file dev cpld0 It implements amongst others the functions getStatus to read the CPLD status register and configFPGA to reset the FPGA and reconfigure it via the SelectMap interface The TIGER class provides wrapper functions for these CPLD functions Two additional ioctl commands are implemented in the pciDriver which are used to call the kernel functions pci save state and pci restore state These are PCI bus service functions allowing to save the PCI configuration space of a device to a pci dev struct normally called before suspending the system and to restore it after wards The wrapper functions savePciState and restorePciState were added to the C interface of the pciDriver class PciDevice and to the TIGER class allowing to ex plicitly call the kernel save and restore functions They must be executed before and after the reconfiguration of the FPGA in order to retain the content of the PCIe configuration registers of the Virtex 6 integrated endpoint 6 2 4 JTAG Driver and Chi
47. in a more narrow way over a limited distance This is especially useful for the fan out region of the FPGA where the signal density is very high see Fig 5 23 Last but not least the vias that can be used for each net class are defined in the physical CSets For example vias with a larger di ameter may be required for power nets The most important physical constraints of the TIGER board are tabulated in Tab 5 5 Table 5 5 Physical constraint sets PCS defined for the TIGER board Single ended PCS Differential PCS Power net PCS PCB layer inner outer all all Line width 0 115 mm 0 140 mm 0 100 mm 0 140 mm DP gap 0 200 mm Line width neck 0 095 mm 0 120 mm 0 090 mm 0 100 mm DP gap neck 0 140 mm Via hole diameter 0 200 mm 0 200 mm 0 200 mm 0 300 mm Via pad diameter 0 500 mm 0 500 mm 0 500 mm 0 600 mm A spacing CSet finally defines the minimum distances between the different object types lines pins shapes vias holes on the PCB For each combination a separate constraint is defined i e line to line line to pin pin to pin and so on The minimal allowed values are specified by the PCB manufacturer but susceptibility to crosstalk must also be taken into account when selecting these values 5 1 7 3 Layout The netlist and the constraints from the previous two steps are exported into a board file for use with the Allegro PCB Editor Before the actual layout proces
48. in leading order given by the GPD H This allows one to constrain parametric models for the GPD H in the subspace x t at a given Q e An analysis of the sum 7 integrated over amp provides information over the transverse size of the nucleon in the COMPASS II x domain as it can be used to obtain the impact parameter dependent parton distributions as shown in section 2 4 3 e The dependent analysis of the difference Y provides the real part of the Compton form factor which is in leading order given by a convolution of the GPD H with a function describing the hard quark photon interaction Fig 2 12 shows the projected azimuthal distribution of the beam charge amp spin difference Y in one exemplary kinematic bin obtained by calculations using two different models as well as a fit to world data The predicted accuracy at COMPASS II for a running time of 280 days is also given in this plot 2 6 Beam Charge amp Spin Asymmetry at COMPASS II 17 o al E 160GeV 1 lt Q lt 4GeV 0 03 lt x lt 0 07 VGG Reggeized x t correlation 2 VGG Factorized x t dependence 2 e a Mueller fit on world data with JLab Hall A eee without JLab Hall A II a A TALI IE er a AA ISI AMAIA ARENA miim e SI A A A AL A A LAA AL BI LEALES o I A H ti o e a 1 Beam Charge and Spin Diff nb GeV rad o 8 3 3 8 3 S 3 3 al Figure 2 12 Projected statisti
49. it can run with a conventional Linux operating system Kontron provides a board support package BSP for the COMe mTT10 which contains board specific drivers and kernel patches for kernel 2 6 37 to support the mod ule s hardware 172 The BSP was integrated into a Fedora 14 distribution 32 bit and installed on the SSD of the CPU module To ease the software maintenance for multiple TIGER boards in the DAQ environment a diskless setup was created allowing to boot all systems from a single network source using PXE For this purpose an initramfs image was created with the dracut tool and the client file system was copied to a NFS share following the instructions in 173 6 2 Operating System and Device Drivers 121 Since the Fedora project is the basis for the Red Hat Enterprise Linux distribution which in turn is the basis for Scientific Linux the standard Linux distribution at CERN the Fedora operating system for the TIGER board was integrated into the COMPASS environment without difficulty The versions of Scientific Linux CERN SLC that are currently used for the COMPASS DAQ are SLC5 kernel 2 6 18 for the online and run control machines and SLCA kernel 2 6 9 for most of the frontend CPUs 6 2 1 Kontron Drivers The Linux BSP for the Kontron COMe mTT10 module includes a set of drivers for the on board PLD called the kempld driver package It consists of a core driver kempld core which pro vides the driver framework and is n
50. n 0 H8 MGT pcie x8 tx n 7 AE2 MGT aurora rx n l1 G6 MGT pcie x8 tx pl0 P3 MGT aurora rx n 2 F8 MGT pcie x8 tx pll Rl MGT aurora rx n 3 E6 MGT pcie x8 tx p 2 T3 MGT aurora rx pI0 H7 MGT pcie x8 tx p 3 Ul MGT aurora rx pll G5 MGT pcie x8 tx p 4 Wl MGT aurora rx pi 2 F7 MGT pcie x8 tx pl5 AAI MGT aurora rx pI3 E5 MGT pcie x8 tx pl6 ACI MGT aurora tx n 0 J2 MGT pcie x8 tx p 7 AEI MGT aurora tx n 1 H4 MGT sfp mgtclk n M7 MGTCLK aurora tx n 2 G2 MGT sfp mgtclk p M8 MGTCLK aurora tx n 3 F4 MGT sfpl_abs AL14 LVCMOS25 aurora_tx_p 0 jl MGT sfp1_los AL15 LVCMOS25 aurora_tx_p 1 H3 MGT sfpl rx n N6 MGT aurora tx p 2 Gl MGT sfpl rx p N5 MGT aurora tx p 3 F3 MGT sfpl tx en ARI8 IVCMOS25 come pcie rx n AG2 MGT sfp1_tx_fault ARI7 IVCMOS25 come pcie rx p AGI MGT sfpl tx n MA MGT come pcie tx n AD4 MGT sfpl tx p M3 MGT come pcie tx p AD3 MGT sfp2 abs AMI4 LVCMOS25 pcie sys clk n Y7 MGTCLK sfp2_los AN15 LIVCMOS25 pcie sys clk p Y8 MGTCLK sfp2_rx_n P8 MGT pcie_x8_rx_n 0 R6 MGT sfp2 rx p P7 MGT pcie x8 rx n 1 U6 MGT sfp2 tx en AU17 IVCMOS25 pcie x8 rx n 2 V4 MGT sfp2_tx_fault AT16 IVCMOS25 pcie x8 rx n 3 W6 MGT sfp2 tx n N2 MGT pcie x8 rx nl 4 Y4 MGT sfp2_tx p N1 MGT pcie_x8_rx_n 5 AA6 MGT pcie x8 rx n 6 ABA MGT Table B 1 Multi Gigabit pcie x8 rx n 7 AC6 MGT Transceiver MGT Signals pcie_x8_rx_p 0 R5 MGT pcie x8 rx pll U5 MGT pcie x8 rx pl2 V3 MGT pcie x8 rx pI 3 W5 MGT pcie x8 rx p 4 Y3 MGT pcie x8 rx p 5 AA5 MGT pci
51. namely the Fast Registers FR ReadoutTiger 100 5 The TIGER Module Table 5 16 Truth table to determine the state of the GANDALF boards on the basis of the handshake signals PP C PP D State 1 X no GANDALF board is placed in the slot or the design is not requesting TCS signals 0 0 a GANDALF board is placed in the slot but it is switched off 0 1 the GANDALF board in this slot is functional and requesting TCS signals Ready and FR_TriggerTigerReady Details about the Fast Register commands can be found in the GANDALF User Guide 152 Finally it should be mentioned that the GA0 signal is used by the CPLD to distinguish be tween the two switch slots in the VXS crate Slot A on the left hand side which is reserved for the TIGER readout concentrator is identified by GA0 0 and slot B on the right hand side which is reserved for the TIGER trigger processor is identified by GA0 1 The slot position is also announced to the FPGA in order to cross check whether the correct design type i e readout or trigger has been loaded 5 3 2 TIGER to TIGER Links Two high speed interfaces between the TIGER modules within a VXS crate are provided e Switch port SP4 on connector P2 is connected to FPGA user I Os in the same manner as the VXS payload ports It is used for a synchronous data transfer between the TIGER boards following the same protocol as described above This includes the forwarding of th
52. not used PWR LEVEL I OD signals the module to switch to a lower power state connected to the CPLD but not used PWR EN I CMOS assert to power on the module connected to the CPLD used for power sequencing PWR GOOD O OD indicates the status of the internal power regulators connected to the CPLD used for power sequencing Table 5 12 System management signal group of the MXM interface Signal Name 1 0 Type Description connected to WAKE I O OD wake up system from suspend or from soft off connected to the CPLD but not used VGA_DISABLE I OD PCI class code select pin for the GPU connected to the CPLD tied to GND to select non VGA device since the primary display adapter is lo cated on the COM Express CPU PRSNT_R O OD MXM module presence detect PRSNT_L connected to the CPLD used for power sequencing 5 2 CPU and GPU Extension Boards 89 Table 5 13 Power rails for the MXM module as provided by the TIGER board Name Voltage Current PWR SRC 12 5V 5A 5V0 5 0 V 2 5A 3V3 3 3V 2A The MXM card requires three power supply rails as listed in Tab 5 13 They may be powered up in any sequence as long as the PWR_EN signal stays low until all voltages are stable After the assertion of PWR_EN the modules internal power regulators ramp up and the PWR GOOD signal indicates that all rails are within the specified tolerance To shut down the module
53. o 26 28 o a offset in ns gt e 30 e 32 e e 34 36 26 t_ToF i i 1 offset offset in ns 28 30 2 32 e 2 e e gt e e A 34 counter 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Figure 7 9 Comparison of the calibration results obtained with the laser method and the CAMERA Helper method The time difference and time of flight calibration offsets are plotted for each counter counter combination Both methods are in good agreement t diff A offset deviation t diff B offset deviation t ToFiii offset deviation t_ToF i i 1 offset deviation 5r offdiff0 8 offdiff1 6 offdiff2 5 offdiff3 Entries 24 7 Entries 24 5 Entries 24 i 24 4r Mean 0 1258 6 Mean 0 4058 Mean 0 2722 4 0 2666 RMS 0 168 5 RMS 0 5028 4 RMS 0 2564 0 2617 4 3 2r 2 3 2 tr 2 1 1 doll ern EA AL 1 0 1 2 1 0 i 2 1 0 1 2 1 0 1 2 off laser off CH in ns off laser off CH in ns off laser off CH in ns off laser off CH in ns Figure 7 10 Deviations between the calibration offsets obtained from laser data and physics data When excluding the deficient counter B11 from the ud B distribution the mean is reduced to 0 33 ns with an RMS of 0 36 ns 7 2 Physics Data 141 1 2 A Ep a u 0 8 0 6 0 4 0 2 Entries 140 42 Bor Figure 7 11 Energy loss AEg of protons in the Ring B scintillators of the CAMER
54. of the CAM ERA detector The TIGER module which constitutes the hardware platform for the new trigger system was designed to operate within this framework Its development is an integral part of this thesis and is described in detail in chapter 5 4 1 Detector Principle Since the detector has already been described in section 3 2 2 its features will only be briefly summarized Two barrels of scintillating material each segmented in 24 slats are surrounding the target The elements of Ring A are located in a distance of 26 cm from the beam axis and the Ring B segments are placed 115 cm away from the beam axis Each slat is read on both ends with PMTs and the signals are digitized and processed by GANDALF modules For each digitized pulse a set of features is extracted containing the pulse arrival time the maximum amplitude and the pulse integral The following nomenclature is used to identify the readout channels of the detector X e where X A B specifies the ring inner or outer i 0 1 23 specifies the index of the element and e up dn specifies the end of the counter upstream or downstream A scintillator is referenced by the name X When dealing with a particular counter the index i may be omitted for easier notation 4 1 1 Time of Flight Measurement In Fig 4 1 one segment of the CAMERA detector with two facing elements is shown to illustrate the measurement principle A particle originating from the ta
55. power button to wake it up from the ACPI soft off state Should one of the handles be opened again during the start up phase power is switched off instantly When the state running is reached the behavior of the VME handles changes insofar as opening one of them will trigger a controlled shut down process which is indicated by flashing the Power LED on the front panel If the second switch is opened before the shut down is completed i e the user did not wait patiently power is disabled instantly In this way it is guaranteed that all on board power is certainly off when the board is ejected from the VXS slot 5 1 Mainboard 75 state off state_running entry disable all hot swap contr d entry signal pwr ok to CPU both switches closed switch opened switch opened state_startup state_shutdown entry enable standby hot swap contr entry press CPU power button AE CPU shutting down state_s2 entry press CPU power button state_s8 exit release CPU power button entry release CPU power button CPU exits suspend ceu halted state_s3 state_s9 entry enable main hot swap contr entry stop DC DCs and LDOs main pwr_good power rails off state_s4 state s10 entry start DC DC converters entry disable main hot swap contr d UCD pg amp amp LTM pg both switches opened state s5 LDO pg entry enable
56. produced by etching 54 Fused silica window 70 Wires Csl Pere reer rrr TITTI TIT ICI CITI TI TT ITITI TITTI TITTI TTI CITI II TI IIITIT III CITI IITITITT ITTICA TITTI TITI TITTI CITI LIITA Drift Field 5 mm THGEM 1 E EN EN E HE E HE DN Transfer Field ji 3 mm THGEM 2 BEE E E E E E E mE gu Transfer Field 1 3 mm THGEM 3 ZI n arcs SI A um Induction Field 1 3 mm Anode with pads we Figure 3 10 Structure of a THGEM based photodetector 53 3 4 The Trigger System 31 3 4 The Trigger System With a beam intensity of up to 3 8 10 muons per spill it is obvious that not all events can be recorded Therefore it is essential to use a trigger system that is able to identify the events which are possibly interesting for the analysis Due to the limited buffer memory of the front end electronics the time available for a trigger decision is typically in the order of 1 us Sev eral trigger types can be combined to form the COMPASS first level trigger which is then dis tributed via the Trigger Control System TCS 56 to all readout modules The TCS signal consists of two channels which are time division multiplexed encoded with a biphase mark code and transmitted by means of a unidirectional optical fiber network Chan nelAis used to transmit a 1 bit first level trigger information only thus providing a fixed latency and low dead time All other information is transmitted consecutively on channel B including ev
57. proton Nuclear Physics B 337 3 509 COMPASS COLLABORATION V Y ALEXAKHIN ET AL 2007 The deuteron spin dependent structure function gt and its first moment Physics Letters B 647 1 8 COMPASS COLLABORATION M ALEKSEEV ET AL 2010 Quark helicity distributions from longitudinal spin asymmetries in muon proton and muon deuteron scattering Physics Letters B 693 3 227 M BURKARDT C A MILLER W D NOWAK 2010 Spin polarized high energy scattering of charged leptons on nucleons Reports on Progress in Physics 73 1 016201 168 Bibliography 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 A W THOMAS W WEISE 2001 The Structure of the Nucleon chap Deep Inelastic Lep ton Scattering Wiley VCH Ist edn J C COLLINS D E SOPER G STERMAN 1989 Factorization of Hard Processes in QCD In Perturbative QCD Adv Ser Direct High Energy Phys vol 5 edited by A H MUELLER World Scientific Publishing F HALZEN A D MARTIN 1984 Quarks amp Leptons An Introductory Course in Modern Particle Physics John Wiley amp Sons C G CALLAN D J GRoss 1969 High Energy Electroproduction and the Constitution of the Electric Current Phys Rev Lett 22 156 J BERINGER ET AL 2012 Review of Particle Physics Phys Rev D 86 010001 U ELSCHENBROICH Diagram 01 105 http www hermes desy de notes pub trans public s
58. quarks of the nucleon are surrounded by a large number of virtual sea quarks with mainly very small momentum fractions Because the resolution of a virtual photon increases with Q one detects more sea quarks at higher Q Consequently more partons at small xg and fewer partons at larger xg can be found with increasing Q 12 The Q dependence of the parton distributions is suppressed in the following notation since it is not essential for the further considerations 2 3 2 Longitudinal Polarized PDFs Besides the momentum distribution qp also the helicity distribution Agy of the quarks can be measured with DIS Longitudinal polarized leptons i e leptons with helicity along or opposite to their momentum are scattered off nucleons which are polarized parallel gt or anti parallel lt to the momentum of the incident leptons The virtual photon which inherits the helicity 2 3 Parton Distribution Functions x 0 00005 Proton x 0 00008 x 0 00013 H1 ZEUS BCDMS x 0 0002 ui x 0 00032 NMC e X 0 0005 SLAC eee put eo x 0 0008 e ae x 0 0013 x 0 0032 eee x 0 005 DE u 90 00009 000000008 x 0 008 XJ ese oe 090 eee x 0 013 oo 00 000 09 909 x 0 020 puree ep one 0000 000 00 000 00909988 x 0 032 gig E EP ee s ug g Paca E e 00 000 00 00000 e x 0 08 s an magnis AAA sara n Cleaner oos Ves 6e A A ar 00000 arenoso A dts Ibl RO A EH REB ca de 000 00
59. requests The flow control of TLPs is based on credits which indicate how much buffer space is left on the receiving side of a link The data link layer is responsible for the data integrity of transferred packets Upon reception ofa TLP itis checked for errors based on the LCRC field and itis either acknowledged by sending an ACK DLLP or the packet is requested again by sending a no acknowledge NAK DLLP When the transmitting device receives a NAK it will send the TLP again from its replay buffer A TLP 43 Cyclic Redundancy Check 44 Tp fact up to eight virtual channels may be defined on a link each comprising dedicated RX TX buffers and credit information This allows to implement multiple traffic classes with different priorities 92 5 The TIGER Module User application Software layer generates receives requests and completions TLP generation reception Transaction layer TX RX buffer flow control configuration management SE LCRC Sequence ID Data link layer sy ACK NAK protocol TLP replay buffer Physical layer link training and status de scrambling logical PIPE interface PCS 8b 10b encode decode se elastic buffer E amp SERDES electrical analog buffers Figure 5 30 PCIe devices are divided into several layers each responsible for a specific part of the com munication The physical layer can be further divided into sublayers In the FPGA the connection between the MAC layer of the PCIe IP cor
60. set to 0ns 80 ns 4 2 2 2 Time Calibration The description of the time measurement in 4 1 1 has not covered the need for calibrations Actually the time stamps that are measured by the GANDALF module have an individual offset for every channel due to differences of the effective speed of light in the scintillators as well as due to different lengths of the light guides and cables and individual transit times of the PMTs This is calibrated by means of the CAMERA laser system Using the optical fibers which are connected to the center of each scintillator a laser pulse is coupled into all slats simultaneously The individual time offsets are then adjusted such that zai 0andzg j 0 Vie 0 1 23 4 12 ttor Aj gt B 0 Vi j 0 1 23 4 13 4 2 Trigger Concept 43 Entries 5719629 10 Mean 70 28 104 E RMS 93 42 0 400 500 600 700 800 Kinetic energy of the protons at the vertex MeV Figure 4 5 Distribution of the kinetic energy of protons at the vertex from DVCS events which have been generated with HEPGen 76 p 85 These are not reconstructed events and no detector response is simulated for this plot Dedicated calibration runs have been taken with the laser system during the commissioning phase of the 2012 DVCS test run From this data 1 offsets for the individual channels have been determined cf 7 1 1 They are needed for an online correction of the time stamps in the trigger module For the offline
61. signals of the CAMERA detector as well as the dynode signals of the SciFi stations FIO1 and FI02 are digitized by GANDALF transient analyzers The data flow emanat ing from the GANDALF modules is assigned to two different sub networks according to the specified tasks trigger generation and detector readout Two TIGER modules with different firmware configurations are forming the crosspoints of the respective sub nets The trigger sub network depicted on the left hand side of the diagram feeds the pulse infor mation the so called trigger primitives to a first TIGER module which is configured to act as a trigger processor To cope with the high hit rate of up to several MHz in single detector chan nels and the low latency requirement for the trigger generation a custom protocol cf 4 3 2 has been designed for the transmission of the trigger primitives The trigger processor operates on this information by means of its high performance FPGA device in order to identify proton 3Nuclear Instrumentation Module 4 3 Electronics Framework 47 CAMERA SciFi GANDALF GANDALF trigger primitives TIGER Trigger Readout Processor Concentrator S Link Ethernet trigger COMPASS Trigger Figure 4 8 Application ofthe TIGER modules for the CAMERA readout and proton trigger candidates and make a trigger decision The functionality of the FPGA firmware is described in section 6 1 2 The CAMERA trigger signal is f
62. the adder subtractor can be fed back via the Z multiplexer to act as an accumulator The specific operation of the DSP slice is dynamically selected by control signals OPMODE ALUMODE The adder subtractor can also be configured as a logic unit to implement bit wise functions or pattern detection Within each input path and behind the multiplier there are additional pipeline registers which can be used to increase the maximum possible clock frequency at the expense of some latency They are denoted with the red rectangles in Fig 5 8 DSP algorithms like finite impulse re sponse FIR filters with several stages are implemented by cascading multiple DSP48El1 slices in a column For this purpose cascadable input and output streams are available between the pipeline registers of adjacent DSP slices to avoid the use of general logic and routing resources 5 1 1 5 Inputs and Outputs Most of the I O pins of the Virtex 6 FPGA are configurable by the designer to match the elec trical signal standard of the interfacing components These are the so called User I O pins in contrast to the power and ground pins and a few configuration pins which serve a dedicated purpose A large number of both single ended and differential standards is supported The pins are grouped in banks of 40 I Os each with additional dedicated Vcco supply voltage pins to power the output buffers Since different standards may require different supply voltages this poses some con
63. the division counter is started However the input to output skew is controlled by the INC and DEC pins 100 p 60 in steps of 1 fpco allowing to align the Si5326A outputs with the COMPASS reference clock provided that the output frequency is an integer multiple of the reference frequency The alignment process also called Si sweep is an iterative process of shifting the skew value until a minimum phase offset is achieved 163 In each step the actual alignment of the output clock and the reference clock is statistically determined as shown in Fig 6 2 Two copies of the high frequency output clock are routed through IDELAY elements of the FPGA in order to delay one of the signals with respect to the other The delay value should approximately equal the skew increment step of the Si5326A Both signals are then registered by flip flops using the lower frequency reference clock In the case that the edges of both clocks are correctly aligned to each other a 1 will be seen on the non delayed branch but on the delayed branch a 0 is still visible since the edge has not yet propagated through the delay Therefore a 1 at the output of the AND gate will indicate a phase match Finally a counter is used to determine the degree of alignment for every skew increment step in order to detect the optimal setting 6 1 FPGA Firmware 111 board to host Virtex 6 integrated PCle block for PCle RX TX Gen1 x1
64. 0 mA through any pin in a powered or unpowered bank A total limit of 100 mA per bank 150 p 2 applies to prevent damage to the device If there is a low impedance path from VCCO to GND excessive current flow could lead to device failure 151 Therefore the FPGA output buffers which drive the VXS signals to the backplane are enabled only after the link partner has indi cated that it is ready to receive data The handshake signals PP_C n and PP_D n are connected to the Virtex 5 FPGA on the GAN DALF side and to the CoolRunner II CPLD on the TIGER side and they are pulled to 3 3 V by weak pullup resistors on the VXS backplane When a GANDALF module is inserted into a slot connected to port n and while its FPGA is still unpowered the corresponding PP_C n and PP_D n signals are forced to GND due to the clamp diodes of the Virtex 5 As soon as the FPGA is powered the I Os enter a high impedance state and as a result the handshake signals are pulled again to 3 3 V When the FPGA firmware is finally loaded it may drive PP_C n low to request TCS signals The TIGER board detects the state of the GANDALF boards based on Tab 5 16 On the other hand the CPLD on the TIGER module is driving PP_D n low as long as the Virtex 6 is not powered up and configured which causes the GANDALF FPGA to tri state its VXS out puts Furthermore there are two software commands which have to be issued to the GAN DALF board in order to enable the VXS outputs
65. 000000 nn TI Bir de ee o Pi e 000 00 000 0000909 amp x 0 05 x 0 13 x 0 18 x 0 25 x 0 4 da ML E tejo or F Q GeV Figure 2 3 Proton structure function F xp Q measured in electromagnetic scattering of leptons on nucleons by various collider and fixed target experiments For better visibility F is multiplied by 2 for the different xg bins i is the number of the xg bin from 1 xg 0 85 to 24 xp 5 1075 Combined statistical and systematic errors are shown by the vertical bars 17 p 241 Figure 2 4 Scattering of longitudinal polarized leptons off protons with same or opposite polarization The virtual photon can only be absorbed by a quark with opposite helicity 18 8 2 Theoretical Motivation EMC E142 E143 SMC HERMES E154 E155 JLab E99 117 COMPASS CLAS 4 vX oe otur Deuteron H sg x rot ee Neutron from He 10 10 10 19 1 Figure 2 5 Spin dependent structure function xg x of the proton deuteron and neutron measured in DIS of polarized leptons by various experiments Combined statistical and systematic errors are shown by the vertical bars 17 p 247 of the lepton to a certain extent depending on the lepton kinematics can only be absorbed by a quark with helicity opposite to the photon helicity Fig 2 4 Thus quarks with a certain helicity can be selectively probed by choosing the nucleon polari
66. 1 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 di 42 ue B EB de Uni EN m i l C Jr ea CS rmm mm or vs ay m mom m 3 4r iL Jr Jr Fe ee ee el we ae ee ah H ll H al al HH eje vie do enm RR RR Rm Re ae eh ER RR E FH RR Re e T ae te e ee e u e ES e EE Eee eee b ele ele ele ele ele lel ele eee ee ele ele eee le Eee etl ele elle lel el lel Eee aco doc e GTX transceiver mu ES Aurora backplane SFP transceiver E PCI Express f r e o User I O EUG Fee e tjt tt aH 0 BE NH ig de HN E LVDS front I O SFP control E Clock inputs de F t ER ERR IRR M x cot I E DDR3 SO DIMM E Configuration port I VXS backplane DOGOODO D Figure 5 9 Package pin plan for the TIGER FPGA The association of the pins to the various interfaces is indicated by the color code 5 1 1 6 Gigabit Transceivers For fast serial data transmission the Virtex 6 FPGA is equipped with GTX transceivers i e pairs of transmitter and receiver capable of data rates up to 6 6 Gbit s Transmitter and receiver are basic
67. 1 50 1 25 3 1 00 0 75 0 50 0 25 T T 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 Time ms Figure 5 19 The normal ramp up and ramp down sequences for the four power rails which are con trolled by the UCD9246 The FPGA core voltage is ramped up first followed by the FPGA I O voltages as soon as VCCINT has reached the power good threshold The power down is performed in reversed order ALinear Technology LTM4605 DC DC uModule regulator 115 is operated in boost mode to provide a 12 5 V supply for the MXM GPU card ATI TPS51200 linear sink source regulator is used to provide the termination voltage for the DDR3 SO DIMM and three LDO regulators TI TPS74401 are employed to generate low noise analog supply voltages for the Virtex 6 multi gigabit transceiver MGT blocks The linear regulators are connected to the 1 5 V rail for lowest possible power dissipation and heat production 5 1 6 3 Power Up Power Down Sequence The CPLD is responsible for the TIGER board s start up sequence which involves enabling the power regulators configuring the FPGA and booting the CPU in a coordinated manner No less important is the shut down sequence before the module can be safely removed from the crate A state diagram of the implemented start and stop behavior is shown in Fig 5 20 The start up process is executed as soon as both VME handles are closed State s2 emulates a push of the CPU s
68. 1149 1 2001 International Rectifier 2011 IRF6718 DirectFET Power MOSFET Data sheet Texas Instruments 2010 UCD9246 Digital PWM System Controller Data sheet Texas Instruments 2010 PTD08A010W and PTD08A020W Digital PowerTrain Module Data sheets Xilinx Inc Xilinx Power Estimator XPE http www xilinx com products design tools logic design xpe htm Linear Technology 2007 LTM4605 High Efficiency Buck Boost DC DC uModule Regula tor Data sheet Cadence Design Systems Inc Allegro PCB Designer Release 16 5 http www cadence com products pcb pcb design M MONTROSE 2000 Printed Circuit Board Design Techniques for EMC Compliance A Handbook for Designers john Wiley amp Sons 2nd edn J KNIGHTEN B ARCHAMBEAULT ET AL Summer 2006 PDN Design Strategies III Planes and Materials Are They Important Factors in Power Bus Design IEEE EMC Society Newsletter 210 58 174 Bibliography 119 Xilinx Inc Device Models http www xilinx com support download 120 Signal Integrity Software Inc What is IBIS AMI http www sisoft com elearning ibis ami html 121 PCI Industrial Computer Manufacturers Group 2010 COM Express Module Base Specifi cation PICMG COM 0 R2 0 122 PCI Industrial Computer Manufacturers Group 2009 COM Express Carrier Design Guide 123 Kontron Europe GmbH 2012 COMe mTT10 User s Guide Rev 120 124 SD Card Association 2006 SD Specifications Part 1 Physical Layer Simplified Spe
69. 16 j phpro 2012 02 504 M B chele et al JINST 7 03 C03008 2012 doi 10 1088 1748 0221 7 03 C03008 T Baumann et al JINST 8 01 C01016 2013 doi 10 1088 1748 0221 8 01 C01016 M Alexeev et al Nucl Instr and Meth A 695 159 162 2012 doi 10 1016 j nima 2011 11 079 M Alexeev et al Nucl Instr and Meth A 732 264 268 2013 doi 10 1016 j nima 2013 08 020 M Alexeev et al Nucl Instr and Meth A Development of a Thick GEM based detector for single photon detection to be published 2013 Contents 1 Introduction 2 Theoretical Motivation 2 1 2 2 2 3 2 4 2 5 2 6 TheNucleonSpin ori dev s eR em UR A a ic UE Rs Deep Inelastic Scattering s sa roo 22cm onen Parton Distribution Functions sasaaa aa 2 3 1 Unpolarized PDES ll 2 lee e an 2 8 2 Longitudinal Polarized PDFs LL Generalized Parton Distributions LL 2 4 1 Kinematic Variables LL 2 4 2 Relation of the GPDs to Known Distributions 2 4 3 Impact Parameter Dependent Parton Distributions 2 4 4 GPDs and the Nucleon Spin Structure cn nennen Deeply Virtual Compton Scattering 0 0 00 ee eee eee Beam Charge amp Spin Asymmetry at COMPASS IL LL 3 The COMPASS II Experiment 3 1 3 2 3 3 3 4 3 5 THE Beam 4 pi AA REESE uw The Target Region iii doi g l 2222 AE BG ERS ae eek 3 2 1 The Liquid Hydrogen Target o on onen 3 2 2 The Recoil Proton Detector
70. 169 T GRUSSENMEYER Diploma thesis in preparation ALU Freiburg Physikalisches Institut Private communications 170 CERN HSI S LINK Products http hsi web cern ch hsi s link products html 171 S HAAS 2012 HOLA LSC implementation with on chip SERDES for Virtex 6 CERN PH ESE 172 Kontron Europe GmbH 2011 Linux BSP amp KEAPI for COMe mTT10 http emdcustomersection kontron com 173 JLAB Data Acquisition Group 2012 CentOS6 Linux Diskless Setup https coda jlab org wiki index php CentOS6_Linux_Diskless_Setup 174 J CORBET A RUBINI G KROAH HARTMAN 2005 Linux Device Drivers O Reilly 3rd edn 175 A K PUGALIA 2011 Device Drivers Part 9 I O Control in Linux http www linuxforu com 2011 08 io control in linux 176 G MARCUS 2006 Linux pciDriver Lehrstuhl f r Informatik V Universit t Mannheim Bibliography 177 177 178 179 180 181 182 183 184 185 G MARCUS 2011 MPRACE Software Library Institut f r Technische Informatik Heidel berg http li5 ziti uni heidelberg de mprace J PECH 2011 Re Cant get the Xilinx cable drivers installed on SL6 1 RHEL 6 1 http newsgroups derkeiler com Archive Comp comp arch fpga 2011 09 msg00047 html AMD Staff 2011 OpenCL and the AMD APP SDK http developer amd com resources documentation articles articles whitepapers Solomon Systech 2010 SSD1306 OLED PLED Segment Common Driver with Controller L FRIED
71. 2 7 s oME M2 2 8 w y 2Mv Q 2 9 where E and E are the energy of the incoming and of the scattered lepton and 0 is the polar angle of the lepton scattering process In addition one defines the dimension less quantities 2 2 Xp a lab Ea Bjorken scaling variable 2 10 2P q 2Mv Pq lab Y fractional loss of the lept 2 11 ractional energy loss of the lepton 3 UTE E i The distribution of cross sections in dependence of the Bjorken variable xg is experimentally accessible In the infinite momentum frame xg can be interpreted as fraction x of the nucleon s four momentum which is carried by the struck quark before the virtual photon is absorbed 12 p 3 The xg distribution hence gives information about the momentum distribution of the quarks in the nucleon 2 3 Parton Distribution Functions In the Bjorken limit Q v 00 Xp const 2 12 the DIS process Fig 2 2 can be separated into a hard leptonic and a soft hadronic part 14 15 p 183 do ana E Eu WP 2 13 dQdE Qt E m The leptonic tensor Luv which describes the scattering of the virtual photon off a quark can be calculated in QED while the hadronic tensor W can only be parametrized by parton dis tribution functions PDFs 6 2 Theoretical Motivation 2 3 1 Unpolarized PDFs The unpolarized parton distribution functions q xg Q are defined as the number densities of partons of type f with momentum fraction xg in the nu
72. 2012 S D1306 OLED Driver Library Adafruit Industries https github com adafruit Adafruit SSD1306 HUBER SUHNER 2012 RF cables catalog http www hubersuhner com COMPASS COLLABORATION PHysics Analysis Software Tools http ges home cern ch ges phast index html A FERRERO Exclusivity Selection with CAMERA Presentation at the COMPASS GPD Meeting June 25th 2013 at CERN P MALM 2013 Myoproduktion exklusiver p Mesonen an Protonen Master s thesis in preparation ALU Freiburg Physikalisches Institut Private communications 178 Bibliography
73. 3 K13 IVDS IN 13 L 14 48 M16 N15 IVDS IN 14 L 15 49 L17 MI7 IVDS IN 15 L 16 50 Bl4 Cl4 LVDS_IN lt 16 gt L 17 51 C13 D12 IVDS IN 17 L 18 52 B18 A19 LVDS_IN lt 18 gt L 19 53 Al7 B17 LVDS_IN lt 19 gt L 20 54 H18 G18 IVDS IN 20 L 21 55 F17 G17 IVDS IN 21 L 22 56 K18 J18 IVDS IN 22 L 23 57 M18 N18 LVDS_IN lt 23 gt L 24 58 M13 N13 LVDS_IN lt 24 gt L 25 59 J15 K15 LVDS_IN lt 25 gt L 26 60 G19 F19 IVDS IN 26 L 27 61 C19 B19 IVDS IN 27 L 28 62 L16 L15 IVDS IN 28 L 29 63 K17 J17 IVDS IN 29 L 30 64 N16 P16 LVDS_IN lt 30 gt L 31 65 E19 E18 IVDS_IN lt 31 gt L 32 66 GND L 33 67 GND L 34 68 162 C Connector Pin Out FPGA Pins Signal Name Connector Pins z Conn AJ21 AJ20 LVDS_OUT lt 0 gt R 1 35 AL20 AL21 IVDS_OUT lt 1 gt R 2 36 AJ17 AKI7 LVDS_OUT lt 2 gt R 3 37 AK20 AK19 LVDS_OUT lt 3 gt R 4 38 AL19 AMI9 LVDS_OUT lt 4 gt R 5 39 AN16 AMI6 LVDS_OUT lt 5 gt R 6 40 AK15 AKI4 LVDS_OUT lt 6 gt R 7 4l AJ16 AJ15 LVDS_OUT lt 7 gt R 8 42 AN20 AP20 LVDS_OUT lt 8 gt R 9 43 AU21 AT21 LVDS_OUT lt 9 gt R 10 44 AMI7 AMI8 LVDS_OUT lt 10 gt R 11 45 AP21 AP22 LVDS_OUT lt 11 gt R 12 46 AP16 API7 IVDS OUT 12 R 13 47 AN18 ANI9 IVDS OUT 13 R 14 48 AT22 AR22 LVDS_OUT lt 14 gt R 15 49 AM21 AN21 LVDS_OUT lt 15 gt R 16 50 AV23 AU22 LVDS_OUT lt 16 gt R 17 51 AV18 AV19 IVDS OUT 17 R 18 352 AP18 ARI9 LVDS_OUT lt 18 gt R 19 53 AY18 AWI18 LVDS_O
74. 5 1 1 1 Configurable Logic lees 56 5 1 1 2 Clock Management eee 57 51 13 Block RAM ia SERM WE aan near 59 5 1 1 4 DSP Sliees 4 2 2 822 PARA ARE evi 60 5 1 1 5 Inputs and Outputs LL 60 5 1 1 6 Gigabit Transceivers llle 62 9 1 2 DDRS M emoEy ed cete ed EE st ne pe SR 63 5 1 3 Clock Distribution Network llle 64 5 1 3 1 PCI Express Reference Clock 64 5 1 3 2 Experiment Synchronous Clocks 65 5 1 3 3 Free Running Clocks o ooo nen 66 5 1 4 CoolRunner II CPLD is 2 leere 67 5 1 5 Configuration Scheme o oo o 68 5 1 5 1 Configuration at Power Up o ooo o 68 5 1 5 2 Online Configuration o o e e 69 5 1 5 3 JTAG Configuration and Flash Programming 70 5 1 6 Power Distribution System 0 002 eee eee ee 71 5 1 6 1 Hot SwapCapability llle 71 5 1 6 2 Secondary Voltage Rails llle 72 5 1 6 3 Power Up Power Down Sequence 74 5 1 6 4 Power Distribution System Summary 75 5 1 7 PCB Layout u a aaa ara Uy be RR RUP E a RA 76 5 1 7 1 Sehematie wu sa is res A EN g ap dtd 76 5 1 7 2 Constraints a OL URS eR A REPE EE a a 77 Thra Layout ae t t an Bi a Dead 78 5 1 7 4 Simulation pur pere a a Der RR 79 5 2 CPU and GPU Extension Boards o e 82 5 2 1 COM Express Module
75. 5 3 3 1 5 2 5 VCC5VO 5 0 33 0 VCCINT 1 0 20 0 15 0 VCCIV5 1 5 20 0 15 0 VITDDR 0 75 3 0 3 5 MGTAVCC 1 0 6 0 7 6 MGTAVTT 1 2 3 0 3 8 VCC2V5 2 5 10 0 6 0 VCC3V3_S0 3 3 10 0 6 0 VCC GPU 12 5 4 0 5 0 currents through the LDO rails The main hot swap controller limits the current of the VCC5VO rail to 33 A which includes the current to the DC DC converters as well as the load on the 5 V rail itself The total power consumption of the board s main power branch may therefore be up to 165 W 5 1 7 PCBLayout The TIGER board has been developed using the Cadence Allegro PCB Designer 116 a powerful EDA suite which provides tools for all steps of the PCB design process The main tasks during the PCB development of the TIGER module will be detailed in this section These are to capture the schematic diagram of the circuit to define constraints tolayout the board to perform simulations of the signal and power integrity 5 1 7 1 Schematic The schematic diagram is captured using the Allegro System Architect This tool allows enter ing circuit diagrams using various methods The spreadsheet method is particularly useful for defining connections between devices with a large number of pins in a tabular form This cov ers for example the buses between the FPGA and the different connectors on the TIGER mod ule Most of these buses are direct connections possibly with some additional discrete com ponents involved
76. 5_T DCI ca 6 AW15 IVCMOS25 ddr3 dq 57 A22 SSTL15 T DCI ca 7 AY15 IVCMOS25 ddr3 dq 58 G21 SSTL15 T DCI ca 8 ARI3 IVCMOS25 ddr3_dq 59 J20 SSTL15_T_DCI ca 9 AP13 IVCMOS25 ddr3 dq 60 D22 SSTL15 T DCI ca 10 AU14 IVCMOS25 ddr3 dq 61 L21 SSTL15 T DCI ca 11 AV15 IVCMOS25 ddr3_dq 62 A21 SSTL15 T DCI ca 12 AT12 IVCMOS25 ddr3_dq 63 H21 SSTL15 T DCI ca 13 ARI2 IVCMOS25 ddr3 dqs n 0 L36 DIFF SSTL15 T DCI ca 14 BAl4 IVCMOS25 ddr3 dqs n l N31 DIFF SSTL15 T DCI ca 15 BA15 IVCMOS25 ddr3 dqs n 2 F34 DIFF SSTL15 T DCI ca 16 AW17 LVCMOS25 ddr3 dqs n 3 F31 DIFF_SSTL15_T_DCI ca 17 AY17 IVCMOS25 ddr3 dqs n 4 C34 DIFF SSTL15 T DCI ca 18 ARI5 IVCMOS25 ddr3_dqs_n 5 K34 DIFF SSTL15 T DCI ca 19 AP15 IVCMOS25 ddr3 dqs n 6 F22 DIFF SSTL15 T DCI ca 20 BB17 LVCMOS25 ddr3 dqs n 7 E23 DIFF SSTL15 T DCI ca 21 BB16 IVCMOS25 ddr3 dqs p 0 L35 DIFF SSTL15 T DCI ca 22 AT14 IVCMOS25 ddr3 dqs p 1 M31 DIFF SSTL15 T DCI cd 0 U31 IVCMOS25 ddr3 dqs p 2 E34 DIFF SSTL15 T DCI cd 1 T31 IVCMOS25 ddr3 dqs p 3 F32 DIFF SSTL15 T DCI cd 2 AL32 IVCMOS25 ddr3 dqs p 4 B34 DIFF SSTL15 T DCI cd 3 AK32 IVCMOS25 ddr3 dqs pl5 K35 DIFF SSTL15 T DCI cd 4 R33 IVCMOS25 ddr3 dqs p 6 G22 DIFF SSTL15 T DCI cd 5 P32 IVCMOS25 ddr3 dqs p 7 E24 DIFF SSTL15 T DCI cd 6 AH33 IVCMOS25 ddr3_odt 0 G37 SSTL15 cd 7 AJ33 IVCMOS25 ddr3_odt 1 F40 SSTL15 cd 8 P33 IVCMOS25 ddr3 ras n D41 SSTLI5 cd 9 N33 IVCMOS25 ddr3 reset n G42 IVCMOSI5 cd 10 AG31 IVCMOS25 ddr3 we n D42 SSTL15 cd 11 AH31
77. 646 6 x10 Mean 1647 RMS 0 296 100 E RMS 0 2966 Underflow 0 i Underflow 0 Overflow 0 Overflow 0 100 80 60 40 20 TTTTITTT TTT TTTT TTTTTIX 0 645 5 646 646 5 647 647 5 648 1645 5 1646 1646 5 1647 1647 5 1648 Trigger Latency in ns Trigger Latency in ns a b TriggerLatency Latency Setting 700 ns TriggerLatency_ TriggerLatency Latency Scan ntries x10 Mean 846 4 100 RMS 0 2959 H Underflow 0 30000 ddnde Overflow 0 80 25000 B 20000 15000 40 10000 20 5000 ce PEPE OR A A ERN ES TI A EE 845 845 5 846 846 5 847 847 5 500 1000 1500 2000 Trigger Latency in ns Trigger Latency in ns c d Figure 7 4 Measurement of the effective trigger latency for different settings 77 p 70 The non flat plateau is an artifact of the measuring method which is explained in the text All these trigger signals with the exception of the random trigger must arrive at the TCS con troller with a fixed delay with respect to the causing event The reference time of an event is defined by the signal of the beam particle in the scintillating fiber detectors For most of the trigger types in the COMPASS experiment the latency is adjusted by inserting delay cables to match the timings The TIGER trigger processor on the contrary is a digital system calculating the event timestamp
78. 70 80 90 100 110 120 130 140 150 160 Time ps Figure 5 25 Eye diagram of the differential GTX transceiver input voltage on the RX side of the Aurora channel Simulated with SigXplorer for a data rate of 6 25 Gbit s with optimized TX emphasis settings 82 5 The TIGER Module 5 2 CPU and GPU Extension Boards The CPU and GPU parts of the TIGER module have been implemented using appropriate add on cards which has been decided for several reasons First sole embedded CPU and GPU chips are almost impossible to obtain in low quantities since they are usually sold only to certi fied manufacturers Second design complexity is reduced a lot when pre assembled modules with standardized form factors are used and finally this approach provides the opportunity for future upgrades when more powerful CPU or GPU models become available 5 2 1 COM Express Module Specification For the CPU module the COM Express standard 121 122 has been chosen a specification for x86 based computer on modules COM released by the PICMG A COM integrates all necessary components for a very compact PC including CPU RAM non volatile memory and I O controllers However instead of standard connectors for various PC peripherals the COM features a high density connector to the baseboard where the I O signals can either be directly connected to other on board components or broken out to external connectors The COM Express specification defines several module size
79. 9 3 P4 C4 D4 C 1 VXS Backplane Connectors 159 Link Name Backplane Pins Conn z FPGA Pins W32 Y33 V33 W33 AA34 Y34 AA32 Y32 AW37 AW38 AT37 AR38 AV39 AV38 AU34 AT34 AU36 AT36 AU37 AU38 AY39 BA39 BB39 BB38 V40 W40 U39 V39 W42 Y42 V41 W41 W35 V35 U42 U41 V38 W38 Y38 AA39 BA37 BB37 AY38 AY37 AW36 AV36 BA35 AY35 AY34 AW35 BB36 BA36 BB34 BA34 AV34 AV35 P36 P35 R40 T40 U36 T36 T41 T42 U32 U33 U37 U38 W36 V36 V34 U34 AM26 AL26 AT31 AU31 BA32 AY33 BB33 BB32 AR33 AT32 AV33 AW33 AP30 AN30 AN29 AM29 VXSPORT_9 lt 4 gt VXSPORT_9 lt 5 gt VXSPORT_9 lt 6 gt VXSPORT_9 lt 7 gt VXSPORT_10 lt 0 gt VXSPORT_10 lt 1 gt VXSPORT_10 lt 2 gt VXSPORT_10 lt 3 gt VXSPORT_10 lt 4 gt VXSPORT_10 lt 5 gt VXSPORT_10 lt 6 gt VXSPORT_10 lt 7 gt VXSPORT_11 lt 0 gt VXSPORT_11 lt 1 gt VXSPORT_11 lt 2 gt VXSPORT_11 lt 3 gt VXSPORT_11 lt 4 gt VXSPORT_11 lt 5 gt VXSPORT_11 lt 6 gt VXSPORT_11 lt 7 gt VXSPORT_12 lt 0 gt VXSPORT_12 lt 1 gt VXSPORT_12 lt 2 gt VXSPORT_12 lt 3 gt VXSPORT_12 lt 4 gt VXSPORT_12 lt 5 gt VXSPORT_12 lt 6 gt VXSPORT_12 lt 7 gt VXSPORT_13 lt 0 gt VXSPORT_13 lt 1 gt VXSPORT_13 lt 2 gt VXSPORT_13 lt 3 gt VXSPORT_13 lt 4 gt VXSPORT_13 lt 5 gt VXSPORT_13 lt 6 gt VXSPORT_13 lt 7 gt VXSPORT_14 lt 0 gt VXSPORT_14 lt 1 gt VXSPORT_14 lt 2 gt VXSPORT_14 lt 3 gt VXSPORT_14 lt 4 gt VXSPORT_14 lt 5 gt VXSPORT_14 l
80. A detector vs their velocity Pror for a sample of exclusive up p py events which have been recorded in the DVCS test run 2012 184 dEB Units 0 2 0 4 0 5 0 6 0 7 Entries 2114 0 8 Beta Figure 7 12 Energy loss AEg of protons in the Ring B scintillators of the CAMERA detector vs their velocity ror for a sample of exclusive up u pp events which have been recorded in the DVCS test run 2012 185 142 7 Commissioning 8 Summary In late 2012 the DVCS measurement campaign was started at the COMPASS II experiment us ing 160 GeV c high intensity muon beams and a 2 5 m long liquid hydrogen target The aim of these studies is to constrain better the GPDs in order to gain more information about the transverse spatial parton distributions and to help with the disentanglement of the total angu lar momentum of the partons For the DVCS measurements the CAMERA recoil proton detector has been built and was com missioned successfully during a test run in 2012 It is designed to cover the liquid hydrogen target with full azimuthal angular acceptance and a polar angular acceptance from 45 to 90 The detector comprises two ring shaped layers each with 24 scintillating counters which are read out on both ends This adds up to 96 channels in total The PMT signals are digitized by 12 GANDALF transient analyzer modules at a sampling rate of 1 GS s with an effective resolu tion of better than 10 bits Th
81. AMERA Helper is a utility class for the COMPASS analysis tool PHAST 183 The correction offsets for fair and trop are also calcu lated from the CAMERA Helper calibration constants which is indicated by the index CH The calculation of the time of flight offset is rather straightforward raw raw _ praw _ raw corr CH Bjup Bjdn Aiup Ai dn ToF i j e Re STI IToF off global PS ToF off i j 7 4 I OE CH ToF i j However an offset for the up down time difference for the Ring A elements is not explicitly calculated in the CAMERA Helper it is rather contained in the equation for the z position of the hits corr CH _ corr CH Ceff A i raw raw 2Zoff A i Ceff A i Zu Gs mii a Zoff Ai lla un CA am Ldiffoff A 7 5 Ai diff Ai 9 011 4 1 Aiup Aidn Hbon zs Ceff A i 2 oft CH diff A i with Zoff A i Zcal A i nom A For the Ring B elements no z offset is assumed so the time difference offset is off CH Cai Bi 7 difb off B i 7 6 x off The time difference offsets t dift x data run 108912 and from the physics data run 108898 are compared in Fig 7 9 Both cal ff ff i p And aig y 110 The reason for the discrepancy between laser and physics calibration at counter B11 is a problem with the laser signal that is coupled into this element Due to a broken fiber the signal ampli tude was very low and almost no hits were detected which resulted in a poor
82. CIe root ports each providing an x1 Genl link with 2 5 Gbit s in each direction One link is used internally on the COMe module to con nect the PCH The three remaining links 0 to 72 are available on the baseboard connector Link 0 of the CPU module is connected to the first PCIe interface of the FPGA at block location X0Y0 Since the bandwidth of the Atom s root complex is rather low it was decided to inter connect the FPGA and the GPU directly instead of hooking the GPU to another root port of the CPU This allows for a high speed low latency data transfer between the two devices utilizing 46A multi port root complex may also implement routing between its ports although this is not required by the PCIe specification 142 p 31 94 5 The TIGER Module COM Express module TIGER FPGA MXM GPU module Figure 5 31 PCI Express structure on the TIGER board while the COM Express module is connected to the FPGA via a x1 Gen link the MXM GPU module is connected to the FPGA via a x8 Gen2 link providing much higher bandwidth the maximum available bandwidth of the Virtex 6 PCIe block The interconnection between the FPGA PCIe block XOY1 and the MXM GPU is designed as an x8 Gen2 link featuring a raw bandwidth of up to 40 Gbit s in each direction 5 2 6 5 TIGER Switch Design Due to the structure of the PCIe links on the TIGER board the FPGA must act as switch in order to create a PCIe conform hierar
83. FPGA design remains un the FPGA by means of a JTAG cable or online via changed the CPU FPGAS configuration SRAM allowing to selectively reconfiguring a specific area of the device 109 The remaining part of the design stays operational during the reconfiguration process However in order to use the PR method the FPGA design must be divided into a static and one or more reconfigurable partitions The static partition contains all I Os and clock related components like the MMCMs and global clock buffers as well as the part of the design which receives the bitstream from the CPU via the PCIe interface and forwards it to the ICAP The development of a PR capable design for the TIGER module is described in section 6 1 1 7 5 1 5 3 JTAG Configuration and Flash Programming The Virtex 6 FPGA provides a JTAG interface according to the IEEE standard 1149 1 110 which defines a test architecture for integrated circuits It contains a set of registers which are addressed via the test access port TAP The mandatory TAP pins are Test Data In Test Data Out Test Mode Select and Test Clock TDI TDO TMS TCK allowing to combine multiple devices in a chain via the TDI TDO pins The standard was originally developed to test the interconnections between ICs on the board level as well as some basic functionalities of the ICs itself which is known as boundary scan Besides this many devices support vendor specific instructions such as configure and
84. GPDs can be extracted from cross section and asymmetry measurements of exclusive scattering pro cesses like deeply virtual Compton scattering DVCS or hard exclusive meson production A theoretical introduction to GPDs and DVCS is given in Chapter 2 of the thesis at hand Started in 2012 the COMPASS II experiment is focusing on a measurement of the DVCS pro cess up upy Several upgrades of the experimental setup have been performed namely the construction of a long liquid hydrogen target and a surrounding recoil proton detector called CAMERA as well as an additional large angle electromagnetic calorimeter Details are given in Chapter 3 The measurement of DVCS is an exclusive measurement and the reconstruction of all particles in the final state is necessary Therefore the CAMERA detector is built for the detection of recoil particles which leave the target under large polar angles and with very low momentum Based on a time of flight measurement between two barrels of scintillators it al lows to detect protons with a kinetic energy down to 35 MeV At the same time protons can be distinguished from other particles resulting from background processes by means of an energy loss measurement in the scintillating material The signals ofthe CAMERA detector are read out by photomultipliers followed by fast digitizers which perform an online pulse shape analysis This serves two purposes On the one hand the information is transferred to the dat
85. Gen2 lanes are supported as well as a wide range of analog and digital display interfaces Due to the low z height MXMs are ideally suited for GPGPU applications on low profile processing boards in VXS or OpenVPX systems Recently MXM GPUs became more important also for military and aerospace applications 136 137 since it has been shown that these modules can survive in harsh environments 138 Two compatible form factors Type A and Type B are defined covering different performance ranges space requirements and power consumptions For the TIGER project the smaller Type A form factor was chosen which limits the dimension of the GPU card to 82 x 70 mm and power consumption to 50 W The MXM electrical interface uses a 0 5 mm pitch 285 pin card edge connection system The module PCB exposes edge finger contacts and is inserted into the baseboard connector similar to a SO DIMM RAM module by placing it in the slot at approxi mately a 25 degree angle and then pushing it down until it is parallel to the baseboard 5 2 4 1 TIGER MXM Interface The MXM interface was integrated in the TIGER design following the recommendations of the MXM system design guide 139 Since the GPU will only be used for general purpose comput ing no display is connected to the MXM card All signals which are connected on the TIGER board are summarized in Tabs 5 10 5 12 The suffix denotes an active low signal or the complementary signal of a differential pair
86. HE 1930 Zur Theorie des Durchgangs schneller Korpuskularstrahlen durch Ma terie Annalen der Physik 397 3 325 Saint Gobain Crystals 2011 Organic Scintillation Materials Brochure MACHINE SHOP OF THE FREIBURG INSTITUTE OF PHYSICS Private communications Hamamatsu Photonics 2012 Photomultiplier Tube R10533 data sheet ET Enterprises Ltd 2012 9823B series data sheet P J RG Time resolutions from A and B tests Presentation at the COMPASS GPD Meeting July 10th 2012 at CERN A GUSKOV ECALO in the 2012 DVCS run Presentation at the COMPASS Collaboration Meeting November 15th 2012 at CERN 170 Bibliography 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 N ANFIMOV V ANOSOV ET AL 2013 Shashlyk EM calorimeter prototype readout by MAPD with superhigh pixel density for COMPASS II Nucl Inst amp Meth A 718 0 75 Proceedings of the 12th Pisa Meeting on Advanced Detectors Zecotek Photonics Inc 2011 Multi pixel Avalanche Photo Diode Enabling the future of imaging and detection Zecotek MAPD White Paper N ANFIMOV I CHIRIKOV ZORIN ET AL 2010 Beam test of Shashlyk EM calorimeter pro totypes readout by novel MAPD with super high linearity Nucl Inst amp Meth A 617 1 3 78 B KETZER A AUSTREGESILO ET AL 2007 A Triple GEM Detector with Pixel Readout for High Rate Beam Tracking in COMPASS IEEE Nuclear Science Symposium Conferen
87. IG core instantiates an additional MMCM to generate required clock signals based on the 266 MHz reference clock To determine the performance of the memory interface on the TIGER module a MIG core con trolling the Hynix module has been created for the Virtex 6 FPGA The MIG example design which is also generated by the wizard together with the memory interface core modules con tains a traffic generator to create various test patterns which are continuously written to the memory and read back to verify the integrity of the data transfer The available test patterns in clude repetitive patterns like walking 1s and walking 0s which are typically used for mem ory testing as well as pseudo random data The example design was integrated in the TIGER Base design and extended by counters for the number of read cycles and the number of de tected bit errors The counters have been connected to a ChipScope core see 6 1 1 5 in order to access the counter values during the test using the ChipScope Analyzer software In addition the traffic generator and the memory interface could be controlled by ChipScope VIO signals The MIG core is specified up to 533 MHz according to the Xilinx datasheet but during the performance evaluation the interface could even be overclocked to 600 MHz before bit errors started to appear This might be at least partly due to the fact that the test design does not usea significant portion of the available FPGA resources an
88. IVCMOS25 cd 12 R30 IVCMOS25 Table B 2 DDR3 SDRAM Signals cdi13 T30 IVCMOS25 cd 14 AF31 IVCMOS25 cd 15 AG32 IVCMOS25 cd 16 AD30 LVCMOS25 cd 17 AD31 IVCMOS25 clk ref 300 n F14 IVDS 25 clk ref 300 p E14 IVDS 25 clk sil n V30 IVDS 25 153 Signal Pin Standard clk sil p W30 IVDS 25 clk si2 n AF30 IVDS 25 clk si2 p AE30 IVDS 25 fcs b AH30 LVCMOS25 fle b AH29 IVCMOS25 foe b AJ30 LVCMOS25 fwe_b V31 IVCMOS25 Ivttloutl_fpga AUl8 LVCMOS25 Ivttlout2_fpga AT17 LVCMOS25 tmc clk v6 n AY13 IVDS 25 tmc clk vo p AYl4 IVDS 25 tmc data n AW16 IVDS 25 tmc data p AV16 IVDS 25 Table B 3 Misc Signals 154 B FPGA and CPLD Pin Plan CPLD Pin Plan Signal Pin Standard CD lt 14 gt J14 LVCMOS25 CD lt 15 gt R15 IVCMOS25 CD lt 16 gt N13 IVCMOS25 CD lt 17 gt M16 IVCMOS25 Signal Pin Standard CLK_40M P5 LVCMOS33 BUSY R14 IVCMOS25 COMe_cb_reset_B C4 IVCMOS33 CA lt 0 gt N11 LVCMOS25 COMe_pwr_ok H2 LVCMOS33 CA lt 1 gt R16 IVCMOS25 COMe pwrbtn B D7 LVCMOS33 CA lt 2 gt P16 LVCMOS25 COMe sleep B G2 LVCMOS33 CA lt 3 gt M15 IVCMOS25 COMe smb alert B A3 LVCMOS33 CA lt 4 gt T15 IVCMOS25 COMe_smb_ck M2 LVCMOS33 CA lt 5 gt R12 IVCMOS25 COMe smb dat MI IVCMOS33 CA 6 RI3 LIVCMOS25 COMe sys reset B F3 LVCMOS33 CA lt 7 gt TIO LVCMOS25 COMe_thrm_B Hl LVCMOS33 CA lt 8 gt P11 LVCMOS25 COMe thrmtrip B Gl LVCMOS33 CA lt 9 gt T14 IVCMOS25 COMe wake0 B F5 IVCMOS33 CA lt 10 gt R9 LVCMOS25 COM
89. Intel Atom LPC controller does not implement DMA or bus mastering cycles or multi byte firmware memory cycles 126 p 187 only standard memory and I O cycles can be used for data transmission between the host CPU and the peripheral CPLD Because of the lower number of required clock periods it was decided to use I O cycles for the LPC communication on the TIGER board The implementation of the LPC interface core in the CPLD is based on an OpenCores project 133 The following fields are accepted by the LPC core START indicates the start or stop of a transaction A LAD 3 0 nibble is interpreted as START field when the LFRAME signal is asserted Accepted values are 0000b start and 1111b stop abort other values are ignored by the core CT D indicates the cycle type and direction of the data transfer Only I O Read 0000b and I O Write 0010b cycles are supported by the core ADDR is a 16 bit wide field for I O cycles which takes 4 clock periods to transmit The most significant nibble is driven out first TAR turn around fields are used to turn control over to the peripheral and back to the host Each turn around sequence takes two clock periods SYNC fields are sent by the peripheral to indicate it is either ready to complete the cycle or it needs some wait states However the CPLD core will always synchronize within 1 clock period and does not require wait states to be inserted DATA is an 8 bit wide field
90. K multiplexer logic of the TIGER readout concentrator design concatenates the data streams of up to 18 GANDALF boards and two TIGER boards to form a so called SMUX event 120 6 Firmware and Software As soon as all active ports indicate the presence of a new event the multiplexing process is started It first snoops on the event headers in the input FIFOs in order to determine the total size of the SMUX event and to compare the spill and event numbers of all ports against each other If the spill event number of an event does not match or a GANDALF module does not send data within a certain timeout period the correspondent port is disabled till the end of the spill Finally all sane events are written to an output FIFO preceded by the SMUX header containing the total event size and the spill event number The S LINK transmission to the COMPASS DAQ is performed with a HOLA LSC core kindly provided by the CERN PH ESE group 171 This core utilizes the Virtex 6 GTX transceivers at aline speed of 2 0 Gbit s to implement the SERDES functionality of the TI TLK2501 transceiver chips which are present on the standard S LINK hardware Together with a multimode fiber transceiver plugged into the SFP socket of the TIGER board the HOLA LSC core provides a usable bandwidth of 160 MB s and is able to interface with the existing DAQ equipment of the experiment The COMPASS trigger information and reference clock are distributed via the TCS optical fiber
91. LDO regulators M Figure 5 20 UML state diagram of the start up and shut down sequences implemented in the CPLD 5 1 6 4 Power Distribution System Summary All power rails available on the TIGER module are summarized in Tab 5 3 stating the voltages nominal currents and current limits The nominal current value is the maximum permanent current which the specific power converter is able to supply according to the manufacturer s datasheet For converters with a programmable current limit a soft cap has been set based on the estimated power consumption The selected values are given in the soft limit column The remaining devices contain a non programmable internal protection circuit which will limit the current to a fixed cap to avoid damage For these devices the hard limit column of Tab 5 3 states the internal limit This value is inherent to the specific device and cannot be changed from external For power rails with subordinated LDO converters i e the VCC5VO S5 and VCCIV5 rails the programmable current limits have to take into account also the current that is drawn by the LDO converters Likewise the measured currents on these rails also include the 76 5 The TIGER Module Table 5 3 Summary of the nominal voltages and current limits of the power rails on the TIGER module Voltage V Current A Rail name nominal nominal softlimit hard limit VCC5VO0 S5 5 0 2 9 4 3 VCCI1V8 S5 1 8 0 4 0 7 VCC3V3 S
92. Multi Inno MI12832DO 158 is used to show general board status information It is controlled via the I C bus interface of the COMe module using a software tool which is described in 6 3 Several LEDs at the front panel and at the rear side ofthe PCB indicate the current status of the module Their meanings are listed in Tab 5 17 While the front panel LEDs are visible from a distance those on the rear side are rather meant for debugging purposes Fig A 1 in appendix A may be used for locating the LEDs on the PCB by means of their reference designator Table 5 17 Front panel LEDs and debug LEDs of the TIGER module Name RefDes Color Meaning Front LED 1 red on main power failure blinking CPU suspended Front LED 2 green on power up completed blinking shutdown in progress Front LED 3 green on FPGA configured Front LED 4 green currently not used HDD ACT LD1 red blinking SSD harddisk activity Sby p_fault LD6 red on standby hot swap controller fault over current Sbyp_good LD7 green on all standby power rails ok Main p fault LD8 red on main hot swap controller fault over current CPLD 1R LD9 red on GPU power failure CPLD 1G LD10 green CPLD 2R LD11 red currently not used CPLD 2G LD12 green 53LEMO EPL 00 250 NTN 156 54 due to the use of a TI SN74LVC2G34 buffer gate 55yery High Density Cable Interconnect 960N Semiconductor NB4N855S 5 3 Interfaces 103
93. OS33 VXS_sysrst_B A16 LVCMOS33 Table B 4 CPLD 1 O Signals C Connector Pin Out C 1 VXS Backplane Connectors FPGA Pins Link Name Backplane Pins Conn AH39 AJ40 VXSPORT_1 lt 0 gt P4 A13 B13 AG42 AH41 VXSPORT_1 lt 1 gt P4 C14 D14 AF32 AG33 VXSPORT 1 2 P4 A15 B15 AH34 AJ35 VXSPORT_1 lt 3 gt P4 C16 D16 AF37 AG37 VXSPORT_1 lt 4 gt P4 E13 F13 AG38 AH38 VXSPORT_1 lt 5 gt P4 Gl4 H14 AG36 AH36 VXSPORT_1 lt 6 gt P4 E15 F15 AJ36 AH35 VXSPORT_1 lt 7 gt P4 Gl6 H16 AK38 AJ38 VXSPORT_2 lt 0 gt P3 Al Bl AK40 AL40 VXSPORT 2 lt 1 gt P3 C2 D2 AJA2 AK42 VXSPORT 2 2 P3 A3 B3 ALA2 AM42 VXSPORT_2 lt 3 gt P3 C4 D4 AH40 AJAl VXSPORT_2 lt 4 gt P3 El Fl AK39 AL39 VXSPORT_2 lt 5 gt P3 G2 H2 AJ37 AK37 VXSPORT_2 lt 6 gt P3 E3 E3 AL41 AMAI VXSPORT_2 lt 7 gt P3 G4 H4 AD32 AE32 VXSPORT 3 0 P4 A9 B9 AE33 AD33 VXSPORT_3 lt 1 gt P4 C10 D10 AE34 AE35 VXSPORT_3 lt 2 gt P4 All Bll AF42 AF41 VXSPORT_3 lt 3 gt P4 C12 D12 AG34 AF34 VXSPORT_3 lt 4 gt P4 E9 F9 AF40 AG41 VXSPORT_3 lt 5 gt P4 G10 H10 AF39 AG39 VXSPORT_3 lt 6 gt P4 Ell Fil AF35 AF36 VXSPORT_3 lt 7 gt P4 G12 H12 158 C Connector Pin Out FPGA Pins Link Name Backplane Pins Conn i AL34 AK34 VXSPORT_4 lt 0 gt P3 AS B5 AK35 AL36 VXSPORT_4 lt 1 gt P3 C6 D6 AM34 AL35 VXSPORT_4 lt 2 gt P3 A7 B7 AN39 AM39 VXSPORT_4 lt 3 gt P3 C8 D8 AN35 AN36 VXSPORT_4 lt 4 gt P3 E5 F5 AP37 AR37 VXSPORT_4 lt 5 gt P3 G6
94. PWRINHIBIT UCD RESET LN Figure A 2 TIGER PCB top view 147 za ma 000008 ds Cno TIGER BOARD v1 i c Eu Phys Institut ga C37 que Universitaet His On WB cos 5 Eg m Freiburg R326 3 1 SG TEA a E C158 Hem 313 50 w dEl GNDwrr2 299 6oy ym 3 ON e e i esi m mu Rigo e onam Eu R329 UD ee o Bym E E 9000000900090 900000000006 900000000006 000000000006 900000090090 COS AN Tas 990000090006 900000000090 900000000009 coeccccccoesse coescceccessse 900000000000 e0e0060000099 990000000006 OOOO 000000000006 e 900000000006 000000000006 000000000000 990000000006 900000000009 e 000000000006 900000000009 e 900000000006 900000000006 s 000000000006 900000000000 990000090000 s Cio 5v0 S5 wi 1v5 e e me g R348 C243 um
95. R LUT A 6 Inputs gt 06 DA O5 gt AMUX MOD 4 FF LAT D QH AQ CE CE D gt CLK CLK gt SR SRL Figure 5 3 Simplified schematic of a Virtex 6 slice The carry logic is not shown 91 p 35 The remaining slices are of the standard SLICEL type The CLBs are arranged in a grid spanning the entire FPGA Fig 5 5 apart from an area in the center which holds specialized elements dedicated to e g clocking or device configuration The array of CLBs is also intercepted regu larly by columns of block memory and DSP elements 5 1 1 2 Clock Management The individual parts of the TIGER FPGA design are clocked with several different frequencies Some of them are predetermined e g by interface specifications while others may be tuned for maximum performance or to match the required throughput To provide all the necessary clock frequencies throughout the device the firmware designer can make use of the mixed 58 5 The TIGER Module CLB CLB A IC cue X Switch i Matrix Sa aT X CLB E CLB general routing matrix Xp es Xj es B CLB Figure 5 4 Left a configurable logic block CLB consists of two slices which are connected to a switch matrix All slices of the same column are linked vertically by a carry chain CIN and COUT Right the switch matrices have access to the general routing matrix to
96. Specification o 82 5 2 2 KontronCOMe mTT10 oo o hn 82 Contents vii 5 2 3 COM Interfaces 5 2 3 1 External Interfaces o 5 2 3 3 On board Interfaces o 5 2 4 MXM GPU Module Specification o o 5 2 4 1 TIGER MXM Interface o 5 2 5 AMD Radeon GPU Card o ee es 5 2 6 PCI Express Interconnection o oo e eee eee 5 2 6 1 PCI Express Sp ecification dd fato oa 5 2 6 2 PCI Express Device Layers o ooo 5 2 6 3 PCI Express Transaction Protocol 5 2 6 4 TIGER PCI Exp ress Link Structure 5 2 6 5 TIGER Switch Design LL 53 Interfaces 5 3 1 VXSInterface 5 3 1 1 High Speed Signals LL 5 3 1 2 Sideband Signals LL 5 3 2 TIGER to TIGER Links 5 3 3 SFP Transceiver Sockets 5 3 4 TCS Interface 5 3 5 General Purpose I O 6 Firmware and Software 6 1 FPGA Firmware 6 1 1 TIGER Base Design 6 1 1 1 CPLD Interface o ooo e e 6 1 1 2 Clock Management Reset Logic and VXS I O Buffers 6 1 1 3 TCS Interface Module 6 1 1 4 PCIe Endpoint Block o as th oa n bove b eos 6 1 1 5 ChipScopeCores 2 2 2 2m onen 6 1 1 6 Optional Cores 6 1 1 7 Partial Reconfiguration Workflow
97. TA 4th edn 82 W IE NE R Plein amp Baus GmbH http www wiener d com sc powered crates vxs 6u vxs 6021 html 83 Hartmann Electronic GmbH http www hartmann elektronik de en products vme vme64 and vme64x vxs html 84 MEN Mikro Elektronik GmbH http www men de products vmebus 2 01A020 html 85 W IE NE R Plein amp Baus Elektronik 2013 Product Catalog 2013 86 T BAUMANN 2013 Entwicklung einer Schnittstelle zur bertragung von Pulsinformatio nen eines R ckstofsdetektors an ein digitales Triggersystem Diploma thesis ALU Freiburg Physikalisches Institut 87 IEEE Computer Society 1996 IEEE Standard for Additional Mechanical Specifications for Microcomputers Using the IEEE Std 1101 1 1991 Equipment Practice IEEE Std 1101 10 1996 88 Xilinx Inc 2012 Virtex 6 Family Overview DS150 89 K T POZNIAK 2010 FPGA based specialized trigger and data acquisition systems for high energy physics experiments Measurement Science and Technology 21 6 062002 90 Xilinx Inc 2013 Zynq 7000 All Programmable SoC Overview DS190 91 Xilinx Inc 2012 Virtex 6 FPGA Configurable Logic Block User Guide UG364 92 Xilinx Inc 2013 Virtex 6 FPGA Clocking Resources User Guide UG362 93 Xilinx Inc 2011 Virtex 6 FPGA Memory Resources User Guide UG363 94 Xilinx Inc 2011 Virtex 6 FPGA DSP48EI Slice User Guide UG369 95 Xilinx Inc 2012 Virtex 6 FPGA Memory Interface Solutions User Guide UG406 96 SK Hy
98. The GANDALF 128 channel Time to Digital Converter Journal of Instrumentation 8 01 C01016 J BIELING G AHLUWALIA ET AL 2012 Implementation of mean timing and subsequent logic functions on an FPGA Nucl Inst amp Meth A 672 0 13 C SCHILL 2012 The ARWEN Optical Mezzanine Card for GANDALF http wwwhad physik uni freiburg de gandalf pages download php P J nG 2013 Untersuchung von Algorithmen zur Charakterisierung von Photomultipli erpulsen in Echtzeit Diploma thesis ALU Freiburg Physikalisches Institut NIST PHYSICAL MEASUREMENT LABORATORY PSTAR Database for Protons http physics nist gov PhysRefData Star Text PSTAR html T SZAMEITAT 2012 Entwicklung einer Monte Carlo Simulation f r das COMPASS II Experiment Diploma thesis ALU Freiburg Physikalisches Institut M GORZELLIK 2013 Entwicklung eines digitalen Triggersystems f r R ckstofsproton Detektoren Diploma thesis ALU Freiburg Physikalisches Institut T Kunz 2012 Entwicklung einer Simulationsumgebung f r das COMPASS II Experiment mit Geant4 Diploma thesis ALU Freiburg Physikalisches Institut R SCHAFER 2013 Charakterisierung eines Detektors zum Nachweis von R ckstofsproto nen am COMPASS Experiment Diploma thesis ALU Freiburg Physikalisches Institut 172 Bibliography 80 VMEbus International Trade Association VITA 2006 VXS VMEbus Switched Serial Stan dard ANSI VITA 41 0 2006 81 W D PETERSON 1997 The VMEbus Handbook VI
99. The PMT signals Fig 7 1 are digitized by the GANDALF modules and the detected pulses are transmitted to the TIGER trigger processor which performs the usual hit and track finding logic cf Fig 6 5 But in contrast to actual par ticles which require a finite time to fly from Ring A to Ring B the laser pulses are coupled into the inner and the outer counter at the same instant thus imitating the track of an imaginary particle with infinite speed i e with a time of flight of zero Dedicated laser runs have been carried out every now and then for example at times when no beam was available from the accelerator The laser was enabled to emit pulses with a rate of approx 8 5 kHz The events were triggered by the TIGER trigger with only the geometric coincidence condition 4 10 between Ring A and Ring B required An energy loss condition cf 4 2 2 3 was not applied and a correlation with the beam was of course not required The collected laser data allows to perform a T0 calibration for the 96 CAMERA detector channels A set of t offsets is determined such that the corrected time difference D and downstream channel of each element is zero and the corrected mean time tn is identical between upstream on all elements COIT corr e COIT COIT i COIT corr taxi Xiup tx ijan 0 N Imean x i 9 5 aer 5 ol Imean 0 7 1 VX A B and ie 0 1 23 The time values r raw 1 denote the raw time stamps from t
100. The existing COMPASS trigger system is in large part based on standard NIM electronics Therefore the final proton trigger signal emitted from the TIGER module has to be a simple logic pulse with a fixed latency with respect to the original event Since the trigger processing has not necessarily a constant run time it can for example depend on the actual hit rate in the counters the calculated trigger time stamp has to be buffered until it is time to assert the trigger signal 4 3 Electronics Framework As it has already been described the readout of the CAMERA detector is carried out with 12 GANDALF transient analyzer modules which digitize the 96 PMT channels and extract pulse features in real time using a dCFD algorithm Thereby each PMT pulse is characterized by a set of three parameters a time stamp an amplitude and an integral This data is subsequently used for two different tasks On the one hand the pulse information is buffered until it is even tually transmitted to the DAQ system upon reception of a trigger this is the classical function of a detector readout module But on the other hand the extracted pulse parameters are in stantly transmitted to the TIGER trigger processor where the information from all detector channels are centralized allowing to generate a trigger signal based on the afore mentioned conditions Fig 4 8 illustrates the application of the TIGER modules for the CAMERA readout and proton trigger The PMT
101. Toves amp Fi F F2 in F amp 2 43 m Here m is the nucleon mass F and F are the elastic Dirac and Pauli form factors and are the Compton form factors These are convolutions of the respective quark GPDs with functions describing the hard Compton scattering y q 2 6 Beam Charge Spin Asymmetry at COMPASS II The COMPASS II experiment will measure the DVCS process to constrain the GPDs The kine matic range for x is between 0 01 and 0 1 which has not yet been covered by other experi ments Hence COMPASS II is bridging the gap between the collider experiments H1 ZEUS and the fixed target experiment HERMES 27 p 12 Since the muon beam at the COMPASS II ex periment is naturally polarized due to the parity violation of the pion decay see section 3 1 a u change of sign of the beam charge gt will also invert the polarization gt The Compton form factor is related to the difference Y and the sum Z of the measured cross sections 27 p 16 g o u o u Q e R G20 coso 2 44 S o u 0 o u Q S F7 sinp 2 45 Since the cross sections depend on the azimuthal angle h between the scattering and produc tion planes different combinations of quark GPDs can be determined by either integrating over o or analyzing the angular distribution 27 p 15 e The dependent analysis of the sum Z provides the imaginary part of the Compton form factor 77 which is
102. UT lt 19 gt R 20 54 AT20 AR2O LVDS_OUT lt 20 gt R 21 55 AV21 AW21 IVDS_OUT lt 21 gt R 22 56 BB18 BB19 LVDS_OUT lt 22 gt R 23 57 AY20 BA20 LVDS_OUT lt 23 gt R 24 58 BA19 AY19 LVDS_OUT lt 24 gt R 25 59 BA22 BA21 LVDS_OUT lt 25 gt R 26 60 AV20 AW20 LVDS_OUT lt 26 gt R 27 61 AW22 AY22 LVDS_OUT lt 27 gt R 28 62 BB24 BB23 LVDS_OUT lt 28 gt R 29 63 AW23 AY23 LVDS_OUT lt 29 gt R 30 64 BB22 BB21 LVDS_OUT lt 30 gt R 31 65 AY24 BA24 LVDS_OUT lt 31 gt R 32 66 GND R 33 67 GND R 34 68 Table C 2 IVDS VHDCI Connector Signals D CAMERA Photos Figure D 1 The CAMERA detector has arrived at its final position in the COMPASS experiment on September 27th 2012 Two physicists are shown for size comparison 164 D CAMERA Photos Figure D 2 The CAMERA detector was assembled in the clean area next to the COMPASS hall Figure D 3 Transport of the CAMERA detector to the COMPASS hall ANP BOUP AMP PAP AIOP BALE 0509 AMON UON ANON QUON eses ea 9009 M yu wt O MOMO O LI Figure D 4 Left VXS crate for the CAMERA readout with 12 GANDALF modules and 2 TIGER modules Right patch panel for the connection of the CAMERA PMT signals to the GANDALF modules Figure D 5 View into the space between Ring A and Ring B of the CAMERA detector The orange fibers are used for the laser calibration Ring Ais held in place by the white strings Bibliography 1 2 3
103. VCS measurement phase after the restart of the CERN accelerator complex in 2015 A TIGER Board Layout The figures A 1 to A 3 in this appendix can be used to locate components on the TIGER board by means of their reference designators Fig A 1 shows the debug LEDs on the back side of the PCB Figs A 2 and A 3 show a top and bottom view of the PCB with only the component pins and the silkscreen visible Fig A 4 shows the cross section of the multi layer PCB ALTAR 284 R267 Mu n a 2 ima e C134 R271 Figure A 1 Photo of the back side of the TIGER module The debug LEDs are marked to ease locating 146 A TIGER Board Layout 323 m e e CPLD JTAG FPGAJTAG me A o nee E ODA 000 00000000 000000 DURING L a g 00000000 K800000000 0000000000008 000000000000 OOOOOOOOUOO AO AOODDDA DINO DDDO OD OOOO MMN 907 101 us c o2 ci um ca e LI ca cte on Le U48 U63 u50 e e 2 o e eje n 3 e H H e t e hd gt 167 3 H eo e oe y e m E E HRS B 28 2
104. VHDL a hardware description language for digital systems which is specified in the IEEE 1076 standard VHDL is a text based method of describing the structure and the behavior of a logic circuit It is adopted to design models for synthesis as well as to write test benches for simulations However only part of the syntax can be synthesized to program actual hardware VHDL supports the description of concurrent systems by the use of processes A comprehensive introduction to digital system design with VHDL may be found in 159 160 In several places throughout this chapter VHDL keywords signal or constant names will be mentioned They will be set in typewriter font for better recognition Constants are written in uppercase letters To implement a firmware design for use on a Xilinx FPGA device the VHDL source code is translated by the synthesis tool to a register transfer level RTL schematic which is subse quently converted into a gate level description called a netlist This file is then used as an Very High Speed Integrated Circuit Hardware Description Language 106 6 Firmware and Software 32x 32x Ivttlout1 in out Ivttlout2 TIGER Base design PCle napon FIFOs pcie sys clk interface i clk_ddr3 RX TX configmem SO DIMM interface registers JTAG tcs clk tcs data clk ref aurora mgtclk clk_sit m 4x RX TX SelectMap A D 18x payload NOR flash i f 1x switch sfp mgtclk 2x RX TX Figure 6 1 Top level
105. _T_DCI ddr3 ck p 1 F35 DIFF SSTLI5 ddr3 dq 31 E32 SSTL15_T_DCI ddr3 cke 0 G41 SSILI15 ddr3 dq 32 A36 SSTL15 T DCI ddr3_cke l F42 SSILI15 ddr3 dq 33 H31 SSTL15_T_DCI ddr3 cs n 0 C40 SSTL15 ddr3 dq 34 L30 SSTL15_T_DCI ddr3 cs n l C41 SSTL15 ddr3 dq 35 M29 SSTL15 T DCI ddr3_dm 0 H39 SSTLI15 ddr3 dq 36 B36 SSTL15_T_DCI ddr3_dm 1 J40 SSTL15 ddr3 dq 37 E33 SSTL15_T_DCI ddr3_dm 2 H30 SSTL15 ddr3 dq 38 G32 SSTL15_T_DCI ddr3_dm 3 A32 SSILI15 ddr3 dq 39 G31 SSTL15 T DCI ddr3_dm 4 G33 SSILI15 ddr3 dq 40 K32 SSTL15_T_DCI ddr3_dm 5 N29 SSILI15 ddr3_dq 41 L31 SSTLI5 T DCI ddr3_dm 6 B22 SSIL15 ddr3_dq 42 J37 SSTLI5 T DCI ddr3_dm 7 J21 SSTL15 ddr3_dq 43 N28 SSTLI5 T DCI ddr3_dq 0 L37 SSILI5 T DCI ddr3_dq 44 L32 SSTLI5 T DCI ddr3_dq l J38 SSTL15_T_DCI ddr3_dq 45 N30 SSTL15_T_DCI ddr3_dq 2 H41 SSTL15_ T DCI ddr3 dq 46 K33 SSTLI5 T DCI ddr3_dq 3 H38 SSTILI5 T DCI ddr3_dq 47 P28 SSTLI5 T DCI ddr3_dq 4 R29 SSIL15 T DCI ddr3_dq 48 G23 SSTL15 T DCI ddr3_dq 5 R27 SSTL15 T DCI ddr3_dq 49 B24 SSTL15_T_DCI 152 B FPGA and CPLD Pin Plan Signal Pin Standard Signal Pin Standard ddr3 dq 50 C23 SSTL15 T DCI ca 0 AW13 LVCMOS25 ddr3 dq 51 H23 SSTL15 T DCI call AW12 IVCMOS25 ddr3 dq 52 C24 SSTL15 T DCI ca 2 BBl4 IVCMOS25 ddr3_dq 53 A24 SSTL15_T_DCI ca 3 BB13 IVCMOS25 ddr3 dq 54 B23 SSTL15 T DCI ca 4 AU13 LVCMOS25 ddr3 dq 55 F21 SSTL15 T DCI ca 5 AU12 IVCMOS25 ddr3_dq 56 K22 SSTL1
106. _scl lt 4 gt A15 LVCMOS33 VXS_pp_scl lt 5 gt H13 LVCMOS33 VXS pp scl 6 C15 LVCMOS33 VXS pp scl 7 C14 LVCMOS33 VXS_pp_scl lt 8 gt Bl4 LVCMOS33 VXS_pp_scl lt 9 gt D14 LVCMOS33 VXS_pp_scl lt 10 gt C13 IVCMOS33 VXS_pp_scl lt 11 gt B12 LVCMOS33 VXS_pp_scl lt 12 gt C12 LVCMOS33 VXS pp sclc13 C10 LVCMOS33 VXS_pp_scl lt 14 gt D9 IVCMOS33 VXS_pp_scl lt 15 gt A12 LVCMOS33 VXS_pp_scl lt 16 gt C9 IVCMOS33 VXS_pp_scl lt 17 gt B10 IVCMOS33 VXS_pp_scl lt 18 gt A9 IVCMOS33 VXS pp sdacl E15 IVCMOS33 VXS pp sdac2 D16 IVCMOS33 VXS pp sdac3 F14 LVCMOS33 VXS_pp_sda lt 4 gt C16 IVCMOS33 VXS_pp_sda lt 5 gt El4 IVCMOS33 VXS_pp_sda lt 6 gt D15 IVCMOS33 VXS pp sdac7 F13 IVCMOS33 156 B FPGA and CPLD Pin Plan Signal Pin Standard VXS pp sda 8 E13 LVCMOS33 VXS pp sda 9 Al4 LVCMOS33 VXS_pp_sda lt 10 gt D13 LVCMOS33 VXS_pp_sda lt 11 gt B13 LVCMOS33 VXS_pp_sda lt 12 gt A13 IVCMOSS33 VXS_pp_sda lt 13 gt Cll LVCMOS33 VXS pp sdacl4 B9 IVCMOS33 VXS_pp_sda lt 15 gt D11 LVCMOS33 VXS_pp_sda lt 16 gt Bll LVCMOS33 VXS_pp_sda lt 17 gt D10 LVCMOS33 VXS_pp_sda lt 18 gt All LVCMOS33 VXS_sera G16 LVCMOS33 VXS_serb G15 IVCMOS33 VXS_sw_se lt 0 gt H12 LVCMOS33 VXS_sw_se lt 1 gt G13 LVCMOS33 VXS sw se 2 G14 IVCMOS33 VXS_sw_se lt 3 gt H16 LVCMOS33 VXS_sw_se lt 4 gt G12 LVCMOS33 VXS_sw_se lt 5 gt F16 IVCMOS33 VXS_sw_se lt 6 gt E16 LVCMOS33 VXS_sw_se lt 7 gt F15 IVCMOS33 VXS_sysfail_B B16 IVCM
107. a acquisition system DAQ upon receiving a first level trig ger FLT but on the other hand the CAMERA detector shall also be able to influence the FLT decision itself This requires the extension of the CAMERA readout framework by a digital trig ger system whose development constitutes the main part of this thesis The trigger system shall be able to select events with a recoil proton in the final state while suppressing back ground events using the particle identification capabilities of the CAMERA detector Challeng ing selection criteria based on both the time of flight and the energy loss measurement call for a powerful programmable logic board At the same time the integration into the existing COMPASS trigger system poses strict constraints on the latency of the trigger decision The concept for the new proton trigger system is introduced in Chapter 4 This paves the way for a new FPGA based trigger and DAQ hardware called TIGER which is detailed in Chap ter 5 The module is operated in two firmware configurations serving two distinct purposes Firstly the trigger processor is responsible for the generation of a trigger signal based on re coil particles which is included in the global FLT decision Secondly a readout concentrator allows to multiplex the data streams of up to 18 readout modules into one link to the DAQ The firmware development is covered in Chapter 6 The CAMERA detector and the corresponding readout and trigger elect
108. access to a 4 GB solid state disk SSD Both RAM and SSD chips are directly soldered on the COM Express board The Kontron COM Express Mini module supports a power supply with a single voltage in the range from 4 75 V to 14 V which allows it to be directly connected to the primary 5 V rail of the TIGER board without intermediate power conversion The power consumption of the COM board is between 5 W when idle and 7 5 W under full load 29PCI Industrial Computer Manufacturers Group 30Part No 34003 1040 16 1 5 2 CPU and GPU Extension Boards 83 5 2 3 COM Interfaces 5 2 3 1 External Interfaces The following interfaces of the COM are made available to the user by breaking them out to external connectors on the TIGER main board A Gigabit Ethernet interface RJ45 is used to connect the TIGER board to the DAQ con trol network The COM can optionally be configured as diskless system and boot from network cf 6 2 using the Preboot eXecution Environment PXE The Ethernet interface is also used for remote access via SSH by the monitoring and control tools A USB 2 0 port is available on the front panel to connect peripheral equipment like a keyboard a mass storage device or a JTAG programming cable for remote debugging cf 6 2 4 A standard USB hub can be used to connect more than one device at a time The maximum continuous output current of the USB port is 500 mA For temporary connections of a hard disk or optical disk drive the
109. ally a parallel to serial converter and a serial to parallel converter respectively with a programmable conversion ratio between 8 and 40 The serial side is connected to CML 13 Current Mode Logic 5 1 Mainboard 63 Table 5 1 Key features of the Xilinx Virtex 6 FPGA XC6VSX315T 2 FF1759 CLB slices 49200 SLICEMs SLICELs 20360 28840 Flip Flops 393600 Distributed RAM 5090 kbit Block RAM 704 x 36kbit 25 Mbit DSP48El slices 1344 fmax of BRAM DSP 540 MHz GTX transceivers 24 User I O pins 720 Package pins 1759 Package size 42 5 x 42 5mm compliant drivers buffers which in turn are connected to dedicated transceiver pins at one edge of the device package The parallel side is interfacing the FPGA logic fabric That way the data processing in the internal logic can take place at much lower clock frequencies than the serial transmission A wide range of industry standard high speed serial protocols is sup ported by the GTX transceivers i e Infiniband Aurora PCI Express S ATA Gbit Ethernet and many more but of course custom protocols like the CERN developed S LINK can also be im plemented see 6 1 3 The built in 8b 10b and 64b 66b algorithms can optionally be used to manipulate the data according to these line codes to achieve DC balancing and allow clock recovery from the serial bit stream by the receiver The GTX transceiver tiles contain internal PLLs to generate the high speed bit c
110. amic scheduling of the tasks across the available cores OpenCL is a royalty free open stan dard and is developed by the Khronos Group The AMD software development kit and device driver which are needed for developing an OpenCL program and running it on the GPU are described in the software chapter 6 2 5 5 2 6 PCI Express Interconnection 5 2 6 1 PCIExpress Specification The PCI Express specification 142 has been released in 2002 by the PCI SIG defining a new generation high performance bus architecture to interconnect devices in computing and com munication applications Itwas designed to replace the old parallel PCI and PCI X buses which 40Type MXM3 0 E6760 VO al Open Computing Language 4 pCI Special Interest Group 90 5 The TIGER Module root complex bus 0 virtual PCI PCI bridge PCle upstream port PCle downstream port Figure 5 29 Topology of a typical PCIe system consisting of a root complex a switch and three end points EP The bus numbers are assigned during the enumeration process Adapted from 143 p 48 were in use since the beginning of the 90 s of the last century PCIe provides vastly increased bandwidths with a lower number of required pins and better scalability while still maintain ing software compatibility with its predecessors by employing the same usage model address space model and transaction types An extensive overview about the PCIe archite
111. an be accessed by measuring deeply virtual Compton scattering or hard exclusive meson produc tion will soon provide information about the total angular momenta of the quarks and gluons The theoretical background will be explained in the following sections 2 2 Deep Inelastic Scattering Deep inelastic scattering DIS can be used to investigate the nucleon structure In DIS pro cesses an incoming lepton l is scattered off a quark of the nucleon N by exchanging a virtual boson For small center of mass energies ys like in the COMPASS experiment this boson is usually a virtual photon y The struck quark escapes from the nucleon which consequently fragments into one or several hadrons Fig 2 1 If the scattered lepton is the only particle that 4 2 Theoretical Motivation Figure 2 1 Schematic diagram of semi inclusive deep inelastic scattering A lepton l is scattered off a quark of the nucleon N The momentum of the nucleon is P the momentum of the lepton is k before and k after the exchange of a virtual photon y with momentum q The helicities of the particles are drawn as wide arrows 12 p 3 is detected in the final state the measurement is called inclusive With at least one additional hadron detected in the final state the measurement is called semi inclusive For an exclusive measurement all outgoing particles have to be detected In the parton model DIS processes are best described in the infinite momentum fra
112. analysis the calibration is performed following a different approach based on physical events instead of laser pulses The two methods are compared in section 7 2 2 4 2 2 3 Cuts on Energy Deposition With the geometric correlation alone it is not possible to trigger only on protons The major ity of the tracks originating from the target are generated by 6 ray electrons or pions leading to a high rate of false trigger signals The B dependent energy loss of charged particles in the scintillator slats according to the Bethe Bloch equation can be exploited to suppress these un desirable coincidences To determine a set of cuts which can be applied to the energy deposition in the counters a large sample of DVCS events has been simulated with a Monte Carlo toolchain 76 78 The events are generated with HEPGen a Monte Carlo generator which is dedicated to studies of exclusive processes in the DIS domain One can see from Fig 4 5 that the kinetic energy of the protons at the vertex is mostly below 500 MeV which corresponds to a velocity of 6 lt 0 75 The generated events are forwarded to TGEANT a Geant4 based simulation of the COMPASS II spectrometer and in particular the CAMERA detector Here the detector response is simulated and the events are reconstructed One obtains that protons with a kinetic energy of less than 35MeV at the vertex do not reach the B scintillator This results in a lower limit for the velocity 2Hard Exclusi
113. ary and subordinate bus number registers in the configuration space headers of the downstream ports Implicit routing is only based on the direction of the TLP which is either upstream or downstream The concept of the switch design for the TIGER FPGA has been implemented using the PCI Express XpressRICH3 IP core by PLDA 146 This highly customizable IP core solution can be configured for rootport endpoint bridge or switch operation and is compliant with the PCIe base specification rev 3 0 On the physical side of the core a PIPE interface is used to connect to the PHY which is contained in the Xilinx Virtex 6 GTX transceivers On the other side of the core a transaction layer interface is provided for the user application An endpoint reference design is included with the XpressRICH3 core which is able to perform basic TLP processing and supports PIO memory read and memory write transactions as well 48Programmed Input Output 96 5 The TIGER Module incoming TLP address routing routing ID routing method header address matches mote routing TLP comes from downstream base limit reg BAR 0 1 of secondary interface secondary subordinate bus number own bus dev func number forward TLP broadcast TLP forward TLP forward TLP consume TLP consume TLP downstream downstream upstream downstream else else downstrea
114. ating significant noise The 1 0 V rail VCCINT for example which provides the core power for the Virtex 6 FPGA is de signed to source up to 20 A At the same time the allowed range for the FPGA supply voltage is between 0 95 V and 1 05 V Two main aspects have been verified with the Allegro PDN Analysis tool Firstly a static IR drop analysis has been executed to determine the expected voltage drop due to the resistance of the power planes The obtained color map indicates the voltage drop at any location with respect to the voltage at the output pin of the power regulator Based on these results the regions with the highest current densities have been identified and the lay out has been adjusted in order to improve the situation for example by adding more vias or by widening the copper areas of the power nets Secondly a power network impedance analy sis was performed to estimate the voltage ripple of the supply nets The lower the impedance of the PDN is the lower is the noise that is caused by the high frequency currents due to the switching of the ICs To keep the impedance of the PDN low decoupling capacitors are placed at the board which quickly provide adequate charge to the ICs But also the closely spaced power and ground planes of the PCB itself act as a capacitance 117 Results of this simulation were used to optimize the quantity and position of the decoupling capacitors The most sensitive signal traces on the TIGER board in te
115. backplane connectors At the same time they control the hot swap process of the board cf 5 1 6 1 ensuring that the main power is shut down before the board is pulled out of the crate The TIGER module is shown in Fig 5 2 with labeling for the major components and for the interface connectors The heat sinks have been removed for this picture to show the devices underneath 5 1 1 Virtex 6 FPGA The central component of the TIGER module is a Virtex 6 FPGA from Xilinx 88 FPGAs are integrated circuit devices which can be configured flexibly to perform complex digital compu tations They are field programmable which means they can be reconfigured in system at any time Since the end of the 1990s FPGAs are widely used in trigger and data acquisition systems for HEP experiments 89 providing certain advantages over traditional ASIC devices For ex ample the re programmability allows to fix design problems or to extend the functionality of the design In addition due to the strongly increased cost and development time for ASIC pro ductions using the current manufacturing processes FPGAs provide a better unit cost for low and medium quantities 4Electromagnetic Compatibility SApplication Specific Integrated Circuit 5 1 Mainboard 55 addi quor ln COM Express DDR3 SDRAM em y with switc i CPU module 12C SMB FPGA Display ni XILINXe SATA TI i VIRTEX 6 FPGA I F I n c TA FO ES sR BOSS eo vo 9 P
116. ber detectors are connected to scaler modules to count the absolute number of beam particles BM03 BM04 031 032 MIB3 beam BM06 BM05 Figure 3 2 The Beam Momentum Stations BM01 BM06 are placed around the last bending magnet B6 of the beam line 22 3 The COMPASS II Experiment l 2610mm 135mmi INOX o 5 flange 2535 mm 120mm 1 I et outer 43 6mm 4 rohacell supports 155mm I inner 40 6mm TI LET _ __ _ aaa ba AR R H Mylar Kapton Target cell end caps window 0 125mm Mylar 0 125mm 0 35 mm thickness thickness thickness Carbon fiber vacuum tube 1 mm thickness 5mm 78 mm inner diameter KM Carbon fiber endcap 2 mm thickness 76 mm inner diameter INOX pipe Y 8mm 5mm superposition Kapton amp Mylar Superposition of 2 layers of Kapton on 10mm Figure 3 3 Sketch of the new liquid hydrogen target not to scale 38 39 3 2 The Target Region 3 2 1 The Liquid Hydrogen Target For the COMPASS II GPD program a new liquid hydrogen target Fig 3 3 has been designed and was successfully put into operation for the DVCS test run in 2012 The target is 255 cm long 0 cm s with the currently available beam intensity thus providing a luminosity of up to 1 The diameter was chosen to be 40 mm to match the transverse size of the beam One of the de sign requirements was to minimize the material that has to be traversed
117. by the DVCS final state particles Photon absorption has to be kept at a minimum and even low momentum protons should be able to escape the target cryostat Therefore the target cell is made of Kapton film with a thickness of 125 um as shown in Fig 3 4 It is wrapped with a super insulation foil con sisting of 30 layers of 11 um thick aluminum foil and placed inside a tubular vacuum cryostat The tube with an inner diameter of 78 mm is made of 1 mm thick carbon fiber and is sealed with a 0 35 mm thick Mylar film window Fig 3 4 right To cool down the target a SHI CH 110 cryocooler 37 is used which provides a cooling power of 30 W at 20 K After 15 hours of cool ing the nominal temperature of 18 K is reached and the isolation vacuum is 7 107 mbar Fora precise determination of the luminosity it is important to maintain a homogeneous hydrogen density Therefore the target has been carefully aligned to be perfectly horizontal 3 2 2 The Recoil Proton Detector For the DVCS reaction the determination of exclusive events is mandatory Since the missing mass method is not sufficient to ensure exclusivity at COMPASS energies the recoiled proton has to be detected as well 27 p 89 For this task a new recoil proton detector the so called CAMERA detector has been built in 2012 It was commissioned during the DVCS test run which is detailed in chapter 7 A 3D drawing is shown in Fig 3 5 some photographs can be found in appendix D The meas
118. cal and systematic accuracy for a measurement of the p dependence of the beam charge amp spin difference 27 p 23 One exemplary bin of the COMPASS II kinematical range is shown 1 GeV Q lt 4 GeV 0 03 lt xp lt 0 07 Predictions are calculated using the VGG model 32 and a fit to world data 33 The analysis of the beam charge amp spin difference Y requires a precise determination of the overall incoming u and j fluxes and the corresponding detection efficiencies at the per cent level since these quantities have a strong impact on the systematics A relative precision of 3 or better is required for the luminosity i e incoming flux times target density 27 p 90 It is easier to measure the beam charge amp spin asymmetry 4 Y Y as certain sys tematic effects cancel at least partially Nonetheless the beam flux determination is still one of the leading contributions to the systematic error The asymmetry is also expected to be less sensitive to theoretical corrections However the interpretation of the asymmetry projections shown in Fig 2 13 is not straightforward since suitable models still have to be developed 27 p 23 In 2009 a DVCS test run was performed at COMPASS to provide a first evaluation of the rela tive contributions of the DVCS and BH terms and the interference term The experimental setup included a 40 cm long liquid hydrogen target surrounded by a 2 m long recoil proton de tector followe
119. card while the second copy is driving the input of a Silicon Labs Si5326A 100 clock multiplier chip The Si5326A device Fig 5 11 generates two programmable clocks by means of a digitally con trolled PLL with programmable loop filter bandwidth A reference crystal is used to actively attenuate the jitter of the input clock resulting in a sub picosecond jitter performance The clock multiplication ratio is defined by four divider values according to the following formula N2 N3 Nlgs Nlis fout i fin 5 3 All settings are programmed through an I C interface via the CPU using the software tool de scribed in section 6 3 66 5 The TIGER Module ref Xtal CLKIN Na Figure 5 11 The Si5326A clock multiplier chip contains a DSPLL which implements a digital loop filter dLF based on a DSP algorithm to regulate the digitally controlled oscillator DCO The phase detector PD compares the input and feedback clocks Adapted from 101 Altogether the TIGER Virtex 6 FPGA receives three clock signals which are related to the global COMPASS reference clock in particular the 155 52 MHz TCS clock which is used to register the TCS data input and to perform TCS synchronous tasks like the reset of certain counters at the begin of spill see 6 1 2 as well as two derived clocks with a fixed frequency and phase relationship with respect to the TCS clock A special procedure called Si sweep see 6 1
120. ce Record NSS 07 p 242 M ALEXEEV F BARBOSA ET AL 2012 Detection of single photons with THickGEM based counters Nucl Inst amp Meth A 695 159 M ALEXEEV R BIRSA ET AL 2013 Development of a Thick GEM based detector for single photon detection Nucl Instr and Meth A to be published M ALEXEEV R BIRSA ET AL 2013 THGEM based photon detectors for the upgrade of COMPASS RICH 1 Nucl Instr and Meth A 732 0 264 B GRUBE 2006 A Trigger Control System for COMPASS and a Measurement of the Trans verse Polarization of A and Hyperons from Quasi Real Photo Production Ph D thesis Technische Universit t M nchen C BERNET A BRAVAR ET AL 2005 The COMPASS trigger system for muon scattering Nucl Inst amp Meth A 550 1 2 217 L SCHMITT H ANGERER ET AL 2004 The DAQ of the COMPASS experiment IEEE Trans Nucl Sci 51 3 439 Cypress Semiconductor 2011 HOTLink Design Considerations AN1162 H C VAN DER BY R A MCLAREN ET AL 1997 S LINK a data link interface specifica tion for the LHC era IEEE Trans Nucl Sci 44 3 398 F CARENA W CARENA ET AL 2006 The ALICE Data Acquisition Software Framework DATE V5 15th International Conference on Computing In High Energy and Nuclear Physics Mumbai India p 25 M BODLAK V FROLOV ET AL 2013 New data acquisition system for the COMPASS ex periment Journal of Instrumentation 8 02 C02009 F HERRMANN 2011 Development
121. ced by sampling calorimeter modules with alternating layers of lead and scintillator the so called Shashlyk modules providing higher radiation hardness and a better energy resolution than the lead glass modules The hadronic calorimeters HCAL1 and HCAL2 are sampling calorimeters with alternating lay ers of iron and plastic scintillator Hadrons generate hadronic showers while interacting with the iron which are then detected in the scintillating layers Since the hadronic interaction length is much greater than the radiation length the depth of the hadronic calorimeters is much larger than the depth of the electromagnetic calorimeters Therefore the HCAL1 and HCAL2 are located in the spectrometer immediately behind ECAL1 and ECAL2 respectively in order to not spoil the good energy resolution of the ECALs As aside effect the hadronic shower generation already starts in the ECALs Gas Electron Multiplier 9Micromesh Gaseous Structure 3 3 The Spectrometer 27 The New ECALO For the GPD program an additional electromagnetic calorimeter ECALO was developed to cover larger photon angles On the one hand this increases the accessible xg region 27 p 99 on the other hand it is also needed for the 7 background suppression ECALO is placed directly after the CAMERA detector to reduce its geometrical size The angular acceptance should match the aperture of the CAMERA detector This results in a total size of 220 x 220 cm with a central
122. chy with the root complex of the COM Express CPU on top but at the same time the FPGA should implement endpoint functionality A schematic overview of the switch design is given in Fig 5 32 It comprises a virtual three port switch which is respon sible for the TLP routing to the specified destination The implemented routing mechanisms will be described below The upstream port of the switch is connected to the COM Express module the external downstream port is connected to the MXM GPU and the internal down stream port is connected to the FPGA s endpoint logic In addition a bus mastering DMA engine is included in the FPGA design which can be programmed to perform memory read and memory write transactions The DMA transactions may target the system RAM as well as memory that is located on another PCIe device and has been mapped to an address range in the system memory map This setup allows to transfer data between the CPU and the FPGA by targeting the internal memory range between the CPU and the GPU by targeting the memory window of the external down stream port and between the FPGA and the GPU by initiating a peer to peer transfer to be performed by the DMA engine The peer to peer transfer is the preferred method to exchange data between the FPGA and the GPU since the CPU and system memory are not involved Hence a lower latency is achieved and the transfer rate is only limited by the bandwidth of the PCIe x8 Gen2 link b
123. cifica tion Version 2 00 125 Intel Corp 2012 Intel Platform Controller Hub EG20T Datasheet Order Number 324211 009US 126 Intel Corp 2013 Intel Atom Processor E6xx Series Datasheet Document Number 324208 005US 127 Hirose Electric Co Ltd 2010 FH12 Series 0 5mm and Imm Pitch Connectors For FPC FFC Part No FH12 40S 0 5SV 55 128 Kontron Europe GmbH 2012 ADA LVDS DVI 24bit User s Guide Rev 123 129 NXP Semiconductors 2012 UM10204 P C bus specification and user manual Rev 5 130 Texas Instruments 2008 4 channel P C and SMBus switch with reset function PCA9546A data sheet 131 SFF Committee 2013 Specification for Diagnostic Monitoring Interface for Optical Transceivers SFF 8472 Rev 11 3 132 Intel Corp 2002 Intel Low Pin Count LPC Interface Specification Document Number 251289 001 133 Artec Design 2012 Artec Dongle II OpenCores project Entity Ipc iow http opencores org project artec dongle ii fpga 134 MXM SIG 2012 Mobile PCI Express Module Electromechanical Specification Version 3 1 135 MXM SIG 2009 MXM Graphics Module Software Specification Version 3 0 136 Mercury Computer Systems 2008 Sensor Stream Computing Platform Data sheet 137 D FRANKLIN Sensor Processing with Rugged Kepler GPUs Presentation at the GPU Tech nology Conference May 16th 2012 in San Jose Bibliography 175 138 139 140 141 142 143 144 145 146
124. cing between the power planes and the corresponding ground planes is minimized to 50 um to form a low impedance power distribution system 118 The placement of the components and the routing of the signal traces were performed manu ally for best possible control over the layout The main challenge while routing the board was the high density of signals below the FPGA The FF1759 ball grid package exhibits 1759 pins in a 42 x 42 array with a pitch of 1 mm To pass through the via array which is needed to fan out the connections the neck mode constraint with reduced trace width was used Fig 5 23 De sign rule checks DRC are continuously performed during the routing process to ensure that all constraints are met As an example the relative propagation delay of the DDR3 SO DIMM signals is calculated and a deviation beyond the specified limit is indicated by a DRC marker The relevant signal traces have thus to be modified with the delay tuning tool by inserting addi tional segments the so called snakes Fig 5 24 All in all the final TIGER board file contains 847 components 4163 vias and 4480 routed connections with a total length of 103 m 5 1 7 4 Simulation Prior to the production of the board various simulations have been performed in order to as sure best possible signal and power integrity The power distribution network PDN of the 80 5 The TIGER Module TIGER board has to supply large currents to the devices without cre
125. cleon when probed at a scale Q In the parton model viewed in the infinite momentum frame the nucleon is a collection of point like non interacting partons hence the DIS cross section o xg Q is just the sum of the cross sections for scattering from individual partons weighted by their number density 13 p 78 a xg Q x Y es app Q 2 14 f The sum runs over the quark and anti quark flavors f u d s ii d 5 and the electric charge ef of the quarks is expressed in units of the elementary charge On the other hand the DIS cross section can be expressed using the structure functions F and F5 which can be measured in lepton scattering on unpolarized nucleons A detailed calcula tion yields the relation between the unpolarized PDFs and the structure functions F and F gt 15 p 192 1 F1 x8 0 5 Lepage xe Q 2 15 f Fa x8 Q xg e qr xB Q 2 16 f From Eqs 2 15 and 2 16 the Callan Gross relation Fo xp Q 2xg Fi xg Q 2 17 is derived whose experimental verification showed that the quarks carry spin 1 2 16 Early measurements of the structure functions showed a scaling behavior i e no dependence of Q but actually a weak logarithmic variation is introduced due to radiative corrections in QCD The Q dependence of the proton structure function F is shown in Fig 2 3 The rise for small xg and the drop for large xg is explained by the proton s quark composition The valence
126. cture is given in 143 The interconnection of all devices in a PCIe system is called a hierarchy Fig 5 29 It is implemented using high speed serial point to point links with a scalable number of lanes usually between x1 one lane in each direction and x16 16 lanes in each direction in depen dence of the required bandwidth The original specification defined the Genl transfer rate of 2 5 Gbit s per lane while the Gen2 transfer rate of 5 0 Gbit s per lane was added in revision 2 0 144 The root complex located at the top of the PCIe hierarchy establishes the connection of the CPU and memory to the PCIe fabric It contains one or multiple root ports which can be utilized to connect endpoint or switch devices The root complex is responsible for the con figuration of the PCIe devices immediately after power up of the system which includes the enumeration process i e the process of discovering all devices and assigning bus and device numbers and the assignment of base addresses for each device by writing to the according configuration registers Endpoints are peripheral devices in a PCIe hierarchy which contain an upstream port only i e a port pointing in direction of the root complex An endpoint may initiate transactions as a requester or respond to transactions as a completer cf 5 2 6 3 For backwards compatibility a legacy endpoint type is defined which may support IO transactions mapped to the system s IO address space Sinc
127. d by the dCFD algorithm while the previous one is still being transmitted the new hit has to be buffered In an extreme scenario with several near following pulses on a detector channel the latency is significantly increased Therefore the buffer size is limited to six data packets and further hits are not considered for transmission to the trigger TIGER A detailed sketch of the entire data flow that is involved in the CAMERA trigger generation is shown in Fig 4 10 The variable t indicates the accumulated latencies for the various stages of the transmission path starting at t 0 with the primary event in the target After a delay of at most 150 ns including the time of flight of the recoil particle light propagation in the scintillator and cable delays the signal arrives at the GANDALF ADC module After an additional delay of 80 ns the hit parameter calculation of the dCFD algorithm is finished and the trigger primitive TP is available in the TP buffer Depending on the fill level some additional latency is caused by the buffering For the exemplary values in this diagram two immediately preceding hits are assumed The subsequent transmission to the TIGER module via the VXS link takes 50 ns and another 30 ns later the trigger primitive has been written to the TP memory where it can be accessed by the trigger logic The proton trigger signal which is generated by the TIGER module is transmitted to the trigger barrack at the far end of the exp
128. d by the spectrometer The beam line provided high intensity u and u beams with an energy of 160 GeV From the recorded data set the exclusive single photon events have been selected and their distribution is plotted in three bins in xg Fig 2 14 Additionally these plots contain the Monte Carlo simulations for the contributions to hard exclusive single photon production by the Bethe Heitler process BH In the low xg bin which is dominated by the BH process 251 events are observed This is used to normalize the BH predictions to the data In the DVCS dominated high xg bin 54 events are selected while the BH contribution is estimated to 20 events The remaining 34 events are due to pure DVCS or 7 background where only one of the photons from 2 decay is detected 18 2 Theoretical Motivation 2 Mp 2 94r 22 4c 8 0 355 0 005 lt xp lt 0 01 32 oasE 0 01 lt x lt 0 02 5 oasE 0 02 lt x lt 0 03 0 35 Compass projected 03H Compass projected 03 E Compass projected de a t GeV c ga 4 2 0 12 E 5 oasE 0 03 lt x lt 0 07 5 oasE 0 07 lt xy lt 0 15 5 oasE 0 15 x 0 30 0 35 Compass projected 03 Compass projected val Compass projected Per Men Hermes JHEP 07 2012 032 ae Hermes JHEP 2012 032 aaa E Hermes JHEP 07 2012 032 E GeV 4 GeVic Figure 2 13 COMPASS II projections for the t dependenc
129. d placement and routing can be therefore realized in an optimal way The maximum clock rate may be reduced in a full design Several 114 6 Firmware and Software long term tests have been performed to determine an upper limit for the bit error rate BER of the memory interface The traffic generator was operated for 21 hours in pseudo random mode and for 24 hours in walking 1s mode thereby transferring 360 TB of data without any bit error This allows an estimation of the BER to lt 3 5 10 16 Aurora Interface A TIGER to TIGER high speed serial interface is available based on the Aurora 64B 66B Logi CORE IP 153 A 4 lane full duplex configuration with a streaming interface has been gener ated for use with the Base design The Virtex 6 GTX transceivers of the Aurora core connect to the RX and TX lanes of the VXS switch port SP3 cf 5 3 2 on the backplane The reference clock is connected to a dedicated MGT clock input of the transceiver tile to minimize jitter The Au rora module contains an additional MMCM to generate the user clock the transmission clock and internal synchronization clocks for the serial transceivers The user interface of the core provides the necessary ports to stream data between the two link partners In the streaming mode the Aurora channel is used as a pipe Words are written into the TX side and get deliv ered to the RX side of the pipe The data width is 64 bits per lane i e 256 bits for the x4 link bet
130. d the time stamp of the input pulse Because the trigger output of the TIGER and the digitization of the GANDALF are operated with synchronous clocks the trigger signal will always occur at the same phase of the sampling period Furthermore the in terpolated time that is calculated by the GANDALF dCFD method is not uniformly distributed for small pulses 74 This results in the excess which is seen at the left side of the latency distributions When measured with an oscilloscope the plateau becomes flat 7 1 3 TheStartCounter Since the uncertainty in the CAMERA time of flight is dominated by the time resolution of Ring A an alternative time of flight between the primary vertex and Ring B would result in a more accurate momentum determination for the recoil protons This requires precise knowledge of the vertex time For this purpose a new scintillating fiber detector has been built for the COMPASS II upgrade the so called StartCounter It is used to determine the time of the beam particles which is then propagated to the vertex position The improvement in the momentum resolution of recoil protons by using the StartCounter is shown in Fig 7 5 The StartCounter is made up of three stations FI12Y FI13U and FI13V which are placed in the beam upstream of the liquid hydrogen target between the existing FI01 and FI02 detectors The main design goals were high detection efficiency good time resolution and high rate stability Each station cons
131. detectors with typical dimensions spatial and time resolutions 35 52 Detector type Active area Spatial resolution Timeres VSAT Scint fibers 3 9 cm 12 3 cm 130 210 um 0 4 ns Silicon strips 5x7cm 8 11 um 2 5 ns Pixel GEM 10x 10 cm 95 um 9 9 ns SAT GEM 31x 31 cm 70 um 12 0 ns Micromegas 40 x 40 cm 90 um 9 0 ns LAT MWPC 178 x 90 120 cm 1600 um Drift chambers DC 180 x 127 cm 190 um Large area DC 500 x 250 cm 500 um Straws 280 x 323 cm 190 um Table 3 3 Electromagnetic and hadronic calorimeters in the COMPASS II spectrometer setup ECALO prototype was installed in 2012 The full detector is currently under construction 35 48 Calorimeter Active area w x h Channels Energy resolution AE ECAL02012 132 x 108 cm 504 0 084 S Y e 0 02 ECALO 220 x 220 cm 2700 0 084 S e 0 02 ECALI 397 x 286 cm 1476 0 064 S amp Y e 0 02 ECAL2 245 x 184 cm 3072 0 064 S Y e 0 02 HCALI 420 x 300 cm 480 0 59 S e 0 08 HCAL2 440 x 200 cm 216 0 664 S Y e 0 05 Figure 3 8 Schematic view of the Shashlyk calorimeter module for ECALO 49 3 3 The Spectrometer 29 the muons which are able to pass the absorbers due to their weak interaction while all other particles are absorbed Therefore particles that are detected both in front of and behind the absorbers can be identified with muons 3 3 4 RICH 1 The RICH 1 detector in th
132. did not produce a single bit error in 12 hours resulting in an error rate of BER5 oGbps lt 5 107 The 6 25 Gbit s channel showed an asymmetric behavior In the direction from switch slot A to switch slot B no bit error oc curred during the test but in the opposite direction 124 bit errors were detected during a total transmission of 2 53 10 bits This results in an error rate of BER6 25Gbps 4 9 107 6 1 1 7 Partial Reconfiguration Workflow It has already been mentioned in the last chapter 5 1 5 2 that the Virtex 6 supports dynamic partial reconfiguration PR This allows to change a specific part of the FPGA firmware on the Integrated Bit Error Rate Tester 6 1 FPGA Firmware 115 partition pins black box static top level RM 3 Figure 6 4 Sketch of a partial reconfiguration project fly while the remaining part of the device keeps operating The implementation of a PR design poses some additional demands because not all FPGA elements may be reconfigured online In addition a well regulated behavior of the static part must be guaranteed while the reconfig uration is in progress To evaluate the partial reconfiguration feature the TIGER Base design was converted into a PR capable test design This was possible without substantial modifications since the basic struc ture already fulfilled most of the requirements The PR workflow in Xilinx PlanAhead includes th
133. dout as defined by the position of the board in the crate Finally the last top level elements controlled by the CPLD interface are the VXS I O buffers The differential I O buffers for the VXS ports are instantiated in the Base design using VHDL if generate statements to choose the signal direction based on the constant TIGER FUNC TIONALITY In case of the Trigger design eight input buffers IBUFDS per VXS port are gen erated while a VXS port of the Readout design consists of six inputs and two outputs cf Tab 5 14 Differential tri state buffers OBUFTDS are used for the outputs with the T pin being controlled by the CPLD interface based on the status of the GANDALF board on the other side of the link 6 1 1 3 TCS Interface Module The TCS clock and data signals provided by the Gimli card cf 5 3 4 enter the TCS interface module of the Base design The serial data stream is decoded by first demultiplexing the A and B channels which carry the first level trigger FLT signal and the TCS commands respectively The phase information which is needed for an unambiguous assignment of the two channels can be obtained from the TCS data itself due to the biphase mark encoding scheme and the bit statistics 64 p 50 This phase information also allows to reconstruct the original 38 88 MHz COMPASS reference clock with a constant latency While the FLT signal is immediately made available to the user logic the B channel data is parallelized
134. e COMPASS spectrometer is a large volume ring imaging Cherenkov detector which is used to identify particles by means of the Cherenkov effect The detector is filled with the radiator gas C4F1o with a refractive index of n 1 0015 at atmospheric pressure and a temperature of 25 C A particle passing the radiator gas emits Cherenkov light under a certain angle which depends on the velocity 6 v c of the particle 1 1 1 COS 3 1 n nyicmHp The light is reflected by spherical mirrors to photon detectors in the focal plane where the image appears as a ring From the ring diameter the Cherenkov angle can be calculated and using the particle momentum p measured in the spectrometer one obtains the mass m of the particle via 3 1 With RICH 1 pions kaons and protons can be separated in the momentum range between 5 GeV c and 43 GeV c In the inner region of the detector plane where the occupancy is highest multi anode pho tomultiplier tubes are used for the photon detection The outer region is currently covered by MWPCs with CsI coated photo cathodes In the course of the COMPASS II upgrades these chambers will be replaced by THGEM based detectors which have been recently developed 53 55 THGEMSs are electron multipliers which are basically derived from GEMs by scaling the ge ometrical parameters and changing the production technology Instead of thin foils as they are used for GEMs the new detector type is built
135. e GANDALF modules perform an online pulse feature extraction based on a digital constant fraction discrimination which results in a representation of the PMT pulses by arrays of time stamp amplitude and integral values The calculated pulse fea tures are available within 130 ns after the signal detection in the PMT Apart from being buffered for subsequent transmission to the DAQ the pulse information also serve as trigger primitives for the so called CAMERA trigger The concept of the CAMERA trigger system has been created within the frame of this thesis Based on the hits in the recoil proton detector a first level trigger signal is generated selecting events with a proton in the final state At the same time background signals from ray elec trons or pions are distinguished to suppress false trigger signals For this particle identification a real time track reconstruction and a combined analysis of the time of flight and the specific energy loss in the detector material is required Furthermore latency constraints due to the existing front end electronics limit the available processing time to roughly 500 ns In order to implement the sophisticated trigger processing algorithms with low latency the development of a high performance programmable logic module was mandatory 144 8 Summary For this reason the TIGER module has been developed in the scope of this thesis It constitutes the underlying hardware for the CAMERA trigger and fo
136. e Q At low Q the scattering angle of the muon is too small for target pointing so these events are triggered by the energy loss trigger The energy loss y of the muon is determined from the deflection in the dipole fields of the spectrometer magnets Using a coincidence of two vertical hodoscope strips in the Ladder system muons with a large deflection in the magnets but very small scattering angle can be selected yielding events with a large energy loss but small Q Finally the Middle trigger system consists of both vertical and horizontal hodoscope layers hence combining the target pointing and the energy loss method 57 32 3 The COMPASS II Experiment SM1 SM2 Hi He H4L HSL Vetos Beam EN Tomi Hai HAM H30 Ho ECAL1 ECAL2 u Filter u Filter Figure 3 11 Position of the trigger hodoscopes in the COMPASS II spectrometer The inner trigger HAI and H5D is not used for the DVCS measurement Adapted from 57 p 223 Veto System Although most of the beam muons are concentrated in a small spot there are still many halo muons which are able to cause a false trigger when crossing the hodoscopes To suppress these events several veto elements are installed upstream of the target They consist of segmented scintillation counters with a central hole where the beam is allowed to pass 27 p 86 Proton Trigger For the DVCS measurements it was decided to develop a new proton trigg
137. e TCS signal to the trigger TIGER to gain access to the COMPASS reference clock and the SPS spill structure as well as the transmission of event data from the trigger TIGER to the readout TIGER for monitoring purposes Further details on the exchanged data will be given in section 6 1 2 Switch port SP3 on connector P5 is connected to GTX transceiver tiles on the Virtex 6 FPGA with AC coupling capacitors on the RX side thus providing a 4 lane high speed serial link between the TIGER boards By using the Xilinx IP core generator an Aurora interface can be customized and integrated in the FPGA user design Aurora 64B 66B 153 is a lightweight link layer protocol which is suited for chip to chip as well as board to board serial communication providing high throughput of up to 25 Gbit s for a 4 lane design with very little protocol overhead The implementation process is detailed in section 6 1 1 6 In addition there are eight single ended signals SW SE 7 0 located on the VXS sideband connector which can be used to transmit status information between the two TIGER boards SW_SE lt 0 gt indicates the status of the readout TIGER and SW SE 1 indicates the status of the trigger TIGER The remaining signals are currently unused 5 3 Interfaces 101 5 3 3 SFP Transceiver Sockets Two sockets for small form factor pluggable SFP transceivers reside on the TIGER board SFP 154 is a de facto standard for compact hot pluggable transce
138. e a PCIe link is always a point to point connec tion multiple devices are connected by using switches which are responsible for the routing of 5 2 CPU and GPU Extension Boards 91 the data packets to the intended destination A switch contains one upstream port and several downstream ports pointing away from the root complex Each port of a switch is associated with a virtual PCI to PCI bridge which are internally connected via a virtual bus see inset in Fig 5 29 During the enumeration process a bus number is assigned to this virtual bus 143 p 50 5 2 6 2 PCI Express Device Layers The PCIe communication protocol consists of three layers Fig 5 30 namely the transaction layer the data link layer and the physical layer For each layer a packet category is defined i e the transaction layer packet TLP the data link layer packet DLLP and the physical layer packet PLP Each packet category originates and terminates at the corresponding layer of the transmitter and receiver device respectively A TLP consists of a header containing the transac tion type and routing information an optional data section and an optional end to end CRC ECRO field It originates at the transaction layer of the transmitting device and is forwarded downwards through the protocol stack where additional information is appended A sequence number and a link CRC LCRC field are added by the data link layer and start and end fram ing characters are
139. e absence of a display which would result in GPGPU applications running at degraded per formance For the development of OpenCL applications the AMD Accelerated Parallel Processing APP Software Development Kit SDK v2 8 has been installed The APP SDK provides AMD s OpenCL implementation consisting of an API and a runtime as well as several sample applications When an OpenCL program is executed a series of API calls configure the system for execu tion an embedded just in time compiler compiles the OpenCL code and the runtime asyn chronously coordinates execution between parallel kernels 179 6 3 Monitoring and Control Tools Anumber of command line tools have been developed to control the TIGER module The most important ones will be described in the following tiger status This tool prints out the status of the TIGER board i e power good status of DC DC converters and linear regulators FPGA configuration status TCS and Si5326A lock status temperature and voltage monitoring information usage tiger status v c optional arguments v verbose mode prints all information default is to print only errors c print the crate overview indicates which VXS ports are ready tigersm The TIGER SelectMap configuration tool is used to load the FPGA firmware It first saves the PCI state located in the PCIe configuration registers then it reconfigures the FPGA with the specified binfile and restores the PCI stat
140. e afterwards usage tigersm binfile arguments binfile path to the FPGA binfile Display Power Management Signaling 6 3 Monitoring and Control Tools 125 tg register A tool to read and write the Fast registers and Static registers i e User registers of the PCIe endpoint interface usage tg register r w f u reg data arguments r register read command w register write command f Fast register access u User register access reg register number data data word required for write commands in conjunction with option f only the values 0 1 or 2 i e toggle are allowed reg and data are either decimal or hexadecimal numbers with prepended Ox tg configmem A tool for read and write access to the Configmem of the PCIe endpoint interface usage tg configmem r w addr data arguments r read access W Write access addr Configmem address data data word required for write commands addr and data are interpreted as hexadecimal number si5326prog A tool to program the Si5326A clock multiplier chip It parses a register file created with the SiLabs Precision Clock EVB Software writes the data via the I C bus to the chip and waits until the PLL has locked usage si5326prog registerfile arguments registerfile path to the Si5326 register txt file si5338prog A tool to program the Si5338 clock generator chip It parses a reg
141. e and the PCS layer located in the GTX transceiver block is made using the PIPE interface Details can be found in the text will be deleted from the replay buffer only after the reception of an ACK which ensures a high reliability of PCIe data transfers In order to be able to associate the ACK NAK DLLPs with the original TLP a sequence ID is contained in the packets The physical layer can be divided into a logical and an electrical part The logical part is fur ther subdivided into the media access layer MAC and the physical coding sublayer PCS The MAC contains the link training and status state machine distributes the bytes across the avail able lanes on the link and scrambles them to reduce EMI noise The PCS performs 8b 10b encoding decoding and contains an elastic buffer on the receiving side which is necessary to compensate for clock differences The physical media attachment layer PMA forms the electrical part of the physical layer which contains the SERDES converters and the differential analog buffers Since the PCS PMA together also called the PHY and the MAC often reside in different devices or IP cores the PIPE interface has been specified for the connection between PHY and MAC 145 5 2 6 3 PCI Express Transaction Protocol Each data transfer in the PCIe fabric is based on a transaction i e a series of packets which are exchanged between a requester and a completer A transaction is started with a request 45 Int
142. e following steps 168 The static top level design instantiates a reconfigurable partition RP as a black box Fig 6 4 The design is synthesized which results in a netlist that does not contain the recon figurable logic One or more reconfigurable modules RM are synthesized independently as separate projects A RM netlist contains the reconfigurable logic which is implemented in the RP A project for the actual implementation is created in PlanAhead with the PR option en abled The top level netlist is imported and the black box instance is defined as RP The RM netlists are added to the RP An area constraint Pblock is attached to the RB in order to define the FPGA region that gets reconfigured during the PR process An implementation run called a configuration in PlanAhead is created for every RM For the first configuration both the static logic and the RM is implemented Afterwards the static partition is promoted for use in subsequent implementation runs In the other configurations the static logic is imported and only the RM is implemented Finally for each configuration there are full and partial bitstream files generated The modules of the TIGER Base design which have been described in this chapter so far con stitute the static top level design of the PR test project This includes all the interfaces at the FPGA boundary as well as the clock related parts MMCM BUFG These elements cannot be 116 6 Firmware and S
143. e has to take the orbital velocity of the electrons into account with the so called shell correction while at the upper limit radiative effects become more important than ionization The term corrects the density effect which is caused by the polarizability of the medium 17 p 326 In Fig 4 2 the stopping power in several materials is plotted as a function of By p Mc Axes for the muon pion and proton momenta are also drawn for comparison The data points for a vinyl toluene based plastic scintillator material like the BC 408 which is used for the CAM ERA detector are taken from the PSTAR database 75 The qualitative behavior of the stopping power functions is similar in most materials They show a broad minimum around fy 3 5 where particles are minimum ionizing At lower energies dE dx falls with the kinemati cal factor 67 3 while the rise at higher energies is caused by the relativistic extension of the transverse electric field The 8 dependence of the stopping power can now be utilized to identify recoil particles with the CAMERA detector The expected energy loss for protons and pions in the CAMERA scintil lators has been simulated with the COMPASS II Monte Carlo toolkit TGEANT and is shown in Fig 4 3 Onecan see that for f 0 6 pions can be separated from protons to a large extent lparticles with mean energy loss rates close to this minimum are called minimum ionizing particles MIP 40 4 CAMERA Proton Tr
144. e of the amplitude of the cos modulation of the beam charge amp spin asymmetry for six bins in xg assuming a running time of 280 days 27 p 24 Recent HERMES results blue triangles 34 and a fit to world data green curves 33 are given for comparison 0 005 lt x lt 0 01 0 01 x 0 03 o o o Tas S70 ec o o o 2 2 2 o o o 60 o o o a a a 50 z c 40 30 20 Eribrrnr bar Dena bora bor ee Lari borra boron bora brad Erba li lilla 150 100 50 0 50 100 150 150 100 50 0 50 100 150 150 100 50 0 50 100 150 deg deg deg Figure 2 14 Preliminary results of the COMPASS DVCS test run in 2009 Distribution in the azimuthal angle for measured exclusive single photon events up u py with Q gt 1 GeV for three xg bins For comparison the expected contributions by the Bethe Heitler process obtained by Monte Carlo simula tions are plotted 27 p 36 3 The COMPASS II Experiment The COMPASS experiment 35 is a fixed target experiment located in the CERN North Area making use of high energy high intensity muon or hadron beams of the SPS accelerator com plex In the past numerous measurements on the helicity and transverse spin structure of protons and neutrons have been performed Starting in 2012 an experimental phase called COMPASS II pursues a new physics program including the determination of GPDs by measur ing the DVCS amplitude The experimen
145. e proton trigger conditions for the CAMERA detector as outlined in section 4 2 2 Details about the development and verification of the trigger logic are to be found in a recent thesis 77 The main features are briefly summarized in the following The trigger processing takes place in four stages as shown in Fig 6 5 The first stage is the slat logic which correlates the PMT pulse information from the upstream and downstream end of each scintillator slat in order to reconstruct the time and position of a hit A slat logic instance exists for every CAMERA scintillator each containing a hit manager up and a hit manager down module The hit managers receive the trigger primitives i e the GANDALF pulse parameters from the VXS interface and store them in form of pmt info data objects They are also responsible for the TO correction to compensate for the individual delays of the detector channels cf 4 2 2 2 The slat_logic operates on the pmt info elements to detect coincidences and finally outputs slat_info objects which are further processed in the second stage Here a track logic instance for each possible A B coincidence is responsible for the reconstruction of possible particle tracks This process results in a series of track info data objects which contain the relevant track parameters In a third stage the cut_logic filters the coincidences found in the previous step based on the event topology and on various condit
146. e readout electronics and the DAQ system There fore it was decided to develop a proton trigger system which can be integrated in the existing TCS cf 3 4 Based on the CAMERA detector information which is provided by the GANDALF readout modules the so called TIGER trigger shall be able to identify events with a proton in the final state Simulations 27 p 19 have shown that the projected total number of DVCS events is in the order of 55 events per day while the number in certain xg Q bins can be as low as one or two events per day It is therefore mandatory for the TIGER trigger to be very efficient On the other hand the rate of particles in the CAMERA detector is expected to be in the order of several MHz mainly due to the production of ray electrons in the target material and the surrounding walls and due to pions from fragmentation processes These background signals should be identified and subsequently suppressed in the trigger decision resulting in a high purity of the proton trigger signal These two key requirements very high efficiency and good purity are best fulfilled by utilizing the advantages of high performance programmable logic devices The TIGER module provides a dedicated hardware platform to implement the proton trigger logic based on the following conditions 4 2 2 Trigger Conditions 4 2 2 1 Geometric Coincidence A particle from the target is passing through Ring A and B of the CAMERA detector producing a
147. e there are possible differences in the length of the laser fibers And for the z position of the counters and the distance between the A and B counters the nominal values are assumed while the actual positions may deviate from these values Therefore the calibration method with physics events is used to achieve best possible momentum resolution for the CAMERA detector 7 2 3 Muon Runs The majority of the data that has been recorded during the DVCS test run in 2012 is with muon beam The total number of good spills i e usable spills for the analysis is 9120 with u beam and 20197 with u beam The intensity ofthe negative beam is roughly one third of the intensity of the positive beam Thus the 1 2 sharing of the beam time between u and u was chosen as a tradeoff between maximizing the total number of events and balancing the statistics for the two polarities Currently for approx 3096 of the data on tape the events are reconstructed and available for analysis Data samples have been selected for two analysis channels exclusive production of photons up u py and exclusive production of 0 mesons up p pp First analyses are currently ongoing To demonstrate the capability of the CAMERA detector the energy loss vs f plots for the exclusive y and 0 events are shown in Fig 7 11 and Fig 7 12 140 7 Commissioning 5 t diff A offset e laser calibration Camera helper offset in ns oo offset in ns E
148. e x8 rx pl6 AB3 MGT pcie_x8_rx_p 7 ACS MGT 151 Signal Pin Standard Signal Pin Standard clk ddr3 n M12 IVDS 25 ddr3_dq 6 K38 SSTIL15 T DCI clk ddr3 p L12 IVDS 25 ddr3_dq 7 H40 SSTL15_T DCI ddr3_addr 0 G36 SSILI15 ddr3 dq 8 M34 SSTL15 T DCI ddr3_addr 1 H36 SSTL15 ddr3_dq 9 L34 SSTIL15 T DCI ddr3 addr 2 A37 SSTL15 ddr3 dq 10 P31 SSTLI5 T DCI ddr3 addr 3 D38 SSTL15 ddr3 dq 11 M32 SSTLI5 T DCI ddr3_addr 4 C38 SSTLI15 ddr3_dq 12 K42 SSTL15_T_DCI ddr3_addr 5 B37 SSTLI5 ddr3 dq 13 J42 SSTLI5 T DCI ddr3 addr 6 B38 SSILI5 ddr3_dq 14 M33 SSTLI5 T DCI ddr3 addr 7 A39 SSTL15 ddr3 dq 15 P30 SSTL15_T_DCI ddr3 addr 8 A41 SSTL15 ddr3 dq 16 K29 SSTL15 T DCI ddr3 addr 9 C39 SSTLI15 ddr3_dq 17 J32 SSTLI5 T DCI ddr3 addr 10 H35 SSTLI5 ddr3 dq 18 D37 SSTL15 T DCI ddr3 addr 11 A40 SSTLI5 ddr3 dq 19 C35 SSTL15 T DCI ddr3 addr 12 F37 SSTLI5 ddr3 dq 20 K30 SSTLI5 T DCI ddr3 addr 13 J35 SSTL15 ddr3 dq 21 J30 SSTLI5 T DCI ddr3_addr 14 B39 SSTLI5 ddr3 dq 22 D36 SSTL15 T DCI ddr3 addr 15 E38 SSTLI5 ddr3 dq 23 C36 SSTL15 T DCI ddr3_ba 0 E42 SSILI5 ddr3 dq 24 A34 SSTLI5 T DCI ddr3 ball B41 SSILI5 ddr3 dq 25 B33 SSTL15_T_DCI ddr3 ba 2 F41 SSTL15 ddr3 dq 26 B32 SSTL15_T_DCI ddr3 cas n B42 SSILI5 ddr3 dq 27 D32 SSTL15_T_DCI ddr3 ck n 0 H34 DIFF SSTLI5 ddr3 dq 28 E35 SSTL15_T_DCI ddr3 ck n l F36 DIFF SSTLI5 ddr3 dq 29 A35 SSTL15_T_DCI ddr3 ck p 0 G34 DIFF SSTLI5 ddr3 dq 30 C33 SSTL15
149. e_wakel_B G4 LVCMOS33 CA lt 11 gt L12 LVCMOS25 COMe wdt E4 LVCMOS33 CA lt 12 gt P9 IVCMOS25 cpld pg E5 LVCMOS33 CA lt 13 gt P10 IVCMOS25 CSI_B T7 LVCMOS25 CA lt 14 gt R10 IVCMOS25 DIN P8 IVCMOS25 CA lt 15 gt T13 LIVCMOS25 DONE R8 IVCMOS25 CA lt 16 gt T16 LVCMOS25 FCS_B M10 LVCMOS25 CA lt 17 gt N8 LVCMOS25 FLE_B N16 IVCMOS25 CA lt 18 gt T9 IVCMOS25 FOE B M12 IVCMOS25 CA lt 19 gt P13 IVCMOS25 FWE_B P15 IVCMOS25 CA lt 20 gt N10 IVCMOS25 gbe0_act_B M7 IVCMOS33 CA lt 21 gt M9 LVCMOS25 gbe0_ledla L4 LVCMOS33 CA lt 22 gt L15 IVCMOS25 gbe0_ledlb R6 IVCMOS33 CCLK N15 IVCMOS25 gbe0_led2a L5 IVCMOS33 CD lt 0 gt M11 IVCMOS25 gbe0_led2b K4 IVCMOS33 CD lt 1 gt L14 IVCMOS25 gbe0_link1000_B R5 IVCMOS33 CD lt 2 gt K14 IVCMOS25 gbe0_link100_B P6 IVCMOS33 CD lt 3 gt Till IVCMOS25 gbeO link B N6 IVCMOS33 CD lt 4 gt Pl4 IVCMOS25 GPU_gpio0 Jl IVCMOS33 CD lt 5 gt L13 IVCMOS25 GPU gpiol K1 LVCMOS33 CD lt 6 gt M13 LVCMOS25 GPU_gpio2 L1 LVCMOS33 CD lt 7 gt J13 LVCMOS25 GPU prsnt 1 B E8 IVCMOS33 CD lt 8 gt T12 IVCMOS25 GPU prsnt r B B5 LVCMOS33 CD lt 9 gt K15 IVCMOS25 GPU pwr en C3 LVCMOS33 CD lt 10 gt N14 LVCMOS25 GPU_pwr_good C1 LVCMOS33 CD lt 11 gt M14 LVCMOS25 GPU_pwr_level C2 LVCMOS33 CD lt 12 gt K16 LVCMOS25 GPU_th_alert_B D2 LVCMOS33 CD lt 13 gt J15 LVCMOS25 GPU_th_overt_B D1 IVCMOS33 155 Signal Pin Standard GPU vga disable B Al LVCMOS33 GPU wake B B3 LVCMOS33 hotl_en A8 LVCMOS33 HSWAPEN T8 LVCMOS25
150. eam particles The SciFis use 16 channel multi anode PMTs for photon detection Apart from the anode outputs which are connected to TDCs for detector readout there is an output of the last common dynode stage of the PMT which basically provides an or signal of 16 channels By digitizing these dynode signals one obtains time stamps of the beam particles which can later be used for a re timing of the trigger output It should be mentioned here that for the offline analysis a precise vertex time is also essential for improvement of the momentum resolution of the CAMERA detector For this purpose an additional SciFi detector has been built and placed upstream of the target cf 7 1 3 46 4 CAMERA Proton Trigger 4 2 3 Further Requirements For a successful implementation of the TIGER system in the existing COMPASS environment some general requirements have to be fulfilled Because of the limited buffer memory of the front end electronics the latency of the first level trigger has to be kept at a minimum typi cally below 3 us Taking the inevitable cable delays and the transmission through the TCS into account the trigger decision has to be made within less than a microsecond This makes high demands on the processing speed of the proton trigger system It will become apparent in the following sections how this challenge has been approached with the development of the TIGER system and its tight integration into the GANDALF framework
151. ecessary for all other kempld drivers and several function drivers The most important ones are the I C driver kempld i2c and the watchdog driver kempld wdt 172 The kempld i2c driver provides access to the I C controller in order to communicate with 1 C devices both on the COMe module and on the TIGER board The driver can be used by other kernel drivers as well as by user space applications Some configuration tools for the TIGER system cf 6 3 utilize this driver for IC communication The kempld wdt driver is used to activate deactivate and trigger the watchdog of the COMe module A watchdog timer is a hardware component used to reset the system in case of a software failure Under normal conditions it is regularly triggered by the software to reset the timer But if the software becomes unresponsive the notifications will fail to appear and the watchdog will timeout and reset the system Since the TIGER modules are located in the COM PASS experimental area which is unaccessible during beam time manual resets of the CPU are not easily possible Therefore the watchdog feature is an important precaution to guaran tee minimum downtime of the system The simplest way to use the watchdog is to open the file dev watchdog and regularly write some characters to it in order to notify the watchdog that the user space program is still alive If no notification is received within a certain timeout period the system is reset 6 2 2 CPLD Driver
152. efine the usage of specific serial fabrics such as InfiniBand Ethernet or PCI Express for a communication between the modules A WIENER crate model UEV 6021 with UEP 6021 power supply 82 equipped with a Hart mann B181182331 backplane 83 provides 18 payload card slots 2 switch card slots and 1 legacy VME64x slot see Fig 4 9 It can therefore house all the GANDALF modules needed for the CAMERA and SciFi dynode readout as well as two TIGER modules and a VMEbus CPU board MEN A20 84 The A20 CPU board is connected via Ethernet to the DAQ control net work and serves as a bridge to the VMEbus for configuration and monitoring of the GANDALF modules Since the switch card slots are not part of the VMEbus the TIGER modules have to be connected separately to the DAQ control network as described in section 5 2 The VXS part of the backplane is designed in a dual star configuration connecting each of the payload cards separately to the two switch cards by direct point to point links According to the VXS protocol layer specifications these links are defined as four bidirectional high speed se rial interconnections i e eight differential pair connections between each payload and switch card Due to the specific requirements of the data transmission between the GANDALF G and TIGER T modules it was decided to use only the physical layer but not the protocol layer of the VXS standard Hence the direction of the signal transmission could be def
153. el LEDs and the debug LEDs are driven by the CPLD 5 1 5 Configuration Scheme The customized behavior of the Virtex 6 FPGA is defined by the firmware design see also 6 1 It is specified using a hardware description language HDL and implemented using the Xilinx ISE PlanAhead toolchain 106 resulting in a bitstream file which can be used for the device configuration The bitstream for the SX315T device has a size of 104 5 Mbit and it contains all the information about the configuration of the slices the routing and the initial values of the registers and block RAM elements Since in many designs most ofthe block RAMs are initialized with 0 s and not all FPGA elements are actually used it is beneficial to enable the so called bitstream compression in the implementation process to reduce the file size by a factor approx between 1 5 and 3 The configuration is written to the FPGA internal SRAM which is a volatile memory and must therefore be reloaded after every power up There are different configuration options available on the TIGER board the parts which are involved in the configuration process are shown in Fig 5 12 5 1 5 1 Configuration at Power Up The Xilinx Platform Flash XL briefly Flash in the following is a 128 Mbit non volatile NOR flash based storage device 107 It has been developed as a dedicated configuration solution for large Virtex 5 and Virtex 6 FPGAs providing bitstream transfer rates of up to 800 Mbit s
154. el PHY Interface for PCI Express 5 2 CPU and GPU Extension Boards 93 packet and can be either posted which means it does not require a response or non posted In the latter case it is executed as split transaction which is finalized with a completion packet at a later time Several transaction types are defined in the PCIe specification These include the memory read write IO read write and configuration read write transactions known from the PCI protocol as well as the new message transaction for packet based in band signaling of errors and interrupts Only memory write and message transactions are posted all others are non posted 143 p 56 For each transaction type a corresponding TLP packet type is defined which is used to issue a request In addition two TLP packet types are available for completions with and without data The packet type is specified in the TLP header together with routing information to iden tify the final destination In contrast to the local link traffic i e DLLPs and PLPs which is always related to a direct connection between two neighboring devices TLPs are forwarded by switches 6 from one link to another in order to carry out a transaction between two devices which are further apart in the PCIe hierarchy Depending on the packet type of the TLP one of three available routing methods is chosen address routing ID routing or implicit routing Ad dress routing is used for memory and IO requests referr
155. ence Detect 15Part number HMT325S6BFR8C H9 16High Speed Current Steering Logic 5 1 Mainboard PA CPU PCI Express ICS9DB102 ide Module ref clock 100 MHz 1 2 fan out dico TCS clock CIk1 cloc TOS Gimli m NEENTIS 155 52 MHz 1 2 fan out CIk2 TCS data Si5326A Virtex 6 CPLD CIkO Aurora clock 40 MHz Si5338A H User clock Quartz Clka DDR3 clock SFP clock Figure 5 10 Clock generation and distribution network of the TIGER module A detailed description is given in the text transceiver tiles The ICS9DB102 clock buffer starts up together with the CPU and the PLL locks within 2 ms after the reference clock is started The outputs are enabled only when the downstream devices request a clock signal via independent CLKREQ pins 5 1 3 2 Experiment Synchronous Clocks The COMPASS trigger information is distributed experiment wide by the TCS system via opti cal fibers The transmitted signal is a serial data stream with a bit rate of four times the COM PASS reference clock fres 4 38 88 MHz 155 52 MHz 5 2 The optical TCS signal is received on the TIGER board by the Gimli card cf 5 3 4 where it is converted back into an electrical signal and the TCS clock is reconstructed with a CLC016 clock recovery chip Thereupon the clock signal is split with a 1 2 IVDS fan out buffer NB6N11S from ON Semiconductor 99 One copy of the TCS clock is directed to the FPGA together with the TCS data from the Gimli
156. ent numbers spill numbers and synchronization signals like Begin of Spill or End of Spill Last but not least the experiment s reference clock which is used to encode the TCS signal is recovered by the readout modules to obtain a global time reference Muon Trigger For the DVCS measurement a trigger on scattered muons in a large kinematic range in Q and xp is mandatory There are four muon trigger systems Fig 3 11 each being responsible for a different kinematic region 27 p 84 the Ladder trigger H4L H5L covers low Q and high y the Middle trigger HAM H5M covers low Q and all y the Outer trigger H3O H40 covers intermediate Q and all y the LAS trigger H1 H2 covers large Q and high xg Each system consists of two scintillating hodoscopes of which at least one is located behind a muon filter To detect the scattered muon two different approaches are used For the target pointing trigger the vertical position of the scattered muon is measured with two horizontal hodoscope layers to determine the scattering angle in the y z plane In this plane the particle tracks are not bent by the spectrometer magnets so the track can be extrapolated to z 0 which is the z position of the target center The vertical position of the track at z 0 is then tested for compatibility with the target position vertical target pointing This method is used by the Outer and the LAS system which trigger on events with medium and larg
157. er system to enrich the sample of exclusive events with a proton in the final state This system shall be able to identify recoil protons based on the information from the CAMERA detector in real time At the same time background signals from 6 ray electrons or pions should be distinguished and prevented from generating a false trigger signal which is only possible by sophisticated digital processing Since the trigger signal has to be included in the first level trigger the available time for the trigger decision is limited The development of the new CAMERA proton trigger system which was conducted during this thesis will be described in detail in the following chapters The underlying concept is presented in chapter 4 The newly created hardware platform the so called TIGER board and the corresponding firmware are detailed in chapters 5 and 6 3 5 TheData Acquisition System The COMPASS data acquisition system DAQ is responsible for the readout of about 300 000 detector channels at trigger rates of up to 100 kHz The high beam intensity results in hit occu pancies of up to several MHz in single channels Altogether the whole spectrometer produces several Gigabytes of data per second during a 9 6 s long spill To handle this amount of data a custom DAQ system was designed 58 which will be described in the following 13 Trigger Implementation for GANDALF Electronic Readout 3 5 The Data Acquisition System 33 300000 Microme
158. erimental hall It has to arrive at the COMPASS trigger system within 1200 ns after the primary event Taking the propagation delay of the cable to the TCS of roughly 330 ns into account the time available for the trigger logic is approximately 510 ns In order to meet the hard real time requirement for the proton trigger generation the TIGER board features powerful programmable logic devices Its development is detailed in the fol lowing chapter The trigger logic is implemented in a custom firmware which is described in section 6 1 2 4 3 Electronics Framework 51 1 0 t 50ns t 280ns Eventin Hitin t 100ns t 150ns t 160ns t 230ns TB E PMT AD gt FD VXS Target ToF Scintillator Cable TP buffer gt Transmitter GANDALF t 290ns DDR Transmitter ISERDEST IODELAYI IBUFDS ___ OBUFDS OSERDES 320ns TIGER t lt 1200 ns t 360ns Atz330ns Trigger son ESTE peon rer ces e Cable Signal Figure 4 10 Diagram of the trigger primitive TP transmission path The entire data flow of the hit information from the primary event at time t 0 to the final trigger signal is sketched and exemplary points in time f are given indicating when the information is available at the respective stages 52 4 CAMERA Proton Trigger 5 The TIGER Module The TIGER module is designed to complement the readout electronics of the CAMERA detec tor Hence it has to fit into the e
159. es 77 p 59 Physical blocks 118 6 Firmware and Software B8 11 A8 11 A4 7 B4 7 B20 23 B16 19 A16 19 A12 15 B12 15 Figure 6 6 PlanAhead screenshot showing the FPGA floorplan of the trigger design The detector chan nel numbers are noted next to the inputs The hierarchical blocks of the trigger logic are placed at de fined locations by the use of Pblocks and area constraints The number of connections between the slat logic and the track logic Pblocks is indicated by the red many and orange fewer lines Pblock 2 contains the coincidence A11 B2 77 p 64 Table 6 3 Configuration of the CAMERA readout crate as it was used during the 2012 DVCS run Slot GA 2 3 4 5 6 7 10 IO bank 22 17 12 16 13 15 14 Detector FI CAMERA ring A Channel 0 5 8 11 12 15 4 7 16 19 0 3 20 23 Slot GA 13 14 15 16 17 20 21 IO bank 14 15 13 16 12 17 22 Detector CAMERA ring B FI2 Channel 20 23 0 3 16 19 4 7 12 15 8 11 0 5 6 1 FPGA Firmware 119 The backplane connection to the TIGER readout concentrator is used to transmit the aforementioned monitoring information in an event based way for inclusion in the S LINK data stream This enables an offline comparison of the trigger data with the detec tor data recorded by the GANDALF modules 77 p 60 The configuration memory Configmem provided by the PCIe inter
160. es which are located on the TIGER mainboard This separation is depicted by the dashed line indicating the transition of the buses from the COM to the base board The device addresses are given in Tab 5 7 and Tab 5 8 respectively The main I C bus fans out to four downstream channels using a PCA9546A 4 channel bus switch 130 These sub buses can selectively be enabled by writing to a control register of the switch cf Fig 5 27 This allows the connection of multiple devices with identical addresses to one bus structure as long as they are connected to different sub buses and only one of the sub buses is active at a time On the TIGER board this feature is necessary for the 12C connec tion of the two SFP transceivers in order to access their enhanced digital diagnostic interface 131 for monitoring purposes Only one of the two SFP sub buses may be enabled at a time otherwise both transceivers would respond to the same address A2h Table 5 7 Slave addresses of the SMBus devices on the COMe module and on the TIGER board Address Device Comment Location 98h Winbond W83771W COMe hardware monitor AOh SPD EEPROM COMe internal use only COMe module D2h Clock Generator COMe internal use only 9Ch TI UCD9246 PWM system controller D4h ICS9DB102 PCle clock buffer TIGER one Low Pin Count Bus The LPC bus 132 has been designed by Intel to replace the parallel ISA bus in modern PCs and it is commonly used t
161. establish connections between the CLBs I O pads buffers I O logic Slice CLBs Ethernet MAC o S T o E E 2 D PCle block 2 c 5 a ta Q o EIER x VO x DSP slices Block RAM MMCMs clock buffers GTX transceivers Figure 5 5 The FPGA floor plan shows the distribution of the CLBs across the chip area as well as the placement of I O RAM DSB clocking and high speed communication elements Details on these elements are given in the text 5 1 Mainboard 59 8 phase taps 1 variable phase tap CLKIN1 gt BERGE s CLKIN2 OP LF VCO O gt CLKOUTO CLKFB CLKOUTI N O L CLKOUTE CLKFBOUT Figure 5 6 The Virtex 6 mixed mode clock manager is based on a phase locked loop to maintain a sta ble phase and frequency relationship of the output clocks to the reference input The phase frequency detector PFD compares the feedback clock to the input clock and in case of deviations it generates up down signals which are converted by the charge pump CP and loop filter LF into a reference voltage for the voltage controlled oscillator VCO Adapted from 92 p 39 mode clock managers MMCMs which are available in the FPGA These PLL based frequency synthesizers are capable of generating eight clock signals with different frequencies and pro g
162. ethods described in 5 1 5 VXS handshake for safe hot plugging of the GANDALF and TIGER modules in the VXS crate a handshaking protocol see 5 3 1 2 has been introduced which regulates the acti vation of the VXS bus output drivers To enable communication between the CPU and the CPLD a LPC bus interface cf 5 2 3 2 has been implemented in the CPLD firmware which acts as a LPC to parallel bridge support 181 ow Pin Count 68 5 The TIGER Module PCle Ethernet E 4 erne gt eu eee JTAG LPC Platform Flash XL Select Map Figure 5 12 Schematic overview of the FPGA configuration interfaces on the TIGER board ing I O Read and I O Write cycles The available registers are listed in section 6 2 2 together with a description of the driver which is provided for a convenient access from the Linux OS In addition the CPLD implements so called glue logic i e simple logic for joining different parts of the board design By providing I O banks with different supply voltages 2 5 V and 3 3 V it can convert the signal levels between the FPGA which supports a maximum I O volt age of 2 5 V and components with 3 3 V LVTTL or IVCMOS interfaces During commissioning of a newly developed module it is also advantageous that the connections may be changed by firmware upgrades in case of unforeseen problems or requested changes in behavior which would not be possible with hard wired connections Finally all the front pan
163. etween the two devices In addition RAM and CPU resources are saved for other tasks The CPU is just running 47 Direct Memory Access 5 2 CPU and GPU Extension Boards 95 COM Express module CPU root complex PCle x1 Gen1 Virtex 6 FPGA Figure 5 32 The PCI Express hierarchy on the TIGER board requires the implementation of a switch in the FPGA The upstream port of the switch is connected to the COM Express module one of the downstream ports is connected to the MXM GPU The second downstream port is a virtual port which is internally connected to the FPGAS endpoint logic the GPU driver and initializing the peer to peer transfer As soon as the transfer starts request and completion packets are exchanged directly between the peers For each TLP which is handled by the switch one of the three available routing mechanisms is employed depending on the TLP type cf 5 2 6 3 as shown in Fig 5 33 If the TLP type indicates address routing the header contains a destination address which is compared by the switch against the memory windows of the downstream ports These windows are defined by the base and limit registers in the type 1 configuration space headers which are initialized during the enumeration process If on the other hand the TLP type indicates ID routing the header contains a destination ID i e bus device and function numbers The routing decision is made based on the second
164. face is utilized to set a multitude of parameters for the trigger logic These include the trigger latency the width of the coincidence windows for the slat and the track logic exclusion of CAMERA segments from the trigger and monitoring settings to name just a few 77 p 61 The TCS information received from the backplane is used to synchronize the system time at each Begin of Spill signal in order to allow the interpretation of the coarse time infor mation which is contained in the GANDALF trigger primitives 77 p 45 The system time is also needed for the release logic to guarantee a constant trigger latency 6 1 3 TIGER Readout Concentrator Design The readout concentrator is the second application of the TIGER module It is used in the VXS crate to multiplex the S LINK data received from the GANDALF modules into a single data stream to the DAQ and it also distributes the TCS signal to all the modules in the crate The development of this application firmware will be described in full detail in a forthcoming thesis 169 The most important aspects are summarized in the following The COMPASS DAQ system adopts the S LINK standard to transmit the event data from the readout modules to the spill buffer PCs cf 3 5 Each S LINK connection is able to transmit up to 160 MB s using the current HOLA protocol or up to 100 MB s using the older ODIN protocol 170 On the receiving side a link destination card LDC mounted on a PCI spill buf
165. fer card is required for every S LINK Up to four of this cards may be inserted in a readout PC To optimize the utilization of the links in order to minimize the number of required PCs and spill buffer cards it is desired to multiplex several readout modules into one S LINK For the CATCH modules the SMUX card 35 p 61 which is plugged into the rear transition connector of a VME crate provides up to 4 1 multiplexing of neighboring modules This is used for detectors producing low or medium data rates in order to save on DAQ equipment For the GANDALF readout framework it was decided to develop a similar S LINK multiplexing system Thanks to the high bandwidth point to point connections on the VXS backplane this could even be achieved without additional rear transition cards Four data signals together with a clock signal are available from each GANDALF module to the TIGER readout concentrator module located in switch slot A using the VXS A port cf Tab 5 14 allowing to transfer the S LINK data words chopped up in nibbles An identical connection is also available from the TIGER trigger module After the S LINK words have been received and parallelized again on the TIGER side they are written into buffer FIFOs Every input port features a dedicated FIFO deep enough to hold several complete events Custom FIFO logic keeps track of the S LINK start and end markers to detect the events boundaries and performs some data integrity checks The S LIN
166. fit result cf Fig and the time of flight offsets pn hj obtained from the laser ibrations are in good agreement The largest deviation is found for 7 2 Physics Data 139 Table 7 4 Calibration constants for the CAMERA Helper class The indices for the detector elements are X A B i 0 1 23 and je i i 1 mod 24 constant name explanation laift off X i offset for the time difference between upstream and downstream PMT EroE off i j offset for the time of flight between A and Bj ToF off global global time of flight offset Zcal A i calibrated z position of the center of the Ring A elements Znom A nominal z position of the center of the Ring A elements Ceff X i effective speed of light in the scintillator elements 7 2 For the deviation on counter B7 no reason could be identified The calibrations for the remaining counters match within a few hundred picoseconds The distributions of the devia tions between laser calibration and CAMERA Helper calibration are shown in Fig 7 10 One may notice that the laser offsets are systematically shifted to larger values but this is still within the RMS error of the distributions Summing up the laser calibration matches the physics calibration quite well providing a suit able set of calibration offsets for online purposes Of course the laser method contains some uncertainties which render impossible its usage for exclusive analyses For exampl
167. from one TIGER board over the backplane to the second TIGER board several topologies are connected to form a complete transmission channel With the Channel Analysis function SigXplorer provides a very efficient method for the sim ulation of high speed serial links Thanks to the IBIS AMI model which includes the pre emphasis equalization and clock recovery algorithms of the GTX transceivers a large num ber of bits can be simulated in a reasonable amount of time 120 The result is presented in form of an eye diagram like in Fig 5 25 combining the effects of noise jitter and intersymbol 27 Simulation Program with Integrated Circuit Emphasis 28Input Output Buffer Information Specification with Algorithmic Modeling Interface 5 1 Mainboard 81 interference on the signal quality The simulation can also be used to study how different TX emphasis settings affect the eye opening The optimum values for the pre cursor and post cursor emphasis settings have been determined for the serial link between two TIGER boards for different data rates as shown in Tab 5 6 Table 5 6 Optimized TX emphasis settings and expected insertion loss obtained from a SigXplorer chan nel analysis simulation Data rate 5 0 Gbit s 6 25 Gbit s pre cursor emphasis 0 61 dB 0 30 dB post cursor emphasis 1 94 dB 2 76 dB differential 4 8 dB 6 9 dB insertion loss 2 5 GHz 9 3 125 GHz Voltage mV 0 10 20 30 40 50 60
168. fter 12 years of successful data taking the even higher data rates of the upcoming COMPASS II physics program will require a new event building system which is currently in development 62 3 5 1 The GANDALF Framework With the GANDALF framework 63 a new generation of readout electronics has been intro duced to accommodate a growing demand for high performance digitizers in particle physics experiments Originally designed at the Freiburg institute of physics for the readout of the CAMERA detector the GANDALF module is now used in various configurations for several de tectors in the COMPASS experiment GANDALF is a versatile detector readout system based on the VME64x VXS form factor con sisting of a generic mainboard and a set of application specific add on mezzanine cards The GANDALF board is capable of implementing sophisticated real time digital signal processing DSP algorithms owing to its high performance FPGA devices and large on board memo ries Two mezzanine slots per board can be populated with add on cards of various types to adapt the signal inputs to the particular requirement High speed data buses with a through put of 48 Gbit s per mezzanine slot transmit the acquired signals directly to the FPGA on the mainboard Currently three types of mezzanine cards are available which can be chosen depending on the intended functionality and the signals that arrive from the detector front end The ADC mezzanine card
169. full license was not obtained as of this writ ing The first generation of FPGA firmware designs for the TIGER module including all designs which were used during the 2012 DVCS data taking did not include this switch functionality A standard endpoint design using the Xilinx Virtex 6 FPGA Integrated Block for PCI Express IP core was used instead Details about the FPGA firmware and the driver software can be found in sections 6 1 1 and 6 2 3 respectively 5 3 Interfaces 97 5 3 Interfaces The interfaces provided by the CPU have already been specified in the previous section The re maining interfaces ofthe TIGER board which are directly connected to the FPGA are described in the following They can be classified into high speed I Os 5 3 1 5 3 3 an experiment syn chronous reference input 5 3 4 and general purpose I Os 5 3 5 5 3 1 VXS Interface Since the TIGER module is designed for the operation in VXS switch slots its backplane con nection consists of a large number of point to point links for communication with other mod ules in the crate As explained in section 4 3 1 each GANDALF payload board is directly con nected to each of the two switch boards via the so called VXS payload ports PP Furthermore the so called VXS switch ports SP provide direct connections between the two switch slots In total the TIGER board exhibits five VXS backplane connectors four of them P2 P5 being Tier 2 high speed connectors and
170. gas Straws Hodoscopes SciFis CAMERA Silicons GEMs Calorimeter Channels DCs MWPCs MWs SciFis BMS RICH 1 RICH 1 1400 Detector F1 TDC Discriminator Discriminator Analog in APV25 SADC dini m 1 B oD 1 0 a 0 D D D 0000000 N ELLI LE AO 119 DEN 250 Readout TCS TCS CATCH TCS GANDALF TCS GANDALF TCS TCS Hot CATCH GeSiCA Modules F1 TDC MI TDC ADC GeSiCA HE MUX TIGER MUX TIGER S Links SMUX SMUX MUX TIGER MUX TIGER l 95 Readout Buffer Spill Buffer in 30 ROB PCs 512 MB each 10m 100 MB s link 150 m 100 160 MB s link Gigabit Ethernet Gigabit Ethernet Switch Network 5km 700 MB s Event Building Event Builder amp Recording CDR CASTOR in 20 EVB PCs and Filter Figure 3 12 Overview of the COMPASS data acquisition system showing the number of involved mod ules and the maximum data transfer rates of the various links A schematic overview of the data flow from the detectors to the final storage in the CERN com puting center is given in Fig 3 12 The signals ofthe various detectors are digitized on front end cards which are either mounted directly on the detectors or located next to them According to requirements the digitizers are of the ADC TDC or Scaler type Upon receiving a trigger the digitized data is transferred via HOTLink based 59 copper or optical links to VME readout modules which can be either CATCH 6 GeSiCA or Ho
171. ges download php Xilinx Inc 2012 LogiCORE IP Aurora 64B 66B Product Guide PG074 SFF Committee 2001 Specification for SFP Small Formfactor Pluggable Transceiver INF 8074i C DECUSATIS 2008 Handbook of Fiber Optic Data Communication A Practical Guide to Optical Networking Elsevier Academic Press 3rd edn LEMO 2011 00 NIM CAMAC amp 01 Coaxial Connectors Honda Tsushin Kogyo 2008 HDRA series 0 8 mm Pitch Connectors Multi Inno Technology 2009 OLED Module Specification Model MI12832DO 176 Bibliography 159 P ASHENDEN 2008 The Designer s Guide to VHDL Morgan Kaufmann 3rd edn 160 P ASHENDEN 2007 Digital Design An Embedded Systems Approach Using VHDL Mor gan Kaufmann 161 Xilinx Inc 2010 Virtex 6 FPGA SelectIO Resources User Guide UG361 162 B GRUBE I KoNonov L SCHMITT 2001 COMPASS TCS documentation COMPASS note 2001 9 163 P KREMSER Diploma thesis in preparation ALU Freiburg Physikalisches Institut Private communications 164 S SABATINI G WENXUE 2012 PCIe SG DMA controller http opencores org project pcie sg dma 165 V L LORENZO 2009 Triple Port WISHBONE SPRAM Wrapper http opencores org project wb 3p spram wrapper 166 OpenCores 2010 WISHBONE System on Chip SoC Interconnection Architecture for Portable IP Cores 167 Xilinx Inc 2013 ChipScope Pro Software and Cores User Guide UG029 168 Xilinx Inc 2013 Partial Reconfiguration Tutorial UG743
172. he GANDALF transient ana lyzers corrected by the TO calibration constants To obtain the 1 offsets from the laser data histograms of the uncorrected time difference and uncorrected mean time are created for each counter and fitted with a Gaussian to determine the mean of the distributions The results are shown in Fig 7 2 The values from the fits are translated into 1 offsets for the individual channels in order to shift the time differences to zero and the mean times to a common value The target mean value fmean o is defined as the smallest integer value such that all t offsets are positive This choice is beneficial for the handling of the calibration offsets in the TIGER trigger processor The resultant offsets are tabulated in Tab 7 3 These 1 offsets are written into the configuration memory of the TIGER trigger processor with the ttconfig tool 77 p 60 They are used to correct the time stamps of the trigger primitives that arrive from the GANDALF modules For each ofthe 96 TIGER input channels the respective calibration offset is added before the pmt info objects are stored in the hit manager buffers cf 6 1 2 In addition the extracted TO calibration constants for the single channels are converted into calibration offsets for tmean fairr and trop They are needed in the online monitoring software coool to correct the time values of various shift plots These are some basic plots that are used 130 7 Commissioning R
173. higher the pulse amplitude is with respect to the ADC dynamic range It has been shown 66 that the dCFD timing resolution is inversely proportional to the pulse amplitude resulting in a timing resolution better than 100 ps for pulses exceeding 296 of the dynamic range and better than 20 ps for pulses which are larger than 10 of the dynamic range Further Detectors Due to the increased beam intensity for the DVCS measurement the readout of several detec tors has been changed from CATCH modules to GANDALF modules In particular the scintil lating fiber detectors in front of the target which sustain the highest hit rate are connected to GANDALF TDGs since 2012 Thereby the limit for the trigger latency could be slightly re laxed without degrading the detectors efficiency thanks to the much larger hit buffers of the FPGA based M1 TDCs 36 3 The COMPASS II Experiment 4 CAMERA Proton Trigger The COMPASS II GPD program includes the study of the DVCS process as described in section 2 5 To enable the selection of the rare DVCS events the development of a new trigger system which is sensitive to the recoil protons was mandatory The first section of the chapter at hand briefly recalls the measurement principle of the CAMERA detector In the following section the concept for the novel proton trigger system is developed Finally this chapter is concluded with a description of the electronics framework which is provided for the readout
174. if we look at a glass of wine closely enough we see the entire universe Richard Feynman The Feynman Lectures on Physics Volume I 3 7 1 Introduction When looking at matter closely enough one can resolve its inner structure The visible mat ter in our universe consists of two classes of particles leptons and hadrons According to the Standard Model of particle physics leptons are elementary particles without substructure whereas hadrons are composed of smaller constituents The most familiar hadrons are protons and neutrons which form the atomic nucleus and are therefore called nucleons The observed cross section for deep inelastic lepton nucleon scattering measured in several experiments in the 1960s led to the development of the parton model 1 2 In this model nucleons are composed of point like quasi free particles the so called partons which were later identified with the quarks and gluons According to current knowledge a nucleon is made up of three valence quarks surrounded by gluons mediating the strong force between them At short time scales the gluons may fluctuate into virtual quark antiquark pairs referred to as sea quarks The composition of the nucleons momentum from its constituents has been determined by several experiments showing that under certain conditions the quarks amp antiquarks and the gluons each carry approximately half of the momentum of the nucleon However the nucleon spin structure is
175. igger 10 Sq TG 8 E IE 6E Hp liquid Ims Boa IE E E J ho 4 E HN gt j o y amp sr ml 8 E L T Al TH 2 NW gq 8e Hi bi L 1 L 1 LIU L 1 HIHI L L LUTHI L 1 LIU L I I I 1111 0 1 1 0 10 100 1000 10000 By p Mc I 1 el I iss I 1 ERU I civil I 1 zl 0 1 1 0 10 100 1000 Muon momentum GeV c 0 1 1 0 10 100 1000 Pion momentum GeV c L L ril L L coil L I pori L L yd L LI 11111 0 1 1 0 10 100 1000 10000 Proton momentum GeV c Figure 4 2 Mean energy loss rate in several materials as a function of By p Mc with additional axes for muon pion and proton momenta Radiative effects are not included 17 p 325 The data points x for plastic scintillator are from 75 Proton 90 deg o o Proton 70 deg co o Proton 50 deg Pion 90 deg Pion 70 deg Pion 50 deg Energy loss in Ring B MeV gt al o N o o o o w o 20 10 Figure 4 3 Energy loss of protons red and pions blue in Ring B vs velocity p for tracks with a polar angle 0 of 50 70 and 90 Simulation with TGEANT 76 p 27 4 2 Trigger Concept 41 4 2 Trigger Concept 4 2 1 Design Objectives The existing COMPASS trigger system is mainly based on the detection of scattered muons However for the 2 5 m long liquid hydrogen target and the maximum possible beam intensity the anticipated trigger rate would saturate th
176. igger out lt 0 gt VXS_B lt 0 gt Tx0 Rx 0 0 Trigger in lt 0 gt Trigger out lt 1 gt VXS_B lt 1 gt Tx1 Rx1 1 Trigger in lt 1 gt A Trigger out lt 2 gt VXS_B lt 2 gt Tx2 Rx 2 2 Trigger in lt 2 gt Trigger out lt 3 gt VXS_B lt 3 gt Tx3 Rx3 3 Trigger in lt 3 gt Trigger out lt 4 gt VXS_B lt 4 gt Rx0 Tx0 4 Trigger in lt 4 gt E Trigger out lt 5 gt VXS_B lt 5 gt Rx 1 Tx1 5 Trigger in lt 5 gt Trigger out lt 6 gt VXS B 6 Rx2 Tx2 6 Trigger in lt 6 gt ES Trigger out lt 7 gt VXS_B lt 7 gt Rx 3 Tx3 7 Trigger in lt 7 gt optimize signal routing The mapping which is used in the GANDALF framework is shown in Tab 5 15 It specifies the relation between the geographic address of the payload slot the VXS port number of the switch slot and the I O bank assignment of the TIGER FPGA 5 3 1 2 Sideband Signals The VXS Tier 1 connector provides single ended pins used for low speed sideband signals Apart from a few system management pins inherited from VME64x SYSRST SYSFAIL GAO there are two signals available for each payload board PPn_SCL and PPn_SDA when referring to the VITA 41 x protocol layer standards In the custom protocol of the GANDALF framework these signals are employed for a handshaking system between the GANDALF and TIGER boards in the VXS crate Since they are not used as a 12C bus as the VITA 41 x names would imply the signals will be called PP_C n and PP D x in the following t
177. in 5 1 Mainboard 69 CPU LPC i CCLK CPLD ccLK CPLD PFXL Init B Program B D lt 15 0 gt Figure 5 13 Initial FPGA configuration from Figure 5 14 Online full reconfiguration of the Platform Flash XL after power up is controlled FPGA is executed by the CPU using the LPC bus by the CPLD to the CPLD burst read mode The device features a standard NOR flash interface which is an interface with full address and data buses for byte addressable random access operations in contrast to the paged access to NAND flash The Flash holds a basic configuration file for the FPGA the so called TIGER Base design see 6 1 1 This design contains only the static parts which are available in all FPGA user designs like the PCIe interface the I O structures and the clocking resources Immediately after the board has been powered up the CPLD initializes the download of the configuration bitstream from the Flash to the Virtex 6 via the 16 bit SelectMap interface Fig 5 13 By using a 40 MHz configuration clock CCLK the bitstream transfer takes only 50 ms Up to 1 ms of additional time is required to allow the FPGA start up sequence to finish which includes waiting for DCI matching and MMCM locking The end of the start up sequence is indicated by the FPGA s DONE signal It is necessary to perform the initial FPGA configuration before the COM Express CPU starts to boot so that the PCIe endpoint in the FPGA is
178. inally included in the global COMPASS trigger system where it is combined with trigger signals from other parts of the spectrometer The readout sub network depicted on the right hand side of the diagram is responsible for the collection of the detector data and its transmission to the DAQ Instead of connecting each GANDALF readout module via a dedicated S LINK to the DAQ a second TIGER module is used asareadout concentrator thus significantly reducing the number of required S LINKs and ROB PCs At the same time the TIGER readout concentrator distributes the TCS signal to all GAN DALF modules providing a synchronous reference clock and the COMPASS first level trigger More information about the FPGA design in this configuration is given in section 6 1 3 4 3 1 VME64x VXS Crate The hardware platform which has been chosen to implement the above described architec ture is based on the VME64x VXS standard ANSI VITA 41 0 80 It extends the parallel VME bus 81 which is widely used in nuclear and particle physics experiments since many years 4VME Switched Serial 48 4 CAMERA Proton Trigger by high speed serial interconnections for increased data transfer rates The VITA 41 standard provides a base specification VXS 0 for mechanics power and backplane connections and it defines two types of VXS boards payload cards and switch cards In addition various proto col layer specifications VXS 1 to VXS 5 are available which d
179. ined as shown in Fig 4 9 to accommodate the data flow which is mainly unidirectional For the connection to the trigger processor all eight lanes are used in the direction G T whereas the connection to the readout concentrator is sub divided into six lanes for event data transmission G T and two lanes for the TCS distribution T G 4 3 2 Backplane Link The challenge while implementing the custom transmission protocol for the VXS backplane was to find a trade off between the different requirements that arose from the task to send the trigger primitives from the GANDALF modules to the TIGER trigger processor The most critical requirement is a predictable latency because hit information which are not received in time cannot be considered for the trigger generation anymore In addition the different detector channels should not influence each other in terms of transmission rate and latency to avoid that for example one noisy channel can saturate the link and suppress hits from other channels Fortunately the number of available transmission lanes of the VXS link to the trigger processor just equals the number of digitization channels of the GANDALF transient analyzer which allowed a one to one mapping The pulse processing algorithm in the GANDALF module is continuously looking for hits on the input channels and calculating its parameters with high resolution 74 p 88 However for the trigger processing the pulse feature informa
180. ing B laser pulse 280 270 P A 260 250 amplitude ADC bins 240 230 220 9 Pain Pa deccm pem 210 av my 200 Patata un uu T gen ren ligero rmm 0 50 100 150 200 250 300 350 400 frame time sample intervals Figure 7 1 Typical PMT signal for a laser pulse in a Ring B counter of the CAMERA detector The plot is generated with the GANDALF Toolbox 163 a GUI for parsing COMPASS raw data 1ADCbin 1mV lsample interval 0 989 ns 2 time difference of A elements 0 Bs 2 E e e ha o o e e e e ee e e e o o 4 e e e e e 2 6 8 0 5 t up t dn in ns 0 5 t up t dn in ns Figure 7 2 Uncorrected time difference and mean time between upstream and downstream PMTs of the CAMERA scintillator elements Results of the Gaussian fits mean and sigma are plotted for each detector element Laser data from run 108912 On counter B11 there was a problem with the laser fiber very low amplitude of the laser signal 7 1 CAMERA Installation 131 tg t in ns 2002 4 6 8 10 12 14 16 18 20 22 4 6 8 10 12 14 16 18 20 22 Channel Channel 2060 2070 t t4 0 5 in ns up t tan 0 5 in ns up 1 N e e 1 N e o 5 10 15 20 15 20 25 30 35 40 45 5 10 15 20 Channel Channel not calibrated TO calibrated Figure 7 3 CAMERA monitoring plots for run 108898 with pion beam The plots in the left column are
181. ing to the system memory and IO map and ID routing is used for completions and configuration requests which refer to the target s logical position in the hierarchy i e bus device function number Implicit routing is based on the knowledge of a switch in which direction the root complex is to be found Message transactions may be routed by any of the three methods 143 p 117 A special case is the routing of configuration requests which can only be issued by the root complex and can be either of type 0 or type 1 A type 0 configuration request is always con sumed by the device which receives it Hence if the root complex wants to send a configura tion request to a device further down in the hierarchy it will issue a type 1 request The only devices paying attention to a type 1 request are PCI to PCI bridges They handle the packet de pending on the target bus number which is specified in the header If the target bus equals the bus directly connected to the bridge s downstream side called the secondary bus the bridge converts the packet from type 1 to type 0 by changing the header and passes it to the secondary bus If the target bus is located beneath the secondary bus the packet is forwarded as a type 1 configuration request without modification 143 p 732 5 2 6 4 TIGER PCI Express Link Structure COM Express CPU Virtex 6 FPGA and MXM GPU of the TIGER board are connected by PCIe links as shown in Fig 5 31 The CPU contains four P
182. ion has been performed the measured PMT pulse amplitudes are used to obtain additional information for the particle identification The geometric mean Ex V Ex up Ex an of the upstream and downstream PMT pulse amplitudes after calibration is pro portional to the energy loss of the particle in the scintillator The mean energy loss of a charged particle traversing material is described by the Bethe Bloch equation 17 p 324 2 p2 2 ekz m me Pme pe PUT 4 8 ABl pn 2 Here Tmax is the maximum kinetic energy transfer in a single collision z is the charge number of the incident particle y 1 0 is the Lorentz factor and K 0 307 MeVg cm combines some constants 4 1 Detector Principle 39 20 A 20 B Figure 4 1 Principle of the Time of Flight measurement with the CAMERA detector A recoil particle p generates scintillation light when crossing the detector elements A and B indicated by the red dots The light pulses propagate to the upstream and downstream ends of the counters red dashed arrows where they are detected by PMTs From the pulse arrival times the particle track can be reconstructed In a specific material with charge number Z atomic mass A and mean excitation energy I the stopping power i e dE dx for charged particles with mass M gt me is approximately dE 1 a In const 8 y 4 9 The Bethe Bloch equation is valid in the region 0 1 lt By S 1000 At the lower limit on
183. ions like energy thresholds or cuts in the energy loss distribution cf Fig 4 7 Coincidences that survive these cuts enter the fourth and last stage the release logic Its purpose is to buffer the trigger decision until it is finally released at the right time to fulfill the constant latency requirement cf 4 2 3 As a result a single pulse is generated at the LVTTL trigger output In total 12 GANDALF modules digitize the 96 PMT channels of the CAMERA detector and trans mit the information about detected pulses via the backplane to the TIGER trigger processor 3However some elements MMCM GTX transceiver provide a dynamic reconfiguration port to change their set tings 6 1 FPGA Firmware 117 slat logic slat info 0 slat info 1 pmt info 0 pmt info 1 hit m down pmt info 0 pmt info 1 pmt info 0 pmt info 1 hit m up rack no T track_info 1 I Signal sys time slat_info 0 slat info 1 pmt info 0 pmt_info 1 Figure 6 5 The TIGER trigger logic is arranged in four stages The trigger primitives arriving from the backplane are combined to form hit information and these are further combined into tracks After applying optional cuts the remaining coincidences cause trigger time stamps being enqueued in the release buffer Subsequently a trigger pulse is emitted at the specified time 77 p 46 hit m down The VXS ports that are used for the reception of trigger primitive
184. ister file created with the SiLabs ClockBuilder Desktop Software writes the data via the I C bus to the chip and initializes the locking process As soon as the PLL has locked it writes the current VCO calibration values to the FCAL registers cf flow chart in 102 p 21 usage si5338prog registerfile arguments registerfile path to the Si5338 register txt file 126 6 Firmware and Software displayinit This tool is used to initialize the front panel display Multi Inno MI12832DO according to the procedure in 158 p 14 The display contains a SSD1306 controller 180 which is programmed using the Adafruit SSD1306 OLED driver library 181 usage displayinit ttconfig and trigger FIFO The tools ttconfig and trigger FIFO are specific to the TIGER trigger processor design Their usage is described in 77 7 Commissioning 7 1 CAMERA Installation During a changeover period in September 2012 the COMPASS experiment was prepared to perform DVCS test measurements Among several other changes the newly designed liquid hydrogen target and the surrounding CAMERA detector have been put into operation The CAMERA detector had been completely mounted and tested in advance in a construction hall clean area next to the COMPASS hall which allowed to crane the whole structure right into the experimental area The preparations included the following steps mounting the scintillator light guide element
185. ists of two layers with an active detector area of 48 x 48 mm The layers are built from round scintillating fibers with a diameter of 2 5 mm which are staggered with a pitch of 1 5 mm Clear light guiding fibers are welded to the active fibers forwarding the light to 16 channel multi anode PMTs The electrical readout is performed with GANDALF TDC modules For the FI13 U and V stations 32 fibers per layer are connected to dedicated PMT channels and the layers are alternately read out to the left and the right hand side This allows to calculate a meantime between the two corresponding layers in order to minimize the variation of the hit time depending on the hit position along the fiber The FI12Y station is constructed with triple layers where columns of three fibers each are coupled to one PMT pad thus yielding more light for an improved time resolution The two layers of this station are also read out one to the left and one to the right hand side for meantiming Compared to the existing SciFi stations FIO1 and FI02 which are built with 0 5 mm fibers and provide a time resolution in the order of 500 ps the StartCounter exhibits a significantly better time resolution thanks to the larger fibers First studies show that resolutions of approx 320 ps can be expected for a single station resulting in a combined resolution of ogc lt 200 ps for the StartCounter 79 Hence the StartCounter can be used to calculate a precise timestamp for the primar
186. ith 10 A each are providing the 2 5 V and 3 3 V rails The UCD9246 device is a four rail digital controller for DC DC step down converters For each rail a set of parameters including the desired output voltage the maximum allowed current and the behavior in case of failures is programmed into the integrated flash memory during the initial start up of the system using an external programming cable A PMBus interface is used to communicate with the UCD9246 PMBus is a variant of the I C based SMBus pro tocol specifically designed to control power supplies A software tool Fusion Digital Power Designer is available from TI which provides a graphical user interface to configure the de vice parameters and monitor the operation for debug purposes Once a valid configuration has 22 Power Management Bus 23 System Management Bus 5 1 Mainboard 73 UCD9246 CPLD PWM controller JTAG VCC5VO control amp monitor U Temp PTD converter modules PTD08A020W QO VCCINT PTD08A020W VCC1V5 TPS51200 O VTTDDR 3x TPS74401 O MGTAVCC AVTT PTD08A010W VCC2V5 PTD08A010W VCC3V3 SO VCC5VO LTM4605 O VCC GPU Figure 5 18 Generation of the secondary voltage rails by switched mode DC DC converters and LDO regulators been written to the flash memory the device
187. ith pull up resistors 31 Secure Digital and Secure Digital High Capacity 32 Flexible Flat Cable 33 Kontron ADA LVDS DVI 24bit Part No 96007 0000 00 1 34Inter Integrated Circuit bus 84 5 The TIGER Module to the supply voltage A device connected to the I C bus can either assert its output low thus forcing the corresponding signal line to GND which indicates a 0 or keep the output floating which results in the signal line being pulled up to indicate a 1 A detailed description of the IC bus protocol can be found in the specification 129 The implementation on the COMe mTT10 supports a clock speed of up to 400 kHz Fast mode 7 bit and 10bit addressing modes and clock stretching which allows slave devices to delay the data transfer In addition the Intel Atom CPU features a SMBus System Management Bus in terface which is also forwarded to the COM Express connector The SMBus protocol is derived from the I C bus specification and it is commonly used on PC mainboards to communicate with power and clock related chips and hardware monitoring devices The maximum sup ported clock speed is 100 KHz From the software point of view the SMBus is handled just like an IC bus The internal bus numbers of the COMe mTT10 are 0 for the SMBus and 1 for the C bus Fig 5 26 shows the structure of the SMBus and the I C bus on the TIGER module Each bus connects devices which are located on the COM Express module as well as devic
188. iver modules commonly used in optical networking applications SFP modules provide an electrical interface to the moth erboard and an optical interface to connect a fiber optic cable Modules with various optical transceiver types are available for both multi mode and single mode fibers The most popular optical connector is the LC type 155 p 57 For communication over short distances of up to 550 m usually multi mode fibers are used in conjunction with LEDs or laser diodes operating at a wavelength of 850 nm transceiver type SX Typical multi mode fibers have a core diameter of 50um and a cladding diameter of 125 um A classification system is used to specify the modal bandwidth i e the bandwidth distance product To this day it defines four classes OMI to OMA The most common multi mode fibers today are OM3 50 125 and OMA 50 125 which can be recognized by their cyan jacket in contrast to the older OM1 and OM2 fibers which are usually orange 155 p 69 For a connection to the DAQ system the TIGER board is equipped with up to two SFP mod ules Currently the CERN developed HOLA S LINK protocol is used at COMPASS to con nect the TIGER data concentrator to the readout buffer PCs providing a peak data rate of 160 MB s per link cf 6 1 3 However a future upgrade of the DAQ readout PCs could yield data rates of up to 6 6 Gbit s which is the limit of the Virtex 6 GTX transceivers 5 3 4 TCS Interface The TIGER module features a
189. latform _ Flash XL ANSITI 2296 2N ur E ASOSHALT 1 aanp ESD strips YE EMC front panel MXM GPU VXS backplane Power connectors connector Figure 5 2 Photo ofthe TIGER module and of its front panel The FPGA and its related components and interfaces are indicated with blue labels The CPU with the corresponding interfaces is labeled in orange color Areas related to the power supply are marked with green borders All components and interfaces are detailed in the chapter at hand 56 5 The TIGER Module While the first devices mainly consisted of programmable logic gates with programmable in terconnects new features were added in the following device generations Nowadays FPGA devices are much more than just programmable gate arrays in fact they contain complex embedded function blocks for digital signal processing DSP clock management or fast serial communication to mention just a few examples Recently a new device family 90 based on a System on Chip architecture has been introduced combining an ARM multi core processing system with traditional programmable logic on the same chip The FPGA device which was chosen for the TIGER module is the XC6VSX315T with speed grade 2 in the FF1759 package Its most important functional elements are briefly described in the following subsections and the key features are summarized in Tab 5 1 The development of the TIGER FPGA firmware which implements the specific behavi
190. lock and the divided word clock offering dedicated reference clock inputs for best possible performance The TX output buffer features a programmable differential voltage swing and a pre emphasis driver to maximize signal integrity The receiver contains equalization circuits to compensate for high frequency losses in the channel Both TX and RX settings must be optimized to ac commodate the transceivers to the specific channel characteristics This is especially relevant for the higher data rates in order to reach a satisfying bit error ratio On the TIGER module GTX transceivers are utilized for PCI Express connections between the FPGA and the CPU and GPU extension boards see 5 2 for a high speed link to the adjacent TIGER module via the VXS backplane 5 3 2 and for optical fiber connections to the DAQ sys tem 5 3 3 5 1 2 DDR3 Memory A SO DIMM socket for a DDR3 memory module is directly connected to the Virtex 6 FPGA on the TIGER board providing a temporary storage capability for example to buffer detector data from the readout modules which may incur at peak rates during the on spill period To gain access to the DDR3 memory in the FPGA design an interface is created using the Xilinx Memory 64 5 The TIGER Module Interface Generator MIG wizard 95 This graphical tool generates IP core files containing the controller and physical layer which are necessary to support external SDRAM modules Detailed information about the s
191. m upstream unsupported forward TLP request UR upstream Figure 5 33 Flow chart for the routing decision of the virtual PCIe switch which is implemented in the TIGER FPGA as DMA transfers To implement the switching functionality outlined above two instances of the core are utilized to form the upstream and downstream switch ports A user application module was developed in the hardware description language Verilog to implement all switch specific functions since they are neither part of the core nor part of the reference design All routing mechanisms which have been described above are supported by the switch module In addition the configuration space interface of the downstream bridge is controlled by the user code to enable access to the configuration space registers via CfgRd and CfgWr transac tions during the enumeration process The design was extensively tested in the course of a product evaluation with simulations in ModelSim 147 using PLDAs PCI Express Bus Functional Model BFM 148 The BFM emu lates a PCIe environment and allows to send and receive packets monitor bus activity and re port errors Two instances of the BFM have been connected to the user design to verify correct behavior of the switching logic and the configuration space access Unfortunately a synthesis of the design for the Virtex 6 hardware was not possible with the evaluation version since the PHY code is not provided with this package and a
192. me where the nucleon carries an infinite momentum opposite to the one ofthe virtual photon 13 p 77 In this reference frame the transverse momenta of the constituents are neglected because they are small compared to the longitudinal components Furthermore time dilatation implies that there is no time for interaction between the nucleon constituents and the scattering process can therefore be treated in impulse approximation The following Lorentz invariant kinematic variables are used to describe the DIS q k ky 0 squared four momentum of the virtual photon 2 2 P q v SE energy loss of the scattered lepton 2 3 s k py squared invariant center of mass energy 2 4 w P gy squared invariant mass of the photon nucleon system 2 5 where k and k are the four momenta of the incoming and scattered lepton M is the mass of the nucleon and P is its 4 momentum Because q is negative the virtuality of the photon is defined as Q q In the lab frame of a fixed target experiment where the nucleon is at rest prior to the interac tion these variables are 0 gb ABE sin gt 2 6 2 3 Parton Distribution Functions 5 I k I k hard soft N P Figure 2 2 Factorization of the deep inelastic scattering into a hard leptonic and a soft hadronic part The soft part is parametrized by parton distribution functions Indicated in brackets are the four momenta of the respective particles yb p ur
193. n interface to the COMPASS trigger control system which is iden tical to the one ofthe GANDALF module The TCS signal which is distributed by optical fibers to every readout board in the experiment provides a global reference clock as well as first level triggers and SPS spill information cf 3 4 In order to permit the employment of the TIGER module also in other experiments which may use a different system for the distribution of trig gers the TCS interface is held flexible thanks to exchangeable add on cards the so called Gimli cards 64 p 45 The standard type provides an input for the optical TCS fiber and performs a clock recovery from the serial data stream Two LVDS signals are finally transmitted from the Gimli card to the TIGER board the TCS reference clock 155 52 MHz at COMPASS and the TCS data stream The connector pin out is to be found in 63 p 167 The trigger data and clock signals from the add on card are available in the Virtex 6 FPGA where the decoding of the TCS information is performed cf 6 1 1 3 Furthermore a copy of the clock signal is provided to the Si5326A clock multiplier chip in order to generate additional experiment synchronous clocks cf 5 1 3 2 An alternative version of the Gimli card is available featuring inputs for clock and trigger signals as well as an oven controlled crystal oscillator which provides a stable low jitter 20 MHz reference clock 5 3 5 General Purpose I O A number of general p
194. n the port VXSPORT_n i The direction of the links is chosen in the FPGA firmware design depending on the application ofthe TIGER board In the trigger design 6 1 2 all eight links are configured as inputs to receive the trigger primitives the readout design 6 1 3 uses six inputs to receive the event data and two outputs to transmit the TCS information The link assignment is detailed in Tab 5 14 The VITA 41 documents do not define the mapping between VXS port numbers and crate slot numbers 80 p 27 This is left to the manufacturer of the backplane and may be chosen to 49Tyco 1410137 1 1410138 1 1410139 1 98 5 The TIGER Module Table 5 14 Usage ofthe VXS links between the GANDALF and the TIGER modules The backplane signal names in the central columns refer to the VITA 41 specification This does however not reflect the link direction as defined in the GANDALF framework GANDALF VXS backplane signal TIGER usage signal name GANDALF TIGER linkindex usage VXSlink clkout VXS A 0 Tx0 Rx 0 0 VXS link clk in S LINK out lt 0 gt VXS_A lt 1 gt Tx 1 Rx 1 1 S Muxin lt 0 gt lt S LINK out lt 1 gt VXS_A lt 2 gt Tx2 Rx2 2 S Mux in lt 1 gt S S LINK out lt 2 gt VXS_A lt 3 gt Tx3 Rx 3 3 S Mux in lt 2 gt S LINK out lt 3 gt VXS_A lt 4 gt Rx 0 Tx0 4 S Mux in lt 3 gt E VXS_A lt 5 gt Rx 1 Tx 1 5 x TCS data in VXS A 6 Rx2 Tx2 6 TCS data out e TCS clock in VXS_A lt 7 gt Rx3 Tx3 7 TCS clock out Tr
195. network It is built up of ten primary laser transmitters with subsequent passive optical 1 32 splitters yielding a total of 320 available fiber connections 56 To cut down on the number of required TCS fiber outlets the readout concentrator is also used to distribute the TCS clock and data signal via the backplane to all GANDALF modules and to the second TIGER module The VXS lines with indices 6 and 7 of each payload and switch port transmit the TCS data and clock signals respectively Virtex 6 output DDR registers ODDR are operated for this purpose in a clock forwarding mode 161 p 117 in order to minimize the skew For locations where a TCS is not available e g at test beams or in the lab the functionality of the TCS controller 162 p 11 has been implemented in a VHDL module which may be in stantiated in the readout concentrator design Different trigger modes are available including an internal pseudo random trigger with variable rate a 1 KHz clock like trigger and an exter nal trigger provided by a NIM signal Event and spill numbers are incremented automatically and the first level trigger and event label information are multiplexed following the same pro cedure as in the COMPASS TCS This allows to run a stand alone DAQ system consisting of a TIGER readout concentrator module and up to 18 GANDALF boards in one VXS crate 6 2 Operating System and Device Drivers Since the COM Express module features a standard x86 CPU
196. nix Inc 2010 DDR3 SDRAM Unbuffered SODIMMs HMT325S6BFR8C Data sheet 97 SK Hynix Inc 2010 DDR3 SDRAM Device Operation 98 Integrated Device Technology Inc 2011 ICS9DB102 Two Output Differential Buffer for PCIe Gen1 8 Gen2 Data sheet 99 ON Semiconductor 2011 NB6N11S 3 3 V 1 2 AnyLevel Input to LVDS Fanout Buffer Translator Data sheet Bibliography 173 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 Silicon Laboratories 2010 Si5326 Any Frequency Precision Clock Multiplier Jitter Atten uator Data sheet Silicon Laboratories 2011 Any Frequency Precision Clocks Family Reference Manual Si53xx RM Silicon Laboratories 2011 Si5338 P C Programmable Any Frequency Any Output Quad Clock Generator Data sheet Silicon Laboratories 2012 Si5335 38 51 56 ClockBuilder Desktop Software Xilinx Inc 2008 CoolRunner II CPLD Family DS090 Xilinx Inc 2007 XC2C512 CoolRunner II CPLD DS096 Xilinx Inc ISE Design Suite http www xilinx com products design tools ise design suite Xilinx Inc 2010 Platform Flash XL High Density Configuration and Storage Device DS617 Xilinx Inc 2012 Virtex 6 FPGA Configuration User Guide UG360 Xilinx Inc 2012 Partial Reconfiguration User Guide UG702 IEEE Computer Society 2008 IEEE Standard Test Access Port and Boundary Scan Archi tecture IEEE Std
197. nnected to a 3 port memory bridge 165 featuring a WISHBONE slave interface for each port This allows up to three WISHBONE 166 compliant masters in the user application to access the Configmem The bridge arbitrates the requests of the masters in a round robin manner A typical use case for the Configmem is the online update of certain registers in a VHDL component First the new values are written to the appropriate address in the Configmem and then a Fast register is toggled which triggers a VHDL process to transfer the values from the Configmem to the registers The third interface between the PCIe core and the user application logic are two 64 bit wide FIFOs with configurable depth 16 k by default A board to host FIFO is available for data transmission from the FPGA logic to the CPU s memory and a host to board FIFO is used to transmit data in the opposite direction For highest performance the FIFOs should be accessed in DMA transfer mode 6 1 1 5 ChipScope Cores Xilinx ChipScope Pro 167 is a powerful tool set for in circuit debugging and verification of FPGA designs It provides the firmware developer with a convenient logic analyzer solution which is able to monitor virtually any signal in the design without the need to bring it out to an external test pin The ChipScope solution consists of a set of IP cores which are customized and generated by means of the Xilinx CORE Generator tool and inserted into the VHDL design under test
198. not clear today The naive assumption that the spin of the nucleon is mainly carried by the quarks amp antiquarks has been disproven in the late 1980s by an experiment of the European Muon Collaboration 3 which was confirmed by further experiments at CERN 4 SLAC 5 and DESY 6 Parton helicity distributions of the nucleon which were extracted from these data sets yield a contribution of the quark amp antiquark helici ties AZ of only about 30 In addition the contribution of the gluon helicity AG to the nucleon spin seems also to be small The gluon helicity distribution Ag x for a mean value of the gluon momentum fraction x 0 2 has been determined recently at the COMPASS experiment at CERN 7 indicating that its first moment is small and in the order of AG 0 25 Since the quark amp antiquark and gluon helicities alone do not suffice to yield the nucleon spin this raises the question about the contributions of the orbital angular momenta of the quarks and gluons Common Muon and Proton Apparatus for Structure and Spectroscopy 2 1 Introduction During the last 20 years the parton model has been advanced in order to clarify the spin struc ture of the nucleon The concept of generalized parton distributions GPDs has been intro duced which provides for the first time access to the total angular momentum of the partons Hopefully this will finally help to solve the last part of the long standing spin puzzle The
199. ntering the experiment hall 35 p 13 To reach sufficiently large beam intensity a deviation ofthe muon momentum ofup to 5 from the nominal value is accepted This requires a momentum measurement for every single beam particle The measurement is performed with the Beam Momentum Stations BMS which are positioned around the last bending magnet B6 of the beam line Fig 3 2 The BMS are hodoscopes to measure the particle tracks in front and behind the magnet which allows to calculate the muon momentum 35 p 15 The muons are naturally polarized due to the parity violation of the pion and kaon decay The degree of polarization depends on the momentum ratio of muons and hadrons With a ratio of Pu Px 160 GeV c 172 GeV c one can reach a polarization of 80 4 and an intensity of 3 8 10 muons per spill for a u beam The maximum possible intensity for a u beam is about one third of that value 27 p 19 The majority of the muons is concentrated in the central beam spot but there are also some muons with larger distance to the beam axis they form the so called halo It has already been mentioned in section 2 6 that the luminosity has to be determined with a precision of the order of 3 This requires an accurate measurement of the muon beam flux which is done by means of scintillating fiber detectors 31 p 18 They are used to sample the beam at random points in time to determine incident beam tracks At the same time these fi
200. nterpreted in a phenomenological way 28 In this case the momentum transfer is purely transverse i e t A3 Here and in the following boldface symbols denote 2 dimensional variables in the transverse plane Like the Fourier transforma tions ofthe nucleon form factors show the distribution of charge in the nucleon the GPDs pro vide information about the spatial distribution of partons with a momentum fraction x The impact parameter dependent parton distributions are defined as the Fourier transform of the A dependence of the GPD Hf x 0 A d de e Au Pt H x 9 A 2 34 qr x b The result is a kind of 3 dimensional picture of the parton distributions in the nucleon One dimension is the longitudinal fractional momentum x which is parallel to the virtual photon The two others give the position in the transverse plane the impact parameter b A simul taneous measurement of both x and the longitudinal position is forbidden by the uncertainty principle The impact parameter dependent parton distributions qr x b for a set of different values of x are visualized in Fig 2 8 which can be interpreted as momentum dissected images of the 2 4 Generalized Parton Distributions 13 Figure 2 9 Qualitative form of the impact parameter dependent PDFs qr x b1 25 nucleon Depending on the momentum fraction x either the valence quarks the pion cloud or the sea quarks and gluons are resolved
201. nto an ideal data buffer between different clock domains For first in first out FIFO applications the BRAM elements contain dedicated Phase Locked Loop 8 Available values 1 2 4 9 18 or 36 bit For a data width smaller than 9 only 32 kbit of the memory can be used 60 5 The TIGER Module control logic for the address counter and status flag generation like FIFO full or FIFO empty Either a synchronous or an asynchronous i e dual clock FIFO can be implemented using one or several cascaded BRAM elements depending on the required depth FIFO elements and block memories are extensively used in the TIGER design to buffer the PMT hit and track information between the individual stages of the trigger processing cf 6 1 2 This is one of the reasons for the choice of an SXT FPGA which has considerably more BRAM elements than the LXT models Another application for BRAM elements is the implementation of read only memory to store constants or look up tables for the use by DSP algorithms e g the energy cuts according to section 4 2 2 3 for the proton selection of the trigger 5 1 1 4 DSP Slices For fast digital signal processing applications the Virtex 6 FPGA provides dedicated DSP48E1 slices A simplified diagram of the slice content is given in Fig 5 8 The central element is a 25 bit x 18 bit multiplier with a pre adder on one input followed by data path multiplexers X Y Z and a 48 bit 3 input adder subtractor The output of
202. o avoid confusion The handshaking was implemented to ensure the hot swap capability of the boards allowing to safely insert them into and remove them from the crate at any time The need for such a protection system arises from the fact that the FPGAs on the TIGER and the GANDALF board are directly connected to each other by the VXS backplane signals without any DC blocking capacitors buffer or driver chips or similar devices This poses a possible danger for the FP GAs in situations when one of the boards is unpowered The FPGAS I O buffers are designed with a conventional CMOS output structure containing two clamp diodes on every pin one to VCCO and one to GND 149 When driving an unpowered bank the clamp diodes will become 50Tyco 1410421 1 5 3 Interfaces 99 Table 5 15 Mapping between the VXS port number and the slot number called geographic address for the Hartmann backplane 83 used in the GANDALF framework as well as the assignment of Virtex 6 I O banks on the TIGER module Since one FPGA I O bank with 20 differential pairs provides connections for 2 5 VXS ports some ports are split between neighboring banks VXS port Slot GA FPGAI O bank 1 10 14 2 13 14 3 9 15 14 4 8 13 12 5 14 15 6 15 13 7 7 15 8 6 13 9 16 16 10 17 12 11 5 16 12 4 12 13 18 17 16 14 19 22 23 15 3 17 16 2 22 17 20 17 18 21 22 forward biased For this case the Virtex 6 data sheet specifies a maximum current of 1
203. o connect low bandwidth devices like legacy I O controllers or au dio chips On the TIGER board it is connected to the CoolRunner II CPLD to provide various options for the FPGA configuration cf 5 1 5 35The ID interface of the SFP modules can t be used due to an address conflict with the JIDA EEPROM 5 2 CPU and GPU Extension Boards 85 I C bus SMBus JIDA EEPROM SPD EEPROM clock generator MXM GPU slot DDR3 SO DIMM SFP transceiver 1 SFP transceiver 2 Figure 5 26 SMBus and I C bus structure with connected devices The dashed line indicates the tran sition of the buses from the COM to the base board Devices shown above this line are located on the COMe module while devices shown below this line are located on the TIGER module Slave Address Control Register spa RBBB EIS ERE EDIDI ERE EIC COSS Start Condition R W ACK From Slave ACK From Slave Stop Condition Figure 5 27 I C data transfer to write to the control register of the PCA9546A switch with slave address EEh The four lower bits of the control register indicate which channels are enabled Any combination of sub buses may be enabled by writing 1 s to the corresponding selection bits Bi 130 Table 5 8 12C bus device addresses Devices listed in the upper section of the table are located on the COMe module devices listed in the middle section of the table are located on the TIGER module The lower section contains the sub buses which are
204. o cover the full length of the target and it has to provide a large polar angular acceptance taking into account the kinematic distribution of the recoil protons The two barrels of the detector are each segmented into 24 scintillator slats The inner barrel Ring A with a diameter of 52 cm is composed of 275 cm long slats with a thickness of only 4 mm This allows even slow protons down to p 260 MeV c to pass the material and reach the outer barrel for a time of flight measurement In contrast the outer barrel Ring B with a diameter of 230 cm is made from 5 cm thick material for good energy resolution The slats are 360 cm long to provide a polar angular acceptance from 45 to 90 with respect to the beam axis For an enhanced azimuthal angular resolution the two barrels are rotated by half a segment i e 7 5 against each other The exact dimensions of the counters are given in Tab 3 1 The scintillating elements are made from BC 408 plastic scintillator material 43 This ma terial was chosen because it has a long light attenuation length which is important for such long elements but is still reasonable fast for time of flight measurements Both ends of the elements are coupled via light guides to photo multiplier tubes PMT which are placed out side of the spectrometer acceptance In particular for the downstream side of Ring A very long light guides had to be used Fig 3 6 Due to space constraints upstream and downstream of the de
205. of standard Cu coated PCB material with a thickness between 0 2 mm and 1 0 mm Therefore the THGEM layers are quite stable allowing to produce large size detectors The layers with a size of up to 60 x 60 cm are manufactured with standard PCB technology by drilling holes with a typical diameter of 0 4 mm and a pitch of 0 8 mm Finally a clearance ring around the hole is produced by Cu etching see Fig 3 9 A THGEM based detector Fig 3 10 is typically constructed of a plane of wires defining a drift field three THGEM layers for electron multiplication and an anode layer to collect the charge The detector is covered with a fused silica window to allow UV photons to enter the chamber and it is operated with a Ar CHy gas mixture The top side of the first THGEM is coated with CsI to act as a photo cathode To each THGEM layer a bias voltage AV is applied between top and bottom side generating an electric field which is particularly strong in the holes leading to electron multiplication Between the layers transfer fields guide the electrons to the next am plification stage and the induction field finally guides the electrons to the anode pads where the signals are read out 1lTHick GEM 12 Printed Circuit Board 30 3 The COMPASS II Experiment Pitch 2 800 uni gt t k Figure 3 9 The THGEM layer is produced of Cu coated halogen free fiberglass PCB material The holes are drilled and the clearance ring rim is
206. oftware reconfigured on the fly The user defined part of a TIGER design is developed as a RM It may contain logic that is mapped to CLBs Block RAM and FIFO elements and DSP blocks All sig nals that cross the boundary between the static partition and the reconfigurable partition have to use so called partition pins These are dedicated logic elements forming a routing bridge basically a 1 input look up table They are inserted automatically during the implementation process to guarantee that the connection points for any RM are at the same position Compared to the standard Base design the PR capable version implements the ICAP compo nent of the Virtex 6 and some additional logic It forwards the partial bitstream from the PCIe host to board FIFO to the SelectMap like interface of the ICAP To ensure a well defined behav ior during the reconfiguration process all signals that enter or leave the RP must be registered with flip flops at the partition boundary It is beneficial to do this on both sides for eased timing closure The static logic must ignore the outputs from the RP during the reconfiguration since the signals at the partition pins are undefined until the configuration is complete Hence the enable signal of the flip flops is used to isolate the logic of the two partitions 6 1 2 TIGER Trigger Processor Design The first application specific project derived from the Base design is the TIGER trigger proces sor Itimplements th
207. ogic devices has been designed during the thesis at hand The development of the trigger system is detailed in the following chapters For the calibration and monitoring of the CAMERA detector a laser system has been installed Optical fibers are connected to the center of each scintillator for the distribution of laser ref erence pulses These pulses create very precise time stamps simultaneously on all detector elements which are used for timing calibration The TO calibration procedure is explained in section 7 1 1 Furthermore the measurement of the pulse amplitudes simplifies the procedure of finding the correct high voltage settings for the PMTs and it allows for the monitoring of the PMT gain stability trapezoidal cross section 6 measured to the center of the scintillator 26 3 The COMPASS II Experiment 3 3 TheSpectrometer 3 3 1 Tracking Detectors In the COMPASS spectrometer a large variety of tracking detectors is used to precisely deter mine the tracks of final state particles They are also utilized to measure the momentum of particles that are deflected in the dipole field of the spectrometer magnets Table 3 2 summa rizes the properties of the different tracking detectors Depending on the distance from the beam axis different spatial and time resolutions and rate capabilities are required Hence the tracking detectors are classified in three regions 35 p 10 Closest to the beam there are the Very small area tracke
208. one P1 being a Tier 1 sideband connector 80 p 18 5 3 1 1 High Speed Signals The VXS Tier 2 connectors are arranged without gaps in between each other to form a mono lithic connection area featuring a total of 192 differential pair signals which are shielded by intermediate ground pins to reduce crosstalk On the TIGER board the signals are allocated as follows pin out tables in appendix C 1 18x8 144 differential pairs to the GANDALF boards 16 differential pairs to the second TIGER board 32unused differential pairs not connected While the VXS interfaces of the GANDALF and TIGER modules are electrically and mechani cally designed according to the VITA 41 0 specification they do not comply with any VXS proto col layer as defined in the VITA 41 x standards These standards are packet switched serial pro tocols like InfiniBand Serial RapidIO 1 10 Gigabit Ethernet or PCI Express But due to specific requirements of the CAMERA trigger and readout application a custom transmission protocol was developed for the VXS communication in the GANDALF framework cf 4 3 2 The TIGER module provides a dedicated VXS port to every payload slot in the crate each consisting of eight unidirectional point to point links based on a 500 MHz DDR LVDS transmission The nomen clature of the TIGER VXS links is given by the following pattern with n 1 2 18 being the number of the VXS port and i 0 1 7 being the link index withi
209. onverters 5 1 6 2 Secondary Voltage Rails The Virtex 6 FPGA as well as the surrounding auxiliary devices require a number of different supply voltages which have to be generated by onboard power converters For efficiency rea sons the application of switched mode DC DC converters is preferred over linear regulators Therefore a digital power management solution based on the TI UCD9246 PWM system con troller 112 and four TI PTD08A PowerTrain modules 113 has been selected which is well suited for high performance FPGA systems The configuration is shown in Fig 5 18 The PowerTrain modules are the actual conversion section of the digital power system and they are available with different current ratings to match the requirements of the specific applica tion The expected power consumption of the TIGER board was thoroughly evaluated based on the device datasheets and the Xilinx Power Estimator 114 spreadsheet Since the power consumption of an FPGA largely depends on the implemented firmware design the estimate was based on a rather busy design utilizing between 75 and 85 of the available logic DSP and BRAM resources running at a clock frequency of 500 MHz This should provide sufficient margin for even the most power consuming FPGA designs which may be implemented for fu ture applications of the TIGER module In conclusion two PowerTrain modules with current ratings of 20 A are selected for the 1 0 V and 1 5 V rails and two modules w
210. or of the board is described in chapter 6 5 1 1 1 Configurable Logic The unit cell of the Virtex 6 programmable logic is the so called slice which consists of four function generators FG eight storage elements some multiplexers and arithmetic carry logic A simplified schematic without the carry logic is shown in Fig 5 3 Each FG can be configured either as 6 input look up table LUT or as a 5 input LUT with two separate outputs Using F7AMUX F7BMUX and F8MUX it is possible to combine up to four LUTS ofa slice to implement functions of 7 or 8 inputs as well as large multiplexers up to 16 1 The storage elements serve as edge triggered D type flip flops FF to register either the LUT outputs O5 O6 or the bypass slice inputs AX DX Alternatively four of the storage elements can be configured as level sensitive latches LAT but in this case the remaining four FF cannot be used All storage elements in the slice are controlled by the common clock CLK clock enable CE and set reset SR signals The slice provides four asynchronous outputs A D four registered outputs AQ DQ and four multiplex outputs AMUX DMUX The spe cific function of a slice is defined by the content of the LUTs and the signal routing through the multiplexers which is selected during the device configuration When implementing arithmetic adders and subtractors each one bit addition is depending on the carry bit from the next less significant stage
211. out towers thus providing 9 individual channels per Shashlyk block The MAPD 3N has its highest photon detection efficiency in the green region which matches the emission of the WLS fibers Because the gain of MAPDs is temperature dependent a peltier element is used to stabilize the temperature of the photo diodes The complete module including calorimeter readout and cooling has a length of 42 cm Test beam measurements 51 confirmed that the MAPD readout is comparable to the traditional PMT readout in terms of linearity and energy resolution when the temperature was stabilized at 15 C 3 3 3 Muon Identification To identify the scattered muons there are several hadron absorbers dubbed Muon Filters MF placed in the spectrometer They are enclosed by detector layers to form the so called Muon Walls MW MW 1 at the end of the LAS consists of a 60 cm thick iron absorber MF 1 between two groups of drift tube detectors MW 2 at the end of the SAS is built from a 2 4 m thick con crete absorber MF 2 accompanied by drift tubes and MWPCs These detectors act as filters for The used material is PS 2 pTP 0 05 POPOP The primary scintillator para Terphenyl pTP emits light with a wavelength of 340 nm The secondary scintillator POPOP is used to shift the emission to a longer wavelength 410 nm 1 Saint Gobain BCF 91A Shifts blue to green Peak emission at 494 nm 28 3 The COMPASS II Experiment Table 3 2 COMPASS tracking
212. pScope Server The Xilinx Platform USB cable drivers were installed on the TIGER system by following the instructions in 178 This allows to use the Xilinx Lab Tools specifically the iMPACT tool for programming a CPLD or Platform Flash XL and the ChipScope server for FPGA firmware de bugging These tools can run directly on the TIGER CPU without the need of a dedicated PC next to the VXS crate AJTAG cable is simply connected to the USB port of the TIGER board and to the JTAG TAP of the device under test Besides the FPGA on the TIGER board itself this may also be the JTAG chain of one of the GANDALF boards in the crate After the ChipScope server has been started on the TIGER CPU the Analyzer GUI can be launched on any PC in the local network in order to perform a debugging session 6 defined in kernel source file drivers pci pci c 124 6 Firmware and Software 6 2 5 AMD Driver and OpenCL SDK The AMD Catalyst 12 10 proprietary Linux x86 display driver has been installed to support the Radeon E6760 for GPGPU applications It should be noted that the Catalyst Control Center op tion has to be enabled during the installation and the XServer must be started even though the system is operated headless otherwise the access to temperature monitoring and per formance settings of the GPU is not possible In addition DPMS has to be disabled in the etc X11 xorg conf in order to prevent the GPU from entering a power saving state due to th
213. pecific memory module being used is requested during the generation process This includes the module organization i e the row column and bank address width and the SDRAM timings This information is usually found in the datasheet of the memory module but may also be read out from the SPD EEPROM A 2GB SO DIMM DDR3 module from Hynix has been selected for the application on the TIGER board It is constructed from eight 256M x 8bit components and is a 64 bit wide single rank module Supported timings are DDR3 1066 533 MHz clock with CL7 and DDR3 1333 667 MHz clock with CL9 96 The CL value specifies the column address strobe CAS latency in clock cycles between the assertion of the CAS signal and the availability of the requested data A detailed description of the DDR3 device operation may be found in 97 For access to the Hynix DDR3 memory module a MIG core has been created according to the FPGA pin out as specified in appendix B Several design rules for the I O bank selection and pin allocation 95 p 135 had to be considered during the FPGA pin planning to satisfy the timing require ments ofthe memory interface Details about the implementation of the MIG core in the TIGER firmware are given in section 6 1 1 6 5 1 3 Clock Distribution Network The TIGER module requires a number of different clock signals for operation While some fre quencies are defined by the various interface standards others have to be synchronized to the
214. polation for t 0 This information is important for the determination of the total angular momentum of the quarks using Ji s sum rule 2 38 2 5 Deeply Virtual Compton Scattering 15 w BH dominates 12 E e DVCS dominates BH DVCS p Xg 0 01 1 BH DVCS x 0 04 Hs Xp 0 1 S Dvcs l I os IDVCS oe 7 30 CHE voto ono TD Loo c0 gt E EL TA ns E Sn i Ew BH DVCS Y eem s 01 honores IDVCSI za 10 E a L E Zo aes LE We Ln WW 205 F BHP E 7160GeV Q 2GeV t 0 1GeV Ey7160GeV Q 2GeV t 0 1Gev T EU 60GeV Q 2GeV t 0 1GeV 10 1 L L 1 1 L L L 1 at T US L L L N fi fi i 150 100 50 0 50 100 p 150 100 50 0 50 100 Eb 150 100 50 0 50 100 gt Figure 2 11 Expected cross section of the reaction up upy as a function of the angle p between the lepton scattering plane and the production plane BH blue DVCS red and total cross section dotted are plotted for different xg regions Simulation was performed for E 160 GeV Q 2 GeV and t 0 1 GeV COMPASS kinematic 30 p 7 The Bethe Heitler process BH which is shown in Fig 2 10 b and c and the DVCS process have the same final state Therefore their amplitudes interfere and the differential cross section for BH and DVCS can be written as 29 do IN 1Ny x taal xir Far Tay tT oyccl BH 2 40 dxdQ d t dp BH DVCS DVCS BH Dvcs BH I where i
215. precisely constrain the GPDs for t 0 This limit is not accessible in deep inelastic scattering but it can be investigated in hard exclusive processes like DVCS or HEMP 2 5 DeeplyVirtual Compton Scattering The GPDs H H E E of the u and d quarks can be experimentally investigated in hard ex clusive photon or meson production Observables like cross sections and asymmetries in de pendency of beam or target polarization or beam charge contain information about the GPDs Because the GPDs are typically convoluted with a hard scattering amplitude they cannot be extracted from the data directly Instead the functional dependency of the GPDs on x and t is parametrized by theoretical models and the parameters are determined by comparison with the data One of the processes that will help to constrain the GPDs is the deeply virtual Compton scat tering where a real photon is produced by scattering a lepton probe off a nucleon target which remains intact IN INY 2 39 Fig 2 10 a describes the DVCS process A quark with longitudinal momentum fraction x absorbs a virtual photon with virtuality Q and then emits a real photon which reduces the longitudinal momentum fraction to x The quark GPD describes the correlation between the initial and final state quarks for given values of t and Q The t dependence of the quark GPDs is directly accessible in DVCS but this requires a high experimental precision to allow an extra
216. r 11 15 2010 http wwwcompass cern ch compass gpd index html Bibliography 169 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 COMPASS COLLABORATION 2009 COMPASS Medium and Long Term Plans Letter of in tent CERN Geneva SPS and PS Experiments Committee CERN SPSC 2009 003 SPSC I 238 M VANDERHAEGHEN P A M GUICHON M GUIDAL 1998 Hard Electroproduction of Photons and Mesons on the Nucleon Phys Rev Lett 80 5064 K KUMERICKI D MULLER 2010 Deeply virtual Compton scattering at small xg and the access to the GPD H Nuclear Physics B 841 1 2 1 HERMES COLLABORATION A AIRAPETIAN ET AL 2012 Beam helicity and beam charge asymmetries associated with deeply virtual Compton scattering on the unpolarised pro ton Journal of High Energy Physics 2012 7 1 COMPASS COLLABORATION P ABBON ET AL 2007 The COMPASS experiment at CERN Nucl Instr and Meth A 577 3 455 T Kunz T SZAMEITAT COMPASS II Spectrometer Model 3D image rendered with TGEANT Private communications Sumitomo Cryogenics Group Inc http www shicryogenics com N DOSHITA LH2 target in the 2012 DVCS run Presentation at the COMPASS Collabora tion Meeting November 15th 2012 at CERN T SZAMEITAT A FERRERO Private communications CERN CRYOGENICS GROUP Private communications E BURTIN N D HOSE A FERRERO Private communications H BET
217. r the connection of the readout mod ules to the DAQ system It has been designed for integration into the GANDALF framework which is based on the VME64x VXS standard The TIGER module takes on the role of a VXS switch board having access to high bandwidth backplane links from each GANDALF transient analyzer module in the crate A high performance FPGA device and a large on board memory are provided for fast execution of complex algorithms and for extensive buffering capabilities An embedded CPU is available for slow control tasks like configuration and monitoring of the system and a GPU can support the FPGA with floating point calculations Apart from this the board features interfaces for the TCS and S LINK connection to the DAQ The author of this thesis has conducted all steps of the development process of the CAMERA trigger system from concept creation to commissioning This includes the schematic design the component selection and the PCB layout of the TIGER board A fundamental firmware design for the Virtex 6 FPGA has been developed which implements the relevant logic to op erate the board and access its interfaces Finally the hardware has been successfully put into operation and all features have been thoroughly tested and characterized The TIGER modules were first employed at the COMPASS II experiment during the DVCS test run in 2012 providing key functionalities for the trigger and data acquisition systems They were operated in two
218. rameters which guarantee in particular adequate signal integrity and correct timings as required by the various interface technologies For example most signal standards require a specific impedance of the transmission line For differential signals addi tional parameters like the phase tolerance between the two lines of a pair and the maximum uncoupled length are defined The TIGER board contains electrical CSets for single ended dif 78 5 The TIGER Module ferential and high speed differential signals see Tab 5 4 Additional sets are defined for the buses of the DDR3 SO DIMM interface in order to constrain the relative propagation delay between the signals Within each data byte bus the signals have to match with an accuracy of 5 ps The address and control lines are required to match with an accuracy of 20 ps Table 5 4 Electrical constraint sets ECS defined for the TIGER board Single ended ECS Differential ECS High speed diff ECS Impedance 50 5 Q 100 5 Q 100 5 Q Static phase tolerance 2 5mm 0 5 mm Dynamic phase tolerance 5 0 mm 1 5 mm max length 15 0 mm 5mm Max uncoupled length 20 0 mm 7 0 mm A physical CSet defines parameters of the copper traces like the line width and the gap be tween the two lines of a differential pair DP in order to achieve the target impedance of the transmission lines In addition a so called neck mode is specified allowing to route nets
219. rammable phase offsets from a given reference clock The principal of operation is sketched in Fig 5 6 the output frequencies are defined by selecting a multiplicator M an input divider D and output dividers O according to the formula M i fin 5 1 fouti fins O 5 1 Since the distribution of the clock signals is crucial for high speed FPGA designs there are a number of dedicated clock lines available Clocks which are required all over the design are transmitted via global clock lines providing the highest fan out Regional clocks can be used to drive logic which is located within a confined area A clock region is 40 CLB high and half the chip wide I O clocks are reserved for use with I O logic like DDR buffers and SerDes 5 1 1 3 Block RAM All data processing algorithms need storage elements to retain data between the clock cycles The registers and distributed RAM in the slices can be used to store small amounts of data whereas dedicated Block RAM BRAM elements are available for larger data sets The basic BRAM element Fig 5 7 of the Virtex 6 is a 36 kbit dual port memory with selectable data width which can be set to different values for port A and port B The data outputs can make use of an optional pipeline register which adds one clock cycle of latency but on the other hand allows for higher operation frequencies Read and write access is possible via both ports using independent clocks which turns the BRAM element i
220. ration methods available for the CAMERA detector The results of both approaches are compared in the fol lowing The correction offsets for t4 and tror are calculated from the two calibration constant data sets 7 2 Physics Data 137 Me Eg 60 50 40 30 20 10 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 B AB Figure 7 7 Energy loss for protons in Ring B vs their velocity Bor all scintillator elements Elastic pion proton events from run 108898 79 Entries 22267 Mean 6 714 RMS 6 828 2500 2000 1500 1000 10 15 20 25 30 Tracks Figure 7 8 Number of tracks that could be reconstructed in the CAMERA detector for events of a pion run triggered by the TIGER trigger 138 7 Commissioning For the laser method the offsets are calculated from the f constants COIT _ corr _ corr _ praw _ raw 0 _ 40 lait x i 7 Lx up Ex dn Ex up X idn Ex up X idn 7 2 _ y pofblaser diff X i corr corr corr corr oor up Bj an lAi up Fai an ToF i j 2 2 raw raw _ praw _ praw 0 0 _ 40 _ 40 Bjupt IBjian Ai up laian pjup iBjdn ai up Ai an Es 2 2 pofblaser ToF i j The calibration with physics events is more complex The procedure which is detailed in 79 yields a set of calibration constants They are enumerated in Tab 7 4 and their values are tabu lated in the CAMERA Helper calibration file 79 p 110 The C
221. re is a SATA 2 0 port available on the TIGER module It may be used during OS or software installation The PCH s SDIO interface is connected to a SD card slot which is located on the TIGER board beneath the COM Express module SD and SDHC memory cards with normal speed and high speed bus interfaces as defined in the SD specifications 124 are sup ported according to the Intel PCH datasheet 125 p 369 The LVDS display port of the Intel Atom s integrated graphics unit 126 p 86 is broken out to a Hirose FH12 40S 0 5SV 55 40 pin FFC connector 127 This type of connec tor is normally used to attach flat panel displays to the base board but with an LVDS to DVI adapter available from Kontron 128 an external monitor with DVI D port can be connected This is mainly used during the OS installation phase and to access the BIOS menu while under normal conditions the board is operated headless i e without monitor and input devices 5 2 3 2 On board Interfaces Several on board devices are connected to the appropriate interfaces of the COM Express mod ule Besides the PCI Express connection to the FPGA which will be discussed in section 5 2 6 there are 12C bus SMBus and LPC bus connections used on the TIGER board as described in the following 12C Bus and SMBus The I C bus is a two wire serial bus for low speed communication consisting of two bidirec tional open drain signals called SDA serial data and SCL serial clock w
222. ready when the OS is enumerating the buses and devices cf 5 2 6 Hence the CPU is held in the reset state until the configuration is finished In this way it is guaranteed that the FPGA will be discovered by the OS as a PCIe device 5 1 5 2 Online Configuration After the OS has completed its boot process the FPGA can be reconfigured at any time from the CPU The necessary software tools are described in section 6 3 For a full reconfiguration the new bitstream file is transferred via the LPC bus to the CPLD Fig 5 14 which forwards the 16 bit data words to the SelectMap interface of the FPGA using non continuous data loading with a controlled CCLK 108 p 42 In this mode the clock is halted and it toggles only once when the next word is ready to be loaded into the FPGA Due to the limited bandwidth of the LPC bus this method is rather slow compared to the configuration from the Flash An alternative approach for the online configuration of the FPGA is the partial reconfiguration PR via PCIe Fig 5 15 For this purpose the Virtex 6 contains an internal configuration ac cess port ICAP 108 p 74 The ICAP provides an interface similar to the SelectMap interface which is accessible from the FPGA fabric It can be used to load partial bitstream files to the 70 5 The TIGER Module PCle n gt Figure 5 15 On partial reconfiguration via PCle Figure 5 16 The Flash can be updated either via the static part of the
223. rget is crossing the two detector 38 4 CAMERA Proton Trigger elements A and B generating scintillation light while traversing the material This event of light creation in a detector element is in the following called a hit The light pulses propagate to the ends up and dn of the counters where they arrive at time 4 jp lA dn tB up and tp dn respectively The time 4 and position za of the hit in counter A is then calculated as follows tA upt tad ta eee 4 1 2 lau LA dn ZA ca 4 2 where Ca err is the effective speed of light in the A scintillator For the hit in counter B time and position tg and zg are calculated analogously The time calibration of the CAMERA detector cf 4 2 2 2 is chosen such that the positions za and zg are given relative to the center of the corresponding counter zo 4 and Zo z respectively By combining the hit information from the inner and the outer counter the particle track can be reconstructed The time of flight trop and the distance of flight s is used to calculate the velocity Prof and the polar angle 0 as follows tror fp ta 4 3 Az zg zATk 4 4 s Vd2 Az 2 4 5 S E 4 6 PToF Duni 4 6 d 0 arctan 4 7 Az where k zo p Zo A is the offset in z direction of the center of counter B with respect to the center of counter A and d is the radial distance between the inner and the outer counter 4 1 2 Energy Loss Measurement After the track reconstruct
224. rigin and frequency of the clocks as well as the destination VHDL block in the FPGA design are specified The signals are either connected to global clock capable user I O pins GC or to the MGTREFCLK inputs of the GTX transceiver tiles Origin Clocksignal Frequency Destination Clock Pins Gimli tcs clk V6 155 52 MHz TCS interface Si5326A SL variable Base design GC clk_si2 505 44 MHZ clockm gmt clk ref 200 0 MHz Si5338 clk ddr3 266 6 MHz Memory interface aurora mgtclk 125 0 MHz Aurora core sfp mgtclk 125 0MHz User app MGTREFCLK COMe pcie sys clk 100 0MHz PCIe core start up A prompt and a delayed version of the reset signal may be generated for every clock domain The reset signals carry names of the form clockdomain startup rst delayed i e the base name startup rst is prefixed by the name of a clock domain and may be suf fixed by the string delayed The prompt reset signal is deasserted synchronously as soon as the correspondent clock signal is present The delayed reset signal is deasserted synchronously a defined number of clock cycles after the CPLD interface enters the flash mode FM NORMAL The software controlled user reset is also synchronized to the various clock domains It should be noted that the delayed reset signals will not be deasserted in the case of a binfile mismatch i e if the loaded FPGA design file does not match the TIGER function Trigger or Rea
225. rms of signal integrity are the high speed serial links which are connected to the Virtex 6 GTX transceivers They have been simu lated with Allegro SigXplorer With maximum data rates of more than 6 Gbit s and also due to the backplane connection the Aurora lanes cf 5 3 2 are the most interesting signals for such a simulation The procedure will briefly be described in the following In a first step the topology of the probed net is extracted This creates a model of the transmis sion line based on the actual PCB routing including vias and AC coupling capacitors There upon accurate signal models for all involved components are added to the topology They are usually obtained from the respective manufacturer For the Virtex 6 FPGA there are models for the GTX transceiver and for the package available from Xilinx 119 A model for the Tyco back plane connector could unfortunately not be obtained instead a generic model for a connector with similar characteristics was used The models are provided in various formats of which some have to be converted before they can be used in SigXplorer The model for the Virtex 6 package which describes the parasitic effects of the bonding wires and the solder bumps ex ists in the SPICE format The behavior of the GTX transmitters and receivers is described by IBIS AMI models The backplane connector model is provided in form of a scattering matrix S parameter file Since an Aurora lane extends
226. ronics was commissioned during a DVCS test run in autumn 2012 In formation on the installation in the experiment and the calibration procedure as well as a first glimpse of the ongoing data analysis is provided in Chapter 7 before the results of this thesis are summarized in Chapter 8 COMPASS Apparatus for Measurement of Exclusive ReActions 3 Trigger Implementation for GANDALF Electronic Readout 2 Theoretical Motivation 2 TheNucleon Spin Spin is a basic property of any quantum mechanical system It is expressed in multiples of the reduced Planck constant Protons and neutrons together called nucleons are spin 1 2 parti cles In the parton model developed by Bjorken 1 and Feynman 2 the nucleons are composed of point like particles called the partons They were later identified with the quarks which had been introduced by Gell Mann 8 and the gluons While quarks are spin 1 2 particles the gluons are the gauge bosons for the strong force and therefore carry spin 1 The spin of the nucleon can be written as 9 p 536 1 1 E Qu AE 2 1 stu where AZ is the spin contribution from the quarks and anti quarks AG is the spin contribution from the gluons Ly and Lg are the orbital angular momenta of the quarks and gluons respec tively While AX 0 3 and AG 0 2 0 3 have been determined by former experiments 10 11 the orbital angular momenta are not known today The generalized parton distributions which c
227. rs VSAT These are silicon strip de tectors scintillating fiber detectors Pixel GEMs and Pixel Micromegas They cover a range of up to 3 5 cm from the beam axis and provide a high rate stability The scintillating fibers provide a good time resolution of 400 ps which allows for a correlation between hits and beam tracks without ambiguity even for the high rates of up to 3 109 s per fiber in the beam spot Inthe intermediate region between 2 5 cm and 40 cm from the beam axis GEM and Micromegas detectors with strip readout represent the Small area trackers SAT Finally the outer regions of the spectrometer are covered by the Large area trackers LAT which are multi wire proportional chambers MWPC drift chambers and straw detectors of various sizes and shapes 3 3 2 Calorimeters In each COMPASS spectrometer stage there is one electromagnetic and one hadronic calorime ter cf table 3 3 to measure the energy of electrons photons and hadrons in the final state Originally the electromagnetic calorimeters ECAL1 and ECAL2 were composed of homoge neous lead glass modules where photons lose their energy in electromagnetic showers by means of bremsstrahlung and pair production The Cherenkov light of the shower electrons is detected with PMTs Since the depth of the calorimeter blocks is approx 16 radiation lengths more than 99 of the particle energy is contained in the shower 35 p 47 In 2008 the central part of ECAL2 was repla
228. s to visualize the acquired data and to control the virtual I Os Automatic data export can be configured to allow an offline analysis of the data 6 1 1 6 Optional Cores Two optional VHDL modules drawn as dashed boxes in the top level diagram Fig 6 1 are available in the TIGER Base design which may be enabled by the application designer in order to access the correspondent interfaces These are the memory interface to the DDR3 SO DIMM and the Aurora interface for a communication between the two TIGER modules in the VXS crate When an optional core is instantiated in the Base design it is necessary to add also the respective UCF file to the project in order to include additional location and timing constraints Memory Interface The DDR3 memory interface core 5 1 2 has been generated with the Xilinx MIG tool 95 by specifying the timing parameters of the Hynix SO DIMM 96 and the pin out of the TIGER board FPGA appendix B The core provides the controller and PHY modules to access the DDR3 SDRAM It is also responsible for the initialization of the memory device after power up and the calibration of the read and write data paths to account for delays The user interface of the MIG core provides the connections for the application module It presents a flat ad dress space to the user and maps it to the native bank column row addressing scheme of the SDRAM A description of the relevant user interface signals is found in 95 p 70 The M
229. s and pin out types to provide a wide range of products in terms of performance interface capabilities and power consumption For the TIGER project the smallest module size Mini 55 x 84mm has been selected with pin out type 10 which is defined since revision 2 0 of the specification 5 2 2 Kontron COMe mTT10 The Kontron COMe mTT10 123 is a COM Express Mini module with pin out type 10 and it is hence suited for the application on the TIGER board where it is mainly used for slow control tasks like configuration and monitoring Therefore it does not have to provide lots of com puting power but rather a convenient access to the various devices that have to be controlled remotely during run time From the several variants of the COMe mTT10 which differ in CPU speed memory size and temperature range the model COMe mTTi10 E680T 1GB 4GB has been chosen for the appli cation on the TIGER module It features an Intel Atom E680T single core CPU with a maximum clock frequency of 1 6 GHz and an integrated graphics core complemented by an EG20T plat form controller hub PCH The Atom s internal memory controller has access to 1 GB of DDR2 800 RAM which is shared between the CPU and GPU core but since the integrated graphics is not used during normal operation of the TIGER module almost the whole memory space is available to the CPU Besides a number of standard PC interfaces which will be described in the next subsection the PCH provides
230. s are located at the left edge of the Virtex 6 device as shown in Fig 6 6 The mapping between the CAMERA channels and the FPGA I O banks has been defined considering an optimized routing of the coincidence logic Since coincidences can occur between the combinations A B and A B 1 the correspond ing slat logic instances should be located close to the track logic instances to which they contribute As it can be seen this is fulfilled for all combinations but for one A B z2 which is connecting the top and bottom end of the device Optimal placement of the logic blocks is ensured by carefully floorplanning the design During this task the design hierarchy is broken apart into so called Pblocks which can be constrained to a specific device area 77 p 62 Due to the allocation of the CAMERA channels to specific I O banks in the trigger design on the one hand and the fixed mapping between VXS ports and backplane slots on the other hand the position of the GANDALF modules in the crate is implied The resultant configuration of the CAMERA readout crate is given in Tab 6 3 To monitor and control the TIGER trigger processor the design adopts some of the interfaces provided by the Base design e The PCIe board to host FIFO may be used to spy on the pmt info slat info and track _info streams of selectable channels for test and debug purposes Scaler values are also transmitted via this interface to monitor the hit and trigger rat
231. s can start the board cross section has to be defined The TIGER PCB consists of 14 copper layers as shown in Fig A 4 There are eight signal layers available for routing of the interconnections two power layers for 26 metallized drill holes in a PCB to connect between different layers 5 1 Mainboard 79 SOLI piii ii in E RUNS NG iii RAD Au e i AD elele e e elele ollajo o ele JS DES i Figure 5 23 The fan out of the FPGA signals is Figure 5 24 The propagation delays of the conducted in neck mode This enables to route DDR3 memory signals are matched in groups both traces ofa differential pair between two via by inserting additional trace length snakes or IOWS trombones the supply voltages and four ground planes The cross section is constructed such that it sup ports good signal and power integrity First of all for each signal layer there is a corresponding reference plane which allows to define the impedance of the transmission lines and which also carries the return currents This is important for the high frequency behavior of the signal traces in order to keep the electronic noise at a low level 117 The distance between the signal and reference layers influences also the physical constraints for single ended and differential transmission lines Proper values for the trace width DP gap and layer spacing are calculated by the Xsection tool in the PCB Editor Furthermore the spa
232. s introduced by M ller 19 Rad yushkin 20 and Ji 21 The factorization in a hard and a soft sub process of the DIS can in more general terms also be applied to other processes where momentum is transferred to the target The deeply virtual Compton scattering DVCS y N yN which is shown in Fig 2 6 and explained in detail in the next section or the hard exclusive meson production HEMP are examples for such processes An extensive overview of the theory for GPDs is given in 22 In the Bjorken limit 2 12 and for a momentum transfer to the nucleon t which is much smaller than Q the DVCS amplitude can be factorized into a convolution of the Compton amplitude for the lepton scattering and a quark correlation function which describes the soft part of the process 23 24 It is shown that the soft part of the DVCS amplitude is the same as the one of the elastic form factors In fact this is true for the soft part of many hard exclusive reactions Hence the GPDs are introduced as a universal description to parametrize this soft part The four spin 1 2 GPDs which will be used in the following sections are listed in table 2 1 There are nucleon helicity conserving H 8 T 8 and nucleon helicity inverting E 8 E 8 GPDs both polarized and unpolarized The polarized GPDs are quark helicity dependent which is not the case for the unpolarized ones The index f stands for the quark flavor u d s and the index g denotes the gluon
233. s of the GTX transceiver tiles which implement the corresponding interfaces In case of future changes to these interfaces for example the adoption of a different transmission protocol for the SFP link a new frequency plan can be calculated using the ClockBuilder software 103 The resultant register file is then loaded to the device via the 12C programming tool as described in section 6 3 Finally a separate 40 MHz quartz oscillator is used for an independent clocking of the CPLD 17Voltage Controlled Oscillator 5 1 Mainboard 67 Table 5 2 Clock signals produced with the Si5338 clock generator chip and connected to the TIGER FPGA Output Purpose Frequency Connected to 0 Aurora transceiver clock 125 MHz GTX ref clock input 1 User reference clock 200 MHz FPGA global clock input 2 DDR3 clock 266 MHz FPGA global clock input 3 SFP transceiver clock 125 MHz GTX ref clock input 5 1 4 CoolRunner II CPLD The Xilinx CoolRunner II is a programmable logic device which consists of up to 32 function blocks FB connected via an advanced interconnect matrix AIM 104 The logic in the FB is implemented within a sea of gates using the disjunctive normal form sum of products Each FB contains a programmable logic array PLA which forms 56 product terms AND rela tions of up to 40 inputs followed by 16 macrocells each capable to provide a conjunction OR gate of any combination of the product terms
234. s on the retaining structure mounting the photo multiplier tubes mounting the high voltage power supply and the VXS crate on the CAMERA platform installing the electronic readout modules in the VXS crate installing all HV and signal cables as well as fibers for the laser pulser checking the counters for light tightness performing readout tests with cosmic particles The barrel of the CAMERA detector is seated on roller bearings allowing to rotate the detector on its platform by 90 for measurements with cosmic particles This possible rotation had to be respected also for the fixation of the cables Therefore the cables were laid out with some additional length forming a loop below the detector To avoid disorder the cables were bun dled in groups of 12 each and the bunches were loosely attached to a cross rail of the platform While the HV cables are directly plugged into the power supply modules the signal cables ter minate at a patch panel which is mounted above the VXS crate This allows to use short patch cables with reduced diameter for the connections between the panel and the analog inputs of the GANDALF modules while low loss cables with larger diameter are used for the connection between the PMTs and the panel The two different cable types are listed in Tab 7 1 Addi tionally the patch panel provides more flexibility in case of a potential module replacement A photo of the patch panel and the VXS crate is found in
235. s the azimuthal angle between the lepton scattering plane and the production plane which is defined by the proton and the real photon In the DVCS process the real photon originates from the nucleon and carries information about the GPDs while the real photon in the BH process resulting from bremsstrahlung does not carry information about the nucleon and therefore can not be used for the determination of the GPDs There are two possibilities to separate the contributions from BH and DVCS On the one hand the DVCS cross section can be measured in a kinematic region where DVCS dom inates over BH Fig 2 11 right In this case the cross section is basically given by the square of the DVCS amplitude which at leading order has the form 22 p 49 IH tovcs BUD ic 2 41 1x 4ie On the other hand one can measure the beam charge difference to separate BH and DVCS When inverting the charge of the lepton beam the DVCS amplitude rpycs changes its sign which is not the case for the BH amplitude Tgp Because 7 gy is real the difference of cross sections is a 1 a 1 TBH R t pvcs 2 42 do IN 1NyY Here and in the following o is the short form for Td dide 16 2 Theoretical Motivation The BH amplitude is well known because it only depends on the elastic nucleon form factors The difference of cross sections is proportional to the real part of the DVCS amplitude which for an unpolarized target is 31 p 29 PIE
236. se design components instantiated at the top level UCF constraint files defining the pin locations IO standards and timing constraints test bench VHDL files for behavioral simulation of the design or parts of it the TG defines package file which declares constants and types not to be changed by the user and the TG parameters package file which declares constants the user may change to customize the design To give an example for a user definable constant in the TG parameters package the constant TIGER FUNCTIONALITY may either take the value TIGER FUNC TRIGGER or the value TIGER FUNC READOUT to specify the intended design type Different code variants will be instanti ated by means of VHDL generate statements at several places in the Base design depending on this constant s value 6 1 1 1 CPLD Interface The CPLD interface module is used to transmit status flags as well as some reset and control signals between the CPLD and the FPGA on the TIGER board The SelectMap A lt gt and D lt gt buses cf Fig 5 12 which are routed using a fly by topology from the CPLD via the Platform Flash XL to the FPGA are reused for this purpose Before the FPGA is initialized the SelectMap signals act as configuration pins used to transfer the bitstream into the FPGA s configuration SRAM As soon as the device configuration is finished these I Os are accessible in the VHDL top level module where they are called CA lt gt and CD lt gt
237. sheet editor U8 MGTREFGLKON 1171 _G9 MGTREFCLKOP_117 _G10 C256 NMGTREFCLKIN 117 E9 AURORA_MGTCLK _DPSERIESCAP AURORA_MGTCLK _DPSERIESCAP AURORA_MGTELK m MGTREFCLK1P_117 E10 AURORA_MGTCLK _DPSERIESCAP MGTRXNO 117 ae 0 AURORA_RX lt 3 0 188nF MGTRXN1 117 MGTRXN2 117 2 C255 MGTRXN3 117 3 AURORA_MGTCLK _DPSERIESCAP AURORA_MGTCLK py METRXPO_117 H7 AURORA RX lt 3 gt 117 vare z 1 188nF MGTRXP2 117 2 METRXP3_117 ES MGTTXNO 117 0 AURORA_TX lt 3 0 MGTTXN1_117 MGTTXN2_117 MGITXN3 117 MGTTXPO 117 AURORA_TX lt 3 gt MGTIXP1_117 MGTTXP2 117 merrxP3 117 _F3 Figure 5 22 Auto generated schematic drawing for the FPGA bank 117 5 1 7 2 Constraints Apart from the pure schematic an extensive set of constraints has to be defined when devel oping a high speed digital device Every component and every interconnect technology has certain requirements to the physical layout of the board and a multitude of design rules has to be observed in order to achieve the best possible performance The Allegro Constraint Man ager is utilized to define so called constraint sets CSets A CSet collects parameters which are later used to control the routing of the board Since different signal classes may have different routing requirements references to any electrical physical and spacing CSet can be assigned to each net in the design An electrical CSet defines pa
238. simulation one obtains the energy loss of the protons in the Ring A and Ring B scin tillators as well as the reconstructed velocity rof The resulting distributions of energy loss vs Tor are shown in Fig 4 6 The left hand plot shows the energy loss in Ring A which basically follows the fall of the Bethe Bloch formula One can clearly see the lower limit of Prop 0 15 since no time of flight calculation is possible for protons which do not reach the B counter The right hand plot shows the energy loss in Ring B The rising branch below prof 0 4 con tains the protons that are fully stopped in the B scintillator releasing all their remaining kinetic energy while protons above this limit can pass through the B counter which again results in the characteristic 6 3 fall according to Bethe Bloch The broadening of the proton band to higher energy losses is caused by two effects Depending on the polar angle 0 of the proton track the distance through the scintillator varies by a factor of up to 2 The energy loss probability distribution is best described by a highly skewed Landau dis tribution with a most probable value for the energy loss well below the mean dE dx that is given by the Bethe Bloch equation Due to the very rare high energy transfer col lisions the energy loss can fluctuate significantly 17 p 327 4 2 Trigger Concept 45 N Entries 3589800 Mean x 37 23 10 Mean y 3 898 RMS x 17 89 RMS y 2 278
239. ss PCIe links A CPLD is responsible for the power up and power down se quences of the board the FPGA configuration process and the handshaking process between the TIGER and the GANDALF modules in the VXS crate It also implements the necessary glue logic to connect different parts of the board All hardware components and interfaces will be described in depth in the following sections 1 Graphics Processing Unit 2 Mobile PCI Express Module 3 Complex Programmable Logic Device 54 5 The TIGER Module COMPASS reference clock TIGER to TIGER communication Aurora detector hit information from GANDALF modules TIGER module VXS interface Trigger out generic I O S LINK Gbit Ethernet configuration amp monitoring Figure 5 1 Block diagram of the TIGER module showing the most relevant components and interfaces All items are detailed in the text 5 1 Mainboard The form factor for a 6U VXS switch board is defined in the VITA 41 0 standard 80 The TIGER mainboard complies mechanically and electrically with this specification The dimensions of the printed circuit board PCB are 233 x 160mm It is equipped with a IEEE 1101 10 compliant 87 EMC front panel electrostatic discharge strips and injector ejector handles The handles facilitate a safe plug unplug procedure by providing the necessary insertion force which can be more than 500 N due to the large amount of
240. straints for the combination of multiple signal standards in one bank The FPGA on the TIGER module uses the following signal standards on the User I O pins IVDS 2 5 V for clock signals for all signals connected to the VXS backplane and for the differential I O connector on the front panel see 5 3 IVCMOS 2 5 V for the interface to the CPLD see 6 1 1 1 and for some general purpose control signals 9Low Voltage Differential Signaling 10Low Voltage Complementary Metal Oxide Semiconductor 5 1 Mainboard 61 36 kbit Block RAM OPMODE wa RSTREGA RSTRAMA PCIN OPMODE ALUMODE Shifters P REGCEB Figure 5 7 Interface signals of the Figure 5 8 Simplified diagram of the DSP48El slice Virtex 6 dual port Block RAM ele adapted from 94 p 18 The red boxes denote optional ment 93 p 14 pipeline registers e SSTL Differential SSTL 1 5 V for the DDR3 memory interface see 5 1 2 Differential LVDS inputs are internally terminated with 1000 resistors This saves a large num ber of discrete components which would have otherwise to be placed very close to the FPGA package Similarly the bidirectional SSTL pins make use of the internal split thevenin termina tion provided by the Virtex 6 DCI feature This allows to properly terminate the signal trace when the I O buffer is tri stated i e used as input while in output mode the termination re sis
241. t 6 gt VXSPORT_14 lt 7 gt P4 El Fl P4 G2 H2 P4 E3 F3 P4 G4 H4 P3 A13 B13 P3 C14 D14 P3 A15 B15 P3 C16 D16 P3 E13 F13 P3 G14 H14 P3 E15 F15 P3 G16 H16 P5 A13 B13 P5 C14 D14 P5 Al5 B15 P5 C16 D16 P5 E13 F13 PS G14 H14 P5 E15 F15 P5 G16 H16 P2 Al B1 P2 C2 D2 P2 A3 B3 P2 C4 D4 P2 El F1 P2 G2 H2 P2 E3 F3 P2 G4 H4 P5 A9 B9 P5 C10 D10 P5 All Bll P5 C12 D12 P5 E9 F9 P5 G10 H10 P5 Ell Fl PS G12 H12 P2 A5 B5 P2 C6 D6 P2 A7 B7 P2 C8 D8 P2 E5 F5 P2 G6 H6 P2 E7 F7 P2 G8 H8 160 C Connector Pin Out FPGA Pins Link Name Backplane Pins t Conn t _ R39 P38 VXSPORT 15 0 P5 A5 B5 M41 M42 VXSPORT 15 1 P5 C6 D6 R35 R34 VXSPORT 15 2 P5 AT B7 T34 T35 VXSPORT_15 lt 3 gt P5 C8 D8 P40 P41 VXSPORT_15 lt 4 gt P5 E5 F5 R37 T37 VXSPORT 15 5 P5 G6 H6 P42 R42 VXSPORT_15 lt 6 gt P5 E7 F7 T39 R38 VXSPORT_15 lt 7 gt P5 G8 H8 BA30 AY30 VXSPORT 16 0 P2 A9 B9 AW31 AV31 VXSPORT 16 1 P2 CIO DIO BA31 BB31 VXSPORT 16 2 P2 All Bill AT29 AR29 VXSPORT_16 lt 3 gt P2 C12 D12 AY32 AW32 VXSPORT 16 4 P2 E9 F9 AT30 AR30 VXSPORT 16 5 P2 G10 H10 AW30 AV30 VXSPORT_16 lt 6 gt P2 Ell Fl AY29 BA29 VXSPORT_16 lt 7 gt P2 G12 H12 N40 N41 VXSPORT_17 lt 0 gt P5 Al B1 M36 M37 VXSPORT_17 lt 1 gt P5 C2 D2 N36 P37 VXSPORT_17 lt 2 gt P5 A3 B3 N38 N39 VXSPORT 17 3 P5 C4 D4 L39 L40 VXSPORT_17 lt 4 gt P5 El Fl L41 L42 VXSPORT_17 lt 5 gt P5 G2 H2 N35 N34 VXSPORT_17 lt 6 gt P5 E3 F3
242. tGeSiCA modules 35 p 54 Recently with the GANDALF module see section 3 5 1 below a fourth type of readout module has been introduced to the COMPASS DAQ system The readout modules collect the data from the front ends and merge it according to the event numbers while adding some header information for the experiment wide unique channel number identification The final data stream is transmitted via S Link 60 to readout buffer PCs ROB where the data of one spill is buffered in RAM before being distributed via Gigabit Ethernet to event builder PCs EVB in a round robin manner The EVBs receive the sub events from the ROBs for a specific event number and concatenate them to form the final event which is then stored in the DATE format 61 The resulting files are transferred to the central data recording service CASTOR where they are stored on magnetic tapes MAnalog to Digital Converter 15Time to Digital Converter 16COMPASS Accumulate Transfer and Control Hardware 17 GEM and Silicon Control and Acquisition 18 Generic Advanced Numerical Device for Analytic and Logic Functions 19CERN Advanced STORage manager 34 3 The COMPASS II Experiment When the current DAQ system was designed back in 2001 it could easily handle the data rate coming from the detectors Since then it has been upgraded several times to keep up with an increased number of detector channels and higher beam intensities and trigger rates Nev ertheless a
243. taking 2 clock periods Contrary to the ADDR field in the DATA field the least significant nibble is driven out first During a read cycle the DATA field is driven by the peripheral and during a write cycle it is driven by the host 36half a byte i e 4 bit 5 2 CPU and GPU Extension Boards 87 Table 5 9 Length of the LAD 3 0 fields for different cycle types I O Read I O Write Memory Read Memory Write and the minimum cycle duration for each type clock periods Field Description VOR I OW MemR Mem W START start ofthe transaction 1 1 1 1 CT D cycle type direction 1 1 1 1 ADDR I O or memory address 4 4 8 8 DATA write data transfer 2 2 TAR turn around bus to peripheral 2 2 2 2 SYNC indicates synchronization 21 21 21 zl DATA read data transfer 2 2 TAR turn around bus to host 2 2 2 2 cycle duration gt 13 gt 13 gt 17 gt 17 5 2 4 MXM GPU Module Specification The Mobile PCI Express Module MXM standard has been designed to ease the employment of PCIe based graphics modules in small form factor computer systems including notebooks blade and rack mount servers mobile workstations as well as all in one and home theater PCs The current version 3 0 of the MXM specification has been released in 2008 and consists of two parts the hardware specification 134 covering the electrical mechanical and thermal aspects and the software specification 135 Up to 16 PCIe
244. tal setup which will be described in this chapter consists of the beam line the target region and the forward spectrometer Fig 3 1 In the beam line the beam particles are guided to the experiment hall and their tracks and momenta are measured The target with its surrounding detectors is placed at the interaction point The outgoing particles are detected in the two stage spectrometer The angular acceptance of the first spectrometer stage LAS is up to 180 mrad Particles that pass through the central hole in the LAS enter the second spec trometer stage SAS which has an acceptance of 30 mrad Each spectrometer stage includes a dipole magnet SM1 and SM2 respectively to deflect charged particles in the horizontal plane as well as detectors for particle identification track reconstruction and energy determination 35 p 7 Fig 3 1 shows the spectrometer setup for the 2012 DVCS run Please note that the laboratory coordinate system is defined such that the z axis points in direction of the beam The most important components of the experimental setup are introduced in the following sections In particular the upgrades that were performed for the DVCS measurements are de scribed in more detail 3 1 The Beam The beam for the COMPASS experiment is provided by the M2 beam line which is part of the SPS accelerator complex Several configurations for muon or hadron beams are available al 1 Super Proton Synchrotron Large Angle Spec
245. tector the PMTs of Ring B had to be mounted orthogonal to the scintillator elements Therefore special light guides with a 90 turn have been manufactured Fig 3 7 3 2 The Target Region 25 Figure 3 7 The light guides for Ring B have been manufactured at the workshop of the Freiburg institute of physics Left The objectis milled from a solid block of acrylic glass using a 5 axis CNC machine Right A finished light guide 44 Table 3 1 Properties of the counter elements of the CAMERA detector Each Ring A inner B outer consists of 24 counters 41 47 Element Property RingA Ring B length 275 0cm 360 0cm thickness 0 4cm 5 0cm Scintillator width 6 5 6 6 cm 29 0 30 3 cm dist from beam axis 25 9cm 114 7 cm material BC 408 BC 408 type Hamamatsu R10533 ET Enterprises 9823B PMT window Y 51mm 130mm photo cathode active Y 46mm 110mm combined time resolution 380 ps 175ps For Ring A the 2 inch PMT model Hamamatsu R10533 45 has been chosen Ring Bis read by 5 inch ET Enterprises 9823B 46 PMTs In total the CAMERA detector comprises 96 channels which are digitized with fast transient analyzer modules Details of the readout will be de scribed in section 3 5 1 Apart from the readout the CAMERA detector information shall also be used for a recoil particle trigger that identifies protons in real time For this purpose new digital trigger electronics based on powerful programmable l
246. that a shutdown is in progress CPLD RESET USER1 _IO toggles the user defined reset signal CPLD RESET TMCRATE _IO toggles the RATE pin of the Gimli card this usually solves a locking problem of the CLC016 Table 6 5 TIGER CPLD status register The meaning of the status signals is explained in Tab 6 1 31 14 13 0 GANDALF Status GS lt 17 0 gt TIGER Status TS lt 13 0 gt status of all the GANDALF boards various status information about the TIGER board in the VXS crate GS i l 0 board at VXS port i 1 is ready TS lt 13 gt OtherTigerReady TS lt 12 gt c2 startup rst TS 11 cl startup rst TS lt 10 gt si los TS lt 9 gt si lol TS lt 8 gt tcs lol TS lt 7 gt ucd_pgood DC DC power good TS lt 6 gt ldo pgood low drop out reg pwr good TS 5 INIT B pin of FPGA TS lt 4 gt DONE pin of FPGA TS lt 3 gt binfile mismatch TS lt 2 gt TigerFunction TS lt 1 0 gt flash mode 6 2 Operating System and Device Drivers 123 devices which may of course also be used for PCI Express devices since the two standards are software compatible The driver maps the PCI BARs to the user space and provides access to the configuration registers of the device The MPRACE library provides C classes for convenient I O transactions with the PCIe board A new class for the TIGER board has been derived from the basic mprace Board class The TIGER class implements public
247. thods to obtain the correction offsets are compared in 79 One possibility is to choose the offset such that the position of the cusp in the proton band i e the velocity of protons which have just enough energy to cross the B scintilla tors matches the value from the simulation A position offset in z direction is determined for each Ring A scintillator element A scaling factor for the energy calibration is determined for every counter allowing to convert the raw amplitude information of the PMT pulses given in ADC digits into an energy loss of the particle in the scintillating material After applying these corrections to the raw data an energy loss distribution for the elastic pion proton sample is obtained as shown in Fig 7 7 It contains the data from all Ring B scintillator elements showing the energy loss of the recoil particles in the CAMERA detector versus their velocity The proton band is nicely visible The pion runs were triggered by the condition TIGER BeamTrigger Veto In order to determine the purity of this trigger signal a histogram of the number of tracks that could be reconstructed in the CAMERA detector for each event is shown in Fig 7 8 Itturns out that only in 396 of the events the TIGER trigger has fired spuriously although no track was found in the CAMERA detector 7 2 2 Comparison of two CAMERA Calibration Methods With the laser data and the elastic pion events there are two independent calib
248. time sub dividing the 1 ns time unit in 64 bins The VXS backplane link which has been developed for the GANDALF framework in 86 is op erating at 500 MHz DDR thus providing a throughput of 1 Gbit s for each channel It is imple mented using a 10 1 SerDes logic which is self calibrating after establishing the connection to compensate for different trace lengths on the backplane and a possible offset between the transmitter and the receiver clock Once the link is synchronized it is long term stable and the bit error rate has been determined to be below 6 7 10 with a confidence level of 99 86 p 93 The format of the trigger primitive data packet is shown in Tab 4 1 For every detected 5Double Data Rate SSerializer Deserializer 50 4 CAMERA Proton Trigger Table 4 1 Format of the trigger primitive data packet for transmission via the VXS backplane Po and Pj are parity bits to allow error detection Bit 9 8 7 6 5 4 3 2 1 0 Word 1 1 amplitude 9 Word 2 coarse time 10 Word 3 high res time 6 0 0 P Po hit three 10 bit data words are transmitted which takes 30 ns Therefore the mean hit rate on a detector channel may be up to 33 MHz The amount of data which is transmitted for every trigger primitive has an immediate effect on both the acceptable mean hit rate and the latency increase in case of pile up pulses When a hit is recognize
249. time and the z position of the hits for every channel are shown in the middle and lower plots In the left column the uncalibrated values are plotted and in the right column the values were corrected by the TO calibration constants which have been obtained from the laser data 7 1 2 TIGER Latency Adjustment In order to include the CAMERA trigger in the COMPASS trigger system the latency of the TIGER trigger processor has to be set correctly This is due to the fact that the readout elec tronics is continuously buffering the last few microseconds of detector data and upon arrival of a first level trigger only a small time window from the past is sent to the DAQ For every readout module a latency setting defines how far in the past the readout window has to start in order to capture the hits corresponding to the triggered event The actual value for each module depends on the length of the TCS fiber which transmits the FLT and on the detector s position in the experiment the further upstream the earlier the particles pass The COMPASS trigger system combines a multitude of trigger signals including several scattered muon trig gers cf 3 4 calorimeter triggers beam triggers the new proton trigger and a random trigger 7 1 CAMERA Installation 133 TriggerLatency Latency Setting 500 ns TriggerLatency TriggerLatency Latency Setting 1500 ns TriggerLatency Entries 6366380 3 Entries 6254549 o Mean
250. tion is not required with full precision Simulations have been performed 77 p 41 to study the influence of limited accuracy of the hit information on the proton identification capabilities Thereupon the precision of the trigger primitives was defined as follows to optimize the transmission rate over the VXS backplane 4 3 Electronics Framework 49 11 12 13 14 15 16 17 18 19 20 21 7 i er 9 payload slots for GANDALF 9 payload slots for GANDALF VME64x slot TIGER A TIGER B e e VXS 4x full duplex link i e 8 differential pair signals VMEbus backplane connector VXS backplane connector Figure 4 9 The VME64x VXS backplane with dual star configuration has a VXS point to point connec tion from every payload board slot to each of the two switch board slots The direction of transmission for the 8 differential pair signals which is deviating from the definition in the VITA 41 standard is in dicated by the wide arrows There are 8 TX lanes from every GANDALF slot to the TIGER slot B for transmission of the trigger primitives The connection to TIGER slot A implements 6 TX lanes for event data transmission and 2 RX lanes for TCS data reception Slot number 1 is reserved for the VMEbus CPU The diagram is based on 85 p 41 e a 9 bit amplitude by discarding the 3 least significant bits of the ADC digit a10 bit coarse time which provides a dynamic range for the time measurement of 1024 ns a 6 bit high res
251. tors are disabled During the design phase of a high pin count FPGA board a careful floor and pin planning is mandatory Floor planning involves the detailed analysis of the data streams in the FPGA to allow for a wise placement of the logical blocks on the chip area which in turn supports the routing process during the firmware implementation to meet the timing constraints and reach a higher performance This procedure is accompanied by the pin planning since the connec tion of data buses to certain package pins naturally defines where these data enter or leave the silicon In practice the result of these two planning processes is always a trade off between op timal routability of the signal traces on the PCB see also 5 1 7 and a reasonably arranged data flow through the FPGA logic The package pin plan for the TIGER FPGA is shown in Fig 5 9 and the pin out is tabulated in appendix B Located directly behind the I O drivers and receivers there are dedicated logic resources to assist the interfacing with high speed signals Amongst others these are DDR buffers IODELAY elements and SerDes logic blocks which are employed for the VXS connection between the GANDALF and TIGER modules as described in section 4 3 2 1 Stub Series Terminated Logic 12 Digitally Controlled Impedance 62 5 The TIGER Module 2 11 12 13 Usi de 4r cocoa ue 4r Jr ue 14 15 16 17 18 19 e D 20 2
252. total of eight differential clock signals enter the Virtex 6 FPGA on the TIGER board for various purposes see Tab 6 2 Three of them are connected to the clock management block namely clk_si1 clk_si2 and clk_ref They are routed through global clock buffers BUFG in order to make them accessible everywhere in the FPGA In addition the following Virtex 6 device primitives are instantiated An IDELAYCTRL module 161 p 112 connected to clk_ref performs the calibration of the individual IODELAY elements to reduce the effects of process voltage and tempera ture variations IODELAY elements are used in the DDR3 memory interface 5 1 2 as well as in the VXS backplane link interface 4 3 2 to dynamically compensate for the different input delays of the signals in the data buses e AMMOM BASE primitive 92 p 40 fed by c1k ref generates a set of clock frequen cies needed for the Readout application in particular 155 MHz 40 MHz 300 MHz and 100 MHz clocks e AMMOM BASE primitive fed by clk_si2 505 44 MHz generates a set of TCS related clock frequencies needed for the Trigger application in particular 0 8 f i 0 2 fii 1 0 fiio and 0 5 fii The second VHDL module which is connected to the CPLD interface is the reset logic It provides various reset signals which are deasserted in a coordinated manner after the device 6 1 FPGA Firmware 109 Table 6 2 List of the clock signals which enter the TIGER FPGA O
253. trometer 3Small Angle Spectrometer 20 3 The COMPASS II Experiment N HCAL1 B N CAL1 M RICH N Straws N SM1 MF1 E LAS ECALO CAMERA Figure 3 1 Spectrometer setup of the COMPASS II experiment for the 2012 DVCS run consisting of a large angle spectrometer LAS and a small angle spectrometer SAS Additional tracking detectors which are not visible in this picture are present in both spectrometer stages The various detectors are described in the text 36 3 1 The Beam 21 lowing for a wide range of hadron structure and spectroscopy measurements For the DVCS process polarized muon beams are required Protons are accelerated in the SPS to momenta up to 400 GeV c They are extracted in so called spills which are approx 9 6 seconds long at intervals of typically 43 seconds The protons are deflected to a primary beryllium target T6 whose length can be chosen depending on the required secondary beam intensity Thereby mainly pions and kaons are produced which are then momentum filtered and sent through a 600 m long tunnel where they partly decay to muons during their transit At the end of the tunnel the remaining hadrons are removed in a beryllium absorber The muons are then focused momentum selected and guided through a 800 m long beam line Here the beam is bent upwards by dipole magnets to bring it from the underground SPS to the surface where it is bent back in a horizontal direction before e
254. ubject html D M LLER D ROBASCHIK B GEYER 1994 Wave Functions Evolution Equations and Evolution Kernels from Light Ray Operators of QCD Fortschritte der Physik 42 2 101 A V RADYUSHKIN 1997 Nonforward parton distributions Phys Rev D 56 9 5524 X Jr 1997 Gauge Invariant Decomposition of Nucleon Spin Phys Rev Lett 78 4 610 M DIEHL 2003 Generalized parton distributions Physics Reports 388 2 4 41 J COLLINS A FREUND 1999 Proof of factorization for deeply virtual Compton scat tering in QCD Phys Rev D 59 074009 X Jr J OSBORNE 1998 One loop corrections and all order factorization in deeply virtual Compton scattering Phys Rev D 58 094018 K GOEKE M V POLYAKOV M VANDERHAEGHEN 2001 Hard exclusive reactions and the structure of hadrons Progress in Particle and Nuclear Physics 47 2 401 X Jr 1997 Deeply virtual Compton scattering Phys Rev D 55 11 7114 COMPASS COLLABORATION 2010 COMPASS II Proposal Proposal CERN Geneva SPS and PS Experiments Committee CERN SPSC 2010 014 SPSC P 340 M BURKARDT 2005 Transverse deformation of parton distributions and transversity de composition of angular momentum Phys Rev D 72 094020 A V BELITSKY D M LLER A KIRCHNER 2002 Theory of deeply virtual Compton scat tering on the nucleon Nuclear Physics B 629 1 3 323 H FISCHER DVCS Program at COMPASS II Presentation at the ECT Trento GPD2010 Workshop Trento Italy Octobe
255. urement principle and detector design will be described below COMPASS Apparatus for Measurement of Exclusive ReActions 3 2 The Target Region 23 Figure 3 4 Left Photo of the downstream end of the liquid hydrogen target cell made of Kapton film with a Mylar end cap Right Photo of the Mylar film window for the vacuum cryostat 40 Figure 3 5 CAD drawing of the CAMERA detector viewed from downstream 41 The overall dimen sions are 4 2 m in length and 3 2 m in diameter The barrel can be rotated on the platform by 90 for calibration measurements with cosmics 24 3 The COMPASS II Experiment light guides photo multiplier tubes polyethylene rings plastic rings scintillator elements uu MN downstream side upstream side Figure 3 6 Individual view of the Ring A elements of the CAMERA detector consisting of scintillators light guides and photo multiplier tubes 36 The velocity of the recoil particle from the target is determined by a time of flight measurement between two barrels of scintillators which surround the target At the same time the energy loss of the particle traversing the scintillating material is measured Using the Bethe Bloch formula 42 these information allow to identify protons and distinguish them from other particles e g pions or ray electrons from the target The CAMERA detector has been designed for full azimuthal angular acceptance In z direction it has t
256. urpose inputs and outputs have been added to the TIGER board which may be used for arbitrary tasks within the Virtex 6 firmware First of all there are two LEMO 51 for example Finisar FTLF8524P2BNV SFP Transceiver 52High speed Optical Link for Atlas 102 5 The TIGER Module 00 sockets available on the front panel to provide the output signals IVTTLOUTI and LVT TLOUT2 These IVTTL compatible signals are controlled via the FPGA user design and can provide up to 24 mA of continuous output current The LEMO outputs are most commonly used for the transmission of the generated trigger signal from the TIGER trigger processor to the COMPASS trigger logic A dual VHDCI female connector Honda HDRA ED136LFZGT 157 is employed to provide 32 IVDS inputs and 32 IVDS outputs The inputs are located on the left hand side connector and the outputs are located on the right hand side connector see Fig 5 35 The connector pin out is to be found in appendix C 2 Both input and output signals are buffered by LVDS receivers drivers before entering the FPGA in order to protect its I Os from short circuits and electrostatic discharges The IVDS I Os may be used in the FPGA design for example to input additional detector information like veto signals and to output additional trigger related signals Finally also the front panel display and the LEDs can be counted to the I Os in a sense Fig 5 34 The 32 x 128 pixel OLED display of the type
257. urrent and protects the system in the event of a failure by limiting the output current and shutting down the MOSFET based on a programmable fault threshold When the load current exceeds this threshold a fault timer is started which will eventually shut down the device if the over current condition is still present on expiration of the timer The 2Positronic VPX6W6M400A1 B1A5A6A 21 Advanced Configuration and Power Interface 72 5 The TIGER Module occurrence of such a fault event is indicated by the red fault LED LD6 on the back side of the TIGER board It can be reset by either toggling the VME switches or power cycling the crate The green power good LED LD7 indicates that the three S5 power rails have been ramped up successfully As soon as the CPLD is initialized it executes the power up sequence for the main branch Almost all devices on the TIGER board are powered by the various voltage rails of this power distribution branch The main 5 V rail VCC5VO is enabled by a TI TPS2331 hot swap controller linked to a IRF6718 MOSFET which is rated at 61 A and features a very low on resistance of 0 5m0 111 Using an external sense resistor the TPS2331 implements a circuit breaker with a programmable overcurrent threshold to shut down the power rail in case of a short circuit A potential fault would be indicated by the red LED LD8 Out of the main 5 V rail all other voltages which are needed on the TIGER board are generated by DC DC c
258. ve Production GENerator 44 4 CAMERA Proton Trigger N E UN 9 Entries 3717991 Meanx 0 4286 Meany 3 898 RMSx 0 1503 RMSy 2 279 Entries 3717991 Mean x 0 4285 Meany 37 22 RMSx 0 1503 RMSy h e o Oo TTT TTT o Oo TTT O o N piu lye ee o Energy loss in Ring A MeV gt oo Energy loss in Ring B MeV N oO Oo TIT T o TTT 9 02 03 04 05 06 0 7 0 8 02 03 04 05 06 B ToF Figure 4 6 Simulated energy loss of the DVCS proton in Ring A left and Ring B right of the CAMERA detector vs the proton velocity Top which is determined from the time of flight 78 p 90 The contin uation of the rising trend above the cusp in the Ring B energy loss distribution is due to proton tracks with a small polar angle 0 Events have been generated with HEPGen and detector response has been simulated with TGEANT of detectable protons of 6 0 27 at the vertex Due to the large energy loss of these slow protons in the target material the A counter and the air the velocity Pror that is reconstructed from the time of flight measurement between the A and B counters is significantly lower Prog 0 15 While this effect is corrected for in the offline analysis details in 79 for the real time trigger processing one is limited to the velocity calculated from the ToF measurement i e the mean velocity between the A and B counter From the
259. verify commands to load firmware data into FPGAs or CPLDs For this purpose Xilinx provides pro gramming cables Platform Cable USB and software Xilinx iMPACT 108 p 67 for convenient prototype development or debugging see also 6 1 1 5 Furthermore the JTAG connection is used to program the Platform Flash XL indirectly through the FPGA since the Flash has no own JTAG interface For this task the Xilinx iMPACT software first initializes the FPGA with a proprietary firmware 107 p 10 featuring an in system pro gramming engine which gains access to the Flash through its standard NOR flash interface Afterwards the configuration file is transferred via JTAG instructions to the programming en gine which executes the necessary write commands on the flash interface This programming path is depicted by the red arrow labeled with in Fig 5 16 An alternative path labeled with 19Joint Test Action Group 5 1 Mainboard 71 standby branch 5V IN VCC5VO S5 zul TPS2420 TPS TPS hot swap controller 73601 74801 Lo vccava ss VCC1V8 S5 main branch 5VINO O VCC5VO 30 A max TPS2331 hot swap controller amp circuit breaker VCCINT 1 5V 2 5V 3 3V 12 5V VTTDDR MGTAVTT MGTAVCC CPLD converter CPLD 4 Figure 5 17 Schematic overview of the power distribution system for the TIGER module The DC DC converter part is detailed in Fig 5 18 for sending
260. ween the TIGER boards Besides the data ports there are data valid and ready flags available to control the data flow Gaps in the data transmission may be generated by the sender at any time by pulling the valid flag low Native flow control signals enable the receiver to pause the data transmission by sending an XOFF message Finally some status signals are available to indicate software or hardware errors as well as the channel initialization status In order to achieve the highest possible data rate the GTX transceiver settings cf 5 1 1 6 must be tuned carefully They greatly depend on the electrical characteristics of the transmission channel The optimum TX emphasis parameters have been obtained by channel simulations as described in section 5 1 7 4 The RX equalization parameters are determined using the Chip Scope IBERT core 167 p 18 With the help of this core a bit error rate test of the high speed serial channels between two TIGER boards is performed Each FPGA implements an instance of the IBERT core which generates a pseudo random test pattern 31 bit PRBS for transmis sion to the link partner and at the same time it checks the pattern that it receives from the link An automatic sweep of the RX parameters can be performed to determine the settings with the lowest bit error rate After the optimum settings had been obtained 12 hour test runs have been conducted for data rates of 5 0 Gbit s and 6 25 Gbit s The 5 0 Gbit s channel
261. will operate according to these parameters on subsequent switch ons For each ofthe four rails the UCD9246 integrates circuitry for DC DC loop management which generates a PWM signal to control the MOSFET driver of the connected PowerTrain module and it monitors its output voltage and compares it to the programmed reference On deviation from the reference the pulse width of the control signal is adjusted to compensate the error In addition the output current and the temperature of the power stage is constantly monitored and actions like shut down or power limiting can be taken as soon as programmable thresh olds are exceeded Power sequencing soft start soft stop and voltage tracking can be defined separately for every rail to satisfy the requirements of the powered devices The start and stop sequences implemented for the TIGER board are shown in Fig 5 19 Dependencies between rails can also be defined for example the FPGA I O voltage rails are immediately shut down in case of a failure on the FPGA core voltage rail whereas the 3 3 V rail stays unaffected to keep the TIGER board responsive It can be seen in Fig 5 18 that there are some additional rails in the main power distribution branch which are not controlled by the UCD9246 They will be briefly summarized in the fol lowing 24pulse Width Modulation 74 5 The TIGER Module 3 254 3 00 2 75 2 50 225 1 2 00 V 1 15 U out
262. xisting GANDALF framework which has recently been devel oped for the COMPASS II experiment As described in section 4 3 the GANDALF framework is based on the VXS ANSI VITA 41 standard which combines the well proven VMEbus with a high speed point to point connection on a single backplane While the GANDALF transient analyzers are VXS payload boards the TIGER module is operated in the VXS crate s switch slots And although it is designed with the form factor of a VXS switch board the TIGER module is a good deal more than just a switch It provides immense com puting power for a sophisticated trigger generation large memories to buffer detector readout data and several interfaces to the existing COMPASS DAQ equipment Fig 5 1 shows the main functional blocks of the TIGER module The centerpiece of the board is a large field programmable gate array FPGA which carries out the actual high performance data processing It also controls most of the high speed interfaces and memory devices The FPGA is accompanied by a CPU which runs a standard Linux operating system OS and pro vides the necessary interfaces for slow control tasks like configuration and monitoring of the TIGER system Optionally a GPU can be added to the system to assist with complex calcu lations Both CPU and GPU are located on exchangeable add on boards complying with the COM Express and the MXM standards respectively see 5 2 They are connected to the FPGA with PCI Expre
263. y vertex and improve the momentum resolution of the CAMERA detector in the offline analysis 7 2 Physics Data 135 0 12 Ad no SC only Ring A 1 with SC 0 11 o p p 0 1 0 09 0 08 i He Fd Fe 0 07 0 06 0 05 0 04 0 03 1 0 7 P GeV o oO o gt o al 0 6 Figure 7 5 Relative momentum resolution for recoil protons 79 The red triangles show the resolution when the time stamp of Ring A is used for the time of flight determination Blue diamonds show the resolution when the StartCounter SC is used to calculate a vertex time for the alternative time of flight determination An improvement of up to 20 for large momenta is achieved 7 2 Physics Data 7 2 1 First Proton Signals The first physics data with CAMERA have been recorded with a pion beam scattered off the liquid hydrogen target From these runs a data sample of elastic pion proton events has been extracted in order to characterize the detector and to obtain precise calibration values which are required for the later analysis of the pion and muon runs For each scintillator the effective speed of light ce and a correction offset for the time difference fair between upstream and downstream pulse arrival time is determined in order to calibrate the z position Eq 4 2 of the hits in the counter 79 This method of calibration is more precise than the T0 calibration with the laser system cf 7
264. zation The quark helicity distribution is defined in the parton model as the difference of the density distribution of quarks with helicity parallel qF or anti parallel 44 to the nucleons helicity Aqr xg q7 xB q xn 2 18 The unpolarized or polarization averaged parton distribution which was introduced above can be written as _ qf xB q7 xB qF xp 2 19 From this follows the positivity limit Aq 25 lt qf xp The spin dependent structure function g Fig 2 5 can be extracted from the difference of the cross sections with same and opposite polarizations of beam and target 13 p 99 measured in deep inelastic scattering 25 2 3 2 dio do 40 E dal anda mer E E cos gi xg 2M xgo xg 2 20 2 4 Generalized Parton Distributions 9 Because E gt M one is not sensitive to g2 in an experiment with a longitudinal polarized target It can only be determined using a transverse polarized target The polarized PDFs contribute to g xg analogous to Eq 2 15 1 amp xB 5 2 ej qp 2 21 27 By integrating the polarized parton distributions over xg one obtains their first moments Aq dxshage 2 22 By summing over all quark and anti quark flavors one gets AL Y Agp 2 23 f the contribution of the quark helicities to the nucleon spin in the sum rule 2 1 2 4 Generalized Parton Distributions The concept of generalized parton distributions GPDs wa

Download Pdf Manuals

Related Search

Related Contents

HP 802.11 b/g Wireless Printer Adapter Quick Start Manual  Kambrook KSM1PA Musical Toy Instrument User Manual  広報うちなだ 平成23年4月号（PDF）  Windsor 86221950 User's Manual  Transcend PF830    QuickSpecs  MSI Megabook GT640 GT663R-i7468BLW7P  Deliverable 18.3 Notification of Delivery of the  User Manual IMPORTANT SAFETY INSTRUCTIONS  

Copyright © All rights reserved. Failed to retrieve file