RU Builder User Manual
Contents
1. nbSuperFragmentInEvent 1 The blockNb field is numbered from to nbBlocksInSuperFragment 1 The buResourceld field is an opaque handle that a FU should use to identify events resources to be discarded The fuTransactionlId field is the FU transaction id of the FU request that caused the BU to reply with the current event DAQ CERN PH CMD 2005 Page 19 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 The 120 BU DISCARD C structure is as follows typedef struct _I20 BU DISCARD MESSAGE FRAME I20_PRIVATE_MESSAGE_ FRAME PvtMessageFrame U32 n U32 buResourceld 1 I20 BU DISCARD MESSAGE FRAME PI20 BU DISCARD MESSAGE FRAME The FU must fill n buResourceld The n field specifies the number of events resources to be discarded The buResourceld field is an array of the ids of the BU resources to be discarded DAQ CERN PH CMD 2005 Page 20 of 32 5 Application state machines RU Builder User Manual Date 8 16 2005 5 Application state machines 5 1 Commonalities of the application finite state machines The finite state machines of the BUs EVM and RUs have commonalities Figure 9 shows the finite state transition network FSTN which all three types of application follow There are three common behaviors Firstly all RU builder applications read and act upon configuration parameters when they receive a Configure SOAP message Secondly all RU builder applications only participate in event building when t2. event step 7 When a FU has finished with an event it tells the BU to discard it step 8 Each BU has its own worker thread that executes the behavior of that BU The eventldFIFO blockFIFO requestFIFOs and discardFIFO are used by the peer transport thread s to store incoming messages ready for the worker thread to process them The fullResourceFIFO IS manipulated solely by the worker thread It is used to store which events were built in which order This enables a BU to service a FU request with the next event that was built lonk re le 1 Confirm TILL Resource Allocate new and or clear previous l fullResourceFIFO discardFIFO Ja CELE FU Allocate discard FU take Figure 2BU FIFOs DAQ CERN PH CMD 2005 Page 10 of 32 3 RU builder application FIFOs RU Builder User Manual Date 8 16 2005 3 2 EVM FIFOs The EVM is responsible for controlling the flow of event data through the RU builder To understand the internal FIFOs of the EVM it is first necessary to know its dynamic behavior Figure 3 shows the internal FIFOs of the EVM The EVM tells the TA the capacity of the RU builder by sending it trigger credits step 1 One trigger credit represents the ability to build one event Given a credit the TA sends the EVM the trigger data of an event step 2 The EVM pairs the trigger data with a free event id step 3 The EVM also requests the RU s to readout the event s data step 4 A BU with the ability to
3. 2005 1 4 References 1 The CMS collaboration The Trigger and Data Acquisition project Volume Il Data Acquisition amp High Level Trigger CERN LHCC 2002 26 ISBN 92 9083 111 4 1 5 Definitions Acronyms and Abbreviations BU Builder Unit 120 Intelligent Input O utput CVS Concurrent Versioning System RU Readout U nit DAQ Data Acquisition system RUI Readout U nit Input EVB Event builder TA Trigger Adapter EVM Event M anager TDR Technical Design Report FED Front End Driver TriDAS Trigger and Data Acquisition FSTN Finite State Transition N etwork XDAQ Cross platform data acquisition toolkit FU Filter U nit DAQ CERN PH CMD 2005 Page 7 of 32 2 RU builder overview RU Builder User Manual Date 8 16 2005 2 RU builder overview 2 1 How the RU builder fits within the EVB The RU builder is a component of a larger system called the event builder EVB The EVB is a distributed application that reads out event fragments from one set of nodes and assembles them into entire events in another set of nodes Figure 1 shows the applications of the RU builder and how they interact with the rest of the EVB e FED builder Super fragments Trigger credits Trigger data Allocates Discards Events FU 0 FU q Permanent storage Figure 1 RU builder applications and how they interact with the rest of the EVB The external interfaces of the RU builder assume that triggers are given to the EVM in the same order as their corre
4. AS cvs login You will be prompted for a password CVS password Please enter 98passwd The RU builder source code needs to be placed in the directory XDAQ ROOT dag evb Enter the following commands to put it there using the CMS CVS server cd XDAQ ROOT cvs export r EVB_S_18305 V3_3 TriDAS dagq evb The RU builder source code depends on two 120 interface files in the directory SXDAQ ROOT dagq interface Enter the following commands to obtain these files from the CMS CVS server cd XDAQ ROOT cvs export r EVB_S 18305 V3_3 TriDAS dag interface evb include i20EVBMsgs h cvs export r EVB_S_ 18305 V3_3 TriDAS dagq interface shared include i2o0XFunctionCodes h DAQ CERN PH CMD 2005 Page 28 of 32 10 RU builder self test RU Builder User Manual Date 8 16 2005 9 2 Building the application libraries The RU builder is composed of seven XDAQ application libraries There is one library for each of the RU builder applications libBU so libEVM so and libRU so Plus there is one library for each example application libFU so libRUI so and libTA so Finally there is one library for the RU BuilderTester application As its name suggests this application performs the self test feature of the RU builder The locations of the libraries are as follows TriDAS daq evb bu 1ib linux x86 1ibBU so TriDAS dagq evb evm 1ib linux x86 1ibEVM so TriDAS dagq evb ru 1ib linux x86 1ibRU so TriDAS daq evb examples fu 1ib linux x86 1ibFU so TriDA
5. CMD 2005 Page 26 of 32 8 Exported configuration parameters RU Builder User Manual Date 8 16 2005 The default values of the BU control parameters were calculated as follows e To prevent a BU from monopolizing event ids each BU has a maximum number of event ids it can acquire at any moment in time Assuming all BU s are equal each BU is allowed to acquire BU maxEvtsUnderConstruction nbEvtIdsInBuilder number of BUs 4096 64 64 e The blockFIFO of a BU see figure 2 in section 3 2 is responsible for buffering incoming event data In the worst case this FIFO would have to buffer the blocks of all outstanding requests for event data BU blockFIFOCapacity maxEvtIdsUnderConstruction x size of an event block size 64x1MB 4KB 64 x256 16384 e ABU hasa single FIFO called discardFIFO for FU discard messages and one FIFO per FU called requestFIFO for FU request messages Knowing that a BU can service a maximum of 64 FUs and assuming that a single FU will never have more than 1024 outstanding requests for events BU discardFIFOCapacity 1024 x maximum number of FUs 1024 x 64 65536 BU requestFIFOCapacity 1024 The default values of the RU exported parameters were calculated as follows e The blockFIFO of the RU is responsible for buffering incoming super fragment data Assuming a RUI only gives as many super fragments to a RU as the TA gives triggers to the EVM then in the worst case the blockFIFO must hold t
6. OsofaRU The EVM sendsaRU an event id trigger event number pair when it asks the RU to readout the corresponding event s data step 1 In parallel the RUI informs the RU of event data that is ready to be processed step 2 A RU places each super fragment for which it has received a pair into the fragment lookup table step 3 BUs ask RUsto send them the super fragments of the events they are building step 4 A RU services a BU request by retrieving the super fragment from its fragment lookup table and asking the BU to cache the super fragment step 5 Each RU has a worker thread that executes the behavior of that RU All of the internal FIFOs of a RU that is to say the pairFIFO blockFIFO and requestFIFOS are used by the peer transport thread s to store incoming messages for the worker thread to process blockFIFO EVM Readout pairFIFO E Fragment request Iros fe Cache Figure 4 RU FIFOs DAQ CERN PH CMD 2005 Page 12 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 4 12O interface All the 120 messages of the EVB including the internal and external messages of the RU builder are defined in the package TriDAS dagq interface The 120 function codes of all the RU builder 20 messages are given in the file TriDAS dagq interface shared include i2o0XFunctionCodes h The C structures that define the 120 messages are in the file TriDAS dag interface evb include i20EVBMsgs h The I2O interface o
7. S daq evb examples rui 1lib linux x86 1ibRUI so TriDAS dagq evb examples ta 1lib linux x86 l1ibTA so TriDAS dagq evb rubuildertester 1l1ib linux x86 libRUBuilderTester so Enter the following to build these libraries cd XDAQ_ ROOT dag evb make IORU builder self test This section explains how to perform the self test of the RU builder This test helps determine whether or not the RU builder has been successfully installed The self test consists of the RU BuilderTester application plus one EVM one RU and one BU all running on the same XDAQ executive The EVM is told to generate dummy triggers the RU is told to generate dummy super fragments and the BU is told to drop the events it builds The step by step instructions to run the self test are Step 1 Obtain and build the RU builder See section 9 for instructions Step 2 Open a terminal and create the XML configuration file for the self test by typing cd XDAQ_ ROOT dagq evb xml produceSelfTestXml pl HOST PORT XDAQ_ ROOT 1x1SingleXDAQ template xml gt 1x1SingleXDAQ xml W here HOST and PORT indicate the port on which the executive shall service HTTP requests DAQ CERN PH CMD 2005 Page 29 of 32 10 RU builder self test RU Builder User Manual Date 8 16 2005 Step 3 Run the XDAQ executive by typing SXDAQ_ ROOT dag xdag bin linux x86 xdaq sh h HOST p PORT c 1x1SinglexXDAQ xml W here HOST and PORT are the same as those given in step 2 Step 4 Open a web browser an
8. The Compact Muon Solenoid Experiment TriDAS Trigger and Data Acquisition RU Builder User Manual Version 3 3 August 16 2005 Version 3 3 Date August 16 2005 Authors S Murray CI identifier EVB D_ 18306 CERNY Fermilab Revision History July 11 2003 July 31 2003 Finalization of version 1 0 J Gutleber May 24 2004 Updated for version 2 0 of the EVB now referred to as S Murray the RU builder November 1 2004 Updated for version 2 1 of the RU builder S Murray May 2 2005 Updated for version 3 0 of the RU builder S Murray July 4 2005 Updated for version 3 2 of the RU builder S Murray August 16 2005 Updated for version 3 3 of the RU builder S Murray 0 1 1 0 2 0 2 1 3 0 3 2 3 Cl Record C CI Identifier EVB_D_18306 Description Describes the RU builder for an integrator who will put the builder into a full DAQ system S Murray Steven Murray cern ch 10 11 Table of Contents Introduction 1 1 Document purpose and scope 1 2 Intended readership 1 3 system requirements and dependencies 1 4 References 1 5 Definitions Acronyms and Abbreviations RU builder overview 2 1 How the RU builder fits within the EVB 2 2 What the EVB does 2 3 The RU builder applications RU builder application FIFOs 3 1 BU FIFOs 3 2 EVM FIFOs 3 3 RU FIFOs I20 interface 4 1 TA EVM interface 4 2 RU RUI interface 4 3 BU FU interface Application state machines 5 1 Commo
9. Us e A RU has 64MB of physical memory for caching super fragments e Aneventis 1MB e An event is made up of 64 super fragments 1 per RU of equal size therefore the size of a super fragment is 16KB e The block size size of an 120 message frame used to transport event data is 4KB e The RUIs only give as many events to the RU s as the TA gives triggers to the EVM e The maximum number of FUs per BU is 64 e A FU will never have more than 1024 outstanding requests for events e Fast control messages are sent if they are older than 1 second e The packing factor of fast control messages is 8 The need to know the total number of event ids in the RU builder is common to all three types of RU builder applications e The total number of event idsin the RU builderis a function of RU memory Assuming each RU has 64MB of memory for buffering super fragments and that the size of an event is 1 MB BU EVM amp RU nbEvtIdsInBuilder sum of the memory of all RUs size of an event 64 x 64MB 1MB 4096 The BUs and EVM send fast control messages e Fast control messages are sent if they are older than 1 second BU amp EVM ageMessages true BU amp EVM msgAgeLimitDtMSec 1000 It has been assumed that all events have the fixed size of IMB If the RU builder is to build events of varying sizes then the appropriate safety factor needs to be taken into account when calculating the number of event ids in the RU builder DAQ CERN PH
10. antee that the EVM will be able to buffer BU requests for event ids or that the RUs will be able to buffer BU requests for event data e The configuration parameters of a RU builder application must be set before it is configured DAQ CERN PH CMD 2005 Page 32 of 32
11. build an event will ask the EVM to allocate it an event step 5 Within such a request a BU will normally give back the id of an event to be cleared For each cleared event id the EVM sends a trigger credit to the TA and makes the id a free event id step 6 The EVM confirms the allocation of an event by sending the requesting BU the event id and trigger data of the allocated event step 7 The EVM has a worker thread that executes the behavior of the EVM The triggerFIFO clearedEventIdFIFO and requestFIFO are used by the peer transport thread s to store incoming messages ready for the worker thread to process them The pairFIFO and freeEventIdFIFO are manipulated solely by the worker thread The pairFIFO keeps track of the event id trigger data pairs that have yet to be sent to requesting BUs The freeEventIdFIFO stores the ids of free events for which trigger credits have been sent to the TA RU Readout EVM pairFIFO creLggerct Iro a HHH Trigger TA Credit clearedEventIdFIFO freekventIdFIFO Allocate and or clear request Iro L BU Confirm Step 7 Figure 3 EVM FIFOs DAQ CERN PH CMD 2005 Page 11 of 32 3 RU builder application FIFOs RU Builder User Manual Date 8 16 2005 3 3 RU FIFOs A RU is responsible for buffering super fragments until they are request by the BUs To understand the internal FIFOs of a RU itis first necessary to know its dynamic behavior Figure 4 shows the Internal FIF
12. control software or how to setup a trigger or event data source It is also not the purpose of this document to describe the internal workings of the RU builder Developers are referred to the source code for such information The code has been structured and commented so that it can be easily read and understood It is recommended to use Doxygen to generate documentation from the code as compatible comment tags have been used If the reader is not familiar with Doxygen then they are referred to its website htto www doxygen org description of the RU builder given in this document can be relied upon to be valid beyond this release N l It must be emphasized that the RU builder is still under development and subject to change No 1 2 Intended readership This document is intended for a system integrator someone that needs to integrate the RU builder into a data acquisition system DAQ Itis assumed that the DAQ system is based on the XDAQ framework If the reader is not familiar with this framework then they are referred to the XDAQ website http xdaq web cern ch xdaq 1 3 System requirements and dependencies Version 3 3 of the RU builder only supports the Linux operating system running on an Intel x86 processor The code was tested using version 3 2 3 of gcc This version of the RU builder is dependent on version 3 2 2 of the XDAQ core framework DAQ CERN PH CMD 2005 Page 6 of 32 1 Introduction RU Builder User Manual Date 8 16
13. d enter the following URL http HOST PORT Where HOST and PORT are the same as those of steps 2 and 3 You should now see the HyperDAQ page of the XDAQ executive you started in step 3 It should look similar to the following screenshot te A HyperDAQ http 137 138 236 89 66432 um xdag application lid 3 O Control Panel Cluster Explore Executive FeerlranspotHT PeerlransportFil ARelay uid ag Urn ag urn xdaq um daq application lid 0 application lid 1 application lid s application lid 4 HyperDAt RUBuilderlester Eh EL BL Urn ag urn ag umd ag urns ag Urn ag application lid 3 application lid 12 application lid 13 application lid 14 applicatian lid 15 Figure 11 HyperDAQ web page for self test DAQ CERN PH CMD 2005 Page 30 of 32 10 RU builder self test RU Builder User Manual Date 8 16 2005 Step 5 Go to the default web page of the RU BuilderTester application by clicking on the RU BuilderTester link Your web browser should now display something similar to TA IJI EMi O RU O BIJ O Fu hE ae eventNb 0 state Halted deltaT 10098836 00 delta J Figure 12 RUBuilderTester web page Step 6 Go to the control web page by clicking on the Ctrl icon You should now see something like start Figure 13 RUBuilderTester control web page Step 7 Start the self test by clicking the start button The button should now be labeled stop Step 8 Retu
14. e 8 BU FU interface sequence diagram A FU requests a BU to allocate it one or more events step 1 In response the BU asks the FU to take the requested event data as a set of event data blocks step 2 When a FU has finished processing one or more events it tells the BU to discard them step 3 Fi The BU FU interface of this version of the RU builder does not support partial events Partial events may be supported in a future version DAQ CERN PH CMD 2005 Page 17 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 The I20_BU_ALLOCATE MESSAGE FRAME C structure and its companion BU_ALLOCATE C structure are as follows typedef struct _BU_ALLOCATE U32 fuTransactionIid U32 fset BU_ALLOCATE PBU_ ALLOCATE typedef struct _I20 BU ALLOCATE MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 n BU_ALLOCATE allocate 1 I20 BU_ALLOCATE MESSAGE FRAME PI20 BU ALLOCATE MESSAGE FRAME The FU must fill n allocate The n field specifies the number of events the FU is requesting The allocate field is an array of FU transaction ids and fragment sets For each event a FU requests the FU fills in the fuTransactionId field and the fset field of a BU_ALLOCATE C structure and puts itin the allocate array The fuTransactionlId field is a transaction id that a FU can use to match its requests with the events it receives A BU treats the fuTransactionId field as being opaque in other words It is not interpre
15. e sequence diagram A super fragment is composed of one or more I20 EVENT DATA BLOCK MESSAGE FRAMES The I20 EVENT DATA BLOCK _MESSAGE_FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE_ FRAME I20_ PRIVATE_MESSAGE_ FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20 EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The RUI must fill eventNumber nbBlocksInSuperFragment blockNb The nbBlocksInSuperFragment field gives the number of blocks the super fragment is composed of The blockNb field indicates the block s position within the super fragment Blocks are numbered from 0 to nbBlocksInSuperFragment 1 DAQ CERN PH CMD 2005 Page 16 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 4 3 BU FU interface The BU FU interface specifies how e A FU requests events from a BU e ABU sends an event to a FU e A FU tells a BU to discard an event Figure 8 is a sequence diagram describing the protocol between a BU and a FU FU BU Step 1 NS Function code I20 BU ALLOCATE C structure I20 BU ALLOCATE MESSAGE FRAME Function code I20 FU_TAKE C structure I20 EVENT DATA BLOCK MESSAGE FRAME Step 3 sega nn Ca I20 BU DISCARD C structure I20 BU DISCARD MESSAGE FRAME Figur
16. f the RU builder is subject to change The description of the interface provided by this document cannot be relied upon to be valid beyond this release Figure 5 shows the external I20 interfaces of the RU builder the TA EVM interface the RUI RU interface and the BU FU interface This chapter is divided into three sections one for each interface RUI RU interface RU builder TA EVM interface TA gt BU FU interface Figure 5 Extemal 120 interfaces of the RU builder DAQ CERN PH CMD 2005 Page 13 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 4 1 TA EVM interface The TA EVM interface specifies how e The EVM gives the TA tngger credits e The TA gives the EVM trigger data Figure 6 is a Sequence diagram describing the protocol between the EVM and the TA TA EVM Function code I20 TA CREDIT Step ae C structure I20_TA_CREDIT_MESSAGE_FRAME Function code I20 EVM TRIGGER C structure I20 EVENT DATA BLOCK MESSAGE FRAME n Figure 6 TA EVM interface sequence diagram The EVM communicates with the TA using a credit based mechanism The EVM tells the TA the current Capacity of the RU builder by sending the TA a trigger credit count step 1 One trigger credit represents the RU builder s ability to build one event The TA should only send the EVM trigger data for as many events as the EVM has given the TA credits step 2 The TA is responsible for getting receiving trigger data from the t
17. he blocks of as many super fragments as there are event ids in the RU builder blockFIFOCapacity nbEvtIdsInBuilder x size of a super fragment block size 4096 x 16K 4KB 16384 The parameters of the form Packing should not normally be modified as they have only been tested with the default value of 8 However the user may modify them if they are experiencing performance problems with the RU builder DAQ CERN PH CMD 2005 Page 27 of 32 9 How to obtain and build the RU builder RU Builder User Manual Date 8 16 2005 9 How to obtain and build the RU builder This chapter is divided into two sections The first explains how to obtain the RU builder source code and the second explains how to build it Both sections assume the following e The user has already installed the XDAQ core framework e The user has defined the shell environment variable XDAQ ROOT to point to the root of their XDAQ installation in other words their TriDAS directory e The shell of the user is tsch e The user accesses the CMS CVS server as an anonymous read only user 9 1 Checking out the source code from CVS The RU builder source code is stored in the CMS CVS server For more information about this server please look at the webpage http cmsdoc cern ch cmsoo projects cvs server html You must login to the CMS CVS server if you wish to use it To do so enter the following setenv CVSROOT pserver anonymous cmscvs cern ch cvs_server repositories TriD
18. hey are enabled Thirdly all RU builder applications throw away their internal data and any incoming I20 message frames when they are halted Configuration parameters are read and acted The application can only participate in event upon when the application is configured building when it is in the Enabled state The application releases all incoming 120 message The contents of all internal data frames when it is in the Halted state structures are discarded when the application is halted Figure 9 FSTN ofa RU builder application DAQ CERN PH CMD 2005 Page 21 of 32 5 Application state machines RU Builder User Manual Date 8 16 2005 5 2 BU EVM and RU finite state machines The FSTNs Specific to each type of RU builder application are shown in figure 10 RU Halted Configure Enable Enable Send request to Send initial EVM for initial Credits Eo TA set of event LOIS Figure 10 BU EVM and RU FSTNs Enabled DAQ CERN PH CMD 2005 Page 22 of 32 6 Starting the RU builder RU Builder User Manual Date 8 16 2005 6 Starting the RU builder The RU builder is part of a larger system called the event builder EVB Besides run control the RU builder communicates with a TA one or more RUIs and one of more FUs The RU builder cannot be started at any arbitrary moment in time its start up must be synchronized with that of the TA RUIs and FUs The RU builder and the EVB components it interacts with are XDAQ applicati
19. messages DAQ CERN PH CMD 2005 Page 24 of 32 8 Exported configuration parameters User Manual S Exported configuration parameters RU Builder Date 8 16 2005 Configuration parameters need to be set before an application is sent a Configure SOAP message Table 1 lists the exported control parameters of each type of RU builder application The type and default value of each parameter is given BU EVMERU ageMessages boot e BU EVM GRU exitonFal o O boo e D Bu blockFOCapaciy unsoed ong 16387 BU discardFiFO Capaciy unsigned long 65536 Bu SO EVM ALLOCATE CLEAR Packing unsinedTong O E BU f maxvisUnderConstruction unsonediong 67 BU 20 RU SEND Packng unsonediong 8 D sendCreditsW ithD ispatchFrame bool _ false 120 RU READOUT Packing unsigned long 8 I20_TA_CREDIT_Packing unsigned long 8 fblockFIFO Capacity unsigned long 16384 Table 1 Exported configuration parameters The default values are set when the RU builder application is instantiated The default values have been chosen with the goal of covering the majority of use cases for the RU builder A user should rarely need to diverge from these default values DAQ CERN PH CMD 2005 Page 25 of 32 8 Exported configuration parameters RU Builder User Manual Date 8 16 2005 The following assumptions were made when calculating the default values of the RU builder s configuration parameters e A RU builder is composed of 64 BUs and 64 R
20. nalities of the application finite state machines 5 2 BU EVM and RU finite state machines Starting the RU builder Stopping the RU builder Exported configuration parameters How to obtain and build the RU builder 9 1 Checking out the source code from CVS 9 2 Building the application libraries RU builder self test Configuration guidelines Oo N NOOO O 24 25 28 28 29 29 32 List of Figures Figure 1 RU builder applications and how they interact with the rest of the EVB 000 0 8 Foo BUTIT O aren meee tr nee daer ean enn Tire A E Arr ee eee eee are 10 Prieur EVM PIE OS sa acenctebatetactnateteeiates dhatncnstcbacentan E E OE A OE EEEO 11 Pieure ARUFI OS srci i E N TCA E E O O 12 Figure 5 External 120 interfaces of the RU Duildef scscsoironiisesinnsna 13 Figure 6 TA EV M interface Sequence diagram sic o assenssueesuts cvassnasaecceescpuabnesbaneeetievawenasseaeeents 14 Fioure RU RUD interlace Sequence diagram ienesco cerie aeereeaeaeheen 16 Figure 8 BU FU interface sequence Giagram ccccccssseesseccceccceeeeeeseeecceeeeeeaesseeeceeeeeeaaaaesees 17 Figure 9 FSTN of a RU builder application eccccsecccccceeeeeseeeeeceeeeesaaeeeeeeeceeeeeeaaeenees 21 Foure 10 BU EVM and RU FSTN eeose E EA T O 22 Fieure tl My perDAO web pace Tor sell TeSt asrcisnieren E E 30 Fis re 12 RU Builder lester Web pace civecete acess aaacenaiidersione wae aes ee nact rer ones eee 31 Figure 13 RU Builder Tester control
21. ons and as such depend on one or more peer transports to communicate with each other These peer transport must be up and running before these applications try to communicate with each other To start the RU builder and the components it interacts with run control should do the following in order 1 Start the necessary peer transports so that the RU builder and its surrounding applications can communicate 2 Start the TA so that is can service credits from the EVM 3 Startthe RU builder so that it can receive super fragments from the RU Is 4 Start the RUIs and FUs so they can start pushing in super fragments and extracting events respectively The RU builder is a distributed application whose nodes BU s EVM and RUs need to be started up in a specific order To put the RU builder into the state where it will build events run control should do the following in order Send Configure to all of the RU builder applications Send Enable to all of the RUs Send Enable to the EVM Send Enable to the BUs NY A Unlike version 2 x of the RU builder the state changes of version 3 0 are synchronous There is no need to poll the state of an application after a request to change state The RUs have to be enabled first because they have to be ready to receive event number event id pairs from the EVM The EVM can start sending these pairs immediately after it has been enable The EVM has to be enabled before the BU s so that it is read
22. rigger and for providing backpressure to the trigger as necessary The 120 TA CREDIT MESSAGE FRAME C structure is as follows typedef struct _I20 TA CREDIT MESSAGE FRAME I20_PRIVATE_ MESSAGE FRAME PvtMessageFrame U32 nbCredits I20_TA_CREDIT_MESSAGE_ FRAME PI20 TA CREDIT MESSAGE FRAME The EVM must fill nbcredits DAQ CERN PH CMD 2005 Page 14 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 The 120 EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The TA must fill eventNumber nbBlocksInSuperFragment blockNb Version 3 0 of the RU builder only supports single block trigger data Therefore the TA must set nbBlocksInSuperFragment to and blockNb to 0 DAQ CERN PH CMD 2005 Page 15 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 4 2 RU RUI interface The RU RUI interface specifies how a RUI passes super fragments to a RU Figure 7 is a sequence diagram describing the protocol between the RUI and the RU RUI RU Function code I20 RU DATA READY C structure I20 EVENT DATA BLOCK MESSAGE FRAME og Figure 7 RU RUI interfac
23. rn to the main web page of the RU builder tester by clicking on the RUB Test icon Clicking the refresh button of your browser should now show events being built For example the eventN b variable of the EVM O should increment TA Rll EVM O RU O BU O Fu hE ae evento 0 state Halted deltaT 10098836 00 deltah J Figure 14 Running RU builder DAQ CERN PH CMD 2005 Page 31 of 32 11 Configuration guidelines RU Builder User Manual Date 8 16 2005 11 Configuration guidelines This chapter summarizes and highlights the most important points with regards to configuring the RU builder The default values of the RU builder control parameters have been chosen for the majority of use cases The user should rarely need to diverge from these values The RU builder is dependent on the instance numbers of the BUs EVM FUs RUs and TA e RUs must be assigned instance numbers from 0 to the number of RUs 1 e BUs must be assigned instance number from 0 to the number of BUs 1 e The EVM must be assigned instance number 0 e The TA must be assigned instance number 0 The RU builder has the following configuration restrictions e A single BU can service a maximum of 64 FUs e The sum of the maximum number of event ids each BU can have at any moment in time BU maxEvtsUnderConstruction must not exceed the total number of event ids in the RU builder BU EVM RU nbEvtIdsInBuilder If this is the case then there is no guar
24. sponding event data is given to the RUs DAQ CERN PH CMD 2005 Page 8 of 32 2 RU builder overview RU Builder User Manual Date 8 16 2005 2 2 What the EVB does For each event the EVB e Reads out the trigger data This trigger data will become the first super fragment of the event e Reads out the fragments of the event from the detector front end drivers FED s e Builds the fragments into RU super fragments using the FED builder e Builds the whole event using the RU builder The whole event is the trigger super fragment plus the set of RU super fragments e Decides whether or not the event is interesting for physics using the filter units FU s e Sends the event to permanent storage if it is interesting for physics or discards it if itis not 2 3 The RU builder applications The RU builder consists of a single event manager EVM one or more readout units RUS and one or more builder units BU s The EVM is responsible for controlling the flow of data through the RU builder The RUs are responsible for buffering super fragments until they are requested by the BUs The BUS are responsible for building and buffering events until they are requested by the filter units FUs The trigger adapter TA readout unit inputs RU Is and filter units FU s are extemal to the RU builder The TA is responsible for interfacing the DAQ trigger to the EVM The RU Is are responsible for pushing super fragment data from the FED builder into
25. ted A BU will send back a copy of the fuTransactionId field in each of the I20_EVENT_ DATA BLOCK _MESSAGE_FRAMES that make up the requested event The fset field isa fragment set identifier Fragment sets are a way to describe partial events The fset field is ignored by the BU in this version of the RU builder because this version does not support partial events DAQ CERN PH CMD 2005 Page 18 of 32 4 2O interface RU Builder User Manual Date 8 16 2005 The 120 EVENT DATA BLOCK MESSAGE FRAME C structure is as follows typedef struct I20 EVENT_DATA_BLOCK_MESSAGE FRAME I20_PRIVATE_MESSAGE FRAME PvtMessageFrame U32 eventNumber U32 nbBlocksInSuperFragment U32 blockNb U32 eventIid U32 buResourceld U32 fuTransactionIid U32 nbSuperFragmentsInEvent U32 superFragmentNb U32 padding I20_EVENT DATA BLOCK MESSAGE FRAME PI20 EVENT DATA BLOCK MESSAGE FRAME The FU should only read nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb buResourcelId fuTransactionIid An event is composed of 120 _EVENT_DATA BLOCK _FRAMES The nbSuperFragmentsInEvent superFragmentNb nbBlocksInSuperFragment blockNb fields are used to identify the position of an event data block within an event An event is composed of one trigger super fragment plus N RU super fragments where N is the number of RUs Therefore the nbSuperFragmentsInEvent field is set to the number of RUs plus 1 The superFragmentNb field is numbered from 0 to
26. the RUs The FUs are responsible for selecting interesting events for permanent storage DAQ CERN PH CMD 2005 Page 9 of 32 3 RU builder application FIFOs RU Builder User Manual Date 8 16 2005 3 RU builder application FIFOs The RU builder applications use FIFOs to keep track of requests trigger data and event data Knowledge of these FIFOs is required in order to correctly configure the RU builder This chapter is divided into three sections one for the BU one for the EVM and one for the RU Each section gives a brief description of the application s behavior and how its FIFO s are used 3 1 BU FIFOs A BU is responsible for building events An event is composed of one trigger super fragment and N RU super fragments where N is the number of RUs To understand the internal FIFOs of a BU itis first necessary to know its dynamic behavior Figure 2 shows the internal FIFOs of aBU With free capacity available aBU requests the EVM to allocate it an event step 1 The EVM confirms the allocation by sending the BU the event id and trigger data of an event step 2 This trigger data is the first super fragment of the event The BU now requests the RUs to send it the rest of the event s super fragments step 3 The BU builds the super fragments it receives from the RUs step 4 into a whole event within its resource table step 5 FUs can ask a BU to allocate them events step 6 A BU services a FU request by asking the FU to take a whole
27. web pate sirensis eiahead ch caeisedniek ete 31 Fiure 14 Running RU Quer scsi eacteoes etna cis ver eteesantete ee etapa ceed 31 List of Tables Table 1 Exported configuration parameters ccccccccccsssseseeecceecceeeeesseecceeeeseaaeeeeeeeeeeeeeaaas 25 1 Introduction RU Builder User Manual Date 8 16 2005 I Introduction The RU builder is a distributed XDAQ application that is part of a larger system called the event builder EVB The CMS data acquisition group is presently developing the EVB as described in the TriiDAS TDR 1 This document explains how to obtain build and configure version 3 3 of the RU builder The following features are introduced with version 3 3 of the RU builder e Example RUI generates super fragments with FRL and FED headers trailers e Example FU checks FRL and FED headers trailers of RU I super fragments e Bug fix RU builder applications no longer crash their executive when they go to the Failed state 1 1 Document purpose and scope The goal of this document is to enable the reader to integrate the RU builder into a running system composed of the RU builder itself a trigger source one or more event data sources one or more data sinks and some form of run control The RU builder cannot run without the components just listed This document describes how to obtain build and configure the RU builder This document does not describe the other components of a running system such as run
28. y to service their requests for event ids BUs can start requesting event ids as soon as they are enabled Enabling the EVM causes it to send an initial trigger credit count to the TA The number of initial credits is equal to the total number of event ids in the RU builder As soon as trigger data arrives at the EVM from the TA the EVM sends event number event id pairs to the RUs As explained in the previous paragraph this is why the RUs have to be enabled before the EVM Enabling a BU causes it to send its initial request for event ids to the EVM The number of initial event ids requested is equal to the maximum number of event ids the BU is allowed to acquire at any single moment in time DAQ CERN PH CMD 2005 Page 23 of 32 7 Stopping the RU builder RU Builder User Manual Date 8 16 2005 7 Stopping the RU builder The current version of RU builder foresees two ways of stopping the RU builder e Stop the trigger and event data entering the RU builder e Haltall of the RU builder application When stopping the trigger and event data entering the RU builder it is useful to know when the RU builder has finished building the events for which it has received triggers and event data This can be found out by reading the following exported parameter of the EVM EVM ruBuilderIsEmpty Halting a RU builder application causes it to discard destroy all the data in its internal data structures and to release all incoming I20
Download Pdf Manuals
Related Search