Home

Bulk Data Transfer

1. 3 1 2 Receiver The analogue attribute for the Receiver is recv_protocols e g see test CDB alma BulkDataReceiver BulkDataReceiver xml 3 1 3 Distributer For the Dsitributer it is necessary to specify two attributes recv_protocols which specifies the number of flows towards the Sender and sender_protocols which specifies the number of flows towards the Receivers see file test CDB alma BulkDataDistributer xml Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 16 of 44 ALMA Error handling ACS Bulk Data Transfer The ACS Bulk Data Transfer uses local and remote exceptions to notify errors and or abnormal program behaviour Besides re throwing all standard CORBA exceptions the ACSBulkDataError idl ACSBulkDataError Known issue following ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError ACSBulkDataError exceptions are also thrown see file AVFlowEndpointError AVStreamEndpointError AVStreamBindError AVSendFrameError AVCouldNotOpenFile AVObjectNotFound AVInitError AVCallbackError AVProtocolError AVInvalidFlowNumber AVConnectError AVDisconnectError AVStartSendError AV PaceDataError AVStopSend
2. ALMA NNNNN Revision 1 1 2009 04 30 User Manual Roberto Girami ACS Bulk Data Transfer User Manual and How to Manual Roberto Cirami INAF Astronomical Observatory of Trieste Paolo Di Marcantonio INAF Astronomical Observatory of Trieste Owner Roberto Cirami cirami ts astro it Approved by Date Signature ALMA ACS Bulk Data Transfer Change Record VISION pe pomor ECTION PAGE REMARKS 11 11 2003 R Cirami Created Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 2 of 44 ALMA ACS Bulk Data Transfer Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 3 of 44 ALMA ACS Bulk Data Transfer Table of Contents DOVER VIEW cisisessssesccsesssisssssscdcbuocssvessssesesteiedessestecedsetsoissonssbestescscesciseseesebustessabediesaocssessosossasecsssbasoosseses 6 1 1 SYNOPSIS OF COMPONENTS IN THE CORBA A V STREAMING SERVICE eee eene 6 1 2 ACS Bu xk DATA TRANSFER FEATURES cccscesssssssssscscecscscseccececescecesevevevusussusususesssssesssecssscececeseneseueuenes 7 2 DESIGN AND IMPLEMENTATION ccssscsssssscsssscccsssccsssssscsssecccsscccssscccssscccesscccsssecesessssssseeees 7 2A REOUIREMENTS a Sat oh ON e do alten a tone es Beats nc P Drag fs T PPA DIIS EN EE A ETA E A N A E E E A OR 8 2 21 DATE AS Em 2 2 2 Distributer A P O mte edpeeede eite 8 2 3 CLASS DESCRIPTION A d i e ee ti a aaa 9 2 3 1 Sender class Bulk
3. Example 2 A Sender connects to the Distributer sending data on two flows The Distributer in turn sends data on two flows as well Two Receivers connect to the Distributer the first receives data on Flowl the second on Flow2 The two Receivers are implemented in different ways only the first one being a template class see below The Sender performs the following operations on the two flows Flowl An ACE Massage Block of 10 kB is sent from the Sender to the Distributer as parameter Then a Message Block of 14 MB of data is transferred Flow 2 The file lt gt test bulkDataInput txt is read on the Sender side and then transmitted to the Distributer The name of the output file bulkDataOutput txt is sent as a parameter Related files for this example bulkDataDistributerEx cpp bulkDataSenderDistrimplh 1 bulkDataReceiverDistrlImpl h i bulkDataReceiverDistr2Impl h cpp bulkDataReceiverCbDistrl h cpp bulkDataReceiverCbDistr2 h cpp Example 3 This example shows the usage of the Notification mechanism The test is divided in two parts in the first one a Receiver is connected to a Sender through the Distributor The user instantiates an ACS callback and then subscribes to the Distributor passing the reference callback to it The Sender sends some data on two flows and at the end an error is generated on flow one Then a completion is created on the receiver side and passed back to the user callback In t
4. VUE mbl new ACE Message Block size WAS 73 for CORBA Long j 0 j lt size 1 j aides ASIS mo gt woe Se WSS mbl gt wr ptr sizeof char ds cy ESE 79 mbl gt wr_ptr X0 80 mbl wr ptr sizeof char GAS 82 flowNumber 1 83 getSender gt sendData flowNumber mb1 84 85 ACS SHORT LOG LM DEBUG flow 1 length sent data d mbl length 86 87 mbl gt release 88 89 ROR RK kok k k kk k kk k kk k kok k kk k k k k flow E Kk k k Ck kckck ck kck k ck ck k ck kckck ck kckck ck ck ck sk kk ek OE 91 ACE Message Block mb BUFSIZ Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 26 of 44 ALMA ACS Bulk Data Transfer 02 8 JSE 94 DSE 96 OE OE DIO TOOR LOLs MES WOSE 104 TOSS 106 LOPE LORS LOSS TEMOR LELY dag 0 JO O1 SP WwW Wal Sg 1203 112318 162223 125 124 1255 LAGS WAS 12195 1258 ESO 3 ESAS 1328 1331 TES SS SiS 115913 2 SS LD 140 ais 142 143 144 qu 146 TAT PILE A Tp AGB SC EEOD eres uai DU o Continue to send data till the file is read to the end while 1 Read from the file into a message block iste A CHAO Same reb miser Alo tice MOR SZ D PRESTON ae iat lt ON ACS SHORT LOG LM DEBUG BulkDataSenderImpl lt gt paceData sending EES break ae a if feof fp At end of file break the loop and end the clie
5. When the size of the received data is egual to the expected dimension the loop ends Database configuration It is possible to specify the number of flows inside the stream the protocol host and port number for the Sender and the Receiver as well as for the Distributer as attributes inside the CDB Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 15 of 44 ALMA 3 1 1 ACS Bulk Data Transfer Sender For the Sender the directory tes CDB alma BulkDataSender contains the file BulkDataSender xml lt xml version 1 0 encoding UTF 8 a gt lt BulkDataSender xmlns urn schemas cosylab com BulkDataSender 1 0 Example Configuration Database Entry for a BulkDataSender MATEO ROSE History 2004 11 23 oat Created xmlns baci urn schemas cosylab com BACI 1 0 xmlns cdb urn schemas cosylab com CDB 1 0 xmlns xsi http www w3 org 2001 XMLSchema instance sender protocols TCP HOST 14000 TCP TCP HOST 14001 TCP The syntax of the sender protocols attribute following the A V OMG specifications is protocol HOST port protocol HOST port Protocol can be either TCP or UDP and the different flows are separated by In this example there are 4 flows inside the stream If only the protocol is specified a default port on the localhost is assigned Is it also possible to leave the string empty In this case the protocol is TCP by default
6. flow endpoint P EN stream connection gt Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 6 of 44 ALMA 1 2 21 ACS Bulk Data Transfer Fig 1 Main components in the CORBA A V Streaming Service The data stream uses out of band streams external to the GIOP IIOP path of the ORB which can be implemented using communication protocols like TCP UDP etc enhancing in this way the performances of the data transfer ACS Bulk Data Transfer features The ACS Bulk Data Transfer module provides classes and ACS Characteristic Components which implement the features described in the above sections It allows to create dynamically as many flows as required specifying inside the CDB the number of flow endpoints requested see CDB configuration section It allows to connect a Sender component with a Receiver one hiding all the underlying TAO A V complexity and provides the necessary tools to mimic a multicast behaviour a Distributer model which connects multiple receivers to one sender The first sections of the document illustrate the design and the corresponding implementation potential users who only wish to use the Bulk Data can safely skip those sections and read the user manual part of the document Design and Implementation Reguirements Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 7 of 44 ALMA AGS Bulk Data Transfer The basic requirements for the ACS Bulk D
7. 40 sender gt stopsend 41 aa sender gt disconnect 43 receiverl gt closeReceiver 44 45 notifCb1 gt remove ref 46 4T Here we release our components and logout from the manager so ji 1 The user notification callback is included 12 14 The user notification callback is instantiated and activated 16 The callback reference is passed to the Distributor to subscribe to the Notification mechanism 26 The method remove ref must be called on the notification callback pointer in order to correctly decrease the reference counting and have the callback destructor called For the second part of the example Distributor model the situation is similar Below the code for the user notification callback is reported bulkDataTestNotificationCb h The done method is called at the end of the notification process and the completion is logged 1 finclude lt baci h gt 2 3 class BulkDataTestNotificationCb public virtual POA ACS CBvoid 4 OE public SE 18 BulkDataTestNotificationCb S BulkDataTestNotificationCb 95 TOS void working const Completion amp comp const ACS CBDescOut amp desc Tie throw CORBA SystemException T2 Es MS 14 void done const Completion amp comp const ACS CBDescOut amp desc 151 throw CORBA SystemException 16 ce CompletionImpl complImp comp 18 complImp log Ao 205 Dalle CORBA Boolean negotiate ACS TimeInterval tim
8. BulkDataReceiverCbEx1 BulkDataReceiverCbEx1 6 las mono OR Sg Jj Se 10 BulkDataReceiverCbEx1 BulkDataReceiverCbExl DE EE 1293 ACS TRACE BulkDataReceiverCbExl BulkDataReceiverCbExl ERST 14 Se shie 16 BulkDataReceiverCbEx1 cbStart ACE Message Block userParam p dor enr Ta ACS TRACE BulkDataReceiverCbEx1 cbStart 119 83 if flowNumber_m 1 20 8 Zales ACS_SHORT_LOG LM_DEBUG flowname 1 s flowname m c str 2 9 ACS SHORT LOG LM DEBUG length param flowname 1 d userParam p gt length ZB 24 countl m 0 2818 26 else if flowNumber_m 3 ele 28 ACS SHORT LOG LM DEBUG flowname 3 s flowname m c str 290 ACS SHORT LOG LM DEBUG length param flowname 3 Sd userParam p gt length 308 Sil char filename 256 B22 aoe ACE OS strcpy filename userParam p rd ptr 34 39 fp_p ACE_OS fopen filename w SIG 78 SSE return 0 BB 40 41 int 42 BulkDataReceiverCbEx1 cbReceive ACE Message Block PS 44 ACS TRACE BulkDataReceiverCbExl cbReceive 45 46 if flowNumber_m 1 47 48 ACS_SHORT_LOG LM_DEBUG flowname 1 s flowname_m c_str 49 ACS SHORT LOG LM DEBUG length param flowname 1 Sd frame p gt length 505 Sig countl m frame_p gt length 528 BA else if flowNumber m 3 pAr E while frame_p 0 GR Create Date 2004 11 11
9. SINE virtual BulkDataSenderExlImpl PIY 1259 virtual void startSend throw CORBA SystemException AVStartSendErrorEx 1S8 ab bo virtual void paceData throw CORBA SystemException AVPaceDataErrorEx O ISS virtual void stopSend throw CORBA SystemException AVStopSendErrorEx The class BulkDataSenderEx1Impl implements a synchronous Sender so it inherits from BulkDataSenderDefaultImpl which provides the implementation for the connect and disconnect methods BulkDataSenderEx1Impl must provide the implementation for the three methods startSend paceData and stopSend which are pure virtual in the base class BulkDataSenderDefaultImpl The implementation is provided in the file bulkDataSenderEx1Impl cpp error handling is not shown include bulkDataSenderExlImpl h 3 BulkDataSenderExlImpl BulkDataSenderEx1Impl const ACE CString amp name KO O UO U MA LOE RO ContainerServices containerServices BulkDataSenderDefaultImpl name containerServices ACS TRACE BulkDataSenderExlImpl BulkDataSenderExlImpl BulkDataSenderExlImpl BulkDataSenderExlImpl ACS TRACE BulkDataSenderExlImpl BulkDataSenderExlImpl 13 void BulkDataSenderExlImpl startSend throw CORBA SystemException AVStartSendErrorEx MEZ SK 1159 ACS TRACE BulkDataSenderExlImpl startSend 16 dug int size dL 19 ROR KR KK k kok Ck k kk k kk ke kk k k k k k k k flow JL KCKCkCkCkCk ck
10. k ck k ck kckckck ck kck ck ck kck ck ck k k ke ke ke x ADE zvl size 10000 2E 23 ACE Message Block mb1 24 mbl new ACE Message Block size DIE 26 for CORBA Long j 0 j lt size 1 j es 287 mbl wr ptr p 29 mbl gt wr ptr sizeof char Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 25 of 44 ALMA ACS Bulk Data Transfer 32 mbl wr ptr 2 N0 SBK mbl gt wr ptr sizeof char 34 353 CORBA ULong flowNumber 1 SIS getSender gt startSend flowNumber mb1 SE Sige ACS_SHORT_LOG LM_DEBUG flow 1 length start parameter sent d mbl length SO 40 mbl gt release 41 42 BOR RK KK KK k k k k k k k k k k k k k k k k k k flow 3 E AA 43 44 size 256 45 46 char fileName size 47 48 ACE OS strcpy fileName bulkDataOutput txt 49 50 const char ptr fileName SLS S flowNumber 3 SES getSender gt startSend flowNumber ptr size DA DS ACS SHORT LOG LM_DEBUG flow 3 start parameter sent s fileName SOE Sys Biers 59 void BulkDataSenderExlImpl paceData throw CORBA SystemException AVPaceDataErrorEx Coe A Gale ACS TRACE BulkDataSenderImpl paceData prx 638 int sraej 64 CORBA ULong flowNumber 65 66 KKK KR KK kA k k ke k k k k k k k e ke flow T E AA G5 68 size 14000000 GIO 03 ACE Message Block mb1
11. shown Create Date KO O U OU MN WN O UKB UN F Oe BulkDataSenderThreadImpl BulkDataSenderThreadImpl const ACE CString amp name ContainerServices containerServices BulkDataSenderDefaultImpl name containerServices ACS TRACE BulkDataSenderThreadImpl BulkDataSenderThreadImpl numberOfFlows 4 CORBA ULong flow for CORBA ULong i 0 i lt numberOfFlows i String str VELO flow og cis stringstream out OU Ue c fI oW Sige Gicie we obio ace ACERO SI Ss AS CES a P SenderThread thread new SenderThread strl this flow ThreadBase defaultResponseTime ThreadBase defaultSleepTime 10 1s thread p push back thread 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 36 of 44 ALMA AGS Bulk Data Transfer 20 5 E 22 void BulkDataSenderThreadImpl paceData throw CORBA SystemException AVPaceDataErrorEx 25 ff 24 ACS TRACE BulkDataSenderThreadImpl paceData Eg 26 for CORBA ULong i 0 i lt numberOfFlows i QE 283 thread_p i gt resume ZB 30 ACE Time Value tv 20 31 BACI CORBA getORB run tv Sea PBE 34 35 void SenderThread run Soo af Aq Low 38 Boe sender p gt getSender gt sendData flowNumber_m mb p 40 ACS SHORT LOG LM INFO flow d length sent data d flowNumber m mb p gt length 41 AMO ECON 43 44 ACS SHORT LOG LM ERROR Error 45 46 se
12. three methods described above In particular he she must call e getSender gt startSend in startSend getSender returns the private member BulkDataSender of the class BulkDataSenderImpl e getSender sendData in paceData e getSender stopSend in stopSend Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 10 of 44 ALMA AGS Bulk Data Transfer Fig 4 shows the class diagram for the idl interfaces for different senders For a complete example see the User Manual section and in particular Example 1 files bulkDataSenderEx1 idl and bulkDataSenderEx1Impl h cpp ACS CharacteristicComponent bulkdata BulkData Sender connect receiver bulkdata BulkDataReceiver void disconnect void startSend void paceData void stopSend void N A Bulkdata BulkDataSenderEx1 Bulkdata BulkDataSenderEx2 MyBulkDataSender M Fig 4 Class diagram for user Sender idl interfaces The Sender can send data either in a synchronous or in an asynchronous way see section 2 2 To allow the user to send asynchronous data BulkDataSenderImpl lt TSenderCallback gt is a template class The class which implements a Sender callback must inherit from TAO AV Callback and implement the get timeout ACE Time Value amp timeout void sargs and the handle timeout methods The TAO A V always calls the get timeout method inside the class that implements the
13. 0 261 If the callback is not destroyed in the handle_destroy method then we have a memory leak If we try to destroy the callback later for example in the destructor of the bulkDataFlowConsumer then we have a crash of the whole system because the TAO AV TCP Object object which owns the callback was already destroyed Shortly we spent really lot of time to figure out all this behavior and the only solution seems to clean up things by using the delete this idiom Note however that the users are never supposed to instantiate callback object by themselves and therefore this should not give any major problem 6 1 4 Remarks on the Makefile In order to be able to compile correctly the code for the A V the following lines must be added to the standard ACS Makefile e USER INC IS TAO ROOTYorbsvcs orbsvcs AV e USER LIB ITAO AV It is necessary to explicitly add TAO ROOT orbsvcs orbsvcs AV to the include path because some include files in the AV directory assume that I option is NOT used and therefore do not prepend AV to the file name when including it 7 Appendix A Hand shake mechanism Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 38 of 44 ALMA ACS Bulk Data Transfer Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 39 of 44 ALMA AGS Bulk Data Transfer In the TAO A V Streaming Service the Sender Receiver architecture is implemented by using the ACE Reactor pattern and use
14. 1 ITReceiverCallback I ITSenderCallback BulkDataSenderDefaultCallback BulkDataDistributerlm pl subscribeNotification notifCb CBvoid void y ITReceiverCallback TSenderCallback BulkDataDistributer locNotifCb_p CBvoid subscribeNotification notifCb CBvoid void notifySender completion const Com pletion void y TReceiverCallback ITSenderCallback BulkDataSenderDefaultCallback BulkDataDistributerNotifCb distr_p BulkDataDistributerlm pl CBvoid working comp const Completion amp desc const CBDescOut amp void negotiate timeToTransmit Timelnterval desc const CBDescOut amp Boolean done comp const Completion amp desc const CBDescOut amp void Fig 11 Class diagram for the Distributor implementation Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 19 of 44 ALMA ACS Bulk Data Transfer Also the BulkDataDistributerImpl class implements the subscribeNotification method delegating it to the BulkDataDistributer class fig 11 This method calls automatically the subscribeNotification method of all the receivers connected to the Distributor passing a reference of the Distributor Notification callback BulkDataDistributerNotifCb to them Similarly to the receiver also the BulkDataDistributer class implements the notifySender meth
15. Author Roberto Cirami ACS_Bulk_Data_Transfer Page 29 of 44 ALMA 16 26 36 42 53 66 72 85 88 ACS Bulk Data Transfer OR int result ACE OS fwrite frame p gt rd ptr SSE frame p gt length 1 fp p DIO if result 0 60 Gules ACS_SHORT_LOG LM_ERROR BulkDataReceiverCbEx1 cbR eceive failed 62 return 6338 64 frame_p frame_p gt cont 59 66 97 8 68 return 0 5981 4 OS 71 int 72 BulkDataReceiverCbEx1 cbStop SEDET 74 ACS TRACE BulkDataReceiverCbExl cbStop 138 GI if flowNumber m 1 TES ACS SHORT LOG LM INFO flow 1 total length d counti m SE eo if flowNumber m 3 80 ACE OS fclose fp p Silo 28 ge tum so Jj 84 85 MACI DLL support functions T IA L EN IT 86 include maciACSComponentDefines h 87 MACI DLL SUPPORT FUNCTIONS BulkDataReceiverImpl BulkDataRec eiverCbEx1 gt The cbStart method is called automatically in response to the Sender startSend call For flow 1 a counter is initialized for statistics purposes For flow 3 the name of the output file is extracted and the file is opened The cbReceive method is called automatically in response to the Sender paceData call For flow 1 the counter variable is incremented by the length of the data received in order to calculate the total amount of data received For flow 3 the content of the input file is extra
16. DataSenderImpl TSenderCallback eee 9 2 3 2 Receiver class BulkDataReceiverImpl TCallback eene 12 2 3 3 Distributer class BulkDataDistributer TReceiverCallback TSenderCallback 13 VECEA MYTH 15 3 DATABASE CONFIGURAT ION eere ee eee eoo een tos e one see to ness eene e ese 00 ee eo 000000000 e 000000006 e esses 15 3 Senders snas cus Omen tern 16 3 12 AA A nna durae A EE 16 313 DIS DUIeE a ita 16 4 ERROR HANDLING c con cian 17 5 NOTIFICATION MECHANISM eee ee eee ene en netten esee eno 00006 Ko e toes ease seen ee senes ee ene e etna esee eS 18 6 USER MANUA Y oec 20 6 1 ACS BuLk DATA TRANSFER EXAMPLES oocococononononnnnonononcnnnnnnncnnncnonononononanananana no nro no e enn enses aser ee esee nun 20 01 1 How to ru the examples s Tee REED 22 6 1 2 How to customize the examples PEE 23 For the second part of the example Distributor model the situation is similar 33 6 1 3 Remarks on callback deletion eene eene n nnn nnne nsns es ense ener nen 37 6 1 4 Remarks on the Makefile esses eese eene eene aes nnne enne tenet nennen ene een nen 38 7 APPENDIX A HAND SHAKE MECHANI SM eee ee eee eese toe tae eese ene seen eset see 0000000000000 oo 38 TT TIMBOUPEPROBEEM A e 44 S R
17. EEERENCGES e irer eost sveieovshsev sends ob suscsesdebcideusetesubdeessvesvssantessbvesssnsseasedssotesesas 44 Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 4 of 44 ALMA ACS Bulk Data Transfer Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 5 of 44 ALMA 1 1 ACS Bulk Data Transfer Overview The ACS Bulk Data Transfer allows to transfer huge amounts of data between two ACS Components bypassing the ORB and using an out of bound connection for enhancing performances In this way the achieved throughput is of the order of 800 Mbits sec on a 1 Gbit Ethernet network see ref 1 The ACS Bulk Data Transfer is based on the TAO Audio Video Streaming Service which in turn implements the OMG CORBA Audio Video Streaming Service specifications This document describes the ACS Bulk Data Transfer and shows how to use it Synopsis of components in the CORBA A V Streaming Service The CORBA A V Streaming Service specification defines flows as a continuous transfer of media between two multimedia devices Each of these flows is terminated by a flow endpoint A set of flows such as audio flow video flow and data flow constitute a stream which is terminated by a stream endpoint A stream endpoint can have multiple flow endpoints One flow endpoint acts as a source of the data and the other flow endpoint acts as a sink see Fig 1 stream endpoint A flow connection
18. Error AVOpenReceiverError AVReceiverConfigError AVCloseReceiverError At present it is not possible to throw exceptions from the receiver callbacks Moreover the receiver callbacks should not return negative values Create Date 2004 11 11 ACS Bulk Data Transfer Author Roberto Cirami Page 17 of 44 ALMA ACS Bulk Data Transfer 5 Notification mechanism In order to inform the Sender when something happens on one or more Receivers the Bulk Data provides a Notification mechanism In this way a Receiver can send information about problems to the Sender or the Distributor using a callback mechanism If the user wants to use the Notification mechanism in the point to point model he instantiates a callback and passes its reference to the receiver subscription method If something happens on the receiver side the receiver callback notifies the receiver with an ACS completion containing all the necessary information Then the receiver automatically calls the user callback passing back the completion In the distributor model the user passes the reference of the instantiated callback to the Distributor Then the Distributor automatically instantiates its own callback and passes its references to all the connected receivers In case of error the receiver callback notifies the Distributor passing a completion to it The Distributor in turn calls its own callback which notifies the Sender with the completion What to do as a result o
19. OVvi d dp HMi9ureens OVL d purge ds Author Roberto Cirami Page 43 of 44 ACS Bulk Data Transfer Create Date 2004 11 11 ALMA 7 1 8 ACS Bulk Data Transfer Fig 13 Sequence diagram for the hand shake mechanism Timeout problem A timeout problem can occur inside the hand shake mechanism if the execution time of the receiver callback method cbReceive takes too much time As described above when there are data available in the receiver buffer the ACE Reactor calls the hook method of the handler registered for receiving the data not visible to the user This hook method calls the receive_frame method of the receiver callback see point 23 in fig 10 which in turn calls coReceive point 24 in fig 10 The return value of the hook method is checked by the ACE Reactor If it is greater or equal to 0 the Reactor continues dispatching data calling the hook method again If the return value is 1 for example when the connection is closed the receiver handler is automatically removed from the Reactor freeing the resources If cbReceive blocks for a long time before returning point 24 in fig 10 the Sender calls stop point 25 in fig 10 and then continues closing the connection In the meantime the timeout inside the handle stop expires point 25 in fig 10 and point e in the description of the hand shake mechanism above and the Receiver continues closing the connection and deleting the resources After that
20. Sender callback If the timeout parameter is zero it means that the Sender callback is not used If the value of timeout is different from zero the TAO A V calls the handle timeout method of the Sender callback every timeout miscroseconds Inside the handle timeout method the user must implement the sending of the data for example calling getSender gt startSend getSender sendData etc The BulkDataSenderImpl lt TSenderCallback gt class is provided with a default template parameter which represents a Sender callback with a zero timeout In this way the Sender sends its data synchronously Via a typedef this default sender class is called BulkDataSenderDefaultImpl In this way the user who wants to inherit from this class can simply do the following class UserSenderImpl public virtual BulkDataSenderDefaultImpl i whereas in the first case he she had to specify the default sender callback explicitly Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 11 of 44 ALMA ACS Bulk Data Transfer class UserSenderImpl public virtual BulkDataSenderImpl BulkDataSenderDefaultCallback 2 3 2 Receiver class BulkDataReceiverlmpl lt TCallback gt Fig 5 and Fig 6 show respectively the relationship between the idl interfaces and the classes which form the Receiver ACS CharacteristicComponent A bulkdata BulkDataR eceiver E 1 openReceiver void closeReceive
21. ata Transfer are e Point to point data transfer Data are sent from a Sender to a Receiver using an out of bound high performance connection e Multicast data transfer A Sender sends data to a Distributer which is responsible to deliver the data to one on more Receivers registered into it 2 2 Design The ACS Bulk Data Transfer provides a wrapper and an adaptation of the CORBA A V Streaming Service in the TAO implementation to ACS hiding from the user most of its complexity In order to fulfill the above reguirements C classes have been created which act as wrappers around the TAO A V Streaming Service and expose the functionalities of the TAO A V with a simplified interface at present only C implementation is provided At a higher level ACS Characteristic Component objects are provided which contain and use these C classes providing the developer with user friendly programming interfaces These interfaces allow the user to create a stream and add to it as many flows as needed establishing a connection between the Sender and the Receiver objects After that the Sender can start sending data to the Receiver and then when all the data are sent close the connection 2 2 Point to point Following the CORBA A V Streaming Service specifications the Sender can send data in two possible ways either asynchronously or synchronously In the asynchronous case the data are sent inside a callback called by the TAO A V with a timin
22. cbReceive returns receive frame returns 0 to the hook method which in turn returns 0 to the ACE Reactor which calls again the hook method The hook method calls receive frame but the receiver callback does not exist any more because in the meantime all the resources have been deallocated and the system crashes At present this problem is under investigation References 1 P Di Marcantonio R Cirami et al Transmitting huge amounts of data design implementation and performance of the Bulk Data Transfer mechanism in ALMA ACS ICALEPCS 2005 Geneva Switzerland October 2005 Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 44 of 44
23. ceiver File bulkDataReceiverDistr1Impl h class BulkDataReceiverDistr2Impl public virtual BulkDataReceiverImpl lt BulkDataReceiverCbDistr2 gt public virtual POA bulkdatadistr BulkDataReceiverDistr2 The class BulkDataReceiverDistr2Impl is not a template and the callback is passed directly as template parameter of the base class File bulkDataReceiverDistr2Impl cpp include lt maciACSComponentDefines h gt MACI DLL SUPPORT FUNCTIONS BulkDataReceiverDistr2Impl In this case the class BulkDataReceiverDistr2Impl is not more a template and therefore is instantiated in the usual way Known issue Also in this case the disconnection order is important for the correct shutdown of the connection The two Receivers must be disconnected from the Distributer after the Sender Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 31 of 44 ALMA ACS Bulk Data Transfer 1 sender gt disconnect 2 distributer gt closeReceiver 3 distributer multiDisconnect receiverl in 4 distributer multiDisconnect receiver2 in Example 3 Notification mechanism This example shows the usage of the Notification mechanism Only the relevant part of the code is shown in the listings below The test executable is bulkDataNotificationTest cpp The test is divided in two parts to show the Notification mechanism usage in the distributor and point to point models respectively After getting the referenc
24. cted and saved inside the output file The cbStop method is called automatically in response to the Sender stopSend call For flow 1 the statistic is printed For flow 3 the output file is closed The Receiver object is instantiated passing as template parameter the receiver callback just implemented Example 2 Distributer Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 30 of 44 ALMA AGS Bulk Data Transfer Two different ways of implementing the Receivers are used in this example Only the first Receiver is a template The following code shows the relevant part of the implementation of the first receiver File bulkDataReceiverDistr1Impl h template class TCallback class BulkDataReceiverDistrlImpl public virtual BulkDataReceiverImpl lt TCallback gt public virtual POA bulkdatadistr BulkDataReceiverDistrl The class BulkDataReceiverDistrlImpl is a template class taking the callback as template parameter This callback is also passed as parameter to the template base class BulkDataReceiverImpl File bulkDataReceiverDistr1Impl cpp include lt maciACSComponentDefines h gt MACI DLL SUPPORT FUNCTIONS BulkDataReceiverDistrlImpl lt BulkDataReceiverCbDistrl gt The specific callback is passed when the class BulkDataReceiverDistrlImpl is istantiated inside the file containing the callback implementation The following code shows the relevant part of the implementation of the second re
25. ctor concrete event handler is the receive frame method as described before The limitation is that internally the TAO A V reads data only in chunks of 8192 bytes It could happen therefore that the Sender receives the acknowledgement of the last frame received even if the data are still not fully Fig 12 Receiver callback class diagram consumed on the Receiver side they are actually stored in the host TCP receive buffer but are not read yet In this case a stop could be issued to early spoiling the last part of the received stream In order to overcome this problem a hand shake protocol on top of this architecture has been implemented by inheriting from the TAO AV Callback as shown in fig 12 and adding internally a new state management see below Before sending the raw data a control frame is sent and analyzed by the BulkDataCallback class The control frame contains the information an ID on whether the forthcoming stream is a parameter or the bulk of data and the number of expected bytes length The ID allows to call internally the appropriate methods cbStart ACE Message Block param cbReceive ACE Message Block frame to distinguish between parameters and data whereas the bytes length information permits to manage and overcome the synchronization problem The hand shake mechanism Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 40 of 44 ALMA ACS Bulk Data Transfer checks
26. d used for starting the transfer and sending parameters It calls automatically cbStart on the 5 Receiver callback 35 Method used for sending the data It calls automatically cbReceiver on the Receiver callback 37 Method used for ending the data transfer It calls automatically cbStop on the Receiver callback 39 41 The Sender and the Receiver close the connection Known issue At present the disconnection order of the Sender and Receiver is important The user must call sender disconnect before receiver gt closeReceiver in order to close the connection properly Sender The first step for creating a Sender component is to create an idl interface for it which inherits from the BulkDataSender idl interface file bulkDataSenderEx1 idl In this new idl interface the user can add his owns methods for the Sender component Then it is necessary to create a class which implements this idl interface For example 1 this class is declared inside the file bulkDataSenderEx1Impl h include bulkDataSenderEx1S h include bulkDataSenderImpl h SS UN H class BulkDataSenderExlImpl public virtual BulkDataSenderDefaultImpl public virtual POA bulkdata BulkDataSenderEx1 Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 24 of 44 y ALMA ACS Bulk Data Transfer 6 publliler Te 8 BulkDataSenderExlImpl const ACE CString amp name ContainerServices containerServices oS
27. de At this point the OS sends an acknowledgment to the Sender which can send the next chunk of data When there are data available in the receiver buffer the ACE Reactor calls the receiver hook method which reads chunks of 8192 KB of data from the buffer The ACE Reactor calls consecutively this hook method until there are data in the buffer Due to this asynchronous behaviour it could happen that the Sender receives the last acknowledgment before the data are actually processed by the user on the receiver side causing in this way a possible data loss To solve this problem an ad hoc state management has been implemented inside the base receiver callback class BulkDataCallback which allows to wait automatically for the data to be processed before closing the connection hand shake mechanism see Appendix A The hand shake mechanism sends the dimension of the transmitted data to the Receiver in a way transparent to the user Inside the receiver callback there is a loop and before each loop iteration the amount of the received data 1s compared to their expected dimension Inside the loop there is a sleep which waits for a fixed amount of time before the next iteration Two methods inside the BulkDataCallback class allow the user to set the timeout e void setSleepTime ACE Time Value locWaitPeriod sets the amount of time to wait in each loop iteration e void setSafeTimeout CORBA ULong locLoop sets the number of loop iterations
28. de the while loop the file bulkDatalnput txt is read saved into a Message Block and then sent on flow 3 stopSend method Inside this method the transmission is stopped for each flow To create the Receiver it is necessary to provide a Receiver callback to the class BulkDataReceiverlmpl lt gt For example 1 the callback class is declared inside the file bulkDataReceiverCbEx1 h aks include bulkDataCallback h Ze include bulkDataReceiverImpl h 3g 4 class BulkDataReceiverCbEx1 public BulkDataCallback 58 6 publics ES BulkDataReceiverCbEx1 Se o BulkDataReceiverCbEx1 moz Lis virtual int cbStart ACE_Message_Block userParam_p 0 128 ISE virtual int cbReceive ACE_Message_Block frame_p 14 TS virtual int cbStop LOE 17 private Lere akej CORBA ULong countl m 207 AIE CORBA ULong count2_m 22 PILE Samer pi y The callback class must inherit from BulkDataCallback and implement the three methods cbStart cbReceive and cbStop To distibguish between the different flows the internal variable flowNumber m is used as shown in the example The class is implemented inside the file bulkDataReceiverCbEx1 cpp Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 28 of 44 ALMA ACS Bulk Data Transfer 1 include bulkDataReceiverCbExl h 2 3 BulkDataReceiverCbExl BulkDataReceiverCbEx1 countl m 0 count2 m 0 4 58 ACS_TRACE
29. eToTransmit const ACS CBDescOut amp desc 228 throw CORBA SystemException PEE if 24 return true Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 33 of 44 ALMA ACS Bulk Data Transfer In order to notify the user when something happens on the receiver side the receiver itself must create a completion and notify the Distributor or the Sender This is done in the receiver callback and is shown in the listing below file bulkDataReceiverCbNotif cpp only the relevant code is shown MD 00 11 O 00 45 20 BO 9 10 SS UNA O KO 0 UO U Zale 2A 2439 24 23 PASE Zu Dio DIO SIDE Silky SIR BBE 34 B9 SS Sg 38 29 40 41 42 include bulkDataReceiverCbNotif h int BulkDataReceiverCbNotif cbStop Ery if flowNumber_m 1 AVFlowEndpointErrorExImpl err AVFlowEndpointErrorExImpl FILE LINE BulkDataReceiverCbNotif cbS top throw err else if flowNumber m 2 ACS_SHORT_LOG LM_INFO BulkDataReceiverCbNotif data on lo NEC ots ceres cptebi e Cone Zan RR catch ACSErr ACSbaseExImpl sex AVFlowEndpointErrorCompletion comp ex __FILE__ __LINE__ BulkDataReceiverCbNotif cbStop comp log if recv p recv_p gt notifySender comp else ACS_SHORT_LOG LM_ERROR BulkDataReceiverCbNotif Receiver reference NULL return 0 void BulkDataReceiverCbNotif setReceiv
30. er AcsBulkdata BulkDataReceiver lt BulkDa taReceiverCbNotif gt recv LE eO CV NUBE ACS SHORT LOG LM ERROR BulkDataReceiverCbNotif setReceiver ra Caner P else TO EXCAWC P ASO An error is generated and thrown for test purposes inside the cbStop method Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 34 of 44 ALMA ACS Bulk Data Transfer 19 In the catch block a completion is created and the error added to it 23 The receiver is notified of the error and the completion is passed to it The method setReceiver must be implemented in order to set the receiver reference inside the receiver 34 callback Example 4 Using threads to send data This example shows how to use different threads to send data on different flows Only the relevant part of the code is shown in the listings below The following code shows the file bulkDataSenderThreadImpl h where the thread class and the sender component class are declared tinclude bulkDataSenderDistrS h tinclude bulkDataSenderImpl h tinclude bulkDataSenderDefaultCb h tinclude lt acsThread h gt include lt vector gt forward declaration class BulkDataSenderThreadImpl KOD Sed Oui aus UND EA F using namespace ACS class SenderThread public ACS Thread Pubic WR SenderThread const ACE_CString amp name BulkDataSenderThreadImpl sender CORBA ULong flowNumber const ACS TimeInterval amp res
31. es to the Sender Distributor and two Receivers not shown the connections are established Then the subscription to the Notification mechanism is done and the data transfer begins At the end all the components are released not shown 1 finclude bulkDataTestNotificationCb h 25 Sig aloe na amo usse Clase usen bo 4 Bes Here we instantiate our components 8 TER First part of the example distributor model ga Ste sender gt connect distributer in Toa distributer gt multiConnect receiver in FI 124 BulkDataTestNotificationCb notifCb new BulkDataTestNotificationCb T3 14 ACS CBvoid var cb notifCb gt this 1158 16 distributer subscribeNotification cb Trt Toz sender gt startSend 193 sender paceData 20x sender gt stopSend DES 22 sender gt disconnect 23 distributer gt closeReceiver 24 distributer gt multiDisconnect receiver in 25i 26 notifCb remove ref Ques 28 Second part of the example point to point model 2 0 39 sender gt connect receiverl in SLA 32 BulkDataTestNotificationCb notifCbl new BulkDataTestNotificationCb 30 34 ACS CBvoid var cbl notifCb1 gt this 55 0 36 receiverl subscribeNotification cb1 Siga Seg sender startSend Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 32 of 44 ALMA ACS Bulk Data Transfer on sender paceData
32. f receiving a notification is entirely up to the Sender stop sending the data continue to send etc BulkDataSender O BulkDataReceiver O subscribeNotification notifCb CBvoid void R A BulkData ReceiverDistr 5 BulkDataDistribute rO Fig 9 Class diagram showing the relationship between the Receiver and Distributor 1dl interfaces Fig 9 shows the interface relationship between the Receiver and Distributor The subscribeNotification method of the BulkDataReceiver interface is used for subscribing to the Notification mechanism This method is inherited by the BulkDataDistributer class Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 18 of 44 ALMA ACS Bulk Data Transfer The class BulkDataReceiverImpl implements the subscribeNotification method delegating its implementation to the BulkDataReceiver class fig 10 This class implements also the notifySender method necessary to notify the Sender when something happens on the receiver side r TCallback ITReceiverCallback e P ee RM i l BulkDataReceiverlm pl BulkDataReceiver M locNotifCb_p CBvoid subscribeNotification notifCb CBvoid void subscribeNotification notifCb CBvoid void notifySender completion const Completion amp void Fig 10 Class diagram for the Receiver implementation
33. form the Sender ACS CharacteristicComponent bulk data BulkDataS ender connect receiver bulkdata BulkDataReceiver void disconnect void startSend void paceData void stopSend void Fig 2 Class diagram for Sender idl interfaces Create Date 2004 11 11 ACS Bulk Data Transfer Page 9 of 44 Author Roberto Cirami ALMA ACS Bulk Data Transfer POA_bulkdata BulkDataSender connect receiver BulkDataReceiver void disconnect void CharacteristicComponentImpl startSend void paceData void N stopSend void jTSenderCallback BulkDataSenderDefaultCallback TSenderCallback BulkDataSenderlmpl BulkDataSender connect receiver BulkDataReceiver void startSend flownumber ULong param ACE Message Block 0 void disconnect void bee startSend flownumber ULong param const char len size t void startSend void sendData flownumber ULong buffer ACE Message Block void paceData void sendData flownumber ULong buffer const char len size t void stopSend void stopSend flownumber ULong void Fig 3 Class diagram for the Sender implementation BulkDataSenderImpl TSenderCallback is an abstract class which provides the implementation for the methods connect and disconnect The connect method is responsible for the c
34. g defined by the user The synchronous case is blocking the data are sent directly inside the sender implementation code without the use of a callback mechanism The Receiver can receive data only in an asynchronous way by means of a callback called automatically by the TAO A V To reflect this behavior inside the ACS Bulk Data Transfer the ACS Characteristic Component objects relative to the Sender and the Receiver are implemented as template classes allowing in this way to specify the appropriate callbacks as template parameters 2 2 2 Distributer Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 8 of 44 ALMA ACS Bulk Data Transfer Besides a point to point communication model the ACS Bulk Data Transfer provides a multicast model Since the TCP protocol does not support multicasting a distributer mechanism has been developed The ACS Bulk Data Distributer mimics a multicast behaviour receiving data from one Sender and dispatching them to one or more registered Receivers The next subsections describe the architecture and the classes of the ACS Bulk Data Transfer We describe in detail only the usage of the ACS Characteristic Component objects For a complete description of the C classes see the online documentation 2 3 Class description 2 31 Sender class BulkDataSenderlmpl lt TSenderCallback gt Fig 2 and Fig 3 show respectively the relationship between the idl interfaces and the classes which
35. he Distributer It calls the openReceiver method of the Receiver passed as parameter and then calls the multiConnect method of the BulkDataDistributer object contained inside the Distributer This method creates a new Sender internal to the Distributer and connects it to the Receiver When the Distributer receives parameters or data from the Sender its receiving callback delegates to the contained BulkDataDistributer class the task of forwarding the incoming parameters and data to the connected Receivers via the methods distSendStart and distSendData distSendStop is called whenever the Sender issues a stopSend During the disconnection phase the closeReceiver method of the Distributer disconnects the Distributer itself from the Sender whereas the multiDisconnect method disconnects the Distributer from the specified Receiver Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 14 of 44 ALMA 2 3 4 ACS Bulk Data Transfer The Distributer class is a template class with two template parameters One is the Sender callback which takes a default value see Sender section 2 3 1 and the other is the callback used to receive the data from the Sender and dispatch them to all registered Receivers Timeout Inside the TAO Audio Video Streaming Service the Sender Receiver architecture is implemented using the ACE Reactor Pattern Whenever the Sender sends some data they are stored in an internal OS buffer on the receiver si
36. he second part of the example the same notification mechanism is shown with a Receiver connected directly to the Sender point to point model Related files for this example bulkDataNotificationTest cpp bulkDataTestNotificationCb h bulkDataSenderNotifImpl h cpp bulkDataReceiverCbNotif h cpp bulkDataReceiverl CbNotif h cpp Example 4 This example shows how to use different threads to send data A Create Date 2004 11 11 Sender connects to the Distributer sending data on four flows The Distributer in turn sends the data to a Receiver The Sender component instantiates four different threads The same Sender object is shared between the four threads and each thread sends data on one flow sending respectively 10 20 30 and 40 MB Author Roberto Cirami ACS Bulk Data Transfer Page 21 of 44 ALMA AGS Bulk Data Transfer Related files for this example bulkDataThreadTest cpp bulkDataSenderThreadImpl h cpp bulkDataReceiverCbThread h cpp 6 1 1 How to run the examples Fist it is necessary to perform the following operations e export ACS CDB lt gt bulkData test e Inside the file bulkData tes CDB MACI Components Components xml uncomment the code referring to the example you want to run and comment all the rest e acsStart Then the paths to follow are different for each example Example 1 i acsStartContainer cpp ContainerEx1 in lt gt bulkData test the directory where the file bulkDatal
37. id lt 7 TReceiverCallback TSenderCallback BulkDataSender DefaultCallback I TReceiverCallback BulkDataDistributerlmpl TSenderCallback connect receiver BulkDataReceiver void BulkDataDistributer multiConnect receiver BulkDataReceiver void multiConnect recvName const ACE CString amp void disconnect void multiDisconnect recvName const ACE CString amp void multiDisconnect receiver BulkDataReceiver A distSendStart flowName ACE CString amp void openReceiver void closeReceiver void setReceiver recvConf const BulkDataReceiverConfig amp void distSendData flowName ACE CString amp frame ACE Message Block int distSendStop flowName ACE CsString amp void Fig 8 Class diagram for the Distributer implementation The Distributer acts as a Receiver towards the Sender and as a Sender towards the Receivers willing to get the dispatched data So it implements both the Sender and the Receiver interfaces The interface BulkDataReceiverDistr is added for internal Distributer purposes The Sender connects to the Distributer in the usual way with the connect method passing the Distributer as parameter The connect method in turn calls the openReceiver method on the Distributer see Sender section 2 3 1 The Distributer multiConnect method allows the Receivers to register themselves into t
38. ie mena nekam ars EA S SE 4 Creates and initializes the SimpleClient object 5 SimpleClient client 6 if client init argc argv 0 7 Sr Suevia ip OE AOE else 2i 22 Must log into manager before we can really do anything 25 ciento 24 25 26 Get the specific component we have requested on the command line 2s bulkdata BulkDataReceiver var receiver client get object bulkdata BulkDataReceiver BulkDataReceiver OP ELE 29 293 bulkdata BulkDataSender var sender client get_object lt bulkdata BulkDataSender gt BulkDataSenderEx1 r Erue SOs Sales sender gt connect receiver in 322 SIS sender gt startSend 34 B57 sender gt paceData Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 23 of 44 ALMA ACS Bulk Data Transfer 3V3 sender gt stopSend 38 aoe sender gt disconnect 40 LS receiver gt closeReceiver 42 43 We release our component and logout from manager 44 client manager release component client handle BulkDataSenderEx1 45 ACS SHORT LOG LM INFO Sleeping 3 sec to allow everything to cleanup and stabilize 46 47 ACE OS sleep 3 48 49 client manager release component client handle BulkDataReceiver 50 Sig client logout 525 SE return 0 5 6 idl generated files for the Sender and the Receiver 27 29 Get references to the Receiver and Sender components Metho
39. internally if the dimension of the received data is equal to the expected data length and waits until all data have been received see below The hand shake mechanism described above is completely hidden from the user To receive fully synchronized parameters data she he must only inherit from BulkDataCallback and implement the three abstract methods see Fig 12 without knowing anything about what happens below The hand shake mechanism introduces some performance penalties in the data throughput bout only of the order of 0 5 see ref 1 Fig 13 shows the sequence diagram for the hand shake mechanism when the user issues the commands startSend paceData stopSend From the main program the user calls startsend on the Sender point 1 in fig 13 calling in turn startSend flwn param on the BulkDataSender object point 2 passing as parameters the flownumber and the parameter The implementation of this last method is hidden from the user Inside this method the hand shake mechanism is implemented consisted of five sequent method calls a streamctrl p start This calls the start method of the TAO StreamCtrl object which is a private member of the class BulkDataSender This method calls asynchronously the handle start method of the BulkDataCallback setting its internal state b dp p gt send frame 1 dim This calls the send frame method of the TAO AV Protocol Object object private member of the class BulkDataSe
40. matically by the TAO A V when the Sender calls startSend cbReceive when it calls paceData and cbstop when it calls stopSend 2 3 3 Distributer class BulkDataDistributer lt TReceiverCallback TSenderCallback gt Fig 7 and Fig 8 show respectively the relationship between the idl interfaces and the classes which form the Distributer bulkdata BulkDataReceiver openReceiver void closeReceiver void bulkdata BulkDataSender connect receiver bulkdata BulkDataReceiver void disconnect void startSend void bulkdata BulkDataR eceiverDistr paceData void stopSend void setReceiver recvConf BulkDataReceiverContig void bulkdata BulkDataDistributer multiConnect receiver BulkDataReceiver void multiDisconnect receiver BulkDataReceiver void Fig 7 Class diagram for Distributer idl interfaces Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 13 of 44 ALMA ACS Bulk Data Transfer POA_bulkdata BulkDataDistributer connect receiver BulkDataReceiver void multiConnect receiver BulkDataReceiver void disconnect void CharacteristicComponentimpl multiDisconnect receiver BulkDataReceiver void openReceiver void closeReceiver void setReceiver recvConf BulkDataReceiverConfig amp void startSend void paceData void stopSend vo
41. nder the first parameter 1 indicates that the data that are going to be received are parameters and dim is the dimension of the parameter itself This method calls asynchronously the hook method of the ACE Reactor called when there are data in the receiving buffer This hook method in turn calls the receive frame method of the BulkDataCallback The index 1 is passed to receive frame as parameter indicating that the data that are going to arrive are parameters c streamctrl p gt stop This method calls asynchronously the handle stop method of the BulkDataCallback setting its internal state d dp p gt send frame param After the internal state of the callback T has been set the actual parameter are sent and the ACE Reactor calls receive frame param in response to this method call receive frame in turn calls the cbStart param method of the user callback passing the parameter to it Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 41 of 44 ALMA AGS Bulk Data Transfer e streamctrl p stop This method calls asynchronously the handle stop method of the BulkDataCallback setting its internal state Inside this last handle stop call there is an automatic sleep that waits until all the sent data are received see section 2 3 4 Timeout The paceData method call triggers the same mechanism for passing the data The only differences occur at point b where the index 2 is passed in
42. nput txt resides li bulkDataEx1 Example 2 1 acsStartContainer cpp ContainerEx1 in lt gt bulkData test the directory where the file bulkDatalnput txt resides Sender 11 acsStartContainer cpp ContainerEx3 Distributer 111 acsStartContainer cpp ContainerEx2 Receivers iv bulkDataDistributerEx Example 3 i acsStartContainer cpp ContainerNotifTest ii bulkDataNotificationTest Example 4 Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 22 of 44 ALMA AGS Bulk Data Transfer 111 acsStartContainer cpp ContainerThreadTest 1v bulkDataThreadTest 6 1 2 Howto customize the examples This section shows in grater detail how the Bulk Data examples are built in order to allow the user to customize them to fit his needs For brevity of four flows only flows 1 and 3 are shown Error handling is not included see Error handling section Example 1 The test executable is bulkDataEx1 cpp After getting the references to the Receiver and Sender the Sender initialize the connection and then transfer parameters and data to the Receiver At the end the connection is closed 1 include lt maciSimpleClient h gt 2 include lt baci h gt 3 include ace Get Opt h 4 include orbsvcs AV AVStreams i h 5 include bulkDataSenderC h 6 include bulkDataReceiverC h 7 include ACSBulkDataError h Se 9 using namespace maci 0 using namespace ACSBulkDataError dL Be W
43. nt ACS_SHORT_LOG LM_DEBUG BulkDataSenderImpl lt gt paceData end of ES break MAIRE PEE a e flowNumber 3 getSender gt sendData flowNumber amp mb ACS_SHORT_LOG LM_DEBUG flow 3 file bulkDatainput txt sent Reset the mb mb reset end while Close the input file ACE OS fclose fp void BulkDataSenderExlImpl stopSend throw CORBA SystemException AVStopSendErrorEx ACS_TRACE BulkDataSenderImpl stopSend CORBA ULong flowNumber 1 getSender stopSend flowNumber flowNumber 3 getSender stopSend flowNumber MACI DLL support functions X include lt maciACSComponentDefines h gt MACI DLL SUPPORT FUNCTIONS BulkDataSenderExlImpl R E Jone o eh m m m m M 13 startSend method Used to start the transfer and to send parameters 21 33 An ACE Message Block of 10 kB is created Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 27 of 44 ALMA 35 36 44 50 52 53 84 68 83 96 115 133 Receiver ACS Bulk Data Transfer The Message Block is sent as parameter on flow 1 getSender returns the BulkDataSender private member of the base class BulkDataSenderImpl lt gt The output filename is created The filename is sent as parameter on flow 3 paceData method Used to send the data A Message Block of 14 MB is created and sent on flow 1 Insi
44. od The example 3 of section 6 shows an example of the usage of the Notification mechanism for the point to point and distributor models 6 User Manual This chapter is divided in two sections The first one illustrates how to run the examples provided with the bulkData module The second one shows how to customize the examples to fit the particular user needs 6 1 ACS Bulk Data transfer examples Inside the test directory of the bulkData module it is possible to find some examples which show the main usage of the Bulk Data Transfer Example 1 A synchronous Sender initializes a connection with a Receiver establishing a stream with 4 flows The flows perform the following operations Flow 1 An ACE Massage Block of 10 kB is sent from the Sender to the Receiver as parameter Then a Message Block of 14 MB of data is transferred Flow 2 An array of 12000 chars is sent from the Sender to the Receiver as parameter Then an array of 12000000 chars of data is transferred Flow 3 The file test bulkDataInput txt is read on the Sender side and then transmitted to the Receiver The name of the output file bulkDataOutput txt is sent as a parameter Flow 4 A Message Block of 37 MB is sent to the Receiver Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 20 of 44 ALMA AGS Bulk Data Transfer Related files for this example bulkDataEx1 cpp bulkDataSenderEx1Impl h cpp bulkDataReceiverCbEx1 h cpp
45. onnection establishment with a Receiver object specified as parameter It initializes the TAO A V and then reads from the CDB the connection parameters for the Sender number of flows of the stream protocol currently only TCP and host and port number for each flow see CDB configuration section After that it creates the stream endpoint and the flow endpoints and then adds the flow endpoints to the stream endpoint At the end it calls the openReceiver method on the Receiver see description of the Receiver class and creates the connection with it The disconnect method simply closes the connection with the Receiver In order to implement his her own Sender the user can either create an idl interface which inherits from the interface BulkDataSender using in this way the connect and disconnect method of the base class ore creating a new one not described If his her interface inherits from BulkDataSender he she must implement it in a new class which inherits from BulkDataSenderImp1 lt TSenderCallback gt and provide the implementation of the startSend paceData and stopSend methods see User Manual section startSend is used to start the data transfer e g send parameters to the Receiver open files etc With paceData the user sends the bulk of the data to the receiver and stopSend ends the data trasfer e g close the open files etc To actually send the data the respective methods of the BulkDataSender class must be called inside the
46. ponseTime ThreadBase defaultResponseTime const ACS TimeInterval amp sleepTime ThreadBase defaultSleepTime I6 SenderThread AL 2 vista E MEO SIE CINES e 20 privater Pile BulkDataSenderThreadImpl sender p 228 ACE Message Block mb p 238 CORBA ULong flowNumber m 24s IB 2g 26 class BulkDataSenderThreadImpl public virtual BulkDataSenderDefaultImpl publi c virtual POA bulkdatadistr BulkDataSenderDistr Create Date 2004 11 11 Author Roberto Cirami ACS_Bulk_Data_Transfer Page 35 of 44 ALMA 13 40 ACS Bulk Data Transfer PE 2 pao LO S02 Sig BulkDataSenderThreadImpl const ACE CStrings name ContainerServices containerServices D2 virtual BulkDataSenderThreadImpl 3325 Anbrtqgal acid startsena throw CORBA SystemException AVStartSendErrorEx 34 virtual void paceData throw CORBA SystemException AVPaceDataErrorEx 35 virtual void stopSend throw CORBA SystemException AVStopSendErrorEx SGE alo private BSE B95 CORBA ULong numberOfFlows 40 vector lt SenderThread gt thread p 4T Vee The SenderThread class inherits from ACS Thread The sender component class BulkDataSenderThreadImpl has a vector of pointers to SenderThread as attribute The file bulkDataSenderThreadImpl cpp contains the definition of the SenderThread and BulkDataSenderImpl classes Below the relevant code related to the thread creation and the sending of data is
47. r void Fig 5 Class diagram for Receiver idl interfaces POA bulkdata BulkDataReceiver CharacteristicComponentimpl openReceiver void closeReceiver void SJ lana TReceiverCallback gt BulkDataReceiverlmpl s BulkDataReceiver openReceiver void closeReceiver void Fig 6 Class diagram for Receiver implementation Author Roberto Cirami Create Date 2004 11 11 ACS Bulk Data Transfer Page 12 of 44 ALMA AGS Bulk Data Transfer The class BulkDataReceiverImpl lt TReceiverCallback gt provides the implementation for the methods openReceiver and closeReceiver The openReceiver method called from the Sender connect method initializes the TAO A V and then reads from the CDB the connection parameters for the Receiver number of flows of the stream protocol currently only TCP and host and port number for each flow After that it creates the stream endpoint and the flow endpoints and then adds the flow endpoints to the stream endpoint The closeReceiver method simply closes the connection on the Receiver side The Receiver can receive data only using a callback mechanism see section 2 2 The template class BulkDataReceiverImpl TReceiverCallback provides the hook for the receiver callback TReceiverCallback is a class which must inherit from BulkDataCallback and must implement the three methods cbStart cbReceive and cbStop cbStart is called auto
48. s a callback mechanism to actually manage the incoming data stream The provided TAO AV Callback class offers three methods to fulfill this purpose handle start and handle_stop which react when a start stop is issued on a specific flow and a receive frame ACE Message block frame which is used to get the received data This mechanism has the following limitations TAO_AV_Callback handle_start int 7 a ag receive_frame frame ACE Message Block int 1 there is no possibility to send nandle stop int short parameters directly when a start is issued for example an UID to characterize the forthcoming frame a string BulkDataCallback handle start int containing a filename to be receive_frame frame ACE Message Block int handle_stop int opened etc cbStart userParam ACE Message Block int cbReceive frame ACE Message Block int cbStop int 2 asynchronization problem occurs UserReceiverCb Point 2 is quite subtle Data sent by the cbStart userParam ACE Message Block z0 int Sender are first received in the TCP cbReceive frame ACE Message Block int chStopQ int receive memory buffer of the involved host whose typical default size for Linux Red Hat 9 0 is around 85 KB Being the ACE Reactor event driven as soon as data are available the pre registered callback method is called and data are consumed the rea
49. tStopped 47 ACS SHORT LOG LM INFO s Stopped thread getName c str a 3 1 19 In the constructor of the sender component class the four threads are instantiated 28 In the paceData method the threads are started The actual sending of data is implemented in the run method of the SenderThread class The ACE 39 message blocks containing the data to be sent are created in the constructor of the SenderThread class not shown 61 8 Remarks on callback deletion Instances of the callback classes are destroyed inside the handle destroy method by invoking a delete this call 1 e commiting a suicide This is not completely safe and in principle should be used only if objects are allocated on the heap Unfortunately we are forced to do this because of the underlying implementation in the ACE TAO A V Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 37 of 44 ALMA ACS Bulk Data Transfer An ACS Callback class is instantiated on the heap in the method get callback for details see the file bulkData include bulkDataFlowConsumer i This method is in turn called automatically inside the ACE TAO A V see ACE ROOT TAO orbsvcs orbsvcs AV TCP cpp line 297 The problem is that when the connection is closed the TAO A V i e the reactor TAO AV TCP Object object calls automatically the method handle destroy see line 259 of ACE ROOT TAO orbsvcs orbsvcs AV TCP cpp and after that destroy itself 11 line 26
50. the send frame method indicating that the data that are going to be received are actual data and not parameters and at point d where receive frame calls the cbReceive data method of the user callback passing the data to it Finally inside stopSend the stopSend flwn method of BulkdataSender is called which in turn calls the stop method of the TAO StreamCtrl object This method calls asynchronously handle stop on the BulkDataCallback which calls the costop of the user callback Create Date 2004 11 11 Author Roberto Cirami ACS Bulk Data Transfer Page 42 of 44 ACS Bulk Data Transfer ALMA rh ds n n Oo d a a g le Odojsqa 1E Odojs ajpuey oe Odoys 5z uuassei k Opuasdajs 2Z M je Odojs aipueu 9z Odojs gz ejep eue283q2 pz ejep awe anaal ez ejep eurejpuas zz Odojs sipueu LZ Odojs 0c 77 z jawey anaal B wip z eurej pues 81 o Ones aipueu 24 Queis OL je g ejep ump ejegpues 6 M Qejegeoed p E p Odojs a pueu EL Odojs zi E weJed peygqa LL weied jawey amada 0 weed jawey puas 6 K hi Odojs ajpueu g Qdos 4 e Jawe anasa 9 wip Jawe pues g Lig Lle OuejsTappuey p Queis l 1X34919A1993y48 2 Q4INg q 1819901 yoequejeyequing j elqojo2ojo1d AY

Bulk Data Transfer

Contents

Download Pdf Manuals

Related Search

Related Contents