EGEE gLite User's Guide
Contents
1. 13 3 34 SUITE WHINE 13 135 GLITE LSEEK ye BEA Ou ect ER E Hot a 13 A ek ond EG 2 ES IHRE 9 dp rw we Rd a 14 MUTE CLOSE ok aris nek p SD bee a eR RO a 14 238 GLITE UNLINE L4 oad EG a HR 9 dep rive wies Rd a 15 139 GLITE ERROR ask reso a 15 3 3 10 GLITE_STRERROR 15 3341 GLITE POSIK OPEN Lue Sexe ed eee a 16 3 3 12 _ 81 _ 16 3 3 13 1 _ 8 _ 17 3 3 14 GLITE POSIX WRITE 17 3 3 15 GLITE_POSIX_LSEEK 17 3 3 16 _ 51 _1 64 18 INFSO RI 508833 PUBLIC 3 25 OQ Enabling Grids for E sciencE 3 3 17 3 3 18 3 3 19 3 3 20 3 3 21 3 3 22 3 3 23 3 3 24 EGEE GLITE USER S GUIDE gLite I O GLITBE POSIX ESTAT m eyes GLITE POSIX FSTAT64A GLITE POSIX CLOSE 1 wae m xm a GLITE POSIX UNLINK GLITE FILBHANDLE iu badeu 4 _ 5 _ GLITE STAT i252 24b ea Ok a TYPE DEFINITIONS AND CONSTANTS 4 KNOWN PROBLEMS AND CAVEATS 4 1 CASTOR TIMEOUT 5 COMPONENT AND INTERACTION DIAGRAMS INFSO RI 508833 Doc Ident2. 24 25 Doc Identifier ecee EGEE GLITE USER S GUIDE P unt Enabling Grids gLite I O for E sciencE Date March 23 2005 AlienlOclient 1 byteswritten NetWritelvoid void int64 int3 2 int32 1 remotewrite QUANT Anet_extendedParallelT cpClient 1 1 1 writelnt3 2 WRITE BLOCK 1 1 2 writeInt64 remoteOffset if cryptdata 1 3 1 Encrypt Data Block 1 1 4 writelnt3 2 buffersize 1 1 5 write buffer buffersize 1 1 6 readint3 2 byteswritten Figure 4 Sequence diagram showing the steps in a client side gLite I O write call INFSO RI 508833 PUBLIC 25 25
3. are fh IN the file handle returned by a previos glite openorglite creat call INFSO RI 508833 PUBLIC 13 25 EGEE GLITE USER S GUIDE EGEE TECH 570771 10 Enabling Grids for E sciencE gLite I O Date March 23 2005 include glite data io client ioclient h glite int64 glite lseek glite handle fh glite int64 offset glite int32 whence offset IN new byte offset to seek to whence N relative file position it can be one of the following values SEEK SET set to offset SEEK CUR set to current position plus offset SEEK END set to the size of the file plus offset Return value The new offset from the beginning of the file or a negative value in case of error 3 3 6 GLITE FSTAT include glite data io client ioclient h glite int32 glite fstat glite handle fh struct glite stat buf Get Information about the file associated with the given handle The parameters are fh IN the file handle returned by a previos glite openorglite creat call buf OUT pointer to a stat structure as defined in iotypes h into which information is placed concerning the file Return value GLITE IO SUCCESS if success or a negative value in case of error 3 3 7 GLITE CLOSE include glite data io client ioclient h glite int32 glite close glite handle fh Close the file The parameters are fh IN the file handle returned by a previos glite openorglite creat call Retur
4. stdout layout ConversionPattern m n log4j appender file org apache log4j FileAppender log4j appender file fileName HOME glite var log glite io client log log4j appender file layout org apache log4j PatternLayout log4j appender file layout ConversionPattern d 6p c m n 3 2 COMMANDLINE INTERFACES There are three CLI tools that can be used to upload download remove files into from a Storage Element These three tools are glite get Retrieve a file from the local Storage Element glite put Upload a file to the Storage Element glite rm Remove a file They all have the following options in common h Print a help screen s Be silent c lt name gt Use configuration with name lt name gt overriding the default glite io client configuration INFSO RI 508833 PUBLIC 9 25 EGEE GLITE USER S GUIDE EGEE TECH 570771n10 Enabling Grids gLite I O for E sciencE Date March 23 2005 By default the g1ite get and glite put command line tools will display the standard error a progress bar showing the status of the transfer For all three tools a report is printed on completion showing information about the transferred or removed file Using the s option instructs the command line tools to suppress these informational messages Errors are still printed to stderr 3 2 1 GLITE PUT UPLOAD A LOCAL FILE TO THE STORAGE ELEMENT The syntax for the put command is glite put lt localfilename gt lfn lt lfn gt or glite
5. 0 read guicl 1 1 7 read remotefilename remotefilename length g if cryptauthentication 1 11 1 Decrypt guid 1 1 12 readint32 openResponse d Doc Identifier EGEE TECH 570771 v1 0 Date March 23 2005 Figure 1 Sequence diagram showing the steps in a client side gLite I O open call INFSO RI 508833 23 25 ecee EGEE GLITE USER S GUIDE EGEE TECH ST0771 10 Enabling Grids gLite I O for E sciencE Date March 23 2005 AlienlOclient 1 closeResult NetClosd void void 1 remoteclose QUANT Anet_extendedParallelT cpClient c int3 2 1 1 1 writelnt3 2 CLOSE FILE 1 1 2 readint3 2 closeResult Figure 2 Sequence diagram showing the steps in a client side gLite I O close call AlienlOclient 1 bytesread NetRead void void int64 int32 int3 2 1 remoteread QUANT Anet extendedParallelTcpClient c int64 1 2 writeInt3 2 READ BLOCK REMOTE STREAM 1 3 1 writelnte 4 remoteoffset 1 3 2 writelnt3 2 size 1 3 3 readint3 2 nreads for i20 lt nreads i 1 3 4 1 2 2 buffersize 1 3 4 2 read buffer buffersize if cryptdata 3 4 3 1 Decrypt Buffer 1 3 5 readint32 bytesread else 1 4 1 read buffer size per Figure 3 Sequence diagram showing the steps in a client side gLite I O read call INFSO RI 508833 PUBLIC
6. 33 PUBLIC 10 25 EGEE GLITE USER S GUIDE EGEE TECH 570771n10 Enabling Grids gLite I O for E sciencE Date March 23 2005 3 2 8 GLITE RM REMOVE A REMOTE FILE FROM THE STORAGE ELEMENT If the given file is the last replica then also the Logical File Name and the GUID will be deleted from the catalog glite rm lfn lfn Or glite rm lfn where 1 n is the logical file name of the file you want to remove In case you want to use GUIDs you can use the syntax glite rm guid lt guid gt or glite rm lt guid gt where lt guid gt is the GUID of the remote file By default that application displays to the standard error some information concerning the remove action If you want to run that application in silent mode you have to use the option s in that case only errors will be printed to stderr 3 2 4 CONFIGURATION In order to use the command line tools you need to set up the environment and configure glite io client To set the environment up you need to acquire your Grid Proxy Certificate and then set the LD_LIBRARY_PATH environment variable in order to locate the glite io client library and its depen dencies Normally this is done automatically on a Grid User Interface and Worker Node Details on how to do this can be found in the gLite Installation Guide In case you want to override the default glite io cient configuration values you have to create a pair of configuration files for example glite
7. CGee Enabling Grids Information Society for E sciencE EGEE EGEE gLite User s Guide GLITE I O Document identifier EGEE TECH 570771 v1 0 Date March 23 2005 Activity JRA1 Data Management Document status DRAFT Document link https edms cern ch document 570771 Abstract This is the user s guide to the gLite Input Output libraries INFSO RI 508833 PUBLIC 1 25 EGEE GLITE USER S GUIDE EGEE TECH 570771n10 Enabling Grids for gLite O Date March 23 2005 Document Change Log Issue Date Comment Author 1 0 1 3 2005 Initial Version Text by Paolo Badino and Peter Kunszt Document Change Record Issue Item Reason for Change Copyright Members of the EGEE Collaboration 2004 See http eu egee org partners for de tails on the copyright holders EGEE Enabling Grids for E science in Europe is a project funded by the European Union For more information on the project its partners and contributors please see http www eu egee org You are permitted to copy and distribute verbatim copies of this document containing this copy right notice but modifying this document is not allowed You are permitted to copy this document in whole or in part into other documents if you attach the following reference to the copied ele ments Copyright 2004 Members of the EGEE Collaboration http www eu egee org The information contained in thi
8. K_SET set to offset SEEK_CUR set to current position plus offset SEEK_END set to the size of the file plus offset Return value the resulting offset location as measured in bytes from the beginning of the file Other wise a value of of f_t 1 is returned and glite_posix_errno is set to indicate the error 3 3 16 GLITE_POSIX_LSEEK64 include glite data io client ioclient posix h off64_t glite_posix_lseek64 int fd off64_t offset int whence Set a file read write pointer The parameters are fd IN the file descriptor returned by a previos glite_posix_open or glite_posix_creat call offset IN new byte offset to seek to whence N relative file position it can be one of the following values SEEK SET set to offset SEEK CUR set to current position plus offset SEEK END set to the size of the file plus offset Return value the resulting offset location as measured in bytes from the beginning of the file Other wise a value of o f64 t 1 is returned and glite posix errno is set to indicate the error 3 3 17 GLITE POSIX FSTAT finclude glite data io client ioclient posix h int glite posix fstat int fd struct stat buf Return information about the specified file The parameters are d IN the file descriptor returned by a previos glite posix openorglite posix creat call buf OUT pointer to a stat structure as defined in sys stat h into which information is placed con cer
9. LL HANDLE is returned 3 3 22 GLITE POSIX ERRNO finclude glite data io client ioclient posix h extern int glite posix errno Number of last error Be aware that is not thread safe INFSO RI 508833 PUBLIC 20 25 OQ Enabling Grids for E sciencE 3 3 23 GLITE STAT EGEE GLITE USER S GUIDE UL gLite O Date March 23 2005 struct glite_stat glite_int32 dev glite_int64 ino glite_int32 mode glite int32 nlink glite int32 uid glite int32 gid glite int32 rdev glite int64 size glite int32 blksize glite_int64 blocks glite_time_t atime glite_time_t mtime glite_time_t ctime char guid GLITE GUID LENGTH char lfn GLITE LENGTH char surl GLITE SURL LENGTH char owner GLITE NAME LENGTH char group GLITE NAME LENGTH remote remote Storag include glite data io client ioclient h The device The inode on the remote Storage System The file protection and type bits The number of hard links on the remote Storage System The user ID of owner on the Storage System The group ID of owner on the Storage System The device type if inode device The total size in bytes The blocksize for remote The nun on the Time of Time of Time of Element filesystem I O ber of blocks allocated remote Storage Syste
10. READS lglobus gss assist S GLOBUS FLAVOR THREADS lglobus gssapi gsi GLOBUS FLAVOR THREADS lglobus common GLOBUS FLAVOR THREADS N N 3 REFERENCE GUIDE The package providing the gLite I O client is called org glite data io client It contains the com mand line tools as well as the API libraries and include files 3 1 CONFIGURATION 3 1 1 CONFIGURATION PARAMETERS Unless a different configuration namespace is used see the INSTALL file coming with the org glite data io client package Creating Different Configurations the gLite I O client initial izes itself at the first glite open or glite posix open call looking at the configuration files in named glite io client properties xml and glite io client log properties in SGLITE LOCATION USER etc GLITE LOCATION etc or SGLITE LOCATION VAR etc That config uration file should contain at least the entries for the glite io client component Configuration parameters used at initialization time are Server Mandatory The name of the host where the gLite I O server is running ServerPort The port number that the gLite I O server is listening on Default value 9999 EncryptName Enable encryption of the file name Allowed values are true and false Default value true EncryptData Enable encryption of the data blocks sent and received Allowed val
11. RM and the underlying storage to actually resolve the file name and to serve up the bytes to the user 2 QUICKSTART GUIDE 2 1 COMMAND LINE QUICKSTART The gLite I O library comes with a set of command line tools to facilitate the user interaction with the grid In order to put a local file into the grid g1ite put command can be used UI glite put mylocalfile lfn tmp myfilel glite put Total 0 00 MB 100 00 0 0 Mb s Transfer Completed LFN tmp myfilel GUID 008b 98e 27aa 1227 be5e 898a04albeef SURL srm gridftp05 cern ch 8443 srm managerV1 SFN castor cern ch use Data Written bytes gt 875 Eff Transfer Rate Mb s 0 000094 The three slash notation is in accordance with RFC 2396 1 Instead the user can also specify just tmp myfilel The verbosity of glite put can be suppressed by using the s silence option In the same way data files can be retrieved from the Grid by issuing glite get UI glite get lfn tmp myfilel myfile grid glite get Total 0 00 MB 100 00 5 0 0 Mb s INFSO RI 508833 PUBLIC 5 25 EGEE GLITE USER S GUIDE EGEE TECH 570771 10 Enabling Grids gt or ESENE gLite Date March 23 2005 Transfer Completed LEN tmp myfilel GUID 008bf98e 27aa 1227 be5e 898a04albeef SURL srm gridftp05 cern ch 8443 srm managerV1 SFN castor cern ch use Data Written bytes 375 Eff Transfer Rate Mb s 0 000041 Finally the data
12. System and an attempt is made to remove the physical instance but if that fails no error are returned In case that replica is the last one also the file logical name and GUID will be removed so they can be reused later The parameters are pathname JN the name of the remote file to delete It could be in the format 1 n file name or guid file guid glite creat also accept the file name without the prefix In that case if file name starts with a is interpreted as an LEN otherwise if it s a valid GUID string represen tation it s interpreted as a GUID INFSO RI 508833 PUBLIC 19 25 Doc Identifier EGEE GLITE USER S GUIDE Won Enabling Grids gLite I O for E sciencE Date March 23 2005 Return value On success zero is returned On error 1 is returned and glite posix errno is set appropriately 3 3 21 GLITE FILEHANDLE finclude glite data io client ioclient posix h glite handle glite filehandle int fd Returns the file handle associated with the descriptor The returned handle could be used to call glite io non posix methods like glite fstat but calling glite close on a file opened with glite posix open may cause an impredictable behavior so should be avoided in any case The pa rameters are d IN the file descriptor returned by a previos glite posix openorglite posix creat call Return value On success the associated file handle if success On error GLITE NU
13. and glite posix errno is set appropriately 3 3 12 GLITE POSIX CREAT finclude glite data io client ioclient posix h int glite posix creat const char pathname mode t mode Create a Remote File This call is equivalent to glite posix open pathname O CREAT mode The parameters are pathname N the name of the remote file to open It could be in the format 1 n file name or guid file guid glite creat also accept the file name without the prefix In that case if file name starts with a is interpreted as an LEN otherwise if it s a valid GUID string represen tation it s interpreted as a GUID mode N specifies what permissions the file has when it is created Return value On success the new file descriptor On error 1 is returned and glite_posix_errno is set appropriately INFSO RI 508833 PUBLIC 16 25 EGEE GLITE USER S GUIDE EGEE TECH 570771n10 Enabling Grids for gLite VO Date March 23 2005 include glite data io client ioclient posix h ssize_t glite_posix_read int fd void buf size_t count 3 3 13 GLITE POSIX READ Read up to count bytes from file descriptor fd into the buffer starting at buf The parameters are fd IN the file descriptor returned by a previos glite posix open call buf IN the buffer where to store the data count N the number of byte to read This parameter should be equal or less the size of the buffer Return value On
14. can be removed by issuing glite rm UI glite rm lfn tmp myfilel The server that is being contacted is specified in the configuration see Section 3 1 For more details on these command line tools please see the reference guide below in Section 3 2 2 2 API QUICKSTART ACCESSING A REMOTE FILE USING GLITE I O In order to access a remote file you need to write your own application linking the gLite I O client library Libglite_data_io_client so and opening the file the same way you open a local file using POSIX calls You may use either the g1ite posix APIs in which case the syntax of open close read write and lseek operations is exaclty the same of the standard POSIX methods or you may choose to use the glite_ API calls which differ from the POSIX ones since they use gLite defined types and accept additional parameters mainly related to optimization and thread safe error reporting The flow will always be the same in both cases Reading a file using extended interface include glite data io client ioclient h glite_result gl_res glite_error err glite_handle fh fd glite open remotefilename O RDONLY 0 0 amp gl res error handling if glite handle GLITE NULL HANDLE err glite error fh fprintf stdout Cannot open file s n glite_strerror err return or exit a int nread 0 char buffer TRANSFERBLOCKSIZE do nread glite_read fh buffer TRANSFERBLOCKSIZE
15. d in future releases of gLite I O 5 COMPONENT AND INTERACTION DIAGRAMS REFERENCES 1 URI Syntactic Components http www faqs org r cs r c2396 html 2 JRA1 Data Management Cluster Overview of gLite Data Management EGEE March 2005 https edms cern ch document 570643 INFSO RI 508833 22 25 OQ Enabling Grids for E sciencE EGEE GLITE USER S GUIDE gLite O AlienlOclient 1 openResponse NetOpen const char void 1 1 writeconnectionblock 1 1 1 1 writelnt3 2 cryptauthentication 1 1 1 2 writelnt3 2 cryptdata 1 1 1 3 writelnt3 2 cacheleve 1 1 1 4 writelnte4 requestsize 1 rremoteopen QUANT extendedParallelTcpClient c cons 1 1 1 5 writeChar serverid 1 1 1 6 writeChar Icachehopcnt 1 1 1 7 writeChar rcachehopcnt Y vv 1 1 1 8 writeChar mcachehopcnt 1 1 1 10 write cacheforwardaddress 1024 1 1 1 9 writeChar cachehopcnt 1 1 1 11 write login 256 1 1 1 12 write protocol 256 1 1 1 13 writeilfn 1024 1 1 1 14 write guid 40 yv 1 1 2 writelnt3 2 openidentifier 31 if cryptauthentication enable 1 1 4 writelnt32 filename length 1 Y Cr 1 1 5 write filename filename length 1 1 1 6 readint3 2 remotefilename length if cryptauthentication 1 8 1 Decrypt remote Filename 1 1 9 readint3 2 guid length 1 1 1
16. de N specifies what permissions the file has when it is created size IN specifies how many bytes should be preallocated remotely May be null result OUT the result code associated with the operation This parameter is used to return additional information in case of errors and can be null if that information is not requested Return value a handle to the opened file or GLITE NULL HANDLE in case of failure 3 3 3 GLITE READ include glite data io client ioclient h glite int32 glite read glite handle fh void buf glite int32 count Read the file The parameters are fh IN the file handle returned by a previos glite open call buf IN the buffer where to store the data count N the number of byte to read This parameter should be equal or less the size of the buffer Return value the number of bytes read or a negative value in case of error 3 3 4 GLITE WRITE include glite data io client ioclient h glite int32 glite write glite handle fh const void buf glite int32 count Write to the file The parameters are fh IN the file handle returned by a previos glite open orglite creat call buf IN the buffer containing the data to be written count N the number of byte to write This parameter should be equal or less the size of the buffer Return value the number of bytes written or a negative value in case of error 3 3 5 GLITE LSEEK Set a file read write pointer The parameters
17. error handling if nread lt 0 err glite_error fh fprintf stdout Cannot read file s n glite_strerror err INFSO RI 508833 PUBLIC 6 25 EGEE GLITE USER S GUIDE EGEE TECH S70771410 Enabling Grids gLite I O for E sciencE Date March 23 2005 break or return or exit or Do something while nread gt 0 glite close fh Reading a file using POSIX interface include lt string h gt finclude glite data io client ioclient posix h int nread 0 char buffer TRANSFERBLOCKSIZE int fd glite posix open remotefilename O RDONLY 0 error handling if fd 0 fprintf stdout Cannot open file s n strerror glite_posix_errno return or exit or do nread glite posix read fd buffer TRANSFERBLOCKSIZE error handling if nread lt 0 fprintf stdout Cannot read file s n strerror glite_posix_errno break or return or exit or Do something while nread 0 glite posix close fd The remotefilename is a logical file name like in the command line example above e g lfn tmp myfilel The flag is a bitwise or from the system values RDONLY WRONLY O_CREAT while mode specifies the permissions the file should have upon creation Upon success a file descriptor is returned otherwise the result is less than zero and errors have to be handled In the posix ca
18. evios glite open orglite creat call Return value GLITE IO SUCCESS if success a negative value in case of error 3 3 10 GLITE STRERROR include glite data io client ioclient h const char glite strerror glite result error Get a string representing a GLite Error code The parameters are error IN the error code to be translated Return value the error strig INFSO RI 508833 PUBLIC 15 25 EGEE GLITE USER S GUIDE EGEE TECH S70771 410 Enabling Grids for gLite VO Date March 23 2005 finclude glite data io client ioclient posix h int glite posix open const char pathname int flags mode t mode 3 3 11 GLITE POSIX OPEN Open a Remote File The parameters are pathname N the name of the remote file to open It can be in the format 1 n file name or guid file guid glite posix open also accepts the file name without the prefix In that case if file name starts with a is interpreted as an LEN otherwise it will assumed it is a GUID and a GUID representation validity check will be performed on it flags IN the requested access flag The value should be a the bitwise inclusive OR of values from the following lists O RDONLY open for reading only O WRONLY open for writing only O CREAT create a file mode N specifies what permissions the file has when it is created Return value On success the new file descriptor On error 1 is returned
19. g The value should be a the bitwise inclusive OR of values from the following lists O RDONLY open for reading only O WRONLY open for writing only O CREAT create a file mode N specifies what permissions the file has when it is created size IN specifies how many bytes should be preallocated remotely It will be used only if the CREAT flag is set and may be null result OUT the result code associated with the operation This parameter is used to return additional information in case of errors and can be null if that information is not requested Return value a handle to the opened file or GLITE NULL HANDLE in case of failure 3 3 2 GLITE CREAT include glite data io client ioclient h glite handle glite creat const char glite int32 mode glite int64 size glite result result Create a Remote File This call is equivalent to glite open name O CREAT mode size result The parameters are pathname JN the name of the remote file to open It can be in the format 1 n file name or guid file guid glite creat also accepts the file name without the prefix In that case if file name starts with a is interpreted as an LEN otherwise if it s a valid GUID string representation it s interpreted as a GUID INFSO RI 508833 PUBLIC 12 25 EGEE GLITE USER S GUIDE EGEE TECH 570771 410 Enabling Grids gLite I O for E sciencE Date March 23 2005 mo
20. ifier EGEE TECH 570771 v1 0 Date March 23 2005 4 25 EGEE GLITE USER S GUIDE EGEE TECH 570771 10 Enabling Grids gLite I O for E sciencE Date March 23 2005 1 INTRODUCTION Input Output librares are used to access files for read and write This is the most basic operation in any system dealing with data stored in files The files on the Grid are known to the user by its Logical File Name LEN The gLite I O library provides the necessary methods and command line tools to interact with the grid storage using LFNs To get a complete overview of the gLite data management services and how the I O fits into them as well as explanations about file names etc the reader is referred to the Overview of gLite Data Management User s Guide 2 1 1 SERVICE ARCHITECTURE From the user s point of view gLite I O is just a set of client libraries that can be used like a posix I O library The server side architecture determines the semantics of the library to a large extent The sequence diagrams in the appendix 5 show how the client interacts with the server for the four most basic operations 1 2 INTERACTIONS WITH OTHER SERVICES The I O client only interacts with the I O server In order to do so the client needs a valid user VOMS proxy certificate so there is an indirect dependency on VOMS or at least grid proxy init The I O server interacts with the gLite File and Replica Catalog the File Authorization Service the S
21. io client customconfig properties xml and glite io client customconfig log properties and store them in one of the predefined configu ration paths SGLITE_LOCATION_USER etc SGLITE LOCATION etc or SGLITE LOCATION VAR etc filling them with the Glite IO Client configuration parameters you want to assign and then invoke the transfer applications by using the c lt config name gt option In our example you ll have to type glite put c glite io client customconfig localfilename lfn lfn glite get c glite io client customconfig lfn lfn localfilename glite rm c glite io client customconfig lfn lfn 3 3 APPLICATION PROGRAM INTERFACES 3 3 1 GLITE OPEN Open a Remote File The parameters are pathname N the name of the remote file to open It can be in the format 1 n file name or guid file guid glite open also accepts the file name without the prefix In that case if INFSO RI 508833 PUBLIC 11 25 EGEE GLITE USER S GUIDE EGEE TECH 570771 10 Enabling Grids for E sciencE gLite I O Date March 23 2005 include glite data io client ioclient h glite handle glite open const char pathname glite int32 flags glite int32 mode glite int64 size glite result result file name starts with a is interpreted as an LEN otherwise it will assumed it is GUID and a GUID representation validity check will be performed on it flags IN the requested access fla
22. m last access last modification last change File GUID EE CE um he logical file name he physical file name he name of the owner he group name of owner IO Stat Type This structure is used to return File information using the glite_fstat method The meaning of each field is the same as the posix stat64 structure which extends addition the guid lfn surl owner and group fields INFSO RI 508833 PUBLIC m OQ Enabling Grids for E sciencE EGEE GLITE USER S GUIDE gLite I O 3 3 24 DEFINITIONS AND CONSTANTS Doc Identifier EGEE TECH 570771 v1 0 Date March 23 2005 typedef int typedef void typedef int typedef long long gli glite int32 te int64 glite handle glite result typedef time t glite define GLITE NULL HANDLE define GLITE LFN LENGTH define GLITE SURL LENGTH define GLITE LENGTH define GLI AM time t 1024 2048 31 1024 4 KNOWN PROBLEMS AND CAVEATS 4 1 CASTOR TIMEOUT There is an as yet unresolved open issue concerning timeouts an internal dependency of gLite I O has a hardcoded timeout of 5 minutes If the SRM is a mass storage system like Castor where staging of data off tape can take longer than that the connection will time out prematurely Currently the only thing that can be done is that the client re tries later This will be resolve
23. n value GLITE IO SUCCESS if success a negative value in case of error INFSO RI 508833 PUBLIC 14 25 EGEE GLITE USER S GUIDE EGEE TECH S70771 10 Enabling Grids for gLite VO Date March 23 2005 include glite data io client ioclient h glite_int32 glite_unlink const char pathname 3 3 8 GLITE UNLINK Deletes a file on the remote Storage element That replica information will be removed from the Logical File System and an attempt is made to remove the physical instance but if that fails no error are returned In case that replica is the last one also the file logical name and GUID will be removed so they can be reused later The parameters are pathname JN the name of the remote file to delete It could be in the format 1 n file name or guid file guid glite unlink also accept the file name without the prefix In that case if file name starts with a is interpreted as an LEN otherwise if it s a valid GUID string represen tation it s interpreted as a GUID Return value GLITE SUCCESS if success a negative value in case of error 3 3 9 GLITE ERROR include glite data io client ioclient h glite int32 glite error glite handle fh Get the Last GLite Error code associated to a File Handle NOTE This method cannot be used after an open or close call since there s no valid file handle The parameters are fh IN the file handle returned by a pr
24. ning the file Return value On success zero is returned On error 1 is returned and glite posix errno is set appropriately INFSO RI 508833 PUBLIC 18 25 EGEE GLITE USER S GUIDE EGEE TECH 570771 10 Enabling Grids for gLite VO Date March 23 2005 include glite data io client ioclient posix h int glite_posix_fstat64 int fd struct stat64 buf 3 3 18 GLITE POSIX FSTAT64 Return information about the specified file To be used for large files The parameters are d IN the file descriptor returned by a previos glite posix openorglite posix creat call buf OUT pointer to a stat structure as defined in sys stat h into which information is placed con cerning the file Return value On success zero is returned On error 1 is returned and glite posix errno is set appropriately 3 3 19 GLITE POSIX CLOSE finclude glite data io client ioclient posix h int glite posix close int fd Close the file The parameters are d IN the file descriptor returned by a previos glite posix openorglite posix creat call Return value On success zero is returned On error 1 is returned and glite posix errno is set appropriately 3 3 20 GLITE POSIX UNLINK finclude glite data io client ioclient posix h int glite posix unlink const char pathname Deletes a file on the remote Storage element That replica information will be removed from the Logical File
25. put localfilename lt 1 gt where lt localfilename gt is the name of the local file you want to upload and 1 n is the logical file name you want to assign to that file Please remember that a valid LEN should always start with a which in the first URL form results in three consecutive slashes 1 In case you want to assign a GUID you can use the syntax glite put localfilename guid guid or glite put lt localfilename gt lt guid gt where lt guid gt is the GUID you want to assign to the file The GUID has always to be a valid GUID string representation 36 bytes that must follow the format string representation 08x 04x 04x 04x 012x If you want to specify explicitly the permission bits you can use the m option glite put localfilename lfn lt lfn gt m mode where mode should be an octal integer in the range 0001 0777 The default value is 0640 3 2 2 GLITE GET COPY A REMOTE FILE FROM THE STORAGE ELEMENT TO A LOCAL FILE The syntax is glite get lfn lt lfn gt localfilename or glite get lfn lt localfilename gt where lt 1fn gt is the logical file name of the file you want to download and 1ocalfilename is the name of the destination local file In case you want to use GUIDs you can use the syntax glite get guid guid localfilename or glite get lt guid gt lt localfilename gt where guid is the GUID of the remote file INFSO RI 5088
26. s document represents the views of EGEE as of the date they are published EGEE does not guarantee that any information contained herein is error free or up to date EGEE MAKES NO WARRANTIES EXPRESS IMPLIED OR STATUTORY BY PUBLISHING THIS DOCUMENT INFSO RI 508833 PUBLIC 2 25 ecee EGEE GLITE USER S GUIDE EGEE TECH ST0771 10 che iced gLite Date March 23 2008 CONTENTS 1 INTRODUCTION 5 1 1 SERVICE ARCHITECTURE 2 5 12 INTERACTIONS WITH OTHER SERVICES 5 2 QUICKSTART GUIDE 5 2 1 COMMAND LINE QUICKSTART 5 22 API QUICKSTART ACCESSING A REMOTE FILE USINGGLITEI O 6 3 REFERENCE GUIDE 8 3 1 CONFIGURATION 4 coc 40 eee eder ex bew adage deen ees 8 3 1 1 CONFIGURATION PARAMETERS 8 3 1 2 CONFIGURATION FILE 9 32 COMMANDLINEINTERFACES 9 3 2 1 GLITE PUT UPLOAD A LOCAL FILE TO THE STORAGE ELEMENT 10 3 22 GLITE GET COPY A REMOTE FILE FROM THE STORAGE ELEMENT ATCA EWES Sak acido de Ob Rok Dee e 10 3 2 3 GLITE RM REMOVE A REMOTE FILE FROM THE STORAGE ELEMENT 11 3 2 4 CONFIGURATION 11 3 3 APPLICATION PROGRAM INTERFACES 11 3 3 1 GLITE_OPEN 11 3 32 1 _ 12 3 33 GLITE_READ
27. se there isa glite posix errno variable defined which may be used just like the usual posix errno variable Once the file has been opened say for read it can be accessed by the posix Iseek and read methods to get blocks of data directly For more details on each method see the reference guide below NOTE Beware that the usage of the glite posix errno variable is not thread safe Don t use the posix interface in threaded programs In order to compile and link an application that uses glite io the following flags have to be passed in to the compiler and linker respectively Compilation ISGLITE LOCATION include INFSO RI 508833 PUBLIC 7 25 EGEE GLITE USER S GUIDE EGEE TECH S70771 410 Enabling Grids gLite I O for E sciencE Date March 23 2005 Linkage Q LOBUS FLAVOR THREADS gcc32dbg LIBS L GLITE LOCATION lib lglite data io client L GLOBUS_LOCATION lib lglobus 951 proxy core GLOBUS FLAVOR THREADS lglobus gsi credential GLOBUS FLAVOR THREADS Lobus gsi callback GLOBUS FLAVOR THREADS Lobus oldgaa GLOBUS FLAVOR THREADS lobus gsi sysconfig GLOBUS FLAVOR THREADS lobus gsi cert utils GLOBUS FLAVOR THREADS 5 551 GLOBUS FLAVOR THREADS Lobus proxy ssl GLOBUS FLAVOR THREADS Lobus openssl error GLOBUS FLAVOR THREADS lglobus ftp client GLOBUS FLAVOR THREADS lssl GLOBUS FLAVOR THREADS 1 GLOBUS FLAVOR TH
28. success the number of bytes read is returned zero indicates end of file and the file position is advanced by this number On error 1 is returned and g1ite posix errno is set appropri ately 3 3 14 GLITE POSIX WRITE finclude glite data io client ioclient posix h Ssize t glite posix write int fd const void buf size t count Writes up to count bytes to the file referenced by the file descriptor fd from the buffer starting at buf The parameters are d IN the file descriptor returned by a previos glite posix openorglite posix creat call buf IN the buffer containing the data to be written count N the number of byte to write This parameter should be equal or less the size of the buffer Return value On success the number of bytes written are returned zero indicates nothing was written On error 1 is returned and g1ite posix errno is set appropriately 3 3 15 GLITE POSIX LSEEK finclude glite data io client ioclient posix h off t glite posix lseek int fd off t offset int whence Set a file read write pointer The parameters are d IN the file descriptor returned by a previos glite_posix_open or glite posix creat call INFSO RI 508833 PUBLIC 17 25 EGEE GLITE USER S GUIDE EGEE TECH 570771n10 Enabling Grids gLite I O for E sciencE Date March 23 2005 offset IN new byte offset to seek to whence IN relative file position it can be one of the following values SEE
29. ues are true and false Default value false NumberOfStreams Number of QUANTA tcp streams Default value 1 CacheLevel AliEn aiod Cache Level value Default value 7 INFSO RI 508833 PUBLIC 8 25 EGEE GLITE USER S GUIDE EGEE TECH S70771 410 Enabling Grids gLite I O for E sciencE Date March 23 2005 3 1 2 CONFIGURATION FILE This section contains an example of the gLite I O client configuration file All the hostname and path name values should be changed according to the environment where the gLite IO client is installed Content of glite io client properties xml lt xml version 1 0 encoding UTF 8 gt service name glite io client components component name io client init lt param gt lt name gt Server lt name gt lt value gt __hostname__ lt value gt lt param gt lt param gt lt name gt ServerPort lt name gt lt value gt 9999 lt value gt lt param gt lt param gt lt name gt EncryptName lt name gt lt value gt true lt value gt lt param gt lt param gt lt name gt EncryptData lt name gt lt value gt true lt value gt lt param gt lt init gt lt component gt components lt service gt An example of a logging configuration file glite io client log properties is log4j rootCategory DEBUG file log4j appender stdout org apache 10g4j ConsoleAppender log4j appender stdout layout org apache log4j PatternLayout log4j appender
Download Pdf Manuals
Related Search