Home

Java CoG Kit Manual

1. gt grid proxy info identity timeleft Destroys a user proxy if present The files containing proxies can be specified The dryrun option prints the names of files that would be destroyed without actually destroying them Usage syntax is gt grid proxy destroy dryrun filel file2 Displays information regarding the long term user certificate It can display the identity that the certificate represents the CA that signed the certificate the validity period and so on Usage syntax is gt grid cert info options 31 grid change pass phrase myproxy 4 4 3 Windows batch files Allows changing of the passphrase used to encrypt the long term private key of the user You need to enter your old passphrase Usage syntax is gt grid change pass phrase options Warning The grid change pass phrase tool echoes your old and new passphrases to the screen The reason for this is currently Java does not have a portable way of reading a passphrase securely from the console without echoing it to the screen first Any solution would sacrifice the portability of the code The only way to avoid this problem is to provide a graphical front end to grid change pass phrase Though this is not currently available we plan to develop this in future Allows storing and retrieval of credentials using a MyProxy server Supports var ious options like the hostname and port number of the server the lifetime of the delegated proxy etc
2. GSSCredential ACCEPT_ONLY Step 3 Load the gridmap file if available Xy GridMap gridMap new GridMap gridMap load gridMapFile Step 4 Start the server el gk new GateKeeperServer gssCred port Step 5 Set the gridmap and log files if needed e gk setGridMap gridMap if logFile null gk setLogFile logFile Step 6 Register with the job manager services Read it from the configuration file if specified otherwise enter the service name directly as shown Aj if props null gk registerServices props else gk registerService jobmanager org globus gatekeeper jobmanager ForkJobManagerService null The above example program is available at the following location jglobus org globus gatekeeper Gatekeeper java 8 2 4 Differences between Java and Globus Personal Gatekeeper The Java CoG Kit Personal Gatekeeper is compatible in many aspects to the Globus Personal Gatekeeper Also a globusrun tool from Globus 2 2 4 version can submit a job to the Java Personal Gatekeeper and get back the result The differences include limitations in our implementations They are given in Section 8 2 2 74 8 3 File Transfer Service 8 3 1 Limitations 8 3 2 Starting the Gass Server Globus Access to Secondary Storage GASS is a mechanism used to transfer data using the HTTP protocol A GASS server uses secure HTTP for authentication and data transfer A GASS server
3. lt cog install path gt will represent the lt cog devel gt build cog 1 1 directory You can now proceed to configure jglobus as shown in Section 3 6 This section will show you how to configure the Java CoG Kit After installing and eventually compiling the Java CoG Kit you will need to set the COoG_INSTALL_PATH environment variable which is used by various tools in side the Java CoG Kit to determine the installation location of the Java CoG Kit COG_INSTALL PATH should point to the lt cog install path gt directory The exact value of lt cog install path gt depends on the Java CoG Kit distribution that you chose to download and it has been explained in its respective installation subsec tion It is also highly recommended that you add the lt cog install path gt bin direc tory to your binary search path named PATH on most systems Most of the ex amples in this manual assume that you have done so If the binary search path is not updated to include the Java CoG Kit bin directory you will have to specify the path to the Java CoG Kit bin directory when running any of the executables shown in the examples Unix gt lt cog install path gt bin lt executable gt Windows gt lt drive letter gt lt cog install path gt bin lt executable gt 22 3 6 2 Time Synchronization Windows The Java CoG Kit requires that your date and time are properly set The recom mended way to do this is by synchronizing your system cl
4. GUI 74 2 Unix Shell Scripts se o 25 664 6 ee we Re ee 743 Windows batch files 7 4 4 Using the API to access MDS To SNEMA oia a e ae 7 6 Performance issues With MDS 76 1 Programming Issues lt o cy ca co eH eae ee ORS 7 7 Implementation Details of MDS 2 2 version 7 8 Differences between Java and Globus tool Server side Java CoG Kit 8 1 Introduction 3 2 Job Execution Service s d a caca e s aa Ce ee ee es S21 CONTEUTAI N session bb eee SR ea ee oS 82 2 Limitations oo lt o sore eee A we e ee aie e 60 60 60 60 61 61 61 61 61 62 62 62 64 64 68 69 70 71 71 72 72 72 73 73 10 8 3 8 2 3 Starting the personal gatekeeper 8 2 4 Differences between Java and Globus Personal Gatekeeper File Transfer Service ooe caa aie ee HR eR aS 331 Limitations se 6 ded Swed ie eR ee ed 3 2 Stang the Gass Server lt o e kk o Ge A 8 3 3 Differences between Java and Globus GASS service Production Tests with the Java CoG Kit g 9 2 oS 9 4 9 5 9 6 IM UCO N sei te a de Requirements spas roda rara ee eS Installation 2 ocios a e Bee CONTESTADO 2 as A A AR amp BS Host Table Format soso Ra be Running the Tests coo A e Epa GridAnt A Client side Grid Workflow System 10 1 10 2 10 3 10 4 10 5 10 6 MitroductGon pd Bh AA A dy Bed Grd Ant Tasks 2 ae s ah A ON ek hs 102
5. Nevertheless you might need to consider the JVM startup cost and other performance issues 69 7 6 1 Programming Issues Right NEW 15 March 2002 Retrieving information from MDS should be performed with thought and care You should be connecting to the MDS server only as long as the connection is required in order to avoid blocking the limited number of ports to an MDS server Hence it is better to disconnect from the server immediately As a connection takes usually some time it is sometimes better to perform a number of queries However you should avoid analysing the result between subsequent queries Instead you should analyse the queries once all queries have been performed or start a parallel thread Wrong This method blocks the port unnecessarily 1 Connect to the server 2 Query the server 3 Analyse and Display the results 4 Disconnect from the server Right This is the prefer ed method 1 Connect to the server 2 Query the server 3 Disconnect from the server 4 Analyse and Display the results For iterative procedures we recommend the same Wrong 1 Connect to the server Query the server Analyse and Display the results goto 2 until all queries done A A Disconnect from the server j Connect to the server Query the server goto 2 until all queries done Disconnect from the server ae Sb Analyse and Display the results Additionally a user need to think about the correlation bet
6. globus url copy supports FTP and GridFTP protocols It also supports HTTP and HTTPS protocols for GASS transfers The protocol specific URL formats are e FTP ftp lt user gt lt password gt lt host gt lt port gt lt file path gt e GridFTP gridftp lt host gt lt port gt lt file path gt e HTTP http lt host gt lt port gt lt file path gt e HTTPS https lt host gt lt port gt lt file path gt e Local files file lt file path gt please note the three slashes Notes 1 lt port gt is optional in all cases 42 5 2 3 Windows Batch Files 5 2 4 Java CoG Kit Shell 5 2 5 APIs 2 In case of FTP username and password should both be provided or omitted In case they are omitted an anonymous connection will be made 3 If the HTTP S URL is referring to a GASS server running on a Unix like operating system lt file path gt would be a hierarchical path relative to the root directory It will look like lt directory gt lt directory gt lt directory gt lt name gt For example home albert document tex 4 If the HTTP S URL is referring to a GASS server running on Windows lt file path gt would be of the form lt drive letter gt lt directory gt lt directory gt lt name gt For example c temp myfolder document txt For example to transfer a file from an anonymous FTP server to a GASS server running on a Windows machine gt globus url copy ftp ftp foo org ba
7. 1 QrOBRECWte ci aka a A ee eS 10 22 STGCOPY eso eee BAS Ra Installation ssc eek el See we a EWE Oe ek oS SECY o eos 4 edo bee ew ak Ge SR Edo ee ee a cs ea dit BS eS oe Si eee oe eS 10 5 1 SnidEXecute ok bw ee ee E 103 2 ENQUOPY sp cee pe wee ER Se wo ere Complex Example 2044245425404 45 0h e084 bees Program Options Al A2 A 3 A4 AS A 6 Al A 8 ElODOSZIAS o a s ow A eA A we ewe dd MIODUS BaSS SOIVED now yk RR SR ee ES globus gass server shutdown o globus personal gatekeeper o ooo o BIODUSTDA A A ee a is RS SIOPS UECH ae a ab POCO ha eke eG A A GES grid change pass phrase oo o 74 75 75 75 76 77 77 77 78 78 79 80 82 82 83 83 84 85 86 86 86 86 86 87 87 87 88 88 89 90 91 91 AD grid info search o o coce a a a a e 92 ALO grid Proxy destroy 4 5 54 eee ee ed ee Oe ee HE 93 Pell ond PEOR AO uti x BS Bh ed doe i EE ed he ER 93 ALZSTIO DIOR NADIE 0644 5 e Pea ee ee he oe 94 o E Oe Se eee Sts 95 Command overview 96 B 1 New Format forthetable c lt 0 012 lt lt 96 Frequently Asked Questions 97 Cal mstallaton corro PEA A eG eee 97 Co ECH lt a A e 4 Bs 97 C 2 1 General Grid security Questions 97 C 2 2 Questions related to user certificates and certificate authority 98 C 2 3 Questions related to proxy certificates 98 C 2 4 Questions related to host certificates
8. CoG Kit to access the Grid Information Services 7 4 1 Using Graphical User Interface GUD The LDAP Browser Editor provides a user friendly Windows Explorer like inter face to LDAP directories with tightly integrated browsing and editing capabilities It is entirely written in Java with the help JNDI class libraries It can connect to LDAP v2 and v3 servers Figure 7 2 shows the user interface of the LDAP browser editor E LDAP Browser Editor v2 8 Idap doegrid es net 389 0 Globus c US File Edit View LDIF Help s e e aja o x le ia aTa T4 o Globus c US E o Globus c US doegrid es net24411 Q CI o DOE Science Grid doegrid es net Attribute Value ou Directory Administrators description Entities with administrative access t objectclass top objectclass organizationalunit 26389 Glou Gra SortTree D en d Refresh lators C ou Peop Manager D ou Special Users 9 C o National Aeronautics and Space Admi C ou Ames Research Center lr VEA DO 22 gt Ready Figure 7 2 LDAP Browser Homepage 2 http www mcs anl gov gawor ldap Out of historical reasons the browser is not distributed with the Java CoG Kit This may change in the future Using Web Browser User friendly Web browser access to Information Services can also be provided through a set of PHP scripts on a PHP enabled Web server These PHP scripts can be added to any Web page and
9. Displays usage version Displays version A 4 globus personal gatekeeper The command globus personal gatekeeper can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus it has the same options as listed below Syntax java Gatekeeper options java Gatekeeper version java Gatekeeper help Options help usage Displays usage p port Port of the Gatekeeper d debug Enable debug mode s services Specifies services configuration file i Srog Specifies log file gridmap 88 A 5 Specifies gridmap file proxy Proxy credentials to use serverKey Specifies private key to be used with serverCert serverCert Specifies certificate to be used with serverKey caCertDir Specifies locations directory or files of trusted globusrun CA certificates The command globusrun can be found in the build cog 1 1a bin direc tory This commandline provides a convenient wrapper to a Java class Thus it has the same options as listed below Syntax java GlobusRun options RSL String java GlobusRun version java GlobusRun help Options help HY usage Display help version Display version f lt rsl filename gt file lt rsl filename gt Read RSL from the local file lt rsl filename gt The RSL must be a single job request quiet Quiet mode do not print diagnostic messages output enable Use the
10. and gridmap files 98 Cass MYPOI os il BA dee Be eek dee S 98 C26 Miscellaneous cocos Re ES 98 C3 Pile YO and Transier o s oo cos chp hao ee a Rae at 99 Gal OVANEN eae ee Sede a eee SBS 99 C32 GudPTP vad raice ee E a Ewe a E be 99 Cao GASS ce coa a ep ae Ee e ew aA 100 C34 Version differences cn s 5 40 44824 84845 2 ees 100 CA Job Exec tion es esd esiu ee AR eee me 100 EAT GRAM Lust ee ee p Pe ES PERE oe eS 100 C3 Grid Information Service s r e bbe dee ead e 101 Col GEES eses daie AA a da 101 C5 2 Architecture of MDS 24 4 ee rias eS 101 C 5 3 SecurityinMDS isc se eke eee ryh ae kp 102 C 5 4 Retrieving information from MDS 102 C 5 5 Performace Issues with MDS 102 C 6 Server Side Java CoG Kit ee oeoa e eaka e epa 102 COl General oros Oa a da 102 C62 Job Execution Service cos ee Re RR ee we ee 103 C63 GASS SEED o sc cg ye a eh SR ie Bae 103 CI GridAnt 1 License 1 1 General Comments The Java CoG Kit is distributed under the Globus Toolkit Public License GTPL which is listed in Section 1 2 We kindly ask you to notify us about projects that you develop with the help of the Java CoG Kit This will allow us to keep track of the use of the Java CoG Kit as this directly affects our ability to motivate additional coding activities Please be so kind to send an e mail to gregor mcs anl gov with the subject JAVA COG KIT USGAE or fill out a form at Form http www unix globus org cog
11. be tampered with During mutual authentication both parties present their certificates to each other For the authentication handshake to take place the parties need to trust the Certi fying Authorities of each other In GSL the private key of a user is stored on the user s machine In order to protect it it is never stored in its original form but is encrypted using a passphrase pro vided by the user Before it can be used it needs to be decrypted Thus whenever a user wants to authenticate to a resource he she has to provide the passphrase in order to decrypt the private key This can be very inconvenient since a Grid computation typically involves obtaining access to many different computational and data resources The need to enter the passphrase repeatedly can be avoided by creating what is called a security proxy hereafter referred to as proxy A proxy contains a public and private key pair different than the key pair that belongs to the user The proxy key pair is used during any authentication dialogs A proxy has a limited lifetime after which the keys are not valid Thus even if the private key gets compromised the damage would be limited This allows storing the private key of the proxy without encrypting it with a passphrase Thus there is no passphrase for the proxy A new certificate is generated for the proxy It contains a mapping between the user s identity slightly modified to denote that it is a pr
12. cog install path gt bin direc tory proxy init gt cog shell Currently the Java CoG Kit shell provides a single command called proxy init that presents a GUI to create a user proxy This GUI is the same as the one ex plained in Section 4 4 1 4 4 5 API In this section we describe selected security APIs provided by the Java CoG Kit Please note that these APIs are different from and not backwards compatible with the APIs in the previous versions of the Java CoG Kit For the convenience of developers used to the old APIs we also provide a comparison of the old and new library APIs in this section Reasons for changing the security library API 1 The old security library was based on a commercial SSL library IAIK which had licensing restrictions not suitable for many of the Java CoG Kit users The old security library was socket oriented it was difficult to write non socket based security modules e g for FTP MDS etc The old security library API was not designed to work with multiple security protocols represent different types of credentials etc Functionality provided by the new library The new security library is based on GSS API and is implemented entirely with open source SSL and certificate processing libraries With the GSS API abstrac tions it is possible to provide transport and security protocol independence Also the new library supports a few new features such as the new proxy certific
13. command to be executed on the Grid resource A boolean flag that specifies if the executable resides on the client machine The default is false Specifies the arguments to be provided with the executed command Specifies the remote directory in which the command is to be executed l This will eventually be an optional parameter and extended by additional optional parameters to more easily specify a task that is GT2 and GT3 portable The additional parameters are server lt hostname gt port lt portnumber gt provider GT3 We call this formulation the uniform hosting environment formulation 83 environment Specifies the environment variables to be set prior to the execution of the command outputFile Specifies the file name to which the output must be redirected If left blank or not specified the output is streamed to the standard output By default output is streamed to the standard output errorFile Specifies the file name to which the error messages must be redirected If left blank the errors are streamed to the standard error By default the errors are streamed to the standard error redirect A boolean flag that specifies if the output and error streams are to be redi rected to the client Default value is true For example assume we like to schedule a job on the machine hot anl gov through port 8080 lt gridExecute factorylocation http hot anl gov 8080 SecureJobManagerFactory security xmlEnc del
14. file system space on all the machines registered on the GIIS needs to be displayed Assume that cold mcs anl gov and hot mcs anl gov are registered on GIIS Part of the output would look as follows dn Mds Host hn hot mcs anl gov Mds Vo name site o Grid Mds Fs freeMB 10 Mds Fs freeMB 21 Mds Fs freeMB 270 Mds Fs freeMB 341 Mds Fs freeMB 4388 Mds Fs freeMB 47 Mds Fs freeMB 73 dn Mds Host hn cold mcs anl gov Mds Vo name site o Grid Mds Fs freeMB 10 Mds Fs freeMB 21 Mds Fs freeMB 270 Mds Fs freeMB 341 Mds Fs freeMB 4388 B B Mds Fs freeM 47 Mds Fs freeM 73 Query CPU data on a single machine on a GIIS This example shows how to query for CPU model and speed on a single machine on a GHS The command is as follows gt grid info search x h giis mcs anl gov p 2135 b Mds Vo name site o Grid amp objectclass MdsCpu Mds Host hn co1d mcs anl gov Mds Cpu model Mds Cpu speedMHz Here we are querying a GIIS server but we specify the name of a single machine cold mcs anl gov in which we are interested So it retrieves the CPU model and speed of that singe machine only The output for the above query is given below dn Mds Host hn cold mcs anl gov Mds Vo name site o Grid Mds Cpu model Pentium III Coppermine Mds Cpu speedMHz 866 7 4 3 Windows batch files The grid info search batch file available for windows machines available in the lt cog install path gt bin directory performs the
15. keystore file Defaults to globus jks A 2 globus gass server The command globus gass server can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus 1t has the same options as listed below Syntax Java GassServer options java GassServer version java GassServer help Options help usage Displays usage s silent Enable silent mode Don t output server URL read Enable read access to the local file system w write Enable write access to the local file system r 87 Enable stdout redirection Enable stderr redirection client shutdown Allow client to trigger shutdown the GASS server See globus gass server shutdown p lt port gt port lt port gt Start the GASS server using the specified port i insecure Start the GASS server without security n lt options gt Disable lt options gt which is a string consisting of one or many of the letters crwoe A 3 globus gass server shutdown The command globus gass server shutdown can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus it has the same options as listed below Syntax java GassServerShutdown usage version lt GASS URL gt java GassServerShutdown help Allows the user to shut down a remotely running GASS server started with client shutdown permissions option c Options help usage
16. made on several levels including the definition of the terminology the design of an architecture and framework the application in the scientific problem solving process and the creation of physical instantiations of Grids on a production level A small overview about the Grid can be found in a draft paper entitled Gestalt of the Grid Article http www mcs anl gov gregor bib papers vonLaszewski gestalt paz This article provides an overview of important influences developments and tech nologies that are shaping state of the art Grid computing In particular we address the following questions What motivates the Grid approach What is a Grid What is the architecture of a Grid Which Grid research activities are performed How do researchers use a Grid What will the future bring Other CoG Kit related papers can be found at References von Laszewski http www mcs anl gov gregor bib 2 1 Intended Audience This manual is intended for the intermediate Grid programmer that would like to access the Globus Toolkit functionality through Java We assume that the reader of this manual is familiar with Java If not general information about Java is available through the Web site at SUN Microsystems or at IBM SUN http java sun com IBM http www ibm com java In general this manual serves as a basic introduction to a subset of functionality provided by the Java CoG Kit This manual does not explain every package cla
17. one or sub search scope P version protocol version default 3 1 limit time limit in seconds for search z limit size limit in entries for search Y mech SASL mechanism D binddn bind DN v run in verbose mode diagnostics to standard output 0 props SASL security properties auth auth conf auth int w passwd bind password for simple authentication 92 A 10 grid proxy destroy The command grid proxy destroy can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus 1t has the same options as listed below Syntax A 11 grid proxy info java ProxyDestroy dryrun filel java ProxyDestroy help Options help usage Displays usage dryrun Prints what files would have been destroyed filel file2 Destroys files listed The command grid proxy info can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus 1t has the same options as listed below Syntax java ProxyInfo options java ProxyInfo help Options help usage Displays usage file lt proxyfile gt f Non standard location of proxy printoptions Prints information about proxy exists options e Returns 0 if valid proxy exists 1 otherwise globus Prints information in globus format printoptions subject Distinguished name DN of subject issuer DN of issuer certificate signer identi
18. order to use GSI while using the Netscape Library or the JNDI library are given below 66 1 javax security sasl client pkgs This property has to be set to org globus mds gsi netscape while using Netscape Directory SDK and org globus mds gsi jndi while using the JNDI LDAP provider It basically specifies the package that provides implementation for the SASL mechanism 2 javax security sasl qop It specifies what Quality Of Protection QOP to use Itis a list of QOP values put in order of preference Allowed QOP values are auth authentication only auth int authentication with integrity protection GSI without encryption auth conf authentication with integrity and privacy protections GSI with en cryption If the property is not specified it defaults to auth 3 javax naming Context SECURITY_CREDENTIALS It specifies the credentials to use for SASL authentication If the property is not set the default credentials will be used 4 javax security sasl strength It specifies the strength of encryption But it is currently not used by the library Authenticated Access Using JNDI LDAP library In order to establish authenticated access to MDS using JNDI you need version 1 2 3 or above of the JNDI LDAP library For setting up a secure connection with MDS replace the Step 2 of Anonymous Access Using JNDI LDAP library Section 7 4 4 with the following steps This property specifies where the i
19. perform MDS queries to gather basic information The scripts can be easily adapted to show the summary data needed by a project For information on PHP please refer to the following link PHP http www php net Example Web Interface The Globus project maintains an MDS index node giis available for anyone to query The web interface for this node is available at the following link Globus giis http giis globus org ldapbrowser login php 7 4 2 Unix Shell scripts The Java CoG Kit provides a Unix shell script grid info search in the lt cog install path gt bin directory The tool has the following syntax 62 gt grid info search options search_filter attributes The usage messages for this command are available the Appendix A of this manual The following examples describe some of the ways of using the grid info search command Query all objects on GRIS Query file system space on a GHS This example shows how to display all of the data objects and resources on a single machine set up as a GRIS Assume the machine hot mcs anl gov has a GRIS service running at port 2135 The command will be given as follows gt grid info search x h hot mcs anl gov p 2135 b Mds Vo name local o Grid objectclass The option x is used to denote anonymous access and denotes the branch point A branch point is the location in the directory from which to start the search The default branch point for GRIS service
20. policies The file maps Grid identities to local usernames on that machine A user is authorized to use a service only if the user s Grid identity can be mapped to his local username using the grid mapfile Thus before you can use any Globus service you have to request the Grid ad ministrator to add you to the grid mapfile The procedure is site specific Please contact the Grid administrator for details Please follow the steps given below in order to protect your security credentials e Make sure that the only permission on your long term private key file userkey pem is the read permission for yourself This should be the case by default and you should never change that 28 4 3 MyProxy e Make sure that the only permissions on your proxy file are read and write permissions for yourself This should be the case by default and you should never change that e If you are running a Globus service make sure that the only permission on the long term private key of the host hostkey pem is the read permission for the superuser root administrator This should be the case by default and you should never change that e In the process of acquiring user credentials you are prompted to enter a passphrase This passphrase is used to encrypt your long term private key Please make sure to select a passphrase that is easy to remember for you but still very difficult to guess for others If by mistake you have chosen a weak passphrase d
21. server needs to be started on the local machine This can be done by instantiating a new GassServer object The first argument passed to the constructor enables or disables the starting of the GASS server in secure mode In the example below the GASS server is started in secure mode The second argument indicates the port on which the GASS server will listen to incoming connections If the second argument is set to zero a port will be chosen automatically The geturz retrieves the URL associated with the GASS server for further reference GassServer gass new GassServer true 0 String gassUrl gass getURL The resulting GASS URL must be passed to the GRAM server through the RSL description for the job Suppose your gassUrl is https 140 221 10 38 4678 and you want to run an executable called c a exe The resulting RSL description would contain the following amp rsl_substitution GLOBUSRUN_GASS_URL https 140 221 10 38 4678 executable GLOBUS_GASS_URL c a exe Retrieving Output and Error Messages for Jobs For batch jobs the output and error streams are redirected to remote files which are not retrieved after the job terminates To avoid this output and error streams can be redirected to the client machine using GASS To redirect the output to the client machine you need to pass the GASS URL to the server through the stdout and stderr parameters in the job RSL This will stream the job output and error strea
22. tar gz gt wewe globus org eog java 1 1l cegq 1 l sre tar gz e zip archive cog 1 1 src zip www globus org cog java 1 1 cog 1 1 src zip After downloading unpack the archive Unix gt tar xzf cog 1 1 src tar gz Windows Double click on the downloaded archive and extract it to a directory of your choice A directory named cog 1 1 will be created This directory will from now on be referred to as lt cog jglobus src gt You can now proceed to compile jglobus as described in Section 3 5 1 3 4 3 JGlobus Development Source The development version of jglobus can be retrieved from our source repository using anonymous CVS access We suggest that you first create a new directory in which to store the development version of jglobus For convenience this directory will be referred to as lt jglobus devel gt gt mkdir lt 3jglobus devel gt gt cd lt jglobus devel gt Login to the CVS server gt cvs d pserver anonymous cvs globus org home dsl cog CVS login Hit ENTER when you are asked for a password After the login step you can check out the jglobus module with the following command gt cvs d pserver anonymous cvs globus org home dsl cog CVs co r jglobus jgss jglobus 3 You need to have CVS installed on your system before downloading the jglobus development version 20 3 4 4 OGCE Stable Source Inside the lt jglobus devel gt directory another directory named jglobus will be created Th
23. x2 4 GB gsiftp 2 2 4 6224 9 6 Running the Tests In order to start the tests all you need to do is run the test script gt chmod x nightly test gt nightly test or gt bash nightly test A log file will be created in the location you chose during the configuration This log file will contain detailed information about the testing process You may need to check the log in case something goes terribly wrong The output produced by the tests will be available in the location specified through the HTMLOUTDIR variable The output directory will contain an index htm file which can be opened using a web browser The index file will contain link the tests performed together with information about the machine the tests were run on and the date and time these tests were performed A sample image of how one of the reports could look like is provided in Figure 9 1 Clicking on any of the links will either provide help with that item or display additional details 80 Java C oG Kit general tests Host machine dvorak mcs anl gov OS Linux 2 4 18 w1 686 unknown Date Thu Mar 27 12 12 17 CST 2003 0 49 30 IBMJava2 131 Time _ Host Os 0 14 43 grieg mcs anl gov Mandrake 7 2 2 4 17 0 14 55 shostakovich mcs anl gov RedHat 7 3 2 4 18 w1 0 15 2 wag
24. 2 8 Limitations of the Java CoG Kit 47 Mo USNS GASS 6 246 624 bee Bee eee Be Phas t bs 47 IJA GUD ereed araute EE REE eo ae eS 47 3 2 Unix Shell SCPE esposos eR Se eS 47 5 3 3 Windows Batch Files 48 5 3 4 JavaCoG Kit Shell 48 202 APIS cis ar a A 48 5 3 6 Limitations of the Java CoG Kit 51 6 Job Submission 52 6 1 Ditroduction db eee eee bees BSS 52 GLI Gatekeeper oe pesata esi a ee eR ee eS 52 612 JobiManager encia ee 52 6 1 3 Batch and Interactive Jobs 53 614 File STABIDE lt o ooo a eS 53 6 2 Globus Resource Specification Language RSL 53 Gl RSL SiGe 66 bide rr ee bie eed 54 6 2 2 RSLintheJavaCoGKit 55 63 Job Submision osorno bes Gal locos oa eae RP ERA ee ee eA 63 2 Ums Shell SApS os dy ee ha eae eee eS wes 6 3 3 Windows Batch Files 6 3 4 Java CoG ESB ooo ee AS 63 5 Job Submission API so cotorra 6 4 Differences from the C Globus Toolkit GAI A OAD RSL Parse comia ca a at e O G Accessing the Grid Information Service 7 1 Introduction 7 2 Architecture Tal GRIS ca is Se eS ie SR oe Bas Taa MURR ui fetes ge Meh Shee ete Bt Mie EA BBO we Bee 123 lt ea a OE ROO ee eS TA WorKME coco ea a a ew kopa T3 Secunty Wia MDS os coea a aa a ede HRS Vel Site PONCIES 2 c c pore a ee 7 4 Accessing Grid Information Services 7 4 1 Using Graphical User Interface
25. 7 Project Registration 8 RSL 53 Schema 68 Server 72 Testing 77 Third party transfer 45 Website 13 108
26. Appendix B Command overview B 1 New Format for the table The main ant build file for building the Java CoG Kit is build xm1 The help target present in each of the xml files gives the details of all the targets supported and their functionality The demos xm1 contain all the gui demos present in ogce and tools xml contains targets for running the command line tools using ant The following table gives an overview of the equivalent ant targets which are available for each of the scripts present in the lt cog install path gt bin directory in the alphabetical order Command Buildfile target Section globus gass server shutdown tools xml globus gass server shutdown A 3 globus gass server tools xml globus gass server A2 globus personal gatekeeper tools xml globus personal gatekeeper A 4 globus url copy tools xml globus url copy A 6 globus2jks N A N A Bel globusrun tools xml globusrun A 5 grid cert info tools xml grid cert info A 7 grid change pass phrase N A N A A 8 grid info search tools xml grid info search A 9 grid proxy destroy tools xml grid proxy destroy A 10 grid proxy info tools xml grid proxy info A 11 grid proxy init N A N A A 12 hellogridftp demos xml N A N A helloworld demos xml N A N A myproxy tools xml myproxy A 13 ogce setup demos xml setup N A setup demos xml old setup N A visual grid proxy init demos xml login N A 96 Appendix C Frequently Asked Questions C 1 Installation C 2 Security What are the requireme
27. Developmental 20645 486 eee a RS S22 DO 24 Oa eta y Raa acho pe ah hee Bee ee aS 3 3 Java CoG Kit Formats 2 2 260s bee ee ea e dom 201 Th elava CoG Kit Pats ocn re a 3 3 2 Stable and Development Distributions O ea op e eA ee ee ea i D286 Whatto Chogse ccs c 0n6 4 Ar ee eee BS 11 11 11 12 12 13 13 13 13 14 15 15 16 16 16 17 17 17 17 17 17 18 18 18 18 3 4 Downloading the Java COGKit 19 3 4 1 JGlobus Stable Binary 19 342 JGlobus Stable Source ceca 20 3 4 3 JGlobus Development Source 20 344 OGCE Stable Source o 6 be eee ea 21 3 4 53 OGCE Development Source 21 ID Compiling 5 ec repast be Eee ee Re ee 21 Jal Compiling IGIOD S 4c 6 6k hae eee AR 21 332 Compiling OGCE score BAS 22 6 Configuration occiso PR Re oe 22 36 1 Environment Variables lt lt 5 44 5 see ee 940 22 3 6 2 Time Synchronization ss o s s ees ne a ee ess 23 3 6 3 Globus Security Credentials 23 S64 Contigua s s eroe con ca Ae RR ee HAO 23 Security 25 4 1 Introduction eo coe coe epa b sep eee eee eee ee 25 4 1 1 Grid Security Infrastructure o 25 4 1 2 Certificates and certifying authorities 26 4 1 3 Proxies and delegati0N 26 4 2 Security Prerequistes cn 5 bv oe Ha ewe et 27 4 2 1 Acquiring a user certificate 27 4 2 2 Acquiring a host
28. GASS Server library to redirect standout output and standard error to globusrun Implies quiet server GLOBUSRUN_GASS_URL can be used to access files local to the submission machine via GASS Implies output enable and quiet write allow Enable the GASS Server library and allow writing to GASS URLs Implies server and quiet r lt resource manager gt resource manager lt resource manager gt Submit the RSL job request to the specified resource manager A resource manager can be specified in the following ways host y host port host port service host service host service host subject host port subject host service subject host service subject host port service subject For those resource manager contacts which omit the port service or subject field the following defaults are used port 2119 service jobmanager subject subject based on hostname This is a required argument when submitting a single RSL request 89 A 6 globus url copy k kill lt job ID gt Kill a disconnected globusrun job status lt job ID gt Print the current status of the specified job b batch Cause globusrun to terminate after the job is successfully submitted without waiting for its completion Useful for batch jobs This option cannot be used together with either server or interactive and is also incompatible with multi request jobs The handle or job ID of the su
29. MHz For a detailed description of object classes and their attributes refer to the following webpage Schemas http www globus org mds Schema html For the syntax of the schemas refer to RFC 2252 available at the following loca tion Syntax http www ietf org ric rfic2252 txt 7 6 Performance issues with MDS The performace of a query depends upon the Information Providers used and the amount of time the data is live and cached When a query to a GRIS arrives it will be answered very quickly if the data requested is live and cached If the data requested has been flushed from the cache because it has expired the GRIS server will invoke the information providers to fetch the information The time taken to deliver depends on the time taken by these providers The performance of a query to a GIIS is dependent upon the performance of the GRIS s that it accesses as well as the amount of time the data is live and cached When a query to a GIIS arrives it will be answered very quickly if the data is present in the cache Otherwise the GIIS might query a GRIS that supplies the information In short there is no appropriate formula for predicting the performance for a query to MDS As the GIIS hierarchy becomes more complex the performance becomes more unpredictable The performance of IPs have a great impact on the performance of a query in general It is possible to write a server side MDS information provider executable in Java
30. Na tional Accelerator Laboratory Chicago Oct 16 20 2000 pp 161 163 http www globus org research papers ACAT3 pdf 38 8 GridFTP Web Page Online Available http www globus org datagrid gridftp html 38 9 J Bester I Foster C Kesselman J Tedesco and S Tuecke GASS A Data Movement and Access Service for Wide Area Computing Systems in Proceedings of IOPADS 99 Atlanta Georgia ACM Press May 1999 Online Available ftp ftp globus org pub globus papers gass pdf 38 10 Globus Access to Secondary Storage Web Page Online Available http www globus org gass 38 53 11 Dsniff A Tool for Penetration Testing Web Page Online Available http naughty monkey org dugsong dsniff 40 12 B Allcock and R Madduri Reliable File Transfer Service Web Page Online Available http www unix globus org ogsa docs alpha3 services reliable transfer html 40 42 13 RFC 2228 FTP Security Extensions Web Page Online Available http www ietf org rfc rfc2228 txt 46 105 14 15 16 19 20 GRAM Job Manager Reference Manual Online Available http www globus org api c globus 2 2 globus_gram_job_manager html main html 52 The Monitoring and Discovery Service Web Page Online Available http www globus org mds 60 G von Laszewski S Fitzgerald I Foster C Kesselman W Smith and S Tuecke A Directory Servi
31. String remoteFilel testDir getFile txt File localFilel new File getFile txt hotClient get remoteFilel localFilel Send a file to the remote server 44 E boolean append true String remoteFile2 testDir putFile txt File localFile2 new File putFile txt hotClient put localFile2 remoteFile2 append Third party transfers Following is an example showing a third party transfer between two GridFTP servers namely hot mcs anl gov and cold mcs anl gov The former is assumed to be the source of the file and the latter is assumed to be the destination Create a GridFTPClient object for cold mcs anl gov and perform authentication El GridFTPClient coldClient new GridFTPClient cold mcs anl gov 2811 coldClient authenticate credential Set the data channel authentication and protection parameters as shown above for hotClient EY The following step is optional unless using Extended Block Mode It is performed here for illustrative purposes Set the receiving server to passive mode so that it starts listening for a data channel connection on any available port Set the sending server to active mode providing it with the above mentioned port and the hostname of the receiving server so that the sending server can open a data channel connection to the receiving server These operations if performed have to be in that order Ea RP ae HostPort h
32. The Java CoG Kit User Manual Draft Version 1 1 MCS Technical Memorandum ANL MCS TM 259 Revisions March 14 2003 July 18 2003 Gregor von Laszewski Beulah Alunkal Kaizar Amin Jarek Gawor Mihael Hategan Sandeep Nijsure Argonne National Laboratory Mathematics and Computer Science Division 9700 S Cass Ave Argonne IL 60439 Coresponding Author 630 252 0472 gregor mcs anl gov Location of Manual http www globus org cog manual user pdf Be kind to your environment and do not print this frequently changing manual c Argonne National Laboratory All rights reserved January 30 2004 Contents 1 License 1 1 General Comments ias eS Bee Rae ee 1 2 Globus Toolkit Public License GTPL L3 Omer Licences oaceae a S E E Sot eee eS LT JEODUS oce sanp ee he we i SR Eae Be Lao Oeae iung o Ne Ee HES BAR ad BS eS 2 Preface 21 Mmtendged Anding sy p Sa ee Be ee ew G Ded WRCSOUICES as e ae Go e E a ao ai AS 221 Project Website o q e r ke ea e a ea Se 222 Bug Reporting lt esc e sewe desi sente ni 223 Malh LIIS e ee kaa aoe e Re eS 2 2 4 Sourcecode Repository 2 3 Manual Guidelines e e Zoek Conventions o era 232 Cribs i nia ee eR 2 4 Administrative Contact 23 Acknowledginents 4 5 23 66k Se Rk ad oe SOE 3 Installation Dol rs se 5 GE ee SARS ae EAS amp BALES wy eS 32 Requif ements 0 244 566 4 46 Bebe ea A ee 321 Java
33. Usage syntax is gt myproxy options command where command is one of put get anonget destroy and info anonget is used for anonymous retrievals For example to store a proxy to host myproxy mcs anl gov With a validity period of 12 hours use gt myproxy h myproxy mcs anl gov c 12 put In case of the put command you will be prompted first for your Grid passphrase and then for the password to be used to protect the credential stored on the MyProxy server When you later try to retrieve the credential using get Or anonget Or any other method you will be asked to enter this password Warning The myproxy program echoes both your Grid passphrase and the cre dential password to the screen The reason is same as the one mentioned above regarding grid change pass phrase This problem will go away when we provide a graphical front end to this tool Each of the tools described in the previous section has a Windows batch file coun terpart These batch files can be found in the lt cog install path gt bin directory Just like the Unix shell scripts each of them supports a help option that prints a usage message The usage details have also been included in the Appendix A for this manual 32 4 4 4 Java CoG Kit shell The Java CoG Kit Shell is a convenience application that allows you to use several Java CoG Kit features from a platform independent command line interface To start the shell execute the following from the lt
34. aining all or a portion of the Software either verbatim or with modifications Each licensee is addressed as you or Licensee 2 The University of Southern California and the University of Chicago as Op erator of Argonne National Laboratory are copyright holders in the Software The copyright holders and their third party licensors hereby grant Licensee a royalty free nonexclusive license subject to the limitations stated herein and U S Government license rights 3 A copy or copies of the Software may be given to others if you meet the following conditions a Copies in source code must include the copyright notice and this li cense b Copies in binary form must include the copyright notice and this li cense in the documentation and or other materials provided with the copy 4 All advertising materials journal articles and documentation mentioning features derived from or use of the Software must display the following ac knowledgement This product includes software developed by and or derived from the Globus project http www globus org In the event that the product being advertised includes an intact Globus dis tribution with copyright and license included then this clause is waived 5 You are encouraged to package modifications to the Software separately as patches to the Software 6 You may make modifications to the Software however if you modify a copy or copies of the Software or an
35. ajor parts jglobus and ogce The Java CoG Kit is available in a number of formats that address different categories of users In the following sections we will try to explain which part and version is suitable for a certain type of user Please note that if you do not plan to compile the Java CoG Kit yourself you could just use the Java Runtime Environment The version requirements still apply 17 3 3 1 The Java CoG Kit Parts jglobus JGlobus contains just the basic components and API s to interface with GT2 0 and GT3 0 OGCE OGCE contains possible future enhancements and showcases that use of some of the features of jglobus 3 3 2 Stable and Development Distributions Stable Distribution The stable distribution is recommended for production environments It comes in two formats binary and source Development Distribution The development distribution contains the latest features of the Java CoG Kit but without being tested extensively The development version is only available from the source repository 3 3 3 Formats Java CoG Kit Binaries The binary format of the Java CoG Kit requires minimal effort for the installation process It is prepackaged in both tar gz and zip archives Java CoG Kit Sources The Java CoG Kit sources are available for users who wish to compile the Java CoG Kit themselves or wish to see the sources of the Java CoG Kit Java CoG Kit Source Repository The source repository contains
36. and Section 7 4 3 How do I invoke the MDS services using API Section 7 4 4 Do I have any problems using Netscape library to access MDS with GSI authentication Section 7 4 4 How can I hook up to the GSI security using JNDI Section 7 4 4 Is the LDAP browser integrated with the Java CoG Section 7 4 1 How do I choose selecting between the JNDI and netscape SDK Section 7 4 4 Why should I not keep a connection to the MDS for a long time Section 7 6 1 What is the difference in updating quality for GIIS and GRIS Section 7 6 Can I write GRIS and GIIS in java Section 7 6 What performace can I expect from MDS Section 7 6 Does the performance of GRIS affect the performance of GRIS Section 7 6 Does Java CoG Kit provide any server side implementations Section 8 1 102 C 6 2 Job Execution Service C 6 3 GASS server What are the server side functionalities provided by Java CoG Kit Section 8 1 Where do I find detailed information regarding the Globus server side func tionalities Section 8 1 What is a personal gatekeeper Section 8 2 What is a Job Manager Service Section 8 2 How do I configure my personal gatekeeper Section 8 2 1 Are there any limitations in Java CoG Kit Job Execution Service Section 8 2 2 Can I allow mulitple uses to access my personal gatekeeper and submit their jobs to it Section 8 2 What are the different ways of starting up the pers
37. at authenticates and authorizes the execution of a service It receives requests from clients and performs mutual authentication with the client After authenticating and authorizing it starts a job manager run ning under the credentials of the authenticated user A gridmap file is used by the gatekeeper to map Globus credentials to local users Figure 6 1 shows a schematic representation of this process The Java CoG Kit provides a personal gatekeeper that can be used as a lightweight alternative to the Globus gatekeeper Details about the differences between the personal gatekeeper and the Globus gatekeeper can be found in Section 6 4 1 A job manager is spawned by the gatekeeper upon receiving each request The job manager processes job specifications sent by the clients most of which result in a job submission to a local scheduler It also provides a mechanism through which the client can check the status of a job or cancel it More information about the job manager can be found at 14 52 User 1 Job Manager Client User 2 User 2 Client Job Manager tee eee eee SAO 2 Job Manager Client Figure 6 1 Gatekeeper Architecture 6 1 3 Batch and Interactive Jobs 6 1 4 File Staging Job execution can be done in two major ways batch and interactive Interactive jobs provide immediate feedback to the user With interactive jobs the
38. ate for mat and delegation at any time API For a detailed list of GSS API implementa tion features and limitations please see the following webpage Java GSI GSS API Implementation http www globus org cog distribution 1 1 api org globus gsi gssapi Java_GSI_GSSAPI html Key differences between old and new library 1 GSS abstractions are used throughout the code instead of the old security API e g previously setCredential org globus security GlobusProxy and now setCredential org ietf jgss GSSCredential 2 All the security classes in the org globus security package and all subpack ages except org globus security gridmap package are now deprecated 33 3 The functionality of the org globus security GlobusProxy class is mostly re placed by org globus gsi GlobusCredential class However it is strongly rec ommended not to use if possible org globus gsi GlobusCredential class as it is security protocol specific representation of PKI credentials Instead it is recommended to use the GSS abstractions as much as possible as shown in the sample code in this section Getting default user proxy credentials Versions of Java CoG Kit before 1 1a GlobusProxy cred GlobusProxy getDefaultUserProxy Java CoG Kit 1 1a ExtendedGSSManager manager ExtendedGSSManager ExtendedGSSManager getInstance GSSCredential cred manager createCredential GSSCredential INITIATE_AND_ACCEPT Saving credentials in a fi
39. atekeeper are available in Section 6 1 72 8 2 1 Configuration 8 2 2 Limitations The Personal Gatekeeper supports two configuration files One of them is used for configuring specific jobmanagers such as fork pbs etc and the other is a gridmap file used for authorizing different users to use the service Both these files can be specified either through the program or by using the command line tools which are described in the next section If the configuration file for job managers is not spec ified the gatekeeper starts the default fork job manager A sample configuration file for specifying job managers is available at jglobus org globus gatekeeper services conf A gridmap file consists of single line entries listing a certificate subject and a userid like this 0 Grid O Globus OU your domain CN Your Name userid where subject name refers to the subject that appears on your certificate and a userid refers to your account login name on the server machine When a client connects to the gatekeeper the subject of the certificate will be searched in the gridmap file If it is not found the connection is rejected If the subject name is found the connection is allowed This file need not be specified if the Gatekeeper is used by a single user For more information on gridmap file refer to Section 4 2 5 Currently the implementation does not support e Caching of files using gass cache e Running services as authenticated u
40. atible with older servers such as Globus Toolkit 2 2 2 0 Thus they will not work for majority of the examples in this manual 30 grid proxy info grid proxy destroy grid cert info To generate a proxy that is compatible with Globus Toolkit 2 2 and 2 0 servers either use the visual grid proxy init tool described in the previous section or use the old option for this tool Other options supported by this tool are lifetime strength policy file etc This tool prompts you for your passphrase Usage syntax is gt grid proxy init options For example to create a proxy that will work with Globus Toolkit 2 2 and 2 0 servers has a validity period of 12 hours and contains 1024 bit keys gt grid proxy init old hours 12 bits 1024 Warning The grid proxy init tool echoes your passphrase to the screen The rea son for this is currently Java does not have a portable way of reading the passphrase securely from the console without echoing it to the screen first Any solution would sacrifice the portability of the code If you want to avoid this behavior please use the visual grid proxy init tool described in the previous section Displays information regarding a proxy It can display various pieces of informa tion such as the issuer Distinguished Name DN of the identity time left on the proxy and so on Usage syntax is gt grid proxy info options For example to observe the identity and validity period of a proxy use
41. ble a out More complicated resource descriptions can be build from the basic relations using compound requests and value sequences Compound re quest can be formed using conjunction disjunction or multi request Value se quences are used to express ordered sets of values The value sequence syntax is used primarily for defining variables and for providing the argument list for a program The operator can be used to denote a conjunct request RSLAttributes Following is a list of commonly used attribute names used in conjunction with GRAM executable describes the application to be executed directory represents the remote working directory used for the execution of the job arguments sets the command line arguments that will be passed to the executable stdin allows input redirection for the job from a file stdout allows output redirection for the job stderr specifies the redirection of the error stream For a complete set of GRAM attributes please consult the following link GRAM RSL parameters http www globus org gram gram_rsl_parameters html Examples e Typical GRAM resource descriptions contain at least a few relations in a conjunction this is a comment amp executable a out lt that is an unquoted literal directory home albert arguments argl arg 2 count 1 e Substitutions can be used to make sure the same substring is used multiple times in a resource descri
42. bmitted job will be written on stdout stop manager lt job ID gt Cause globusrun to stop the job manager without killing the job If the save_state RSL attribute is present then a job manager can be restarted by using the restart RSL attribute fulldelegation Perform full delegation when submitting jobs Diagnostic Options p parse Parse and validate the RSL only Does not submit the job to a GRAM gatekeeper Multi requests are not supported a authenticate only Submit a gatekeeper ping request only Do not parse the RSL or submit the job request Requires the resource manger argument d dryrun Submit the RSL to the job manager as a dryrun test The request will be parsed and authenticated The job manager will execute all of the preliminary operations and stop just before the job request would be executed Not Supported Options n no interrupt The command globus url copy can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus 1t has the same options as listed below Syntax java GlobusUrlCopy options fromURL toURL java GlobusUrlCopy help Options s lt subject gt subject lt subject gt Use this subject to match with both the source and destination servers ss lt subject gt source subject lt subject gt Use this subject to match with the source server ds lt subject gt dest subject lt subject gt Use this subject to match
43. ce for Configuring High Performance Distributed Computations in Proceedings of the 6th IEEE Symposium on High Performance Distributed Computing 5 8 Aug 1997 pp 365 375 Online Available http www mcs anl gov gregor papers fitzgerald hpdc97 pdf 60 Globus Toolkit 2 2 MDS Technology Brief Web Page Online Available http www globus org mds mdstechnologybrief_draft4 pdf 60 Open Grid Services Architecture OGSA Web Page Online Available http www globus org ogsa 82 85 DAGMan Directed Acyclic Graph Manager Web Page Online Available http www cs wisc edu condor dagman 83 BPEL4WS Business Process Execution Language for Web Services Version 1 0 Web Page Online Available http www 106 ibm com developerworks webservices library ws bpel 83 XLANG Web Services for Business Process Design Web Page Online Available http www gotdotnet com team xml_wsspecs xlang c default htm 83 Web Services Flow Language WSFL Web Page Online Available www ibm com software solutions webservices pdf WSFL pdf 83 G von Laszewski I Foster J Gawor and P Lane A Java Commodity Grid Kit Concurrency and Computation Practice and Experience vol 13 no 8 9 pp 643 662 2001 Online Available http www mcs anl gov gregor papers vonLaszewski cog cpe final pdf 106 Index Acknowledgments 16 Administrative Contact 16 ant 17 Bugs 13 Clo
44. certificate Section 4 2 4 What information can I get from a certificate How Please see the grid cert info tool in Section 4 4 2 and 4 4 3 Questions related to proxy certificates How do I create renew destroy a proxy Please see the tools described in Section 4 4 2 and 4 4 3 How do I get information about my proxy Please see the grid proxy info tool in Section 4 4 2 and 4 4 3 Questions related to host certificates and gridmap files MyProxy Miscellaneous How do I get added to the gridmap file Section 4 2 5 how to store retrieve credentials to myProxy Please see the myproxy tool in Section 4 4 2 and 4 4 3 How do I configure the Java CoG Kit with the security files Please see the Java CoG Kit configuration wizard in Section 4 4 1 Does the Java CoG Kit provide API for security related tasks How do I use them Section 4 4 5 Portability and design issues regarding the security API in previous versions of the Java CoG Kit and in version 1 la Section 4 4 5 98 C 3 File I O and Transfer C 3 1 Overview C 3 2 GridFTP I m getting following error when connecting to a gatekeeper Server cer tificate rejected by Chain Verifier What does it mean and how can I fix it In most cases it means that the client either does not trust or does not have the CA certificate that signed the server certificate Please see the Section 4 2 4 I m getting following error when connecting t
45. certificate optional 27 4 2 4 Renewing a certificate so 224 4 64s Gee be ERS 28 4 2 4 Obtaining the certificates of the trusted CAs 28 4 25 Gridmapiles s lt lt es cae ekee ke ae ee Ho 28 4 2 6 Protecting credentials seas oe Se SS ES 28 45 MYyPPOXy o 6 6 064468 465 406 babe eae ee 29 4 4 Managing certificates and proxies 30 A Mil as a Bid a Seeds Soe GE Ai A ey he EE 30 4 4 2 Unix shell ScriptS 30 4 4 3 Windows batch files 32 444 JavaCoG Kitshell 33 445 APL swe eee eee we ee A 33 4 5 Firewall Issues oso race A HE 36 4 6 Random number generation issues 37 5 File VO and Transfer 38 dl Tamoduchom con dd a dra BO 38 5 1 1 Requirements for File Access and Transfer over the Grid 38 Se AE AER A ee ow 38 MALS GASS ok eee ea eR EE RR eR eS a a A 39 5 1 4 Other file transfer mechanisms 40 5 1 5 Security Requirements 2 64 55 eee e 40 32 USMECGHIFRDO cocte cane ed eee Se eR Eee oS 40 32d GUL e ea be dy Se eS E Re Bas 40 S22 Ums Swel Scripts csi doy et bw eS eee eS 42 5 2 3 Windows Batch Files 43 3 24 JavaCoG Kit Shell c0 43 i APD regete E we Rg Saad bw A EE SE BIS 43 5 2 6 Differences between Java CoG Kit version 0 9 13 and 1 la 46 5 2 7 FTP GridFTP protocol features supported by the Java CoG I ne a Re ee ae eh ee eae we 46 5
46. certificate is a matter of generating a public private key pair and sending it to the CA for identity verification and signing Thus it is site specific as explained in the previous subsection Please note that for acquiring a host certificate you need to have administrative privileges on the host Using Globus tools to acquire a host certificate Please refer to the following webpage Acquiring GSI certificates http www globus org security vl 1 certs html Follow the hyperlink Host certificates for instructions about acquiring a host certificate As in the previous subsection please replace the Globus specific pro cedures with those specific to your site After you receive the certificate from the CA please store the certificate and private key with appropriate file permissions in the host certificate directory as mentioned on the above webpage 27 4 2 3 Renewing a certificate You will get a notification from the CA when your user or host certificate is about to expire Renewing a new certificate involves generating a new public private key pair creating a renewal request and getting it signed from the CA Thus it is partly site specific as explained before Globus provides a tool called globus cert renew for this Please refer to the following webpage for a documentation of this tool Renewing a GSI Certificate http www globus org details programs globus cert renew html 4 2 4 Obtaining the certificates of the tr
47. ch and develop ment This work would not have been possible without the help of Ian Foster and the Globus Project team 16 3 Installation 3 1 Introduction 3 2 Requirements 3 2 1 Java Development Kit 3 2 2 Ant In this chapter you will learn how to to download install and configure the Java CoG Kit Installation is the first step that needs to be accomplished before the Java CoG Kit can be used It ensures that the Java CoG Kit exists on your local machine in a proper state After installation configuration is needed to adjust various parame ters that are specific to your environment The Java CoG Kit has a minimal installation requirement In most cases it is only necessary to have a Java Virtual machine In case you also like to make use of the GridAnt system you will also need ant In order to be able to compile and run the Java CoG Kit you will need to have a recent version of the Java Development Kit The recommended version is 1 4 1 The minimum required version of the Java Development Kit is 1 3 1 JDK nttp java sun com The Java CoG Kit uses the Apache Ant build system At least version 1 5 2 of Apache Ant is required by the Java CoG Kit Please make sure that along with Ant you also install any libraries required by Ant The Ant binaries sources and information about Ant requirements can be found on the Ant web site 1 Ant http ant apache org 3 3 Java CoG Kit Formats The Java CoG Kit contains two m
48. ck synchronisation 23 cog properties 23 Command globus gass server 48 75 globus gass server shutdown 48 globus personal gatekeeper 73 globus url copy 48 globusrun 56 grid info search 62 Commands globus url copy 42 Contact 16 Contributors 16 FAQ 97 File I O 99 GASS Server 103 GRAM Server 103 GridAnt 104 gridFTP 99 Information Service 101 Installation 97 Job Execution 100 MDS 101 Security 97 Transfer 99 File I O 38 FIle Transfer Third party 45 File Transfer 38 File Transfer GUI 40 GASS 39 Gatekeeper 52 GIIS 61 GIS 60 Performance 69 schema 68 Use Cases 69 globus gass server 48 75 globus gass server shutdown 48 globus personal gatekeeper 73 globus url copy 42 48 globusrun 56 grid info search 62 GridAnt 82 gridCopy 84 gridExecute 83 Installation 85 Security 86 Tasks 83 GridFTP 38 API 43 GRIS 60 GUI File Transfer 40 Job Submission 55 Desktop 56 Form 55 LDAP Browser 62 Installation 17 Clock synchronisation 23 cog properties 23 configuration 23 IPs 61 JNDI Anonymous 65 Authenticate 67 Job Execution Service 72 Job Manager 52 Job Submission 52 API 57 LDAP Browser 62 License 8 bouncycastle 11 cryptix 11 Globus Toolkit 9 GPTL 9 junit 11 log4j 11 puretls 11 soaprmil 1 11 xerces 11 xml4j 11 107 Mailing List 13 MDS 60 Nescape SDK Anonymous 65 Netscape SDK Authenticate 67 Production testing 7
49. clients gridExecute Executes an arbitrary job on a remote machine using the Java Job Manager service provided by GT3 alpha gridCopy Provides third party file transfers between GridFtp enabled Grid resources using the Reliable File Transfer service provided by GT3 gridQuery Provides capabilities to query the service data of different Grid services This is a tentative list and is by no means final Neither have we implemented all of the above tasks The initial prototype for GridAnt has the functionality for job submission and file transfer Other tasks are under development We release the current version as a technology preview in order to obtain feedback and to engage the community in its further development 10 2 1 gridExecute The gridExecute task executes an arbitrary job on a Grid resource It requires the following input parameters specifies a mandatory argument factorylocation security delegation executable localExecutable arguments directory Specifies the location of the Java Job Manager factory service available in GT3 Specifies the XML security parameters Valid options are xmlSig and xmlEnc for XML signature and XML encryption respectively The default is XML signature Specifies the parameters for credential delegation for GSI security Valid options are full and limited for full delegation and limited delegation respec tively The default id limited delegation Specifies the
50. could be run as part of a job submission to transfer standard input and output files and prestage executables to remote servers as explained in Section 6 3 It can also be used to transfer data as explained in Section 5 3 The Java CoG Kit provides client and server GASS functionality It provides a pure Java Globus GASS server for transferring files via HTTPS The server is multi threaded and accepts HTTPS connection from GASS clients to copy from copy to and append to files that are local to the server It also provides a pure Java Globus GASS client for transferring files via HTTPS The GASS servers does not support the cache management functionality The GASS server can be invoked in any of the following ways Command Line To start the GASS server run the script or batch file globus gass server available in the lt cog install path gt bin directory as follows gt globus gass server If you wish to shut down the server using the command line tool you need to specify the option c or client shutdown While starting the server In that case server can be shut down using the globus gass server shutdown Let us assume that gass server is started on machine named hot mcs anl gov at port number 4573 It can be stopped using the following command gt globus gass server shutdown hot mcs anl gov 4573 Using the API It can be started from within a program using the API A sample code is shown below Step 1 Initialize the va
51. d by Date http www unix globus org mail_archive java maillist html 2 2 4 Sourcecode Repository Note that this list may result in daily mails sent by the Java CoG Kit community Please use the bug tracking system for reporting bugs If you use the bug track ing system your message has a higher chance of being answered There is no guarantee that we answer a mail sent to the Java CoG Kit mailing lists We maintain all source code in a CVS repository that can be accessed anony mously You can find more details about this in Section 3 4 5 14 2 3 Manual Guidelines This manual is constantly being improved and your input is highly appreciated Please report suggestion errors changes and new sections or chapters to this doc ument to Gregor von Laszewski gregor mcs anl gov When you report bugs please do not use page line or section numbers Remember new sections may appear due to community contributions Instead please quote the section title or make corrections by hand and FAX it to us Even better submit a corrected document as you can check out the manual through our CVS archive 2 3 1 Conventions If you see a or a in the text there is no reason to send us a report on it It simply means that the section to which we refer has not yet been integrated in this manual Regular text is written using the Times font Code examples use the Courier font For code example contributions we recommend not exceeding the
52. d info search in Java CoG are slightly different from those of the C version For example for the grid info search command we have not enabled config file option that specifies a different configuration file to obtain MDS defaults and nowrap option that passes the output through a line unwrapping filter first In Java CoG Kit implementation the search filter needs to be specified in order to get the results Whereas this is optional in C version For details please check grid info search help in both Java and C versions 71 8 Server side Java CoG Kit 8 1 Introduction This chapter gives an overview of how to start up Job Execution and File Transfer Services present in Java CoG Kit The Java CoG Kit provides client side as well as partial server side functionality for enabling operations on Grid While the other chapters of this manual focus on the client side functionality this chapter focuses on the server side functionality The Java CoG Kit provides experimental implementations of a Job Execution Service and a File Transfer Service Job Execution service includes a Personal Gatekeeper and a Job Manager while the File Transfer service includes the GASS mechanism The Java CoG Kit does not include GridFTP server for file transfers MDS server for storing and retrieving resource information and a full fledged Job Execution Service for executing jobs securely on remote machines These services are pro vided by C Globus Toolkit Detai
53. dential to the MyProxy server This is done from a machine that has the user s long term credentials From these credentials a proxy is created and sent to the MyProxy server The lifetime of the proxy on the MyProxy server can be controlled by the user The proxy can be secured using a username and a password Also the user can restrict the hosts which can later retrieve and or renew the proxy At a later time a proxy can be retrieved by supplying the username and password that the user has set The lifetime of the retrieved proxy can be controlled The proxy can also be renewed if needed Grid administrators may refer to the Administrator s Guide on the MyProxy home page mentioned above for instructions on how to maintain a MyProxy server For users MyProxy software comes with tools to store and retrieve credentials The Java CoG Kit also provides command line tools for this purpose Please refer to sections 4 4 2 and 4 4 3 for information about using these tools 29 4 4 Managing certificates and proxies Some of the tools described in this section need that the environment variable COG_INSTALL_PATH Is Set to lt cog install path gt as discussed in Section 3 6 1 4 4 1 GUI Currently the Java CoG Kit provides the following GUI based tools for credential management Visual grid proxy init This tool allows creation of a proxy Lifetime and cryptographic strength of the proxy can be specified Also the locations of user s l
54. e filepath string containing absolute path of the remote file For example home albert foo txt for a Unix host c temp myfolder document txt for a Windows host A A A HF FH GassInputStream in new GassInputStream host port filepath Read 10 bytes starting at offset 0 from this stream E byte buf new byte 10 in read buf 0 10 GassInputStream supports some other functions like available and getSize Please refer to the Javadocs documentation for the usage information of these methods Close the stream xi in close Create an output stream to write data to a remote file length this parameter specifies the total size of E the data you want to write If unknown use 1 KY boolean append true GassOutputStream out new GassOutputStream host port filepath length append Write 10 bytes starting at position 0 to this stream Ri out write buf 0 10 50 Close the stream out close 5 3 6 Limitations of the Java CoG Kit Currently Java CoG Kit does not provide support for GASS cache This means that the experimental Job Execution Service Section 8 2 provided by the Java CoG Kit does not cache executable and data files staged from clients Also the Java CoG Kit does not provide any replacement for the globus gass cache command line utility available in the Globus Toolkit 51 6 Job Submission 6 1 Introduc
55. e several Java CoG Kit features from a platform independent command line interface Cur rently an equivalent of globus url copy is under development for this shell To start the shell execute the following from the lt cog install path gt bin directory gt cog shell This section discusses the Java CoG Kit APIs for GASS Many of the APIs de scribed here need security credentials in the form of an object of the class org ietf jgss GSSCredential Please refer to the Section 4 4 5 for details about how you can use the Java CoG Kit APIs for getting GSSCredential objects from your GSI proxies In the following sections we assume that you have already created objects of the GSSCredential class 48 Starting a local GASS server Starting a remote GASS server The procedure to start a local GASS server is described in detail in Section 8 3 2 As mentioned before a GASS server is started on the local machine mainly for staging files and receiving output errors from jobs submitted to remote Globus GRAM servers Create a RemoteGassServer instance cred the security credential identity certificate private key the server will use for authentication Object of class org ietf jgss GSSCredential port the port on which the server should listen if 0 a dynamic port will be assigned el boolean secure true RemoteGassServer server new RemoteGassServer cred secure port Set the options for read write access t
56. eams desired for the file transfer De fault is 1 Indicates the tcp buffer size desired for the file transfer Default is 16384 For example assume we like to schedule a transfer from machine machine hot anl gov to machine cold anl gov through machine rft anl gov on port 8080 lt g gt ridCopy factorylocation http rft anl gov 8080 ReliableTransferFactoryService security xmlSig delegation full fromURL gsiftp hot anl gov home amin from txt toURL gsiftp cold anl gov home amin to txt parallelStreams 3 or in uniform hosting environment formulation notation lt g J gt 10 3 Installation ridCopy server hot anl gov port 8080 provider GT3 fromURL gsiftp gridftpServer home amin from txt toURL gsiftp gridftpServer home amin to txt parallelStreams 3 We are following the GT3 development to provide a set of tasks that can be orches trated with GT3 Grid services The following are the tools required in order to use the GridAnt framework for GT3 Java 1 3 1 The GridAnt system also works with Java 1 4 however it re quires certain additional configuration for the new security libraries If you intend to use Java 1 4 0 you will have to copy the Xalan jar available in the gridant lib directory to j2sdk1 4 0 jre lib endorsed directory Apache Ant 1 5 1 Java Cog Kit GT3 alpha2 Server side components 18 Specifically you will need the Java Job Manager service in t
57. eed to compile OGCE as described in Section 3 5 2 This section will explain the steps required to compile the Java CoG Kit To compile jglobus simply do the following gt cd lt cog jglobus srce gt gt ant dist This will compile and build jglobus The build process will create a build direc tory in the lt cog jglobus src gt directory The build directory will contain all the compiled classes the Java CoG Kit directory and a set of examples 4 You need to have CVS installed on your system before downloading the jglobus development version 21 3 5 2 Compiling OGCE 3 6 Configuration 3 6 1 Environment Variables lt cog jglobus srce gt build classes lt cog jglobus sre gt build cog 1 1 lt cog jglobus src gt build cog 1 1 bin lt cog jglobus src gt build examples From this point on lt cog install path gt will represent the lt cog jglobus src gt build cog 1 1 directory You can now proceed to config ure jglobus as shown in Section 3 6 To compile ogce simply do the following gt cd lt cog ogce src gt gt ant dist This will compile and build jglobus The build process will create a build direc tory in the lt cog globus src gt directory The build directory will contain all the compiled classes the Java CoG Kit directory and a set of examples lt cog devel gt build classes lt cog devel gt build cog 1 1 lt cog devel gt build cog 1 1 bin lt cog devel gt build examples From this point on
58. egation full executable bin 1s localExecutable true arguments 1 directory home amin outputFile myOutput txt errorFile myError txt redirect false gt or in uniform hosting environment notation lt gridExecute server hot anl gov port 8080 provider GT3 executable bin 1s arguments 1 directory home amin outputFile myOutput txt errorFile myError txt redirect false le 10 2 2 gridCopy The gridCopy task performs third party file transfers between grid resources ca pable of supporting the GridFtp protocol This task requires the following input arguments specifies a mandatory arguments factorylocation Specifies the location of the Reliable File Transfer factory service security Specifies the XML security parameters Valid options are xmlSig and xmlEnc for XML signature and XML encryption respectively The default is XML signature delegation Specifies the parameters for credential delegation for GSI security Valid options are full and limited for full delegation and limited delegation respec tively The default is limited delegation fromURL Specifies the url of the file to be copied The url must be in the form gstftp machineName portName absolutePathName 84 toURL parallelStreams tcpBuffer Specifies the url of the destination address The url must be in the form gsiftp machineName portName absolutePathName Indicates the number of parallel tcp str
59. ent years to automate complex busi ness tasks using sophisticated workflow management tools Such tools are ex tremely useful in expressing complicated business activities as a set of independent work units and orchestrating a series of dependencies across these units In other words a workflow management system helps in combining a set of specialized tasks by expressing intricate dependencies between these tasks and exposing them as a single complex activity To the heart of any workflow system is the workflow engine The workflow engine is a central controller that handles task dependencies failure recoveries performance analysis and process synchronization Most of the work done in workflow management systems concentrate on the business aspects of the workflow Little consideration is given to the needs of the client in terms of mapping the process flow of the client In the Grid community it is essential that the Grid users have such a tool available to their disposal that enable them to orchestrate complex workflows on the fly without substantial help from the service providers At the same it is also important that such a workflow system does not burden the Grid user with the intricacies of the workflow system With the perspective of the Grid user in mind a simple yet powerful client side workflow management system has been developed and is named as GridAnt Gri dAnt which makes use of commodity technologies such as Apache Ant 1 and XML T
60. erface to the RFT server are copied into the destination directory 41 5 2 2 Unix Shell Scripts globus url copy 4 You need to run GT3 Reliable File Transfer service by following the instruc tions at 12 The setup is complete You can run the tool using ant as follows gt cd ogce gt ant f demos xml ftp You can also run the shell script cog ftp or the Windows batch file cog ftp bat in the lt cog install path gt bin directory Inorder to interface to the RFT service you need to edit the following options 1 Edit the server Textfield available in the Options Tab RFT section in the GUI Tool to specify the location of the Reliable File Transfer service 2 Select the Remote GT3 Provider in the Options Tab When you drag and drop the directories or files the requests are send to the remote Reliable File Transfer Service which does the actual transfer If the RFT service is not setup then the tool uses the transfer service provided by the Java CoG Kit The tools described in this section can be found in the lt cog install path gt bin directory Allows file transfers between a local system and a remote system or between two remote systems The source and target locations are specified as URLs The usage syntax is as follows gt globus url copy options fromURL toURL A complete list of available options can be obtained by running gt globus url copy help This list has also been included in the Appendix A
61. es makes any warranty express or im plied or assumes any legal liability or responsibility for the accuracy completeness or usefulness of any information apparatus product or process disclosed or represents that its use would not infringe privately owned rights IN NO EVENT WILL THE UNITED STATES THE UNIVERSITY OF CHICAGO OR THE UNIVERSITY OF SOUTHERN CALIFOR NIA OR ANY CONTRIBUTORS TO THE GLOBUS PROJECT OR GLOBUS TOOLKIT BE LIABLE FOR ANY DAMAGES INCLUD ING DIRECT INCIDENTAL SPECIAL OR CONSEQUENTIAL DAM AGES RESULTING FROM EXERCISE OF THIS LICENSE AGREE MENT OR THE USE OF THE COMMERCIAL PRODUCT LICENSEE AGREES THAT THE EXPORT OF GOODS AND OR TECH NICAL DATA FROM THE UNITED STATES MAY REQUIRE SOME FORM OF EXPORT CONTROL LICENSE FROM THE U S GOVERN MENT AND THAT FAILURE TO OBTAIN SUCH EXPORT CONTROL LICENSE MAY RESULT IN CRIMINAL LIABILITY UNDER U S LAWS Portions of the Software resulted from work developed under a U S Gov ernment contract and are subject to the following license the Government is granted for itself and others acting on its behalf a paid up nonexclusive ir revocable worldwide license in this computer software to reproduce prepare derivative works and perform publicly and display publicly The Software was prepared in part as an account of work sponsored by an agency of the United States Government Neither the United States nor the University of Chicago nor The University of S
62. following steps Step 1 Provide the host and port information of the MDS server that is to be queried Ey Hashtable env new Hashtable env put Context PROVIDER_URL ldap host port Step 2 Specify the anonymous access El env put Context SECURITY_AUTHENTICATION simple Step 3 Create the Initial Dir Context DirContext ctx null ctx new InitialDirContext env Step 4 Search for required information 5 String baseDN mds vo name local o grid String filter objectclass NamingEnumeration results ctx search baseDN filter null Step 5 Display the results Ey SearchResult si Attributes attrs while results hasMoreElements si SearchResult results next attrs si getAttributes System out printin si getName System out println attrs System out println Anonymous Access Using the Netscape Directory SDK Anonymous access to MDS is described in this section using the Netscape Library Please make sure to include the Netscape library jar file in your classpath before 65 compiling your programs You can get the jar file by following the instructions provided in the given link Netscape http www mozilla org directory javasdk html Authenticated Access to MDS A patched version of the Netscape library 1dapjdk patched jar is distributed with the Java CoG Kit in the src org globus mds gsi netscape directory You can
63. for the random number generator On some platforms this may be a very computationally expensive process However the seed for the random num ber generator only needs to be initialized once per Java Virtual Machine instance The Java CoG Kit can be configured to use an arbitrary SecureRandom implemen tation which can be optimized for particular platform s by adding the following properties into the lt user home gt globus cog properties file random provider lt Provider class gt random algorithm lt algorithm name gt For example if you are using the ISNetworks implementation of SecureRandom add the following into the lt user home gt globus cog properties file random provider com isnetworks provider random InfiniteMonkeyProvider random algorithm InfiniteMonkey SecureRandom implementation by ISNetworks http www isnetworks com infinitemonkey Of course you must first install the provider correctly The next time you use a Java CoG Kit tool or library the startup time should be faster on the platforms supported by the provider For platforms not supported by the provider the default seed generator will be used 37 5 File I O and Transfer 5 1 Introduction This chapter begins with a discussion of file access and transfer issues in a Grid en vironment It then introduces various methods of data access and transfer over the Grid It gives an overview of the GridFTP and GASS protocols It then describes various f
64. gate directory services such as GIIS so that those services can pass on information about the ma chine to others GRIS authenticates and parses each incoming information request and then dispatches those requests to one or more local Information Providers de pending on the type of information named in the request Results are then sent back to the client In order to get collective information about two or more resources present in a single site the queries can be sent directly to GIIS In that case the GIIS directs the query to GRIS MDS uses the Grid Security Infrastructure GSI which enables the use of cer tificates to provide authentication and authorization MDS provides both authen ticated as well as anonymous accesses by the users For authenticated access to MDS the user requires a user certificate and certain other credentials as described in Section 4 2 1 The Site Policies specify the restrictions on registration of resources with GIIS by the system administrator An open policy for a GIIS allows all the GRIS or GIS resources to be registered with it Whereas in a closed system only specified re sources can register with a GIIS The default is for the GIIS to accept registrations 61 only from itself By default the GIIS service runs on port 2135 Please contact your system administrator for your local site policies 7 4 Accessing Grid Information Services This section explains the different methods provided by the Java
65. gt globusrun help A simple example which lists the current directory on the remote machine and prints the result on the client machine is provided below gt globusrun r hot mcs anl gov o amp executable bin ls The r parameter allows you to specify the remote machine to which the job is being submitted The o parameter instructs both the client and the server to treat this job as an interactive job redirecting the input and the output from and to the client machine 56 6 3 3 Windows Batch Files 6 3 4 Java CoG Kit Shell 6 3 5 Job Submission API Remote Executables Submitting the Job In Windows the globusrun bat file can be used to execute a remote job from the command line The syntax of the command line is identical to the one for the Unix shell script Please refer to the previous subsection for details The Java CoG Kit Shell is a convenience application that allows you to use sev eral Java CoK Kit features from a command line like interface To start the shell execute the following from the lt cog install path gt bin directory gt cog shell From inside the console you can use the globusrun command The syntax and options for the globusrun command inside the console are identical to those of the Unix globusrun shell command or the Windows globusrun bat batch The Java CoG Kit provides an extensive API for handling the execution of jobs and other tasks associated with job execution A remote executab
66. he GUI and submit the job To test the gridCopy GridAnt task gt gt gt ES gt gt cd gt3 gridant ant build create your proxy certificate start the GT3 service container edit the build xml in the gridant directory such that the arguments in that target rftDemo reflect the appropriate values ant rftDemo To test a simple GUI client for FileTransfer gt gt gt gt gt gt cd gt3 gridant ant ant build create your proxy certificate start the GT3 service container rftGul make the necessary entries in the GUI and submit the file transfer To be completed 86 Appendix A Program Options A 1 globus2jks The command globus2jks can be found in the build cog 1 1a bin di rectory This commandline provides a convenient wrapper to a Java class Thus it has the same options as listed below Syntax Java KeyStoreConvert options java KeyStoreConvert help Converts Globus credentials user key and certificate into Java keystore format JKS format supported by Sun Options help usage Displays usage version Displays version debug Enables extra debug output cert lt certfile gt Non standard location of user certificate key lt keyfile gt Non standard location of user key alias lt alias gt Keystore alias entry Defaults to globus password lt password gt Keystore password Defaults to globus out lt keystorefile gt Location of the Java
67. he availability of the GridAnt framework provides a much needed func tionality for developing and testing Grid applications with the Globus Toolkit 3 GT3 18 GridAnt uses Apache Ant as its workflow engine Apache Ant is a popular build tool that is extensively used in the Java community Its current func tionality allows the management of complex dependencies and task flows within the project build process We extend the functionality of Apache Ant by providing customized Ant tasks to access the Grid GridAnt proves to be an excellent tool not only to map complex client side work flows but also as a simplistic client to test the functionality of different Grid ser vices GridAnt will help applications to make a smooth transition from GT2 to GT3 GridAnt is not claimed as a substitution for more sophisticated and powerful 82 10 2 GridAnt Tasks workflow engines that map complex business processes 19 20 21 22 Never theless applications with simple process flows tightly integrated to work with the Grid technology can benefit from GridAnt without having to endure any complex workflow architectures The philosophy adopted by the GridAnt project is to use the workflow engine available with Apache Ant and develop a Grid workflow vo cabulary on top of it The following is a partial list of GridAnt tasks that we plan to implement gridSetup The Grid environment setup gridAuthenticate Initializes the proxy certificate to be used by
68. he program_execution module and the Reliable File Transfer service in the data management module To install GridAnt you need to checkout the latest source code compatible with GT3 alpha2 in the cvs repository gt gt gt gt gt Note mkdir cog cd cog cvs d pserver anonymous cvs globus org home dsl cog CVS co gt3 cd gt3 gridant ant build To install the GridAnt components for GT3 alpha use cvs d pserver anonymous cvs globus org home dsl cog CVS co r alpha gt3 85 10 4 Security 10 5 Examples 10 5 1 gridExecute 10 5 2 gridCopy 10 6 Complex Example GridAnt uses the Grid Security Infrastructure GSI for authentication authoriza tion and credential delegation Please refer to Chapter 4 for a detailed description on obtaining the required credentials and the initial setup to make GridAnt GSI compliant Several examples are available in the build xml file To test the gridExecute GridAnt task gt gt gt gt gt Vv cd gt3 gridant ant build create your proxy certificate start the GT3 service container edit the build xml in the gridant directory such that the arguments in that target submitDemo reflect the appropriate values ant submitDemo To test a simple GUI client for Job submission gt gt gt gt gt gt cd gt3 gridant ant build create your proxy certificate start the GT3 service container ant submitGUI make the necessary entries in t
69. hitecture of MDS 7 2 1 GRIS GRIS is information service that runs on a single resource and can answer queries from a user about that particular resource by directing these queries to an informa tion provider deployed on that resource 60 7 2 2 IPs An Information Provider IP is a service that generates information about a spe cific aspect of a resource The query from GRIS to a resource could be requesting any or all of the following types of data e Platform type and architecture e Operating system host OS and version e CPU information type number of CPUs version speed cache e Physical and virtual memory size and free space e Network interface information machine names and IP addresses e File system summary size free space The following link gives a set of core Information Providers available for MDS Information Providers http www globus org mds DefaultGRISProviders html 7 2 3 GIIS 7 2 4 Working 7 3 Security with MDS 7 3 1 Site Policies A GIIS is an aggregate directory service that can supply a collection of information gathered from multiple GRIS and GIIS resources available at a site It supports queries against information spread across multiple GRIS resources Every resource running MDS has a GRIS A GRIS can respond to queries from other systems on the Grid asking for information about a local machine or other specific resource It can be configured to register itself with aggre
70. how a user can use the various tools that the Java CoG Kit provides for managing creating destroying examin ing etc certificates and proxies A discussion of the security API provided by the Java CoG Kit follows The next section describes the issues that need to be faced when using GSI across network boundaries guarded by firewalls We con clude with a discussion of feature differences between the Java CoG Kit and the C implementations of the Globus Toolkit This section assumes some knowledge of the fundamentals of information security and Public Key cryptography If you are not familiar with these concepts please refer to a book such as 4 Working knowledge of Secure Sockets Layer SSL is also assumed 4 1 1 Grid Security Infrastructure The Grid infrastructure allows users to access computational and data resources that may span organizational and perhaps national boundaries Thus it is very important to ensure that this access is secure The basic security requirements are user authentication confidentiality and communication integrity Additionally single sign on is desired in order to allow the user to only authenticate once ir respective of the number of resources he she needs to access This lets the user access resources with the least amount of manual intervention In addition to sat isfying these requirements the security infrastructure needs to interoperate with the various security paradigms being used today in d
71. iable file transfer with GridAnt Section 10 5 How is GridAnt compatible with GT2 Yet to describe 104 Bibliography 1 Ant a Java based Build Tool Web Page Online Available http ant apache org 17 82 85 2 I Foster C Kesselman G Tsudik and S Tuecke A Security Architecture for Computational Grids in 5th ACM Conference on Computer and Communications Security ACM Press Nov 2 5 1998 pp 83 92 Online Available ftp ftp globus org pub globus papers security pdf 25 ios J Novotny S Tuecke and V Welch An Online Credential Repository for the Grid MyProxy in Proceedings of the Tenth International Symposium on High Performance Distributed Computing HPDC 10 San Francisco IEEE Press Aug 2001 Online Available http www globus org research papers myproxy pdf 25 29 4 A Menezes P van Oorschot and S Vanstone Handbook of Applied Cryp tography CRC Press 1996 25 5 Grid Security Infrastructure Web Page Online Available http Iwww globus org security 25 6 W Allcock J Bester J Bresnahan A Chervenak L Liming S Meder and S Tuecke GridFTP Protocol Specification Web Page September 2002 Online Available http www globus org research papers GridftpSpec02 doc 38 39 7 W Allcock I Foster and S Tuecke Protocols and Services for Dis tributed Data Intensive Science in ACAT2000 Proceedings Fermi
72. ifferent organizations This is necessary since it is not convenient for those organizations to abandon the exist ing infrastructure and switch to a new one The Grid Security Infrastructure GSI 2 5 satisfies all the requirements mentioned above GSI uses the Public Key Infrastructure PKI X 509 certificates and Secure Sock ets Layer SSL as its basis It extends these standards for single sign on Thus users do not have usernames passwords in GSI Instead they have public private key pairs and identity certificates 25 4 1 2 Certificates and certifying authorities 4 1 3 Proxies and delegation Every user and service on the Grid has a public and private key These keys are used during the SSL handshake for mutual authentication and to establish a se cure channel This method is not secure unless a public key can be reliably mapped to an entity a user or a service GSI uses a third party called the Certifying Au thority CA to certify this mapping Users and services generate public and private key pairs and send the public keys to the CA for certification The CA verifies using some non cryptographic means that the public key belongs to that entity Upon successful verification it gener ates a document that contains the identity of the entity called subject name its public key and the identity of the CA This document is called a certificate This certificate is digitally signed by the CA so that it cannot
73. ific Each Grid should either have its own CA or use an established commercial CA that is trusted by the users and services in that Grid Depending on the CA the procedure for getting your certificate signed will vary Using Globus tools to acquire a user certificate To use this method you need to have an account on a machine where Globus is installed and permission to run Globus tools This method is described in detail on the following webpage Acquiring GSI certificates http www globus org security vl 1 certs html Follow the hyperlink User certificates on this page Please note that the method specified on this page for sending your public key to the CA for signing is relevant only when using the Globus CA and should be replaced by a procedure specific to your site Please contact the Grid administrator at your site for details After you receive the certificate from the CA please store the certificate and the private key file with appropriate file permissions in the globus directory inside your home directory as instructed on the above webpage 4 2 2 Acquiring a host certificate optional You need to acquire a host certificate if you are going to run a Globus service on your host Examples of Globus services are Globus GRAM server Section 6 1 GridFTP server Section 5 1 2 and so on A host certificate is a binding between the identity of the host and its public key Just as with user certificates acquiring a host
74. ile transfer tools and APIs provided by the Java CoG Kit Some example code demonstrating the use of file transfer APIs is provided An important aspect of distributed computing is access to distributed data Many Grid based scientific and engineering applications require transfer of large amounts of data terabytes or petabytes between storage systems and access to this data by applications running on remote hosts For example data generated by a particle accelerator for a multinational physics collaboration may need to be transferred to analysis centers in different continents from where the data would be accessed by multiple analysis applications 5 1 1 Requirements for File Access and Transfer over the Grid 5 1 2 GridFTP Among the most important requirements are high performance security reliability and restartability A few more requirements are imposed by the high heterogeneity involved in the Grid environment The Grid being a multi organizational envi ronment different storage systems operating systems security infrastructures re source namespaces etc are used at different sites It is clearly inconvenient for the applications to use different protocols and APIs to interact with different systems The file access and transfer mechanism chosen for the Grid should therefore pro vide an abstract layer for access irrespective of the underlying heterogeneity At the same time it should impose as few requirements as possible on
75. iliarity with the Unix command line and Bash scripts In order to be able to run the Java CoG Kit tests you need to be sure that the following are available e A Unix like operating system BSD Linux Solaris HP UX e Bash e Concurrent Versions System CVS e Jakarta Ant e Atleast one Java Development Kit 1 3 1 or higher e GNU wget We have not spend effort in making this a 100 Java framework 77 9 3 Installation To run the Java CoG Kit tests all you need is the night1y tests script available from the Java CoG Kit web site Test script 9 4 Configuration http www globus org cog java nightly test Configuration of the tests is done by changing the values of the variables found in the beginning of the test script A detailed list of these variables together with their meaning and sample values is provided below LOCAL BUILDDIR HTMLOUTDIR JDKSDIR JDKS Syntax LocaL yes no Specifies whether the sources are going to be fetched from the source repos itory no or already downloaded sources will be used yes Example LocaL no Syntax BUILDDIR lt directory gt If the value of Loca is yes this variable represents the location of the Java CoG Kit sources If the value of Locat is no it points to the directory where the sources will be downloaded by the script This directory will be created if it does not already exist Example BUILDDIR SHOME tmp c
76. in MDS is Mds vo name local o grid The final argument in this example is the search filter It specifies the category of object class you wish to search The search filter objectclass here indicates that the information regarding all the ob ject classes needs to be displayed Please refer to Section 7 5 for the syntax and attributes of the objectclass A part of the output for the above query would look as follows dn Mds Host hn hot mcs anl gov Mds Vo name local o Grid Mds Cpu speedMHz 866 Mds Memory Ram Total freeMB 304 Mds Fs freeMB 10 Mds Fs freeMB 21 Mds Fs freeMB 270 Mds Fs freeMB 341 Mds Fs freeMB 4428 Mds Fs freeMB 47 Mds Fs freeMB 73 Mds Cpu Free 5minX100 134 Mds Net Total count 2 Mds validfrom 20030303165825Z Mds Cpu Total count 2 Mds Memory Vm sizeMB 243 Mds Cpu vendor Genuinelntel Mds Net name eth0 Mds Net name lo Mds validto 20030303165825Z This example shows how to query for the amount of free file system space on all machines on a GHS running for a site The command is as follows gt grid info search h giis mcs anl gov p 2135 b Mds Vo name site o Grid objectclass Mds Fs freeMB Here it is assumed that GIIS is running on machine giis mcs anl gov at port 2135 The branch point option b has the value Mds Vo name site o grid This is the default branch point for GIIS server The attribute Mds Fs freeMB specifies that the information regarding the amount of free 63
77. input can be redirected from a file whether local or remote The output and error streams of remote jobs are redirected to remote files which can be monitored from the local machine In contrast batch jobs have their output error streams stored into remote files which can be retrieved after the job completes Batch jobs are suitable when immediate feedback from the job is not needed when multiple jobs are launched in parallel or when the execution time is expected to be very large GRAM also provides the ability to stage in data or executables using a facility called Global Access to Secondary Storage GASS File staging allows you to automatically transfer any files required by your job from the client machine to the server machine It is also possible to transfer the output files back to the client machine after the job ends Details about GASS can be found at 10 6 2 Globus Resource Specification Language RSL RSL is a common interchange language used to describe resources irrespective of the scheduler or batch system used RSL provides skeletal syntax to describe resources and various resource management components resulting in lt attribute value gt pairs Each attribute in the resource description serves as a parameter to control the behavior of one or more components in the resource management system 53 6 2 1 RSL Syntax The core syntax of the RSL is the relation of the form lt attribute value gt pair e g executa
78. ints to the location of the private key associated with the Globus user certificate proxy points to the location of the user proxy The proxy is located in a temporary directory and has its name composed of the string x509up_u and the a user id OS specific In the above example the user id is 1000 cacert contains a comma separated list of certificate authorities that the user trusts ip represents the IP address of the machine the Java CoG Kit will be run from 23 EE m cee i License Agreement ICENSE iroject Name sio ain Contact mal Our license follows the Globus Toolkit License Nevertheless we require that you notify 59 we can keep track of the use of the Java CoG Kit as this clirectly affects our abil to Mativate additional coding activities Please be sa kind to send an e mail ta regar mcs anl gavwith the following description Pape escrigiion of your project urhermore if your project uses Java CoG Kit ar you use a softw Cot Setup S lava CoG Kit we require that you cite the following paper in your the globus project crid Ki Crecer von Laszemsia lan Foster a MO eet Previous Setup Java Commodity Grici Kit Gregor von Laszewski lan Foster Jar Concurrency and Computation Practice and Experience pages A A AAA 2 i Progress You can ether load a previous Java CoG Kit configuration or start a new one Y agree to these terms and co
79. is directory will be represented by lt cog jglobus sre gt You can now proceed to compile jglobus as described in Section 3 5 1 The OGCE stable source is not available at this time Please use the development OGCE source 3 4 5 3 4 5 OGCE Development Source 3 5 Compiling 3 5 1 Compiling JGlobus The development version of OGCE can be retrieved from our source repository using anonymous CVS access Please not that jglobus is needed in order to use OGCE This section will provide instructions to download both jglobus and OGCE We suggest that you first create a new directory in which to store the development version of jglobus For convenience this directory will be referred to as lt cog devel gt We recommend that you name this directory cog gt mkdir lt cog devel gt gt cd lt cog devel gt Login to the CVS server gt cvs d pserver anonymous cvs globus org home dsl cog CVS login Hit ENTER when you are asked for a password After the login step you can check out the jglobus module with the following command gt cvs d pserver anonymous cvs globus org home dsl cog CVS co r jglobus jgss jglobus gt cvs d pserver anonymous cvs globus org home dsl cog CVS Y co r jglobus jgss ogce Inside the lt cog deve1 gt directory two directories named jglobus and ogce will be created These directories will be represented by lt cog jglobus sre gt respec tively lt cog ogce sre gt You can now proc
80. jordomo globus org with the single word which in the body of the message You will receive in re sponse a message listing the lists to which your email address is subscribed If this mailing list does not appear in the list you receive you are probably subscribed to the list under a different address and you will not be able to post messages to the list using your current address If you would like to be notified of CoG Kit release updates visit our convenient subscription center at Subscribe http www globus org cog contact Other Globus related mailing lists can be found on the Globus web page Subscribe http www globus org about subscriptions html News Note that you can use these web pages to unsubscribe from the lists All mailing list are maintained with majordomo However we did have to disable the who function in order to protect the members from spam bots News about the Java CoG Kit is sent in irregular intervals the frequency is monthly to every four month by means of the following list CoG News cog news globus org Sorted by Thread http www unix globus org mail_archive cog news threads html Sorted by Date http www unix globus org mail_archive cog news maillist html Discussions and Community Developers Discussions and general questions can be send to the high volume e mail list at Java List java globus org Sorted by Thread http www unix globus org mail_archive java threads html Sorte
81. k with the newest version of the code Section 3 4 3 For users interested in OGCE the following choices are available ogce stable source Users that are interested in also seeing the source to the stable binary version Section 3 4 4 ogce development source Users that like to work with the newest version of the code Section 3 4 5 Most users may just be interested in the stable ogce and jglobus sources distribu tion Hence we refer to the Java CoG Kit in this manual as the combined contri butions presented in the jglobus and ogce directories 3 4 Downloading the Java CoG Kit This section instructs you on how to download various Java CoG Kit versions 3 4 1 JGlobus Stable Binary The stable binary distribution of the jglobus is available from our web site e tar gz archive cog 1 1 bin tar gz www globus org cog java 1 1 cog 1 1 bin tar gz e zip archive cog 1 1 bin zip www globus org cog java 1 1 cog 1 1 bin zip 19 After downloading unpack the archive Unix gt tar xzf cog 1 1 bin tar gz Windows Double click on the downloaded archive and extract it to a directory of your choice A directory named cog 1 1 will be created This directory will from now on be referred to as lt cog install path gt You can now proceed to configure jglobus as described in Section 3 6 3 4 2 JGlobus Stable Source The stable source distribution of the jglobus is available from our web site e tar gz archive cog 1 1 src
82. l C 5 2 Architecture of MDS How do I use the Java CoG Kit API to start a job Section 6 3 5 What are interactive batch jobs when to chose what Section 6 1 3 How do I stream output errors back to my machine Section 6 3 5 What is a directory service Section 7 1 What is MDS Section 7 1 What are the differences between Java CoG Kit and Globus grid information search tool Section 7 8 Where do I get detailed information for Globus MDS Section 7 1 What are the main components in Mds Section 7 2 What is GRIS Section 7 2 What is the functionality of GIIS Section 7 2 How do GRIS and GIIS interact with each other Section 7 2 What are the different kinds of information you can retrive using MDS Section 7 2 What is the architecture of the MDS Section 7 2 What is an information provider Section 7 2 Can I use GRIS without a GIIS Section 7 2 Where do I find MDS information providers Section 7 2 101 C 5 3 Security in MDS How GSI work with MDS Section 7 3 What is SASL authentication in regards to MDS Section 7 4 4 Are there any site policies attached with GIIS and GRIS Section 7 3 1 Can I share information that I got from GRIS Section 7 3 1 C 5 4 Retrieving information from MDS C 5 5 Performace Issues with MDS C 6 Server Side Java CoG Kit C 6 1 General How do I retrieve MDS information using the command line tool Section 7 4 2
83. le Versions of Java CoG Kit before 1 1a GlobusProxy cred 2 FileOutputStream out new FileOutputStream file cred save out out close Java CoG Kit 1 1a ExtendedGSSCredential cred E byte data cred export ExtendedGSSCredential IMPEXP_OPAQUE FileOutputStream out new FileOutputStream file out write data out close Loading credentials from a file Versions of Java CoG Kit before 1 1a FileInputStream in new FilelnputStream file GlobusProxy cred GlobusProxy load in null in close Java CoG Kit 1 1a byte data new buffer 1024 FileInputStream in new FilelnputStream file read in the credential data in read data 34 in close ExtendedGSSManager manager ExtendedGSSManager ExtendedGSSManager getInstance GSSCredential cred manager createCredential data ExtendedGSSCredential IMPEXP_OPAQUE GSSCredential DEFAULT_LIFETIME null use default mechanism GSI GSSCredential ACCEPT_ONLY Getting the remaining lifetime of a credential Versions of Java CoG Kit before 1 1a GlobusProxy cred int time cred getTimeLeft Java CoG Kit 1 1a GSSCredential cred int time cred getRemainingLifetime Getting the identity of the credential Versions of Java CoG Kit before 1 1a GlobusProxy cred String identity CertUtil toGlobusID cred getSubject Java CoG Kit 1 la GSSCredential cred gt String identity cred getName t
84. le can be executed by simply passing its name through the RSL description amp executable bin 1s After the RSL description was built it must be submitted to the server First you should ensure that the gatekeeper is alive on the remote machine Gram ping hot mcs anl gov Next a GramJob Object is instantiated passing the RSL string to the constructor GramJob job new GramJob RSLString Feedback from the remote server is provided in order to interact with the job A listener can be used to receive notifications about job status from the server class GramJobListenerImpl implements GramJobListener public void statusChanged GramJob job String status job getStatusAsString job addListener new GramJobListenerImpl The job is now ready for submission The actual submission is done through the request method which takes two arguments e The first argument specifies the remote server e The second argument indicates whether the job is submitted in batch or in teractive mode A value of true denotes batch mode job request hot mcs anl gov false 57 Local Executables and File Staging By default the job manager will look for the executable and input output files on the remote machine on which the job is scheduled for execution In case the executable file resides on the local machine or if the job requires a local file as input file staging needs to be used In order to use file staging a GASS
85. led information of these C based services can be found at the following links Globus GridFTP http www globus org datagrid gridftp html Globus GRAM http www globus org gram 8 2 Job Execution Service Globus MDS http www globus org mds Details on how to start up these services for Globus Toolkit 2 2 are available at the following link Globus install www globus org gt2 2 admin guide startup html Java CoG Kit contains an experimental and elementary Job Execution Service The implementation includes a Personal Gatekeeper and a Job Manager A client submits a job request to the Personal Gatekeeper The Personal Gatekeeper performs authentication with the client and starts a Job Manager The Job Manager receives the job requests interprets them and executes the jobs either interactively using the fork jobmanager or through batch schedulers such as PBS LSF Normally a Gatekeeper has to map the identity of the client to a local user and start the Job Manager as that local user But since the Java Virtual Machine allows only limited interaction with the Operating System this functionality cannot be implemented in Java As such the Personal Gatekeeper cannot map between the root and user id like the C Globus Gatekeeper Hence this service can be used for personal grids or adhoc grids controlled by a singe user All jobs submitted will be executed with that user account settings Details about the full fledged Globus C based G
86. ll 5 3 5 APIs Starts a GASS server on the local machine and prints its URL Port number may be specified You can control the level of access this server will have to the local file system Access can be read only or write only or read write Redirection of standard output and error streams of a job can be controlled You can also specify whether this server can be shut down with a request from a client Usage syntax is gt globus gass server options For example to start a server listening on port 2222 with read only access to local file system gt globus gass server p 2222 read Stops a GASS server given its URL For this to succeed the server must allow client initiated shutdowns The GASS server can be local or remote Usage syntax 1s gt globus gass server shutdown options lt GASS URL gt For example to shut down a GASS server running on hot mcs anl gov on port 2345 gt globus gass server shutdown https hot mcs anl gov 2345 Please refer to Section 5 2 2 for a description of globus url copy Each of the tools described in the previous section has a Windows batch file coun terpart These batch files can be found in the lt cog install path gt bin directory Just like the Unix shell scripts each of them supports a help option that prints a usage message The usage details have also been included in the Appendix A for this manual The Java CoG Kit Shell is a convenience application that allows you to us
87. margin width of the paper and make the lines no longer than 79 characters An example is shown below int a a 1 2 Interactive commands issued by a user in a shell are preceded with a gt at the beginning of the line gt mkdir directory gt cd directory In case interactive commands exceed the 79 character limit they are wrapped into the next line and are not proceeded by the gt character A backslash is included at the end of such lines to explicitly indicate that the command ins continued on the next line gt echo This is s very long text that is continued on the next lines The leading blanks in the next lines are to be ignored gt echo This is a new command References to variables or other important text that is part of a program or shell script is written in Courier To illustrate this on an example Hence a reference to the variable int a form our previous example uses also the Courier font Generic entities are wrapped between angle brackets Each such entity is not to be taken literally In general such constructs are explained as they occur throughout the manual The use of such entities is shown in the example below gt ping lt machine name gt Here lt machine name gt is to be replaced with an actual machine name gt ping hot mcs anl gov 15 Web links are proceeded by a meaningful name for the link An example is Java CoG Kit Website http www globus org cog Links to code sou
88. mplementation They are given in Section 8 3 1 76 9 Production Tests with the Java CoG Kit 9 1 Introduction 9 2 Requirements new March 28 2003 Testing is a significant part of contemporary software development practices With a proper design it can uncover a high percentage of problems before software is released Tests can also be used with released software to reveal compatibility issues The Java CoG Kit contains two testing methodologies First it contains a number of unit tests that are run prior to a release to increase the code correctness Second it contains a number of production tests that are intended to check if elementary tasks such as job submission and filetransfer can be performed In this section we concentrate on the later The Java CoG Kit production testing framework is designed to perform production tests in a flexible manner It tests multiple Java Development Kits JDKs Globus Toolkit versions for a variety of essential Globus Toolkit services The results are displayed in convenient reports in HTML format that may be published on demand to a Web server Hence the framework can be used by Grid administrators to per form simple production tests helping to provide a report about the functionality of a Grid However this framework can also be used by individual users to test their ability to access Globus Toolkit Services based on a configuration file the user may maintain This chapter assumes some fam
89. mplementation of the GSI SASL mechanism for JNDI can be found env put javax security sasl client pkgs org globus mds gsi jndi This property specifies the quality of protection value K env put javax security sasl qop auth Specify the particular SASL mechanism to use env put Context SECURITY_AUTHENTICATION GSIMechanism NAME Authenticated Access Using Netscape Directory SDK In order to establish a secure connection using Netscape library you need to use Version 4 1 or above of the Netscape Directory SDK 67 Example Location To provide authenticated access instead of anonymous access include these steps after Step 2 in the Anonymous Access Using the Netscape Directory SDK Section TAA Hashtable props new Hashtable This property specifies where the implementation of the GSI SASL mechanism for Netscape Directory SDK can be found EL props put javax security sasl client pkgs org globus mds gsi netscape This property specifies the quality of protection value props put Javax security sasl qop auth Authenticate to the server over SASL E ld authenticate null new String GSIMechanism NAME props null The same example program for the Netscape using GSI security mechanism is available at the following location jglobus org globus mds gsi NetscapeTest java For JNDI example program using GSI please refer to the f
90. ms to the local GASS server GassServer gass new GassServer true 0 rsl new RslAttributes rsl add stdout gass getURL dev stdout rsl add stderr gass getURL dev stderr Register the JobOutputListener class with the Gass server JobOutputListenerImpl outListener new JobOutputListenerImpl JobOutputStream outStream new JobOutputStream outListener gass registerJobOutputStream out outStream gass registerJobOutputStream err outStream class JobOutputListenerImpl implements JobOutputListener public void outputClosed Job has finished no more output is available 58 Sample code Sample program testing various features is available at jglobus src org globus gram Gram15Test java It can be run from jglobus directory using the following ant command gt ant buildfile progs xml GramTest3 lt machine name gt 6 4 Differences from the C Globus Toolkit 6 4 1 Gatekeeper Due to the lack of operating system specific programming interfaces of the Java programming language the personal gatekeeper does not allow user remapping Hence the job managers spawned by the personal gatekeeper can only run with the same priviledges as the gatekeeper itself 6 4 2 RSL Parser The following features avaiable in the Globus C RSL parser are not supported by the Java CoG Kit RSL parser 1 User specified delimiter for quoted literals 2 RSL strings that only contain relations outside
91. nditions A If the second option is selected below it means that a default prewous configuration was Previous Setup not round About Cancel Previous Next gt user Certificate Ciria tag The default configuration can usually be found in your globus directory which ls located in your home directory The defauk name for the Java CoG Kit configuration file Is E cortmene aunnomy 25 oroperes e Use a previous configuration home mike globus cog properties view Toa Senp gt onfiguration the globus project Ou User Certificate Cancel lt Previous Next gt Finish Progress Please enter the location ef your Globus user certificate To browse for your Globus certificate please press the browse button To Mew the selected file press view If you do not yet have a Globus certificate please consul the Globus manual on how to get fone Jhome mike globus usercert em About Cancel lt Previous Figure 3 2 Screen shot of the setup wizard An additional list of properties that can but set in the cog properties file but which are not configured by the Setup Wizard is provided below tcp port range A range of ports in the form lt minport gt lt maxport gt that limits the local ports used for services by the Java CoG Kit org globus dev random A true Or false value specifying whether the Java CoG Kit should use the Unix style dev u
92. ner mcs anl gov Solaris 9 0 15 2 mozart mcs anl gov Solaris 9 0 15 2 mahler mcs anl gov Solaris 7 0 15 2 schumann mcs anl gov IRIX 6 5 IBMJava2 14 Time Host os 0 32 7 grieg mcs anl gov Mandrake 7 2 2 4 17 0 32 18 shostakovich mcs anl gov RedHat 7 3 2 4 18 w1 0 32 23 wagner mcs anl gov Solaris 9 0 32 23 mozart mcs anl gov Solaris 9 0 32 23 mahler mcs anl gov Solaris 7 0 32 23 schumann mcs anl gov IRIX 6 5 j2sdk1 4 0_03 Time _ Host OS grieg mcs anl gov Figure 9 1 Sample general test report Mandrake 7 2 2 4 17 81 10 GridAnt A Client side Grid Workflow System 10 1 Introduction This chapter focuses on a sophisticated client side workflow management system that can orchestrate complex task dependencies It gives an overview of process workflows and workflow engines It further describes the applicability of a client side workflow system for Grid technologies and introduces the functionality of the GridAnt workflow system It provides detailed instructions for the user to install the GridAnt system and other dependent packages An introductory set of examples is discussed that helps the end user to understand the working of the GridAnt system The current version of GridAnt is not an integral part of the Java CoG Kit and requires a separate installation However efforts are being made to integrate the GridAnt module in the Java CoG Kit for future releases Significant research has been conducted in rec
93. nl gov CN Beulah Alunkal CN proxy Time Left 8 h 23 min 15 sec Strength 512 bits Create Destroy Refresh Gatekeeper Gatekeeper wiggum mes anl gov v Test Remove Job Specificiation Executable bin ls J Local file Directory home a Arguments Environment Stdout job stdout v Local file Es Stderr job stderr v Local file Ex Stain Cj Local file Job Options C Redirect stdout err to Window Run C Full delegation Figure 6 2 The CoG Form 55 Drag and Drop Desktop 6 3 2 Unix Shell Scripts A more experimental and hopefully more intuitive interface for submitting jobs can be started by executing gt cog desktop With Drag and Drop Desktop multiple jobs and servers can be configured graph ically A job submission is a simple matter of dragging the icon of a job over the icon of a configured server A Drag and Drop Desktop sample screen shot can be seen in Figure 6 3 Java CoG Kit Drag N Drop Desktop iof x File Edit View Security localhost list pitcairn 0 A date terra a LS eny wiggum 3 windows Figure 6 3 The Drag and Drop Desktop You can use globusrun to execute remote jobs from the command line The format for running globusrun is gt globusrun options RSL string For a complete list of options accepted by globusrun please run the following command
94. nner msg https hot anl gov 2222 c temp banner msg globus url copy explained in the previous section has a Windows batch file counterpart named globus url copy bat It can be found in the same location as globus url copy The capabilities are identical The Java CoG Kit Shell is a convenience application that allows you to use sev eral Java CoG Kit features from a platform independent command line interface Currently an equivalent of globus url copy is under development for this shell Text based interactive interfaces for FTP and GridFTP are also in progress These will be similar to the ftp program available on many Unix systems To start the Java CoG Kit shell execute the following from the lt cog install path gt bin directory gt cog shell The Java CoG Kit provides a set of APIs for file transfers using FTP and GridFTP We show here some examples that use the GridFTP APIs For a detailed program mer s guide and complete documentation in Javadocs format please refer to the following website Java CoG Kit File Transfer API guide http www globus org cog jftp Specifically the programmer s guide addresses the following issues e File storage and retrieval to and from FTP and GridFTP servers e Third party direct server to server transfers between FTP and GridFTP servers e Parallel and Striped transfers using GridFTP 43 e Measuring performance of a file transfer e Restarting failed transfers Transfe
95. ntax Indicates the location of the source repository of the Java CoG Kit This variable is only used if you set LOCAL to no above We recommend you ne this variable unmodified The tests were designed to run without user n and modification of the cvsroor variable may lead to CVS pone while waiting for a password Caution Due to bid me ne ae doeg not pol the a ae to be ae is eet a script uses a aiei a walle ente any hannar a that are already locally stored on your machine The side effect is that the next time you access a CVS archive for which you had the password stored in pserver mode you will have to retype it Example cvsrooT pserver anonymoustcvs globus org home ds1 cog CVS Syntax COG_PROPERTIES lt file gt Allows you to specify a cog properties file to be used for the tests You can safely leave this blank in which case the default sHomE globus cog properties will be used Example COG_PROPERTIES HOME globus cog properties esi HOSTLIST lt URL gt lt URL gt to bet beato pl e tests A detailed po of the ae format i is piers g in Section 9 5 Example HOSTLIST http www lpt usb com machines txt file tmp machines2 txt Syntax TIMEOUT lt integer gt Specifies in seconds the time after which a test is killed if it has not termi nated This seems to be necessary since in some instances while running on IBMJava 2 14 the Java CoG Kit appears to hang indefini
96. nts for the Java CoG Kit Section 3 2 How do I download the stable distribution Section 3 4 How do I download the development distribution Section 3 4 5 How do I compile the Java CoG Kit sources Section 3 5 How do I configure the Java CoG Kit Section 3 6 A script complains about COG_INSTALL_PATH not being set why Section 3 6 1 A program complains about a missing proxy certificate why Section 3 6 3 C 2 1 General Grid security Questions Why is Grid security so important Section 4 1 1 What is the difference between a normal UNIX Windows username pasword to the Grid security infrastrucure Section 4 1 1 What is a certificate Section 4 1 2 What is a CA Section 4 1 2 What is a proxy Section 4 1 3 What is the difference between a certificate and a proxy Section 4 1 3 What needs to be protected from others and how Section 4 2 6 What is a gridmap file Section 4 2 5 97 What is MyProxy Section 4 3 Can the The Java CoG Kit work behind a firewall How do I limit the range of ports that the Java CoG Kit will use Section 4 5 C 2 2 Questions related to user certificates and certificate authority C 2 3 C 2 4 C 2 5 C 2 6 How do I aquire a certificate Section 4 2 1 How do I renew a certificate Section 4 2 3 How do I change the pass phrase Please see the grid change pass phrase tool in Section 4 4 2 and 4 4 3 How do I get the CA s
97. o a server Handshake fail ure What does it mean and how can I fix it Probably you have a proxy not compatible with the server Please see the grid proxy init tool described in Section 4 4 2 What are the issues involved in file transfer over the Grid Section 5 1 1 What is GridFTP Section 5 1 1 and 5 1 2 What is GASS Section 5 1 1 and 5 1 3 Can I still use FTP and SCP Section 5 1 4 What is the difference between GridFTP and GASS Section 5 1 1 What is a third party transfer See the GridFTP section 5 1 2 What are parallel and striped transfers See the GridFTP section 5 1 2 Are GridFTP and GASS standard services that run on every Globus enabled resource See the GridFTP Section 5 1 2 and GASS section 5 1 3 Is there a provision to monitor the progress of a transfer and restart it if fails Yes See restart markers mentioned in the GridFTP section 5 1 2 How do I store and retrieve files using GridFTP Different methods of doing this are described in Section 5 2 How do I transfer files between two GridFTP servers Different methods of doing this are described in Section 5 2 How do I monitor progress of my transfers Currently you can do this only using the GridFTP APIs Please check the GridFTP Programmer s Guide available at http www globus org cog jftp guide html 99 C 3 3 GASS C 3 4 Version differences C 4 Job Execution C 4 1 GRAM How do I come to kno
98. o be generated from this one cert lt certfile gt Non standard location of user certificate key lt keyfile gt Non standard location of user key out lt proxyfile gt Non standard location of new proxy cert pkcs11 Enables the PKCs11 support module The cert and key arguments are used as labels to find the credentials on the device 94 A 13 myproxy The command myproxy can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus it has the same options as listed below Syntax java MyProxy options command java MyProxy version java MyProxy help Options help s lt subject gt command lt host gt lt port gt lt username gt lt time gt lt time gt Displays usage version Displays version host lt host gt Hostname of the myproxy server port lt port gt Port of the myproxy server default 7512 username lt username gt Username for the delegated proxy portal_lifetime lt time gt Lifetime of delegated proxy on the portal default 2 hours cred_lifetime lt time gt Lifetime of delegated proxy default 1 week 168 hours Note Only used by PUT operation subject lt subject gt Performs subject authorization One of the following put lt put proxy get get proxy anonget get proxy without local credentials destroy remove proxy info credential information 95
99. o remote file system output error redirection and for client initiated shutdowns options a bitwise OR of zero or more of the following flags READ_ENABLE WRITE_ENABLE STDOUT_ENABLE STDERR_ENABLE CLIENT_SHUTDOWN_ENABLE These flags are static variables defined in the class org globus io gass server GassServer AA A F F For example if you want to allow client initiated shutdowns and read only access to remote file system use a bitwise OR of those two flags as shown in the code below int options GassServer READ_ENABLE GassServer CLIENT_SHUTDOWN_ENABLE server setOptions options Start the server on the specified host String resourceManagerContact new String hot mcs anl gov server start resourceManagerContact Get the URL for the remote server String url server getURL Later Shut down the remote server server shutdown 49 Remote file I O with GASS Once you start a GASS server remotely you can get input and output streams to read and write data for any remote file you have access to Get the host and port information of the remote GASS server created in the previous section and stored in an object called server E URL remoteGassUrl new URL server getURL String host remoteGassUrl getHost int port remoteGassUrl getPort Create an input stream to read data from a remote fil
100. oString GlobusCredential GSSCredential conversion As mentioned before it is not recommended to use the GlobusCredential class directly To convert an instance of GlobusCredential to a GSSCredential instance you must first wrap it in org globus gsi gssapi GlobusGSSCredentialImpl class as shown below GlobusCredential cred x GSSCredential cred new GlobusGSSCredentialImpl cred GSSCredential ACCEPT_ONLY It is also possible to retrieve the org globus gsi GlobusCredential object from the GSSCredential instance if it is of the right type GSSCredential cred A if GSSCredential instanceof GlobusGSSCredentialImpl GlobusCredential globusCred 35 4 5 Firewall Issues GlobusGSSCredentialImpl cred getGlobusCredential Grids usually involve multiple organizations or at least multiple departments in the same organization Thus the interactions between Grid clients and servers span network boundaries Some of these networks may have firewalls Since the network communication in Globus is based on TCP sockets firewalls may block 1t You may face the following issues due to network firewalls while using the Java CoG Kit to communicate with Globus servers e Connecting to Globus servers If the port a Globus server is listening on is blocked by the firewall the connection will fail This applies to all servers like GridFTP servers Globus Gatekeepers and so on This problem can be solved if the person maintaining
101. ock through the NTP protocol Please consult your system administrator about the NTP protocol and time synchronization Alternatively you could synchronize your system clock us ing one of the following methods NT Atomic Clock Synchronizer www worldtimeserver com Atomic Clock Synchronizer download http www worldtimeserver com atomic clock UNIX Linux On UNIX you can configure automatic synchronization through with a nearby NTP server 3 6 3 Globus Security Credentials 3 6 4 Configuration Configuration with the Wizard Configuration with an Editor Using the Java CoG Kit requires you to have a proper set of Globus credentials including but not limited to a Globus certificate For details about Globus security credentials please consult Section 4 2 This subsection will explain the different methods that can be used to configure the Java CoG Kit To start the configuration wizard for the Java CoG Kit run the setup script avail able in the lt cog install path gt bin directory A sample screen shot of the setup wizard is shown in Figure 3 2 Manual configuration of the Java CoG Kit is also possible The configuration file is named cog properties and is located in the lt user home gt globus directory A sample Java CoG Kit configuration file is provided in Figure 3 3 It includes a number of important properties These properties are usercert points to the location of the Globus user certificate userkey po
102. of specifications 59 7 Accessing the Grid Information Service This chapter gives a brief overview of the Grid Information Service architecture and explains the different ways in which a user can access the Grid Information Servers using the Java CoG Kit 7 1 Introduction Grid technologies enable large scale sharing of resources within groups of individ uals and organizations In these settings the user might be interested in discovering and monitoring the resources in a secure and efficient way The Globus Toolkit supports a Monitoring and Discovery Service MDS 15 to provide information about Grid resources In short MDS provides directory services for resources in the Grid A directory service provides information about different entities in the environment such as resources and services to applications and their users Extensive documentation for MDS is available at Homepage http www globus org mds Manual http www globus org mds mdsusersguide pdf More technical details are available at 16 7 2 Architecture The structure of MDS is hierarchical 17 It consists of Grid Resource Information Service GRIS Grid Index Information Service GIIS and Information Providers IPs as shown in Figure 7 1 User 2 Resources B gt Direct query a Client GRIS User 3 3 Client lt GIIS GIIS requests information Indirect query Resource A gt GRIS Figure 7 1 Arc
103. og test Syntax HTMLOUTDIR lt directory gt This variable points to the directory where the output of the tests will go Example HTMLOUTDIR HOME public html tests Syntax JDKSDIR lt directory gt This variable represents a directory where at least a Java Development Kit can be found This directory will be searched for valid Java Development Kits If inside the JDksDIR you have a symbolic link pointing to a Java Development Kit directory also within the JDKSDIR that specific Java De velopment Kit will still only be used once You can also have directories that contain other things than a Java Development Kit Such directories will be ignored Example JDKSDIR usr local Syntax JDKS lt directory gt lt directory gt 1 11 This variable can be used as an alternative to the JDksDIR variable It must contain a list of Java Development Kit distribution directories If you wish to use the JDKSDIR method instead this variable must remain blank Example JDks usr local jdk1 3 1 07 usr local j2sdk1 4 1 01 2 Currently this does not work If you choose a local test the script will assume that it was executed from the ogce bin directory 78 Ys ANT_HOME CVSROOT COG_PROPERTIES HOSTLIST Host Table Format Syntax ANT_HOME lt directory gt Specifies the location of Jakarta Ant Example ANT_HOME usr local jakarta ant 1 5 1 Syntax See the CVS manual for details about cvsRoot sy
104. ollowing program loca tion jglobus org globus mds gsi JndiTest java Adding Updating Entries using API in MDS 7 5 Schema MDS is a READ ONLY directory service Nevertheless you can add or update entries in MDS using the API if the MDS server supports a backend read write database In that case you populate the backend database yourself Normally the backend is automatically populated by information providers For further informa tion please refer to the following link FAQ UpdateMads http www globus org mds FAQ htmltadddatatomds The information model used in MDS is based on entries arranged in an hierarchical tree like structure The tree is called the Directory Information Tree and the con tents include object classes and entries Object classes describe what information can be stored in the directory The values of the object class determine the schema 68 rules the entries must obey Few of the descriptions of the schema object classes and their attribute types are shown below Object class Mds Attribute type Mds validfrom Attribute type Mds validto Attribute type Mds keepto Object class MdsHost Attribute type Mds Host hn Object class MdsOs Attribute type Mds Os name Attribute type Mds Os release Attribute type Mds Os version Object class MdsCpu Attribute type Mds Cpu vendor Attribute type Mds Cpu model Attribute type Mds Cpu version Attribute type Mds Cpu features Attribute type Mds Cpu speed
105. om To Status Current Errors checa C splash_p4f 5 npa files Finis 1448 NA Nor work gt m su pita file Finis 3004 NA No er ecoa E systems file gsifp Finis 80870 100 No er Dan Q E tech_reports fpa asiftp Finis 384 No er ec cvs C abstracts sift siftp Finis NIA No er o seit sit Finis NJA No er E data_management a teat o ip gsifp asip Finis NA Noer Pa onan o A gsifip gsiftp Finis NJA No er E multim E E piepen gsiftp filed Finis 806 No er o ogsa prop flex files Finis 118 No et files fle Finis 151 No er E Remote System GridFTP files fle Finis 80870 100 No er asa Brea gsiftpymotmes anl gov 281 1 homestalunkalibackup osito cold mes anl gov 281 1 thomesfalunkallbacky 9 CI pertest may22 tp uir 2 C perest C grami CI perttest copy SCI perest gass c out C perttest may22 z null gass gsifip c 64 out programs null gass gsifip c out Al gass gsiftp java auth out E windows README gass gsifip java out noram pase otal time in millisec 10 WE j z 3 Welcome to File Transfer Component Figure 5 1 File Transfer GUI due to network outrage the tool alerts the user and continues the transfer after recovering from the failure It allows the user to save transfer re
106. onal gatekeeper Section 8 2 3 Are there any features the gatekeeper does not support when compared with the Globus Personal Gatekeeper implementation Section 8 2 4 Can I transfer data using Java CoG Kit Section 8 3 What is GASS Section 8 3 What is the protocol does GASS use to transfer data Section 8 3 Are there any limitations in Java CoG Kit GASS implementation Section 8 3 1 Does Java CoG GASS support cache management Section 8 3 1 Are there any features GASS in Java CoG Kit does not support when com pared with the Globus implementation Section 8 3 3 What are the various ways I can start up the GASS server Section 8 3 2 103 C 7 GridAnt What is a process workflow Section 10 1 What is the difference between server side and client side workflows Section 10 1 What workflow engine is used in GridAnt and why Section 10 1 What is the list of tasks that GridAnt must implement Section 10 2 What is the current status of GridAnt Section 10 2 What version of Java is required for GridAnt Section 10 3 What version of Ant is required for GridAnt Section 10 3 What version of GT3 is required for GridAnt Section 10 3 How do I setup GT3 modules to work with GridAnt Section 10 3 How do I setup the Java CoG Kit to to work with GridAnt Section 3 2 How do I execute a remote job with GridAnt Section 10 5 How do I execute a third party rel
107. ong term credentials and the location of the resulting proxy file can be specified Password 1 Options Create Exit Figure 4 1 Visual grid proxy init To run this tool run the shell script visual grid proxy init or the Windows batch file visual grid proxy init bat in the lt cog install path gt bin direc tory Java CoG Kit configuration wizard This tool lets the user configure the Java CoG Kit by specifying various security related parameters such as the locations of the user s long term and proxy cre dentials locations of the files containing trusted CA certificates and some other options The tool then creates a configuration file called cog properties which is used by the Java CoG Kit software This tool is described in detail along with screenshots in Section 3 6 4 To run this tool run the shell script ogce setup or the Windows batch file ogce setup bat in the lt cog install path gt bin directory 4 4 2 Unix shell scripts The Java CoG Kit provides a number of Unix command line tools All of these tools can be found in the lt cog install path gt bin directory Each of these tools supports a help command line option that prints a detailed usage message de scribing various options These usage messages have also been included in the Appendix A of this manual grid proxy init Allows creation of a proxy By default this tool generates a GSI 3 style proxy The GSI 3 style proxies are not comp
108. otocol features of FTP and GridFTP that are sup ported by the Java CoG Kit FTP file storage and retrieval to from FTP server client server transfer third party transfer data channel protection level 13 clear safe private ASCII and IMAGE data types file data structure non print format control stream transmission mode operation in passive and active server mode GridFTP 1 0 in addition to the aforementioned Mode E parallel transfers striped transfers IMAGE data type if in mode E restart markers performance markers data channel authentication SBUF setting TCP buffer size 46 5 2 8 Limitations of the Java CoG Kit Unsupported features of GridFTP Following are the GridFTP 1 0 features not provided by the Java CoG Kit e ABUF e PIPE pipelining of commands e partial file transfer e any combination of transfer parameters that is not mentioned above for in stance mode E with ASCII If you need any of these features please send a request using the Bugzilla system as explained in Section 2 2 2 Please be sure to include a brief description of your project and how the particular feature may help the project Support for limited directory listing formats 5 3 Using GASS 5 3 1 GUI 5 3 2 Unix Shell Scripts The output of the list function in FTP servers depends on the particular FTP server operating system and the architecture of the machine that the server is running on Even the same FTP se
109. outhern California nor any contributors to the Globus Project or Globus Toolkit nor any of their employees makes any warranty express or implied or assumes any legal liability or responsibility for the accuracy completeness or usefulness of any information apparatus product or process disclosed or represents that its use would not infringe privately owned rights IN NO EVENT WILL THE UNITED STATES THE UNIVERSITY OF CHICAGO OR THE UNIVERSITY OF SOUTHERN CALIFORNIA OR ANY CONTRIBUTORS TO THE GLOBUS PROJECT OR GLOBUS TOOLKIT BE LIABLE FOR ANY DAMAGES INCLUDING DIRECT INCIDEN TAL SPECIAL OR CONSEQUENTIAL DAMAGES RESULTING FROM 10 EXERCISE OF THIS LICENSE AGREEMENT OR THE USE OF THE SOFTWARE END OF LICENSE 1 3 Other Licences We distribute a number of other libraries with the Java CoG Kit These libraries come with their own licences We strongly encourage you to inspect these licenses The can be found in the lib directories of the Java CoG Kit 1 3 1 jglobus The jglobus lib directory contains the following licences jglobus bouncycastle LICENSE jglobus cryptix LICENSE jglobus log4j LICENSE jglobus junit LICENSE jglobus puretls LICENSE 1 3 2 ogce The ogce lib directory contains the following licences ogce soaprmil1 LICENSE ogce xerces LICENSE ogce xml4j LICENSE 11 2 Preface Grids are an important development in the discipline of computer science and en gineering Rapid progress is being
110. oxy and the new public key This certificate is signed by the user rather than a CA The certificates thus form a trust chain with the user s certificate signed by the trusted CA and the proxy certificate signed by the user In GSI the long term private key of the user cannot be used for authentication It can only be used to sign the proxy and that is the only time the user needs to enter his passphrase There are cases when a service needs to acquire resources on the behalf of the user The user s proxy cannot be used for this since it resides on the user s machine and not on the service machine GSI uses a technique called delegation in such cases When a user authenticates to a service and establishes an SSL connection the user creates another proxy that is passed to the service This proxy is signed by the private key of the user proxy adding another link to the trust chain The service can use this proxy to authenticate to other resources on behalf of the user 26 4 2 Security prerequisites Most of the software provided by the Java CoG Kit uses the GSI security To be able to start using GSI you need to perform certain steps This section describes these steps in detail 4 2 1 Acquiring a user certificate As mentioned in the introduction getting a certificate for yourself is a matter of generating a public private key pair and sending it to the CA for identity verifica tion and signing The latter is site spec
111. p coldClient setPassive hotClient setActive hp Transfer a file The transfer function blocks until the transfer is complete Ry String remoteSrcFile testDir srcFile String remoteDstFile testDir dstFile append true hotClient transfer remoteSrcFile coldClient remoteDstFile append null Close both the servers This is very important as it releases the resources and saves you from running out of memory as explained in the GridFTP Programmer s Guide hotClient close coldClient close 45 A full fledged running example is available at the following location ogce org globus examples HelloGridFTP java 5 2 6 Differences between Java CoG Kit version 0 9 13 and 1 1a Java CoG Kit 0 9 13 when initially released only provided the library org globus io ftp Later Jftp was released containing package org globus ftp This package was the new implementation of the GridFTP protocol and was compatible with the Java CoG Kit 0 9 13 The two packages co existed for some time but the use of the package org globus io ftp was discouraged Now that package has been re moved from the distribution and is no longer supported Users should use the org globus ftp package A list of the FTP and GridFTP protocol features supported by the latter is provided in the next section 5 2 7 FTP GridFTP protocol features supported by the Java CoG Kit The following is a list of all pr
112. projects add with the following description Project name Institution Main contact E mail Web page Description of your project References References citing the Java CoG Kit In case you like to cite the Java CoG Kit in your papers we recommend that you use the following paper Gregor von Laszewski Ian Foster Jarek Gawor Peter Lane A Java Commodity Grid Kit Concurrency and Computation Practice and Experience Pages 643 662 Volume 13 Issue 8 9 2001 http www globus org cog java We also would like to be notified about your publications that involve the use of the Java CoG Kit as this will help us to document its usefulness We like to feature links to these articles with your permission on our Web site Additional references to Java CoG Kit and other Grid related activities can be found at Some Refernces von Laszewski http www mcs anl gov gregor bib or Some References Globus Project http www globus org research papers html 1 2 Globus Toolkit Public License GTPL Copyright c 1999 University of Chicago and The University of Southern Califor nia All Rights Reserved 1 The Software below refers to the Globus Toolkit in either source code or binary form and accompanying documentation and a work based on the Software means a work based on either the Software on part of the Software or on any derivative work of the Software under copyright law that is a work cont
113. ption amp rsl_substitution TOPDIR home albert DATADIR TOPDIR data EXECDIR TOPDIR bin executable EXECDIR a out directory TOPDIR arguments DATADIR filel environment DATADIR S DATADIR count 1 This is equivalent to the following RSL string amp rsl_substitution TOPDIR home albert DATADIR home albert data EXECDIR home albert bin home albert bin a out home albert executable directory 54 6 2 2 RSL in the Java CoG Kit 6 3 Job Submission 6 3 1 GUI Form arguments home albert data filel environment DATADIR home albert data count 1 The Java CoG Kit RSL Parser used by the Java CoG Kit job manager does not sup port the full functionality of the C Globus RSL parser Details about the differences can be found in Section 6 4 Some of the tools mentioned in this section require that you have the environment variable COG_INSTALL_PATH Set For details on configuring the Java CoG Kit please refer to Chapter 3 6 The executables described in this section can all be found in the lt cog install path gt bin directory A simple graphical interface is available by executing gt cog form The program allows you to specify job parameters in a convenient form A sample screen shot is shown in Figure 6 2 Java CoG Kit Form Submission File olx Credentials Subject O Grid O Globus OU mcs a
114. quests in a file and make the transfers at a later time Server Side Reliability The setting of RFT requires few additional steps during the setup which is ex plained in the next paragraph Given the source and the destination of the transfer this service performs the transfer reliably recovering automatically from certain types of failures such as server crashes and network outages The service after fork ing off the transfer client monitors the transfer by waiting on the transfer client If the client returns a fatal error e g when the source URL or destination URLs are not valid among other things which means the transfer is impossible to do then the service will not restart the failed transfer but if the client returns a non fatal error which can be anything from a crashed server to network outage the service will restart the transfer The transfer is started from the point where it failed before If you want to use the RFT feature you need to build the GT3 RFT client The following steps are needed 1 Get the source code distribution of the Java CoG Kit and compile it as de scribed in Section 3 4 5 2 You need to check out the gridant module from the cvs repository into the same directory where ogce and jglobus are checked out gt cvs d pserver anonymous cvs globus org home dsl cog CVS co gridant 3 Build the gt3 RFT client by using the following command gt cd ogce gt ant gta All the jar files needed to int
115. random device for random number generation random provider random algorithm numbers proxy strength proxy lifetime Java CoG Kit Configuration File Tue Feb 25 22 30 30 CST 2003 usercert home albert globus usercert pem userkey home albert globus userkey pem proxy tmp x509up_u1000 cacert usr local globus share certificates 42864e48 0 ip 140 221 56 12 Specifies the Java random provider to be used by default Specifies the random algorithm to be used for generating secure random Indicates in bits the default strength of the security proxy Specifies the lifetime in hours of the security proxy Figure 3 3 A sample cog properties file for the user albert 24 4 Security 4 1 Introduction Security is of paramount importance in the Grid computing paradigm The Globus Toolkit uses the Grid Security Infrastructure GSI 2 for secure access to Grid resources Users of the Java CoG Kit thus need to interact with the GSI in order to access the Grid resources This chapter starts with a brief discussion about the security issues involved in the Grid paradigm and how GSI addresses these issues It then provides an introduc tion to various GSI concepts like certificates certifying authorities proxies and gridmap authorization The subsequent section explains the procedures to acquire the necessary credentials A web based credential management software called MyProxy 3 is then discussed We then describe
116. rce are proceeded by the repository tag An example is jglobus org globus gram Gram java 2 3 2 Contributions This manual contains in alphabetical order contributions from Beulah Alunkal ANL Kaizar Amin ANL Jarek Gawor ANL Mihael Hategan ANL Sandeep Nijsure ANL Gregor von Laszewski ANL Additional contributions during the course of the Java CoG Kit development have been made by sorted in alphabetical order Peter Lane ANL Jason Novotny LBL now MPI Nell Rehn ANL now IBM Mike Russell UC now MPI Pawel Plaszczak ANL Carlos Pefia ANL now NYU Warren Smith ANL now NASA Andreas Schreiber DLRZ Patrick Wagstrom ANL now IIT If we have forgotten to include your name in the list of contributors please notify us We invite you to contribute to the manual or the code 2 4 Administrative Contact 2 5 Acknowledgments The project is managed by Gregor von Laszewski To contact him please use the information below Gregor von Laszewski Argonne National Laboratory Mathematics and Computer Science Division 9700 South Cass Avenue Argonne IL 60439 Phone 630 252 0472 Fax 630 252 1997 gregor mcs anl gov This work was supported by the Mathematical Information and Computational Science Division subprogram of the Office of Advanced Scientific Computing Re search Office of Science U S Department of Energy under Contract W 31 109 Eng 38 DARPA DOE and NSF support Globus Project resear
117. refer to the Sections 4 2 1 4 2 4 and 4 2 5 for instructions about acquiring the required credentials Some of the tools described in this section need that the environment variable COG_INSTALL _PATH is Set to lt cog install path gt as discussed in Section 3 6 1 The Java CoG Kit provides a tool called File Transfer GUI with an easy to use interface for connecting to multiple FTP and GridFTP servers and transferring files You can also browse the local file system with this tool File and directory transfers between the local system and remote systems and between two remote systems direct third party transfers are provided File System operations such as creating deleting and renaming files and directories are also provided This tool provides both client and server side reliability The client side reliability is provided by the tool itself using the Java CoG Kit File Transfer Service The server side reliability is provided by interfacing to the Globus Toolkit 3 OGSA based Reliable File Transfer RFT service 12 developed at Argonne National Laboratory At the client side the tool allows the user to make directory transfer requests store them in a queue and monitor the transfers If there is any failure during the transfer 40 Java CoG Kit File Transfer File Connect Security Options Help E Local System a Bus Bas q A CAcygwinihomerAdministratorworkigt31 fp Mp hotmes anl gow 21 4 Jobid Fr
118. riables ie int port 0 boolean secure true int options org globus io gass server GassServer READ_ENABLE org globus io gass server GassServer WRITE_ENABLE org globus io gass server GassServer STDOUT_ENABLE org globus io gass server GassServer STDERR_ENABLE Step 2 Start up the server in secure mode at the given port el org globus io gass server GassServer gassserver new org globus io gass server GassServer secure port Step 3 Set the appropriate Options 75 gassserver setOptions options Step 4 Display the GASS url el System out println gassserver getURL The example program is available at the following location jglobus org globus tools GassServer java In order to shut down the GASS server url using its url use the following code Step 1 Create a GlobusURL String url null GlobusURL gassURL new GlobusURL url Step 2 Shut down the server ad org globus io gass server GassServer shutdown null gassURL The example program is available at the following location jglobus org globus tools GassServerShutdown java 8 3 3 Differences between Java and Globus GASS service The Java CoG Kit GASS implementation is compatible with Globus GASS It al lows for example a Java GASS client to connect and transfer a file from a Globus GASS server or a Globus GASS client to connect and transfer a file from a Java GASS server There are certain limitations in the i
119. rring files between a client and a server As described before GridFTP uses the GSI security mechanism Thus APIs described here need security credentials in the form of an object of the class org ietf jgss GSSCredential Please refer to the Section 4 4 5 for details about how you can use the Java CoG Kit APIs for getting GSSCredential objects from your GSI proxies Get a GSSCredential object as explained above GSSCredential credential Create an instance of the GridFTPClient class String host hot mcs anl gov int port 2811 GridFTPClient hotClient new GridFTPClient host port Authenticate to the server hotClient authenticate credential Set security parameters such as data channel authentication defined by the GridFTP protocol and data channel protection defined by RFC 2228 If you do not specify these data channels are authenticated by default hotClient setProtectionBufferSize 16384 hotClient setDataChannelAuthentication DataChannelAuthentication SELF hotClient setDataChannelProtection GridFTPSession PROTECTION_SAFE Get a list of files and directories in the current directory The function returns a vector of Filelnfo objects Each of these objects contains information about a remote file such as name size modification time etc Vector fileInfoVector hotClient list Get a file from the remote server Y
120. rror files can be sent to any host on the Grid Both file staging and output errors redirection need a GASS server to be run on the machine submitting the job Please refer to Section 6 3 5 for more information about this A single GASS server can be used to stage files and receive output errors for multiple jobs running on multiple remote sites Unlike a GridFTP server a GASS server presently cannot serve files to multiple users Thus it is not possible to have just a single GASS server running as root on a host Instead a user wishing to access files using GASS has to start his own GASS server which is basically a HTTPS server On the local machine this can be done simply by creating a HTTPS server process For starting up a GASS server on a remote host however the Globus gatekeeper has to be used The Java CoG Kit provides tools and APIs for starting local and remote GASS servers To increase performance of file access GASS supports the concept of a file cache on a host running the Globus GRAM service As mentioned before executable and data files needed by jobs can be staged from different hosts If multiple jobs access the same remote file it can be cached for better performance The Globus Toolkit also allows the users to add delete and list files in the GASS cache with a command line utility called globus gass cache Adding a file could be useful for example to stage files before job execution starts in order to avoid any dela
121. rsion file private_key_file Changes the passphrase that protects the private key If the file argument is not given the default location of the file containing the private key is assumed home mike globus userkey pem Options help usage Display usage version 91 Display version file location Change passphrase on key stored in the file at the non standard location location grid info search The command grid info search can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus 1t has the same options as listed below grid info search options lt search filter gt attributes Searches the MDS server based on the search filter where some options are help Displays this message version Displays the current version number mdshost host h The host name on which the MDS server is running The default is none mdsport port p The port number on which the MDS server is running The default is 2135 mdsbasedn branch point b Location in DIT from which to start the search The default is mds vo name local o grid mdstimeout seconds T The amount of time in seconds one should allow to wait on an MDS request The default is 30 anonymous X Use anonymous binding instead of GSSAPI grid info search also supports some of the flags that are defined in the LDAP v3 standard Supported flags s scope one of base
122. rver software running on various Unix platforms may produce different results Any non Unix FTP server may produce a completely different representation The FTP library in the Java CoG Kit is designed to handle the following Unix like file list formats rw r r 1 gawor globus 528 Nov 23 15 10 Makefile and rw rw r 1 globus 117579 Nov 29 13 24 AdGriP pdf Any other file list format will not be parsed and an exception will be returned to the user If you are using the API you can write your own parser for the par ticular format you are interested in For this you have to use the parameterized list function in FTPClient Or GridFTPClient Class and intercept the input to the Datasink interface Please refer to the Javadocs for more details Some of the tools described in this section need that the environment variable COG_INSTALL_PATH is Set to lt cog install path gt as discussed in Section 3 6 1 Currently the Java CoG Kit does not provide any GUI tools for using GASS Java CoG Kit provides a number of Unix command line tools All of these tools can be found in the lt cog install path gt bin directory Each of these tools supports a help command line option that prints a detailed usage message describing various options These usage messages have also been included in the Appendix A of this manual 47 globus gass server globus gass server shutdown globus url copy 5 3 3 Windows Batch Files 5 3 4 Java CoG Kit She
123. s data from a single file can be split over multiple data connections A striped transfer distributes data in a file over multiple independent data nodes A third party transfer takes place between servers A and B while the client C manages the transfer GridFTP allows monitoring the progress of a transfer using performance markers which are essentially progress indicators sent by the server periodically GridFTP servers may also send restart markers which act as checkpoints for the transfer If a transfer fails at any time it can be resumed from the last check point The bytes already transferred before the last checkpoint do not have to be transferred again GridFTP is typically made available as a standard service on a server running the Globus Toolkit By default it listens on port 2811 GASS is a mechanism to read and write remote files using secure HTTP protocol GASS clients developed in C provide applications with special functions to open and close remote files After this applications can use the normal C library read and write functions Since Java uses the concept of streams for I O Java CoG Kit client APIs provide input and output streams to access remote files GASS can also be used by Globus GRAM servers to transfer executables and data files needed for a computational job from any host on the Grid This method called file staging frees the user from transferring the file manually Similarly job output and e
124. s org globus To use it you need to first create an account To report a bug you need to be precise in your description and include operating system JVM version and other information that can be used to better identify or replicate the condition of your error This also includes the version of Globus Toolkit services you use 2 2 3 Mailing Lists We have established a number of mailing lists to simplify the communication with the group of developers and users Restrictions on the use of the mailing list are outlined below Policy No Advertisements We do not allow you to use the mailing lists in any form of advertisement for your products or services In response to spam mail on this mailing list we have disabled the ability to post messages to this list if you are not subscribed to 1t Subscription Required If you send a message to the list and are not subscribed or you use an email address different from the one you subscribed with your message will not be posted to the list and you will not receive any notification that your mes sage was not posted Hence if you send a message to the list and do not subsequently see your message on the list or in the list archive verify that you are using an email address that is subscribed to the list and then retry your posting 13 Subscribed Lists To verify that you are subscribed to the list send an email message from the Subscription Center email account you subscribed from to ma
125. same way as discussed in Sec tion 7 4 2 7 4 4 Using the API to access MDS Netscape Directory SDK and JNDI with LDAP the provider are libraries that can be used to retrieve resource information from an MDS server The Netscape API is LDAP specific It is used for low level access to LDAP directories JNDI is a generic API for retrieving directory information In addition to these libraries there is an MDS library distributed with the Java CoG Kit Although it is deprecated we still provide it to maintain backward com patibility It is a simple layer built on top of JNDI with LDAP specific calls It 64 was originally written as a work around for some bugs found in early versions of JNDI However JNDI is much more stable now It provides a powerful and flexible interface to directory based services and is more appropriate for accessing MDS The MDS server allows both anonymous and authenticated access to the resource information Anonymous access does not require the user to have any specific credentials Details of how to connect as anonymous or authenticated user using both the Netscape and JNDI libraries are explained in the following subsections Anonymous Access Using JNDI LDAP library Here we describe the how to access MDS anonymously using JNDI The tutorial for using JNDI is available at the following link INDI nttp java sun com products jndi tutorial TOC html Setting up a connection with MDS using JNDI includes the
126. ser You can be authenticated But no remapping is done by the Gatekeeper therefore all jobs are run as the same user e Poe or mpirun for fork job manager or condor submissions However they can be performed using full delegation The implementation of the gatekeeper has a synchronization problem due to which the output might not get appropriately streamed or redirected to the client We recommend using it in batch mode without IO redirection 8 2 3 Starting the personal gatekeeper The Gatekeeper can be invoked in any of the following ways Command Line The Java CoG Kit provides a Unix shell script and a window batch file to start up the personal gatekeeper To start the Gatekeeper run the script or batch file globus personal gatekeeper available in the lt cog install path gt bin directory as follows gt globus personal gatekeeper Using the API The Personal Gatekeeper can be started from within a program using the API provided in jglobus A sample code is shown below 73 Step 1 Initialize the variables f import org globus gatekeeper GateKeeperServer GateKeeperServer gk null GSSCredential gssCred null String logFile null String gridMapFile null Properties props null int port GateKeeperServer PORT Step 2 Obtain the credentials xf GlobusCredential credentials null credentials GlobusCredential getDefaultCredential gssCred new GlobusGSSCredentiallImpl credentials
127. ss and method This manual is intended to show you that the Java CoG Kit provides an effective way of accessing the Grid through Java Developers are encouraged to inspect the JavaDoc documentation We further expect that you are familiar with the Globus Toolkit and have access to a Globus Toolkit 2 installation If you do not the Globus web page provides information about the details and how to install it Globus Toolkit http www globus org 12 2 2 Resources We support our efforts through a web site on which you find a bug tracking system Mailing lists and the code repository 2 2 1 Project Website Online information about the Java CoG Kit can be found on its home page Home page http www globus org cog java Here you can find links to the manual the code and some basic information about the project Besides this page we also maintain a project related Web page that reports on the Java and Python Commodity Grid Kits Project gt http www cogkits org 2 2 2 Bug Reporting We are using the Bugzilla system from mozilla org to track bugs and requests for enhancements for the Java CoG Kit Bugzilla provides you with an interface that guides you on submitting the bug The link to the bug system is located at CoG Kit Bugzilla http www globus org cog contact bugs In case you like to report bugs for other components of the Globus Toolkit you can use the main link at Globus Toolkit Bugzilla https bugzilla globu
128. tely Example TIMEOUT 300 sas opie text file I Each row i dividual fields are separated using po The pa ney fending of the fields are as follows Host name Operating System The name or IP address of the target machine The operating system running on the target machine This field is copied into the reports generated by the tests It has no functional role 79 CPU The type of CPU s that the target machine has This is just an informative field as well Available RAM The amount of available memory This field has no functional significance Service A Globus service available on the target machine The format of this field is lt name gt lt version gt The tests will only recognize the following service names gram gsiftp and mds Port Number Indicates the TCP port number on which the service is running The following example shows how such a table may look like hot mcs anl gov Slackware Linux 8 1 2 4 18 PIIl 866MHz x2 512 MB gram 2 2 2 5222 hot mcs anl gov Slackware Linux 8 1 2 4 18 PIIl 866MHz x2 512 MB gram 2 2 4 5224 hot mcs anl g lackware Linux 8 1 2 4 18 PIIl 866MHz x2 512 MB gsiftp 2 2 hot mcs anl g lackware Linux 8 1 2 4 18 PIIl 866MHz x2 512 Me Re deg ee E 5224 hot mcs anl gov Slackware Linux 8 1 2 4 18 PIIl 866MHz x2 512 hot mcs anl gov Slackware Linux 8 1 2 4 18 PIIl 866MHz x2 512 Mee 224 7224 cold mcs anl gov Solaris 9 Sparc 900 MHz
129. the absolute latest version of the Java CoG Kit 3 3 4 What to Choose We identified a list of possible types of users which may help you quickly decide which version is best for you Normal Users Users who want to use stable and tested Java CoG Kit tools and do not plan to modify or extend the Java CoG Kit Developers Users who want to integrate the features of the Java CoG Kit inside their own Grid applications while using the Java CoG Kit APIs Contributors Users who want to extend the features of the Java CoG Kit GT3 Users Users that will use the GT3 distribution 2 OGCE stands for Open Grid Computing Environment 18 A pictural representation of a mapping between the various user types and the Java CoG Kit distributions is provided in Figure 3 1 Java CoG Kit jglobus binary 1 1 Figure 3 1 Distribution chart of the Java CoG Kit The following summary may help you further in your decision on which version you need to obtain and how to proceed Following each item a link to the section that describes details for that item is provided For users only interested in jglobus the following choices are available jglobus stable binary Users that are interested in just the jar files without modifying them Section 4 1 jglobus stable source Users that are interested in also seeing the source to the stable binary version Section 3 4 2 jglobus development source Users that like to wor
130. the resource providers to make it easy for them to incorporate their resources into the Grid environment The Globus Toolkit provides two methods for accessing distributed data As a data transfer protocol it provides Grid File Transfer Protocol GridFTP 6 7 8 which is a common protocol independent of the underlying architectures It supports GSI and Kerberos security It also provides various features for high performance reliable and restartable data transfers as mentioned in the next section The other method Globus Access to Secondary Storage GASS 9 10 allows applications to use standard file I O interfaces open read write close for distributed access It defines a global name space using Uniform Resource Locators It also allows the use of GSI security for file access Thus it makes porting of applications to the Globus environment easy GridFTP is a set of extensions to the FTP protocol that provide increased security reliability and performance to data transfers 38 GridFTP Protocol Specification http www globus org research papers GridftpSpec02 doc 5 1 3 GASS GASS cache FTP was chosen as the basis because of its widespread use easy extensibility sep aration of control and data channels and so on GridFTP protocol 6 provides features such as GSI security for both control and data channels parallel transfers striped transfers partial file transfers and third party transfers In parallel trans fer
131. the server requests the administrators of the server network to configure the firewall such that it allows traffic destined for Globus servers Since most of the Globus servers listen on fixed well known ports this is possible Please refer to the following webpage for a list of common Globus servers and the ports they listen on Globus Toolkit firewall requirements gt http www globus org security vl 1 firewalls html e GridFTP data channels While transferring data using GridFTP introduced in Section 5 1 2 if the client your computer is set in Passive mode it starts listening on an available port and conveys this port number to the server The server will then try connecting to that port Since this port is neither fixed nor well known a firewall on the client s network will probably block it Thus the server will not be able to connect to the port The solution is to enforce the client to listen on a port number that lies in a specific range of ports and then request the network personnel to allow these ports through the firewall The methods used to restrict the client to a specific port range are described later in this section GASS servers setup for file staging and output error retrieval As explained in Section 6 3 5 a GASS server needs to be run on a client machine your machine for staging executables and data files to and retrieving output errors from jobs running on a remote GRAM server The GRAM Job Manager will tr
132. tion 6 1 1 Gatekeeper 6 1 2 Job Manager This chapter provides information about job submission using the Java CoG Kit Job submission in the Java CoG Kit is done using the Globus Resource Allocation Manager GRAM The Globus Resource Allocation Manager processes the requests for resources for remote application execution allocates the required resources and manages the ac tive jobs The Java CoG Kit provides a GRAM API for submitting and canceling a job request as well as checking the status of a submitted job The job spec ifications are written by the user in the Resource Specification Language RSL and are processed by GRAM as part of the job request The GRAM service is mainly provided by a combination of two programs the gatekeeper and the job manager When a job is submitted the request is sent to the gatekeeper of the re mote computer The gatekeeper handles the request and creates a job manager for the job The job manager starts and monitors the remote program communicating state changes back to the user on the local machine When the remote application terminates successfully or by failing the job manager terminates as well GRAM is responsible for the following e Parsing and processing the Resource Specification Language RSL specifi cations that specify job requests e Job process creation and job control e Enabling remote monitoring and managing of jobs already created The gatekeeper is a remote service th
133. ty DN of the identity represented by the proxy type Type of proxy timeleft Time in seconds until proxy expires strength Key size in bits all All above options in a human readable format text All of the certificate path Pathname of proxy file options to exists if none are given H B 0 are assumed 93 A 12 grid proxy init hours H tsh time requirement for proxy to be valid bits B b strength requirement for proxy to be valid The command grid proxy init can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus 1t has the same options as listed below Syntax java ProxyInit options java ProxyInit help Options help usage Displays usage version Displays version debug Enables extra debug output verify Verifies certificate to make proxy for quiet q Quiet mode minimal output limited Creates a limited proxy independent Creates a independent globus proxy old Creates a legacy globus proxy hours H Proxy is valid for H hours default 12 bits B Number of bits in key 512 1024 2048 4096 globus Prints user identity in globus format policy lt policyfile gt File containing policy to store in the ProxyCertInfo extension pl lt aLa gt OID string for the policy language policy language lt oid gt used in the policy file path length lt l gt Allow a chain of at most 1 proxies t
134. uring this process please change the passphrase using the tools described in Section 4 4 2 or 4 4 3 e Please don t store the private key and proxy files mentioned above on re movable media like floppy disks or zip disks which may get stolen e If you want to copy the private key and proxy files mentioned above to some other host please don t use insecure methods like FTP or rcp Users may use different computers to access services on the Grid These may in clude computers at work computers at home and public access terminals All of these machines may not be very secure and trustworthy On each of these ma chines the user needs to have his security credentials in order to authenticate to the Grid services But it is not secure to copy the long term credentials user s private key and certificate to every machine as they may be compromised Instead it is desirable to have a central secure server trusted by the user where the user can store his credentials and later retrieve a proxy whenever needed for authentication Since proxies have a limited lifetime that can be controlled the compromise of a proxy does not cause much damage MyProxy 3 serves this purpose A securely managed MyProxy server that is trusted by the user can provide an effective way of credential management MyProxy is available from the following website MyProxy Homepage gt http www ncsa uiuc edu Divisions ACES MyProxy First the user has to store a cre
135. use this jar file The patched library has a bug fix for certain security related issues Setting up an anonymous connection using Netscape Directory SDK includes the following steps Step 1 Create an LDAPConnection String binddn null LDAPConnection ld null ld new LDAPConnection Step 2 Connect to host and port E ld connect host port Step 3 Retrieve the results Eb String baseDN mds vo name local o grid String filter objectclass LDAPSearchResults myResults null myResults ld search baseDN LDAPv2 SCOPE_ONE filter null false Step 4 Display the results while myResults hasMoreElements LDAPEntry myEntry myResults next String nextDN myEntry getDN System out println nextDN LDAPAttributeSet entryAttrs myEntry getAttributeSet System out printin entryAttrs System out println The org globus mds gsi library provides bindings for both Netscape Directory SDK and JNDI with LDAP provider for establishing secure connection with GSI enabled Idap servers such as an MDS 2 server The bindings are based on the SASL protocol defined in the RFC document available at the following link RFC http www ietf org rfc rfc2222 txt The library is used in the same manner as any other SASL mechanism The only differences are the properties that can be passed to the underlying SASL mech anism The properties that needs to be set in
136. usted CAs You should only use a service if you trust the CA that signed the certificate This essentially depends on whether you trust the administrators of the domain that hosts the service If you decide to trust a particular CA you need to obtain the certificate of that CA from its administrators For example administrators of the Globus CA make the certificate available for download at Globus CA Certificate ftp ftp globus org pub gsi globus protect unhbox voidb x kern 4 2 5 Gridmap files 4 2 6 Protecting credentials 06em vbox hrulewidth 3em ca 42864e48 0 Once you obtain the CA certificate you need to let the Java CoG Kit know that you trust that CA You can do this either by manually editing the lt user home gt globus cog properties file or using the Java CoG Kit configura tion wizard explained in Section 3 6 4 Successful authentication alone is not sufficient for a user to use a service Au thentication only convinces the service that the user is indeed who he she claims to be In addition the service has a right to check whether the user is authorized to use the service It checks this on the basis of the user s Grid identity i e his Distinguished Name as found in the user s certificate Currently Globus services perform authorization using a file called grid mapfile that has to be present on every machine hosting a Globus service This file is prepared by the GSI administrator of that site depending on local
137. w if my transfer fails How do I restart it Currently you can do this only using the GridFTP APIs Please check the GridFTP Programmer s Guide available at http www globus org cog jftp guide html Why am I not able to obtain a list of files in a directory on a FTP GridFTP server Section 5 2 8 I m getting the following error when I am trying to transfer a file or do a file listing 425 Can t build data connection Connection refused What does it mean and how can I fix it Your computer may be behind a firewall that does not allow the data con nection Please see the GridFTP data channels paragraph in Section 4 5 How do I start a GASS server on my local machine Different methods of doing this are described in Section 5 3 How do I start a GASS server on a remote machine Section 5 3 5 How do I store and retrieve files using GASS Different methods of doing this are described in Section 5 3 How do I transfer files between two machines using GASS Different methods of doing this are described in Section 5 3 What are the differences in file transfer APIs between CoG 0 9 13 and later versions What is GRAM Section 6 1 What is a Gatekeeper Section 6 1 1 What is a job manager Section 6 1 2 What is file staging Section 6 1 4 What is RSL Section 6 2 How do I start a job from command line Section 6 3 2 and Section 6 3 3 100 C 5 Grid Information Service C 5 1 Genera
138. ween query frequency and information update frequency of a value in the MDS For example if a user requests every second information that is only updated every thirty seconds this will lead to a waste of resources We encourage Grid programmers to avoid such situations by having a clear understanding on how the information is updated You can find out more about this from the MDS web pages 70 7 7 Implementation Details of MDS 2 2 version MDS 2 2 uses OpenLDAP Version 2 0 22 which implements LDAP Version 3 The security in OpenLDAP is provided by the Simple Authentication and Security Layer SASL which also uses GSS API SASL is a method for adding authenti cation support to connection based protocols MDS 2 2 uses Cyrus SASL Version 1 5 27 SASL is a convenient generic interface for secure application development By itself SASL does not provide any security It relies on underlying technolo gies to provide the actual identity authentication and message protection services desired by applications communicating over a network Applications may install and request the use of particular mechanisms or use a default mechanism provided by the SASL implementation MDS 2 2 also uses OpenSSL Version 0 9 6b OpenSSL provides the Secure Sock ets Layer SSL implementation used by the GSI OpenSSL is an open source implementation of SSL used to build the GSS API 7 8 Differences between Java and Globus tool The command line arguments for the gri
139. with the destination server notpt no third party transfers Turn third party transfers off on by default nodcau no data channel authentication Turn off data channel authentication for ftp transfers Applies to FTP protocols only Protocols supported gass http and https ftp ftp and gsiftp 90 A 7 grid cert info file The command grid cert info can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus 1t has the same options as listed below Syntax A S grid change pass phrase java CertInfo help file certfile all subject Displays certificate information Unless the optional file argument is given the default location of the file containing the certficate is assumed home mike globus usercert pem Options help usage Display usage version Display version file certfile Use certfile at non default location globus Prints information in globus format Options determining what to print from certificate lT Whole certificate subject Subject string of the cert issuer Issuer startdate Validity of cert start date enddate Validity of cert end date The command grid change pass phrase can be found in the build cog 1 1a bin directory This commandline provides a convenient wrapper to a Java class Thus it has the same options as listed below Syntax java ChangePassPhrase help ve
140. y in processing 39 Currently Java CoG Kit does not provide support for GASS cache Please see Section 5 3 6 for more details 5 1 4 Other file transfer mechanisms 5 1 5 Security Requirements 5 2 Using GridFTP 5 2 1 GUI Client Side Reliability In addition to the specific mechanisms discussed above methods like regular FTP and Secure Copy scp can also be used for file transfers over the Grid though they may not satisfy the different requirements discussed in Section 5 1 1 The Java CoG Kit provides client APIs for FTP transfers For these transfers you have to use the username password authentication method You cannot use the GSI authentication Also these transfers cannot use the features provided exclusively by GridFTP as mentioned in Section 5 1 2 Furthermore the use of FTP over untrusted networks is discouraged because it sends passwords across the network in cleartext Secure copy scp which is based on SSH does not have the problem of cleartext passwords It may suffer however from man in the middle attacks due to the lack of certificates Toolkits like dsniff 11 demonstrate this vulnerability We do not discuss these mechanisms in more detail as they are beyond the scope of this document Instead we concentrate on GridFTP and GASS Both GridFTP and GASS use the GSI for authentication and secure data transfer Thus you will need to acquire the GSI credentials before you can transfer any data Please
141. y portion of it thus forming a work based on the Software and give a copy or copies of such work to others either in source code or binary form you must meet the following conditions a The Software must carry prominent notices stating that you changed specified portions of the Software b The Software must display the following acknowledgement This product includes software developed by and or derived from the Globus Project http www globus org to which the U S Govern ment retains certain rights 7 You may incorporate the Software or a modified version of the Software into a commercial product if you meet the following conditions 10 11 a The commercial product or accompanying documentation must display the following acknowledgment This product includes software developed by and or derived from the Globus Project http www globus org to which the U S Govern ment retains a paid up nonexclusive irrevocable worldwide license to reproduce prepare derivative works and perform publicly and display publicly b wm The user of the commercial product must be given the following notice Commercial product was prepared in part as an account of work sponsored by an agency of the United States Government Neither the United States nor the University of Chicago nor University of South ern California nor any contributors to the Globus Project or Globus Toolkit nor any of their employe
142. y to establish a connection with the GASS server on the client machine Since the ports used by GASS servers are neither fixed nor well known a firewall in the client s network will probably block the connection The so lution is the same as the one mentioned above for GridFTP data channels restricting the port range Connecting to Globus servers behind a NAT firewall Some networks em ploy a firewall that performs Network Address Translations for the hosts in those networks Please refer to the following webpage for a discussion of problems posed by NAT firewalls and possible solutions to these problems Globus Toolkit firewall requirements gt http www globus org security vl 1 firewalls html 36 Restricting the port range used by GridFTP data channels and GASS servers The port range can be set either in the Java CoG Kit configuration file Le lt user home gt globus cog properties or through the Java system proper ties e g set from the command line To set the port range in the configuration file just add the following line to the file tcp port range lt min gt lt max gt For example tcp port range 6000 6060 To set the port range using the system properties set the org globus tcp port range property For example gt java Dorg globus tcp port range 6000 6060 classpath 4 6 Random number generation issues For security related tasks the Java CoG Kit tools and APIs must initialize a se cure seed

Java CoG Kit Manual

Contents

Download Pdf Manuals

Related Search

Related Contents