Garuda User`s Handbook
Contents
1. gt 9 2 gt gt lt gt ED ED CE DE nm Fig 5 2 GridFTP GUI window 5 Upload your certificate to GridFTP client by clicking Tools gt Options as shown in Fig 5 3 mm K Bona aumentcaton in Een Kamen eG Name iele Fig 5 3 Click Tools gt Options Chapter 5 GARUDA GridFTP Client Page 46 GARUDA User Handbook Fig 5 4 Click Security tab in Options dialog box 6 Click the Browse to upload your certificate usercert pem to the client GUI as shown in Fig 5 4 amp Fig 5 5 mi Open E4 igca cert p12 5 usercert pem U userkey pem File Name usercertpem Files of Type All Files v Ceana Fig 5 5 Upload usercert pem to the client tool Chapter 5 GARUDA GridFTP Client Page 47 GARUDA User Handbook What if I don t have a pem certificate file Sol Refer Chapter 4 IGCA command reference to convert your p12 certificate file to pem certificate file How to get my p12 file Sol You need to export your certificate from the web browser Refer the below link for more information http ca garudaindia in index php information fag export the certificates obtain pfxps12 file 7 Click Tools gt Grid Authentication to authenticate with the GridFTP server Enter the passphrase and click Authenticate button as shown in Fig 5 6 j Grid Authentication gt Status of current proxy Identity No v2. Example RANK CPU MHZ 2 FREE MEM MB Chapter 4 GARUDA Command Reference Page 35 GARUDA User Handbook How to set necessary environment variables to access gridway commands Fig 4 13 depicts the necessary commands to set up the gridway user environment at gridfs machine Note The path of gridway installation may vary on different clusters henryogridfs henryllgridfs export GW LOCATION usr local GARUDA gu henryllgridfs export PATH GW LOCATION bin P TH henryvlgridfs I Fig 4 13 Exporting gridway location How to submit a job to gridway scheduler gwsubmit is the command used to submit jobs in gridway as shown in Fig 4 14 E g gwsubmit vt lt filename gt henry gridfs henry gridfs gwsubmit wt test jt JOB ID 22559 henrylgridfs ij Fig 4 14 gwsubmit command to submit jobs in gridway Chapter 4 GARUDA Command Reference Page 36 GARUDA User Handbook How to check the status of my job As shown in Fig 4 15 gwps is the command used to check the status of gridway Job E g gwps lt job id gt henryogridfs henry gridfs gwps 22559 henry 22559 prol 14 39 50 0 00 00 0 01 03 test jt henryBgridfs E Fig 4 15 gwps command to check the status of the job How will I troubleshoot my gridway job Debugging job execution GridWay reporting and accounting facilities provide information about overall performance
3. henry8gridfs Fig 6 1 Setting up a GSRM client environment Setting up of GSRM Libraries in LD LIBRARY PATH As shown in Fig 6 2 export LD LIBRARY PATH to locate GSRM libraries henry eridfs henry gridfs export LD LIBRARY PATH usr local GARUDR GSRH gsrm clients sto rm cgsi soap lib usr loeal GRRUDA GSRH gsrm clients dpm lib henryBgridfs I Fig 6 2 Setting LD_LIBRARY_PATH Chapter 6 GARUDA Storage Resource Manager GSRM Page 55 How to download credentials with VO attributes In order to access GSRM services user needs to have a valid VO credentials Fig 6 3 shows how to download credentials from MyProxy repository and add VO attributes to it Command myproxy logon voms lt VO_NAME gt E g Below example generates credentials for VO called GarudaAdmin you need to replace this with your VO name henry gridfs myproxy logon voms GarudaAdmin Four VO Name Enter MyProxy pass phrase Your identity DC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar henrys cd acb ernet in CN 793469285 CN 1837609429 CN 1911907363 Contacting garudasec ctsf cdac org in 15002 DC IN DC GARUDAINDIA OFC DAC OUSC TSF CN garudasec etsf edac org in Garudahdmin Done Creating DEGEY cw ww ence ee tann med es MUD O WN WM O UN OW DW WG OG NUW GDB GU vad AS AMT Eg nen Done Your proxy is valid until Tue Hay 17 02 58 34 2011 A credential has been received for user henry in tmp x509up 4500 he
4. local GARUDA GLOBUS LOCATION usr local GARUDA GLOBUS 4 0 8 GridPortal http 192 168 60 40 GridPortall 3 IGCA Commands http ca garudaindia in index php information fag Convert p12 cert to pem usr local GARUDA bin convert pl12 pem mailto igcahcdacb ernet in Single sign On facility in grid 1 grid proxy init 2 gsissh p 2222 lt remote cluster gt e g gsissh p 2222 gg blr 3 gsiscp P 2222 lt file name gt lt remote cluster gt e g gsiscp P 2222 bashrc gg blr Fig 3 4 Login to Gridfs machine GARUDA HEAD NODE gridfs ctsf cdac org in 192 168 60 40 This node is called as GRIDFS You need a GARUDA Account to access this node Chapter 1 This node is reachable only via NKN Network As per the policy internet users cannot login to GRIDFS They have to use Grid Portal Chapter 2 Chapter 3 Accessing GARUDA via Command line Page 20 Chapter E KL TB e e Grid Command reference GARUDA User Handbook Chapter 4 Grid Command reference What is Globus Toolkit Globus Toolkit 1s a popular grid middleware among the scientific grid computing platforms Globus incorporates not just job management but also storage management security provisioning data movement monitoring It 1s also a toolkit for developing additional services based on the same infrastructure including agreement negotiation notification mechanisms trigger services and information aggregation How to set n
5. q Fig 6 16 Using ptg function to make a get request to GSRM Chapter 6 GARUDA Storage Resource Manager GSRM Page 65 GARUDA User Handbook Step 2 Download the file from GSRM As shown in Fig 6 17 globus url copy command is used to get the file from GSRM This command reguires the fransfer URL obtained in step 1 Refer Fig 6 16 fhenry gridfs globus url copy gsiftp gsrm ctsf cdac org in gsrm ctsf cdac gt rg in storage3 GerudaAdmin 2D47705 16 sample_file 11243 0 AES THOS FYNED YFFED n garm henryBgridfs Heil URL obtained from stepi Local file path Fig 6 17 globus url copy command is used to download the file How to reserve a space in GSRM As shown in Fig 6 18 reservespace function of the clientSRM command is used to reserve space in GSRM This function is used to reserve a space in advance for the upcoming reguests to get some guarantee on the file management Unit of the space should be specified in bytes Fig 6 18 reserves a total space of 10MB and reguests for a minimum space of size 5MB The returned space token have to be used 1n all the reguests that work on this reserved space green colored rectangle in the image henryBgridfs clientSRM reservespace e httpg gsrm 8446 a 10000000 b 5000000 r O Sending ReserveSpace request to httpg gsrm 3446 Request status Total space te reserve Minimum size of the space statusCode 5RM SUCCESS O fo reserve SRM Response
6. 257108 072567320 PRS prlhpc lan prl res in b l Linux 6 18 B e 406 6 3158 300 14094 16051 210197 230993 0 35 304 PBS gg che local 7 i Linux2 6 10 53 x66 6 3159 800 15364 16051 211057 230993 0 236 304 PBS gg hpd cdac org in B 1 Linux2 6 16 53 x86 6 3166 183 11 amp 16D51 193393 230993 0 120 474 PRS og blr t g 9 1 Linux 6 9 42 E 406 6 2600 300 5043 7978 2944366 4180562 1 13 1102 PBS master iitg ernet in 10 1 0 0 ad 0 0 0 0 0 begabung cc litd ac in 11 1 Linux2 6 5 7 24 power 1554 400 082 3659 43254 86761 0 13 64 PRS cnode63 SP20C 113C Cnet henryloridts 1 Fig 4 5 gwhost command to check the available resources in grid How to check my authentication to remote cluster globusrun command is used to check user authentication to remote cluster as shown n Fig 4 6 henry gridis henry gridfs globusrun a r gg blr GRAM Authentication test successful henry gridfs globusrun a r gg hyd GRAM uthentication test successful Cluster hastname henry gridfs J Fig 4 6 globusrun command to check user authentication to remote cluster Chapter 4 GARUDA Command Reference Page 25 GARUDA User Handbook How will I login to remote cluster using my proxy certificate You can login to the remote cluster using the single sign on feature in Garuda Single sign on SSO is a property of access control of multiple related but independent software systems With this property a user logs in
7. How to check the status of the job and view or download output error file Click Job Management gt gt gt gt Job Info as shown in Fig 2 4 Jeb Nanagemont File Management Help Notices FAOS Administration Jobinfo Select Job id Cancel the job Job Status the job has been completed Click Job Infe Fig 2 4 Get Job Status and view or download output error files How to manage files stored in GRIDFS GARUDA Head node and GSRM Garuda Storage Resource Manager via Grid Portal Click File Management tab as shown in Fig 2 5 Administration Job Management Help Notices FAQ File Management File Management on Grid Head Node Click File Management tab Source Destination kestes Fig 2 5 Grid Portal File Management Section Chapter 2 Accessing GARUDA via Grid Portal Page 14 Accessing GARUDA via Command line GARUDA User Handbook Chapter 3 Accessing GARUDA via Command line GARUDA Grid is connected across the country through NKN network of 1Gbps Steps to access via NKN 1 An interface node has been installed in each partner institution which is hereafter referred to as GARUDA INTERFACE NODE Any user who wants to connect to GARUDA should get an access to GARUDA INTERFACE NODE at his her institution 2 Contact your local system administrator to get the information about the GARUDA INTERFACE NODE located at your institution Ex IP address reachab
8. Knowledge Network The backbone of GARUDA Grid Appendix I Glossary Page 81 GARUDA User Handbook P Public Key Certificate key to be shared with other party for communication Private Key Certificate key that should not be shared with anyone PKI Enables users to securely and privately exchange data and money through the use of a public and a private cryptographic key pair that is obtained and shared through a trusted authority PEM Privacy Enhanced Mail Certificate format used in grid Pl2 amp PFX PKCS 12 contains certificate s public and private keys password protected PKCS Public Key Cryptographic System Proxy Certificate A short lived certificate for accessing grid services Generated using public and private keys Paryaveekshanam Grid monitoring application for GARUDA O OOS Ouality of Service R Revoked Certificate A certificate is said to be invalid once it is revoked Re key Certificate The lifetime of IGCA user certificates and host certificates are valid for 12 months User must rekey the related certificate when his or her certificate will expire After you rekey the certificate you will own the brand new certificate and private key The previous certificate and private key will be revoked Appendix I Glossary Page 82 Registration Authority A Registration Authority RA is an authority in a network that verifies user reguest for a digital certificate
9. and tells the Certificate Authority CA to issue certificate S SOA lt Service Oriented Architecture U User Certificate Certificate issued to user for accessing Grid V VOMS VOMS is a system for managing authorization data within multi institutional collaborations VOMS provides a database of user roles and capabilities and a set of tools for accessing and manipulating the database and using the database contents to generate Grid credentials for users when needed W WS GRAM The Grid Resource Allocation and Management GRAM web service provides a single interface for reguesting and using remote system resources for the execution of jobs The most common use of GRAM is remote Job submission and control It is designed to provide a uniform flexible interface to job scheduling systems Bibliography References Garuda Official Website http garudaindia in Grid Portal http portal garudaindia in Indian Grid Certification Authority http ca garudaindia in GARUDA VOMS Registration http voms garudaindia in Grid Support http gridsupport garudaindia in Globus Toolkit http globus org Gridway http gridway org GARUDA GridFTP Client http portal garudaindia in gridftp gui GSRM http storm forge cnaf infn it documentation client examples 0 GSRM http csngwanfo in2p3 ir mediawiki index php SRM StoRM 1 GSRM http portal garudaindia in GARUDA REPO gsrmv1 1 gsrmv1 1 Manuals gs
10. cdac org in home Garuda dmin sample file 0 status statusCode SRH SUCCESS 0 Denotes the transfer status Fig 6 15 Verifying the transfer performed in step 2 Chapter 6 GARUDA Storage Resource Manager GSRM Page 64 GARUDA User Handbook How to download a file from GSRM using clientSRM command Downloading a file from GSRM server involves two steps Step 1 Prepare to get reguest from GSRM clientSRM ptg Step 2 Download the file globus url copy Step 1 Prepare to get reguest from GSRM As shown in the Fig 6 16 put to get reguest ptg function of the clientSRM command 1s used to make get reguest to GSRM server Command clientSRM pig e lt GSRM End Point gt s lt SURL_to_get_file gt p henryBgridfs clientSRM ptg e httpg gsrm 8446 s srm gsrm dpm ctsf cda c org in home Garuda dmin sample file p Sending PtG request to httpg gsrm 3446 Polling request status Current status SRM REQUEST OUEUED Ctrl c to stop polling Request status statusCode 5RH SUCCESS 0 SRH Response ceguestToken db141255 ce581 404b b6Ze 69645f51d47c2 arrayOfFileStatuses size 1 0 status statusCode 5RH FILE PINNED 22 0 sourceSURL srm gsrm dpm ctsf cdac org in home GarudakAdmin sample_ fi le URL required to download the file from GSRM 0 remainingPinTime 7199 0 tzansferURL gsiftp gsrm ctsf cdac org in gsrm ctsf cdac org in sto rage3 Garudakdmin sample
11. generated using op If you are from resourcg providing Garuda partner institute ensure that you have a login in institute s C p12 cert password upload p12 cert file Fig 1 4 Garuda Credential upload What if I don t have a p12 certificate file Sol Visit http ca garudaindia in index php information fag export the certificates obtain pfxps12 file Refer http voms garudaindia in for more information Chapter 1 Steps to access GARUDA Page 6 GARUDA User Handbook 3 The image below shows the acknowledgement page that you will receive once your certificate has been successfully uploaded to the GARUDA credential repository Welcome Henry gt ukumar Your have a valid credential on the Garuda repository Validity of your credential 3751 02 41 156 3 days Click on the VO names listed below to register Your certificate is uploaded to repository Fig 1 5 GARUDA Credential upload acknowledgement 4 Now you can see a list of available VO s Select your VO Name from the list At least one VO registration is mandatory E g It can be the name of your working domain List of Os configured on this server Atleast a VO Registration is mandatory GeoPhysics Bioinformatics Garuda dmin OSDD ClimateModelling CAE euindia ToolsDeveloper MaterialScience IndianHeritage Fig 1 6 Select your VO What if my VO is not listed here Sol Send a mail to voms cdacb ernet in Chapter 1 Steps
12. igca cert pl2 out usercert pem nokeys clcerts Enter Import Password AC verified OK C my cert gt Fig 4 23 Generate usercert pem 8 Extracting userkey pem As shown in Fig 4 24 type the below command to extract userkey pem openssl pkcs12 in igca cert p12 out userkey pem nocerts Chapter 4 GARUDA Command Reference Page 41 GARUDA User Handbook When prompted for Import Password enter the password which you had set during your certificate export from the browser When prompted for Enter PEM pass phrase set new password to protect your userkey pem file Note You can also enter the same password which you had set during your certificate export from the browser Mmy cert gt openssl pkcs12 in igca cert pl2 out userkey pem nocerts Enter Import Password C verified OK Enter PEM pass phrase erifying Enter PEM pass phrase Fig 4 24 Generating userkey pem What is p12 or pfx file e p12 or pfx Certificate format that contains both public and private key e This file is normally protected with a password What is pem file e Certificate format used in grid e usercert pem public key e userkey pem private key Chapter 4 GARUDA Command Reference Page 42 Chapter y Ydd gt MOON Done GARUDA GridFTP Client GARUDA User Handbook Chapter 5 GARUDA GridFTP Client What is GARUDA GridFTP client Garuda GridFTP Client is a GUI for performing file transfers using
13. nn 3 Chapter 2 Accessing GARUDA via Grid Portal eeeeeeennenenenennnnannnneennee 11 Chapter 3 Accessing GARUDA via Command line mmiimieeeeeeeeeeenann enne 17 Chapter 4 Grid Command referenc vv veeeenYY YY Y YY YY Y YE Y EE EE EE EE ELE CELF EC CDC CCD LD LED EEDD 23 Chapter 5 GARUDA Grid EP Client een 45 Chapter 6 Garuda Storage Resource Manager GSRM ccccecceceeeeeeeeeeeeeeeeeaeens 53 Chapter 7 Ond SUPPORT ec RG AAA AR ET FAR 73 PRIS 11015 AN kl Canada HER TER AEA T E A EF FY FF NSF 78 BILD OS ODRY ae ee TRUTH ee Mut 84 GARUDA User Handbook Introduction GARUDA is India s National Grid Computing initiative bringing together academic scientific and Research communities for developing their data and compute intensive applications with guaranteed QoS GARUDA grid is an aggregation of resources comprising of computational nodes mass storage and scientific instruments distributed across the country The nation wide computational grid GARUDA spans across 17 cities with an aim to bring Distributed Grid networked infrastructure to academic labs research labs and industries in India Reference http garudaindia in Who This Book Is For This book is intended for the people who are interested in using the GARUDA Grid for meeting their computational needs It expects basic knowledge about grid computing and UNIX It provides an easy to follow user s guide of Garuda w
14. of the clientSRM command Refer Fig 6 13 for more information about ptp function of a clientSRM command Note Type clientSRM lt function gt help to view detailed options for a particular function Chapter 6 GARUDA Storage Resource Manager GSRM Page 67 GARUDA User Handbook How to release the reserved space in GSRM As shown in Fig 6 20 ReleaseSpace function of a clientSRM is used to release the reserved space This function reguires spaceToken as an input which was returned during the reservespace reguest Refer the green colored rectangle in Fig 6 18 henryBgcridfs client5RH ReleaseSpace e httpg csrm 0446 t feO0Obef1d 069c 417a ad42 56647f1405bc Sending ReleaseSpace request to httpa gsrm 8446 seeeeeeseseseessesesessesssessesesesssssesesesssssseees 2 Space token returned during reserve space Peguest status request atatusCodc SRH SUCCESS O Fig 6 20 Release the reserved space How to view the reserved space information As shown in Fig 6 21 userSpacelnfo command can be used to view the reserved space information e It allows the user to view the respective space usage details in KB MB GB e User can get the reserved space details of all the space tokens associated with the user e User can also view overall space usage i e reserved and unreserved Command userSpacelnfo e lt GSRM End Point gt s Chapter 6 GARUDA Storage Resource Manager
15. or grs AbortReguest or ar AbortFiles or af ExtendFileLifeTime or efl Space management functions Reservespace or rs GetSpaceMetadata or gsm ReleaseSpace or rsp GetSpaceTokens or gst Type client5RM lt requestName gt h to get help about lt requestName gt options Fig 6 10 clientSRM command and its various options Note Type clientSRM lt function gt help to view detailed options for a particular function Chapter 6 GARUDA Storage Resource Manager GSRM Page 61 GARUDA User Handbook How to upload a directory to GSRM As shown in Fig 6 11 putfiles to gsrm sh is the command used to upload a directory to GSRM server Command putfiles to gsrm sh lt Local source path gt lt SURL gt henry gridfs henryfgridfs putfiles to gsrm sh home henry weka srm gsrm 8446 dpm ctsf cdac org in home Garudaddmin test_dir source home henry weka destination srm gsrm 8446 dpm ctsf cdac org in home Garudaddmin test dir henry gridfs Fig 6 11 Uploading a directory to GSRM How to download a directory from GSRM As shown in Fig 6 12 getfiles from gsrm sh is the command used to download a directory from GSRM server Command getfiles from gsrm sh lt SURL gt lt directory_level gt lt Local destination path gt henry gridfs getfiles from gsrm sh srm gsrm 8446 dpm ctsf cdac org in hom e Garudaadmin test dir 2 home henry source srm gsrm
16. provided by the system administrator in your institute organization Chapter 3 Accessing GARUDA via Command line Page 18 GARUDA User Handbook 2 192 168 63 92 PuTTY Fig 3 2 Login to GARUDA Interface Node G Make sure that you can reach GRIDFS using PING command as shown in Fig 3 3 gridfs ctsf cdac org in GARUDA Grid Head Node henryBxnD2 ping gridfs ctsf cdac org in PING gridfs ctsf cdac org in 192 168 60 4D 56 54 bytes o 64 bytes from gridfs ctsf cdac org in 192 165 60 40 icmp 346 ms gridfs ctsf cdac org in ping Statistics 1 packets transmitted 1 received US packet loss time Oms ret min avgy max mdew 0 346 0 346 0 346 0 000 ms pipe 2 henry xno2 f Fig 3 3 Check for reachability to gridfs What is PuTTY PuTTYisa program that allows windows users to connect to UNIX like machines Chapter 3 Accessing GARUDA via Command line Page 19 GARUDA User Handbook H Use SSH command to login to the GARUDA HEAD NODE gridfs ctsf cdac org in with your username and password Refer Chapter 1 Step 2 obtained from the GARUDA The command is given below Refer Fig 3 4 SSH lt username gt gridfs ctsf cdac org in henry eridfs henry xnO2 ssh henry gridfs ctsf cdac org in henryBgridfs ctsf cdac org in s password Last login Wed Jun 8 09 35 00 2011 from xn02 ctsf cdac org in Welcome to Garuda Head node FRERETEREERETERKERETERKEEEETEE GARUDA HOME usr
17. sizeOfTotalReservedSpace 10000000 sizeOfGuaranteedReserved5pace 5000000 lifetimeOfReserved5pace 6042500 spaceToken of the reserve reguest spaceToken fe0bef1d 069c 417a ad43 56647f1405bc f 4 Fig 6 18 Reserve space in GSRM Chapter 6 GARUDA Storage Resource Manager GSRM Page 66 GARUDA User Handbook How to view the information of a reserved space As shown in Fig 6 19 getspacemetadata function of the clientSRM command 1s used to view the information of a reserved space This function reguires a space token spaceToken that was returned during reservespace reguest Refer green colored rectangle in Fig 6 18 henry gridfs clientSRM getspacemetadata e httpg gsrm 8426 s fe0bef1d 069c 417a ad43 56547 1405hc Ly y Ng y Space token returned during VESEVESPACE Reguest Status request statusCode SRA_SUCCESS 0 SRH Response arrayOfSpaceDetails size 1 0 spaceToken fe0befi1d D6Sc 417a ad43 5t654271f140Sbc 0 status statusCode S5RH SUCCESS 0 0 zetentionPolicyInfo 10 0 0 caner Garuda Admin al 0 Buaranteed5ize 50000DD 0 unused5ize 5000009 0 lifetimeissigned 1935 0 lifetimeLeft 602865 Total and minimum guranteed space Fig 6 19 View the information of a reserved space Note The reserved space token can be made use during your file upload request to GSRM This can be achieved using f option to the prepare to put ptp function
18. test dir Reguest status statusCode 5RH SUCCESS 0 Fig 6 6 Creating directory in GSRM server Chapter 6 GARUDA Storage Resource Manager GSRM Page 58 GARUDA User Handbook How to remove a directory in GSRM As shown in Fig 6 7 rmdir function of the clientSRM command is used to the remove directory 1n GSRM server clientSRM rmdir e lt GSRM End Point gt s lt SURL gt 5 henry eridfs henry gridfs clientSRM rmdir e httpg gsrm 8446 s srm gsrm 8446 dpm e tsf cdac org in home GarudaAdmin test dir Sending Rmdir request to httpg gsrm 446 Reguest status statusCode S5RHM SUCCESS 0 henry8Bgridfs J Fig 6 7 Remove the directory in GSRM Note The above command is used to remove empty directory Use r option to remove the directory recursively Type clientSRM rmdir help for more information How to remove a file in GSRM As shown in Fig 6 8 rm function of the clientSRM command is used to remove a specific file clientSRM rm e lt GSRM End Point gt s lt SURL gt henry gridfs henry gridfs clientSRM rm e httpg gsrm 8446 s srm gsrm 8446 dpm ctsf cdac org in home GarudaAdmin test dir sample file txt Fig 6 8 Removing a specific file in a GSRM server Note Type clientSRM rm help for more information Chapter 6 GARUDA Storage Resource Manager GSRM Page 59 GARUDA User Handbook How to list the files
19. to peer data grid solution for GARUDA An SRM can be defined as a middle ware component that manages the dynamic use of storage resources in the grid GSRM provides e File and Directory management Upload download of files and directories e Dynamic space management Reserving amp release of storage spaces and setting lifetime for the files e User friendly interfaces like command line APIs and web interface e Interfacing with the Compute Grid Allows users to transfer their required data from compute grid to GSRM and vice versa What are the steps to access GSRM Below are the steps to access GSRM services a Login to GRIDFS machine Refer Chapter 3 b Generate valid credentials with VO attributes c Setting up of GSRM user environment d Access GSRM via GSRM client commands What are the GSRM end points in GARUDA GSRM end points are referred as GSRM servers Below are the GSRM end points in GARUDA Location End point URL Hostname Bangalore httpg gsrm ctsf cdac org in 8446 gsrm Hyderabad _ httpg srmhyd cdac in 8446 srmhyd Table 6 1 GSRM End points in GARUDA Note End points use protocol httpg Chapter 6 GARUDA Storage Resource Manager GSRM Page 53 GARUDA User Handbook What is SURL A le or a directory located in a GSRM storage space is identified by a Site URL SURL Below 1s the format of SURL srm host port StFN root path site file name An example of SURL is srm gsrm 8446 dpm ctsf cda
20. usage How to check the status of my credential proxy grid proxy info command is used to verify the status of user credentials as shown in Fig 4 4 henryogridfs henry gridfs grid proxy info subject DC IN DC GARRUDAINDIR O C DAC OU SCTSF CN Henry Sukumar ernet in CN 387787009 CN 211132 6649 CN 1176384196 issuer DC IN DC GARUDAINDIA O C Dac OU CTSF CN Henry Sukumar ernet in CN 387787009 CN 2111326649 identity DC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar ernet in type Proxy draft pre RFC compliant impersonation proxy strength 2048 bits path tmp xsOSup 4500 timeleft 11 56 37 henry gridfs i Fig 4 4 grid proxy info command to check the validity of your credential Chapter 4 GARUDA Command Reference Page 24 GARUDA User Handbook How to check the list of available resources in grid gwhost command 1s used to check the available resources in grid as shown in Fig 4 5 henry gridis henryllgridfu guhost D 1 NULLHULL NULL U 0 ad 0 0 0 4 4 Fork garudaprine ctsi cdacr or i i NULLHULL NULL 0 io 0 0 0 34 36 PRS mn00 npst cdac ernet in 2 1 Linux2 6 18 92 66 2933 0 56809 64331 81932 103308 0 44 3083 PBS ywa png cdac ho 100 ga 3 1 Linux2 6 13 92 x86 6 1592 200 1421 2010 16522 32243 0 42 45 PES cluster jnu ac in 4 j X86 6 2200 385 7620 9991 2032077218027 0 0 8 PRS crypt ctat cdac org in 5 i Linux2 6 16 53 x86 6 2511 1594 108764443 176851
21. which helps to debug job execution Gridway generates log files under the GW LOCATION var directory henryogridfs usr local GARUDA gw var henry gridfs cd GW LOCATION var henry gridfs var ls 20004 Job ID job contact job history _job rsl O job template stdout wrapper D job env job log job state stderr wrapper U henry grid s var J ig 4 16 Gridway job log information directory Chapter 4 GARUDA Command Reference Page 37 GARUDA User Handbook The below table explains each file in detail e gwd log System level log You can find log information of the activity of the middleware access drivers and coarse grain log information about jobs e SJOB ID job log Detailed log information for each job it includes details of job state transitions resource usage and performance e JOB_ID stdout wrapper Standard output of the wrapper executable e JOB_ ID stderr wrapper Standard error output of the wrapper executable By default wrapper is executed with shell debugging options xv active so this is usually the best source of information in case of failure IGCA Command Reference For Linux Users How to convert p12 or pfx files to pem Assume the file name as igca henry new p12 As shown in Fig 4 17 type the below command to extract usercert pem openssl pkcs12 in igca henry new pl2 out usercert pem nokeys clcerts lt henryagridfs henry gridfs op
22. 8446 dpm ctsf cdac org in home Garudaddmin test_dir no of directory levels 2 destination home henry Fig 6 12 Downloading a directory from GSRM Chapter 6 GARUDA Storage Resource Manager GSRM Page 62 GARUDA User Handbook How to upload a file to GSRM using clientSRM command Uploading a file to GSRM storage server involves 3 steps Step 1 Prepare to put request to GSRM clientSRM ptp Step 2 Transfer the file globus url copy Step 3 Verify the transfer clientSRM pd Step 1 Prepare to put reguest to GSRM As shown in Fig 6 13 clientSRM command with prepare to put reguest ptp function is used to make a put request to transfer a file named sample_file to Bangalore GSRM server gsrm Command clientSRM ptp e httpg gsrm 8446 s srm gsrm dpm ctsf cdac org in home GarudaAdmin sample file p henryBgridfs client5RH ptp e httpg gsrm 8446 s srm gsrm dpm ctsf cda c org in home GarudaAdmin sample file p Sending PtP request to httpg gsrm 0446 Polling request status Current status SRM REQUEST OUEUED Ctrl c to stop polling Request status Request token to verify the statusCode SRM SUCCESS 0 transfer SRH Response eguestToken d0f90983 42e0 4558 a23c f350b95685bc arrav fFileStatuses size 1 0 SURL srm gsrm dpm ctsf cdac org in home Garuda dmin sample file 0 status statusCode SRM SPACE AVAILABLE 24 0 fileSize 0 0 remainingPin
23. DA User Handbook Chapter 2 Accessing GARUDA via Grid Portal What is job submission Portal Grid portal is used for submitting sequential and parallel jobs to the grid Grid is a cluster of heterogeneous workstations located at geographically distant locations with the ultimate aim of sharing computing resources The portal provides a web interface for viewing the resources for submitting and monitoring Jobs What is the URL to access Grid Portal via Internet http portal garudaindia in How to access portal via NKN National Knowledge Network http 192 168 60 40 gap2 Which username and password should I use to access portal You need a GARUDA account for accessing Grid Portal Refer Fig 1 8 from chapter I User Name xyz211 Garuda Account Password VO Name seen M Select your VO select ____ test lacce others V am a First Time BEES Bioinformatics GarudaAdmin OSDD ClimateModelling plicy Change of VO q t CAE 91899 IndianHeritage MaterialScience Help GeoPhysics Fig 2 1 Shows the login form to access Grid Portal Note You need to select a VO Name only for the first time Chapter 2 Accessing GARUDA via Grid Portal Page 11 GARUDA User Handbook Which VO should I select during my login to Grid Portal Select the VO that you had registered with GARUDA in Chapter 1 How to view the list of available resources in GARUDA Click Resourc
24. ERIAL No in your application form ADDITIONAL ATTRIBUTE ORGANIZATION CDAC ADDITIONAL ATTRIBUTE TELEPHONE 08066116511 NOTBEFORE Wed Jun 8 06 09 17 2011 UTC RA C DAC Bangalore ROLE User SERIAL 244512 SUBJECT ATT NAME email xyz mail com TYPE SPKAC Fig 1 3 Get your reguest no SERIAL e Choose your nearest RA from the below link and arrange for a Face to Face meeting http ca garudaindia in index php ra contact ra f Mail the scanned copy of your application form and ID proof to igca cdacb ernet in g Your certificate information will be mailed to you Note Don t uninstall browser or format your machine until your certificate is issued Refer http ca garudaindia in for more information Chapter 1 Steps to access GARUDA Page 5 GARUDA User Handbook Step2 GARUDA Registration 1 Visit http voms garudaindia in Make sure you have your IGCA certificate uploaded in the web browser 2 Upload your IGCA certificate file pl2 format using the browse button as shown in Fig 1 4 and enter your password the same password which you entered while exporting your certificate from the browser Garuda ID is generated automatically by the system Please upload credential for Henry Sukuma PKCS12 File Jatest 010 2011 p12 B ja O Proxy File a p Browse Garuda id is automatically gene Garuda ID henryde 0000 For PKCS12 file password sho Password 137779777 Proxy can be
25. G ZOMBIE PRE Fig 4 12 Various states of a gridway job A job can be in one of the following dispatch states DM state e Pending pend The job is waiting for a resource to run on The job reaches this state when it 15 initially submitted by the user or when it is restarted after a failure stop or self migration e Hold hold The owner or GridWay administrator has held the job It will not be scheduled until it receives a release signal e Prolog prol The job is preparing the remote system by creating the execution directory in the remote host and transferring the input and restart in case of migration files to 1t e Pre wrapper prew The job is making some advanced preparation tasks in the remote resource like getting some data from a service obtaining software licenses etc Chapter 4 GARUDA Command Reference Page 30 GARUDA User Handbook e Wrapper wrap The job is executing the Wrapper which in turns executes the actual application It also starts a self monitoring program if specified This monitor watches the raw performance CPU usage obtained by the application e Epilog epil The job is finalizing In this phase it transfers the output and restart in case of failure stop or self migration files and cleaning up the remote system directory e Migrate migr The job is migrating from one resource to another by canceling the execution of Wrapper and performing finalization tasks in the ol
26. GARUDA User Handbook Job template options overview General Variable NAME Execution Variable EXECUTABLE ARGUMENTS ENVIRONMEN T TYPE NP Description Name of the job filename of the Job Template by default Description The executable file Example EXECUTABLE bin ARCH EXECUTABLE bin ls Executable file can be also be taken from GSRM EXECUTABLE gsiftp gsrm ctsf cdac org in dpm ctsf cdac org in home lt VO NAME gt lt executable_file gt Arguments to the above executable Example ARGUMENTS 1TASK ID User defined comma separated environment variables Example ENVIRONMENT SCRATCH DIR tmp LD LIBRARY PATH usr local lib Type of job Possible values are single default multiple and mpi with similar behavior to that of GRAM jobs Number of processors in MPI jobs For multiple and single jobs it defines the count parameter in the RSL Chapter 4 GARUDA Command Reference Page 33 VO files INPUT_FIL ES OUTPUT_FI LES GARUDA User Handbook A comma separated pair of local remote filenames If the remote filename is missing the local filename will be preserved in the execution host Example INPUT FILES param TASK_ID param inputfile INPUT_FILES gsiftp machine tmp input_exp1 input INPUT_FILES file etc passwd INPUT_FILES test_case bin INPUT_FILES gsfip gsrm ctsf cdac org in dpm ctsf cdac org in home lt VO NA ME gt input txt In
27. GSRM Page 68 GARUDA User Handbook henry gridfs GSRH userSpaceInfo e httpg gsrm 8446 s usr local GARUDA GSRH gsrm clients storm bin userSpaceInfo line 206 test unary or Token Total size Free Space Free Bf33324b d6 b8 45fa 9294 a69fflif5S7bf SMB 4MB 50 00 Owner Garuda dmin Has Total available Free Space 4MB 50 00 Fig 6 21 userSpacelnfo command to view reserved space tokens How to view user audit information As shown in Fig 6 22 selfAudit can be used to view audit information of the user data stored in GSRM Audit information includes filename file size time last modified and last accessed Command selfAudit s lt hostname gt Status t Thu May 19 17 29 25 2011 status At Thu May 19 17 29 25 2011 File Name weka FIle Size 0 Bytes Last Accessed On Mon May 16 11 46 55 zoll Last Modified On Mon May 16 14 12 20 2011 Fig 6 22 selfAudit command to view the audit information Chapter 6 GARUDA Storage Resource Manager GSRM Page 69 GARUDA User Handbook Does GSRM have a web interface Yes GSRM web client can be accessed from https gsrm ctsf cdac org in https srmhyd cdac in Note The above URL can be accessed only via GARUDA NKN Network Where can I find complete user manual for GSRM GSRM User manual can be downloaded from http portal garudaindia in GARUDA REPO gsrmv1 1 gsrmv1 1 Manuals gsrmv1 1 usermanual pdf Whom should I contact if I
28. Garuda User s Handbook Roorkee Lhandigarh L Y Luckno Delhi y YW Kanpur D et QI Varanasi Allahabad Ahmedabad Mumbai pb Hyderabad 2 gt Thiruv an anthap a MM First Edition GARUDA User s Handbook Version 1 0 19 May 2011 CDAC CENTRE FOR DEVELOPMENT OF ADVANCED COMPUTING Acknowledgement Ouite a few people are responsible directly and indirectly in the successful completion of this book to whom we owe our sincere gratitude First of all we would like to thank our respected Executive Director Dr Sarat Chandra Babu for always being so enthusiastic and encouraging towards us Special thanks to Dr Subrata Chattopadhyay for his direction and guidance in all our Garuda activities Thanks are also due to Mr R Sridharan for having the faith in us and increasing our confidence time and again We also wish to thank Dr Prahalad Rao and Ms Mangala for lending their support to us whenever reguired Many thanks to Ms Divya and the whole Garuda operational team for their unending efforts for the Garuda project and in turn helping us in all our activities Finally we wish to express our appreciation for Ms Prachi for helping us to construct meaningful grammatical sentences which hopefully are also error free Authors S Henry Sukumar amp Santhosh J GARUDA Operational Team Table of Contents ba EO eL eu ENTRY YRR ee ee ae ANF FNAN YRFA 1 Chapter 1 Steps to access GARUDA
29. Lifetime 7198 TURL 0 remainingFileLifetime 0 0 Fig 6 13 clientSRM command to make a ptp request Chapter 6 GARUDA Storage Resource Manager GSRM Page 63 GARUDA User Handbook Step 2 Transfer the file globus url copy command is used to transfer the file to GSRM server This command requires a Transport URL TURL which is obtained from step 1 Refer Fig 6 13 The complete command 15 shown in Fig 6 14 Command globus url copy lt source_path gt lt TURL gt henry gridfs globus url copy file home henry weka jar gsiftp gsrm ctsf c dac org in gsrm ctsf cdac org in storage3 Garuda dmin 20 5 18 sample file 11 243 0 i henry8gridfs J TURL Obtained from step i Fig 6 14 globus url copy is used to transfer the file to GSRM Step 3 Verify the transfer The transfer operation that performed in step 2 can be verified using the put done pd function of the clientSRM command As shown in Fig 6 15 this command requires a request token which is obtained during step 1 Refer Fig 6 13 henry gridfs clientSRM pd e httpg gsrm 8446 s srm gsrm dpm ctsf cdac org in home Garudaidmin sample file t dDf90983 4220 4558 a23e f350b95685bc Sending PutDone request to htXpg gsrm 8446 Request token obtained in step 1 Reguest status statusCode SRM SUCCESS 0 Lecation of the transfered file SRM Response arrayOfFileStatuses size i 0 SURL srm gsrm dpm ctsf
30. SCP command to securely copy data to a remote machine GSRM GARUDA Storage Resource Manager GRIDWAY The GridWay Metascheduler enables large scale reliable and efficient sharing of computing resources clusters supercomputers stand alone servers It supports different LRM systems PBS SGE LSF and Condor within a single organization or scattered across several administrative domains Appendix I Glossary Page 80 GARUDA User Handbook H Host Certificate Certificate issued by Certificate Authority for a particular host to participate in Grid I IGCA The Indian Grid Certification Authority GCA provides X 509 certificates to support the secure environment in grid computing IGCA is an accredited member of the APGridPMA Asia Pacific Grid Policy Management Authority for Grid Authentication M Middleware Middleware is an interface between the user and resources Globus Toolkit is one of the grid middleware MyProxy Credential Repository MyProxy is aGlobus project that develops software for managing X 509 Public Key Infrastructure security credentials certificates and private keys MPI Message Passing Interface MPI is an API specification that allows processes to communicate with one another by sending and receiving messages It is typically used for parallel programs running on computer clusters and supercomputers where the cost of accessing non local memory is high N NKN National
31. UDA User Handbook Help us to serve you better We reguest you to kindly spare some of your valuable time and fill in the online survey form Your input would help us to improve our services and allow us to serve you better Online survey form http gridsupport garudaindia in survey index php sid 31228 Whom should I contact if Ihave any question regarding this book You can send a mail to rt Ocdacb ernet in or Quick Response Raise tickets in appropriate queue Set your priority level to indicate the urgency of the problem Describe your problem clearly Attach screen shots if applicable Maintain the same subject name until the ticket is resolved Chapter 7 Grid Support Page 76 Appendix I GLOSSARY GARUDA User Handbook Appendix I Glossary The following glossary defines some commonly used terms and acronyms associated with the GARUDA Environment A AGSG Automatic Grid Service Generator Authentication A system says how you identify yourself to the computer The goal behind an authentication system is to verify that the user is actually who they say they are Authorization Authorization checks that authentic user have proper permission or not to access particular service C Certificate A certificate is a digital document that certifies that a certain public key is owned by a particular user This document is signed by Certificate Authority CA Certificate Authority Certific
32. alid proxy Authenticated No Time remaining VO membership Details Update Enterpassphrase sened E Lifetime days mew Ja Fig 5 6 Authentication to the GridFTP server Chapter 5 GARUDA GridFTP Client Page 48 GARUDA User Handbook 8 Upon successful authentication you should be able to view the status of your proxy as shown in Fig 5 7 FA Grid Authentication Status of current proxy Identity sanlhust sanltinshj fedael seemet in Authcnticated Yes Tune remaining 2R7h Sarnin ASHE VO membership Nune Details Update Enter passphrase OOo kama r a sonvo Fig 5 7 View the status of your proxy 9 Connect to the GridFTP Server with Hostname egridfs ctsf cdac org in port 2811 Type GridFTP and click Connect button Upon successful connection you should be able to view a transfer window as shown in Fig 5 8 A GARUDA Gridl IP GUI Client Port 2011 Type GridFTD 4 A Eidg i gt M yb nome santhosh vw amp B8 ce X ial ia Name Size Ilme Modmed Ainbutss Destnpac a us Folde 2 MACOLX JUKU Iolcer Leer 5 15 19 dnr a 2node 3 0KD Folder May O 16 S5 dimar sr x CO Arrant Crratinn RAER Falter heyy 12 ANR AENT v amp My Documents CO Al AAKR Fun mb 173R adneverer e Start Men 1 upurte lurritul 5 5 25 3 3KB Fulder Muy 18 14 27 Urez ar z nb 150 Rasyirieng Iraresfi mg inguil lrisurr ul 220 Transf Carrigila IGCA Jnuug
33. ation authority CA is an entity that issues digital certificates for use by other parties CP CPS Certificate Policy and Privacy Statement CRIN PIN Certificate Revocation and Identification Number The CRIN mail contains a pin code that the user must enter when revoking their own certificates The user should be able to decrypt the message as they would have created the private key during their certificate request process The message is encrypted using the public key in the certificate request Appendix I Glossary Page 78 GARUDA User Handbook D DN Distinguished Name Unique name for each user in the grid E End Entity Certificate User certificate or Host certificate used by the end user 1s called as End Entity Certificate G GARUDA lt Global Access to Resources Using Distributed Architecture GARUDA Account An account used for accessing GARUDA Grid GAP Grid Access Portal Grid Computing Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach a common goal The grid can be thought of as a distributed system with non interactive workloads that involve a large number of files What distinguishes grid computing from conventional high performance computing systems such as cluster computing is that grids tend to be more loosely coupled heterogeneous and geographically dispersed Although a grid can be dedicated to a specialized ap
34. c org in home GarudaAdmin weka jar What is StFN StFN stands for Storage File Name SFN is a part of SURL What is my StFN root path Below is the general format of StFN for gsrm ctsf cdac org in dpm ctsf cdac org in home lt VO_NAME gt Below is the general format of SFN for srmhyd cdac in dpm cdac in home lt VO_NAME gt E g consider a user who belongs to OSDD VO His StFN for gsrm ctsf cdac org in is dpm ctsf cdac org in home OSDD Allthe GSRM examples in this book are of a user who belongs to GarudaAdmin VO So the StFN will look like dpm ctsf cdac org in home GarudaAdmin Replace this with vour corresponding StFN Chapter 6 GARUDA Storage Resource Manager GSRM Page 54 GARUDA User Handbook Setting up GSRM User Environment GSRM Client commands are located at GRIDFS machine as shown below Table 6 2 Clients Location storm usr local GARUDA GSRM egsrm clients storm bin dpm usr local GARUDA GSRM gsrm clients dpm bin bestman usr loca GARUDA GSRM gsrm clients bestman bestman client bin Table 6 2 Location of GSRM clients Setting up GSRM client commands in PATH variable Fig 6 1 shows setting up of GSRM client commands in PATH environment variable henryagridfs henry gridfs export PATH usr local GARUDA GSRM gsrm clients dpm bin usr 1 ocal GARUDA GSRH gsrm clients storm bin usr local GARUDA GSRH gsrm clients best man bestman client bin PATH
35. d resource like in Epilog state and preparation tasks in the new resource like in Prolog state e Stopped stop The job is stopped If restart files have been defined in the Job Template they are transferred back to the client and will be used when the job is resumed e Failed fail The job failed e Done done The job is done and the user can check the exit status Overview of gridway job template Syntax lt VARIABLE gt lt VALUE gt E g EXECUTABLE bin ls Table 4 1 and 4 2 shows the sample job templates of a serial and parallel application respectively Chapter 4 GARUDA Command Reference Page 31 GARUDA User Handbook Job Template for serial program Below 1s the sample gridway job template for serial program EXECUTABLE bin ls ARGUMENTS la file txt STDIN FILE dev null STDOUT FILE stdout JOB ID STDERR FILE stderr JOB ID ENVIRONMENT LD LIBRARY PATH usr local lib REOUIREMENTS HOSTNAME gg blr tfg INPUT FILES file txt Table 4 1 Job template of a serial program Job Template for parallel program Below is the sample gridway job template for parallel program EXECUTABLE home tfg santhosh test pi STDIN FILE dev null STDOUT FILE JOB ID out STDERR FILE JOB_ID err REOUIREMENTS LRMS TYPE Fork REOUIREMENTS HOSTN gg blr tfg TYPE mpi NP 80 Table 4 2 Job template of a parallel program Chapter 4 GARUDA Command Reference Page 32
36. e Info tab as shown in Fig 2 2 to view the available resources in the Grid Administration Job Management File Management Accounting Help Notices l Resources Resources available in the Grid Total number of Clusters 9 Total number of Available Nodes 887 Click Resaurces Infe tab garudaprime ctsf cdac or NULLNULL xn00 npsf cdac ernet in NULLNULL yuva png cdac hg A 100 gar Linux2 6 18 92 EN 18602 prihpe lan prl res in 86_6 x86 6 14249 onodesaserc ise emet nus2 65 7 24 power s Pes e e Fig 2 2 Grid Portal Resource Information Grid Portal http portal garudaindia in Chapter 2 Accessing GARUDA via Grid Portal Page 12 GARUDA User Handbook How to submit jobs using Grid Portal Click Job Management tab to submit jobs as shown in Fig 2 3 Administration File Management Help Noti BASIC Job Submission Portlet Welcome kalasagar211 Click Job Managertent tab En EEE Inh Type selert v lotal number ot 1 processes Operating System v Processor Fxerutahle Ratrh File Select Remote File Input File Seleri Remnte File Addin list STDIN Select Remote Filo Wall Time N day kA N hr Amin ne HD arguments Environment Variables User Specific Oulpul File __ O Fig 2 3 Grid Portal Job Management Section Chapter 2 Accessing GARUDA via Grid Portal Page 13 GARUDA User Handbook
37. ecessary environment variables in command line Fig 4 1 depicts the necessary command to set up Globus environment variable GLOBUS_LOCATION at gridfs machine Note The installation path of globus may vary on different clusters and environment variable would have been already set by the administrator If not set users can set as shown in the below fig 4 1 henry gridfs henryBgridfs export GLOBUS LOCATION usr local GARUDA GLOBUS 4 0 8 henryBgridfs E Fig 4 1 Setting globus environment variable Exporting Globus commands As shown in Fig 4 2 source command is used for exporting the globus commands and libraries in PATH environment variable in gridfs machine Note The installation path of globus may vary on different clusters henryigridts source GLOBUS LOCATION etc globus user env sh henry gridfs A Fig 4 2 Exporting globus commands and libraries in path Chapter 4 GARUDA Command Reference Page 23 GARUDA User Handbook How to download my credentials for using grid myproxy logon command is used to download user credentials from the credentials repository as shown 1n Fig 4 3 Later these credentials can be used for accessing grid Services lt henryogridfs henryBgridfs myproxy logon Enter MyProxy pass phrase GARUDA Account password A credential has been received for user henry in tmp x509up 4500 henryBgridfs J Fig 4 3 Download your proxy for grid
38. enssl pkesi2 in igca henry new pl2 out usercert pem nokeys clcerts Enter Import Password HAC verified OK henry gridfs ls l usercert pem TW YWeI 1 henry henry 1919 Hay 5 15 50 usercert pem henry gridfs i Enier the password that you had set during certificate export from browser Fig 4 17 Generation usercert pem Chapter 4 GARUDA Command Reference Page 38 GARUDA User Handbook As shown in Fig 4 18 type the below command to extract userkey pem openssl pkcs12 in igca henry new p12 out userkey pem nocerts henry gridfs hcnry Bgridfo opensol pkeoi2 in igca henry ncw pl2z out uscrkcy pcm noccrto Enter Import Passvord Enter the passward that ynu had set during certificate apart fram Fac ified OK t ee browser Euter PEM pass phrase Verifying Enter PEN pass phrase henry gridfs i Set pass phrase to protect the converied userkey pem file Yuu can sei the same password as above Fig 4 18 Generating userkey pem For Windows Users If you are a windows user then follow the below steps 1 Download and Install OpenSSL for Windows http www slproweb com download Win320penSSL_Light 0 9 Sr exe 2 Install OpenSSL into the folder CAOpenSSL 3 As shown in Fig 4 19 place your pfx or p12 files in the folder C my cert Chapter 4 GARUDA Command Reference Page 39 GARUDA User Handbook n C my cert Burn New folder yd Favorites BE Desktop y
39. er Form VI pdf b Register online to get a request no https ca garudaindia in cgi bin pub pki cmd basic_csr Supported browsers Internet Explorer and Mozilla Firefox only As depicted in Fig 1 1 Enter the necessary details e Name Full name of the applicant e E Mail Valid Email address for communication e Organization Name of your institution organization e Telephone Valid telephone no for communication e Role Default value User for user certificate e Choose your nearest RA Choose your RA for face to face meeting e Key size Default value 2048 is the strength of certificate IGCA Indian Grid Certification Authority RA Registration Authority Refer Appendix I for glossary and terms Chapter 1 Steps to access GARUDA Page 3 GARUDA User Handbook Certificate Data Name Enter Your Full Name E Mal User Data eee iphone Eole Uzer Choose your nearest KA C DAC Bangalore Keysize Fig 1 1 Online Registration c Clicking Continue will generate a private key as shown in Fig 1 2 Certificate Data Name Enter Your Full Name xyz Generating A Private Key Key Generation in progress This may take a few minutes Registration Authority C DAC Bangalore Keysize High Grade Fig 1 2 Key generation process Chapter 1 Steps to access GARUDA Page 4 GARUDA User Handbook d Fig 1 3 Shows the details of your request Note down the S
40. fs globus url copy vb file home henry weka jar gsiftp gg blr tmp henryBgriafs I Here file protocol used to contact source machine and gsiftp protocol to contact remote machine Fig 4 9 globus url copy command file to gsiftp Transferring files from remote machine to local machine henryagridfs henry gridfs globus url copy gsiftp gg hyd tmp weka jar file home henry weka jar henry gridfs I Fig 4 10 globus url copy command gsiftp to file For transferring data to GSRM Garuda Storage Resource Manager Refer chapter 6 Third Party Transfer Transferring files between machine B and machine C from machine A E g transferring files between gg hyd located in Hyderabad and gg che located in Chennai from GRIDFS located in Bangalore henry gridfs henryBgridfs globus url copy gsiftp gg hyd tmp weka jar gsiftp gg che tmp veka jar henry gridfs Fig 4 11 globus url copy command gsiftp to gsiftp In the above examples the local machine refers to the machine from where the globus url copy command is invoked GSIFTP GSIFTP provides more reliable and high performance file transfer for Grid computing applications User should have valid proxy before invoking gsiftp transfer globus url copy is the command used for invoking gsiftp transfers GSIFTP can also be called as GridFTP For more information about globus and its usage visit htt
41. i Downloads Recent Places E igca cert pi2 Fig 4 19 Place your p12 or pfx files in c my cert 4 As shown in Fig 4 20 open your command prompt and enter the folder C OpenSSL bin es C AWINDOWS is ystem37icmd exe C Documents and Settings gt cd C OpenSSL bin C OpenSSL b n gt Fig 4 20 Change directory to openssl installed folder 5 Setthe OpenSSL commands in the PATH as shown in Fig 4 21 C OpenSSL bin gt set PATH PATH CD C OpenssL bin gt _ Fig 4 21 Set the OpenSSL command in the PATH 6 Change to the directory c Wny cert where your IGCA certificate p12 is stored Chapter 4 GARUDA Command Reference Page 40 GARUDA User Handbook C OpenSSL bin gt cd C my cert C my cert gt dir Volume n dr ve C has no label volume Serial Number is B42A A367 Directory of C my cert 05 13 2011 01 04 PM lt DIR gt 05 13 2011 01 04 PM lt DIR gt 11 18 2010 11 37 AM 2 985 igca cert p12 1 File s 2 985 bytes 2 Dir s 13 413 625 856 bytes free Fig 4 22 Change to the p12 certificate directory 7 Extracting usercert pem As shown in Fig 4 23 type the below command to extract usercert pem openssl pkcs12 in igca cert pl2 out usercert pem nokeys clcerts When prompted for Import Password enter the password which you had set during your certificate export from the browser C my cert gt openss version penSSL 0 9 8r 8 Feb 2011 C my cert gt openss pkcs12 in
42. ility to gridfs ctsf cdac org in GRIDFS 3 Get the login access to GARUDA INTERFACE NODE in your institution 4 Make sure you have IGCA certificate and GARUDA Grid account Refer Chapter 1 For WINDOWS Users A Download the program called puTTY from the below link http the earth li sgtatham putty latest x86 putty exe B Double click and open putty exe This will open the puTTY configuration as shown in Fig 3 1 Chapter 3 Accessing GARUDA via Command line Page 17 GARUDA User Handbook ES Pul TY Configuration Session Basic options for your PuTTY session Logging Specify your connection by host name or IP address Terminal Host Name or IP address Port Keyboard 22 Bell 2 Features A Window ORlegn OSSH Appearance Load save or delete a Meyed sessinn Behaviour Ba GARUDA Interface Translation NODE IP address Selection J efault Settings A Load i Coni ca garudaindia in E Connection 10 g 4 Save Proxy ge Telnet garudaca Rlogin gridfs backup SSH henm gndfs w Auth nn Close window on exit ugs O was ONever Only on clean exit Coes Fig 3 1 PUTTY Interface for SSH connection C Enter the GARUDA INTERFACE NODE IP ADDRESS in the Hostname or IP address text box Refer Fig 3 1 D Protocol Select SSH Refer above Fig 3 1 E Click open button you can see a terminal as shown in Fig 3 2 F Enter the username and password
43. in a directory As shown in Fig 6 9 Is function of the clientSRM command is used to list the files in a directory clientSRM Is l e lt ZGSRM End Point gt s lt SURL gt henryogridfs henry gridfs clientSRM ls 1 e httpg gsrm B446 s srm gsrm 8446 dpm ctsf cdac org in home Garudaddmin Reguest status statusCode 5RM SUCCESS 0 SRH Response details 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 size 1 path dpm ctsf cdac org in home Garuda dmin size 0 createditTime 2011 05 187T05 47 382 lastModificationTime 2011 05 15T05 47 332 type Directory ownerPermission userID root mode RWX groupPermission groupID Garuda dmin mode RWXZX otherPermission NONE status statusCode 5RM SUCCESS 0 arrayOfSubPaths i size 3 path dpm ctsf cdac org in home Garuda dmin generated size 0 createditTime 2011 04 30T23 18 142 lastModificationTime 2011 04 30T23 13 132 type Directory ownerPermission userID root mode RWX groupPermission groupID Garuda dmin mode RWXZX otherPermission R X status statusCode 5RM SUCCESS 0 path dpm ctsf cdac org in home Garuda dmin weka size 0 createdatTime 2011 05 16T08 42 202 lastModificationTime 2011 05 16T08 42 202 type Directory ownerPermission userID DC IN DC GARUDAINDIA O C Dac OU CTSF CN Henry Sukumar henrys cdach ernet in groupPermission g
44. ith detailed step by step illustration of getting access to the Garuda and using it Chapter Organization The first chapter focuses on steps to access Garuda i e getting a grid certificate and a Garuda account The second and third chapters explain about the access mechanisms available in Garuda i e the Grid Job Submission Portal and the Command Line Interface Introduction Page 1 GARUDA User Handbook The fourth chapter explains about the usage of the commonly used commands for the grid services It 1s divided into three sections which deal with Credential Management Data Management and Job Management The fifth chapter explains about the usage of Garuda Grid FTP client software Garuda GridFTP Client is a GUI for performing file transfers using the GridFTP protocol The sixth chapter explains about the usage of commonly used commands of Garuda Storage Resource Manager The chapter seven explains about the mechanism by which users can report any problem they face while using Garuda Introduction Page 1 teps to access GARUDA GARUDA User Handbook Chapter 1 Steps to access GARUDA Getting access to GARUDA involves two steps Step 1 Get Grid User Certificate from IGCA Visit http ca garudaindia in index php certificate reguest a new certificate Follow the steps mentioned below to apply for a user certificate a Download and fill up IGCA User Application Form http ca garudaindia in docs IGCA Us
45. need a support for GSRM You can reach our GSRM team at 080 66116692 and 080 66116457 GSRM End Points in GARUDA e Bangalore httpg gsrm ctsf cdac org in 8446 e Hyderabad httpg srmhyd cdac in 8446 Note The protocol used for contacting the end point is httpg Chapter 6 GARUDA Storage Resource Manager GSRM Page 70 Chapter Ka d Grid Support GARUDA User Handbook Chapter 7 Grid Support How will I log my support reguest 1 Go to http gridsupport garudaindia in 2 Login with your GARUDA Account 3 Raise ticket in appropriate queue Which username and password should I use for accessing Grid Support web page You must have a GARUDA account for accessing Grid Support Refer Fig 1 8 http gridsu pport garudaindia in Login enn cana Username Password Forgot your password Fig 7 1 GARDUDA RT Reguest Tracker Login page Chapter 7 Grid Support Page 73 GARUDA User Handbook How to raise a problem ticket 1 Select the queue for which you would like to raise a ticket 2 Click New ticket in button as shown in Fig 7 2 Logged in as henrys Preferences Logout Search Click to raise a new ticket Select the support queue Fig 7 2 How to raise a ticket How to check the status of my ticket User can check the status of their ticket using the ticket ID E g Ticket Id 12958 as shown in Fig 7 3 Subject Status Re
46. nry gridfs H Fig 6 3 Generating VO credentials If you are not sure which VO you belong to Send a mail to voms cdacb ernet in GARUDA User Handbook How to view my VO credentials As shown in Fig 6 4 voms proxy info is the command used to view your credential information henryogridfs henry gridfs voms proxy info all WARNING Unable to verify signature Server certificate possibly not installed Error AC has been granted to a different certificate than the passed one subject DC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar henrysBcdacb ernet in CN 793469365 CN 183 7609429 CN 19116873 63 CN 19118873 63 issuer DC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar henrysBcdacb ernet in CN 793469385 CN 1837609429 CN 1911887363 identity DC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar henrysBcdacb ernet in CN 793469385 CN 1837609429 CN 1911887363 type unknown strength 2048 bits E z path tmp x509up u500 VO credential information timeleft 10 04 06 VO GarudaAdmin extension information VO Garuda dmin subject DC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar henrysBcdacb ernet in CN 793469385 CN 1837609429 CN 1911887363 CN 1911887363 issuer DC IN DC GARUDAINDIA O C DAC OU CTSF CN garudasec ctsf cdac org in attribute GarudaAdmin Role NULL Capability NULL timeleft 11 03 02 henryBgridfs J Fig 6 4 voms pr
47. once and gains access to all systems without being prompted to log in again at each of them Single sign on is achieved using the command gsissh as shown in Fig 4 7 2 moab pe hyd henry gridfs gsissh p 2222 gg hyd Last login Fri May 6 13 44 17 2011 from gridfs ctsifi cdac org in Rocks 5 0 V Profile built 19 05 09 Aug 2006 Eickstarted 01 39 10 Aug 2008 moab gg hyd hostname gg hyd cdac org in moab8gg hyd J Fig 4 7 gsissh command used for logging into remote cluster How to copy data to remote cluster using GSISCP gsiscp command is used to transfer data to remote cluster as shown in Fig 4 8 henry eridfs henry gridfs gsiscp P 2222 weka jar gg hyd weka jar 1004 6094K5 henryi gridfs a Fig 4 8 gsiscp command ts used for transferring data to remote cluster GSISSH and GSISCP e Use p small p option for GSISSH e Use P capital P option for GSISCP e 2222 GSISSH service port Chapter 4 GARUDA Command Reference Page 26 How to copy data to remote cluster using GRIDFTP The GridFTP protocol provides secure robust fast and efficient transfer of especially bulk data Data can be transferred from Transferring files from local machine source to remote machine destination In this case file protocol is used to contact the source machine and gsiftp protocol should be used to contact remote machine henry gridfs henry grid
48. oxy info command shows VO credentials information How to check the status of GSRM server clientSRM command along with ping function is used to check the state of the SRM and it works as an Are you alive type of call The version of the SRM specification implemented by the server is returned As shown Fig 6 5 ping function of the clientSRM command is used to ping a GSRM server gsrm and srmhyd clientSRM ping e httpg gsrm ctsf cdac org in 8446 clientSRM ping e httpg srmhyd 8446 Chapter 6 GARUDA Storage Resource Manager GSRM Page 57 GARUDA User Handbook henry gridfs clientSRM ping e httpg srmhyd S446 Sending Ping request to httpg srmhyd 8446 Request status statusCode S5RH SUCCESS 0 explanation SRM server successfully contacted SRM Response versionlnfo v2 2 other Info size 2 0 key backend type 0 value DPH 1 key backena version 1 value 1 5 0 1 Fig 6 5 clientSRM command is used to ping the GSRM server Note Type clientSRM ping help for more information How to create a directory in GSRM As shown in Fig 6 6 mkdir function of the clientSRM command is used to create directory in GSRM server Type clientSRM mkdir help for more information clientSRM mkdir e lt GSRM End Point gt s lt SURL gt henry gridts clientSRM mkdir e httpg gsrm 8446 s srm gsrm 8446 dpm c tsf cdac org in home Garudal min
49. p globus org GARUDA User Handbook GARUDA Grid Meta Scheduler Gridway GridWay enables you to treat your jobs as if they were Unix processes Each job is given a numerical identifier analogous to the PID of a process This value is called the Job identifier JID for short If the job belongs to an array Job it will also have an array 1dentifier AID for short A Job index within an array is called the task identifier TID for short Jobs are submitted using the gwsubmit command Refer Fig 4 14 A Job is described by its template file Here you can specify the job s executable file its command line arguments input output files standard stream redirection as well as other aspects Jobs can be monitored using the gwps command Refer Fig 4 15 You can control your Jobs at runtime using the gwkill command You can synchronize your jobs using the gwwait command You can find out what resources your Job has used with the gwhistory command System monitoring commands allow you to gather information of the GridWay system and the Grids you are using These commands are gwuser to show information about the users using GridWay gwhost to monitor the available hosts in the testbed For more information about Gridway and its usage visit http gridway org Chapter 4 GARUDA Command Reference Page 29 GARUDA User Handbook What are the various states of my job h da Mi En PENDING PROLOG gt WRAPPER EPILO
50. plication it is more common that a single grid will be used for a variety of different purposes Grids are often constructed with the aid of general purpose grid software libraries known as middleware Globus Toolkit The Globus Toolkit is a fundamental enabling technology for building grids that allow distributed computing power storage resources scientific instruments and other tools to be shared securely across corporate institutional and geographic boundaries Appendix I Glossary Page 79 GARUDA User Handbook GRIDFS Name of the machine that acts a GARUDA HEAD NODE gridfs ctsf cdac org in GARUDA INTERFACE NODE NODE located at each partner institution which connects to garuda Grid Portal Grid portal is for submitting seguential parallel This portal provides a web interface for viewing the resources and submitting and monitoring Jobs GRAM Grid Resource and Allocation Manager GRAM is a Globus project that produces technologies which enable users to locate submit monitor and cancel remote jobs on Grid based compute resources GRAM enables remote execution management in contexts for which reliable operation stateful monitoring credential management and file staging are 1mportant GridFTP GridFTP is a Globus project that produces high performance secure reliable data transfer technologies optimized for high bandwidth wide area networks GSISSH command to securely login to a remote machine GSI
51. put files can be read from GSRM server given above A comma separated pair of remote filename local filename If the local filename is missing the remote filename will be preserved in the client host Example OUTPUT FILES output file binary binary ARCH TASK_ID OUTPUT_FILES output bin tmp outfile GridFTP URL The file will be copied to the given GridFTP url OUTPUT_FILES out gsiftp storage_servere output TASK ID OUTPUT FILES test_case bin OUTPUT FILES http gsrm ctsf cdac org in dpm ctsf cdac org in home lt VO NA ME gt output out Output files can be directly stored in GSRM server as given above Chapter 4 GARUDA Command Reference Page 34 Standard streams GARUDA User Handbook STDIN FILE Standard input file Example STDIN FILE dev null STDIN FILE In input JOB_ID STDOUT FILE Standard output file Example STDOUT FILE stdout file f JOB ID STDOUT FILE Out ofile STDERR FILE Standard error file Example STDERR FILE stderr file JOB ID Resource selection REOUIREMENTS RANK A Boolean expression evaluated for each available host if the evaluation returns true the host will be considered to submit the job Example REQUIREMENTS ARCH i686 amp CPU MHZ gt 1000 A numerical expression evaluated for each candidate host those for which the requirement expression is true Those candidates with higher ranks are used first to execute your Jobs
52. questors Created 12958 still not getting the required output on prihpe open nehak Pcdacb emet in 8 days ago Fig 7 3 Status of the ticket What are the support gueues available Sl no Project Name Support queue Email name 2 Any Grid related GDeployment grid help Ocdacb ernet in problem 3 Grid Portal Portal rt gp cdacb ernet in Chapter 7 Grid Support Page 74 Can I send my support reguest via email Service Oriented Architecture GARUDA VOMS Grid IDE Tool OSDD OSDD Galaxy GARUDA Network Related Paryavekshanam Gridhra Automatic Grid Service Generator GARUDA SRM Not sure about Oueue SOA Voms GridIDE OSDD Osddgalaxy Netops Gridmon Gridhra AGSG GDS Other GARUDA User Handbook rt soa Ocdacb ernet in voms Ocdacb ernet in rt gide Ocdacb ernet in rt osdd Ocdacb ernet in rt osddgalaxy Ocdacb ernet in netops Ocdacb ernet in rt gmon Ocdacb ernet in rt agsg Ocdacb ernet in rt gds Ocdacb ernet in rt Ocdacb ernet in Table 7 1 GARUDA support gueues Yes refer Table 7 1 for corresponding email addresses Can I speak to the GARUDA Operation team over phone Why not We will be glad to speak to you The support phone nos are given below Grid Support 080 66116511 080 66116472 080 66116474 Network Support 080 66116310 080 66116473 Grid Portal 080 66116493 080 66116494 080 66116459 GSRM 080 66116692 080 66116457 Chapter 7 Grid Support Page 75 GAR
53. rmv1 1 usermanual pdf FN Important Links and Email ids to remember R n j N i l I j Important Links Garuda Website http garudaindia in JARUD A grid help Ocdacb ernet in inola s Natoma Grid Computing iniisa NIGCA IGCA http ca garudaindia in Indian Grid Certification Authority igca Ocdacb ernet in Grid Support http eridsupport garudaindia in
54. roupID Garuda dmin mode NOME otherPermission NONE status statusCode 5RM_SUCCES5 0 path dpm ctsf cdac org in home Garuda dmin weka rpm size 0 createdatTime 2011 05 16T07 14 512 lastModificationTime 2011 05 16T07 14 512 type Directory ownerPermission userID DC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar henrys cdach ernet in groupPermission groupID Garudaldmin mode N NE otherPermission NONE status statusCode SRM_ SUCCESS 0 ThenrwRaridfs 1 Bf Fig 6 9 listing the files in a directory Note Type clientSRM Is help for more information Chapter 6 GARUDA Storage Resource Manager GSRM Page 60 GARUDA User Handbook What are the various functions that can be performed using clientSRM command clientSRM is one of the SRM client commands that can be used to perform various storage operations As shown in Fig 6 10 type clientSRM to view the help page henry gridfs clientSRH Usage client5RH lt requestName gt lt reguestOptions gt lt requestName gt can be it is not case sensitive Discovery functions Ping GetTransferProtocols or gtp Directory functions Mkdir Rmdir Fm Ls Hy Data transfer functions PtP PtG Copy or cp StatusPtP or sptp StatusPtG or sptg Statuscopy or scp PutDone or pd ReleaseFiles or rf BringOnline for bol StatusBringOnline for sbol GetReguestTokens or grt GetReguest5unmarvy
55. the Grid FTP protocol It is intended to be an easy to use program for remote file operations such as upload download directory and file listing and file delete The software 1s available in versions that will run on many different flavors of the Linux operating system as well as on 32 bit flavors of Microsoft Windows XP Vista and 7 What are the Pre reguisites You need Java at least 1 6 to run this software Also you need Garuda certificates properly setup on your machine to connect to the remote resources of the grid Please refer to http portal garudaindia in gridftp gui tor a detailed list of pre reguisites Where can I find online documentation for this tool Online Documentation is maintained at http portal garudaindia in gridftp gui GridF TP GUI Usage For Windows Users 1 Download the Garuda Gridftp software for windows from the below link http gridfs ctsf cdac org in gridftp gui download Garuda GridFTP Setup exe 2 Install the Garuda GridFTP software by just double clicking the setup file 3 Once installed you can invoke the program using Start Menu Item gt Garuda gt Grid FTP GUI as shown in Fig 5 1 Chapter 5 GARUDA GridFTP Client Page 45 GARUDA User Handbook fam Garuda Grid M winscp AD xilisoft All Programs 7 Xmanager2 m Adobe Reader X Fig 5 1 Invoking GridFTP GUI software 4 Step 3 will open a GUI window as shown in Fig 5 2 Re GARUDA Grid TP GUI Client
56. to access GARUDA Page 7 GARUDA User Handbook 5 Fill the form as shown in Fig 1 7 and click on Register button Your distinguished name DN fDC IN DC GARUDAINDIA O C DAC OU CTSF CN Henry Sukumar h Your CA fDC IN DC GARUDAINDIS CN Indian Grid Certification Authority Your email address henrys cdach ernet in Your institute Your phone number Comments for the YO admin C You agree on the VO s usage rules Register Fig 1 7 VO Registration form VO stands for Virtual Organization Chapter 1 Steps to access GARUDA Page 8 GARUDA User Handbook As shown in Fig 1 8 you will receive an e mail from GARUDA Admin containing information about your access details which is hereafter referred to as your GARUDA Account From Grid Support subject Your Garuda Grid Account Details to Abhishek Srivastava Hi The Username and Password for accessing the portal or Gridfs is as below Username abhishekS61 Password s2222222 You can use below Portal link for Job submission http 192 168 60 40 gap2 gap Through NEN Link http portal garudaindia in Through Internet https gridzsupport garudaindia in Raising Bugs Fig 1 8 Mail about your GARUDA account Note Save this mail for your future reference For Garuda account related issues grid help Ocdacb ernet in Chapter 1 Steps to access GARUDA Page 9 Chapter Accessing GARUDA via Grid Portal GARU
57. urulivn y LJ IG CA Inuuguruliun opl g TYPE I L PUTT RND 0 ui 2NN Ty 1 yiid s utsf cdac org in linen Nianris Rimar Mieealirsal nan Pann Parallitliirni Slrigu Fig 5 8 Connected to GridFTP Server Chapter 5 GARUDA GridFTP Client Page 49 GARUDA User Handbook For Linux users If you are a Linux user then follow the below steps d b Download the Zip file from http portal garudaindia in gridftp sui download garuda gridftp zip Extract the contents of the download zip file The extracted folder will be the TOOL HOME directory Set Java to be available on the PATH The extracted content will contain a folder called certificates This contains the CA certificate files Login as root and move this folder to etc grid security If the grid security folder is not present in efc then please create it Use run sh script available in the TOOL HOME folder to invoke the software Refer the Fig 5 2 to Fig 5 8 for more information tool usage Where can I find online documentation for this tool Garuda GridFTP Client manual can be found from the below URL http portal garudaindia in gridftp gui gridftp gui htm How to access GridFTP in command line Refer Chapter 4 Chapter 5 GARUDA GridFTP Client Page 50 Garuda Storage Resource Manager GSRM GARUDA User Handbook Chapter 6 Garuda Storage Resource Manager GSRM What is GSRM GARUDA Storage Resource Manager GSRM is a peer
Download Pdf Manuals
Related Search