Home
gridbus.org
Contents
1. Application description file in XPML format a application lt arg gt optional Default application xml e e ee Gade caros Credential description file in XML optional g Default credentials xml Runs the broker in a graphical interface g gui optional Default lt no value gt Broker runs in command line mode h help Prints the usage help and exits Me e pacovery lt arg gt Activates Recovery mode with given id optional 5 X g Default normal mode i e starts a new application Service description file in XGRL optional s services lt arg gt Default services xml Prints the current Gridbus broker version and v version exits Runs the broker WSRF client when the Broker WSRF u url lt arg gt Service is hosted at the given URL Default lt no value gt The broker is run locally When combined with the u option specifies the application to connect to on the remotely hosted Broker WSRF Service Default lt no value gt The broker is run locally e epr lt arg gt 4 2 The Broker input output and configuration files The main input and config files that are needed by the broker are as follows The Broker properties configuration file The XPML application description file format The Service description file format Configuration files Note the default ones distributed should work fine for most cases Each of these files and their purpose is described
2. e Middleware installation which is one of 0 0 Oo O oO O oO 0 Globus 2 4 more info http www globus org Globus 3 2 with the pre WS globus gatekeeper and gridftp services running Globus 4 0 Alchemi 1 0 Cross platform manager more info http www alchemi net Unicore Gateway 4 1 experimental support within the broker more info http Awww unicore org Condor 6 6 9 more info http www cs wisc edu condor downloads Open PBS 2 3 more info http www openpbs org SGE Sun N1 Grid Engine 6 more info http Awww sun com software gridware index xml XGrid Technical Preview 2 experimental support for 1 0 For a data host one of the following services should be running e SRB v 3 x OR e Globus GridFTP service Additionally the user should have permissions to access the remote resources In case of Globus the user s credentials should be mapped to an account on the remote node Please consult the administrator of the resource for more details 2 2 Installation process Installing the broker is a simple process The broker is distributed as a tar gz and a zip archive that can be downloaded from http Awww gridbus org broker The installation just involves unzipping the files to any directory and optionally setting the PaTH environment variable to include the broker executable script gbb sh or gbb bat depending on your OS Following are the steps to be followed e Unzip the archive to the directory
3. examples apps calc xml e The Service Description The Service description specifies the available services on the grid and is provided to the broker in an XML format A service description file has a description of the services that are to be used by the broker for executing the grid application The broker distribution has a service description template which are has to be modified to specify the services the user has access to For example examples services services xml The following instructions assume the broker is being started from the directory where it was installed since it uses relative paths to refer to the broker input files It also assumes that the PATH Variable includes the broker binary To run the broker with the default configuration the following command is used at the command prompt from the broker s installation directory For nix lt broker install dir gt gbb sh a examples apps calc xml s examples services services xml c examples credentials credentials xml For Windows C lt broker install dir gt gbb bat a examples apps calc xml s examples services services xml c examples credentials credentials xml Where lt broker install dir gt refers to the directory where the broker is installed This will now start the broker and there should be some output scrolling by which informs the user about what the broker is doing For more detailed description about available command line options flags
4. 0 So the number of jobs created is 15 x 1 15 jobs the first parameter can take 15 possible values and the second parameter can have one possible value In case gridfile type parameters are used the number of jobs can be ascertained only at runtime since the broker has to actually resolve the file names to physical files before creating one job for each file A gridfile parameter can be defined as shown below lt parameter name infile type gridfile domain file gt lt IEsIKe joreiqueocoul Vesaly Welk S E DE IDAN aA S lt parameter gt For multiple grid files multiple lt file gt elements are placed within the lt parameter gt element as shown lt parameter name infile type gridfile domain file gt lt files protocols bw ua l ES ond birar U fishes pirEOcoc ou EIEE M url lfn somedirectory someotherdirectory abc txt gt lt file protocol srb url srb sample example gridfile stdout j gt lt parameter gt An application can have only one task with any number of commands in any order Tasks A task consists of commands such as copy execute substitute etc e A copy command specifies a copy operation to be performed Each of the copy commands has a source and destination file specified e An execute command is where actual execution happens The execute command specifies an executable to be run on the remote node It also specifies any arguments to be passed to the command on the
5. S Application Te a Portal J Broker i o Client te S p Alchemi Manager Alchemi Executor y Alchemi gt Executor Net Now a i Alchemi N Executor J Alchemi Ka Executor A Figure 13 Running the broker to submit to Alchemi middleware via Web Services Application Portal Broker Alchemi Client Console Alchemi N Manager Alchemi Executor Alchemi Executor Figure 14 Running the broker to submit to Alchemi using command line client
6. an information service which can be queried to extract data about available SRB storage resources and files located on them lt service type information gt lt information type srbMCAT gt lt SEDMCAT host srbhost cs mu oz au defaultResource defres domain dom home myhom port 9999 gt lt information gt lt service gt 5 TROUBLESHOOTING This section has some information on troubleshooting the broker Please refer to the users lists on source forge for more up to date information on the broker e If you get a ClassNotFoundException while running the broker please make sure the broker jar and all other jar libraries are in the classpath e If you geta source not found error it means you are not running the bash shell source is a bash command which sources the commands in a shell script In the broker source is used to set the appropriate classpath for the broker to find all the jars and libraries In such case please change the existing shell to bash shell e Ifyou get an error like Cannot Eind Llog4 properties Logging will mot be setup Continuing Creating configuration from hibernate cfg xml Exception in thread main org hibernate HibernatelBxception could not find file hibernate cfg xml It means the database config file is not found and the classpath is not set properly Since the config file is located in the classpath You need to set the GB
7. as comments Please note that the options are all specified in upper case and the broker is particular about the case sensitivity Note The file names are case sensitive or not depending on each operating system nix es are case sensitive Win9x is not Win NT 2000 XP preserves case but ignores them when reading accessing the files It is advised in general to pay attention to the case of the name of these files always and set the values accordingly HEHH Polling interval optional the time interval in milliseconds for scheduler polling Reducing this value will cause the broker to query job status more frequently but will result in more I O and processing Find a balance suitable to your application HEEE SCHEDULER_POLL_INTERVAL 5000 Ht tH The PERSISTENCE_DB option specifies whether an in built database should be started by the broker AUTO or the user will use an existing db server MANUAL The Default value is AUTO Users requiring good performance should consider using a seperate RDBMS server such as MySQL etc Appropriate changes will need to be made in the Broker hibernate cfg xml file to specify database driver connection settings If this setting is omitted the default setting AUTO is used PERSISTENCE_DB AUTO Figure 6 Broker properties configuration 4 2 2 The XPML application description file format XPML eXtensible Parametr
8. default application description interpreter for a language called XPML eXtensible Parametric Modelling Language which is designed to describe dynamic parameter sweep applications on distributed systems in a declarative way As such the broker can easily execute parameter sweep applications on the grid A parameter sweep application is one in which there is a program which operates on multiple sets of data and each instance of the running program is independent of the other instances Such applications are inherently parallel and can be readily adapted to distributed system For more information about research on grids and distributed systems please refer to http Avww gridbus org In addition the broker can interpret GGF defined JSDL documents only a subset of JSDL is supported New application descriptions can be easily plugged in 1 4 Gridbus Broker Architecture The Gridbus broker follows a service oriented architecture and is designed on object oriented principles with a focus on the idea of promoting simplicity modularity reusability extensibility and flexibility The architecture of the broker is shown in Figure 2 Application Description Service Credentials Description Persistence Interface Application Service Credential interpreter Interpreter interpreter Remote services on the grid Core Resource J Services Catalog e g Computation Data and 4 Information i Grid Information Service Databas
9. information drive the discovery of services such as computational nodes application and data services The service discovery module connects to the servers to find out if they are available and if they are suitable for the current application The broker uses credentials of the user supplied via the credential description whenever it needs to authenticate with a remote service Once the jobs are prepared and the services are discovered the scheduler is started The scheduler maps jobs to suitable servers based on its algorithm The dispatcher submits mapped jobs to the servers using the actuator component in the execution sub system which wraps the job in an agent that is sent to the remote node The actuator is a middleware specific component which communicates with the remote grid service using protocols understood by it The job monitor periodically monitors the jobs using the services of the execution sub system and updates the broker s internal state which is persisted to non volatile storage when needed As they jobs get completed the agents take care of clean up and gathering the output of the jobs The scheduler stops after all the jobs have been completed The scheduling policy determines whether failed jobs are restarted or ignored 1 5 Broker Usage Scenarios The broker can operate in a number of different hardware and software configuration scenarios Some examples include forking jobs on the local machine or a remote machine usi
10. where you want to install the broker In case of Windows you can use WinZip if you download the zip file or WinRar for the tar gz In case of nix run the command tar zxvf gridbusbroker 3 0 tar gz e The following directory structure is created under the main gridbusbroker 3 0 directory lt install dir gt bin Scripts to run the broker setup the database etc docs API docs examples Examples of xml description config files api etc lib Broker jar and its dependencies licenses Third party licenses manual Broker User and programmer s manual sre Broker source code xml XML schemas for the broker s application service and credential description files e Set the cBB_HOME variable to the directory where you have installed the broker e Additionally it is recommended to have the directory gridbus broker2 4 bin added to the system PATH variable For example for a Bash shell export PATH PATH lt broker install directory gt bin e Set the permissions for the all the scripts chmod 755 bin sh bin util sh 2 3 Recompiling the broker with ant The broker is distributed with the java source code An Ant build file is provided so running ant will recompile the broker jar and copy it to the dist directory The class files built are copied to the build directory There are separate build files for the broker and the WSRF broker service which can be used to build just the broker or bot
11. B_HOME environment variable to the directory in which you installed the broker For example if all the broker directories and files are in home belle gridbusbroker 3 0 the GBB_HOME is home belle gridbusbroker 3 0 While running the broker if you get an Error creating storage instance message it means the database is not set up properly Normally you should run the setup script from the broker s home directory Because the database is created where ever you actually run the setup script That means if you run the setup script from a certain directory then the application should also be run from the same directory 6 KNOWN ISSUES LIMITATIONS As with any software made by anyone the broker has its own limitations Some of these are listed here The Condor Unicore and XGrid implementations are NOT supported in this version Support for these middleware will be evaluated as time goes on and updated only if someone from the user base is willing to maintain it Under a Windows environment data aware scheduling may not be highly optimised since the broker depends on Network Weather Service NWS for bandwidth data NWS is currently not available for Windows and hence the scheduling may not reflect the correctness of the algorithm 7 FUTURE DEVELOPMENT PLANS The Gridbus Broker is a project in continuous development As part of the Gridbus project it aims to implement the innovations that come out of the research work performe
12. One can observe here that the value of an attribute of a parent element generally determines which child element can be placed inside it For example the type attribute of a service element determines whether a compute storage information or application child element can appear within a compute element This pattern is followed throughout the service description schema Credentials to access these services are specified in a separate file This separation of credential and service elements helps to support situations where the same credential such as a proxy certificate or a username password pair is to be used for authenticating to more than one service It is common experience that such a situation is frequently seen Compute services are servers to which the users jobs can be submitted for execution Storage services are used to store the results of execution and hence can be considered as data sinks Information and application services are those which provide generic services that can be used by the broker A compute service is associated with a middleware where the name of the middleware is to be specified In the above example it is globus To describe a globus node a description similar to the following is used lt service type compute gt lt compute middleware globus gt lt globus hostname manjra cs mu oz au version 4 0 fileStagingURL gsiftp manjra cs mu oz au tmp jobmanager jobma
13. The Gridbus Grid Service Broker and Scheduler v 3 0 User Guide Krishna Nadiminti Hussein Gibbins Xingchen Chu Srikumar Venugopal and Rajkumar Buyya Grid Computing and Distributed Systems GRIDS Laboratory Department of Computer Science and Software Engineering The University of Melbourne Australia Email kna hag xchu srikumar raj csse unimelb edu au http www gridbus org broker oO ON DUH INTRODUCTION 4 24 stevia a aa A ennai dadina dda dais Oneida 1 1 Grid Computing 1 2 REEK SIKO EE E E E E E A A E 3 1 3 GNlODUS Be CEREA T EE A E O E TE 3 1 4 Gridb s BrokerArchitoct re isir n a r de A ae 6 1 5 Broker Usage Scenarios ns minner keniana n aa Eaa aae AAEE N eas daana taaa iiaei 8 1 6 Sample Applications of the Broker ee eeeesseeseeseeeneeseneeeneseeeeseeeeseeseeesseeeaeseaeesaeesaeeesneeseeeeeeees 8 INSTALLATIONS 330 EA Re es ein cach els shee ct A TTE 2 1 Requirements karorini iosart a i 2 2 Installation process 2 3 Recompiling the broker With Ant cece ceeeeeseeseeeseeceeeeseeceaeeaeseseseaeeeseeeaeeeaeesseeseeeseneseaeeeatesaeees 10 GETTING STARTED USING THE BROKER criaria iiei aa Aa a i 10 3 1 Command Line Interface CLI running the broker as a stand alone program sseesseserserene 10 3 2 The Broker Workbench a Java GUI for the Broker ese eee cesses eeee tees sees teeeseaeteaeeeaeeeee 11 3 3 Application Programming Interface API eee eee cece cene eee etr
14. be noted that Grids aim at exploiting synergies that result from cooperation ability to share and aggregate distributed computational capabilities and deliver them as service The next generation of scientific experiments and studies popularly called as e Science is carried out by large collaborations of researchers distributed around the world engaged in analysis of huge collections of data generated by scientific instruments Grid computing has emerged as an enabler for e Science as it permits the creation of virtual organizations that bring together communities with common objectives Within a community data collections are stored or replicated on distributed services to enhance storage capability or efficiency of access In such an environment scientists need to have the ability to carry out their studies by transparently accessing distributed data and computational services This is where the concept of resource brokers comes into picture 1 2 Resource Brokers A Resource on a grid could be any entity that provides access to a resource service This could range from Compute servers to databases scientific instruments applications and the like In a heterogeneous environment like a grid resources are generally owned by different people communities or organizations with varied administration policies and capabilities Naturally obtaining and managing access to these services is not a simple task Resource Brokers aim to simplify this proces
15. command line e A substitute command specifies a string substitution inside a text file This operation is used to substitute the names of user defined variables for example Parameter names can be used as variables with a pre fixed Apart from this certain special default parameters variables are also defined such as o os which specifies the Operating system on the remote node jobname which refers to the job ID of a job created by the broker The example XPML file shown above specifies a task with three commands For the grid application described in the file above there are no requirements With this application description the broker creates 3 jobs with job IDs j1 j2 and j3 Each job performs the same set of operations or commands as specified in the tasks section A copy command has a source and a destination child element each with attributes location and file The location can take the values local and node The local value is interpreted as the machine on which the broker is executing and node is understood by the broker as the remote node on which the job is going to execute These values are substituted at runtime A substitute command is meant for substitution of parameter also known as variables values in local files which are then used during local operations remote job execution Typically the values of the parameters are determined at runtime and there could be scenarios in which certain inpu
16. d Distributed Systems Laboratory University of Melbourne Australia June 13 2005 Rafael Moreno Vozmediano Krishna Nadiminti Srikumar Venugopal Ana B Alonso Conde Hussein Gibbins and Rajkumar Buyya Distributed Portfolio and Investment Risk Analysis on Global Grids Technical Report GRIDS TR 2005 14 Grid Computing and Distributed Systems Laboratory The University of Melbourne Australia Nov 14 2005 Xingchen Chu Andrew Lonie Peter Harris S Randall Thomas Rajkumar Buyya KidneyGrid A Grid Platform for Integration of Distributed Kidney Models and Resources Technical Report GRIDS TR 2006 13 Grid Computing and Distributed Systems Laboratory The University of Melbourne Australia Aug 22 2006 APPENDIX I The following table identifies a number of possible hardware software configuration and provides details on how the Gridbus Broker can be utilised within such configurations and what steps need to be taken Although Gridbus Broker is primarily designed to support global Grids it can also be used to schedule applications on local nodes or one or more remote clusters This ensures scalability of broker from the user desktops to global Grids OS Platform Hardware Configuration LAN based Gridbus Broker Solution distributed system Single Batch Win32 server cluster Nix a c z If you have multiple systems whose configurations may be varied the broker can utilise these different systems at the same time Figur
17. d by the members of the GRIDS lab Below is a brief about the current plans about the future development path of the broker The list below mainly deals with new features Obviously there will be work going on to rid the broker of its current limitations and also improvement in existing features Application Description Interfaces and Programming models o Support for new application description languages and programming models such as Grid superscalar Others o Advanced scheduling algorithms o Support for Grid Marker Directory service VO directories and other information services as Community Authorization Service CAS o Support for grid economy model via GridBank o Ability to invoke any webservice using the broker dynamically at runtime 8 CONCLUSION AND ACKNOWLEDGMENTS This user manual attempts to explain the main concepts in the Gridbus Broker including how to install configure and use it The Gridbus Broker team would be very happy to answer any queries that you may have regarding the Broker For more details please contact Dr Rajkumar Buyya raj Srikumar Venugopal srikumar Krishna Nadiminti kna Hussein Gibbins hag and Xingchen Chu xchu append csse unimelb edu au for emails We would like to take this opportunity to acknowledge all the help and support extended to us by all the members of the GRIDS lab CSSE Department Melbourne University We would like to thank Dr Lyle Winton School of Physics and Brett Beeso
18. dbus Broker version 3 0 are o Discovery of services on the grid o Transparent Access to computational services running middleware such as Globus 2 4 Globus 3 2 pre WS Globus 4 0 Alchemi 1 0 o And queuing systems such as OpenPBS 2 3 Sun N1 Grid Engine 6 SGE This includes support for all basic services like Job scheduling and execution for batch jobs Job monitoring and status reporting Gathering output of completed jobs and directing it to user defined locations o Economy based scheduling with built in algorithms for cost time and cost time optimizations o Data aware scheduling which considers network bandwidths and proximity of data to Computational services o XML based application description format XML based service description format o Support for data sources managed by systems such as Storage Resource Broker SRB and the Globus Replica Catalogue o Persistence to enable failure management and recovery of an executing grid application o Extensibility the broker is engineered to support extensions in the form of custom schedulers middleware plug ins application description interpreters and persistence providers o Platform independence which is a natural consequence of a java implementation Home Node Portal o n A S655 P Credential Repository MyProxy PBS SGE Alchemi Gridbus Broker Figure 1 Broker Block Diagram The Gridbus broker uses a
19. e it does its own monitoring Extensibility is the key In Grid environments transient behaviour is not only a feature of the resources but also of the middleware itself Rapid developments in this still evolving field have meant that middleware goes through many versions and unfortunately interface changes are a norm rather than the exception Also changing requirements of Grid users require that the broker itself be flexible enough for adding new features or extending old ones Thus every possible care has been taken to keep the design modular and clean The advantages due to this principle o Extending broker to support new middleware is a zip Requires implementation of only three interfaces For more details refer to Programming section o Getting broker to recognize the new information sources is also easy o The differences in middleware are invisible to the upper layers such as the scheduler and vice versa Thus any changes made in one part of the code remain limited to that section and are immediately applicable For example after adding a new middleware the scheduler is immediately able to use any resource using that middleware o XPML is extensible Adding any new constructs is easy using the same reflection framework see Programming Section You could also do away with XPML altogether and implement your own favourite interface to describe applications Figure 1 shows the block diagram of the broker The main features of the Gri
20. e Fm C J Scheduler f status feedback Job Monitor ie Query Job Status Broker node Secure access using Data Catalog Service Monitor Network Information Service Execution Results user s credentials Remote node on the grid Remote Service E g Globus Alchemi PBS Remote Data bg Eg SRB GridFTP User Process Figure 2 Broker Architecture The broker can be thought of as a system composed of three main sub systems e The interface sub system e The core sub system e The execution sub system The input to the broker is an application description which consists of tasks and the associated parameters with their values and a resource service description which could be in the form of a file specifying the hosts available or information service which the broker queries and a set of credentials to use to authenticate to remote services The interface sub system has modules that convert these inputs into entities called jobs and services with which the broker works internally in the core sub system A job is an abstraction for a unit of work assigned to a node It consists of a task and variables A variable holds the designated parameter value for a job which is obtained from the process of interpreting the application description A service represents a service on the grid which could be a compute storage information or application service The task requirements and the service
21. e 7shows the broker interacting with three different hardware configurations and also different OS platforms Individual configurations are also described in this table a Install OpenPBS http Awww webmo net support pbs html or PBS Pro http www openpbs org about_pbspro html or Sun N1 Grid Engine http Awww sun com software gridware inde x xml on the batch cluster and enable the shared file system For the broker use PBS or SGE resources refer to this manual If the broker is not installed on the head node of the cluster configure the head node to enable SSH http www openssh com connections from outside Then configure the broker to use an SSH dispatcher instead of a local dispatcher which is the default setting If Globus Toolkit 2 4 x http www globus org toolkit downloads 2 4 3 has been installed on the batch cluster simply use GLOBUS_2_4 resource in the broker refer to this manual If Globus Toolkit 4 x http www globus org toolkit downloads 4 0 2 has been installed on the batch cluster simply use GLOBUS_4_0 resource in the broker refer to this manual For running the broker from a client of any of the above middleware see Figure 10 and Figure 8 Executing from a non client will first require an SSH to a client node as shown in Figure 9 E Use the Fork resource in the broker to submit and execute jobs on the local machine Figure 11 If the broker is not installed on the server configure
22. e eee eene tees teas eeaeeseeeseeeteaeeeaeeeee 14 3 4 Remotely hosted Broker WSRF Service 0 ceccecceceeeeee cess eens teas seaeeeseeeaeeeaeeeaeeseaeseaeseeeeneeneess 14 END USER GUIDE Swans eincddehti ne a a nah Senha ddd tains 15 4 1 Using the broker on the CLI with various flags 0 0 0 0 ee ee cece cee eeee cee eee eeaeeeeeeeeeseaeseaeseeeetaeeneeess 15 4 2 The Broker input output and configuration files wi 15 4 2 1 The Broker configuration filOS oo eee cece eeseeeeeeeseeeeeeesneeeseeeaeesaeesaeeceeseeseeeeeneseseesneneneeenanseaeeees 16 4 2 2 The XPML application description file format cee cee eee eee ee eee eteee teas sae eeaeeeaeeeeeneeeteaes 17 4 2 3 The Service description file wc sina sea at nee Waite es a a A aes 20 FROUBEESHOOTING ayers E ance cd co2essivas ashes T KNOWN ISSUES LIMITATIONS niaire baw atcha dai eatin de dee te FUTURE DEVELOPMENT PLANS CONCLUSION AND ACKNOWLEDGMENTS REFERENCES 2 shtavetied a es ee A ee E A AR PEND A AA d si ceepaskssvcenaschaeeyast A banca neue cote snes easiest ca snvdzsaabusssiagtsuer ahi eacansecaaun taba getapuaacuaee Gana 1 INTRODUCTION 1 1 Grid Computing A Grid is a type of parallel and distributed system that enables the sharing selection and aggregation of geographically distributed autonomous services dynamically at runtime depending on their availability capability performance cost and users quality of service requirements It should
23. h the broker and the WSRF service 3 GETTING STARTED USING THE BROKER The Broker can be used as a stand alone command line program or it can be used in your own Java programs or web portals This section describes the use of the Gridbus Broker in command line and embedded within a Java app modes The first thing that needs to be done is to setup the broker database The Broker uses an embedded HSQL server which is easy to setup from the broker install directory run bin util setupDB sh 3 1 Command Line Interface CLI running the broker as a stand alone program The broker can be invoked from the command line just like any other java program The broker distribution comes with a shell script and a batch file for Windows which just sets the correct class path and then calls the broker with any command line args passed to it In this mode the broker outputs its messages to both the console and a log file by default This behaviour can be modified by change the settings in the Broker properties configuration file found in the lib directory When running the broker on the command line it needs the following inputs e The Application Description The Application description is provided to the broker as an XML file which describes the grid application The value for this input can be any absolute or relative path The broker distribution comes with some sample app description files found in the examples directory For example
24. hieve separation of files from different jobs and applications 4 2 1 The Broker configuration files The broker needs the following configuration files e Broker properties e Broker log4j properties e Broker hibernate cfg xml for configuring hibernate persistence The broker distribution comes with a default set of these files which will work in almost all cases Logging for the broker is provided by the log4j library and persistence by the hibernate system The log4j properties and hibernate configuration and object mapping requires detailed knowledge of the log4j and hibernate systems The description of these files is out of the scope of the user manual The programmer s manual has some discussion of the hibernate mappings the broker uses This section briefly describes the Broker properties file The Broker properties file is a standard java properties file which is just a plain text file with a name value pairs one on each line The default Broker properties file supplied with the distribution is shown in figure 3 In the Broker properties config file shown above the broker is configured to look for the application description file named calc xm1 in the examples apps directory relative to the current directory from where the broker is executing The service description option points to the services xml in the examples services directory in this case The Broker configuration file ignores the lines starting with a and considers them
25. ic Modelling Language is an XML based language which is used by the broker to create parametric applications Simply put an XPML application description file is an XML file with special elements as defined in the XML Schema that comes with the broker XPML supports description of parameter sweep application execution model in which the same application is run for different values of input parameters often expressed as ranges A simple application description file is shown below An XPML app description consists of various sections such as parameters tasks and requirements Parameters Each parameter has a name type and domain and any additional attributes Parameters can be of various types including integer string gridfile and belong to a domain such as single range or file e A single domain parameter specifies a variable with just one value which is of the type specified in the type attribute of the parameter element e A range domain specifies a parameter which can take a range of values A range domain parameter has a range element inside it The range child element has from and to and step attributes which specify the starting ending and step values of the range e A file domain parameter specifies a gridfile which is the URL of a remote grid file A gridfile url can have embedded wildcards which are resolved to the actual physical file names by the broker file resolver A gridfile URL cu
26. in the subsections below The output files produced by the broker include e The execution logs from the broker itself broker log e The standard output and standard error files for each job that runs on the grid e The script files generated by the broker for each job these files exist only if the broker is run in DEBUG mode Otherwise they are deleted e Any other output files produced by the application that runs on the grid by the broker Each run of the broker creates a separate directory under a temporary directory which is in the current directory This directory has a structure as follows GBB TMP lt broker ID gt lt job ID gt stdout lt job ID gt stderr lt job ID gt broker log lt other temporary files gt For example GBB TMP jl sh Binary jobhandle ji1 Binary output j1 Binary gt queryJobStatus sh jl Binary gt queryJobStatus Result j1 Binary gt stderr jl Binary stdout j1 Binary j2 HS j3 aos gt j broker log Binary cpuinfo_manjra cs mu oz au_s1 Binary gt meminfo_manjra cs mu oz au_s1 Binary where brokerID is 93EF27FD 0A64 C8E 1 4671 95FDD222EBD4 jobID is j1 j2 j6 All the output files for each job are collected in its own directory ensuring clean separation of files Note The SAME directory structure with a prefix REMOTE is also created on the remote side to ac
27. ion with any of the middleware it supports It uses the minimum set of services that are required to run a job on a resource supported by the middleware The advantages of this are Ina Grid with multiple services configured differently the broker tries to make use of every service possible by not imposing a particular configuration requirement For example in the case of Globus 2 4 all is required is that the GRAM service be set up properly on the grid node The broker can run jobs on resources with different middleware at the same time The broker needs not to be refactored if there is a new version of the middleware o The broker is able to handle gracefully jobs and services failing throughout an execution The job wrapper and job monitor code is written to handle every failure status possible The scheduler does not fail if a service drops out suddenly o The failure of the broker itself is taken care of by the recovery module if persistence has been configured Client centric design The scheduler has just one target that is to satisfy the users requirements especially if the deadline and budget are supplied Even in the absence of these the scheduler strives to get the jobs done in the quickest way possible Thus services are evaluated by the scheduler depending on how fast or slow they are executing the jobs submitted by the broker In keeping with Principle 1 the broker also does not depend on any metrics supplied by the servic
28. l copy co Sget Spud l l 4 Execution Node S 1 Sh Ny l Cluster with Globus fork 5 Application D or PBS job manager Portal Globus i Execution Node Broker Execution Node Globus Client Node A Figure 8 Running the broker from a Globus client Cluster SGE PBS Space pei Middleware wee Head Node a Middleware Y a Na Client Node ee Node Execution Node te r Application S f Portal f Cluster SGE PBS a Broker Shiac Client Nod be ae Middleware i Sa A SSH Head Node N 2 a A ma S Client Node J Execution Node ae ES Execution ane Figure 9 Multi clustering Running the broker from a non client of the target middleware a Application Portal Broker ALW Ty Middleware Client Node Figure 10 Running the broker from a client of the target middleware le Space Middleware Head Node Execution Node a Cluster SGE PBS Execution Node S Application Single r Portal SEINE Z J lt Broker J Fork a N User Space A A Figure 11 Forking jobs on the Local machine nix only Application Portal d L a Broker Client Node A 2 Va Fork SSH a User Space Server Node Figure 12 Forking jobs on a remote machine using SSH n ieee Service Xs j a
29. n from QUT for their comments and contribution 9 REFERENCES 1 2 3 4 5 6 7 R Buyya S Date Y Mizuno Matsumoto S Venugopal and D Abramson Neuroscience Instrumentation and Distributed Analysis of Brain Activity Data A Case for eScience on Global Grids Journal of Concurrency and Computation Practice and Experience 2005 Online Available http www gridbus org raj papers neurogrid ccpe pdf S Venugopal R Buyya and L Winton A Grid Service Broker for Scheduling Distributed Data Oriented Applications on Global Grids in Proceedings of the 2 Workshop on Middleware in Grid Computing MGC 04 5th ACM International Middleware Conference Middleware 2004 Toronto Canada October 2004 B Hughes S Venugopal and R Buyya Grid based Indexing of a Newswire Corpus in Proceedings of the 5th IEEE ACM International Workshop on Grid Computing GRID 2004 Pittsburgh USA IEEE Computer Society Press Los Alamitos CA USA Nov 2004 B Beeson S Melnikoff S Venugopal and D G Barnes A Portal for Grid enabled Physics In Proceedings of the 2005 Australasian Workshop on Grid Computing and e Research AusGrid 2005 Newcastle NSW Australia Australian Computer Society January 2005 H Gibbins K Nadiminti B Beeson R Chhabra B Smith and R Buyya The Australian BioGrid Portal Empowering the Molecular Docking Research Community Technical Report GRIDS TR 2005 9 Grid Computing an
30. nager pbs gt lt queue name workq cost 1 priority 1 limit 50 gt lt globus gt lt compute gt lt service gt In the above service description the node manjra cs mu oz au is specified to be running globus v 4 0 Similarly other compute services can be described as defined in the schema The type attribute can be any of compute information and application The middleware type tag can optionally contain a set of queues The jobmanager attribute of this tag identifies the queuing system If no queue tags are defined the available queues on that node will be discovered at run time Otherwise the queues can be defined explicitly as shown below providing a higher level of control Note Data sources are discovered by the broker at runtime using the application description which contains file parameters and information catalog services defined as service elements in the service description Hence the need for explicitly specifying data sources in the service description is obviated Information services are typically entities which provide information about other services These could be LDAP directories web services data catalogs etc Currently supported service types include the SRB MCAT and the Replica Catalog Application services provide applications hosted on nodes that can be accessed as a service The example below shows the definition of a SRB Metadata Catalog This is modelled as
31. ng SSH or submitting a job to a remote queuing system without the need of Globus For more details on possible configurations please see APPENDIX I 1 6 Sample Applications of the Broker The Gridbus Broker has been used in Grid enabling many scientific applications successfully in collaboration with various institutions worldwide Some of these are listed below Neuroscience Brain Activity Analysis 1 School of Medicine Osaka University Japan High Energy Physics 2 School of Physics The University of Melbourne Natural Language Engineering 3 Dept of Computer Science The University of Melbourne Astrophysics 4 School of Physics The University of Melbourne Molecular Docking Drug Discovery 5 WEHI The University of Melbourne Finance Portfolio analysis 6 Complutense University of Madrid Spain Kidney Grid Distributed Kidney modelling 7 Faculty of Medicine The University of Melbourne It has been also been utilised in several Grid demonstrations including the 2003 IEEE ACM Supercomputing Conference SC 2003 HPC Challenge demonstration 2 INSTALLATION 2 1 Requirements Broker side i e on the machine running the broker e Java Virtual Machine 1 4 or higher More info htto Awww java com e Valid grid certificates properly set up if using remote Globus nodes By default the certificates are placed in the lt USER_HOME gt globus directory Where lt USER_HOME gt is the user s home directory For a use
32. please refer to the User Manual section If invoked via the command line the broker is always a non interactive program This behaviour can be altered to suit the user s needs by using the broker APIs in programs built on top of the broker The next section has some information about how to do that 3 2 The Broker Workbench a Java GUI for the Broker The broker distribution comes with a GUI called Gridbus Broker Workbench which is a convenient MDI java swing GUI for users to prepare execute and monitor a new grid application using the broker There are some useful features that the workbench provides to the end user e A visual XML editor to create and modify the description files CPML XGRL XCL JSDL and ADL required by the broker e A broker execution manager console that can execute and monitor the grid application running on top of the broker e A set of dynamic visual charts to demonstrate the statistics information of the application execution To run the workbench the user can just utilize the shell script provided in the distribution with g argument Visual XML Editor Once the user launches the broker workbench user will see the main window Users can now open the XML editor to prepare modify and validate their application XPML JSDL and ADL services XGRL and credentials XCL description files Users can have difference views against the xml file including the tree source structure and graph view Fig 1 sho
33. r belle on a UNIX machine this would be home belle globus For a user belle on a Windows NT 2000 XP machine this would be C Documents and Settings belle globus For more information on how to acquire and setup x 509 certificates please consult http www globus org security v1 1 certs html e Additionally some ports on the local node should be configured to be open so that the jobs can connect back to the broker Please refer to the Globus documentation for more details e Optional Components O OpenPBS v 2 3 Portable Batch System Required if running jobs on a local cluster managed by a PBS system More info http Awww openpbs org Network Weather Service NWS v 2 8 client tools Required for improving the accuracy of scheduling data intensive applications that access remote data hosts More info http nws cs ucsb edu Note NWS client tools are only available for nix Grid applications that need remote data can still be run using the broker on Windows however optimal selection of data hosts is not assured since the absence of NWS will mean the broker cannot get that information by itself We are working on some way to avoid workaround this dependency in future versions of the broker SCommands Client tools v 3 x for SRB Storage Resource Broker Required if running applications that need to access SRB data More info http www sdsc edu sro scommands index html Remote Grid node side Fora compute resource
34. r Workbench 1 00000 A LEE or B Application Statistics Help GH Open XML Editor GB Gridbus Broker Visual Editor FJ Exit Workspace File Application oe OHO Oo IF xPML XGRL CREDENTIAL JSDL ADL le xpmi Source View Structure View mins ntpwschemas grid c E parameter X c E task pg ay p gt pal Validation Information 28 07 2006 1208 44 INFO xprnl is valid Figure 3 Visual XML Editor Main Application Statistics Help amp Gridbus Execution Monitor Console Basic information Start Time Fri Jul 28 12 23 14 EST 2006 Time Spent 00 02 56 994 Time Remaing 320594 20 17 683 Budget Spent WBRescy PEeScheduled B Stage in RA bette csse unimetb edu au HB stage out Pscomtes Running ngicg vpac org HB Done Mi Faitea Bh vnmown 4 e mn fF Status Indicator Application Logger al Job Status Done Failed Active Pending Submtd Schedid Stageln StageOut Unknown Ready Total IsAlive Deadline 10 44 41 842 Yved 26 Jul 2006 Time lem 23 59 20 142 Budget 9 223372E18 Gt Budg spent 0 0 O Figure 4 Execution Monitor ces server XY Job On Server Chart bak ase unimad aduau ngeowpetog taletsseunindbeduas mafifacsae nimed manyacssaunimabadsau manya Figure 5 Performance Statistics 3 3 Application Programming Interface API The Gridbus broker is designed to be very flexible and extensible It i
35. r manual and then go on to the programmers manual which has more detailed explanation of the broker architecture and common APIs The last section of the programmer s manual also has descriptions of how to extend the broker s APIs and build upon them to suit your needs 3 4 Remotely hosted Broker WSRF Service Starting from version 3 0 the Gridbus Broker can be hosted as a remote WSRF service This allows applications to avoid managing the dependencies and configuration on the client machine and using the Broker web service The Broker WSRF Service has been developed using the Globus WSRF implementation and tooling and can be hosted in any WSRF container To build and deploy the Broker WSRF Service you will need the WSRF Core Java implementation The wsrf broker service build xml distributed with the broker assists in building and deploying the broker The broker specific configuration for the WSRF Service is similar to the case when the broker is run on the command line The broker command line starter can also be used as a WSRF client see section 4 1 This makes use of the WSRFClient API that is found in the distribution 4 END USER GUIDE 4 1 Using the broker on the CLI with various flags The broker provides the following usage options on the command line gbb g a lt application xml file gt r lt appID gt s lt services xml file gt c lt credentials xml file gt u lt broker service url gt e lt epr file gt
36. rrently supports the URL protocols LFN and SRB Note The name of the child element must match with the value of the domain attribute of the parameter element lt xml version 1 0 encoding UTF 8 gt lt xpml xmlns xsi http www w3 org 2001 XMLSchema instance xmlns http schemas gridbus org xpml 2006 01 xpm1 xsi schemaLocation http schemas gridbus org xpml 2006 01 xpm1 XPMLSchema xsd gt lt parameter name X type integer domain range gt Leao EEOAE LY oS METAS Ys lt parameter gt lt parameter name time_base_value type integer domain single gt lt single value 0 gt lt parameter gt lt task gt lt substitute gt lt source file output gt lt destination file output jobname gt lt substitute gt lt copy gt lt source location local file calc OS gt lt destination location node file calc SOS gt lt copy gt lt execute gt lt command value calc SOS gt lt arg value X gt lt arg value Stime_base_value gt lt execute gt lt copy gt lt source location node file output gt lt destination location local file output jobname gt lt copy gt lt task gt lt xpml gt A grid application can have any number of parameters The number of jobs created is the product of the number of all possible values for each parameter In the example show above parameter X ranges from 1 to 15 The second parameter has a constant value
37. s The supported parameters are the same as the ones supported in Globus RSL 4 2 3 The Service description file The Service description file is just an xml file describing the services that can be used by the broker and their properties as defined in the service description schema that comes with the broker The Service description can be used to describe two types of entities services and credentials to access the services A service as defined currently can be of three types Compute services Data services Information services Application services A sample service description is shown below lt xml version 1 0 encoding UIE 8 2 gt lt ServiceDescription xmlns http schemas gridbus org xgrl 2006 01 xgrl xmlns xsi http www w3 org 2001 XMLSchema instance xsi schemaLocation http schemas gridbus org xgr1 2006 01 xgrl ServiceDescriptionSchema xsd gt lt service type compute gt lt compute middleware globus gt lt globus hostname belle cs mu oz au fileStagingURL gsiftp belle cs mu oz au tmp gt lt compute gt lt service gt lt service type compute gt lt compute middleware globus gt lt globus hostname manjra cs mu oz au version 4 0 Pile T GS LEbot ran ica es mi Oe aus ume jobmanager jobmanager pbs gt lt Qqueue name Wworkg cGost 1 priority 1 Llimit 50 gt lt globus gt lt compute gt lt service gt lt ServiceDescription gt Note
38. s by providing an abstraction layer to users who just want to get their work done In the field of Grids and distributed systems resource brokers are software components that let users access heterogeneous services transparently without having to worry about availability access methods security issues and other policies The Gridbus service broker is a resource broker designed with the aim of solving these issues in a simple way 1 3 Gridbus Broker The Gridbus broker is designed to support both computational and data grid applications For example it has been used to support composition and deployment of neuroscience compute intensive applications and High Energy Physics Belle Data Grid applications on Global Grids The architecture of the broker has emphasis on simplicity extensibility and platform independence It is implemented in Java and provides transparent access to grid nodes running various middleware The main design principles of the broker include Assume Nothing about the environment No assumptions are made anywhere in the Broker code as to what to expect from the Grid resource except for one that the resource provides at least one way of submitting a job and if running a flavour of Unix will provide at least a POSIX shell Also no assumption is made about service availability throughout an execution The implications of this principle have a huge impact throughout the broker such as o The broker has no close integrat
39. s targeted at both basic usage and customisation by providing programmers the ability to access most of the common APIs which are used internally The full functionality of the XPML files is available for programmatic access via the API This makes it easy to integrate the broker into your own programs Using the broker in your programs is as simple as copying the gridbusbroker jar into a place where the libraries for your program are located and invoking a single class to start the broker in the simplest case as shown below try Create a new Farming Engine for a single application Broker For a multi application Broker use the Broker class GridbusFarmingEngine fe GridbusFarmingEngine createInstance ealeo xml services xmlL credentials xml System out println Starting broker scheduling Start scheduling fe schedule ihe sehic cuikemmetsho cigs ele utsm sieimme diate lyan ers anene scheduling To wait for results monitor jobs use the following loop K while fe isSchedulingComplete Thread sleep 5000 catch Exception e e printStackTrace The samples provided with the broker distribution show some common ways in which the broker can be invoked from a java program or a JSP web application The programmer s manual has a more detailed explanation of how to use the common broker APIs Those who want to use the APIs are suggested to first read through the use
40. t text files need to be tailored for each job using these parameter values Any of the parameters can be used as a variable in the files used in a substitute command by pre fixing to the parameter name So the parameter X is the variable sx A substitute command has source and destination file names and a location attribute which must be local The following is an example of a substitute command lt substitute gt lt source file input gt lt destination file input jobname gt lt substitute gt In the substitute command shown above the destination element itself has another variable 4jobname which refers to the job s unique id So after substitution the input file is tailored to each job and saved as input j1 input j2 etc for each job Requirements Certain jobs need a particular environment during execution This environment needs to be setup before the job actually starts executing on the remote node For this purpose the Requirements element is provided It can be used to specify a set of initialisation tasks and in the future conditions Requirements are used to specify the resource filtering requirements for a job This is specified as a set of name value pairs as shown below lt job requirements gt lt property name minmemory value 256 gt lt property name maxmemory value 1024 gt lt job requirements gt Note Currently the job requirements feature works with only Globu
41. the server to enable SSH http www openssh com connections Then configure the broker to use an SSH dispatcher instead of a local dispatcher i i S a re CL 7 which is the default setting Figure 12 aa a a Install Alchemi 1 0 http Awww alchemi net Manager on the selected head node install Alchemi 1 0 Executor on the other nodes and make sure they ve connected to the manager For the broker use Alchemi resource which will interact with the manager via web services or the command line client refer to this manual See Figure 13 Figure 14 below globus url copy ws Ea Sget Spud gt l Execution Node l ss 4 d Cluster with Globus fork or PBS job manager Globus pas Execution Node KALEH N f Execution Node a N gt Application Portal F ace Net Now K J ag j A S Alchemi e Client Node yy Web Service 7 T Manager j Alchemi EN a Alchemi Ur Executor chemi Executor yf a Alchemi N gt z C Executor wie SSH AS Cluster SGE PBS Space a Middleware y M Head Node _ N W Middleware 7 xe Client Node eA Execution ia Alchemi Alehemi Executor ae Executor F Execution Node NS Pi Figure 7 Global Grid Utilising multiple hardware software configurations at the same time globus ur
42. ws a example of editing a XPML file Users can add element or attribute to each named node for example the task node the editor guarantees that only the valid sub elements or attributes appear while adding them As soon as the user have completed all the xml files can be validated before saving them on to the disk or sending to the broker Broker Execution Manager Console Besides editing the description files the workbench also allows users to execute and monitor their newly created grid application Within the console users can get the basic information related to Qos Jobs and Servers Furthermore as long as the users click the job or the server name another two windows will show more details about the job and server In addition the application logger area is also refreshed and scrolled automatically during some period of time for user to know the exactly information of the execution of the broker To use the facilities related to the application execution and monitor uses just need to choose the items in application menu Statistics Charts Another interesting feature that the workbench provides is the dynamic visual charts for the statistics information related to the application execution such as job status job completion rate for server and Qos budget and schedule The screenshots above show the job status and job running on server charts Users can choose other charts under the statistics menu H Gridbus Broke
Download Pdf Manuals
Related Search
gridbus.org gridbusters grid bus grid business
Related Contents
Infineon C515C-8R / -8E, C515C-L User`s Manual Image Repainter User`s Manual Bedienungsanleitung kurz 03 vom 29.11.05 取扱説明書 Copyright © All rights reserved.
Failed to retrieve file