The Supercomputing Facility for Bioinformatics
Contents
1. Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 6 2 Running programs After compiling a program the default executable is created as a out file which can be run using the following syntax S gt a out Java programs after compilation produces intermediate byte code class files which need to be interpreted using the following command gt Java source class Note You are discouraged from running a job in interactive mode for an extended period of time by using a out OR java source class It s OK to do so for testing purpose but not for more than 1 2 minutes Such job slow down the entire system and are liable to be killed 7 Submitting Jobs 7 1 Submitting serial single processor jobs For firing a job through SGE you need to use a submit script say submit sh Example of submit sh bin bash wo loam do sin io cwd source etc profile Soules esla olor ls mee Ole a a A aa jo source bash_profile only if you have any local settings in bash_profile S gt qsub submit sh The qsub command is issued by the user to submit a file into the batch execution queue Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 10 7 2 Submitting MPI Jobs For firing a job through SGE you need to use a submit script say submit sh Example of submit sh ap loa loss la fo o Buin Dash 5 cwd HS pe cre np source etc profile sou2. CTORY IN THE REMOTE HOST sccscesceceececscecscscescsceccscecescsceececeecececscsscscesescnsescesescusescecescs 9 5 7 CLOSING THE CONNECTION ies 9 COMPILING amp RUNNING PROGRAMS Guin toilet 9 6 1 COMPILINO PROGRAMS a EEEE E E ENA NA a e 9 6 2 BRINN rR OG IN E EEEN AE OA E A A alae OA E Nes OE ea 10 SUBMITTING JOBS aaaeei coo 10 TAI SUBMITTING SERIAL SINGLE PROCESSOR JOBS cooccccnocncnocnnnnccnoncononacnnnncnnonocnonornonnononnrononcnnonornonnrnnnnccnonccnananos 10 7 2 SUB IMI T TING VER Ld OBS one dde E T A EE E 11 7 3 CHECK THE STATUS OF SUBMITTED IO BS lidia 11 7 4 REMOVING SUBMITTED JOBS FROM QUEUE csccsccssccsccsccsccsccsscescescessescesscsscescescessesscsscescescessessessesscesceseescens 11 OPTIMIZA ION a EI E E EIEE TE soca 12 AVAILABLE SOFTWARE 00 cidos 12 9 1 AMPER O ia id id ld 12 9 2 SS A em ON E EER REE EE E REN METER NOAM AE TET CNRS MEELIS METER NUM MTEC TE TET ERROR LET NEEE EA 12 9 3 AE ARA E A a ee E A E RMT Tne E A IR A 12 FREQUENTLY ASKED QUESTIONS FAOS iii a ia cana 12 10 1 WHEN I RUN A MPI JOB USING MPRUN NP X COMMAND IT GIVES THE ERROR MPRUN MISSING ENVIRONMENT VARIABLE JOB ID INVALID REQUEST enpe E EE A 12 10 2 MY JOB TAKES A LONG TIME TO START OR AT SOME TIME IT DOESN T START ALTOGETHER cecescecececes 12 10 3 HOW DO I ACCESS SCFBIO FROM WINDOW MACHINES sccsceccscecccceccececccscccsceccscuccscecescssscscescscecescecesces 12 10 4 IN GROMACS SOME COMMANDS ARE SUFFIXE
3. D _ MPT AND MANY OTHERS NOT csscsceccececcececcececescecess 13 10 5 HOW DO I CHECK THE STATUS OF A RUNNING PARALLEL JOB cccececcecececcecececcscscscecescuccscecescscscscesescusesces 14 Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 3 10 6 I GET THE ERROR MESSAGE TIMEOUT WHILE LAUNCHING A MPI JOB AND IT ASKS ME TO USE T FLAG WITH MPRUN 14 10 7 I AM A BEGINNER AND I WANT TO LEARN MORE ABOUT UNIX IN GENERAL CAN YOU PLEASE GIVE SOME REFERENCES sia Ni cette 14 10 8 WHENEVER I DELETE MULTIPLE FILES USING WILD CARD SAY BY RM NEED TO ANSWER Y FOR EACH FILE 14 10 9 I DON T LIKE THE DEFAULT EXECUTABLE NAME LE A OUT HOW CAN I CHANGE IT oo ccc cc cece cecceeccesceeseeeseesees 14 10 10 JS THERE ANY AVAILABLE DEBUGGER scscsasissecvcnenesaa gh sc eceuvduasew ie 14 10 11 AM UNABLE TO TELNET FTP SSH SFTP TO ANY OTHER MACHINE FROM THE SUPERCOMPUTER ssceeeeeeees 14 10 12 WHEN I OPEN A FILE USING VI EDITOR I FIND M AT THE END OF EVERY LINE sccsccsccssccsccscesccsscsscescescens 15 LOS HOW DOLECHECK MY OUOTASTA TUS ino 15 10 14 AM UNABLE TO CONNECT TO THE PROVIDED IP ADDRESS AND THE IP ADDRESS ISN T RESPONDING TO PING PE OS eSpace tetas teed eee tan te sete etcetera esas cee tect ates A ceen asta eet een ane atn ines eet eit 15 10 15 HOW CAN I LEARN MORE ABOUT A COMMAND ccccccssccssccscccsscecceesccesccessceusccesscuscseusseesccuscceusseescce
4. Supercomputing Facility for Bioinformatics amp Computational Biology MT Delhi a meat My d The Supercomputing Facility for Bioinformatics amp Computational Biology IIT Delhi a PPP E e User Manual Dr B J ayaram Professor of Chemistry and Coordinator Surojit Bose Pankaj Sharma Vidhu S Pandey Praveen Agrawal Supercomputing Facility for Bioinformatics amp Computational Biology 3 Floor Synergy Building IIT Delhi Hauz Khas New Delhi 110 016 Ph 91 11 26596786 www scfbio iitd org Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 The resources and facilities provided by the Supercomputing Facility for Bioinformatics amp Computational Biology IIT Delhi SCFBio are geared to cater for specific scientific research work in life science area only Understandably the resources at SCFBio are critical and limited It is therefore expected that users play a constructive role ensuring a responsible usage conduct of the resources provided by SCFBio Biology easily has 500 Years of exciling problems lo work on Donald E Knuth SH would oppear we have reached the timtts of what it ts possible lo achieve wtth computer technology although one should be careful with such statements they tend to sound pretty silly after a Years John von Neumann Wherever You go lake your Supercomputer atong with You SCFBio Supercomputing Facility for B
5. Tunnels Bugs Basic options for your PuTTY session Specify Your connection by host name or IF address Host Name or IF address Port 55H Protocol O Raw Rlogin Load save or delete a stored session Telnet Saved Sessions Default Settings Close window on exit CO Always Never Only on clean exit Please use the values from the figure Host Name amp Port 10 3 2 psftp a C Documents and Settings Administrator Desktop psftp exe Bpsf tp no hostname specified use open host name to connect psftp open userl Please use the above command to work replacing user by your provided userID 10 4 In GROMACS some commands are suffixed _ mpi and many others not Gromacs doesn t have all the programs implemented in parallel There are only 4 which run parallel namely disco_mpi ffscan_mpi g_disre_mpi amp mdrun_mpi To run any of these 4 Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 programs please use Submitting MPI Jobs while for others Submitting serial single processor jobs 10 5 How do check the status of a running parallel job gt mpps Apf If you find any process in the status of EXIT CORING SIGNALLING kindly check your submit script If any be in the status of SPAWNING for a prolonged period of time please mail us detailing time amp date 10 6 get the error mes
6. ioinformatics amp Computational Biology User Manual 1 0 TABLE OF CONTENTS N 10 OVERVIEW dioss 5 THE FACT eLo 5 GETTING STARTED ainan ae a ae a a 5 3 1 LOGGINGINTO THE REMOTE MACHINE a a le E nl ae 5 3 2 FOGGING OUT OP THE PPO MACHINE taa avis 5 3 3 LOGIN PROFICES ANDSHELES a te a a tad e 5 MANIPULATING FILES AND DIRECTORIES ccccccccscccccccccccccccccccccccccccccccccccccccccccccscccccccccccsccecs 6 4 1 ENANCINO DIRECTOR T acct es teresa e ea ates E e la soe e iat a 6 4 2 UA iy DC eee ea ee Br AT A CET FTN AI II REPO TNR OREN We a 6 4 3 Td BS SNK NR A SC ge A A RO eT Ei 6 4 4 MARP ANEW DIRECTOR cath at thas ct Tt ii oso tag te rater boe ee ron ice es uba oan at a ieee 6 4 5 IVIO VBR BI EEES e eae ae acto tt od pa 7 4 6 PRESENT WORKING DIRECTORY avancen th tes argc sets heel tates eae eras saad ees aod ae 7 4 7 FEMO EFE na E o o E MO RR CTE nen eer ee er 7 4 8 REMOVE DIRECTOR on eto oia 7 4 9 LASTING FILES AND DIRECTORIES sencor a o locas dec J FILE TRANSFER oaa aa a a ERM nee nee ERIS IN eNO A 8 5 1 ESTABLISHING REMOTE CONNECTION a eave tahoe 8 5 2 PCE UPON DIN G AEE O E E E E E E E 8 5 3 FILE DO WNLOADING leas 8 5 4 MAKING A NEW DIRECTORY ON REMOTE HOST csccececcececcececcsceccsceccecsceecscscscescscesescesescsseecssescscscscesescesesces 9 5 5 CHANGING DIRECTORY IN LOCAL HOST ccceccscececceccsceccececscscscsceecsceccscucesceceecsceecsceecscecscsescesescesescecescececs 9 5 6 CHANGING DIRE
7. irectory and all subdirectories 5 File Transfer 5 1 Establishing remote connection To establish a connection to a remote system use the sftp command After the connection is established provide the valid password gt sftp user aa bb cc dd 5 2 File uploading Move a file from the local host to remote host gt put filename To put multiple files using wild cards gt mput pattern 5 3 File downloading Move a file from remote host to local host gt get filename To get multiple files using wild cards gt mget pattern Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 g 5 4 Making a new directory on remote host gt mkdir directoryname 5 5 Changing directory in local host gt lcd directorypath 5 6 Changing directory in the remote host gt cd directorypath 5 7 Closing the connection gt bye 6 Compiling amp Running Programs The following compilers are available C cc C CC FORTRAN 90 90 FORTRAN 77 f77 JAVA 6 1 Compiling programs gt cc source c compiles C program source c and produces executable a out gt CC source cpp compiles C program source cpp and produces executable a out gt f77 source f compiles FORTRAN program source f and produces executable a out gt 90 source f compiles FORTRAN program source f and produces executable a out gt javac source java compiles Java program source java and produces byte code source class
8. le JOB_ID Invalid request We have prohibited such direct running of processes as 1t by passes the queuing facility provided by SGE and overloads the machine That effectively slows down every other job running on that node Please follow the correct procedure given in the Submitting MPI Jobs 10 2 My job takes a long time to start or at some time it doesn t start altogether The waiting period of a job is directly dependent upon the number of jobs already waiting in the queue and the number of processors required by your job A job is started only when the required number of processors is free If your job requires large number of processors 1t may very much happen that those number of processors never be free With our testing amp benchmarking we have found that the optimum number of processors to be 4 8 For 4 your queue waiting time should be smaller 10 3 How do l access SCFBio from Window machines There are many free ssh or sftp applications available for window machines e g you can download putty ssh amp psftp sftp from http www chiark greenend org uk sgtatham putty download html Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 12 10 3 1 putty 2 PuTTY Configuration Category Session Logging B Terminal Keyboard Eell Features window Appearance Behaviour Translation Selection Colours Connection Proxy Telnet Alogin E 55H Auth
9. pulating Files and Directories 4 1 Changing directory cd without arguments puts the user in the users home directory With a directory name as an argument the command moves the user to that directory gt cd directorypath 4 2 Copy files cp makes copies of files in two ways gt cp filel file2 makes a new copy of filel and names it file2 gt cp list of files directory puts copies of all the files listed into the directory named Contrast this to the mv command which moves or renames a file 4 3 Making a link In creates a link between files Example The following links the existing file example c to ex c gt ln example c ex c The following creates symbolic links gt ln s usr include incl See the online man pages for many other ways to use In 4 4 Make a new directory mkdir makes a new subdirectory in the current directory gt mkdir directoryname makes a subdirectory called directoryname Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 6 4 5 4 6 4 7 4 8 4 9 Move rename files mv moves or changes the name of a file S gt mv filel file2 changes the name of filel to file2 If the second argument is a directory the file is moved to that directory One can also specify that the file have a new name in the directory direc S gt mv filel direc file2 would move filel to directory direc and give it the name file2 in that directory Pre
10. r find M at the end of every line Whenever a file is opened in Windows it appends M at the end of every line It is just a matter of format difference between windows amp UNIX To remove M please use gt dos2unix originalfile convertedfile 10 13 How do check my quota status gt quota v 10 14 l am unable to connect to the provided IP address and the IP address isn t responding to ping request For the security reasons the IP address doesn t reply to ping request If you are unable to connect the primary reason would be that the network is down on your end To check please ping www mail yahoo com If yahoo com responds and you are unable to connect even after waiting for more than 1 2 minutes then please drop us a mail detailing time amp date 10 15 How can I learn more about a command S gt man command name Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 15
11. rce oats jo Ote Ma ese aos eS sa O eS np to be replaced by number of processors Recommended 4 8 source bash_profile only if you have any local settings in bash_profile S gt qsub submit sh The qsub command is issued by the user to submit a file into the batch execution queue For submitting a Gamess job on job inp submit sh bin bash jo 8 loba lesa is Ol FS pe gamess np source etc profile Sonics als pio ml opt soft gamess_parallel gamess job 01 np gt amp submit log gt gsub 1 g 1 submit sh 7 3 Check the status of submitted jobs gt qstat 7 4 Removing submitted jobs from queue gt qdel job_id job_id is obtained using qstat output and errors if any can be checked in submit sh o lt job_id gt amp submit sh e lt job_id gt Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 11 8 Optimization Compile using fast option Example CC fast test cpp mpCC fast lmpi test cpp 9 Available Software 9 1 Amber 8 Installed Path opt amber8 exe http amber scripps edu 9 2 Gamess Installed Path opt soft gamess_parallel gamess http www msg ameslab gov GAMESS GAMESS html 9 3 Gromacs Installed Path opt soft gromacs sparc sun solaris2 8 ultrasparc3 bin http www gromacs org 10 Frequently Asked Questions FAQs 10 1 When I run a MPI job using mprun np x command it gives the error mprun missing environment variab
12. sage timeout while launching a MPI job and it asks me to use t flag with mprun Kindly report this error to us with time amp date and any other output messages you may have 10 7 I am a beginner and want to learn more about UNIX in general Can you please give some references There are many good tutorials available on internet for beginners e g http www ee surrey ac uk Teaching Un1x You can also refer to books such as UNIX Concepts amp Applications by Sumitabha Das 10 8 Whenever delete multiple files using wild card say by rm need to answer y for each file Please use rm for the purpose Be careful as it would delete all the files in your current working directory 10 9 don t like the default executable name i e a out How can change it Please use o flag to compiler with the executable name e g CC my_program cpp o my_program exe This will compile the program and executable will be formed with the name my_program exe 10 10 Is there any available debugger S gt dbx 10 11 am unable to telnet ftp ssh sftp to any other machine from the Supercomputer We have denied the permission to make a connection from the supercomputer to any other machine You are only allowed to make connection to supercomputer from any other machines but not the other way Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 14 10 12 When I open a file using vi edito
13. scenssens 15 Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 4 1 Overview This manual is provided for the users of SCFBio as the primary reference The manual covers the supercomputer configuration available software and hardware access methods and user support Suggestions for additions or corrections to this manual may please be directed to biogrid scfbio itd org 2 The Facility The facility currently hosts a 70 processor 2GB shared memory dual CPU Sun 280R 900 MHz 64 bit Ultra Sparc HI Cu cluster over a dual network of 1 Gbps amp 100 Mbps switched Ethernets Sun Grid Engine and HPC Cluster Runtime Environment CRE software together provide a single window for parallel job creation load balancing capabilities across the nodes queue functionality and share entitlement 3 Getting Started 3 1 Logging into the remote machine Interactive access to the remote machine is obtained through the ssh command S gt ssh userfgaa bb cc dd 3 2 Logging out of the remote machine To logoff the remote machine enter exit and a return S gt exit 3 3 Login profiles and shells At login the user s default shell is bash We have setup your PATH and a few important aliases in etc profile Though if you want to customize something to your taste please make corresponding entries in bash_profile Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 5 4 Mani
14. sent working directory pwd returns the name of the current working directory It simply tells you the current directory Remove files rm removes each file in a list from a directory By default option i to rm inquires whether each file should be removed or not Option r causes rm to delete a directory along with any files or directories in it S gt rm filename Remove directory rmdir removes an empty directory from the current directory gt rmdir directoryname removes the subdirectory named directoryname if it is empty of files To remove a directory and all files in that directory either remove the files first and then remove the directory or use the rm r option described above Listing files and directories Is lists the files in the current directory or the directory named as an argument There are many options Is a directory lists all files including files whose names start with a period Is c directory lists files by date of creation Is 1 directory lists files in long form links owner size date and time of last change Supercomputing Facility for Bioinformatics amp Computational Biology User Manual 1 0 7 Is p directory subdirectories are indicated by Is r directory reverses the listing order ls s directory gives the sizes of files in blocks Is C directory lists files in columns using full screen width Is R directory recursively lists files in the current d
Download Pdf Manuals
Related Search