Phylogenetics: Bioinformatics Cluster
|EEB 5349: Phylogenetics|
|The goal of this lab exercise is to show you how to log into the Bioinformatics Facility computer cluster and perform a basic PAUP* analysis.|
Using the UConn Bioinformatics Facility cluster
The Bioinformatics Facility is part of the UConn Biotechnology/Bioservices Center, which is located near the Up-N-Atom Cafe in the lower level of the Biology/Physics building. Jeff Lary maintains a 17-node Dell Linux cluster that can be used by UConn graduate students and faculty to conduct bioinformatics-related research (sequence analysis, biological database searches, phylogenetics, molecular evolution). You by now have an account on the cluster, and today you will learn how to start analyses remotely (i.e. from your laptop), check on their status, and download the results when your analysis is finished.
Obtaining the necessary communications software
You will be using a couple of simple (and free) programs to communicate with the head node of the cluster.
If you use MacOS 10.x...
The program ssh will allow you to communicate with the cluster using a protocol known as SSH (Secure Shell) that encrypts everything sent over the internet. You will use ssh to send commands to the cluster and see the output generated.
The other program you will use is called scp. It allows you to transfer files back and forth using the Secure Copy Protocol. If you find yourself wanting a more user-friendly SCP client, check out Cyberduck.
Start by opening the Terminal application, which you can find in the Applications/Utilities folder on your hard drive. Using the Terminal program, you can connect to the cluster with the following command:
where username should be replaced by your username on the cluster.
You may wish to install iTerm2, which is a terminal program that makes some things easier than Terminal, but the built-in Terminal will work just fine.
If you want to avoid having to type everytime you want to connect to the cluster, create an alias for this command as follows (in Terminal, but do this before you connect to the cluster):
cat - >> .bash_profile alias cluster="ssh email@example.com"
Ctrl-d The cat command copies what you type (the - means take input from the user) and appends (the >> part) to the file .bash_profile. Be sure to replace username with your actual username! The Ctrl-d part tells the cat command you are finished typing. After you are done, type the following to cause your new .bash_profile file to be reloaded (it is normally loaded only once when you start Terminal):
(Note: there are supposed to be two periods separated by a space at the beginning of this command - the first period causes the file .bash_profile to be executed.)
If you use Windows...
The program PuTTY will allow you to communicate with the cluster using a protocol known as SSH (Secure Shell) that encrypts everything sent over the internet. You will use PuTTY to send commands to the cluster and see the output generated. In the old days, a protocol known as Telnet was used for this purpose, but it is no longer used because it did not encrypt anything, making it easy for someone with access to the network to see your username and password in plain text.
The other program you will use is called PSFTP. It allows you to transfer files back and forth using SFTP (Secure File Transfer Protocol). It replaces the old protocol (FTP) that, like Telnet, sent usernames and passwords unencrypted across the network. If you find yourself wanting a more user-friendly SFTP client, check out FileZilla or WinSCP.
Double-click the PuTTY icon on your desktop to start the program. In the Host Name (or IP address) box, type bbcxsrv1.biotech.uconn.edu. Now type Bioinformatics cluster into the Saved Sessions box and press the Save button. This will save having to type the computer's name each time you want to connect. Now click the Open button to start a session. The first time you connect, you will get a PuTTY Security Alert. Just press the Yes button to close this dialog.
Now you should see the following prompt:
Type in your username and press Enter. Now you should see the password prompt:
Type in your password and press Enter. If all goes well, you should see something like this:
Welcome to Darwin! [bbcxsrv1:~] plewis%
except that your username should appear instead of mine (plewis).
Learning enough UNIX to get around
I'm presuming that you do not know a lot of UNIX commands, but even if you are already a UNIX guru, please complete this section anyway because otherwise you will fail to create some files you will need later.
The cluster comprises MacIntosh G5 computers running MacOSX, but MacOSX is essentially a UNIX operating system with a very nice user interface. But today you will not be using the nice user interface! Instead, you will be communicating using the UNIX command console, so the first step is to learn a few important UNIX commands.
ls command: finding out what is in the present working directory
The ls command lists the files in the present working directory. Try typing just
If you need more details about files than you see here, type
instead. This version provides information about file permissions, ownership, size, and last modification date.
pwd command: finding out what directory you are in
shows you the full path of the present working directory. The path shown should end with your username, indicating that you are currently in your home directory.
mkdir command: creating a new directory
Typing the following command will create a new directory named pauprun in your home directory:
Use the ls command now to make sure a directory of that name was indeed created.
cd command: leaving the nest and returning home again
The cd command lets you change the present working directory. To move into the newly-created pauprun directory, type
You can always go back to your home directory (no matter how lost you get!) by typing just cd by itself
If you want to go down one directory level (say from pauprun back down to your home directory), you can specify the parent directory using two dots:
Creating run.nex using the pico editor
One way to create a new file, or edit one that already exists, is to use the pico editor. You will now use pico to create a run.nex file containing a paup block. You will later execute this file in PAUP* to perform an analysis.
First use the pwd command to see where you are, then use cd to go into the pauprun directory if you are not already there. Type
This will open the pico editor, and it should say [ New File ] at the bottom of the window to indicate that the run.nex file does not already exist. Note the menu of the commands along the bottom two rows. Each of these commands is invoked using the Ctrl key with the letter indicated. Thus, ^X Exit indicates that you can use the Ctrl key in combination with the letter X to exit pico.
For now, type the following into the editor:
#nexus begin paup; log file=algae.output.txt start replace flush; execute algae.nex; set criterion=likelihood autoclose; lset nst=2 basefreq=estimate tratio=estimate rates=gamma shape=estimate; hsearch swap=none start=stepwise addseq=random nrep=1; lset basefreq=previous tratio=previous shape=previous; hsearch swap=tbr start=1; savetrees file=algae.ml.tre brlens; log stop; quit; end;
Once you have entered everything, use ^X to exit. Pico will ask if you want to save the modified buffer, at which point you should press the Y key to answer yes. Pico will now ask you whether you want to use the file name run.nex; this time just press Enter to accept. Pico should now exit and you can use cat to look at the contents of the file you just created:
Create the gopaup file
Now use pico to create a second file named gopaup in your home directory (the parent directory of the pauprun directory). Thie file should contain this text:
#$ -o junk.txt -j y cd $HOME/pauprun paup -n run.nex
(Windows users only) Using PSFTP to upload the algae.nex data file
Download the file algae.nex from here and save it on your hard drive. Hint: you may need to hold down the Shift key when you click the link so that your browser saves the file (rather than displaying its contents).
Make sure that algae.nex is in the same place as the PSFTP program, then start PSFTP by double-clicking it.
PSFTP should say something like this:
psftp: no hostname specified; use "open host.name" to connect
To open a connection to the cluster, type
then supply your username and password when prompted.
To upload algae.nex to the cluster, type
If you do not see any error messages, then you can assume that the transfer worked. Type
to exit the PSFTP program.
(Mac users only) Using Fugu or scp to upload the algae.nex data file
Download the file algae.nex from here and save it on your hard drive. Hint: you may need to use the option key when you click the link so that your browser saves the file (rather than displaying its contents).
You have two options for uploading files to the server. Fugu is the easiest to use, but will require downloading and installing the program Fugu on your computer. Use the other option, scp, only if you are a masochist or are otherwise averse to installing Fugu on your computer.
Using Fugu to upload (or download) files to the cluster
I assume that you have downloaded and installed Fugu on your computer. Start Fugu and type bbcxsrv1.biotech.uconn.edu in the "Connect to:" box, and type your username in the "Username" box. At this point, the Fugu main window should look like the image on the left (click to enlarge). After supplying your password, you will see a window showing local files (i.e. files on your Mac) on the left and remote files (i.e. files on the cluster) on the right. You can simply drag files from the left to the right, or vice versa. You can even drag files from a Finder window onto the remote pane.
Using scp to upload (or download) files to the cluster
Open the Terminal application and navigate to where you saved the file. If you saved it on the desktop, you can go there by typing cd Desktop.
Type the following to upload algae.nex to the cluster:
scp algae.nex firstname.lastname@example.org:
where username should be replaced by your own user name on the cluster. (Don't forget the colon on the very end of the line!)
A few more UNIX commands
You have now transfered a large file (algae.nex) to the cluster, but it is not in the right place. The algae.nex file should be in your home directory, whereas the run.nex file is in the pauprun directory. The run.nex file contains a line containing the command execute algae.nex, which means that algae.nex should also be located in the pauprun directory. Use the following commands to ensure that (1) you are in your home directory, and (2) algae.nex is also in your home directory:
cd $HOME ls algae.*
Note the use of a wildcard character (*) in the ls command. This will show you only files that begin with the letters algae followed by a period and any number of other non-whitespace characters. The $HOME is a predefined shell variable that will be relaced with your home directory. It is not necessary in this case - typing cd all by itself would take you to your home directory - but the $HOME variable is good to know about (especially for use in scripts).
mv command: moving or renaming a file
Now use the mv command to move algae.nex to the directory pauprun:
mv algae.nex pauprun
The mv command takes two arguments. The first argument is the name of the directory or file you want to move, whereas the second argument is the destination. The destination could be either a directory (which is true in this case) or a file name. If the directory pauprun did not already exist, mv would have interpreted this as a request to rename algae.nex to the file name pauprun! So, be aware that mv can rename files as well as move them.
cp command: copying a file
The cp command copies files. It leaves the original file in place and makes a copy elsehwere. You could have used this command to get a copy of algae.nex into the directory pauprun:
cp algae.nex pauprun
This would have left the original in your home directory, and made a duplicate of this file in the directory pauprun.
rm command: cleaning up
The rm command removes files. If you had used the cp command to copy algae.nex into the pauprun directory, you could remove the original file using these commands:
cd rm algae.nex
The first cd command just ensures that the copy you are removing will be the one in your home directory (typing cd by itself acts the same as typing cd $HOME). If it bothers you that the system always asks your permission before deleting a file, you can force the issue using the -f option (but just keep in mind that this is more dangerous):
rm -f algae.nex
To delete an entire directory (don't try this now!), you can add the -r flag, which means to recursively apply the remove command to everything in every subdirectory:
rm -rf pauprun
The above command would remove everything in the pauprun directory (without asking!), and then remove the pauprun directory itself. I want to stress that this is a particularly dangerous command, so make sure you are not weary or distracted when you use it! Unlike the Windows or Mac graphical user interface, files deleted using rm are not moved first to the Recycle Bin or Trash, they are just gone. There is no undo for the rm command.
Starting a PAUP* analysis
If you've been following the directions in sequence, you now have two files (algae.nex and run.nex) in your $HOME/pauprun directory on the cluster, whereas the gopaup file should be in $HOME. Use the cd command to make sure you are in your home directory, then the cat command to look at the contents of the gopaup file you created earlier. You should see this:
#$ -o junk.txt -j y cd $HOME/pauprun paup -n run.nex
This file will be used by software called the Sun Grid Engine (SGE for short) to start your run. SGE provides a command called qsub that you will use to submit your analysis. SGE will then look for a node (i.e. machine) in the cluster that is currently not being used (or is being used to a lesser extent than other nodes) and will start your analysis on that node. This saves you the effort of looking amongst all 17 nodes in the cluster for one that is not busy.
Here is an explanation of each of the lines in gopaup:
- Lines beginning with the two characters #$ are interpreted as commands by SGE itself. In this case, the command tells SGE to send any output from the program to a file named junk.txt and the -j y part says to append any error output to this as well (the j stands for join and the y for yes)
- The second line is simply a cd command that changes the present working directory to the pauprun directory you created earlier. This will ensure that anything saved by PAUP* ends up in this directory rather than in your home directory. Note that $HOME is like a macro that will be expanded to the full path to your home directory.
- The third and last line simply starts up PAUP* and executes the run.nex file. The -n flag tells PAUP* that no human is going to be listening or answering questions, so it should just use default answers to any questions it needs to ask during the run.
Submitting a job using qsub
Now you are ready to start the analysis. Make sure you are in your home directory, then type
Checking status using qstat
You can see if your run is still going using the qstat command:
If it is running, you will see an entry containing gopaup and the state will be r, for running. Here is what it looked like for me (I've omitted the rightmost part):
job-ID prior name user state submit/start at queue ----------------------------------------------------------------------------------------------- 5540 0.55500 gopaup plewis r 02/18/2007 13:38:47 email@example.com 5535 0.55500 bskinkultr jockusch r 02/16/2007 16:18:49 firstname.lastname@example.org 5525 0.55500 mb.sh plapierre r 02/15/2007 10:46:17 email@example.com 5433 0.55500 mb.sh plapierre r 02/08/2007 18:40:50 firstname.lastname@example.org 5539 0.55500 bskinkultr jockusch r 02/18/2007 12:22:55 email@example.com
My run is listed first, and is currently running on node 3 of the cluster.
Killing a job using qdel
Sometimes it is clear that an analysis is not going to do what you wanted it to. Suppose that just after you press the Enter key to start an analysis, you realize that you forgot to put in a savetrees command in your paup block (so in the end you will not be able to see the results of the search). In such situations, you really want to just kill the job, fix the problem, and then start it up again. Use the qdel command for this. Note that in the output of the qstat command above, my run had a job-ID equal to 5540. I could kill the job like this:
SGE will say that it has scheduled the job for deletion, but in practice it kills it almost instantaneously in my experience. Be sure to delete any output files that have already been created before starting your run over again.
While PAUP* is running
While PAUP* is running, you can use cat to look at the log file:
cd pauprun cat algae.output.txt
(Windows users only) Using PSFTP to download the log file and the tree file
When PAUP* finishes, qstat will no longer list your process. At this point, you need to use PSFTP to get the log and tree files that were saved back to your local computer. Start up PSFTP and type
Note that both the log file (algae.output.txt) and the tree file (algae.ml.tre) are in the pauprun directory. PSFTP dropped you in your home directory, but you can tell PSFTP to change to the pauprun directory in the same way you tell UNIX that you want to change directories:
You can likewise type ls in PSFTP to get a listing of files. Do this now to make sure you see the two files you want to download. Now, use the get command to download the files:
get algae.ml.tre get algae.output.txt
Finally, close PSFTP using any of the following commands: quit, exit, bye.
(Mac users only) Using scp to download the log file and the tree file
When PAUP* finishes, qstat will no longer list your process. At this point, you need to use scp to get the log and tree files that were saved back to your local computer. Start the Terminal app on your Mac and type the following (being careful to separate the final dot character from everything else by a blank space):
scp firstname.lastname@example.org:pauprun/algae.output.txt . scp email@example.com:pauprun/algae.ml.tre .
Again, be sure to replace username with your own user name on the cluster. These commands copy the files algae.output.txt and algae.ml.tre to your current directory (this is what the single dot at the end of each line stands for).
Why both junk.txt and algae.output.txt?
In your home directory, SGE saved the output that PAUP* normally sends to the console to a file named junk.txt (we specified that it should do this in the gopaup file). I had you name this file junk.txt because you will not need this file after the run: the log command in your paup block ends up saving the same output in the file algae.output.txt. Why did we tell PAUP* to start a log file if SGE was going to save the output anyway? The main reason is that you can view the log file during the run, but you cannot view junk.txt until the run is finished. There will come a day when you have a PAUP* run that has been going for several days and want to know whether it is 10% or 90% finished. At this point you will appreciate being able to view the output file!
Delete junk.txt using the rm command
Because you do not need junk.txt, delete it using the rm command (the -f stands for force; i.e. don't ask if it is ok, just do it!):
cd rm -f junk.txt
You also no longer need the log and tree files because you downloaded them to your local computer using PSFTP:
cd pauprun rm -f algae.ml.tre rm -f algae.output.txt
It is a good idea to delete files you no longer need for two reasons:
- you will later wonder whether you downloaded those files to your local machine and will have to spend time making sure you actually have saved the results locally
- our cluster only has so much disk space, and thus it is just not possible for everyone to keep every file they ever created
Tips and tricks
Here are some miscellaneous tips and tricks to make your life easier when communicating with the cluster.
Command completion using the tab key
You can often get away with only typing the first few letters of a filename; try pressing the Tab key after the first few letters and the command interpreter will try to complete the thought. For example, cd into the pauprun directory, then type
If algae.nex is the only file in the directory in which the first three letters are alg, then the command interpreter will type in the rest of the file name for you.
I've already mentioned this tip, but it bears repeating. When using most UNIX commands that accept filenames (e.g. ls, rm, mv, cp), you can place an asterisk inside the filename to stand in for any number of letters. So
will produce output like this
algae.ml.tre algae.nex algae.output.txt
If you want to learn more options for any of the UNIX commands, you can use the man command to see the manual for that command. For example, here's how to see the manual describing the ls command:
It is important to know how to escape from a man page! The way to get out is to type the letter q. You can page down using Ctrl-f, page up through a man page using Ctrl-b, go to the end using Shift-g and return to the very beginning using 1,Shift-g (that is, type a 1, release it, then type Shift-g). You can also move line by line in a man page using the down and up arrows, and page by page using the PgUp and PgDn keys.