http://acesgrid.org/http://acesgrid.org/
Overview
The ACES clusters stands for Alliance for Computational Earth Science (ACES) ad I got access from [greg] by emailing him, and he provided access within 24 hours. This came with an email from the grid system itself with a link to their website with lots of useful information. This includes the times for office hours (Tuesdays from 11:30-1:30) and a list of software.
Information
Their website has some useful information, although it's not perfect. If you have any specific questions, you might find it there.
http://acesgrid.org/getting_started.html
Some of the information might be old. For example, there are not currently office hours and you should just email [greg] if you need help with something specific. If you sign up for the aces-support list, you will get some emails (although I imagine very infrequently) about this.
I'll just summarize a few things I am aware of.
Storage
Because we don't have our own Alm lab storage and back-up system, you are only allowed to have access to 1GB on your home drive. Although I think there is quite a lot of memory available on a scratch drive, but it is for short term storage and will be deleted automatically. So this might be a good option when you want to do something which requires a lot of computational power to finish processing something, then you can move it off and store else where. Look at the webpage for specifics.
Interactive computing
After normal login to the head node, ssh to a dedicated compute node with:
ssh qubert
or get a node through the queue interactively:
qsub -I -l nodes=1
The Queue system
The website has some examples of how to submit a job. Try to follow their examples.
From the login (head) node, you can find out about the queue system with the command:
qstat -q
And from the login (head) node, you can find out about the qsub command-line options and what they mean with (if you do this from an interactive node you will get a different manual):
man qsub
Note: You can not qsub when you are logged into an interactive node (qubert), it says "qsub: command not found". Instead, qsub from the head node upon logging in.
This script ran on aces by qsubbing the file below like this:
qsub multiple_scripts1-10
multiple_scripts1-10 is:
#!/bin/csh
#filename: pbs_script
#PBS -N pbs_script
#PBS -l nodes=1
#PBS -e stderr
#PBS -o stdout
# o Export all my environment variables to the job
#PBS -V
if ( -f /etc/profile.d/modules.csh ) then
source /etc/profile.d/modules.csh
endif
module load matlab
matlab -nojvm -r "run(0.2985,1.0,75.0,10000.0,600.0,25.0,0.1,'rates_0_1.csv'); exit;"
MatLAB
I was specifically looking for a new way to use Matlab since I was using it on beagle but that went down. This is how to get matlab working with Aces
http://acesgrid.org/matlab.html
However, if you want run interactively, you reverse a node:
qsub -I -l nodes=1
Then in the next terminal window, you login as you would normally with -X (maybe not as directed on the above webpage):
ssh -X aces
Then you ssh from aces onto the reserved node:
ssh -X reserved.node.name
This should work and bring up X11 (that's what I'm using):
module add matlab
matlab
QIIME
You can also get qiime to work, using the qsub command above to get interactive computing and use:
module add qiime
It's QIIME version 1.6 (I think) so it's got a few quirks that are different from the version 1.3 that was on beagle. You might need to change some variables names etc, but the QIIME documentation should be helpful for this.
Space
So I was able to make my own directory in /data/ and /scratch/, but it didn't exist before I made it:
mkdir /scratch/spacocha
mkdir /data/spacocha
I'm able to write to that folder, so I think it should work fine.
File transfer
Although I can make the /scratch/spacocha folder, I can't scp to it. Instead, I can scp big files, like fastq files, to /data/spacocha/ just fine.
Notes
I have been having trouble getting and interactive node using qsub for a few days. Seems like the cluster fills up occasionally making it difficult to get stuff done in a hurry.
Summary
In summary, this cluster might be good for a few specific tasks, but it's not good for long term storage, and it's geared towards the earth science crowd and modeling ocean circulation etc. It might have some functional capabilities that you could use (i.e. Matlab, QIIME), but be careful not to leave data on their scratch because it will be deleted.