We recommend working through Software Carpentry workshops to learn basic shell programming skills: https://swcarpentry.github.io/shell-novice/
The basic building block of a cluster is the node. There’s two types: 1) the head node (where you login to from your personal computer) and 2) the compute node (all business, where everything gets done). If you need to run something, you log onto the head node and from there, make contact with the compute nodes via the job scheduler.
Using the scheduler
- What is the job scheduler? Basically, it’s just a computer application for controlling the background execution of programs (i.e. the batch processing). Unless you request an interactive node, you won’t have direct access to all of Hyak’s computational resources. Instead, you pass your directions to the job scheduler and it will put it in the queue.
- There’s a ton of different job schedulers, which one do we use? Ikt uses the portable batch system (PBS) which was developed by NASA. **Mox (Next-Gen Hyak) uses the Slurm scheduler (see the documentation)**.
- Okay, so how to I use the job scheduler? You have to write a job script that you submit using the “qsub” command.
- What does this script have to look like? It totally depends on what you want to do! There’s some standard PBS commands that need to come at the start of every job script. The Hyak wiki has some great directions. It also has some example scripts for more complicated job submissions.
Useful PBS resources: