Because our clusters are highly complex environments, we do expect our users to acquire a basic set of skills before they get started. We understand that the range of computing experience among researchers can vary widely. In any cross-disciplinary field, there exists a need for introductory material to get researchers up to speed on fundamental concepts. We've gathered the following list of tutorials on varying computing subjects that we think will be of use.
At the least, a cluster user should understand:
- Basic file/directory manipulation under Linux
- Creating/Editing files under Linux
- Writing and submitting job scripts
- How to login to the clusters
- How to copy data in and out of the clusters
- Know whether your jobs are serial or parallel
All of the clusters that make up our shared computing resources run the Linux operating system. Using Linux is very similar to using Windows or MacOS on the desktop level, but that changes when it comes time to perform tasks remotely, as we do with the CRC clusters. In order to work on CRC clusters, users must use the command line interface (CLI).
- An Introduction to Linux (Cornell Virtual Workshop) provided by Cornell University's Center for Advanced Computing
- The Linux Documentation Project's list of guides Start with the Introduction to Linux and explore
- Unix/Linux Commands Cheat Sheet Quick reference to print and keep near the keyboard
- Unix/Linux Tutorial for Beginners A beginners guide to the Unix and Linux operating system. Eight simple tutorials which cover the basics of UNIX / Linux commands provided by the University of Surrey.
In order to submit compute jobs to the clusters, a basic understanding of shell scripting is required. Job submission scripts typically use Bash, but can employ more advance scripting techniques as well.
- TLDP's Advanced Bash Scripting Guide Another great resource from the Linux Documentation Project. Advanced users (CRC members included!) may still refer to this guide from time to time.
- Perl Tutorial A good starting point for learning the powerful Perl scripting language
- Free Intro to Perl ebook
- Perl Programming Documentation The official reference for Perl
- Python Tutorial Official tutorial for the Python programming language.
- Python for High Performance (Cornell Virtual Workshop)
Parallel Programming (MPI)
- HPC University Highly recommended tutorials and training materials related to various HPC topics.
- Cyber Infrastructure Tutor Highly recommended tutorial available at National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign. The NCSA tutorial web site requires free registration.
- What is MPI? (ANL) Description of MPI with links to tutorials and documentation from Argonne National Laboratory.
- Tutorial on MPI (ANL)
- Message Passing Interface (LLNL) Tutorial developed by Lawrence Livermore National Laboratory.
- MPI Tutorials (ANL)
- MPI Tutorials (LAM-MPI)
Parallel Programming (OpenMP)
- Totalview Tutorial on the basic functions of Totalview plus how to debug parallel programs. Tutorial developed by Lawrence Livermore National Laboratory.
- Profiling and Debugging (Cornell Virtual Workshop)