Skill: Job scheduling and resource management
Competency pathway
Understanding of job schedulers and resource management to allocate compute resources effectively. This includes writing job submission scripts, understanding queue policies, optimising resource requests, and supporting fair and efficient use of shared HPC infrastructure.
Related skills
Computational tools
| Name | Description |
|---|---|
| Cgroups | A Linux kernel feature that limits and accounts for the resource usage of a collection of processes. |
| HTCondor | A specialised workload management system for compute-intensive jobs. |
| Load Sharing Facility (LSF) | A workload management and job scheduling software for distributed computing environments. |
| Oracle Grid Engine | Oracle Grid Engine, previously known as Sun Grid Engine (SGE), is a grid computing computer cluster software system (otherwise known as a batch-queuing system) |
| Portable Batch System (PBS) | A computer software that performs job scheduling in high-performance computing. |
| Simple Linux Utility for Resource Management (SLURM) | An open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. |
Methodologies
| Name | Description |
|---|---|
| Load balancing | Distributing work evenly across processors or nodes to maximise resource utilisation and minimise idle time. Workload distribution is usually decided in the algorithm design but can appear in HPC runtime tuning. |
| Job scheduling | Controlling unattended background program execution of jobs (commonly called batch scheduling). |