Multithreading

Published

December 16, 2025

This example applies multithreading in Python.

Be aware that the Python interpreter process is constrained by the Global Interpreter Lock (GIL) and therefore can only use the resources made available to that process (generally, 1 logical core). Therefore, this multithreading example is inherently concurrent because of the GIL constraint. Although code is not executed in parallel, multithreading in Python can still be beneficial when, for example:

you are waiting for I/O operations to complete and want to compute in the meantime
you wish to prioritize the order of execution within your python script

Slurm batch submission script:

multithread.slurm

#!/bin/bash
#SBATCH --job-name=multithread_1n_8t
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1  
#SBATCH --time=00:10:00
#SBATCH --output=%x_output.log   # Auto-generated log file name (x = job-name)
#SBATCH --error=%x_error.log    # Auto-generated error file name (x = job-name)

# Load in Software
module load 2024 
module load Python/3.12.3-GCCcore-13.3.0

# Move to scratch space and copy the submit directory to the scratch space
cd "$TMPDIR"
cp -r $SLURM_SUBMIT_DIR/* .

echo "My scratch space directory: $TMPDIR"
echo "My slurm submission directory: $SLURM_SUBMIT_DIR"
echo

# Submit the program
python -u multithread.py --threads 8 --message "Running ..."

Python script

multithread.py

import threading
import time
import argparse

def worker(thread_id, message):
    """Worker function that prints a message and simulates work"""
    print(f"Thread {thread_id} says: {message}")
    time.sleep(5)  # Simulate some work
    print(f"Thread {thread_id} is done")

def main():
    parser = argparse.ArgumentParser(description="Multithreading example in SLURM")
    parser.add_argument("--threads", type=int, default=8, help="Number of threads to use")
    parser.add_argument("--message", type=str, default="Hello from thread (default)", help="Message for each thread")
    args = parser.parse_args()

    threads = []
    for i in range(args.threads):
        t = threading.Thread(target=worker, args=(i, args.message))
        threads.append(t)
        t.start()

    for t in threads:
        t.join()

    print("\nAll threads finished execution")

if __name__ == "__main__":
    main()

Running the script

Assuming you are inside your slurm submission directory:

sbatch multithread.slurm