Running Mathematica Jobs

If you wan to run Mathematica on your local computer you can find information and an application form on this page.

This tutorial assumes that the user has a good working knowledge of Mathematica. This tutorial is not about how to use Mathematica, for more information about Mathematica please consult Wolfram Documentation Center.
In the following we show how to run Mathematica computation on the cluster using job scripts.

Running Mathematica with GUI
Running Mathematica in interactive mode
Basic Mathematica job script
Parallelism through multithreading (single node)
Parallelism through SubKernels (multiple nodes)

Running Mathematica with GUI

From users of Linux, MacOSX with XQuartz, or Windows with either Xming or Cygwin/X, Mathematica can be directly run on Sango with its GUI. The steps will be as bellow

1. Login into Deigo using ssh with the -X (or -Y on MacOSX) option

 $ ssh -X deigo

2. Load the latest version of Mathematica

 $ module load mathematica

3. Run mathematica(this example asks for 8 gigabytes of memory)

 $ srun -p short --mem=8g --x11=last --pty Mathematica

Running Mathematica in interactive mode

For users of PuttY or Teraterm on windows, or for users that want to use Mathematica without graphical interface (for example from a place where the network is slower than the campus internal network), the steps will be:

1. Login into Deigo using ssh

 $ ssh deigo

2. Set up your shell environment to use latest version of Mathematica

 $ module load mathematica

3. Run Mathematica (this example asks for 8 gigabytes of memory)

 $ srun -p short --mem=8g --pty math

Basic Mathematica job script

The following example job script, math_basic.slurm launches Mathematica then runs a computation script that computes the sum of A and B.

#!/bin/bash

#SBATCH --job-name=math_basic
#SBATCH --partition=compute
#SBATCH --mem=512m
#SBATCH --time=00:10:00
#SBATCH --ntasks=1
#SBATCH --input=none
#SBATCH --output=math_basic_%j.out
#SBATCH --error=math_basic_%j.err

module load mathematica
math -run < math_basic.m

The contents of the Mathematica computation script, math_basic.m, reads:

A = Sum[i, {i,1,100}]
B = Mean[{25, 36, 22, 16, 8, 42}]
Answer = A + B
Quit[];

To submit the computation to the cluster queue enter

 $ sbatch math_basic.slurm

The computation result will be available in the file math_basic_*.out

Parallelism through multithreading (single node)

Mathematica can exploit multiple cores to do parallel computation using the built in Parallel commands or by utilizing parallel API. Parallel Mathematica jobs are limited to one node, but can utilize all cores on the node if allocated. A parallel Mathematica script must specify the number of processors to be allocated. The following example script will request 8 cores to run a Mathematica script that compute Mersenne prime numbers.
Below is the slurm script math_par_mp.slurm to use for this example

#!/bin/bash
#SBATCH --partition=short
#SBATCH --job-name=math_par_mp
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --input=none
#SBATCH --output=math_par_mp_%j.out
#SBATCH --error=math_par_mp_%j.err

module load mathematica

math -run < math_par_mp.m

The contents of the Mersenne prime computation Mathematica script, math_par_mp.m, read as below

(*Limits Mathematica to requested resources*)
Unprotect[$ProcessorCount];$ProcessorCount = 8;
 
(*Prints the machine name that each kernel is running on*)
Print[ParallelEvaluate[$MachineName]];

(*Prints all Mersenne PRime numbers less than 2000*)
Print[Parallelize[Select[Range[2000],PrimeQ[2^#-1]&]]];

To submit the computation to the cluster queue enter

 $ sbatch math_par_mp.slurm

The computation result will be available in the file math_par_mp_*.out

Parallelism through SubKernels (multiple nodes)

To scale further computation, Mathematica can also exploit multiple cores on multiple nodes parallel computation using the built-in commands LaunchKernels and RemoteMachine to launch sub-kernel computation on different remote machines together with the Parallel command.
This allows you to exploit parallelism of the cluster without the single node limitation of the previous method.
Before starting using sub-kernels, because Mathematica executes each sub-kernel using a newly created login shell, only the environment variables set in your $HOME/.bashrc file are visible by the sub-kernels. Therefore as an exception here, the module load mathematica command must be done in your $HOME/.bashrc file, and does not need to be done in the slurm batch script. You can do it by appending the following code AT THE VERY END of your bashrc file

MHOSTNAME="$(/bin/uname -n)"
if [[ "${MHOSTNAME:0:5}" == "sango" ]]; then
  module load mathematica/12.0
fi

In the following example, prime number computation is distributed on several cores from different nodes. Precisely, the computation is distributed over 6 nodes where 4 cores are used in each node for a total of 24 cores used for parallel computation

#!/bin/bash

#SBATCH --partition=compute
#SBATCH --job-name=math_par_dis
#SBATCH --nodes=6
#SBATCH --mem-per-cpu=400m
#SBATCH --ntasks-per-node=4
#SBATCH --input=none
#SBATCH --output=math_par_dis_%j.out
#SBATCH --error=math_par_dis_%j.err

get_abs_filename() {
  echo "$(cd "$(dirname "$1")" && pwd)/$(basename "$1")"
}

### -------------- BEGIN USER INPUT -------------

## Put the full pathname of *** your Mathematica script here **
myscript="math_par_dis.m"

### -------------- END USER INPUT -------------


## Set up cores/nodes information
nlname="${WORK_SCRATCH}/nodelist_${SLURM_JOB_NAME}_${SLURM_JOB_ID}"
msname="${WORK_SCRATCH}/script_${SLURM_JOB_NAME}_${SLURM_JOB_ID}"
mfname="${WORK_SCRATCH}/machines_${SLURM_JOB_NAME}_${SLURM_JOB_ID}"
mathscript="$(get_abs_filename "${myscript}")"
mkdir -pv "${WORK_SCRATCH}" >&2
scontrol show hostnames $SLURM_NODELIST > "${nlname}"
for ((line=1; line<=${SLURM_JOB_NUM_NODES}; line++)); do
  for ((tN=1; tN<=${SLURM_NTASKS_PER_NODE}; tN++)); do
    head -$line "${nlname}" | tail -1 >> "${mfname}"
  done
done
cd "${WORK_SCRATCH}"
sed -e "s;MACHINES_LIST_FILE;${mfname};g" ${mathscript} > ${msname}

## Run the script
math -run < "${msname}"

## Clean-up
cd "${SLURM_SUBMIT_DIR}"
[[ -d "${WORK_SCRATCH}" ]] && /bin/rm -fvr "${WORK_SCRATCH}" >&2

where the Mathematica computation script math_par_dis.m consists of the following code. Note that in this example, it is important that you do not modify the line hosts=Import["MACHINES_LIST_FILE","List"].

(* configuration for starting remote kernels *)

Needs["SubKernels`RemoteKernels`"]

(* initialize the kernels on all machines defined in the host file *)

hosts=Import["MACHINES_LIST_FILE","List"]

(* on the master node initialize only one kernel less, since one is already running *)
imin=2;
imax=Length[hosts];
idelta=1;

Do[
  Print["starting Kernel: ",i," on ",hosts[[i]]];
  LaunchKernels[RemoteMachine[hosts[[i]]]];,
  {i,imin,imax,idelta}
]

(* actual calculation *)
primelist = ParallelTable[Prime[k], {k, 1, 2000}];
Print[primelist]

To submit the computation to the cluster queue enter

$ sbatch math_par_dis.slurm

The computation result will be available in the file math_par_dis_*.out