This document is still under development and is being written/edited by RCSG and MathWorks staff. |

There are several ways in which to submit MATLAB jobs to a cluster. This document will cover the various ways to run MATLAB compute jobs on the Shared Research Compute clusters, which will include using the Parallel Computing Toolbox (PCT) and the MATLAB Distributed Compute Server (MDCS) to submit many independent tasks as well as to submit a single task that has parallel components. Examples are included.

**Task Parallel Application** - The same application that runs independently on several nodes, possibly with different input parameters. There is no communication, shared data, or synchronization points between the nodes.

**Data Parallel** **Application** - The same application that runs on several labs simultaneously, with communication, shared data, or synchronization points between the labs.

**Lab** - A MATLAB worker in a multicore (Data Parallel) job. One lab is assigned to one worker (core). Thus, a job with eight labs has eight processor cores allocated to it and will have eight workers each working together as peers.

**MDCS** - Matlab Distributed Compute Server. This is a component of MATLAB that allows our clusters to run MATLAB jobs that exceed the size of a single compute node (multinode parallel jobs). It also allows jobs to run even if there are not enough toolbox licenses available for a particular toolbox, so long as the university owns at least one license for the particular toolbox.

**PCT** - Parallel Computing Toolbox.

**MATLAB Task** - One segment of a job to be evaluated by a worker.

**MATLAB Job** - The complete large-scale operation to perform in MATLAB, composed of a set of tasks.

**MATLAB Worker** - The MATLAB session that performs the task computations. If a job needs eight processor cores, then it must have eight workers.

**Job** - Job submitted via the PBS job scheduler (also called PBS Job).

TBA take from old FAQ

The recommended method of submitting Task Parallel jobs is to use PCT as described below. However, if you need to submit one or more **single core jobs**, and are not encountering problems with the lack of specific toolbox licenses, then you might wish to submit your jobs directly with PBS as described in the following steps.

Multicore (parallel) jobs are not supported using this submission method. If you need to submit a parallel job then please use PCT . |

1. Build a MATLAB .m code file

Include all of the MATLAB commands that you need to execute in a MATLAB .m file, such as *sample.m* and place it somewhere in your home directory or subdirectory. The creation and contents of a .m file are beyond the scope of this document. Consult the MATLAB documentation for information on .m files.

2. Create a PBS batch script

You will need to execute Matlab from within a PBS batch script, such as *sample.pbs* as follows. In this example, this file is saved in the same directory as *sample.m*.

#PBS -N JobNameHere #PBS -l nodes=1:ppn=1,walltime=24:00:00 #PBS -W x=NACCESSPOLICY:SINGLEJOB #PBS -M YourEmailAddressHere #PBS -V #PBS -m abe # echo "I ran on:" cd $PBS_O_WORKDIR cat $PBS_NODEFILE # matlab -r sample |

In this example the *sample.pbs* script calls *matlab* with the -r option followed by the MATLAB script name that was created in step #1. Leave off the trailing .m from the script name when callingMATLAB this way.

For more information about PBS job scripts, please see our FAQ.

3. Submit the Job

After you have created *sample.m* and *sample.pbs*, go to the directory where *sample.pbs* resides, load the MATLAB module (if you have not already done so) and submit the job to the job scheduler:

[user@host ~]$ module load matlab/2009a [user@host ~]$ qsub ./sample.pbs |

This type of job submission will compete with all other jobs on the system for run time. If no processors are available then the job submission will wait in the queue until it is able to run. |

This type of job may fail due to license checkout errors. MATLAB shares a site license with the rest of the campus. If your job is using a toolkit for which there are no licenses available for checkout, the job will fail when it starts to run. You will need to use PCT to avoid this problem. |

In order to submit multiple Matlab batch jobs, simply repeat this section for each job.

PCT provides an API that allows you to submit a job that has multiple independent tasks (Task Parallel) or submit a job that has a single task that is a multiprocessor, and possibly multinode, task (Data Parallel). In order to run this type of job you must first configure MATLAB for this type of job submission by following these steps;

These steps need to be performed only once. Subsequent sessions of MATLAB do not need to be configured. |

1. In your home directory create the MdcsDataLocation subdirectory.

mkdir ~/MdcsDataLocation |

2. Load the MATLAB 2011a environment:

module load matlab/2011a |

3. Run MATLAB on the login node:

matlab |

4. In MATLAB, add the */opt/apps/matlab/2011a-scripts* folder to your MATLAB path so that MATLAB will be able to find the scripts necessary to submit and schedule jobs.

- Click
**File > Set Path** - Click
**Add Folder** - Specify the following folder:

/opt/apps/matlab/2011a-scriptsIf MATLAB reports that it is unable to save

*pathdef.m*in your current folder, then follow the prompts to select your home folder before saving the file.

5. Import the cluster configuration for the cluster you are running on:

- On the MATLAB Desktop menu bar, click
**Parallel > Manage Configurations**. - Click
**File > Import** - In the Import Configuration dialog box, browse to find the MAT-file for the configuration that you want to import. Navigate to
*/opt/apps/matlab/2011a-scripts*and select the configuration for the system you are using, such as*sugar.mat*,*davinci.mat*,*stic.mat*, and so forth. Select the file and click**Import**. - Select the configuration you have imported and click
**Start Validation**.- All four stages should pass: Find Resources, Distributed Job, Parallel Job, Matlabpool
If the cluster is busy such that a job submission must wait before it will run then the validation steps will fail.

- All four stages should pass: Find Resources, Distributed Job, Parallel Job, Matlabpool

If all validation stages succeed, then you are ready to submit jobs to MDCS.

The following is an example of a Task Parallel job. The task-parallel example code, *frontDemo*, calculates the risk and return based on historical data from a collection of stock prices. The core of the code, *calcFrontier*, minimizes the equations for a set of returns. In order to parallelize the code, the *for* loop is converted into a parfor loop with each iteration of the loop becoming its own independent task.

To submit the job, copy *submitParJobToCluster.m* into your working directory, make the necessary modifications for your job environment, and then run the code from within MATLAB. This will submit the job. An explanation of the code follows:

function job = submitParJobToCluster() if nargin==0, sz = 3; end % Set the walltime to 5 minutes ClusterInfo.setWallTime('00:05:00'); % change this to the actual walltime that you need. ClusterInfo.setEmailAddress('YourEmailAddressHere') % include your email address here. ClusterInfo.setUserDefinedOptions('-W x=NACCESSPOLICY:SINGLEJOB') % restrict access to the compute node to your job only. Node will not be shared. job = batch(@frontDemo,2,{},'Matlabpool',sz,'CaptureDiary', true); % this submits the frontDemo.m job. % @frontDemo is the function to submit. % 2 is the number of output arguments % {} is an empty array of input arguments % sz is the number of processor cores (workers) % CaptureDiary is set to true job.wait % the MATLAB GUI will pause here until the job finishes try error(job.Task.ErrorMessage) out = job.getAllOutputArguments(); % get all output arguments from the completed job r = out{1}; v = out{2}; plot(r,v) % plot the results job.diary catch ME error(ME.message) end if nargout==0 job.destroy clear job end |

Code

When you run this code within MATLAB, the *frontDemo* code will be submitted to the PBS job scheduler. Use the showq command from a cluster terminal window to look for your job in the job queue.

The above is only an example used to illustrate how to submit a job and retrieve the results from within the same MATLAB session. In most cases using |

The maximum number of workers per job submission is constrained by the queue policy on each cluster, with one worker per processor core. For example, Sugar will not accept more than 8 workers per submission. |

If you have one or more single core jobs to run then you would simply set the size parameter, |

For more information on the batch() command and all of its input arguments and how to use the diary, please see the MATLAB online help or the MathWorkshttp://www.mathworks.com/help/toolbox/distcomp/batch.html website. |

Be sure to destroy your job after the job has finished. |

The data-parallel example code calculates the area of pi under the curve. The non parallel version, *calcPiSerial*, calculates with a for loop, looping through discrete points. The parallel version, *calcPiSpmd*, uses the *spmd* construct to evaluate a part of the curve on each MATLAB instance. Each MATLAB instances uses its *labindex* (i.e. rank) to determine which portion of the curve to calculate. The calculations are then globally summed together and broadcasted back out. The code uses higher level routines, rather than lower level MPI calls. Once the summation has been calculated, it’s indexed into and communicated back to the local client MATLAB to calculate the total area.

To submit the job, copy submitSpmdJobToCluster.m into your working directory, make the necessary modifications for your job environment, and then run the code from within MATLAB. This will submit the job. An explanation of the code follows:

function job = submitSpmdJobToCluster(sz) if nargin==0, sz = 3; end % Set the walltime to 5 minutes ClusterInfo.setWallTime('00:05:00'); % change this to the actual walltime that you need. ClusterInfo.setEmailAddress('YourEmailAddressHere') % include your email address here. ClusterInfo.setUserDefinedOptions('-W x=NACCESSPOLICY:SINGLEJOB') % restrict access to the compute node to your job only. Node will not be shared. job = batch(@calcPiSpmd,1,{sz},'Matlabpool',sz); % this will submit the calicPiSpmd function % @calcPiSpmd is the function to submit. % 1 is the number of output arguments % {sz} is an array of input arguments % sz is the number of processor cores (workers) job.wait % the MATLAB GUI will pause here until the job finishes try error(job.Task.ErrorMessage) out = job.getAllOutputArguments(); % get all output arguments from the completed job p = out{1} catch ME error(ME.message) end if nargout==0 job.destroy clear job end |

When you run this code within MATLAB, the *calcPiSpmd* code will be submitted to the PBS job scheduler. Use the *showq* command from a cluster terminal window to look for your job in the job queue.

Code

The above is only an example used to illustrate how to submit a job and retrieve the results from within the same MATLAB session. In most cases using |

The maximum number of workers per job submission is constrained by the queue policy on each cluster, with one worker per processor core. For example, Sugar will not accept more than 8 workers per submission. |

For more information on the batch() command and all of its input arguments and how to use the diary, please see MATLAB's online help or the MathWorks website. |

Be sure to destroy your job after the job has finished. |

When you submit a job with *batch*, you will notice that each submission is labeled *Job1*, *Job2,* and so forth. Temporary directories associated with each job can be found in ~/MdcsDataLocation as the jobs are running. When *job.destroy* is called, these temporary directories are deleted. The above examples call *job.destroy*. If you close your MATLAB session before executing *job.destroy*, which is likely unless you are using the full example with *job.wait*, you will need to manually cleanup temporary directories in ~/MdcsDataLocation.

In order to run MATLAB code with a parallel component on your desktop locally, you must first start up a MATLAB Pool, as such:

matlabpool open local 8 |

where 8 is the number of MATLAB processes to attach to the job. At this point you will have access to eight MATLAB workers for use with parallel code, such as code with *parfor* loops, and so on.

This should not be more than |

After running the code, close the MATLAB Pool:

matlabpool close |

Calls to |