1. CRC Documentation
2. …
3. CRC Documentation Home
4. Research Computing
5. CRC Common Documents
6. General Software Tools FAQ
7. Using MATLAB on the Shared Research Computing Systems

Page History

Table of Contents

Table of Contents

Introduction

There are several ways in which to submit Matlab jobs to a cluster.  This document will cover the various ways to run Matlab compute jobs on the Shared Research Compute clusters, which will include using the Parallel Computing Toolbox (PCT) and the Matlab Distributed Compute Engine (MDCE) to submit many independent tasks and to submit a single task that has parallel components.   Examples are included.

Definitions

Task Parallel - Multiple independent iterations within your workflow.

Data Parallel - Single program working on a problem across multiple processors.

...

...

...

...

Matlab Task - Individual Matlab compute job.

Matlab Job - Submission from within the Matlab GUI that can contain one or more tasks.

Job - Job submitted via the PBS job scheduler (also called PBS Job).

Interactive Jobs

take from old FAQ

Running Jobs with PBS qsub

Good for jobs that are single processor jobs and do not encounter any toolbox license issues.  Take from old FAQ.

Using MDCE for Task Parallel and Data Parallel Jobs

Using MDCE will allow you to submit multiple jobs with a single job submission (Task Parallel) or submit a single task that is a multiprocessor (and possibly multinode) job.  In order to run this type of job you must first configure Matlab for this type of job submission by following these steps;

Configuring Matlab

1.   In your home directory create the MdcsDataLocation/sugar subdirectory.

mkdir -p ~/MdcsDataLocation/sugar

2.  Load the Matlab 2011a environment:

module load matlab/2011a

3.  Run Matlab on the login node:

matlab

4.  In Matlab, add the ddd folder to your Matlab path so that Matlab will be able to find the scripts necessary to submit and schedule jobs.

1. Click on File and then Set Path
2. Click the Add Folder button
3. Specify the following folder:
   /opt/apps/matlab/2011a-scripts

   Note
   title Error saving pathdef.m
   If Matlab reports that it is unable to save pathdef.m in your current folder, then follow the prompts to select your home folder before saving the file.

5.  Import the cluster configuration for sugar

1. Click on Parallel and then Manage Configurations
2. Click on File and then Import
3. Navigate to /opt/apps/matlab/2011a-scripts and select the configuration for the system you are using, such as sugar.mat, davinci.mat, stic.mat, and so forth.
4. Select the configuration for the system you are using and click on Start Validation
   1. All four stages should pass:  Find Resources, Distributed Job, Parallel Job, Matlabpool

      Note
      title Validation will fail on a busy cluster
      If the cluster is busy such that a job submission must wait before it will run then the validation steps will fail.

If all validation stages succeed, then you are ready to run jobs with MDCE.

Submitting Task Parallel Jobs

The following is an example of a Task Parallel job.  The task-parallel example code, frontDemo, calculates the risk and return based on historical data from a collection of stock prices. The core of the code, calcFrontier, minimizes the equations for a set of returns. In order to parallelize the code, the for loop is converted into a parfor loop.  View the m code here.

To submit the job, copy submitParJobToCluster.m into your working directory, make the necessary modifications for your job environment, and then run the code from within Matlab.  This will submit the job.   The code can be downloaded from here.  An explanation of the code follows:

function job = submitParJobToCluster()

if nargin==0, sz = 3; end

% Set the walltime to 5 minutes
ClusterInfo.setWallTime('00:05:00');                  % change this to the actual walltime that you need.
ClusterInfo.setEmailAddress('YourEmailAddressHere')   % include your email address here.

job = batch(@frontDemo,2,{},'Matlabpool',sz,'CaptureDiary', true);   % this submits the frontDemo.m job.
           % @frontDemo is the function to submit.
           % 2 is the number of output arguments
           % {} is an empty array of input arguments
           % sz is the number of processor cores (workers)
           % CaptureDiary is set to true
job.wait                                                             % the Matlab GUI will pause here until the job finishes
try
   error(job.Task.ErrorMessage)
   out = job.getAllOutputArguments();                                % get all output arguments from the completed job
   r = out{1}; v = out{2};
   plot(r,v)                                                         % plot the results
   job.diary
catch ME
   error(ME.message)
end

if nargout==0
   job.destroy
   clear job
end

Submitting Data Parallel Jobs