MATLAB and Dexy

MATLAB is a programming language and environment which is widely used in many academic and technical fields. This blog post will cover the basics of running MATLAB code using Dexy. If you aren’t familiar with Dexy, then the Getting Started tutorial may help clarify some of the examples in this blog post.

Here are a few fun MATLAB examples to get us started.

Here’s some code which generates a 3d plot:

>> [X,Y] = meshgrid(-8:.5:8,-8:.5:8); % Create a mesh grid on [-8,8]*[-8,8].
>> R = sqrt(X.^2 + Y.^2) + eps; % + eps to avoid divide-by-zero error.
>> Z = sin(R)./R;
>> mesh(X,Y,Z)
>> print -dpng 'mesh.png'

And here’s the resulting plot:

Image may be NSFW.
Clik here to view.

MATLAB allows creating anonymous functions:

>> cs=4;
>> myfun = @(x) (x(1)-1).^4+cs*(x(2)-2).^2

myfun =

    @(x)(x(1)-1).^4+cs*(x(2)-2).^2

Here, we use this anonymous function to create another 3D plot:

>> [X,Y] = meshgrid(0:.1:3,0:0.1:6);
>> tic, for i=1:size(X,1), for j=1:size(X,2), F(i,j)=myfun([X(i,j) Y(i,j)]); end, end, toc
Elapsed time is 0.003296 seconds.
>> surfc(X,Y,F); xlabel('x_1');ylabel('x_2');zlabel('myfun(x_1,x_2)')
>> print -dpng 'mesh-with-function.png'

Image may be NSFW.
Clik here to view.

Here’s an example of solving an ODE representing a Van der Pol oscillator.

First, we define the dynamical system:

>> xdot=@(t,x) [x(2);(1-x(1)^2)*x(2)-x(1)];

Next, we solve it:

>> [t,x_t]=ode45(xdot,[0 20],[2 0]);

Finally we graph it:

>> plot(t,x_t);grid;xlabel('time');ylabel('x');
>> title('Van der Pol Equation');legend('x_1','x_2');
>> print -dpng 'van-der-pol.png';

Image may be NSFW.
Clik here to view.

The same system can also be modelled visually using Simulink (R):

>> open('vdp');
>> set_param('vdp/Scope','LimitDataPoints','off','SaveToWorkspace','on');
>> sim('vdp');
>> plot(ScopeData(:,1),ScopeData(:,[2 3]));grid;xlabel('Time (s)');ylabel('x');title('Van der Pol Simulation');
>> print -dpng 'van-der-pol-simulink.png';

This is the standard Van der Pol simulation (‘vdp’) included in MATLAB distributions:

Image may be NSFW.
Clik here to view.

The -s argument to print also allows us to render the visual model:

>> print -dpng -svdp 'van-der-pol-simulink-model.png'

Image may be NSFW.
Clik here to view.

>> close_system('vdp',0);

Running MATLAB Code in Dexy

Now that we’ve seen some examples, we’ll look in more detail at how we can use
Dexy to run MATLAB code and display the results.

Here’s a short MATLAB script showing two ways to solve A*x=b:

A = [10 -7 0; -3 2 6; 5 -1 5]
b=[7; 4; 6];
inv(A)*b
A\b

To evaluate this script in the MATLAB interpreter, apply the matlabint filter. The configuration in the dexy.yaml is:

    - basic.m|matlabint

Here’s what the output will look like:

                            < M A T L A B (R) >
                  Copyright 1984-2014 The MathWorks, Inc.
                    R2014a (8.3.0.532) 64-bit (glnxa64)
                             February 11, 2014


To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

>> A = [10 -7 0; -3 2 6; 5 -1 5]

A =

    10    -7     0
    -3     2     6
     5    -1     5

>> b=[7; 4; 6];
>> inv(A)*b

ans =

         0
   -1.0000
    1.0000

>> A\b

ans =

     0
    -1
     1 

The matlabint filter launches the matlab interpreter and sends commands, one line at a time. It returns a full session transcript including input prompts, commands and output. You can end a line with ; to prevent its output from being echoed (this is a feature of the MATLAB interpreter).

Sections and Syntax Highlighting

The syntax highlighting in this blog post is applied using the pyg filter. The .m file extension used by matlab does not uniquely identify code as matlab code within the Pygments syntax highlighter, so we need to specify the lexer manually. Here’s how:

    - .m|pyg:
        - pyg: { lexer : matlab }

Now here’s the script we used to generate all the examples in the first section, with syntax highlighting applied:

%%% "3d-plot"
[X,Y] = meshgrid(-8:.5:8,-8:.5:8); % Create a mesh grid on [-8,8]*[-8,8].
R = sqrt(X.^2 + Y.^2) + eps; % + eps to avoid divide-by-zero error.
Z = sin(R)./R;
mesh(X,Y,Z)
print -dpng 'mesh.png'

%%% "anon-functions"
cs=4;
myfun = @(x) (x(1)-1).^4+cs*(x(2)-2).^2

%%% "meshgrid-using-anon-fn"
[X,Y] = meshgrid(0:.1:3,0:0.1:6);
tic, for i=1:size(X,1), for j=1:size(X,2), F(i,j)=myfun([X(i,j) Y(i,j)]); end, end, toc
surfc(X,Y,F); xlabel('x_1');ylabel('x_2');zlabel('myfun(x_1,x_2)')
print -dpng 'mesh-with-function.png'

%%% "van-der-pol-def"
xdot=@(t,x) [x(2);(1-x(1)^2)*x(2)-x(1)];

%%% "van-der-pol-solve"
[t,x_t]=ode45(xdot,[0 20],[2 0]);

%%% "van-der-pol-plot"
plot(t,x_t);grid;xlabel('time');ylabel('x');
title('Van der Pol Equation');legend('x_1','x_2');
print -dpng 'van-der-pol.png';

%%% "van-der-pol-simulink"
open('vdp');
set_param('vdp/Scope','LimitDataPoints','off','SaveToWorkspace','on');
sim('vdp');
plot(ScopeData(:,1),ScopeData(:,[2 3]));grid;xlabel('Time (s)');ylabel('x');title('Van der Pol Simulation');
print -dpng 'van-der-pol-simulink.png';

%%% "print-simulink-model"
print -dpng -svdp 'van-der-pol-simulink-model.png'

%%% "close"
close_system('vdp',0);

%%% "write-js"
addpath('jsonlab')
foo = 5; bar = 7;
vars = struct('foo', foo, 'bar', bar, 'pi', pi)
fid = fopen('values.json', 'w');
fprintf(fid, '%s', savejson(vars));
fclose(fid);

%%% "help"
help publish

This script is divided into named sections using specially formatted comments. The idio filter knows how to split this script up. We’ll apply this filter before we execute the code, so we can capture each section’s output separately (this concept is covered in the Getting Started dexy tutorial mentioned above).

Here’s the dexy configuration applied to this file:

    - sections.m|idio|matlabint|pyg:

The idio filter splits code into sections, then the matlabint filter runs each section through the matlab interpreter, and finally the pyg filter applies syntax highlighting to the output.

We also specify some custom configuration for the matlabint filter:

        - matlabint: {
            timeout : 30,
            add-new-files : true,
            additional-doc-filters : {.png : botoup}
          }

The add-new-files setting tells dexy to pick up any generated files and add them to the dexy run so other documents can make use of them, and the additional-doc-filters setting tells dexy to apply the botoup filter to any new files found which use the .png extension. This means that any .png files generated by the script will be uploaded to S3.

And we specify a custom lexer for the pyg filter, this time the matlabsession lexer because we are applying syntax highlighting to the output from the matlab interpreter, rather than a matlab .m file:

        - pyg: { lexer : matlabsession }

We also need to specify some other files as dependencies so they will be
available on the local file system when we run the matlab script:

        - jsonlab/*.m

Here’s the full configuration:

    - sections.m|idio|matlabint|pyg:

        - matlabint: {
            timeout : 30,
            add-new-files : true,
            additional-doc-filters : {.png : botoup}
          }

        - pyg: { lexer : matlabsession }

        - jsonlab/*.m

Here is how we then include one section of the processed script output in a document using a jinja2 template:

{{ d['sections.m|idio|matlabint|pyg']['3d-plot'] }}

It looks like this when rendered:

>> [X,Y] = meshgrid(-8:.5:8,-8:.5:8); % Create a mesh grid on [-8,8]*[-8,8].
>> R = sqrt(X.^2 + Y.^2) + eps; % + eps to avoid divide-by-zero error.
>> Z = sin(R)./R;
>> mesh(X,Y,Z)
>> print -dpng 'mesh.png'

We applied the botoup filter to any .png files which were generated by our script, and this filter uploads files to S3 and then returns their URL:

{{ d['mesh.png|botoup'] }}

Here is the URL of the generated plot:

https://s3.amazonaws.com/blog-dexy-it-2014/mesh.png

We can then use this URL to generate an image tag using Markdown syntax:

![Mesh Plot]({{ d['mesh.png|botoup'] }})

Image may be NSFW.
Clik here to view.

Or do the same thing using raw HTML:

<img title="Mesh Plot" src="{{ d['mesh.png|botoup'] }}" />

Image may be NSFW.
Clik here to view.

Referencing Individual Values

Sometimes you want to embed calculated values within the text of a document, either instead of or in addition to showing the code which calculated them. One recommended way to do this in Dexy is by writing the data to a JSON file. Here’s one way to do this in MATLAB using JSON functions provided by the jsonlab library:

>> addpath('jsonlab')
>> foo = 5; bar = 7;
>> vars = struct('foo', foo, 'bar', bar, 'pi', pi)

vars =

    foo: 5
    bar: 7
     pi: 3.1416

>> fid = fopen('values.json', 'w');
>> fprintf(fid, '%s', savejson(vars));
>> fclose(fid);

The values.json file created in this script is detected by the dexy run because we have set add-new-files to true. Here’s the contents of the generated file:

{
    "vars": {
        "foo": 5,
        "bar": 7,
        "pi": 3.141592654
    }
}

If you reference this file from a document template and try to access the elements within it, Dexy will automatically parse this JSON file into a data structure for you, and thanks to jinja2’s dot syntax, you can refer to elements in a JSON object using either dots (as if the values were attributes) or the more usual python dict accessors:

{% set vars = d["values.json"]["vars"] %}
Foo is {{ vars.foo }} and bar is {{ vars["bar"] }}. Pi is {{ vars.pi }}.

Here’s how it looks when this document template is evaluated and the values calculated in the script are embedded within the generated document:

Foo is 5 and bar is 7. Pi is 3.141592654.

Dexy or MATLAB Publish?

Finally, let’s compare using Dexy to using MATLAB’s built-in Publish feature. Both options let you create documents incorporating MATLAB code and the output from running it, but there are many important differences.

Dexy is a more flexible, general-purpose tool that lets you write many different kinds of documents, both in terms of document format and in terms of the content and intent of a document. With Dexy you can write a report showing just the values resulting from a calculation and generated graphs, like this:

Foo is 5 and bar is 7. Pi is 3.141592654.

Image may be NSFW.
Clik here to view.

Or you can write some documentation which includes a section of unprocessed source code:

cs=4;
myfun = @(x) (x(1)-1).^4+cs*(x(2)-2).^2

Or you can show the results of running that source code:

>> cs=4;
>> myfun = @(x) (x(1)-1).^4+cs*(x(2)-2).^2

myfun =

    @(x)(x(1)-1).^4+cs*(x(2)-2).^2

With Dexy you can mix and match source code from many different MATLAB files and even include and execute code in other languages.

MATLAB’s Publish feature is much more limited, but it has the advantage of being simpler and not requiring you to install any additional software if you already have MATLAB installed.

Here’s an example of some MATLAB source code with comments written for the Publish command:

%% Hello
% This is an example document.

%% Patients

%%
% Create a table from individual workspace variables.
load patients
patients = table(LastName,Gender,Age,Height,Weight,Smoker,Systolic,Diastolic);

%%
% Select the rows for patients who smoke, and a subset of the variables.
smokers = patients(patients.Smoker == true, {'LastName' 'Gender' 'Systolic' 'Diastolic'})

%%
% Convert the two blood pressure variables into a single variable.
patients.BloodPressure = [patients.Systolic patients.Diastolic];
patients(:,{'Systolic' 'Diastolic'}) = [];

%%
% Pick out two specific patients by the LastName variable.
patients(ismember(patients.LastName,{'Smith' 'Jones'}), :)

Here’s what HTML generated via the Publish command looks like:

The Publish feature is a great way to generate a presentable record of an interactive session of work, or to document a single script and show its full execution. You have a choice of several document formats and settings to control the output.

>> help publish
 PUBLISH Publish file containing cells to output file
    PUBLISH(FILE) evaluates the file one cell at a time in the base
    workspace.  It saves the code, comments, and results to an HTML file
    with the same name.  The HTML file is stored, along with other
    supporting output files, in an "html" subdirectory within the script's
    directory.

    PUBLISH(FILE,FORMAT) saves the results to the specified format.  FORMAT
    can be one of the following:

       'html'  - HTML.
       'doc'   - Microsoft Word (requires Microsoft Word).
       'pdf'   - PDF.
       'ppt'   - Microsoft PowerPoint (requires Microsoft PowerPoint).
       'xml'   - An XML file that can be transformed with XSLT or other
                 tools.
       'latex' - LaTeX.  Also sets the default imageFormat to 'epsc2'
                 unless figureSnapMethod is 'getframe'.

    PUBLISH(FILE,OPTIONS) provides a structure, OPTIONS, that may contain
    any of the following fields.  If the field is not specified, the first
    choice in the list is used.

        format: 'html' | 'doc' | 'pdf' | 'ppt' | 'xml' | 'latex'
        stylesheet: '' | an XSL filename (ignored when format = 'doc', 'pdf', or 'ppt')
        outputDir: '' (an html subfolder below the file) | full path
        imageFormat: '' (default based on format)  'bmp' | 'eps' | 'epsc' | 'jpeg' | 'meta' | 'png' | 'ps' | 'psc' | 'tiff'
        figureSnapMethod: 'entireGUIWindow'| 'print' | 'getframe' | 'entireFigureWindow'
        useNewFigure: true | false
        maxHeight: [] (unrestricted) | positive integer (pixels)
        maxWidth: [] (unrestricted) | positive integer (pixels)
        showCode: true | false
        evalCode: true | false
        catchError: true | false
        createThumbnail: true | false
        maxOutputLines: Inf | non-negative integer
        codeToEvaluate: (the file you are publishing) | any valid code

    When publishing to HTML, the default stylesheet stores the original
    code as an HTML comment, even if "showcode = false".  Use GRABCODE to
    extract it.

    MY_DOC = PUBLISH(...) returns the path and filename of the generated
    output document.

    Example:

        opts.outputDir = tempdir;
        file = publish('intro',opts);
        web(file)

    See also NOTEBOOK, GRABCODE.

    Reference page in Help browser
       doc publish

In comparison, Dexy is a more flexible and powerful tool, but it’s harder to learn and requires Python to be installed. Dexy gives you not only a powerful tool for writing documents incorporating code and results, but also a way of automating many other aspects of your project. If you have a complex workflow, Dexy can run all your scripts in the correct order, help you to share data files between them, and finally help you to communicate your results with confidence in their reproducibility.

If you have questions about using Dexy with MATLAB, please feel free to leave a comment or get in touch via one of the suggested methods here.