3. Running individual tests

Testdefinitions are designed in such a way that simply running the function from the command line will evaulate the test. If the test is not successfull it should crash when running command line, providing error information on what went wrong in which file.

Example:

mte_simple_test;

Advantage

One command leads to an immediate testresult. This method is therefore suitable for obtaining a quick look at the current state of the functions that are tested by this testdefinition.

Disadvantage

It is not possible to print either documentation or vizualisation of the result with one command.

A testdefinition can also be run with the use of the mtest toolbox. To do this first an instance of an mtest object must be created.

t = MTest('mte_simple_test');

This object contains all information from the testdefinition in the property fields (fields like a struct). Several functions can be applied to this object.

For example run the test:

run(t);

     Test started: mte_simple_test
     Test finished: mte_simple_test

The testresult is now stored in the field: "TestResult":

t.TestResult

ans =

     1

With the same object it is also possible to publish the description and result (if specified in the test definition):

t.Publish = true;
t.run;

     Test started: mte_simple_test
     Test finished: mte_simple_test

Advantage

• With this method additional to running the test, the documentation can also be generated. It
• requires a relatively small effort.

Disadvantage

• he testdefinition must be converted to an object before it can be run or published. Compared to command line usage this increases the amount of actions that must be undertaken.
• Next to that, the mtest toolbox is programmed with the use of object oriented programming as introduced with Matlab version 7.4 (2008a). This method cannot be used with matlab versions prior to 2008a.

The third way to run a test is with the use of the MTestRunner that is also used to scan the toolbox for testdefinitions and automatically run them. This is done in the same way as with the mtest object:

Create an mtestengine object

mtr = MTestRunner('Template','oet');

More information about calling MTestRunner:

help MTestRunner

  MTESTRUNNER - Object to autmatically run tests in a toolbox
 
  The mtest object is designed to run and publish a large amount of tests. Based on a main
  directory it can assamble all tests in the dir (and/or subdir if specified) and convert the
  test definition files to mtest objects. With its methods run and runAndPublish the tests can
  be run as well.
 
  Publishing the results is done based on a template (which is the default deltares template if
  not specified and created).
 
  See also MTestRunner.MTestRunner MTestRunner.cataloguetests MTestRunner.run MTest TeamCity

For automatic running test the engine can look for all tests in a particular maindir:

help MTestRunner.cataloguetests

 CATALOGUETESTS  Lists all tests in the maindir of the mtestobject and converts them to mtest objects.
 
    This function lists all files in the maindir (and subdirs if recursive = true) of an
    mtestobject. Test or directory names that contain one of the exclusion strings are
    not taken into account. For each test an mtest object is created and stored in the
    mtestengine object.
 
    Syntax:
    outobj = obj.catalogueTests;
    outobj = catalogueTests(obj);
 
    Input:
    obj     -   an mtestengine object.
 
    Output:
    outobj  -   The same mtestengine object, but with mtest objects for all tests in the
                maindir. It is not necessary to have an output, since the mtestengine is
                of type handle. This automatically adjusts all copies of the object that
                are in the matlab memory. The one that is in the base workspace is
                therefore automatically updated and does not need to be output of the
                function.
 
    See also mtestengine mtestengine.mtestengine mtestengine.run mtestengine.runAndPublish mtest mtestcase

We only want to execute one test, so we add this to the tests in the MTestRunner object:

mtr.Tests(1) = MTest('mte_simple_test');

Now we can use the run command to run the test and publish the result.

help MTestRunner.run

 RUN  Runs all tests (that are in the "Test" property)
 
    This function executes the run function of all mtest objects in the Tests property.
 
    TODO:
        - include input argument to specify test numbers (instead of just all tests).
 
    Syntax:
    outobj = obj.run;
    outobj = run(obj);
 
    Input:
    obj     -   an MTestRunner object.
 
    Output:
    outobj  -   The same MTestRunner object as the input argument obj. It is not 
                necessary to have an output, since the MTestRunner is of type handle. 
                This automatically adjusts all copies of the object that are in the 
                matlab memory. The one that is in the base workspace is therefore 
                automatically updated and does not need to be output of the function.
 
    See also MTestRunner MTestRunner.MTestRunner MTestRunner.run MTest MTestFactory TeamCity

TODO: simple method to directly load multiple tests in the MTestRunner TODO: simple method to print a summary of the test info to the command window after running