Building an mtestengine
Creating an MTestRunner is as simple as the following code suggests:
mte = MTestRunner;
Similar to a struct we can now visualize the properties of this object:
mte
mte =
MTestRunner handle
Properties:
MainDir: 'e:\Temp\tp75c7b7fb_f2ac_4331_896f_056c3548a3b4'
Recursive: 1
Verbose: 0
IncludeCoverage: 1
TestID: '_test'
Exclusions: {'.svn' '_tutorial' '_exclude'}
Tests: [1x1 MTest]
WrongTestDefs: {}
TestsCatalogued: 0
...It directly shows the default options for all fields. Of course we would like to set some of the properties. This can be done either when creating the object itself (with property value pairs) or afterwards in the same way as altering fields of a struct:
mte.MainDir = fileparts(which('MTestRunner'));
mte.Verbose = true;
mte.Recursive = true;
mte.IncludeCoverage = false;
or:
mte = MTestRunner(... 'MainDir',fileparts(which('mte_simple_test')),... 'Verbose',true,... 'Recursive',true,... 'IncludeCoverage',false);
Properties (fieldnames) of the MTestRunner
The MTestRunner has a couple of properties that can be used to define its behaviour while running tests. The following table lists the properties of an MTestRunner object.
| Property Name | Class |
Description |
|---|---|---|
| MainDir | char Default=cd |
Root directory of the toolbox that must be analysed and tested. |
| Recursive | logical Default=true |
Flag to determine whether the mtestengine only lists tests in the maindir or also in all subdirs of the maindir. |
| Verbose | logical Default=false |
Flag that determines whether intermediate information must be printed in the command window. |
| IncludeCoverage | logical Default=true |
Flag to turn on / off the profile function during execution of the tests to obtain function coverage information. |
| TestID | cell Default={'_test'} |
The character arrays included in this cell determine which functionnames are identified as testdefinition files. |
| Exclusions | cell Default={'.svn','_tutorial'} |
Any function or pathname that includes one of the character arrays filling this cell is excluded from the list of testsdefinitions. |
| Tests | mtest Default=[] |
mtest object with the loaded testdefinitions. The mtestengine takes this list whenever the run command is given. |
| WrongTestDefs | cell Default={} |
Files identified as tests according to the testid and exclusion keywords, but failed to load in an mtest object. This is probably due to a testdefinition that is not according to the convention used for the mtest toolbox. |
Setting properties is already adressed in the previous section.
Specifying tests to run
After creation of an mtestengine we can see that it does not contain any testdefinitions yet. The tests contains an empty test.
mte.Tests
ans =
MTest handle
Properties:
Name: []
Ignore: 0
IgnoreMessage: ''
Category: 0
Verbose: 1
AutoRefresh: 0
TestResult: 0
Time: 0
Date: NaN
...manually add tests
Specifying tests can be done in two different ways. Of course we can add tests manually to the field:
mte.Tests(1) = MTest('mte_simple_test');
search for tests
The catalogueTests method (or function) searches for all testdefinitions that match the following requirements:
- Tests should be in the maindir (or one of the subdirs if mte.recursive = true)
- The filename should include one of the strings specified in the field testid of the mtestobject
- The filename should not include one of the strings specified in the field exclusion of the mtestobject
- mfiles that match the above mentionned criteria are converted to mtest objects. they must therefore match the layout criteria described in the tutorial on testdefinitions
By default testdefinition files include the following string in their filename:
mte.TestID
ans = _test
but we exclude files with one of these strings in the filename
mte.Exclusions
ans =
'.svn' '_tutorial' '_exclude'
After automatically searching for testdefinitions and adding them to the testengine ...
gathertests(mte);
Collecting tests
we can see that the content of the tests field has changed. This is of course due to the fact that we just collected all tests that match the description as stated above:
mte
mte =
MTestRunner handle
Properties:
MainDir: 'F:\OpenEarthTools\trunk\matlab\maintenance\1_MTest\examples'
Recursive: 1
Verbose: 1
IncludeCoverage: 0
TestID: '_test'
Exclusions: {'.svn' '_tutorial' '_exclude'}
Tests: [1x10 MTest]
WrongTestDefs: {1x2 cell}
TestsCatalogued: 1
...Run all tests
To run all tests specified in the tests field of the mtestengine object there is a very simple function:
run(mte);
## start running tests ##
mte_wrongdefinition_test Could not be interpreted as a valid testdefinition.
1. mt_definitionwithdescription_newstyle_test (mt_definitionwithdescription_newstyle_test)
Test started: mt_definitionwithdescription_newstyle_test
Error occurred: Assertion failure:
Expected and actual vallue differ at location (1 1)
Expected: 4
Actual: 5
Test finished: mt_definitionwithdescription_newstyle_test
2. mte_concepttest_test (mte_concepttest_test)
Test started: mte_concepttest_test
Error occurred: 1 is not 2
Test finished: name of the test
3. mte_fulldefinition_test (mte_fulldefinition_test)
Test started: mte_fulldefinition_test
...Each mtest object under the field "tests" contains a field that stores the testresult:
mte.Tests(2).TestResult
ans =
0
Run all tests and publish documentation and results
The run function runs all tests and publishes the description and vizualization of results as specified in the testdefinition files. TODO: It also publishes the coverage of the tests (which percentages (and lines) of which functions did we actually test). More information on this function can be found in its help documentation:
help MTestRunner.run
RUN Runs all tests (that are in the "Test" property)
This function executes the run function of all mtest objects in the Tests property.
TODO:
- include input argument to specify test numbers (instead of just all tests).
Syntax:
outobj = obj.run;
outobj = run(obj);
Input:
obj - an MTestRunner object.
Output:
...