Subsections

• C.4.1 Numerics
• C.4.2 Nomenclature and Definitions
• C.4.3 Test Levels
• C.4.4 File Structure
• C.4.5 Model Identification System
• C.4.6 The References

C.4 The Minimos-NT Test

MINIMOS-NT is always under development since there are always new ideas for additional features. However, in order to preserve the already achieved efforts, their functionalities have to be tested on a regular basis.

The MINIMOS-NT test (mmnttest) is a feature-based test: a given set of simulations is run and the produced output files are compared with references. In case differences are detected, the reason for them should be found out. The test is based on the following ideas:

• Before new features enter the CVS repository of MINIMOS-NT, their independence from already tested features has to be proven.
• As new features enter the CVS repository of MINIMOS-NT, appropriate tests have to be constructed and added to mmnttest.

C.4.1 Numerics

Unfortunately, comparing output files is a cumbersome task. The main reason for difficulties is the limited representation of floating point numbers. The output files heavily depend on

• the compiler,
  
• the compile mode,
  
• the machine(s) used for compiling and linking, and
  
• the machine used for running the tests.

Therefore, the mmnttest references have to be created multiple times to cover the range of development environments used at the institute.

Note that all vprojects have to be compiled with the same mode in order to ensure consistent numerical representations. In addition, compiling and linking should be restricted to one machine only, even if the full paths are specified in the configuration file.

The problem of the numerical representation has been partly resolved by changing the platform to AIX. This has one particular reason: The main development platform of the institute is Linux. So here a variety of compilers and compilation modes can be found, because each developer has personal preferences which are widely accepted and supported due to the academic context. But once the development is ready to be ported to AIX, all members of the development team use the same compiler and (almost) the same compiler configurations. Since AIX is not the major development platform, it is also easier to recommend respective guidelines.

Since the test can be run on more than one platform, also inter-platform comparisons can be made. These analyses can detect implementation errors which become obvious by running a binary compiled and linked by another compiler. As the SEILIB test system conveniently allows to repeatedly access the results files, several checks can be additionally performed during a post-processing of a test run. These checks may also include comparisons with references generated for other platforms.

C.4.2 Nomenclature and Definitions

As stated above, mmnttest is a feature-based test:

• A feature is a single MINIMOS-NT functionality, in most cases as described in the documentation.
• A test combines a set of related MINIMOS-NT features, for example stepping functions or optical model functionalities.

A test consists of general settings such as the name or id as well as of different models.

• A name is the name of the template input-deck.
  
• An id is a unique three-character abbreviation of the test.
  
• devices is a list of one or more input pif files.
  
• schemes is a list of one or more iteration schemes.
  
• args is a list of variables equally used for all models.
  
• models is a list of one or more model combinations.

For all devices and all iteration schemes all model combinations are performed. The small optical test is given as example:

import    seitestlib
example = seitestlib.Example()

example.newConstant("NAME",     "Optical")
example.newConstant("NAMEID",       "OPT")
example.newConstant("DEVICE", "Diode-PIN")
example.newConstant("SCHEME",        "DD")
example.newConstant("LVL",              1)

example.numCombinations = 2

example.newVariable("ID",    [ "01",   "02"   ])
example.newVariable("J0",    [ "1e12", "1e15" ])
example.newVariable("alpha", [ "0.5",  "0.7"  ])

C.4.3 Test Levels

In contrast to former mmnttest versions, a level system was introduced. A level is defined by the argument {LVL}. Generally, it was decided to provide nine levels, which are defined as follows:

Table C.2: Definition of the test levels.

Level	Execution Time	Started
1	$< 30 s$	always
2	$< 60 s$	always
3	$< 120 s$	always
4	$>= 120 s$	always
5	$< 30 s$	not always
6	$< 60 s$	not always
7	$< 120 s$	not always
8	$>= 120 s$	not always
9	deactivated	never

The ninth level is reserved for deactivated models, for example unsuccessful simulations. The other two blocks are divided into four categories each based on the execution time. This time refers to a specific computer, compiler, and mode. At the moment there are over 800 tests available, thus, a complete run takes several days on a single computer. This would be obviously very inconvenient for fast tests during development. In addition, not all tests have to be run really every time. Despite some models are maybe still important in context of full tests, they can be skipped in order to speed-up pre-check-in tests. So before changes may be committed to the CVS repository, a full level 1-4 test should be performed. Level 5 to 8 are additionally tested by automated test runs, which are started independently of feature check-ins.

The -level option of the test script requires a range or level numbers, all separated by commas, for example:

• -level 1: selects level 1.
  
• -level 1-4: selects level 1,2,3,4.
  
• -level 1,3-5,7,8: selects level 1,3,4,5,7,8

C.4.4 File Structure

Core of the test is the main library seitestlib.py, which provides two main classes: Example and CompleteTest. The former is derived from seiclass, the latter from Example (see Figure C.2). The new test library is therefore a full SEILIB application. This library is imported and used by each test script file. To run all tests automatically, the driver script mmnttest.py can be used.

Each test consists of several files:

• the test script file in mmnttest which contains all settings and model combinations,
  
• the test template file which is a parameterized input-deck stored in mmnttest/tpl_ipd,
  
• the test device file stored in mmnttest/in_pif,
  
• maybe test data files which are stored in mmnttest/data (see the stepping test for an example).

The following directories, which may further depend on the host platform, are created by the library if they do not exist:

• in_ipd: the actual input-decks. Their creation is based on the template in tpl_ipd.
  
• in_pbf: the input-pbf files, which are the converted pifs from in_pif.
  
• out_crv: the output curve files of the tests.
  
• out_log: the log files contain the MINIMOS-NT output of the respective test.
  
• out_pbf: the output pbf files of the tests.
  
• out_pif: the output pif files of the tests.

Obviously, there are connections between the general settings and filenames:

• the name specifies the template input-deck in the directory tpl_ipd,
  for example tpl_ipd/Optical.ipd.tpl
• one device is specifying the input pif file in the directory in_pif,
  for example tpl_ipd/Diode-PIN.pif

In the former versions, the input device names were connected with the test name, which is actually not necessary. Furthermore, this connection avoids a quite useful sorting by device classes such as diodes. In the new system, the device name must start with a class and continue with an optional description.

Since the test system combines input information to filenames, several conventions for naming test files must be adhered to. Devices start with a general device description, for example Diode, the first letter is capitalized. To further specify the device, for example the material Si-Si, an appropriate string is concatenated, separated by a hyphen. So it is easy to see all already existing devices of a specific category and to pick one for a new test if appropriate. Of course, the extension has to be pif.

Templates start with a capital letter, the name describes the test. The extension has to be ipd.tpl. Note that the name of the test is connected with the general setting name in the test script.

Test scripts start with a specific prefix, which is mmnt- for all tests of MINIMOS-NT, and a specific name of the test script. It is recommended to use the same name as for the template. The extension is .py.

As already shown above, the test script test-optical.py contains:

example.newConstant("NAME",   "Optical")   # connected with the template
example.newConstant("DEVICE", "Diode-PIN") # connected with the device

C.4.5 Model Identification System

A system was defined to create unique file names for the actual input and output files. To identify a test, two strings are used:

1. The {NAMEID} of the test itself, which is now a three-character abbreviation, for example IMD.
   
2. The {ID} of the model, which is given as first argument of a model

Why is {ID} a separate argument and not automatically generated by taking the combination numbers into account? First, related tests can so have a relationship expressed by the combination number. Second, skipped or added tests do not destroy already generated references (note that the combination number is part of the filenames). Based on the idea of the former versions, the device name and the iteration scheme are added to the filename.

C.4.6 The References

References are created by specifying the -genref option together with a new reference directory: -refdir x. The directory x should be an existing directory with a descriptive name. This directory is not automatically created if it does not exist. However, the shell script automatically creates the following four subdirectories if they do not exist: x/ref_crv,  x/ref_pif,  x/ref_pbf, and x/ref_log.

Since the output log contains information about the simulation time and date, the output log of the simulations is not automatically compared. However, it might be interesting for manual comparisons.