SEILIB - the Simulation Environment Interaction Library -- is designed for handling parameterized input-decks and commands. In addition, several tools can be integrated to complete simulation flows based on these parameterized input files and optimization setups can be configured.
Values in text input-decks can be replaced by so-called arguments, which are generally encoded with preceding and succeeding brackets: {SEIARG}. The parameterized input-decks are regarded as templates or meta input-decks. They are used to create the actual input files for the simulators. The library provides several additional features, such as process administration, multi-threading, and host management.
The library is dedicated to all users of an arbitrary number of arbitrary programs, scripts, or functions. Many similar simulations depending on an arbitrary number of input parameters shall be processed and (partly) repeated. It is therefore essential to keep the inputs and outputs consistent.
Furthermore, all available computational resources should be employed to reduce the execution time. The setup is extensible and several tools can be integrated to complete tool flows. In context of the GMRES(M) evaluation discussed in Section 5.5.3, the library successfully processed 23,000 processes on four IBM cluster nodes. It is important to note, that the use of arguments is not a must. So the host management can also be used for simulations based on conventional (non-parameterized) input-decks.
The library is written in PYTHON [167]. The implementation of the library itself can be found in the file seilib.py, which has to be accessible for the interpreter.
This section addresses the most important steps how to set up a new SEILIB application. Note that the optimization system is not covered here, see Appendix C.3.9 for this information.
A new PYTHON script normally starts with an execution directive for the operating system. As PYTHON is an interpreted language, the following line tells the loader which interpreter to use:
#! /usr/bin/env python
After importing the SEILIB library file, a new instance of the main class can be created, the name for it chosen here is p:
import seilib p = seilib.seiclass()
At this point, the proper processing can be already tested. After setting execute permissions for the script (chmod u+x name_of_script.py), it can be directly started like other programs by typing on the command line: ./name_of_script.py. Alternatively, the name of the script can be passed to the interpreter: python name_of_script.py.
Note that the seilib.py file must be accessible for the interpreter. If the library is stored in another directory, the search path for libraries can be extended in order to avoid import errors:
import sys sys.path.append(``name_of_directory'')
If the script is properly executed, only the welcome message of the SEILIB is printed. Of course, the newly created instance can now be set up for the actual SEILIB application. The setup can be divided in two steps:
In order to understand the functionality of the script, the definitions of tool, command, host, argument, template, and auxiliary have to be given:
The following examples give a first glance on how the script is configured:
p = seilib.seiclass()
p.hostman.newHost("tcad01", 0, 8, 1)
ecmd = seilib.seicmdexec("<{CMD}> > <{LOG}>")
p.newTool("mmnt", # name of the tool
1, 1, # mode and number of sub-modes
ecmd, # command
"short", # prefix
"<{AUX1}>", # command line options
[ "seimos.ipd" ]) # array with template files
p.newTool(merge, 12, 1, 0, "test", "", [ ])
The instance of the main class seiclass is p. A new host, tcad01, is registered at the host management system of p. The nice level for all processes on that host will be zero, eight processes may run simultaneously, and the once parameter is set to one (see below).
Next, a command is created, which is here an instance of the simple execution class seicmdexec. One purpose of this class is to compose the command line of the actual tool execution depending on the execution method. The user is absolutely free in defining these commands. In the example, the command line consists of {CMD} and the pipe of standard output to {LOG}. The special-purpose argument {CMD} contains the name of the tool and its command line options as specified in the parameter of the newTool call.
Finally, the tools are specified which is here a call of MINIMOS-NT (mode one, one sub-mode, prefix ``short'') with one input-deck only. In the command line, this input file is represented by the special-purpose argument {AUX1}. The template is stored in array, that could hold an arbitrary number of templates which are consecutively numbered. Note that all automatically generated numbers start with one, whereas PYTHON counting starts with zero.
More than one template is particularly useful if the include statement of the input-deck is used in order to construct a hierarchy of input-decks (as an alternative to conditional inheritance). For example, the physical definition is stored in a base input-deck, which is then included in various application input-decks like for simulating Gummel plots and extracting S-parameters. As all of these input-decks may contain SEILIB arguments, the library therefore supports an arbitrary number of templates. Note that template names may be arguments themselves.
The second tool is a PYTHON function (not shown here) and does therefore not require a command. It is supposed to combine all MINIMOS-NT output curve files to one. It belongs to mode twelve.
There are five methods for executing processes:
The library always waits for all spawned processes before it terminates.
What happens to the template and its arguments? This brings the argument system into play which is going to be discussed in this section. The template input-deck seimos.ipd contains the following four arguments:
aux infile = "<{CWDIR}>/" + if(getenv("HOSTTYPE") == "i386", "mos",
"mosibm");
aux outfile = "<{DEVOUT}>";
aux crvfile = "<{CRV}>";
aux numSteps = <{SN}>;
As stated above, arguments are addressed by their names extended by preceding and succeeding brackets. The constant {CWDIR} is always provided by the library and stands for the current working directory. The names of the input device is added to this directory, but depends on the host type due to the binary ordering. The name of the device output and curve files ({DEVOUT} and {CRV}) depend on the actual state of the script processing, otherwise they would be overwritten all the time. The only input parameter is the number of steps {SN}, which is referenced by a stepping function. This example is for demonstration purposes only - the number of scattering events in a Monte Carlo simulator might be a better one.
Note that in the parameterized input-deck the double quotes must still be used for all represented strings such as the file names. But there are no double quotes for {SN}, which actually stands for an integer. Basically, the script processes strings only, and all non-string values specified as arguments are automatically converted to strings. However, this is no restriction for system handling text-based input-decks.
The last point is how the arguments and their values are registered. This should be demonstrated for the example:
p.newVariable("SN", getSN)
p.newConstant("NAME", "Test")
p.newConstant("AUXDIR", "<{CWDIR}>/aux/", "dir")
p.newConstant("LOGDIR", "<{CWDIR}>/log/", "dir")
p.newConstant("OUTDIR", "<{CWDIR}>/out/", "dir")
p.newConstant("ALLDAT", "<{NAME}>_<{FLATB}>_<{PREFIX}>")
p.newConstant("AUX", "<{AUXDIR}>/<{ALLDAT}>_<{AUXNR}>_<{AUXNAME}>")
p.newConstant("LOG", "<{LOGDIR}>/<{ALLDAT}>.log", "log")
p.newConstant("CRV", "<{OUTDIR}>/<{NAME}>_<{FLATB}>.crv")
p.newConstant("DEVOUT", "<{OUTDIR}>/<{ALLDAT}>_out.pbf")
Although this does not look like the ultimate selling point at the first glance, there are quite many arguments for such a small parameterized simulation. Obviously it would have been possible to skip some of them. But this example is also intended to be a template for further applications where the underlying concepts might be useful.
Starting with the variable SN, its values are coming from a PYTHON function, namely getSN. The function itself is shown below. Then, nine constants are defined: a name, three directories, one constant being used as intermediary, the name convention for auxiliary files, and finally the three constants which were used in the template input-deck. The following statements characterize the argument system:
Note the different definition of {CRV}. In contrast to the other constants, it does not include the unique tool prefix. While setting up tool flows, the output of one tool is normally used as input for another tool. In the example, the MINIMOS-NT output curve files are the inputs for the merge function which is supposed to combine them. This can be easily achieved if the definition does not depend on the tool prefix.
The auxiliary files are created after all arguments have got their respective values. The stream editor is used to transform the templates to auxiliary files. The user can define an argument {AUX}, which contains the template for a name of an auxiliary file (see the example above). If such a variable is missing, a unique name is generated by combining {FLATB}, {PREFIX}, the number of the auxiliary and its name. The special-purpose arguments {FLAT} and {FLATB} contain the current meta-level and combination (for example M1_C1), and can be extended by a certain number of variable values (cf. maxVarDepth). A list of all special-purpose arguments is given in Table C.1.
The library is basically process-oriented and so the complete implementation is based on this idea. Therefore, it is useful if the user keeps the idea of creating processes in mind, which are finally executed on one host. The underlying simulations differ from each other by an arbitrary number of argument values. How many processes are potentially started?
This depends on the user configuration, which can be completed now:
def getSN(name, currProcess): return currProcess.procComb * 10 p.numCombinations = 7
The function getSN was already referenced by argument {SN} and should return the number of steps. This number should obviously depend on the processing state. In the example, the number of steps is the tenfold value of the current combination number. Note that the integer value is automatically converted to a string. Furthermore, it should be again emphasized that functions like getSN are native PYTHON code without any restrictions coming from the library. The library provides the following information in the function parameters: as first parameter the name of the argument the library is requesting a value, since one function could be used for several variables. The second parameter is the current process (the name of the parameter is up to the user: here currProcess was chosen) and an instance of the class seiprocess, which stores all information on this specific process.
But there is also a second way for defining values of variables: the array. The library automatically uses the combination number as index for the array. If the array is too small, the combination number modulo the array length is taken. The alternative configuration for {SN} could simply look as follows:
p.newVariable("SN", [ 10, 20, 30, 40, 50, 60, 70 ])
Four members of the process class are essentially defining the state of processing:
Since this looks a bit complicated, it is inevitable to note that the library can only be as powerful as its state system. Since the user already specified a set of tools which have to be additionally taken into account, the hierarchy is given by:
Inside the fifth loop, an instance of the process class is allocated, a host is selected, all arguments get their values, the auxiliary files are created, and the tool is finally started.
The data stored in the process class is not always complete. There are three levels:
The library offers a flexible execution control. Before a tool is really executed, the function assigned to the doExecute class member is called. It takes the current process as the only parameter. If this function returns zero, the process is not executed. Note that the values of all arguments and the complete state information is available. The user can of course use the complete PYTHON syntax to write this function. The post-execution function afterExecute works in a similar way. It takes the complete process information (including the exit code) as only parameter. If it returns zero, the library processing will be terminated as soon as possible. Examples for these two functions are given now:
def doExecute(currProcess):
if currProcess.procMode != 12: return 0
return 1
def afterExecute(currProcess):
if currProcess.procExit != 0 and not currProcess.procIsCF:
print("log file: " + currProcess.procLog)
return 1
The function doExecute allows only tools of mode twelve to be executed. The function afterExecute prints the log file of the process in the case of a non-zero exit code, but only if the process represents a program and not a callable function.
The host management system offers the possibility to distribute the load over several CPUs and/or over a network (see Section 5.3.1). The user can specify an arbitrary number of hosts which are subsequently used to execute processes. The system retrieves information about the load average of each host, which can be used to select or reject hosts. By using the once parameter of the host, the user can influence the way how hosts are selected. This should be clarified by an example. It is supposed that more than eight processes should be executed, and four hosts (tcad01 - tcad04) are registered allowing two processes at the same time. Depending on the parameter once, the following host selection sequences are possible:
once = 1: tcad01 => tcad02 => tcad03 => tcad04 =>
tcad01 => tcad02 => tcad03 => tcad04 => waiting...
once = 2: tcad01 => tcad01 => tcad02 => tcad02 =>
tcad03 => tcad03 => tcad04 => tcad04 => waiting...
Hence the parameter once can be used to priorize some hosts, for example to keep the first processes local or to fully employ the local resources before remote ones are allocated.
Two classes are responsible for the host management: whereas the seihost class represents one particular host and stores all information about it, the hostManagement class administers the list of all hosts and organizes the communication to them.
As already mentioned above, some commands can be restricted to particular hosts. The following example should demonstrate this:
tcad01 = p.hostman.newHost("tcad01", 0, 1, 1, [ "powerpc" ])
tcad02 = p.hostman.newHost("tcad02", 0, 2, 2, [ "powerpc" ])
tcad03 = p.hostman.newHost("tcad03", 0, 2, 1, [ "powerpc" ])
tcad04 = p.hostman.newHost("tcad04", 0, 2, 2, [ "powerpc" ])
scmd1 = seilib.seicmdssh("source <{CWDIR}>/seisetup;
nice -n <{NICE}> <{CMD}> > <{LOG}>",
[ tcad01, tcad04 ])
Four hosts are registered, but the command scmd1 is restricted to tcad01 and tcad04. Note that in the restriction array the return values of newHost are used. If there is no host for such a restricted process available, it is simply postponed. Thus, while the process is waiting for an appropriate host, other unrestricted processes can still be executed on hosts tcad02 and tcad03. However, postponed processes have priority over new processes for the hosts they are waiting for.
Since a new shell is opened for all spawned processes, the user has to ensure the setting of the environment. Therefore, a setup script can be sourced as shown in the example above. In addition, the user is responsible for taking the nice level into account. The library offers a wide variety of possibilities in the parameter space, but the user is totally free in applying them.
The diagram of the major SEILIB classes is depicted in Figure C.1. In the center, the main class seiclass is shown. This class combines own functionalities and those of other classes in order to provide the complete set of features. Therefore, it is the base class for all SEILIB applications.
In addition, an auxiliary class provides basic functions such as the administration of command line arguments, exit codes, output functions etc. The name of this class is Auxiliary and a global instance seilib.auxiliary is publicly available.
The formerly single-threaded MINIMOS-NT test system was parallelized mainly for two reasons:
So the platform of the complete test system was changed from Linux to AIX, which did not have any consequences for the PYTHON code since the interpreter is also available for AIX. Furthermore, the new platform has additional advantages regarding the numerical representation (see Appendix C.4.1).
Instead of parallelizing the existing test system, a new test system has been developed as a SEILIB application. Since both systems are based on the same ideas, the adaptation of the old tests was very simple. Due to extended features of SEILIB, new tests can be written in a much more flexible and easier way.
In Figure C.2, the class diagram of the new MINIMOS-NT test system is shown. The main class Example is derived from seiclass and is responsible for processing one test. The process and host management systems are directly available, the argument system is processing the settings found in the test script.
There is also one test which does not fit entirely in the concept. This test is intended to evaluate all MINIMOS-NT examples as found in a exa directory. So the specialized class ExaExample was written in order to process all input-decks found in this directory and to provide all features known from the other tests.
The new test system is not only able to parallelize the execution of one test, but also to parallelize the execution of all test scripts found in the test directory. This functionality is regarded as a test itself, and therefore a specialized class CompleteTest derived from Example was implemented. At this point of the test system a cascaded application of the SEILIB can be seen, which is an important aspect regarding the flexibility and stability of the library.
Another proof of concept can be conducted based on the next considerations. If the set of the implemented SEILIB subsystems is analyzed, interesting similarities to the SIESTA framework [232] can be found. Basically, SEILIB is able to process template input-decks, employs a process and host management system, and allows to define tool flows consisting of an arbitrary number of arbitrary tools.
In contrast to the SIESTA system, SEILIB was designed to process a fixed number of combinations in a pre-defined and deterministic way. However, this should be no obstacle for providing an optimization feature by implementing a new optimizer class derived from seiclass. The optimization system is a highly-specialized SEILIB application which can be employed as a full alternative for SIESTA optimization models.
The description of the optimizer system starts with some details on the technical implementation: each supported optimizer provides an interface program to the actual optimizer system. The optimizers themselves and their interfaces are not strictly speaking part of the SIESTA framework. SIESTA starts the interface program as a separate process and communicates with this process by writing to standard input and reading from standard output. This part can be easily reproduced in PYTHON.
Each optimization model includes one or more so-called free parameters which are varied by the optimizer in order to find a minimum of a given scalar (cost or score function) or vector (cf. Levenberg-Marquart optimizer) target. The free parameters are defined by specifying a default, minimum, and maximum value. The variation of the free parameters depends on the results of the respective simulations. So during the optimization process, the interface class of the chosen optimizer requests one or more variable combinations per step. For each of these combinations, the defined tool flow must be processed after the values of the free parameters have been replaced in the template input-decks. The respective results are collected and written to standard output.
In the SEILIB context, the free parameters are arguments with boundaries. Furthermore, in the core results were not of interest at all up to now. For that reason, also slight extensions of the core modules were necessary and implemented. However, each request of the interface class represents a new setup of a pre-defined number of combinations. Thus, the concept fits perfectly in the already existing system and very few adaptations were necessary for providing the optimization setups with all features known from SEILIB and SIESTA. In fact, the largest part of the optimization system deals with preparing the settings for the optimizers.
SEILIB-based optimizations are based on one of four optimizer classes, which are derived from a base optimizer class (see the class diagram in Figure C.3).
The following four systems are currently supported:
Note that these optimizer systems are not part of the SEILIB code, but merely coupled to SEILIB. Thus, although their code is redistributed and provided, the license agreements of the respective code or package (see more information in [232]) have to be considered.
In the following, a short example should be discussed, which employs the DONOPT optimizer:
p = seiopt.seidonopt()
Besides of normal arguments, free parameters are defined, for example:
![\includegraphics[width=10.8cm]{figures/seihierarchy2.eps}](img998.png)
![\includegraphics[width=8.0cm]{figures/seimmnttest.eps}](img999.png)
![\includegraphics[width=8.0cm]{figures/seiopthierarchy.eps}](img1000.png)
p.newVariable("a", p.getVariable, [ 1.0, -10.0, 10.0 ] )
p.newVariable("b", p.getVariable, [ 0.0, -10.0, 10.0 ] )
p.newVariable("c", p.getVariable, [ 0.0, -10.0, 10.0 ] )
Free parameters are variables and get their values from the optimizer. The name of the method is getVariable (getValue is already used in the argument system). The optional third argument was used for dir and log before, now it is used to specify the default, minimum, and maximum value of the variable in an array.
The most critical point of the optimizer model is the feedback path of the result. The tool flow is processed depending on the input values requested by the optimizer. In order for the results to be written back to the optimizer, the user is responsible for
A frequently used place for doing this is the afterExecute method. If more than one tool is involved, the user must furthermore ensure that this is done for the last tool in the tool flow only.
For example:
currProcess.setResult(scalarResult) p.writeResult(currProcess)
Now the definition of the optimization model is done - the only missing part is the configuration of the optimizer. Whereas DONOPT and LMMIN provide configuration methods, the settings of the GENOPT and SIMAN optimizers have to be done directly by accessing the public members of the classes.
The advantages of the SEILIB over the SIESTA system are:
The disadvantages are: