Running GEMPACK Programs from another program

Prerequisite: You should be familiar with running GEMPACK from the command-prompt (see here), and with controlling GEMPACK programs with BAT (batch) files (see here).

Introduction

If you are familiar with some programming language, it is not too hard to build a visual interface to control and run GEMPACK programs. An example is the RunGTAP GUI program used to run the GTAP model. GEMPACK Windows programs are constructed using Delphi Pascal, but other systems, such as Java, C++ or Visual Basic could be used. You could freely distribute your own GUI program, together with many (but not all) GEMPACK programs, to users so that they could run the system without any licence (see note about redistribution at end).

We describe below the main components of such a system under Windows. A note on non-Windows environments appears at the end.

GTAP example

To illustrate, we'll suppose that your GUI program should:

accept choices from the user, using buttons and list boxes
run a GTAP simulation depending on above choices
present results to the user

Running a program

Your programming language should support:

launching another program (aka process: an "instance" of a program)
detecting whether that process has finished running
detecting whether that process finished with an Error status or code

To run a GTAP simulation from the command line, you could type:

GTAP -cmf Sim3.cmf

This assumes you have GTAP.EXE and associated files GTAP.AXS and GTAP.AXT, which someone (perhaps not you) produced using source code GEMPACK. Instructions defining this particular simulation are contained in the text file Sim3.cmf, which will also contain the names and location of data (usually binary HAR) files used for the simulation. If successful the simulation will produce a binary result file, probably called Sim3.sl4. Whether or not successful, the simulation should produce a Log file, probably called Sim3.log.

Therefore the first task of your GUI program might be to create a text CMF file which implemented user choices (made by clicking buttons and list boxes). Simultaneously the GUI program might do many checks (for example, that all files needed were present in the right folder). It will be easier to detect mistakes now than to discover, in the aftermath of an unsuccessful simulation, what went wrong. Another wise practice is for your GUI program to delete any files which have the same name and location as those you expect the simulation to produce.

Launching a program

Under Delphi, external programs are launched using the Windows CreateProcess API. Your programming language will have a similar method (eg, Java's Runtime.exec). Typically you specify:

the name and location of the program to run (eg, C:\mydir\GTAP.EXE)
any command-line parameters(eg, -cmf Sim3.cmf)
the directory in which the program will start running (eg, C:\mydir)
whether the launched program will run minimized or be visible
whether or not the GUI program should wait (freeze) until the launched program has finished. Since simulations can take some time, you typically do NOT want the GUI program to freeze while a GEMPACK model program is running. Therefore your GUI program needs a way to monitor and detect when the simulation is done. Typically the launch routine returns a handle or process object which you can use to ask the OS whether your process (as identified by the handle) is finished.

Detecting whether the launched program has finished running

Under Delphi, the progress of the launched process is monitored using a Timer object which periodically makes a Windows WaitforSingleObject call (say 3 times a second). Your programming language should have a similar method (eg, Java's waitFor or exitValue catching IllegalThreadStateException).

The message that the launched program has finished will usually contain the "exit code" of the process. All GEMPACK programs are designed to return 0 on successful completion; a number >0 indicates an error.

Detecting whether the run was successful

Your GUI program should now make other checks -- for example that all expected output files are produced. If there does seem to be a problem, the GUI program should display (or direct the user's attention toward) the Log file (in this example, sim3.log).

Viewing the results

Assuming the simulation ran ok, the next problem is to present results to the user. These results will be contained in a binary file, perhaps called Sim3.SL4.

You could view these by typing, from the command line:

ViewSOL Sim3.sl4

Therefore, following the method explained above, your GUI could launch ViewSOL (the SL4 viewer). However, often you will need to translate the SL4 file into a more readable form (such as CSV, TXT, or HAR). You could do that with the command-line utility SLToHt:

SLToHt -sti sim3.sti

Here, sim3.sti is a text file (created by your GUI program) which lists the various options and filenames which SLToHt needs to know. For example, sim3.sti will specify which output format is desired (SLToHt can produce several).

Again, following the method explained above, your GUI could launch SLToHt to translate some or all of the the SL4 file contents into a text file which your GUI can read and present.

Creating HAR input and reading HAR output

As well as producing SL4 files, TABLO-generated programs (like GTAP.EXE) can read inputs and produce outputs in text or HAR form. Very often, the HAR binary format is the most convenient. Your GUI may need to read or even produce such files. You can download Fortran Fortran or Delphi subroutines to do this. But nearly always it is easier to use a command-line utility program to translate between HAR and a more familiar format.

To translate HAR files, you could use one of: seehar.exe, har2csv.exe, har2gdx.exe or head2xls.exe.

To produce HAR files, you could use one of: modhar.exe, txt2har.exe, xls2head.exe, csv2har.exe or gdx2har.exe.

Any of these command-line utilities may be run from a GUI program as described above.

BAT files to combine several commands

We see from the above that to run a simulation you may actually need to run several programs: programs to format inputs for the model; the model itself; and programs to reformat model output. It may well be easier to combine these in a BAT script. For example, such a BAT file might read (comments, in red, would not actually be in the BAT):

REM %1 will be sim name, %2 will be work dir
cd %2         change to specified folder  
del *.flg     delete expected outputs  
del *.log     delete expected outputs  
del %1.sl4

Echo ERROR at gdx2har stage >job.flg
gdx2har inputs.gdx inputs.har >gdx2har.log  format GDX inputs as HAR     
if errorlevel 1 goto error

Echo ERROR at GTAP stage >job.flg
GTAP -cmf %1.cmf  run simulation   
if errorlevel 1 goto error

Echo ERROR at SLToHt stage >job.flg
SLToHt -sti %1.sti >sltoht.log     translate SL4 to results.HAR   
if errorlevel 1 goto error

Echo ERROR at har2gdx stage >job.flg
har2gdx results.har results.gdx >har2gdx.log  translate results.har to results.gdx   
if errorlevel 1 goto error 

Echo SUCCESSFUL completion >job.flg
echo Job done OK     
goto endbat          

:error               
type job.flg
dir/od *.log         
echo PROBLEM !!!! examine most recent Log
:endbat

Notice that the BAT file offers additional opportunities for checking. For example, your GUI program could intermittently check the contents of the file job.flg to see which stage had been reached, or where the job had failed.

The BAT approach also lends itself to prototyping, and could be useful if the GUI programmer and the GEMPACK modeller were two different people. The GEMPACK modeller could first develop and test the BAT script to use inputs of agreed format (ultimately to be created by the GUI program) and produce outputs of agreed format (ultimately to be read by the GUI program). Both GUI program and BAT script could evolve independently (as long as input and output formats were agreed).

Non-windows environments

Much of the above will apply also to other OS, with differences of detail (eg, in the scripting language). You can run source-code GEMPACK natively on other OS, such as Linux. In this case:

The GEMPACK user is supplied with all the GEMPACK Fortran code, which they compile for their system.
The GEMPACK Windows programs ViewHAR, ViewSOL, WinGEM, etc, will not run natively (they may run under, say, WINE).
Certain utility programs are written in Delphi and are Windows-only (eg, gdx2har). Usually there is a work-around using a portable (Fortran-based) alternative.

A cautionary note

You may well succeed in writing a GUI to run and monitor GEMPACK programs. The program should run fine on your PC, and maybe on the PCs of your office colleagues. But will it run under a restrictive corporate environment, or under Chinese versions of Windows, or with a full hard disk? To ensure robustness and portability, even amongst Windows systems, requires many refinements, and much testing. Another task is creating an install package, so that you can distribute your system.

Redistribution

In many cases you could redistribute a complete system containing TABLO-generated programs and other GEMPACK programs, and users could run this system without any GEMPACK licence. In other cases, where the redistributed model was "large", users might need the cheapest form of GEMPACK licence. For more about the redistribution policy, see here.