Introduction

The aim of this short introduction is to help trace_suite developers outside DLR to use the standard quality management tool Pavayo used and developed at DLR. The Python script ‘pavayo.py’ enables to launch, post-process and summarize automatically a large number of test cases. The test cases are specified using the XML format and are sorted in test levels.

Please note that all test cases are stored at DLR. Consequently, the download capability of Pavayo cannot be used outside of the DLR network. Therefore, external users must either receive the test cases from DLR or create their own. If they want to set up their own download server, a few variables have to be changed and the directory concept of DLR has to be used.

For support and bug reports please contact trace-info@dlr.de.

System requirements

Python package dependencies will be automatically installed.

Additional modules, which can not be automatically installed are:

  • Tecplot360 as ‘tec360’

  • Latex (pdflatex)

  • pysvn [optional]

Setup

  1. Download the MOJO package from the TRACE team site and untar the archive ($tar -xf MOJO.tar.gz).

  2. Install MOJO package and all dependent python packages using $source mojo-env.sh [--venv <name-of-virtual-env-directory>] in ‘mojo’ directory. ‘<name-of-virtual-env-directory>’ can be an existing environment or a non-existing path. If omitted a virtual environment under a default name is generated in the current working directory. This will automatically add MOJO to the PYTHONPATH and the executable scripts, e.g. pavayo.py to the PATH.

Note

A manual installation can be done via $pip install <path-to-MOJO>.

Warning

It is recommended to use $pip install only in a virtual environment or at least install packages only for your user, i.e. use option '--user'.

Changelog

3.0.30

  • MOJO was restructured and is now a python package which can be installed with pip

    • Executable scripts were moved to the bin folder

  • MOJO uses Python 3

  • Many interfaces were cleaned up, often removing to specific default parameter

  • Add support for queuing system slurm

  • Add support of DLR cluster CARA

  • The gmcplay module was extended by some commands

  • tecplot * Add tecplots VARAUXDATA functionality to TecplotDataset * Add convenience functions to instantiate tecplot dataset directly from a file: TecplotDataset.fromFile() * Add convenience functions to load a list of flattened numpy arrays directly from tecplot dataset: TecplotDataset.getFlattenedZoneDataFromFile()

  • New performance benchmarking tools in mojo.performance

  • pavayo.py:

    • Introducing –unusedLiteratureDatabase allowing test sutie runs without literature database

    • Set default weights for every job

    • More compact progress visualization in an interactive console

    • Writing failedJobs.json instantly during a pavayo run, allowing to debug failed test cases before pavayo has finished the whole suite

    • New –includeSanitizerLogs option parsing log files for GCC sanitizer output and including it in reports

    • Dump test case runtimes using the all new –saveTestCaseRuntimes option

    • Integrate PyMesh as available executable, allowing test case setups with a meshing stage

    • Add reference checks for tecplot-style ascii files, cgns files and json files configurable via testsuite xml files

  • Use gitpython to fetch the literature data base

  • contrib:

    • new analyzeTRACEMetadata.py script allows to analyze test suites and selectivly execute test cases using pavayo

    • new applyJournal.py to apply a specified gmcPlay journal to multiple CGNS files

2.0.49

  • Tecplot: Allow arbritrary characters in file header and zone title

  • Allow wild cards when selecting test cases

  • User defined executables in pavayo

  • Allow multiple Tecplot macro executions in one directory

  • Cluster: Jobs for help information should be executed on the master

  • Test suite report may show that test case is modified even if this is not the case

  • pavayo should work without reference to a svn repository

  • New method to create file listings of folders to be displayed in the console using colors and indentations

  • Tool that converts basic LaTeX to Doxygen syntax

  • kill job started with trace.py

  • trace.py to start TRACE with threads

  • Support for qsub walltime limit in qSubJob, trace/prep/post.py

  • JobManagement: add a job executed as a shell job but reserving a given number of processors

  • Facilitate handling of test suite reports in case of additional pages due to test case errors

  • bugs in tecplotData.py

  • Testsuite run with coverage

  • reactivate valgrind feature in pavayo.py

  • Read 1D cellcentered Tecplot data

  • Analysis of time samples

Mandatory changes in MOJO

Under MOJO/src/mojo/jobManagement/management/clusterInfo.py

  1. Complete the list of nodes with the nodes that are available on your cluster according to the following pattern. An example is given for the westmere nodes:

    # cluster node types
# [type(#threads, #coresPerSocket, #threadsPerSocket)]
__westmere = ['w', [24, 6, 12]]
Complete the corresponding dictionary.
cluster_node_types = {'w'  : __westmere, 'westmere'    : __westmere,
                 …
}

  2. Complete the list of supported cluster with the name of your own cluster:

    supportedClusters = ["SCL-EIP", "PROTEC",…]

  3. Complete the class ClusterInfo

    • A new command line for your cluster must be added similarly to _SCL_EIP_COMMAND_LINE

    • Modify the method getCommandLine() by including your cluster and make sure that the method _clustername() works properly.

    • When this is done, add MOJO/import to the PYTHONPATH of your system.

If you want to use the scripts ‘trace.py’, ‘prep.py’ and ‘post.py’ you shoud also consider the following step

  1. Set default node type for the command line parser

    Change the default value of the node type to the abbreviation of your cluster in method commandLineClusterOptionParser.checkSetup() (MOJO/src/mojo/traceSuiteControl/commandLineParser.py):

    self.options.nodeTypeOption.value = "ib"

In order to check if MOJO is now working, try the command pavayo.py -h. The help menu of pavayo.py should be printed on your screen.