Test and Validation Environment (pavayo)

Usage of Pavayo

Automated test utility for TRACE and its tools beginning starting with version 9.0

$ pavayo.py -h
usage: pavayo.py [-h] [-i PATH] [-w PATH] [--testLevel LEVEL [LEVEL ...]] [--basePath KEY [PATH ...]]
                 [--traceSuite PATH] [--trace PATH] [--prep PATH] [--post PATH] [--addExecutable KEY [PATH ...]]
                 [-g PATH] [--noFixedGmcPlay] [--pymesh PATH] [--noFixedPyMesh] [--debug]
                 [--passCommandLineArgument EXECUTABLE ARGUMENT] [-tbp] [-d] [-c] [-p] [-l] [-a] [--skipReferenceChecks]
                 [--show [{testcases,keywords}]] [--showSelected [{testcases,keywords,status}]] [-o PATH]
                 [--forceReferencePlots] [--evaluateReferences]
                 [--selectiveCompute SELECTIVE_COMPUTE [SELECTIVE_COMPUTE ...]]
                 [--step {reset,gmc,prep,trace,post} [{reset,gmc,prep,trace,post} ...]] [-t TESTCASE [TESTCASE ...]]
                 [-T] [-A ACCOUNT] [--queue QUEUE] [-n {ib,w,sb}] [--np NUM] [--numberOfThreadsPerProc NUM]
                 [--numberOfProcsPerNode NUM] [--allowFewerProcs] [--deactivateClusterJobGrouping]
                 [--slurmSwitches SLURMSWITCHES] [--literatureDatabase PATH] [--unusedLiteratureDatabase]
                 [-m MAILADDRESS] [-f] [--maxPriority MAX_PRIORITY] [--confidentiality ALLOW [ALLOW ...]]
                 [--keyword KEYWORD [KEYWORD ...]] [--exclude] [--knownErrorsFile KNOWN_ERRORS_FILE]
                 [--knownErrorsSuiteName KNOWN_ERRORS_SUITE_NAME] [--updateReferenceData] [--updateRestartData]
                 [--updateMeshData] [--prepareTraceSuiteXMLforUpload] [-u PATH] [--failedJobsFile PATH]
                 [--junitXml PATH] [--retriesComputation N_RETRIES]
                 [--releaseReports RELEASEREPORTS [RELEASEREPORTS ...]]
                 [--singleTecplotLicense | --ntecplot TECPLOTLICENSE] [--valgrind]
                 [--valgrindExec VALGRIND_EXEC [VALGRIND_EXEC ...]] [--valgrindSuppFile VALGRIND_SUPP_FILE]
                 [--includeSanitizerLogs] [--sshUser SSHUSER] [--saveTestCaseRuntimes] [--verbosity VERBOSE]
                 [--additionalProcessStatus] [--outputOnError] [--version]

Perform a test on TRACE.

optional arguments:
  -h, --help                            show this help message and exit
  -i PATH, --input PATH                 read the selected XML file
  -w PATH, --workingDirectory PATH      path for the working directory
  --literatureDatabase PATH             Specify the path to the checked out literature data base
  --unusedLiteratureDatabase            Mark the database as unnecessary for the run. It will not be downloaded.
  -m MAILADDRESS, --mail MAILADDRESS    Mail address to send mail to when job is finished
  -f, --forceUpdate                     force update of all specified test cases, only available on test level 2 and 3
  --releaseReports RELEASEREPORTS [RELEASEREPORTS ...]
                                        Generate additional validation reports in consideration of the confidentiality
  --verbosity VERBOSE                   level of verbosity (0 - minimal output, 1 - Jenkins, 2 - default, 3 - debug)
  --additionalProcessStatus             activates further process status output (i.e. RUNNING..., DONE...) as formerly
                                        known
  --outputOnError                       write the output of failed jobs to stdout if available
  --version, -v                         show program's version number and exit

Test case selection:
  --testLevel LEVEL [LEVEL ...]         The test level you want to execute (3 (default) = test suite, 5 = validation)
  -t TESTCASE [TESTCASE ...], --testcase TESTCASE [TESTCASE ...]
                                        select specific test cases
  -T, --testcasesFromStdin              select specific test cases by reading them from stdin
  --maxPriority MAX_PRIORITY            maximal priority of test cases to run; only available for the test suite
  --confidentiality ALLOW [ALLOW ...]   select only test cases which are free for the given group
  --keyword KEYWORD [KEYWORD ...]       only handle test cases with (or without; see '--exclude') this keyword
  --exclude                             exclude test cases with the keywords specified via '--keyword'
  --knownErrorsFile KNOWN_ERRORS_FILE   Path to JSON file containing known errors
  --knownErrorsSuiteName KNOWN_ERRORS_SUITE_NAME
                                        Name of the suite to use from the knownErrorsFile

Executables:
  --basePath KEY [PATH ...]             list of base paths ({key}=<path> or {key} [path]
  --traceSuite PATH                     path to the TRACE suite folder
  --trace PATH                          path of the TRACE executable
  --prep PATH                           path of the PREP executable
  --post PATH                           path of the POST executable
  --addExecutable KEY [PATH ...]        path to additional executables ({key}=<path> or {key} [path]
  -g PATH, --gmcplay PATH               path of the GMC_PLAY executable
  --noFixedGmcPlay                      gmcPlay version may differ from the version stated in the XML file
  --pymesh PATH                         path of the PyMesh executable
  --noFixedPyMesh                       PyMesh version may differ from the version stated in the XML file
  --debug                               use the *debug versions of the TRACE suite executables; only works in
                                        combination with '--traceSuitePath'
  --passCommandLineArgument EXECUTABLE ARGUMENT
                                        Specify a flag for an executable; Only works for the test level 3; NB: Do not
                                        include the leading dashes; only works for long options
  -tbp, --useThreadsInsteadOfProcesses  Use shared memory parallelization only for TRACE; Warning: this may lead to
                                        invalid configurations!

Execution (at least one option mandatory):
  -d, --download                        download the test cases
  -c, --compute                         do the computation
  -p, --postProc                        do the post processing and check the results against references
  -l, --latex                           create the report file
  -a, --autorun                         perform all steps and do it without interaction
  --skipReferenceChecks                 skip check of tolerances of results against references
  --show [{testcases,keywords}]         show available test cases (By adding the optional argument keywords the test
                                        cases are listed by keywords.)
  --showSelected [{testcases,keywords,status}]
                                        show the selected test cases (By adding the optional argument keywords the test
                                        cases are listed by keywords.)
  -o PATH, --outputTestcases PATH       select output file for test case list
  --forceReferencePlots                 force the creation of reference plots even if checks succeed
  --evaluateReferences                  compare current against original references without running the testcases
  -u PATH, --uploadTestcases PATH       Upload test cases. Don't forget to specify a file with a check-in message.
  --failedJobsFile PATH                 Specify file for failed jobs output.
  --junitXml PATH                       Specify file for a JUnit compatible XML file.
  --retriesComputation N_RETRIES        Number of times every computation is retried.

Sub-set for the 'compute' step (expert options):
  --selectiveCompute SELECTIVE_COMPUTE [SELECTIVE_COMPUTE ...]
                                        use the specified list of job names
  --step {reset,gmc,prep,trace,post} [{reset,gmc,prep,trace,post} ...]
                                        execute the specified steps for validiation test cases

Job management:
  -A ACCOUNT, --account ACCOUNT         Account for cluster costs
  --queue QUEUE                         queue for the queueing system when running on a cluster
  -n {ib,w,sb}, --nodeType {ib,w,sb}    Choose a node type
  --np NUM                              number of processes
  --numberOfThreadsPerProc NUM          number of threads per processor for all TRACE jobs
  --numberOfProcsPerNode NUM            number of processors per node for all TRACE jobs
  --allowFewerProcs                     If there are less processes available than a job asks for use fewer processes
  --deactivateClusterJobGrouping        deactivates simultaneous execution of small jobs on shared cluster nodes
  --slurmSwitches SLURMSWITCHES         <count>[@minutes] requests a node allocation with at most <count> network
                                        switches within <minutes> by SLURM

Modifying test cases and prepare Trace Suite for check in:
  --updateReferenceData                 update reference data of selected test cases: only available for the test suite
  --updateRestartData                   update restart data of selected test cases: only available for the test suite
  --updateMeshData                      update mesh data of selected test cases and activates --updateRestartData: only
                                        available for the test suite
  --prepareTraceSuiteXMLforUpload       modify the 'referenceTraceSuite' version string in the <testcase>.xml as
                                        prepararation for the check-in according to changes in the test cases

Tecplot:
  --singleTecplotLicense                only one tecplot license is available
  --ntecplot TECPLOTLICENSE             specify arbitrary number of Tecplot licenses

Code Analysis:
  --valgrind                            run all executables under observation of valgrind (for a specific executable use
                                        option 'valgrindExec')
  --valgrindExec VALGRIND_EXEC [VALGRIND_EXEC ...]
                                        specify which executables are observed by their exact name
  --valgrindSuppFile VALGRIND_SUPP_FILE
                                        specify your own valgrind suppression file
  --includeSanitizerLogs                Include the logs from GCC sanitizers in the report

SVN:
  --sshUser SSHUSER                     Specify a different SSH username for accessing the Testcase SVN

Metrics:
  --saveTestCaseRuntimes                Dump TestCase runtimes to stdout and to testCaseTimings.json

Testsuite and validation management

As part of the design merge of test suite and validation test cases the controls in Pavayo have been revised. All test case configurations are now stored in an XML file. They can be accessed by the different test level (3- test suite, 5 - validation)

The easiest way to run the test suite for a specified Trace suite is the following:

pavayo.py --traceSuite ../trace_suite -a

Note

By specifying a Trace suite only the executables and the XML file are set implicitly. If no number of processors is given all available procs are used. The test level is set to leve 3 (test suite) as default. The default XML file is stored in ‘distrib/testing/standard/TRACE_SUITE.xml’ (check mojo.pavayo.computeData.py).

TRACE, PREP and POST binaries as well as the XML file can also be specified individually:

pavayo.py -i TEST.xml --trace $TRACE --prep $PREP --post $POST --np 24 -dcpl

Note

Omission of executables is no longer possible and will result in an error.

To force the download of test cases the option ‘-f/–forceUpdate’ must be added:

pavayo.py --traceSuite ../trace_suite -t bump_hybrid cuboids -d -f

Test levels

To start the validation the test level has to be set to level 5:

pavayo.py --traceSuite ../trace_suite -a --testLevel 5

Test case selection

The list of test cases of an XML file will be displayed by:

pavayo.py --traceSuite ../trace_suite --show
pavayo.py -i TEST.xml --show

Single test cases or a group of test cases can be selected by adding the following option:

pavayo.py --traceSuite ../trace_suite -a -t bump_hybrid cuboids

If the user is uncertain what the name of the test case is exactly asterisks can be used as wildcards:

pavayo.py --traceSuite ../trace_suite -a -t 'bump_*'
pavayo.py --traceSuite ../trace_suite -a -t bump_\*

Warning

Quotation marks should be used in these cases. Otherwise the wildcards could be replaced by a list of valid files before the arguments are handed to Python.

A group of test cases can be seperated by whitespaces and/or commas:

pavayo.py --traceSuite ../trace_suite -a -t bump_hybrid, cuboids POST_bump_hybrid,NACA0012_DG

For a listing of the selected test cases the following command can be used:

pavayo.py -i TEST.xml --showSelected

Keywords can be specified to select test cases:

pavayo.py --traceSuite ~/workspace/trace_suite -d --keyword HB PREP

By adding the argument ‘keywords’ to the ‘show’ options a listing of the keywords will be generated:

pavayo.py -i TEST.xml --show keywords
pavayo.py -i TEST.xml --showSelected keywords -t cuboids MT1

By adding the argument ‘status’ to the ‘showSelected’ option the selected test cases are listed based on their SVN status.

pavayo.py -i TEST.xml –showSelected status

Note

Here the test case folder in the current suite folder are checked. So you must be in the correct working directory.

Selective Compute

Since the compute steps can become extremely complex options have been added to execute a subset of the computation step.

For the validation test cases the argument ‘–step’ can be used. It subdivides the computation in five substeps:

• reset: reset directories in the computation folder

• gmc: setup of the CGNS files using gmcPlay

• prep: execute all jobs in the pre-processing list

• trace: compute the TRACE jobs

• post: execute all jobs in the post-processing list and compare against references

The user can specify an arbitrary subset:

pavayo.py --compute --step reset,gmc
pavayo.py --compute --step post

Note

‘reset’ does also work for test suite test cases.

For test suite test cases the argument ‘–selectiveCompute’ is introduced. Here the names of simulation groups, simulations or jobs can be specified.:

pavayo.py --compute --selectiveCompute Deformation

Additional Executables

If a test case needs further executables they can be specified using the argument ‘–addExecutable’ by specifying a name and the path to the executable. In the XML files a leading ‘$’ has to be added to the key. The key is case sensitive. Either ‘=’ or spaces can be used:

pavayo.py --traceSuite ~/workspace/trace_suite --addExecutable echo=/bin/echo
pavayo.py --traceSuite ~/workspace/trace_suite --addExecutable echo /bin/echo --addExecutable echo2 /bin/echo

Modification of test cases

To update test suite test cases the following options can be added:

pavayo.py --traceSuite ../trace_suite -t cuboids -cpl --updateRestartData --updateReferenceData

The command ‘–prepareTraceSuiteXMLforUpload’ updates the Trace Suite reference version of all updated test cases in the XML files and adjusts the revision number in the main XML file.:

pavayo.py --traceSuite ../trace_suite --prepareTraceSuiteXMLforUpload
pavayo.py --traceSuite ../trace_suite --input TEST.xml --testLevel 4 --prepareTraceSuiteXMLforUpload

..note::

The test level has to be specified if it is not the default level.

..warning::

The option ‘–testcase’ will be ignored.

A text file has to be given to upload the changes in a test case:

pavayo.py --traceSuite ../trace_suite -t cuboids -u upload.txt

Structure of Pavayo

The compute package offers all the classes and methods, which are necessary to compute the test or validation suite. To use the compute package use pavayo.py in the bin folder or import single files or classes into a session in the interactive python console.

compute.py regulates the overall execution of the test or validation suite. computeData.py contains all constants, which are necessary for some of the methods in compute.py. computeMethods.py offers some methods for compute.py which are not related directly to the overall procedure. computeOptions.py contains the option parser for pavayo.py.

In testcase are located all modules which are necessary to read a test case from a XML test suite definition file.

For further description on the modules look into their respective documentation.

Executables

To use arbritrary executables within Pavayo the executables can be stored in the XML. Based on these settings a dictionary of executables will be created and used.

Modules in executables

List of executables

In the class ExecutableResources all relevant executables are stored. Using this class Pavayo becomes independent of the settings for the TRACE Suite.

class ExecutableResources

Class to manage a set of executables

addExecutable(environmentName='', path='', commandLineArguments=None, numberOfLicenses=None, maxProcsPerExecution=None, displayVersion=False, successStatement=None, failStatement=None, ignoreExitCode=False, versionCheck=True, versionCheckCommand=None, explicitlyRequested=False)

Add a single executable to the dictionary.

Parameters

• environmentName (str) – environment name

• path (str) – path to executable

• commandLineArguments (list(str) or None) – additional command line arguments

• numberOfLicenses (int or None) – number of available licenses

• maxProcsPerExecution (int or None) – maximum number of processors per execution

• displayVersion (bool) – flag whether executable is shown in screen output and report

• successStatement (list or None) – list of success statements

• failStatement (list or None) – list of fail statements

• ignoreExitCode (bool) – flag whether the exit code will be ignored

• versionCheck (bool) – flag whether version should be checked

• versionCheckCommand (str or None) – command line argument for version check

• explicitlyRequested (bool) – command was requested explicitly, this may enforces stricter checks

addMultipleExecutables(searchPath, environmentName='', commandLineArguments=None, numberOfLicenses=None, maxProcsPerExecution=None, displayVersion=False, successStatement=None, failStatement=None, ignoreExitCode=False, versionCheck=True, replaceSuffix='')

Add one or more executables to the dictionary by searching a given path.

Parameters

• searchPath (str) – path where to search for executables

• environmentName (str) – environment name

• commandLineArguments (list(str) or None) – additional command line arguments

• numberOfLicenses (int or None) – number of available licenses

• maxProcsPerExecution (int or None) – maximum number of processors per execution

• displayVersion (bool) – flag whether executable is shown in screen output and report

• successStatement (list or None) – list of success statements

• failStatement (list or None) – list of fail statements

• ignoreExitCode (bool) – flag whether the exit code will be ignored

• versionCheck (bool) – flag whether version should be checked

• replaceSuffix (str) – suffix to be removed from the end of the executable name

static getKeyAndEnvironmentName(executableName)

Get key and environment variable name from name of executable

Parameters

executableName (str) – name of executable

Returns

key to executable in upper case, name for environment variable in upper case

Return type

str, str

getVersion(options=None)

Determines the versions of all executables in the resource dictionary

Parameters

options (None or argParse.ArgumentParser) – options parsed from the command line

updateDictFromXML(xmlListOfExecutables: mojo.pavayo.testcase.testcase_generated.ListOfExecutables, basePathDict: Mapping[str, str], replaceSuffix: str = '')

Fills the resource dictionary using the resource fabric class. Replaces -1 with sys.maxint for easier handling in the other methods of this class.

Parameters

• xmlListOfExecutables – a dictionary containing names and number of resources

• basePathDict – base path dictionary

• replaceSuffix – suffix to be removed from the end of the executable name

writeToJSON(fileName='executables.cfg')

Writes the executables to a JSON file :param fileName: file name :type fileName: str

Settings for an executable

In the class ExecutableResources all relevant executables are stored. Using this class Pavayo becomes independent of the settings for the TRACE Suite.

class SingleExecutableResource(environmentName='', path='', commandLineArguments=None, numberOfLicenses=None, maxProcsPerExecution=None, displayVersion=False, successStatement=None, failStatement=None, ignoreExitCode=False, versionCheck=True, versionCheckCommand=None)

Settings for a single executable

Parameters

• environmentName (str) – environment name

• path (str) – path of executable

• commandLineArguments (list(str) or None) – additional command line arguments

• numberOfLicenses (int or None) – number of available licenses

• maxProcsPerExecution (int or None) – maximum number of processors per execution

• displayVersion (bool) – flag whether executable is shown in screen output and report

• successStatement (list or None) – list of success statements

• failStatement (list or None) – list of fail statements

• ignoreExitCode (bool) – flag whether the exit code will be ignored

• versionCheck (bool) – flag whether version should be checked

getVersion(options=None)

Determines the versions of the executable

Parameters

options (argParse.ArgumentParser) – options parsed from the command line

property version

Getter for the versin of the executable

Returns

version

Return type

StrictVersion or LooseVersion or str

Settings for an executable

In the class ExecutableResources all relevant executables are stored. Using this class Pavayo becomes independent of the settings for the TRACE Suite.

class Version(versionArgument=None)

Settings for the version of an executable.

getVersion(path)

Get the version of the given executable

Parameters

path (str) – path of executable

Returns

version of executable

Return type

StrictVersion or LooseVersion

callSoftwareVersionOption(software, versionArgument='--version')

Calls <SOFTWARE> –version. Returns the output string.

Parameters

• software (str) – path of the executable

• versionArgument (str) – argument to get the version

Returns

output of the call

Return type

str

getSoftwareVersionNumberString(software, versionArgument='--version', versionKey='version')

Get just the version number from the output of <SOFTWARE> <VERSION_ARGUMENT>.

Parameters

• software (str) – path of the executable

• versionArgument (str) – argument to get the version

• versionKey (str) – key to find version string in output

Returns

version string

Return type

str

is_version_str_equal(version1: str, version2: str) → bool

Compare to version strings piece by piece

Parameters

• version1 – first version string

• version2 – second version string

Returns

flag whether versions are identical

Test Case Types

Several different test case designs are available based all on the abstract test case. New test case types must be stored added to the XSD files and stored in this folder.

Concept on the folder structure

As part of the design merge of test suite and validation test cases, the following concept on the folder structures of test cases was developed.

Each test case must comprise the following folders:

• analysis:

  Tecplot layouts and Python analysis scripts are located in this folder. These files will be executed in their actual location. So all paths used in the files have to be relative to that folder.

• sources:

  All data files required to run the computation are stored in this folder. The files are either copied or linked to the input folder of the related computation folder. Files that will be modified have to be copied.

• references:

  All reference data is stored here. Reference compared to the current computation has to be located in the same same path as in the computations folder. This is very important for the test suite test cases.

• computations:

  All calculations are performed in this folder split in several sub folders.

• figures:

  The figures and latex fragments to be included in the report have to be located in this folder.

• logs:

  Here the screen output, one file per job, is saved.

Note

The analysis, sources and references folders have to be created for each test case. They are usually the only folders that are part of the SVN repository.

Warning

Currently, the validation test cases (test level 5) do not follow this concept.

abstractTestCase

abstractTestCase contains a abstract class all test case classes inherit from. Its constructor initializes the ConfigParser instance for the error handling and sets default values for some attributes.

It forces the inheriting classes to implement the methods PAVAYO needs to execute a test suite. For some methods it implements a default behaviour. This methods have to be overwritten if a test case wants to do something in the case this method is called.

test case classes

The test case classes are defined in an XML Schema Document (XSD) file. This file is located in the TRACE suite in the contrib folder. Therefore you have to commit into the TRACE suite if you want to adapt the test case structures.

The python test case classes are generated by the tool ‘generateDS’. generateDS is available from the python package index. The generated file is located in MOJO/import/pavayo/testcase/testcase_generated.py.

Warning

Never change testcase_generated.py manually!

The generated module contains python classes for every XML/XSD object with its particularly attributes. The test other test case classes in the testcase package inherit from these generated classes and add some methods unique to the particularly test case type. testSuiteComponents contains all the other subclasses from the test case definitions.

Usage

If you want to alter the test case definition change the XSD file first. Before continuing with further tests adapt the XML file and test if it is still valid. Now switch directories to MOJO/import/pavayo/testcase.

Execute generateDS here. For the command line look into the current version of testcase_generated.py. If your doing things right the tool will ask if to overwrite testcase_generated.py. Answer with yes (y).

Modules in testcase

validationSuiteParser

The validationSuiteParser module provides methods to read and write XML files, according to the structure in testcase_generated.py.

The parsePavayo method can be used to parse an XML file which fulfills this structure. The method writeXmlFile a list of test cases can be packed into a suite.

parsePavayo(inFilename, silence=True)

Parses the XML file, applies some operations necessary for PAVAYO and returns stuff in PAVAYO format.

Parameters

• inFilename (str) – name of the XML file to parse

• silence (bool) – prints the parsed XML file if true

Returns

tuple containing a list of all read test cases and the parsed suite object

Return type

tuple(list(AbstractTestCase), Suite)

writeXmlFile(TClist=None, suiteName='unknown_suitename', mojoversion='-1', exit_file='MyTTCML')

Writes the given test case list to the exit_file.

Parameters

• TClist (list(AbstractTestCase)) – list of test cases for the suite

• suiteName (str) – name of the test suite

• mojoversion (str) – current MOJO version (default: -1)

• exit_file (str) – path to the XML file to be generated

generalTestCase.py

generalTestCase offers the GeneralTestCaseSub class. The class wants to represent each test case which can be described as dependencies between jobs. Jobs in these context are atomical executables to be called from the command line or single python functions.

As first test case type it implements two methods to get respectively a job list for the computation part and for the post processing of the test case. Plans are to implement these methods for every test case type so PAVAYO does not have to care about the type of a test case.

GeneralTestCase aims to replace ParallelTestCase in the midterm.

class GeneralTestCaseSub(name=None, publicName='', author=None, email=None, confidentiality=None, testLevel=None, keywords=None, inactive=0, description=None, figures=None, references=None, referenceTraceSuite='', originalReferenceRevision=None, numProcsLocal=1, numProcsCluster=1, priority=1, baseCompute='.', compute=True, rowList=None, jobList=None, gds_collector_=None, **kwargs_)

Class to represent a general test case. You can iterate over the job lists of a instance.

All possible parameters to the constructor of this class can be found in the XSD file used to generate testcase_generated.py.

append(jobListObject)

Appends a job list to this test case

Parameters

jobListObject (TestcaseJoblist instance) – a test case job list to append

getComputationJobList(options, executableDict, resourcesDict=None)

Collects all job objects from the jobs in the current test case instance and creates a job list of them.

Parameters

• options (ArgumentParser) – options instance to extract the executables and the queue to use

• executableDict (ExecutableResources) – dictionary of executables

Returns

job list of all computation jobs

Return type

JobManagementJobList

getMeshingJobList(options, executableDict, resourcesDict=None)

Collects all job objects from the jobs in the current test case instance and creates a job list of them.

Parameters

• options (ArgumentParser) – options instance to extract the executables and the queue to use

• executableDict (ExecutableResources) – dictionary of executables

Returns

job list of all update-restart jobs

Return type

JobManagementJobList

getPostprocessingJobList(options, executableDict, resourcesDict=None)

Returns a job representing the post script.

Parameters

• options (ArgumentParser) – options instance to extract the executables and the queue to use

• executableDict (ExecutableResources) – dictionary of executables

Returns

job list of all post-processing jobs

Return type

JobManagementJobList

getRestartJobList(options, executableDict, resourcesDict=None)

Collects all job objects from the jobs in the current test case instance and creates a job list of them.

Parameters

• options (ArgumentParser) – options instance to extract the executables and the queue to use

• executableDict (ExecutableResources) – dictionary of executables

Returns

job list of all update-restart jobs

Return type

JobManagementJobList

getUsedExecutables(compute=False, restart=False, meshing=False, postprocessing=False)

Returns the version of a software the test case was computed with.

Parameters

• compute (bool) – add executables of compute step to set?

• restart (bool) – add executables of restart step to set?

• meshing (bool) – add executables of meshing step to set?

• postprocessing (bool) – add executables of postprocessing step to set?

Returns

the used executables

Return type

set(str)

insert(index, value)

Inserts a job list to this test case

Parameters

• index (integer) – place to insert in

• value (TestcaseJoblist instance) – a test case job list to append

ValidationTestCase

The ValidationTestCase is designed for the computation and evaluation of several setup including several operating point lists. This test case class uses the same folder structure as the TestSuiteTestCase. Related test cases comprise the sources folder and the analysis folder. For the computation the data is linked or copied from the sources folder to the computation folder.

Each operating point is considered as independent. Thus there are no dependencies between operating points possible. Before the TRACE computation is started the control file can be modified and several settings in the CGNS file can be set via gmcPlay. When the computation is done post-processing jobs can be executed.

Warning

For the post-processing jobs the base folder is the CGNS folder. If a CGNS directory is defined in the setup it will be set via gmcPlay and it will be used here. Otherwise it is expected that the CGNS file are at the default location (‘../output/cgns/).

class ValidationTestCaseSub(name=None, publicName='', author=None, email=None, confidentiality=None, testLevel=None, keywords=None, inactive=0, description=None, figures=None, references=None, referenceTraceSuite='', originalReferenceRevision=None, numProcsLocal=1, numProcsCluster=1, priority=1, baseCompute='.', compute=True, rowList=None, setup=None, restartSetup=None, plots=None, analysisGroup=None, extensiontype_=None, gds_collector_=None, **kwargs_)

Represents a speedline test case. Inherits from collections. Sequence so you can iterate over its operation point lists.

All possible parameters to the constructor of this class can be found in the XSD file used to generate testcase_generated.py.

append(setup)

Appends a given setup to the test case.

Parameters

setup (SetupSub) – setup to append to the testcase

createPlot(plot, verbose=True)

Creates a plot

Parameters

• plot (supermod.MatplotlibFigure) – plot instance

• verbose (bool) – verbosity

find(name)

Finds a given setup to the test case.

Parameters

name (str) – name of the setup searched for

Returns

setup instance

Return type

SetupSub

findSubsetInJobList(nameList, step='COMPUTATION')

find a subset in a job list using a list of keywords

Parameters

• nameList (list of str) – list of keywords

• step (str) – working step

Returns

dictionary of all jobs

Return type

collections.defaultdict

static getArguments(cgnsFilePath, opList, op)

Static method to retrieve the basic settings for the TRACE command line.

Parameters

• cgnsFilePath (str) – path to the CGNS file

• opList (ValidationOpListSub) – current operating point list

• op (ValidationOpSub) – current operating point

Returns

TRACE command options as a list

Return type

list()

getComputationJobList(options, executableDict, resourcesDict=None)

Returns a job list containing all jobs for the computation step of the test case.

Parameters

• options (Argparse) – argparse instance of the current PAVAYO run

• executableDict (ExecutableResources) – dictionary of executables

• resourcesDict (dict) – resource dictionary

Returns

job list representing the computation of this test case or None

Return type

JobList or None

static getPostCommands(options, executableDict, testCaseJobList, opList, op, opLastJobId, cgnsDir, outputFileDir, resourcesDict=None)

Static method to add the post-processing commands to the job list.

Parameters

• options (Namespace) – pavayo options

• executableDict (ExecutableResources) – dictionary of executables

• testCaseJobList (list) – job list of a test case

• opList (ValidationOpListSub) – current operating point list

• op (ValidationOpSub) – current operating point

• opLastJobId (list) – index list of the last jobs (used for dependencies)

• cgnsDir (str) – start directory

• outputFileDir (str) – directory for the log files

• resourcesDict (dict) – resource dictionary

Returns

list of last jobs (used for dependencies)

Return type

list

getPostprocessingJobList(options, executableDict, resourcesDict=None)

Returns a job containing the post script of this test case.

Parameters

• options (argparse) – Argparse instance of the current PAVAYO run

• executableDict (ExecutableResources) – dictionary of executables

• resourcesDict (dict) – resource dictionary

Returns

job list representing the post-processing job list

Return type

JobList or None

static getPrepCommands(options, executableDict, testCaseJobList, opList, op, opLastJobId, inputDir, outputFileDir, resourcesDict=None)

Static method to add the prep-processing commands to the job list.

Parameters

• options (Namespace) – pavayo options

• executableDict (ExecutableResources) – dictionary of executables

• testCaseJobList (list()) – job list of a test case

• opList (ValidationOpListSub) – current operating point list

• op (ValidationOpSub) – current operating point

• opLastJobId (list()) – index list of the last jobs (used for dependencies)

• inputDir (str) – input directory

• outputFileDir (str) – directory for the log files

• resourcesDict (dict) – resource dictionary

Returns

list of last jobs (used for dependencies)

Return type

list

getRestartJobList(options, executableDict, resourcesDict=None)

Returns a job list containing all jobs for the restart step of the test case.

Parameters

• options (Argparse) – argparse instance of the current PAVAYO run

• executableDict (ExecutableResources) – dictionary of executables

• resourcesDict (dict) – resource dictionary

Returns

job list representing the computation of this test case or None

Return type

JobList or None

plotConvergenceReport(fig, convergenceReport)

Creates a convergence plot

Parameters

• fig (FigureWrapper) – handle of matplotlib figure

• convergenceReport (supermod.ConvergenceReport) – settings for convergence plot

plotPerformanceMap(fig, performanceMap)

Creates a performance map

Parameters

• fig (FigureWrapper) – handle of matplotlib figure

• performanceMap (supermod.PerformanceMap) – settings for performance map

plotRadialDistribution(fig, radialDistribution)

Creates a plot of the radial distribution

Parameters

• fig (FigureWrapper) – handle of matplotlib figure

• radialDistribution (supermod.RadialDistribution) – settings for radial distribution

plotResidualReport(fig, residualReport)

Creates a residual plot

Parameters

• fig (FigureWrapper) – handle of matplotlib figure

• residualReport (supermod.ResidualReport) – settings for residual plot

ValidationTestCaseSpeedline

The test case class ValidationTestCaseSpeedline is derived from the class ValidationTestCase. The only difference between the two test case classes is the processing order of the operating points. Here it is assumed that all operating points in one operating point list depend on each other. So they are executed serially.

class ValidationTestCaseSpeedlineSub(name=None, publicName='', author=None, email=None, confidentiality=None, testLevel=None, keywords=None, inactive=0, description=None, figures=None, references=None, referenceTraceSuite='', originalReferenceRevision=None, numProcsLocal=1, numProcsCluster=1, priority=1, baseCompute='.', compute=True, rowList=None, setup=None, restartSetup=None, plots=None, analysisGroup=None, gds_collector_=None, **kwargs_)

Represents a speedline test case. Inherits from collections. Sequence so you can iterate over its operation point lists.

All possible parameters to the constructor of this class can be found in the XSD file used to generate testcase_generated.py.

getComputationJobList(options, executableDict, resourcesDict=None)

Returns a job list containing all jobs for the computation step of the test case.

Parameters

• options (Argparse) – argparse instance of the current PAVAYO run

• executableDict (ExecutableResources) – dictionary of executables

• resourcesDict (dict) – resource dictionary

Returns

job list representing the computation of this test case or None

Return type

JobList or None

getRestartJobList(options, executableDict, resourcesDict=None)

Returns a job list containing all jobs for the resart step of the test case.

Parameters

• options (Argparse) – argparse instance of the current PAVAYO run

• executableDict (ExecutableResources) – dictionary of executables

• resourcesDict (dict) – resource dictionary

Returns

the job list representing the computation of this test case or None

Return type

JobList or None

speedlineTestCase.py

speedlineTestCase offers the class SpeedlineTestCaseSub. SpeedlineTestcaseSub represents a test case those computations use the solution of a previous computation as initial solution. At the moment the back pressure is changed between every computation, but other scenarios are imaginable. To change the settings in the CGNS file PAVAYO has to use gmcPlay.

class SpeedlineTestCaseSub(name=None, publicName='', author=None, email=None, confidentiality=None, testLevel=None, keywords=None, inactive=0, description=None, figures=None, references=None, referenceTraceSuite='', originalReferenceRevision=None, numProcsLocal=1, numProcsCluster=1, priority=1, baseCompute='.', compute=True, rowList=None, dependency=None, preScript=None, postScript=None, component=None, typeComponent=None, controlVolume=None, rotor=None, inletPanel=None, outletPanel=None, radialEquilibrium=None, interfacePanels=None, messageBlocks=None, outputDir='', inputDir='', cgnsInDir='', cgnsFile='', cgnsOutDir='', cgnsInterval=None, movie3DPeriod=None, movie3DNPictures=None, movie2DPeriod=None, movie2DNPictures=None, movieOutDir='', residualDir='', residualFile='residual.dat', residualInterval=None, boundaryDir='', boundaryFile='d0', boundaryInterval=None, radialBdDir='', radialBdFile='d1-spanwise', radialBdInterval=None, controlFile='', numCPUs=None, numThreads=0, opList=None, extensiontype_=None, gds_collector_=None, **kwargs_)

Represents a speedline test case. Inherits from collections. Sequence so you can iterate over its operation point lists.

All possible parameters to the constructor of this class can be found in the XSD file used to generate testcase_generated.py.

append(opList)

Appends a given operation point list to the test case.

Parameters

opList (OperationPointList instance) – a operation point list to append to the testcase

getComputationJobList(options, executableDict, resourcesDict=None)

Returns a job list containing all jobs for the computation step of the test case. It starts with a pre script, if existing, followed by the speed lines and the post commands.

Parameters

• options (argparse) – Argparse instance of the current PAVAYO run

• executableDict (ExecutableResources) – dictionary of executables

Returns

job list representing teh computation of this test case or Noen

Return type

JobList or None

getPostCommandJobs(options, executableDict, isParallelTestCase=False, resourcesDict=None)

Generates jobs from the post commands in a test case.

Parameters

• options (argparse) – the Argparse instance of the current run of PAVAYO

• executableDict (ExecutableResources) – dictionary of executables

• isParallelTestCase (bool) – flag to indicate if the instance calling is a parallel test case; in this case the folder strucure is slightly different

• resourcesDict (dict) – resources used by the jobs

Returns

a job list containing all post command jobs

Return type

list of Job

getPostprocessingJobList(*args, **kwargs)

Returns a job containing the post script of this test case.

Parameters

options (Argparse instance) – Argparse instance of the current PAVAYO run

parallelTestCase.py

parallelTestCase offers the class ParallelTestCaseSub, which represents a test case consisting of many independent calculations. It offers the possibility to use a TRACE control file and add lines to this control file.

As soon as GeneralTestCase can handle TRACE control files ParallelTestCase will be replaced.

class ParallelTestCaseSub(name=None, publicName='', author=None, email=None, confidentiality=None, testLevel=None, keywords=None, inactive=0, description=None, figures=None, references=None, referenceTraceSuite='', originalReferenceRevision=None, numProcsLocal=1, numProcsCluster=1, priority=1, baseCompute='.', compute=True, rowList=None, dependency=None, preScript=None, postScript=None, component=None, typeComponent=None, controlVolume=None, rotor=None, inletPanel=None, outletPanel=None, radialEquilibrium=None, interfacePanels=None, messageBlocks=None, outputDir='', inputDir='', cgnsInDir='', cgnsFile='', cgnsOutDir='', cgnsInterval=None, movie3DPeriod=None, movie3DNPictures=None, movie2DPeriod=None, movie2DNPictures=None, movieOutDir='', residualDir='', residualFile='residual.dat', residualInterval=None, boundaryDir='', boundaryFile='d0', boundaryInterval=None, radialBdDir='', radialBdFile='d1-spanwise', radialBdInterval=None, controlFile='', numCPUs=None, numThreads=0, opList=None, gds_collector_=None, **kwargs_)

Class to represent a parallel test case.

All possible parameters to the constructor of this class can be found in the XSD file used to generate testcase_generated.py.

getComputationJobList(options, executableDict, resourcesDict=None)

Collects all job objects from the jobs in the current test case instance and creates a job list of them.

Parameters

• options (ArgparseParser) – options instance to extract the executables and the queue to use

• executableDict (ExecutableResources) – dictionary of executables

• resourcesDict (dict) – resources used by the jobs

Returns

a job list representing this test case or None

Return type

jobList or None

getPostCommandJobs(options, executableDict, isParallelTestCase=True, resourcesDict=None)

Generates jobs from the post commands in a test case.

Parameters

• options (argparse) – the Argparse instance of the current run of PAVAYO

• executableDict (ExecutableResources) – dictionary of executables

• isParallelTestCase (bool) – flag to indicate if the instance calling is a parallel test case; in this case the folder strucure is slightly different

• resourcesDict (dict) – resources used by the jobs

Returns

a job list containing all post command jobs

Return type

list of Job

getPostprocessingJobList(*args, **kwargs)

Returns a job containing the post script of the test case.

Returns

a job containing the post script of the test case

Return type

Job instance

testSuiteTestCase.py

testSuiteTestCase offers the TestSuiteTestCaseSub class. The class describes a test case in the test suite. A TestSuiteTestCase consists of simulation groups, which consist of simulations. Simulations can consist of several jobs. Every simulation is executed in a unique folder.

The class implements the interface demanded by AbstractTestCase. For the generation of the job lists the respective methods of the congregated classes are used.

Tutorial 1

class TestSuiteTestCaseSub(name=None, publicName='', author=None, email=None, confidentiality=None, testLevel=None, keywords=None, inactive=0, description=None, figures=None, references=None, referenceTraceSuite='', originalReferenceRevision=None, numProcsLocal=1, numProcsCluster=1, priority=1, baseCompute='.', compute=True, rowList=None, simulationGroup=None, restartGroup=None, meshingGroup=None, analysisGroup=None, gds_collector_=None, **kwargs_)

Class to represent a test suite test case. You can iterate over the job lists of a instance.

All possible parameters to the constructor of this class can be found in the XSD file used to generate testcase_generated.py.

addCommandLineForTex(latexDocumentInstance: mojo.latex.pyTex.PyTex, updateRestart: bool = False, chapterType: int = 2)

Formats the commands to a readable TEX Format.

Parameters

• latexDocumentInstance (PyTex) – PyTex instance of the current report to build

• updateRestart (bool) – add job list of restart groups if True

addDescriptionAndCommandline(options: argparse.Namespace, latexDocumentInstance: mojo.latex.pyTex.PyTex)

Returns the TEX output for the test case. The output contains the description and the commandline. The top level of the output is the section level.

Parameters

• options – options of the current PAVAYO run

• latexDocumentInstance – PyTex instance of the current report to build

addFigures(latexDocumentInstance: mojo.latex.pyTex.PyTex)

Returns the TEX output for the test case. The output contains the figures. The top level of the output is the section level.

Parameters

latexDocumentInstance – PyTex instance of the current report to build

addReferenceCheckSummary(latexDocumentInstance: mojo.latex.pyTex.PyTex, forcePlot: bool, comp_label: str, ref_label: str)

Add summary of reference checks to latex document.

Parameters

• latexDocumentInstance – PyTex instance of the current report to build

• forcePlot – Force generation of plots even if everything is identical

addSanitizersForTex(latexDocumentInstance: mojo.latex.pyTex.PyTex)

Adds the output of a compiler sanitizer to the test suite report.

Parameters

latexDocumentInstance (PyTex) – PyTex instance of the current report to build

addValgrindForTex(latexDocumentInstance: mojo.latex.pyTex.PyTex)

Adds the valgrind output of a test suite test case to the report.

Parameters

latexDocumentInstance (PyTex) – PyTex instance of the current report to build

append(simGroup: mojo.pavayo.testcase.testcase_generated.SimulationGroup)

Appends a simulation group to the test case

createJobListDictionary(options: argparse.Namespace, executableDict: mojo.pavayo.executables.executableResources.ExecutableResources, jobListGroupTag: Optional[str] = None, jobListTag: Optional[str] = None, jobTag: str = 'job', workingDirFct: Optional[Callable] = None, step: str = 'COMPUTATION', selectDict: Optional[Dict[str, Union[str, Dict[str, Union[str, List[str]]]]]] = None, resourcesDict: Optional[Dict[str, int]] = None) → DefaultDict[str, Union[Return, Tuple[Union[mojo.pavayo.testcase.testcase_generated.AnalysisJob, mojo.pavayo.testcase.testcase_generated.SimulationJob, mojo.pavayo.testcase.testcase_generated.Job], mojo.jobManagement.jobs.job.Job]]]

Factory method to call the method in AbstractTestcase with the correct arguments.

Parameters

• options – options instance to extract the executables and the queue to use

• executableDict – dictionary of executables

• jobListGroupTag – XML tag for the job list group / level

• jobListTag – XML tag for the job list / sub-level

• jobTag – XML tag for the job / job

• workingDirFct – method to determine the working directory for each job

• step – permitted steps (computation, restart and postprocessing)

• selectDict – dictionary of all jobs

• resourcesDict – resources required by the job

Returns

job dictionary representing the structure of the selected step

createPlot(plot: mojo.pavayo.testcase.testcase_generated.MatplotlibFigure, verbose: bool = True)

Create plot using TestSuiteXYPlot

Parameters

• plot – plot object

• verbose – show output

getCheckReferencesJobList(options: argparse.Namespace) → mojo.jobManagement.jobs.jobList.JobList

Create a joblist to check references of test case.

Parameters

options – options instance to extract the executables and the queue to use

Returns

job list representing this test case or None

getComputationJobList(options: argparse.Namespace, executableDict: mojo.pavayo.executables.executableResources.ExecutableResources, resourcesDict: Optional[Dict[str, int]] = None) → Optional[mojo.jobManagement.jobs.jobList.JobList]

Collects all job objects from the jobs in the current test case instance and creates a job list of them.

Parameters

• options – options instance to extract the executables and the queue to use

• executableDict – dictionary of executables

Returns

job list representing this test case or None

getEvaluateReferencesJobList(options: argparse.Namespace) → mojo.jobManagement.jobs.jobList.JobList

Create a joblist to check current against original references of test case.

Parameters

options – options instance to extract the executables and the queue to use

Returns

job list representing this test case or None

getMeshingJobList(options: argparse.Namespace, executableDict: mojo.pavayo.executables.executableResources.ExecutableResources, resourcesDict: Optional[Dict[str, int]] = None) → Optional[mojo.jobManagement.jobs.jobList.JobList]

Collects all job objects from the jobs in the current test case instance and creates a job list of them.

Parameters

• options – options instance to extract the executables and the queue to use

• executableDict – dictionary of executables

Returns

job list representing this test case or None

getPostprocessingJobList(options: argparse.Namespace, executableDict: mojo.pavayo.executables.executableResources.ExecutableResources, resourcesDict: Optional[Dict[str, int]] = None) → mojo.jobManagement.jobs.jobList.JobList

Returns a job representing the post script.

Parameters

• options – options instance to extract the executables and the queue to use

• executableDict – dictionary of executables

Returns

a jobManagement.jobs.job instance representing the job

getRestartJobList(options: argparse.Namespace, executableDict: mojo.pavayo.executables.executableResources.ExecutableResources, resourcesDict: Optional[Dict[str, int]] = None) → Optional[mojo.jobManagement.jobs.jobList.JobList]

Collects all job objects from the jobs in the current test case instance and creates a job list of them.

Parameters

• options – options instance to extract the executables and the queue to use

• executableDict – dictionary of executables

Returns

job list representing this test case or None

getUsedExecutables(compute: bool = False, restart: bool = False, meshing: bool = False, postprocessing: bool = False) → Set[mojo.pavayo.testcase.testcase_generated.Executable]

Returns the version of a software the test case was computed with.

Parameters

• compute – add executables of compute step to set?

• restart – add executables of restart step to set?

• meshing – add executables of meshing step to set?

• postprocessing – add executables of postprocessing step to set?

Returns

the used executables

get_state_by_step(step: str) → mojo.pavayo.testcase.abstractTestCase.TestCaseState

Returns the state of the step passed.

Parameters

step (string) – step of interest

Returns

True if an error occurred processing the test case

Return type

TestCaseState

insert(index: int, value: mojo.pavayo.testcase.testcase_generated.SimulationGroup)

Inserts a simulation group to this test case

Parameters

• index – place to insert in

• value – a test case job list to append

static prepareSourceFiles(testcase: mojo.pavayo.testcase.testSuiteTestCase.TestSuiteTestCaseSub, simulation: mojo.pavayo.testcase.testcase_generated.Simulation, simGroupName: str, computationDir: str)

Purges the computation, log and figure folder. Copies files needed by the test cases.

Parameters

• testcase – TestSuiteTestCaseSub instance

• simulation – simulation

• simGroupName – name of the simulation group

• computationDir – name of the computation directory

classmethod updateAccuracyDict(accuracy_dict_base: mojo.referenceCheck.referenceCheckContainer.CheckResult, accuracy_dict_update: mojo.referenceCheck.referenceCheckContainer.CheckResult)

Merges accuracy entries from ‘accuracy_dict_base’ and ‘accuracy_dict_update’ into ‘accuracy_dict_base’.

Parameters

• accuracy_dict_base – dictionary with accuracies

• accuracy_dict_update – dictionary with accuracies

updateReferences() → Tuple[bool, str]

Updates reference data for the files stored

Returns

flag whether updating the reference data has been successful, list of failed files

testSuiteComponents.py

testSuiteComponents contains all the subclasses which are to small to get an own file. The only changes to the parent classes are implemented in OpListSub and JobListSub. These classes also inherit from collections.Sequence . JobSub got an method to return the job represented as job from the jobManagement package.

class AnalysisJobSub(name='', executable=None, args=None, useTecplot=True, ignoreExitCode=False, dependency=None, gds_collector_=None, **kwargs_)

getJob(_options, executableDict, workingDirectory, outputDir, resourcesDict=None)

Returns the instance of a job from the job management. The job represents the job hold by the instance.

Parameters

• _options (OptionParser instance) – options instance with options parsed from the command line

• executableDict (ExecutableResources) – dictionary of executables

• workingDirectory (str) – base path of the test case

• outputDir (str) – output folder

Returns

a job instance which represents the instance itself

Return type

JobManagementJob or JobManagementJobList

class CommandsSub(job=None, gds_collector_=None, **kwargs_)

class DependencySub(job=None, subLevel=None, level=None, gds_collector_=None, **kwargs_)

class JobListSub(name='', step=None, job=None, gds_collector_=None, **kwargs_)

append(job)

Appends a new job to the job list represented by the instance.

Parameters

job (TestcaseJob instance) – the job instance to append

insert(index, value)

Inserts a job object at the given index.

Parameters

• index (integer) – index in the job list where the job will be inserted

• value (TestcaseJob instance) – the new job object

class JobSub(name='', executable=None, nProcs=1, args=None, workingDirectory='', outputFile='', procsPerNode=None, threadsPerProc=None, useMpirun=True, nodeType=None, executeOnMaster=False, wallTime=None, jobType=None, outputDir='', successStatement=None, failStatement=None, ignoreExitCode=False, dependency=None, gds_collector_=None, **kwargs_)

getJob(options, executableDict, basePath, outputDir, resourcesDict=None)

Returns the instance of a job from the job management. The job represents the job hold by the instance.

Parameters

• options (OptionParser instance) – options parsed from the command line

• executableDict (ExecutableResources) – dictionary of executables

• basePath (str) – base path of the test case

• outputDir (str) – path of output directory

Returns

job instance which represents the instance itself

Return type

Job

class OpInitialisationGlobalPerRowSub(name=None, inflowDensity=None, inflowPressure=None, inflowVelocityX=None, inflowVelocityThetaAbs=None, inflowVelocityR=None, inflowVelocityYAbs=None, inflowVelocityZ=None, outflowDensity=None, outflowPressure=None, outflowVelocityX=None, outflowVelocityThetaAbs=None, outflowVelocityR=None, outflowVelocityYAbs=None, outflowVelocityZ=None, xStart=None, xEnd=None, gds_collector_=None, **kwargs_)

class OpInitialisationGlobalSub(blockGroup=None, gds_collector_=None, **kwargs_)

class OpListSub(name='', compute=True, controlFile='', addCtrlCommand=None, addPostCommand=None, op=None, gds_collector_=None, **kwargs_)

class OpSub(name='', referenceLength=None, referenceTemperature=None, referencePressure=None, initFile='', inletFile='', stagnationPressure=None, stagnationTemperature=None, radialAngle=None, circumferentialAngle=None, turbulenceIntensity=None, turbulenceLengthScale=None, turbulenceKineticEnergy=None, turbulenceDissipationRate=None, correspondingMach=None, outletFile='', pressure=None, inflowDensity=None, inflowPressure=None, outflowDensity=None, outflowPressure=None, inflowVelocityX=None, inflowVelocityR=None, inflowVelocityTheta=None, outflowVelocityX=None, outflowVelocityR=None, outflowVelocityTheta=None, XStart=None, XEnd=None, inflowBCTypeSteady='', outflowBCTypeSteady='', inflowBCTypeUnsteady='', outflowBCTypeUnsteady='', inflowAverageType='', outflowAverageType='', statisticModel='', transitionModel='', stagnationPointAnomalyFix='', rotationalEffects='', compressionEffects='', preconditioning='', pdeWakeModel='', timeSteps=0, rpm=0, specificHeatRatio=None, specificGasConstant=None, timeMode='', timeStepsPeriod=None, timeStepSize=None, subIterations=None, timeMethod='', interfaceType='', frequency=None, timeAccuracy=None, spaceAccuracy=None, spatialScheme='', cfl=0, prepCommand='', opControlFile='', addCtrlCommand=None, addPostCommand=None, opInputDir='', cgnsFile='', numCPUs=None, numThreads=0, wallTime=None, outputDir='', gds_collector_=None, **kwargs_)

class RestartSetupSub(name='.', cgnsFile=None, component=None, typeComponent=None, controlVolume=None, rotor=None, inletPanel=None, outletPanel=None, interfacePanels=None, messageBlocks=None, output=None, nProcs=None, procsPerNode=None, threadsPerProc=0, opList=None, saveFiles=None, gds_collector_=None, **kwargs_)

find(name)

class RestartValidationOpListSub(name=None, cgnsFile=None, controlFile=None, balanceFile=None, sourceFiles=None, addCtrlCommand=None, prepCommands=None, postCommands=None, op=None, saveFiles=None, gds_collector_=None, **kwargs_)

class RestartValidationOpSub(name=None, cgnsFile=None, controlFile=None, balanceFile=None, sourceFiles=None, addCtrlCommand=None, prepCommands=None, postCommands=None, nProcs=None, procsPerNode=None, threadsPerProc=None, cgnsSettings=None, wallTime=None, saveFiles=None, gds_collector_=None, **kwargs_)

class RowListSub(name='', row=None, gds_collector_=None, **kwargs_)

class RowSub(name='', inletPanel='', outletPanel='', rotating=None, gds_collector_=None, **kwargs_)

class SetupSub(name='.', cgnsFile=None, component=None, typeComponent=None, controlVolume=None, rotor=None, inletPanel=None, outletPanel=None, interfacePanels=None, messageBlocks=None, output=None, nProcs=None, procsPerNode=None, threadsPerProc=0, opList=None, gds_collector_=None, **kwargs_)

find(name)

class SimulationJobSub(name='', executable=None, nProcs=- 1, args=None, procsPerNode=None, threadsPerProc=None, useMpirun=True, nodeType=None, executeOnMaster=False, wallTime=None, jobType=None, successStatement=None, failStatement=None, ignoreExitCode=False, saveLog='', dependency=None, gds_collector_=None, **kwargs_)

getJob(options, executableDict, workingDirectory, outputDir, resourcesDict=None)

Returns the instance of a job from the job management. The job represents the job hold by the instance.

Parameters

• options (OptionParser instance) – options instance with options parsed from the command line

• executableDict (ExecutableResources) – dictionary of executables

• workingDirectory (str) – base path of the test case

• outputDir (str) – output folder

Returns

a job instance which represents the instance itself

Return type

JobManagementJob or JobManagementJobList

class SliceSub(min=0, max=- 1, step=1, gds_collector_=None, **kwargs_)

getSlice()

Return the selected index slice as a python slice object. :return: slice object :rtype: slice

class StrSub

Contains the string method only.

class SuiteSub(mojoversion=None, gmcplayversion=None, pymeshversion=None, suitename=None, repository=None, listOfExecutables=None, report=None, tclist=None, gds_collector_=None, **kwargs_)

class TclistSub(generalTestCase=None, validationTestCase=None, validationTestCaseSpeedline=None, speedlineTestCase=None, parallelTestCase=None, testSuiteTestCase=None, gds_collector_=None, **kwargs_)

class ValidationOpListSub(name=None, cgnsFile=None, controlFile=None, balanceFile=None, sourceFiles=None, addCtrlCommand=None, prepCommands=None, postCommands=None, op=None, gds_collector_=None, **kwargs_)

class ValidationOpSub(name=None, cgnsFile=None, controlFile=None, balanceFile=None, sourceFiles=None, addCtrlCommand=None, prepCommands=None, postCommands=None, nProcs=None, procsPerNode=None, threadsPerProc=None, cgnsSettings=None, wallTime=None, extensiontype_=None, gds_collector_=None, **kwargs_)

compute

compute.py controls the execution of PAVAYO. It contains all functions to download test cases, to build a job list from these test cases and to generate reports. In addition it contains methods to update the reference data for given test cases and upload test cases into the test case repository.

showTestcases(options, testCaseList)

Prints all test cases available in the XML file.

Parameters

• options (instance) – options instance parsed from the command line

• testCaseList (list) – test cases

showKeywords(options, testCaseList)

Prints all keywords available in the the XML file.

Parameters

• options (instance) – options instance parsed from the command line

• testCaseList (list) – test cases

showSelected(options, executableDict, readTcList, suiteObject)

Prints the selected test cases only either by name or keywords.

Parameters

• options (instance) – options instance parsed from the command line

• executableDict (ExecutableResources) – dictionary of executables

• readTcList (list) – test cases read from XML

• suiteObject (object) – xml file suite object

prepareTestcases(options, executableDict, suite, readtclist)

Creates a test case list from a XML file.

Parameters

• options (options instance) – values from the option parser

• executableDict (ExecutableResources) – dictionary of executables

• suite (Suite) – main handle of meta data

• readtclist (list(AbstractTestCase)) – list of test cases

Returns

a tuple consisting on the svnclient, a list of test cases to compute, the suite name and a list of the inactive test cases

Return type

tuple[Optional[pysvntools.PYSVNClient], list[AbstraceTestCase]]

downloadTestcases(options, svnclient, testcaseList)

Download the test cases from a server.

Parameters

• options (options instance) – values from the option parser

• svnclient (class) – pysvn connections

• testcaseList (list) – list of test cases to download

lookForDownloadErrors(testcaseList)

Looks for missing files and saves the errors in the test cases.

Parameters

testcaseList (list) – list of test cases

createJoblist(options, executableDict, testCaseList, resourcesDict=None, postOnly=False)

Creates the main job list; containing the computation jobs and post processing jobs.

Parameters

• options (options instance) – options instance from command line parser

• executableDict (ExecutableResources) – dictionary of executables

• testCaseList (dict) – list of test cases

• resourcesDict – resources available

• postOnly (bool) – puts only post jobs in the jobList

Returns

main job list

Return type

JobList

lookForErrorsInAJobList(testcaseList, jobList, step)

Looks into the job list to get errors which happened during the computation.

Parameters

• testcaseList (list) – list of test cases

• jobList (JobList instance) – job list which was computed

• step (string) – step whose errors are looked for

scanTexFile(testcase)

Scans the TEX file of the test case in order to know which pictures are required to generate the pdf.

Parameters

testcase (Testcase instance) – Testcase instance which will be analyzed

Returns

list of expected pictures

Return type

list

lookingForPictures(expectedPictures, addEndings=True)

Looks if All the pictures have been produced. If it is not the case, a list of strings containing the name of the missing picture(s) will be written.

Parameters

• expectedPictures (list) – a list of picture expected

• addEndings (bool) – add different endings to the image file

Returns

a list containing strings with the missing pictures in latex format

Return type

list

cleanErrorFile(testcaseList, step)

Cleans the given section in the test cases error files.

Parameters

• testcaseList (list) – list of test cases

• step (string) – step for which to clean the error file

updateReferences(testcaseList)

Updates reference data for the test suite.

Parameters

testcaseList (list) – list of test cases to update

Returns

status whether updating has been successful

Return type

Status

uploadTestcases(svnclient, testcaseList)

Upload the modified test cases to thr svn repository.

Parameters

• svnclient (class) – pysvn connection

• testcaseList (list) – list of test cases to upload

Returns

success statement

Return type

bool

createAndRunJobListForComputationPostprocessing(options, executableDict, testcaseList, resourcesDict=None, junitWriter=None)

Executes and manages the computation and/or post processing part.

Parameters

• options (options instance) – options instance filled with the arguments from the command line

• executableDict (ExecutableResources) – dictionary of executables

• testcaseList (list) – list of test cases

• resourcesDict (dict) – resources available

• junitWriter (JUnitXMLOutput) – JUnit Writer if junit output is desired

Returns

a JobList instance containing all jobs for the steps chosen

Return type

JobList instance

manageTestcases(options, svnclient, testcaseList, executableDict, known_error_test_cases, resourcesDict=None, reportSettings=None, suiteName='TEST')

Executes and manages the computation and/or post processing part.

Parameters

• options (argparse.Namespace) – options instance filled with the arguments from the command line

• svnclient (Optional[pysvntools.PYSVNClient]) – pysvn connection

• testcaseList (list) – list of test cases

• executableDict (ExecutableResources) – dictionary of executables

• known_error_test_cases (dict[str: mojo.pavayo.knownErrorsParser.FailingTestCase]) – list of test cases with attributes known to fail

• resourcesDict (dict) – resources available

• reportSettings (ReportSub) – settings for the report

• suiteName (str) – suite name

Returns

error code

Return type

int

createDependenciesDictionary(tclist)

Creates a dictionary containing the test case names as key and a tuple of the test cases the test case is dependent of as values.

Parameters

tclist (list) – list containing test cases

Returns

dictionary the test cases mapped to the dependencies

Return type

dictionary

readErrorFiles(testcaseList, errorType)

Reads the error files from the given test cases, updates the error containers and returns a list of test cases which failed in the given step.

Parameters

• testcaseList (list) – list of test cases to analyze

• errorType (string) – error type to search for

Returns

list of erroneous test cases

Return type

list

createReports(options, testCaseList, known_error_test_cases, runTime, svnclient, executableDict, reportSettings=None)

Creates the latex reports for a computation and returns the name of the report and status.

Parameters

• options (options instance) – options parsed from the command line

• testCaseList (list) – list of test cases

• known_error_test_cases (dict[str: mojo.pavayo.knownErrorsParser.FailingTestCase]) – list of test cases with attributes known to fail

• runTime (int) – runtime of the computation

• svnclient (Optional[pysvntools.PYSVNClient]) – pysvn connection

• executableDict (ExecutableResources) – dictionary of executables

• reportSettings (ReportSub) – settings for the report

Returns

name of reports, return code of LaTeX compilation

Return type

tuple(list(str), int)

Job List Package

In the package ‘joblists’ the methods of all job list types are located.

Available Job Lists

parallel	Build Parallel Job List
speedline	Build Speed Line Job List

Build Parallel Job List

A parallel job list allows the user to perform several computations simultaneously depending on the provided resources. All computation jobs are treated independently and are added to a job list.

Build Speed Line Job List

Here the computations of the operating points are performed successively. For all operating points the same CGNS file is used. Therefore it is backed up at first. For each operating point several steps have to be done.

1. A gmcPlay journal file is created and used to update the CGNS file

2. The TRACE computation is done

3. Parts of the result files have to moved and/or renamed so that they cannot be overwritten.

computeOptions

Module containing the option parser and its arguments for pavayo.py

class AddBasePathAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Action class to fill base path dictionary

class AddExecutableAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Action class for additional executables other than TRACE, PREP, POST, GMCPLAY or PYMESH

class AutorunAction(option_strings, dest, const=True, default=False, required=False, help=None, metavar=None)

Action class for the autorun option of PAVAYO.

class PostProcAction(option_strings, dest, const=True, default=False, required=False, help=None, metavar=None)

Action class for the –postProc/-p option of PAVAYO.

class UpdateMeshDataAction(option_strings, dest, const=True, default=False, required=False, help=None, metavar=None)

Action class for the –updateMeshData option of PAVAYO.

getOptionsFromParser(parser=None, args=None)

Reads the command line options and prepares the options instance for pavayo. Performs some sanity checks on the options.

Parameters

• parser (argparse.ArgumentParser) – option instance providing all arguments

• args (string list) – Provide arguments for parser; if None sys.argv is used

Returns

option instance for pavayo

Return type

Namespace

getParser()

Parses the command line arguments and saves them in a options instance. All paths have to be converted to absolute paths in routine ‘getOptionsFromParser’!

Returns

a options instance containing all command line arguments

Return type

options instance

printOptions(options, executableDict)

Method to give a overview over the selected options of PAVAYO.

Parameters

• options (Argparse) – option parser instance to show

• executableDict (ExecutableResources) – ExecutableResources

computeData

Module containing general methods and constants for compute.py ergo pavayo.py.

computeMethods

Module containing methods for compute.py

convertErrorCodeToString(errorCode)

Gets an error code and returns the related string

Parameters

errorCode (int) – error code

Returns

related text message

Return type

str

createGeneralTitlePageContent(executableDict, suiteName, userName, hostName, svnRepoPath=None, svnRevision=None, runTime=None)

Create the content of the title page for the report.

Parameters

• executableDict (ExecutableResources) – dictionary of executables

• suiteName (str) – Name of the test suite specified in the xml file

• userName (str) – user name

• hostName (str) – host name

• svnRepoPath (str) – path of SVN repository

• svnRevision (int) – SVN revision number

• runTime (str) – duration of current run

Returns

content of title page

Return type

str

createResourceDictionary(options, listOfExecutables)

Create a dictionary of all executables read from the XML file

Parameters

• options (argParse.ArgumentParser) – options parsed from the command line

• listOfExecutables (ResourcesSub) – list of executables read from the XML file

Returns

dictionary of executables

Return type

ExecutableResources

createValidationTitlePageContent(executableDict, basePathDict, reportSettings=None)

Create the content of the title page for the report.

Parameters

• executableDict (ExecutableResources) – dictionary of executables

• basePathDict (dict) – base paths

• reportSettings (object) – cumulated settings for the report

Returns

content of title page

Return type

str

getMOJOPythonPath()

Builds the path to the MOJO src folder.

Returns

absolute path to MOJOs source folder

Return type

str

getSoftwaresVersionNumbers(options, executableDict, readVersionsFromExecutables=False)

Reads the software version numbers by calling <SOFTWARE> –version or reading the version from a options file. Returns a dictionary which maps the software to its version.

Parameters

• options (argParse.ArgumentParser) – options parsed from the command line

• executableDict (ExecutableResources) – dictionary of executables

• readVersionsFromExecutables (bool) – if true the method reads the version from the options file instead of calling the executables

Returns

a dictionary containing the software versions mapped to the software name

Return type

dict

processTraceSuitePath(options, executableDict)

Sets the TRACE, PREP and POST path from the path to the trace suite.

Parameters

• options (argParse.ArgumentParser) – options parsed from the command line

• executableDict (ExecutableResources) – dictionary of executables