In order to reach the `pass` verdict, the TTCN-3 tool under test, the IUT, must
accept the TTCN-3 module as test input. The result of its execution must be the
TTCN-3 verdict pass. When needed, a manual inspection of the execution results
is required. Therefore, if the TTCN-3 verdict is pass after the execution of
the module, the IUT passes the conformance test. If the tool output is anything
else, or it does not comply with the manual inspection, it fails the
conformance test.
The keywords in use to describe the third column, i.e. the expected output by
the IUT, are (currently) as follows:
reject
accept, noexecution
accept, ttcn3verdict:none
accept, ttcn3verdict:pass
accept, ttcn3verdict:inconc
accept, ttcn3verdict:fail
accept, ttcn3verdict:error
accept, ttcn3verdict:xxx, manual:"Validation inspection, free text" (see Note)
Note: `xxx` for ttcn3verdict is a placeholder for verdict value. The actual test cases must
have one of the allowed values `none`, `pass`, `inconc`, `fail`, or `error` in place of `xxx`.
`reject` implies that the TTCN-3 module is either rejected at compile-time or at execution time.
In the conformance test, we do not differentiate between these two cases as the standard does not
make any statement where semantic checks have to be performed.
`accept, noexecution` implies that the TTCN-3 module should be accepted by the TTCN-3 tool after
the syntactic check. For passing the conformance test, the module simply has to be accepted and
an execution is not necessary.
`accept, ttcn3vercit:xxx` implies that the TTCN-3 module should be accepted by the TTCN-3 tool
after the syntactic check and that an execution should take place. The result of the execution
should be a TTCN-3 verdict and the `xxx` denotes what verdict is the exepcted verdict. If the
verdict differs from the specified `xxx`, the conformance test fails. Otherwise, it passes.
In the usual case, each TTCN-3 file contains only one test case. In these cases the verdict
determination is clear. In a few cases, the TTCN-3 file contains more than one test case.
In that case, the overall conformance verdict is determined according to the TTCN-3 verdict
overwrite rules applied to the results of each test case. Let's say we have two test cases.
The first test case ends with the verdict `fail` and the second one ends with the verdict `pass`.
Then the overall verdict is `fail`.
The manual results inspection is a free text describing how a valid out should
look like. The text is informal since different tools have different logging
formats and facilities. The instruction is surrounded by single or double
quotas on a single line:
@verdict pass, testverdict:pass, manual:'The following elements are logged: charstring "Extra", record { 1, "HELLO"}, integer template "?". Make sure the elements are logged at setverdict, at MTC end, and at test case end.'
All notations are designed to be easily machine readable. Therefore, the test automation using
some sort of scripting will only take a small amount of time.
### Other prerequisites
In order to test communication and matching behavior, we expect that the test cases are executed
