# CONS-TEST $Id: README,v 1.1 1998/09/25 22:41:20 software Exp $ INTRODUCTION This is the cons-test package, a test suite for the Cons software construction system. Cons was originally written by Bob Sidebotham, and is currently (September 1998) maintained by Rajesh Vaidheeswarran. The home page for Cons itself can be found at: http://www.telerama.com/~bgarcia/cons/ CONTENTS The contents of this release: ChangeLog List of changes. README What you're looking at right now. cons-test.pl Wrapper script to execute tests. t*a.sh Tests. template.tst Template for creating additional tests. THE cons-test.pl SCRIPT By default, the cons-test.pl script will test the script 'cons' in the current directory. An alternate script name may be specified via the -x flag: $ perl cons-test.pl -x cons.NEW $ perl cons-test.pl -x /usr/foo/cons.experiment/cons The script must have execute permissions set. By default, the cons-test.pl script simply executes all the tests in the local directory with the specified script, and reports a summary of the pass/fail score: cons-test: 'cons' passed all 27 tests. cons-test: 'cons' passed 25 tests, failed 2: t0003a.sh t0026a.sh The cons-test.pl script can be given one or more tests as arguments, in which case it will only execute the specified tests: $ perl cons-test.pl t0007a.sh t0023a.sh cons-test.pl: sh t0007a.sh PASSED cons-test.pl: sh t0023a.sh PASSED cons-test.pl: 'cons' passed all 2 tests. $ TESTS Tests conform to requirements of the Aegis project change supervisor, which integrates creation and execution of regression tests into the software development process. Information about Aegis can be found at: http://www.tip.net.au/~millerp/aegis.html Each test is a completely self-contained shell script, and should be executed by explicitly passing it to your (Bourne or Korn) shell: sh t0001a.sh The cryptic names are by Aegis convention only, and could have been named to reflect the functionality being tested. I've preserved numbering them to give order to the tests: Simpler, underlying functionality is tested before more complicated functionality that relies on functionality tested by prior tests. Each test creates a temporary work directory under $TMPDIR (under /tmp by default) and populates it with files from in-line here documents. Each test checks the return value from every command it executes, expecting successful exit codes from all commands. When it's important, it compares actual command output against expected output. (Note that this actually refers to the output of the executables generated by Cons, not the "cc" command lines reported by Cons as part of its building the test executables.) Collectively, the tests rely on having a C compiler named 'cc' in the environment's PATH and expect that printf() is available through the default Cons environment. Each test reports one of three results: PASSED The Cons script being tested passed this test. FAILED test of [functionality] The Cons script being tested failed this test. NO RESULT for test of [functionality] The Cons script could not be tested due to some non-Cons problem (e.g., unable to create temporary directory or file). Each test removes its temporary work directory upon completion. For post facto debugging, the work directory can be preserved (and its name reported) by setting environment variables: PRESERVE Don't remove the work directory. PRESERVE_PASS Don't remove the work directory if the test passes. PRESERVE_FAIL Don't remove the work directory if the test fails. PRESERVE_NO_RESULT Don't remove the work directory if there is no valid test result. Changes, enhancements, portability fixes are welcome. Steven Knight knight@baldmt.com