NAME Test::Cmd - Perl module for testing commands and scripts SYNOPSIS use Test::Cmd; $test = Test::Cmd->new('prog' => 'program_or_script_to_test', 'interpreter' => 'script_interpreter', 'workdir' => '', 'subdir' => 'dir', 'verbose' => 1); $test->prog('program_or_script_to_test'); $test->interpreter('script_interpreter'); $test->workdir('prefix'); $test->subdir('subdir', ...); $test->verbose(1); $test->write('file', <<'EOF'); contents of file EOF $test->writable('dir', rwflag); $test->preserve(condition, ...); $test->cleanup(condition); $test->run('dir', 'arguments'); $test->pass(condition, funcref); $test->fail(condition, funcref); $test->no_result(condition, funcref); $test->stdout(); $test->stderr(); $test->diff(); $test->here(); DESCRIPTION The Test::Cmd module provides a framework for portable automated testing of executable commands and scripts, especially commands and scripts that require file system interaction. This module is not restricted to testing Perl scripts; the command or script to be tested may be any executable or a script in any language, provided a means exists to execute the script on the local system. In addition to running tests and evaluating conditions, the Test::Cmd module manages and cleans up one or more temporary workspace directories, and provides methods for creating files and directories in those workspace directories from in-line data (that is, here documents). This allows tests to be completely self-contained. The Test::Cmd module manipulates filenames and pathnames using the File::Spec module to support writing tests portably across a variety of operating and file systems. The Test::Cmd class is in fact a subclass of the File::Spec class, and File::Spec methods (File::Spec->catfile(), File::Spec->file_name_is_absolute(), etc.) are available through the Test::Cmd class and its instances. Consequently, tests written using Test::Cmd need not separately import the File::Spec module. A Test::Cmd environment object is created via the usual invocation: $test = Test::Cmd->new(); Arguments to the Test::Cmd->new() method are keyword-value pairs that may be used to initialize the object, typically by invoking the same-named method as the keyword. The Test::Cmd module may be used in conjunction with the Test module to report test results in a format suitable for the Test::Harness module. A typical use would be to call the Test::Cmd methods to prepare and execute the test, and call the ok() method exported by the Test module to test the conditions: use Test; use Test::Cmd; BEGIN { $| = 1; plan => 2 } $test = Test::Cmd->new(prog => 'test_program', workdir => ''); ok($test); $wrote_file = $test->write('input_file', <<'EOF'); This is input to test_program, which we expect to process this and exit successfully (status 0). EOF ok($wrote_file); $test->run('.', 'input_file'); ok($? == 0); Alternatively, the Test::Cmd module provides pass(), fail(), and no_result() methods that report test results differently. These methods terminate the test immediately, reporting PASSED, FAILED, or NO RESULT respectively, and exiting with status 0 (success), 1 or 2 respectively. This allows for a distinction between an actual failed test and a test that could not be properly evaluated because of an external condition (such as a full file system or incorrect permissions): use Test::Cmd; $test = Test::Cmd->new(prog => 'test_program', workdir => ''); Test::Cmd->no_result(! $test); $wrote_file = $test->write('input_file', <<'EOF'); This is input to test_program, which we expect to process this and exit successfully (status 0). EOF $test->no_result(! $wrote_file); $test->run('.', 'input_file'); $test->fail($? != 0); $test->pass; It is not a good idea to intermix the two reporting models. If you use the Test module and the Test->ok() method, do not use the Test::Cmd->pass(), Test::Cmd->fail() or Test::Cmd- >no_result() methods, and vice versa. METHODS Methods supported by the Test::Cmd module include: `new' Create a new Test::Cmd environment. Arguments with which to initialize the environment are passed in as keyword-value pairs. `verbose' Sets the verbose level for the environment object to the specified value. `prog' Specifies the executable program or script to be tested. Returns the absolute path name of the current program or script. `interpreter' Specifies the program to be used to interpret `prog' as a script. Returns the current value of `interpreter'. `workdir' When an argument is specified, creates a temporary working directory with the specified name. If the argument is a NULL string (''), the directory is named `testcmd' by default, followed by the unique ID of the executing process. Returns the pathname to the temporary working directory. `subdir' Creates new subdirectories under the temporary working dir. Subdirectories multiple levels deep must be created via a separate argument for each level: $test->subdir('sub', 'sub/dir', 'sub/dir/ectory'); Returns the number of subdirectories actually created. `write' Writes the specified text (second argument) to the specified file name (first argument). The file is created under the temporary working directory. Any subdirectories must already exist. `writable' Makes the specified directory tree writable (rwflag == TRUE) or not writable (rwflag == FALSE). `preserve' Arranges for the temporary working directories for the specified Test::Cmd environment to be preserved for one or more conditions. If no conditions are specified, arranges for the temporary working directories to be preserved for all conditions. `cleanup' Removes any temporary working directories for the specified Test::Cmd environment. If the environment variable PRESERVE was set when the Test::Cmd module was loaded, temporary working directories are not removed. If any of the environment variables PRESERVE_PASS, PRESERVE_FAIL, or PRESERVE_NO_RESULT were set when the Test::Cmd module was loaded, then temporary working directories are not removed if the test passed, failed, or had no result, respectively. Temporary working directories are also preserved for conditions specified via the preserve() method. `run' Runs a test of the specified program or script, first changeing directory to the specified subdir (first argument) of the temporary working directory. The script is run with the arguments specified as the second argument of the run() method call. Standard output and error output are saved for future retrieval via the stdout() and stderr() methods. Returns the return value from the system() call used to invoke the program or script. `pass' Exits the test successfully. Reports "PASSED" on the error output and exits with a status of 0. If a condition is supplied, only exits the test if the condition evaluates TRUE. If a function reference is supplied, executes the function before reporting and exiting. `fail' Exits the test unsuccessfully. Reports "FAILED test of {prog} at line {line} of {file}." on the error output and exits with a status of 1. If a condition is supplied, only exits the test if the condition evaluates TRUE. If a function reference is supplied, executes the function before reporting and exiting. `no_result' Exits the test with an indeterminate result (the test could not be performed due to external conditions such as, for example, a full file system). Reports "NO RESULT for test of {prog} at line {line} of {file}." on the error output and exits with a status of 2. If a condition is supplied, only exits the test if the condition evaluates TRUE. If a function reference is supplied, executes the function before reporting and exiting. `stdout' Returns the standard output from the specified run number. If there is no specified run number, then returns the standard output of the last run. Returns the standard output as either a scalar or an array of output lines, as appropriate for the calling context. Returns undef if there has been no test run. `stderr' Returns the error output from the specified run number. If there is no specified run number, then returns the error output of the last run. Returns the error output as either a scalar or an array of output lines, as apporpriate for the calling context. Returns undef if there has been no test run. `diff' To Be Written. `here' Returns the absolute path name of the current working directory. (This is essentially the same as Cwd::cwd, except that this preserves the directory separators exactly as returned by the underlying operating-system-dependent method. The Cwd::cwd method canonicalizes all directory separators to '/', which makes for consistent path name representations within Perl, but may mess up another program or script to which you try to pass the path name.) ENVIRONMENT Several environment variables affect the default values in a newly created Test::Cmd environment object. These environment variables must be set when the module is loaded, not when the object is created. `PRESERVE' If set to a true value, all temporary working directories will be preserved on exit, regardless of success or failure of the test. The full path names of all temporary working directories will be reported on error output. `PRESERVE_FAIL' If set to a true value, all temporary working directories will be preserved on exit from a failed test. The full path names of all temporary working directories will be reported on error output. `PRESERVE_NO_RESULT' If set to a true value, all temporary working directories will be preserved on exit from a test for which there is no result. The full path names of all temporary working directories will be reported on error output. `PRESERVE_PASS' If set to a true value, all temporary working directories will be preserved on exit from a successful test. The full path names of all temporary working directories will be reported on error output. `VERBOSE' When set to a true value, enables verbose reporting of various internal things (path names, exact command line being executed, etc.). PORTABLE TESTS Although the Test::Cmd module is intended to make it easier to write portable tests for portable utilities that interact with file systems, it is still very easy to write non-portable tests if you're not careful. The best and most comprehensive set of portability guidelines is the standard "Writing portable Perl" document at: http://www.perl.com/pub/doc/manual/html/pod/perlport.html To reiterate one important point from the "WpP" document: Not all Perl programs have to be portable. If the program or script you're testing is UNIX-specific, you can (and should) use the Test::Cmd module to write UNIX-specific tests. That having been said, here are some hints that may help keep your tests portable, if that's a requirement. Use Test::Cmd->here() for current directory path. The normal Perl way to fetch the current working directory is to use the Cwd::cwd() method. Unfortunately, the Cwd::cwd() method canonicalizes the path name it returns, changing the native directory separators into the forward slashes favored by Perl and UNIX. For most Perl scripts, this makes a great deal of sense and keeps code uncluttered. Passing in a file name that has had its directory separators altered, however, may confuse the command or script under test, or make it difficult to compare output from the command or script with an expected result. The Test::Cmd::here() method returns the absolute path name of the current working directory, like Cwd::cwd(), but does not manipulate the returned path in any way. Use File::Spec methods for manipulating path names. The File::Spec module provides a system-independent interface for manipulating path names. Because the Test::Cmd class is a sub-class of the File::Spec class, you can use these methods directly as follows: if (! Test::Cmd->file_name_is_absolute($prog)) { my $prog = Test::Cmd->catfile(Test::Cmd->here, $prog); } For details about the available methods and their use, see the documentation for the File::Spec module and its sub- modules, especially the File::Spec::Unix modules. Use Config for file-name suffixes, where possible. The standard Config module provides values that $foo_exe = "foo$Config{_exe}"; ok(-f $foo_exe); Unfortunately, there is no existing $Config value that specifies the suffix for a directly-executable Perl script. Avoid generating executable programs or scripts. How to make a file or script executable varies widely from system to system, some systems using file name extensions to indicate executability, others using a file permission bit. The differences are complicated to accomodate in a portable test script. The easiest way to deal with this complexity is to avoid it if you can. If your test somehow requires executing a script that you generate from the test itself, the best way is to generate the script in Perl and then explicitly feed it to the Perl executable on the local system. To be maximally portable, use the $^X variable instead of hard-coding "perl" into the string you execute: $line = This is output from the generated perl script."; $test->write('script', <write('script', <workdir); chmod(0755, 'script'); # POSIX-SPECIFIC $output = `script`; ok($output eq "$line\n"); Addtional hints on how to write portable tests are welcome. SEE ALSO perl(1), File::Find(3), File::Spec(3), Test(3), Test::Harness(3). A rudimentary page for the Test::Cmd module is available at: http://www.baldmt.com/Test-Cmd/ AUTHORS Steven Knight, knight@baldmt.com ACKNOWLEDGEMENTS Thanks to Greg Spencer for the inspiration to create this package and the initial draft of its implementation as a specific testing package for the Cons software construction utility. Information about Cons is available at: http://www.dsmit.com/cons/ The general idea of managing temporary working directories in this way, as well as the test reporting of the pass(), fail() and no_result() methods, come from the testing framework invented by Peter Miller for his Aegis project change supervisor. Aegis is an excellent bit of work which integrates creation and execution of regression tests into the software development process. Information about Aegis is available at: http://www.tip.net.au/~millerp/aegis.html