Commit b6b03c8a authored by Dan Fandrich's avatar Dan Fandrich
Browse files

Added feature in runtests.pl to select tests based on key word.

parent 432945e4

CHANGES

+5 −0
Original line number Diff line number Diff line
@@ -21,6 +21,11 @@ Daniel Stenberg (26 Jul 2008) 
Daniel Fandrich (26 Jul 2008)

- Added test 1044 to test large file support in ftp with -I.



- Eliminate a unnecessary socket creation in Curl_getaddrinfo for an IPv4

  address in an IPv6 capable libcurl.



- Added feature in runtests.pl to select tests based on key word.



Daniel Fandrich (23 Jul 2008)

- Changed the long logfile elision code in runtests.pl to properly handle

  lines ending in \r.

RELEASE-NOTES

+1 −0
Original line number Diff line number Diff line
@@ -17,6 +17,7 @@ This release includes the following changes: 
 o Added --remote-name-all

 o Now builds for the INTEGRITY operating system

 o Added CURLINFO_APPCONNECT_TIME

 o Added test selection by key word in runtests.pl



This release includes the following bugfixes:

tests/FILEFORMAT

+5 −2
Original line number Diff line number Diff line
@@ -29,7 +29,10 @@ Main sections are 'info', 'reply', 'client' and 'verify'. 
<keywords>

A newline-separated list of keywords describing what this test case uses and

tests. Try to use an already used keyword.  These keywords will be used for

statistical/informational purposes.

statistical/informational purposes and for choosing or skipping classes

of tests.  "Keywords" must begin with an alphabetic character, "-", "["

or "{" and may actually consist of multiple words separated by spaces

which are treated together as a single identifier.

</keywords>

</info>

tests/runtests.1

+5 −2
Original line number Diff line number Diff line
@@ -25,7 +25,7 @@ 
.SH NAME

runtests.pl \- run one or more test cases

.SH SYNOPSIS

.B runtests.pl [options] [test number]

.B runtests.pl [options] [test number] [!test number] [key word] [!key word]

.SH DESCRIPTION

\fIruntests.pl\fP runs one, several or all the existing test cases in curl's

test suite. It is often called from the root Makefile of the curl package with
@@ -39,7 +39,10 @@ the numbers with a leading exclamation point, like "!66". 
.P

It is also possible to specify tests to skip based on a key word describing

the test. These are specified with a leading exclamation point and the

key word or phrase, like "!HTTP NTLM auth".

key word or phrase, like "!HTTP NTLM auth".  Likewise, tests to run can

be specified simply by specifying the unadorned key words, like "FTPS".

Remember that the exclamation marks and spaces will need to be quoted somehow

when entered at many command shells.

.SH OPTIONS

.IP "-a"

Continue running the rest of the test cases even if one test fails. By

tests/runtests.pl

+16 −4
Original line number Diff line number Diff line
@@ -45,7 +45,7 @@ 
# to do with cookies, those that set environment variables, or those that

# do more than touch the file system in a <precheck> or <postcheck>

# section). These can be added to the $TESTCASES line below,

# e.g. $TESTCASES="!8 !31 !63..."

# e.g. $TESTCASES="!8 !31 !63 !cookies..."

#

# Finally, to properly support -g and -n, checktestcmd needs to change

# to check the remote system's PATH, and the places in the code where
@@ -197,6 +197,7 @@ my $skipped=0; # number of tests skipped; reported in main loop 
my %skipped;    # skipped{reason}=counter, reasons for skip

my @teststat;   # teststat[testnum]=reason, reasons for skip

my %disabled_keywords;	# key words of tests to skip

my %enabled_keywords;	# key words of tests to run



my $sshdid;      # for socks server, ssh daemon version id

my $sshdvernum;  # for socks server, ssh daemon version number
@@ -1820,13 +1821,20 @@ sub singletest { 


    if(!$why) {

        my @keywords = getpart("info", "keywords");

	my $match;

        my $k;

        for $k (@keywords) {

            chomp $k;

            if ($disabled_keywords{$k}) {

            	$why = "disabled by keyword";

            } elsif ($enabled_keywords{$k}) {

            	$match = 1;

            }

        }



	if(!$why && !$match && %enabled_keywords) {

	  $why = "disabled by missing keyword";

	}

    }



    if(!$why) {
@@ -2781,7 +2789,7 @@ while(@ARGV) { 
    elsif($ARGV[0] eq "-h") {

        # show help text

        print <<EOHELP

Usage: runtests.pl [options] [test number(s)]

Usage: runtests.pl [options] [test selection(s)]

  -a       continue even if a test fails

  -bN      use base port number N for test servers (default $base)

  -c path  use this curl executable
@@ -2797,7 +2805,8 @@ Usage: runtests.pl [options] [test number(s)] 
  -v       verbose output

  [num]    like "5 6 9" or " 5 to 22 " to run those tests only

  [!num]   like "!5 !6 !9" to disable those tests

  [!keyword] like "!cookies !IPv6" to disable tests with those key words

  [keyword] like "IPv6" to select only tests containing the key word

  [!keyword] like "!cookies" to disable any tests containing the key word

EOHELP

    ;

        exit;
@@ -2824,6 +2833,9 @@ EOHELP 
    elsif($ARGV[0] =~ /^!(.+)/) {

        $disabled_keywords{$1}=$1;

    }

    elsif($ARGV[0] =~ /^([-[{a-zA-Z].*)/) {

        $enabled_keywords{$1}=$1;

    }

    else {

    	print "Unknown option: $ARGV[0]\n";

    	exit;
@@ -2940,7 +2952,7 @@ if ( $TESTCASES eq "all") { 
    for(@cmds) {

        $_ =~ s/[a-z\/\.]*//g;

    }

    # the the numbers from low to high

    # sort the numbers from low to high

    foreach my $n (sort { $a <=> $b } @cmds) {

        if($disabled{$n}) {

            # skip disabled test cases