MD5/SHA1 and MDC-2 have special-case logic beyond the generic DigestInfo
wrapping. Test that each of these works, including hash and length
mismatches (both input and signature). Also add VerifyRecover tests. It
appears 5824cc29

 added support for
VerifyRecover, but forgot to add the test data.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>

GH: #1474

PKCS #1 v2.0 is the name of a document which specifies an algorithm
RSASSA-PKCS1-v1_5, often referred to as "PKCS #1 v1.5" after an earlier
document which specified it. This gets further confusing because the
document PKCS #1 v2.1 specifies two signature algorithms,
RSASSA-PKCS1-v1_5 and RSASSA-PSS. RSA_sign implements RSASSA-PKCS1-v1_5.

Refer to the document using the RFC number which is easier to find
anyway, and refer to the algorithm by its name.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>

GH: #1474

RFC 3447, section 8.2.2, steps 3 and 4 states that verifiers must encode
the DigestInfo struct and then compare the result against the public key
operation result. This implies that one and only one encoding is legal.

OpenSSL instead parses with crypto/asn1, then checks that the encoding
round-trips, and allows some variations for the parameter. Sufficient
laxness in this area can allow signature forgeries, as described in
https://www.imperialviolet.org/2014/09/26/pkcs1.html

Although there aren't known attacks against OpenSSL's current scheme,
this change makes OpenSSL implement the algorithm as specified. This
avoids the uncertainty and, more importantly, helps grow a healthy
ecosystem. Laxness beyond the spec, particularly in implementations
which enjoy wide use, risks harm to the ecosystem for all. A signature
producer which only tests against OpenSSL may not notice bugs and
accidentally become widely deployed. Thus implementations have a
responsibility to ho...

This partially reverts commit c636c1c4

. It also tweaks the documentation
and comments in this area. On the client side the documented interface for
SSL_CTX_set_verify()/SSL_set_verify() is that setting the flag
SSL_VERIFY_PEER causes verfication of the server certificate to take place.
Previously what was implemented was that if *any* flag was set then
verification would take place. The above commit improved the semantics to
be as per the documented interface.

However, we have had a report of at least one application where an
application was incorrectly using the interface and used *only*
SSL_VERIFY_FAIL_IF_NO_PEER_CERT on the client side. In OpenSSL prior to
the above commit this still caused verification of the server certificate
to take place. After this commit the application silently failed to verify
the server certificate.

Ideally SSL_CTX_set_verify()/SSL_set_verify() could be modified to indicate
if invalid flags were being used. However these are void functions!

The simplest short term solution is to revert to the previous behaviour
which at least means we "fail closed" rather than "fail open".

Thanks to Cory Benfield for reporting this issue.

Reviewed-by: Richard Levitte <levitte@openssl.org>

1) Remove some unnecessary fixtures
2) Add EXECUTE_TEST_NO_TEARDOWN shorthand when a fixture exists but has
no teardown.
3) Fix return values in ct_test.c (introduced by an earlier refactoring,
oops)

Note that for parameterized tests, the index (test vector) usually holds all the
customization, and there should be no need for a separate test
fixture. The CTS test is an exception: it demonstrates how to combine
customization with parameterization.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Prior to TLS1.3 we check that the received record version number is correct.
In TLS1.3 we need to ignore the record version number. This adds a test to
make sure we do it correctly.

Reviewed-by: Rich Salz <rsalz@openssl.org>

The record layer version field must be ignored in TLSv1.3, so we remove the
check when using that version.

Reviewed-by: Rich Salz <rsalz@openssl.org>

We may get failures if we run it in TLS1.3, and it makes no sense anyway
so force TLS1.2

Reviewed-by: Rich Salz <rsalz@openssl.org>

We have one TLS1.3 ciphersuite, but there is a typo in the id that should
be corrected.

Reviewed-by: Rich Salz <rsalz@openssl.org>

We read it later in grow_init_buf(). If CCS is the first thing received in
a flight, then it will use the init_msg from the last flight we received. If
the init_buf has been grown in the meantime then it will point to some
arbitrary other memory location. This is likely to result in grow_init_buf()
attempting to grow to some excessively large amount which is likely to
fail. In practice this should never happen because the only time we receive
a CCS as the first thing in a flight is in an abbreviated handshake. None
of the preceding messages from the server flight would be large enough to
trigger this.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1772)

If zlib-dynamic was given but not --with-zlib-lib, LIBZ was defined to
the empty string.  Instead, give it the default "ZLIB1".

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1772)

VMS only unloads shared libraries at process rundown, so tell the
OpenSSL code so by pretending we linked with -znodelete.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1862)

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1856)

Since the local symbol table is looked up before the global symbol
table, 'arch' assigned in the local symbol table of the DCL where MMS
is called would be seen before the 'arch' defined in descrip.mms.
Assigning it to the local symbol table in descrip.mms removes that
issue.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1853)

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1849)

The sources for internal tests were sometimes badly formed, assuming
perl variables such as $target{cpuid_asm_src} contains only one file
name.  This change correctly massages all file names in such a
variable.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1850)

This is related to a lack in path merging involding includes of includes

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1846)

The logic around avoiding MULDEF warnings was flawed.  Simplifying it
makes it better.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1846)

The test fixtures are (meant to be) useful for sharing common
setup. Don't bother when we don't have any setup/teardown.

This only addresses simple tests. Parameterized tests (ADD_ALL_TESTS)
will be made more user-friendly in a follow-up.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Richard Levitte <levitte@openssl.org>

Clang on Linux seems to catch things that we might miss otherwise.
Also, throw in 'no-deprecated' to make sure we test that as well.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1839)

pqueue_size() now returns a size_t, but the variable that gets returned
was still declared as an int.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Travis is reporting one file at a time shadowed variable warnings where
"read" has been used. This attempts to go through all of libssl and replace
"read" with "readbytes" to fix all the problems in one go.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Give more detail on what constitutes success/failure.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Rename "read" to "readbytes"

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Improvements to style, grammar etc.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Based on review feedback.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Travis was failing in some builds due to a bogus complaint
about uninit variables.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Also document SSL_peek() which was missing from the docs.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Also implement the using of them

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>