CRYPTO_mem_leaks attempts to adjust the count of bytes leaks to not
include the BIO that is being used to print the results out. However this
does not work properly. In all internal cases we switch off recording
the memory allocation during creation of the BIO so it makes no difference.
In other cases if the BIO allocates any additional memory during
construction then the adjustment will be wrong anyway. It also skips over
the BIO memory during print_leak anyway, so the BIO memory is never
added into the total. In other words this was broken in lots of ways and
has been since it was first added.

The simplest solution is just to make it the documented behaviour that
you must turn off memory logging when creating the BIO, and remove all
the adjustment stuff completely. The adjustment code was only ever in
master and never made it to a release branch so there is no loss of
functionality.

This commit also fixes a compilation failure when using
enable-crypto-mdebug.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Fix buffer overrun in asn1_get_length().

Reproducer: asn1parse-reproduce crash-6bfd417f47bc940f6984f5e639b637fd4e6074bc

Fix length calculations.

Reproducer: asn1parse-reproduce crash-1819d0e54cd2b0430626c59053e6077ef04c2ffb
Reproducer: asn1parse-reproduce crash-9969db8603e644ddc0ba3459b51eac7a2c4b729b

Make i long.

Reviewed-by: Rich Salz <rsalz@openssl.org>

The argument 'argv' in 'main' is a short pointer to a short pointer on
VMS, regardless of initial pointer size.  We must therefore make sure
that 'copy_argv' gets a 32-bit pointer for argv, and that the copied
argv is used for the rest of main().

This introduces the local type argv_t, which will have correct pointer
size in all cases (and be harmless on all other platforms) as well as
the macro Argv, which is defined as 'copied_argv' or 'argv', as the
case may be.

Reviewed-by: Andy Polyakov <appro@openssl.org>

setbuf() is only for 32-bit pointers.  If compiled with /POINTER_SIZE=64,
we get a nasty warning about possible loss of data.  However, since
the only pointer used in the call is a FILE *, and the C RTL shouldn't
give us a pointer above the first 4GB, it's safe to turn off the
warning for this call.

Reviewed-by: Andy Polyakov <appro@openssl.org>

On VMS, the C compiler can work with 32-bit and 64-bit pointers, and
the command line determines what the initial pointer size shall be.

However, there is some functionality that only works with 32-bit
pointers.  In this case, it's gethostbyname(), getservbyname() and
accompanying structures, so we need to make sure that we define our
own pointers as 32-bit ones.

Furthermore, there seems to be a bug in VMS C netdb.h, where struct
addrinfo is always defined with 32-bit pointers no matter what, but
the functions handling it are adapted to the initial pointer size.
This leads to pointer size warnings when compiling with
/POINTER_SIZE=64.  The workaround is to force struct addrinfo to be
the 64-bit variant if the initial pointer size is 64.

Reviewed-by: Andy Polyakov <appro@openssl.org>

'proto' wasn't properly used as a fallback in all appropriate cases.

Reviewed-by: Rich Salz <rsalz@openssl.org>

It displays the command's exit code as well.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

If the command file that app(), test(), perlapp(9 and perltest() are
looking for doesn't exist in the build tree, look for it in the source
tree as well.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Since OpenSSL::Test only redirects stderr to /dev/null when being run
through non-verbose test harness, this change allows the stderr output
to be displayed when verbosity is requested.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

The intention with that option is to allow extra flags to the perl
interpreter itself.

Reviewed-by: Emilia Käsper <emilia@openssl.org>

Cache the decoded public key when an X509_PUBKEY structure is initially
parsed so no locking is required. Ignore any decode errors.

When an application calls X509_PUBKEY_get0() subsequently it will either
get the cached key or the decode operation will be repeated which will
return an appropriate error.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

BIO_f_linebuffer() is now built by default instead of just on VMS, but
the prototype in the header was still only available on VMS.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Andy Polyakov <appro@openssl.org>

There are rare cases when an object file will only be used when
building a shared library.  To enable this, we introduce
SHARED_SOURCE:

    SHARED_SOURCE[libfoo]=dllmain.c

Reviewed-by: Andy Polyakov <appro@openssl.org>

Also, have it always be built, even though it's only (currently) used
on VMS.  That will assure it will get the same changes as all others.

Reviewed-by: Matt Caswell <matt@openssl.org>

Reviewed-by: Andy Polyakov <appro@openssl.org>

Previously, it was sufficient to have certSign in keyUsage when the
basicConstraints extension was missing.  That is still accepted in
a trust anchor, but is no longer accepted in an intermediate CA.

Reviewed-by: Rich Salz <rsalz@openssl.org>

The zlib config option was broken by the BIO opacity changes.

Reviewed-by: Rich Salz <rsalz@openssl.org>

The API contract is more tight than was previously documented -- the returned
string must be NUL-terminated, and the supplied buffer includes space for
the trailing NUL, so the maximum length that can be read in is reduced.

Clarify that the NUL is not included in the returned length, and fix the
spelling of "NUL-terminated" in a nearby spot.

Adjust punctuation to make a modest improvement to the grammar.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

The SSL, SSL_CTX, and SSL_SESSION indices were being referenced
incorrectly in the "_get_ex_new_index" functions.

Remove the STORE EX_DATA index; that functionality is gone.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

"vms-generic" already has some values, which were discarded.

Reviewed-by: Rich Salz <rsalz@openssl.org>

The warning MAYLOSEDATA3 is one you will always get when compiling
source that calculates the difference between two pointers with
/POINTER_SIZE=64.

The reason is quite simple, ptrdiff_t is always a 32-bit integer
regardless of pointer size, so the result of 'ptr1 - ptr2' can
potentially be larger than a 32-bit integer.  The compiler simply
warns you of that possibility.

However, we only use pointer difference within objects and strings,
all of them well within 2^32 bytes in size, so that operation is
harmless with our source, and we can therefore safely turn off that
warning.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

The private key is a scalar and should be sized by the order, not the
degree. See RFC 5915.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>

Reviewed-by: Richard Levitte <levitte@openssl.org>

Add the documentation for new BIO functions added as a result of
making BIO and BIO_METHOD opaque.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Richard Levitte <levitte@openssl.org>

Move the the BIO_METHOD and BIO structures into internal header files,
provide appropriate accessor methods and update all internal code to use
the new accessors where appropriate.

Reviewed-by: Richard Levitte <levitte@openssl.org>

BIO_METHODs are soon to be opaque so we need to have functions available
to set them up.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

RT#4439

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>

The space saving is >5x on x86_64 at ~40% performance penalty.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

MR: #2452

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

MR: #2452

Session resumption involves a version check, so version negotiation must
happen first. Currently, the DTLS implementation cannot do session
resumption in DTLS 1.0 because the ssl_version check always checks
against 1.2.

Switching the order also removes the need to fixup ssl_version in DTLS
version negotiation.

Signed-off-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

RT: #4392, MR: #2452