With read pipelining we use multiple SSL3_RECORD structures for reading.
There are SSL_MAX_PIPELINES (32) of them defined (typically not all of these
would be used). Each one has a 16k compression buffer allocated! This
results in a significant amount of memory being consumed which, most of the
time, is not needed.  This change swaps the allocation of the compression
buffer to be lazy so that it is only done immediately before it is actually
used.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Read pipelining is controlled in a slightly different way than with write
pipelining. While reading we are constrained by the number of records that
the peer (and the network) can provide to us in one go. The more records
we can get in one go the more opportunity we have to parallelise the
processing.

There are two parameters that affect this:
* The number of pipelines that we are willing to process in one go. This is
controlled by max_pipelines (as for write pipelining)
* The size of our read buffer. A subsequent commit will provide an API for
adjusting the size of the buffer.

Another requirement for this to work is that "read_ahead" must be set. The
read_ahead parameter will attempt to read as much data into our read buffer
as the network can provide. Without this set, data is read into the read
buffer on demand. Setting the max_pipelines parameter to a value greater
than 1 will automatically also turn read_ahead on.

Finally, the read pipelining as currently implemented will only parallelise
the processing of application data records. This would only make a
difference for renegotiation so is unlikely to have a significant impact.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add dummy pipline support to dasync for the aes128_cbc_hmac_sha1 cipher.
This is treated as an AEAD cipher.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add the options min_send_frag and max_pipelines to s_server and s_client
in order to control pipelining capabilities. This will only have an effect
if a pipeline capable cipher is used (such as the one provided by the
dasync engine).

Reviewed-by: Tim Hudson <tjh@openssl.org>

Use the new pipeline cipher capability to encrypt multiple records being
written out all in one go. Two new SSL/SSL_CTX parameters can be used to
control how this works: max_pipelines and split_send_fragment.

max_pipelines defines the maximum number of pipelines that can ever be used
in one go for a single connection. It must always be less than or equal to
SSL_MAX_PIPELINES (currently defined to be 32). By default only one
pipeline will be used (i.e. normal non-parallel operation).

split_send_fragment defines how data is split up into pipelines. The number
of pipelines used will be determined by the amount of data provided to the
SSL_write call divided by split_send_fragment. For example if
split_send_fragment is set to 2000 and max_pipelines is 4 then:
SSL_write called with 0-2000 bytes == 1 pipeline used
SSL_write called with 2001-4000 bytes == 2 pipelines used
SSL_write called with 4001-6000 bytes == 3 pipelines used
SSL_write_called with 6001+ bytes == 4 pipelines used

split_send_fragment must always be less than or equal to max_send_fragment.
By default it is set to be equal to max_send_fragment. This will mean that
the same number of records will always be created as would have been
created in the non-parallel case, although the data will be apportioned
differently. In the parallel case data will be spread equally between the
pipelines.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Implement aes128-cbc as a pipeline capable cipher in the dasync engine.
As dasync is just a dummy engine, it actually just performs the parallel
encrypts/decrypts in serial.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Add a flag to indicate that a cipher is capable of performing
"pipelining", i.e. multiple encrypts/decrypts in parallel. Also add some
new ctrls that ciphers will need to implement if they are pipeline capable.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

- Only build & test two configurations. Make all the
  other build variants buildonly on gcc (clang on osx).
- Don't build with default clang at all on linux.
- Only use gcc-5 and clang-3.6 for the sanitizer builds. Re-running
  e.g. CONFIG_OPTS="shared" with them seems redundant.

Reviewed-by: Richard Levitte <levitte@openssl.org>

This reverts commit 963bb621

.

Reviewed-by: Tim Hudson <tjh@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>

Reviewed-by: Richard Levitte <levitte@openssl.org>

Strictly speaking, it isn't stdio and file access which offend me here;
it's the fact that UEFI doesn't provide a strdup() function. But the
fact that it's pointless without file access is a good enough excuse for
compiling it out.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

This isn't a file access function; it's still present.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

UEFI needs this too. Don't keep it only in the Windows/DOS ifdef block.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

This replaces SHUTDOWN/SHUTDOWN2 with BIO_closesocket.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Avoids modifying certificate reference count, and thereby avoids locking.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Without this, the peer certificate would never be deleted, resulting in
a memory leak.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

It was unexpected that OpenSSL::Test::setup() should be called twice
by the same recipe.  However, that may happen if a recipe combines
OpenSSL::Test and OpenSSL::Test::Simple, which can be a sensible thing
to do.  Therefore, we now allow it.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>

This gets rid of the BEGINRAW..ENDRAW sections in engines/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Rich Salz <rsalz@openssl.org>

This gets rid of the BEGINRAW..ENDRAW sections in crypto/bn/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Should it be needed because the recipes within a RAW section might
clash with those generated by Configure, it's possible to tell it
not to generate them with the use of OVERRIDES, for example:

    SOURCE[libfoo]=foo.c bar.c

    OVERRIDES=bar.o
    BEGINRAW[Makefile(unix)]
    bar.o: bar.c
        $(CC) $(CFLAGS) -DSPECIAL -c -o $@ $<
    ENDRAW[Makefile(unix)]

Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Rich Salz <rsalz@openssl.org>

In some cases, one might want to generate some source files from
others, that's done as follows:

    GENERATE[foo.s]=asm/something.pl $(CFLAGS)
    GENERATE[bar.s]=asm/bar.S

The value of each GENERATE line is a command line or part of it.
Configure places no rules on the command line, except the the first
item muct be the generator file.  It is, however, entirely up to the
build file template to define exactly how those command lines should
be handled, how the output is captured and so on.

Reviewed-by: Rich Salz <rsalz@openssl.org>

Add the ASYNC_is_capable() function and use it in speed.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>

Swap the use of CRYPTO_LOCK_INIT in the init code to use the new threading
API mechanism for locking.

Reviewed-by: Richard Levitte <levitte@openssl.org>

The init code was using its own thread local code. Now we have a central
API for it we should use that instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>

The init code was using its own "once" implementation. Now that we have
the new thread API we should use that instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>