Commit 5c9e3447 authored by Jon Spillett's avatar Jon Spillett Committed by Richard Levitte
Browse files

Add Python Cryptography.io external test suite 



Add python cryptography testing instructions too

Reviewed-by: default avatarAndy Polyakov <appro@openssl.org>
Reviewed-by: default avatarRichard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2885)
parent 946a515a

.gitmodules

+4 −0
Original line number Diff line number Diff line 
[submodule "boringssl"]

	path = boringssl

	url = https://boringssl.googlesource.com/boringssl



[submodule "pyca.cryptography"]

	path = pyca-cryptography

	url = https://github.com/pyca/cryptography.git

pyca-cryptography @ 722235c4

Original line number Diff line number Diff line 
Subproject commit 722235c46721acfe8b601e7846730c3c1fa588c5

test/README.external

+42 −11
Original line number Diff line number Diff line
@@ -2,18 +2,18 @@ Running external test suites with OpenSSL 
=========================================



It is possible to integrate external test suites into OpenSSL's "make test".

This capability is considered a developer option and may not work on all

This capability is considered a developer option and does not work on all

platforms.



At the current time the only supported external suite is the one used by

BoringSSL.





The BoringSSL test suite

========================



In order to run the BoringSSL tests with OpenSSL, first checkout the BoringSSL

source code into an appropriate directory:

source code into an appropriate directory. This can be done in two ways:



1) Separately from the OpenSSL checkout using:



  $ git clone https://boringssl.googlesource.com/boringssl boringssl
@@ -23,13 +23,17 @@ BoringSSL repository. Later commits may or may not pass the test suite: 
  $ cd boringssl

  $ git checkout 490469f850e



From the OpenSSL source code configure to use the external tests:

2) Using the already configured submodule settings in OpenSSL:



  $ git submodule update --init



Configure the OpenSSL source code to enable the external tests:



$ cd ../openssl

$ ./config enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers \

  enable-external-tests



Note that using other config option than those given above may cause the tests

Note that using other config options than those given above may cause the tests

to fail.



Run the OpenSSL tests by providing the path to the BoringSSL test runner in the
@@ -67,3 +71,30 @@ within the OpenSSL source code. 
The community is encouraged to contribute patches which reduce the number of

suppressions that are currently present.





Python PYCA/Cryptography test suite

===================================



This python test suite runs cryptographic tests with a local OpenSSL build as

the implementation.



First checkout the PYCA/Cryptography module into ./pyca-cryptography using:



$ git submodule update --init



Then configure/build OpenSSL compatible with the python module:



$ ./config shared enable-external-tests

$ make



The tests will run in a python virtual environment which requires virtualenv

to be installed.



$ make test VERBOSE=1 TESTS=test_external_pyca



Test failures and suppressions

==============================



Some tests target older (<=1.0.2) versions so will not run. Other tests target

other crypto implementations so are not relevant. Currently no tests fail.

test/recipes/95-test_external_pyca.t

0 → 100644
+30 −0
Original line number Diff line number Diff line 
#! /usr/bin/env perl

# Copyright 2015-2016 The OpenSSL Project Authors. All Rights Reserved.

#

# Licensed under the OpenSSL license (the "License").  You may not use

# this file except in compliance with the License.  You can obtain a copy

# in the file LICENSE in the source distribution or at

# https://www.openssl.org/source/license.html





use OpenSSL::Test;

use OpenSSL::Test::Utils;

use OpenSSL::Test qw/:DEFAULT bldtop_file data_file srctop_file cmdstr/;



setup("test_external");



plan skip_all => "No external tests in this configuration"

    if disabled("external-tests");



plan tests => 1;



SKIP: {

    skip "PYCA Cryptography not available", 1

        if ! -f srctop_file("pyca-cryptography", "setup.py");

    skip "PYCA tests not available on Windows or VMS", 1

        if $^O =~ /^(VMS|MSWin32)$/;



    ok(run(cmd(["sh", data_file("cryptography.sh")])),

        "running Python Cryptography tests");

}

test/recipes/95-test_external_pyca_data/cryptography.sh

0 → 100755
+67 −0
Original line number Diff line number Diff line 
#!/bin/sh

#

# Copyright 2017 The OpenSSL Project Authors. All Rights Reserved.

#

# Licensed under the OpenSSL license (the "License").  You may not use

# this file except in compliance with the License.  You can obtain a copy

# in the file LICENSE in the source distribution or at

# https://www.openssl.org/source/license.html

#

# ====================================================================

# Copyright (c) 2017 Oracle and/or its affiliates.  All rights reserved.

#



#

# OpenSSL external testing using the Python Cryptography module

#

set -e



O_EXE=`pwd`/$BLDTOP/apps

O_BINC=`pwd`/$BLDTOP/include

O_SINC=`pwd`/$SRCTOP/include

O_LIB=`pwd`/$BLDTOP



export PATH=$O_EXE:$PATH

export LD_LIBRARY_PATH=$O_LIB:$LD_LIBRARY_PATH



# Check/Set openssl version

OPENSSL_VERSION=`openssl version | cut -f 2 -d ' '`



echo "------------------------------------------------------------------"

echo "Testing OpenSSL using Python Cryptography:"

echo "   CWD:                $PWD"

echo "   SRCTOP:             $SRCTOP"

echo "   BLDTOP:             $BLDTOP"

echo "   OpenSSL version:    $OPENSSL_VERSION"

echo "------------------------------------------------------------------"



cd $SRCTOP



# Create a python virtual env and activate

rm -rf venv-pycrypto

virtualenv venv-pycrypto

. ./venv-pycrypto/bin/activate



cd pyca-cryptography



pip install -q --requirement dev-requirements.txt



echo "------------------------------------------------------------------"

echo "Building cryptography"

echo "------------------------------------------------------------------"

python ./setup.py clean



CFLAGS="-I$O_BINC -I$O_SINC -L$O_LIB" python ./setup.py build



echo "------------------------------------------------------------------"

echo "Running tests"

echo "------------------------------------------------------------------"



CFLAGS="-I$O_BINC -I$O_SINC -L$O_LIB" python ./setup.py test



cd ../

deactivate

rm -rf venv-pycrypto



exit 0