unified build scheme: add and document the "unified" driving engine (ddf1847d) · Commits · CYBER - Cyber Security / TS 103 523 MSP / TLMSP / TLMSP OpenSSL

Configurations/README

+172 −1

Original line number	Diff line number	Diff line
		Configurations of OpenSSL target platforms
		------------------------------------------
		==========================================

		Target configurations are a collection of facts that we know about
		different platforms and their capabilities. We organise them in a
		@@ -427,3 +427,174 @@ or:
		RENAME[libcrypto]=ossl_libcrypto
		RENAME[libssl]=ossl_libssl
		ENDIF


		Build-file programming with the "unified" build system
		======================================================

		"Build files" are called "Makefile" on Unix-like operating systems,
		"descrip.mms" for MMS on VMS, "makefile" for nmake on Windows, etc.

		To use the "unified" build system, the target configuration needs to
		set the three items 'build_scheme', 'build_file' and 'build_command'.
		In the rest of this section, we will assume that 'build_scheme' is set
		to "unified" (see the configurations documentation above for the
		details).

		For any name given by 'build_file', the "unified" system expects a
		template file in Configurations/ named like the build file, with
		".tmpl" appended, or in case of possible ambiguity, a combination of
		the second 'build_scheme' list item and the 'build_file' name. For
		example, if 'build_file' is set to "Makefile", the template could be
		Configurations/Makefile.tmpl or Configurations/unix-Makefile.tmpl.
		In case both Configurations/unix-Makefile.tmpl and
		Configurations/Makefile.tmpl are present, the former takes
		precedence.

		The build-file template is processed with the perl module
		Text::Template, using "{-" and "-}" as delimiters that enclose the
		perl code fragments that generate configuration-dependent content.
		Those perl fragments have access to all the hash variables from
		configdata.pem.

		The build-file template is expected to define at least the following
		perl functions in a perl code fragment enclosed with "{-" and "-}".
		They are all expected to return a string with the lines they produce.

		src2dep - function that produces build file lines to get the
		dependencies for an object file into a dependency
		file.

		It's called like this:

		src2dep(obj => "PATH/TO/objectfile",
		srcs => [ "PATH/TO/sourcefile", ... ],
		incs => [ "INCL/PATH", ... ]);

		'obj' has the dependent object file as well as
		object file the dependencies are for; it's without
		extension, src2dep() is expected to add that.
		'srcs' has the list of source files to build the
		object file, with the first item being the source
		file that directly corresponds to the object file.
		'incs' is a list of include file directories.

		src2obj - function that produces build file lines to build an
		object file from source files and associated data.

		It's called like this:

		src2obj(obj => "PATH/TO/objectfile",
		srcs => [ "PATH/TO/sourcefile", ... ],
		deps => [ "dep1", ... ],
		incs => [ "INCL/PATH", ... ]);

		'obj' has the intended object file without
		extension, src2obj() is expected to add that.
		'srcs' has the list of source files to build the
		object file, with the first item being the source
		file that directly corresponds to the object file.
		'deps' is a list of dependencies. 'incs' is a list
		of include file directories.

		obj2lib - function that produces build file lines to build a
		static library file ("libfoo.a" in Unix terms) from
		object files.

		called like this:

		obj2lib(lib => "PATH/TO/libfile",
		objs => [ "PATH/TO/objectfile", ... ]);

		'lib' has the intended library file name without
		extension, obj2lib is expected to add that. 'objs'
		has the list of object files (also without
		extension) to build this library.

		libobj2shlib - function that produces build file lines to build a
		shareable object library file ("libfoo.so" in Unix
		terms) from the corresponding static library file
		or object files.

		called like this:

		libobj2shlib(shlib => "PATH/TO/shlibfile",
		lib => "PATH/TO/libfile",
		objs => [ "PATH/TO/objectfile", ... ],
		deps => [ "PATH/TO/otherlibfile", ... ],
		ordinals => [ "word", "/PATH/TO/ordfile" ]);

		'lib' has the intended library file name without
		extension, libobj2shlib is expected to add that.
		'shlib' has the correcponding shared library name
		without extension. 'deps' has the list of other
		libraries (also without extension) this library
		needs to be linked with. 'objs' has the list of
		object files (also without extension) to build
		this library. 'ordinals' MAY be present, and when
		it is, its value is an array where the word is
		"crypto" or "ssl" and the file is one of the ordinal
		files util/libeay.num or util/ssleay.num in the
		source directory.

		This function has a choice; it can use the
		corresponding static library as input to make the
		shared library, or the list of object files.

		obj2dynlib - function that produces build file lines to build a
		dynamically loadable library file ("libfoo.so" on
		Unix) from object files.

		called like this:

		obj2dynlib(lib => "PATH/TO/libfile",
		objs => [ "PATH/TO/objectfile", ... ],
		deps => [ "PATH/TO/otherlibfile",
		... ]);

		This is almost the same as libobj2shlib, but the
		intent is to build a shareable library that can be
		loaded in runtime (a "plugin"...). The differences
		are subtle, one of the most visible ones is that the
		resulting shareable library is produced from object
		files only.

		obj2bin - function that produces build file lines to build an
		executable file from object files.

		called like this:

		obj2bin(bin => "PATH/TO/binfile",
		objs => [ "PATH/TO/objectfile", ... ],
		deps => [ "PATH/TO/libfile", ... ]);

		'bin' has the intended executable file name
		without extension, obj2bin is expected to add
		that. 'objs' has the list of object files (also
		without extension) to build this library. 'deps'
		has the list of library files (also without
		extension) that the programs needs to be linked
		with.

		in2script - function that produces build file lines to build a
		script file from some input.

		called like this:

		in2script(script => "PATH/TO/scriptfile",
		sources => [ "PATH/TO/infile", ... ]);

		'script' has the intended script file name.
		'sources' has the list of source files to build the
		resulting script from.

		In all cases, file file paths are relative to the build tree top, and
		the build file actions run with the build tree top as current working
		directory.

		Make sure to end the section with these functions with a string that
		you thing is apropriate for the resulting build file. If nothing
		else, end it like this:

		""; # Make sure no lingering values end up in the Makefile
		-}

Configurations/common.tmpl

0 → 100644

+118 −0

Original line number	Diff line number	Diff line
		{- # -- Mode: perl --

		my $a;

		# resolvedepends and reducedepends work in tandem to make sure
		# there are no duplicate dependencies and that they are in the
		# right order. This is especially used to sort the list of
		# libraries that a build depends on.
		sub resolvedepends {
		my $thing = shift;
		my @listsofar = @_; # to check if we're looping
		my @list = @{$unified_info{depends}->{$thing}};
		my @newlist = ();
		if (scalar @list) {
		foreach my $item (@list) {
		# It's time to break off when the dependency list starts looping
		next if grep { $_ eq $item } @listsofar;
		push @newlist, $item, resolvedepends($item, @listsofar, $item);
		}
		}
		@newlist;
		}
		sub reducedepends {
		my @list = @_;
		my @newlist = ();
		while (@list) {
		my $item = shift @list;
		push @newlist, $item
		unless grep { $item eq $_ } @list;
		}
		@newlist;
		}

		# doobj is responsible for producing all the recipes that build
		# object files as well as dependency files.
		sub doobj {
		my $obj = shift;
		(my $obj_no_o = $obj) =~ s\|\.o$\|\|;
		my $bin = shift;
		if (@{$unified_info{sources}->{$obj}}) {
		$OUT .= src2obj(obj => $obj_no_o,
		srcs => $unified_info{sources}->{$obj},
		deps => [ reducedepends(resolvedepends($obj)) ],
		incs => [ @{$unified_info{includes}->{$bin}},
		@{$unified_info{includes}->{$obj}} ]);
		$OUT .= src2dep(obj => $obj_no_o,
		srcs => $unified_info{sources}->{$obj},
		incs => [ @{$unified_info{includes}->{$bin}},
		@{$unified_info{includes}->{$obj}} ]);
		}
		}

		# dolib is responsible for building libraries. It will call
		# libobj2shlib is shared libraries are produced, and obj2lib in all
		# cases. It also makes sure all object files for the library are
		# built.
		sub dolib {
		my $lib = shift;
		if (!$config{no_shared}) {
		my %ordinals =
		$unified_info{ordinals}->{$lib}
		? (ordinals => $unified_info{ordinals}->{$lib}) : ();
		$OUT .= libobj2shlib(shlib => $unified_info{sharednames}->{$lib},
		lib => $lib,
		objs => [ map { (my $x = $_) =~ s\|\.o$\|\|; $x }
		@{$unified_info{sources}->{$lib}} ],
		deps => [ reducedepends(resolvedepends($lib)) ],
		%ordinals);
		}
		$OUT .= obj2lib(lib => $lib,
		objs => [ map { (my $x = $_) =~ s\|\.o$\|\|; $x }
		@{$unified_info{sources}->{$lib}} ]);
		map { doobj($_, $lib, intent => "lib") } @{$unified_info{sources}->{$lib}};
		}

		# doengine is responsible for building engines. It will call
		# obj2dynlib, and also makes sure all object files for the library
		# are built.
		sub doengine {
		my $lib = shift;
		$OUT .= obj2dynlib(lib => $lib,
		objs => [ map { (my $x = $_) =~ s\|\.o$\|\|; $x }
		@{$unified_info{sources}->{$lib}} ],
		deps => [ resolvedepends($lib) ]);
		map { doobj($_, $lib, intent => "lib") } @{$unified_info{sources}->{$lib}};
		}

		# dobin is responsible for building programs. It will call obj2bin,
		# and also makes sure all object files for the library are built.
		sub dobin {
		my $bin = shift;
		my $deps = [ reducedepends(resolvedepends($bin)) ];
		$OUT .= obj2bin(bin => $bin,
		objs => [ map { (my $x = $_) =~ s\|\.o$\|\|; $x }
		@{$unified_info{sources}->{$bin}} ],
		deps => $deps);
		map { doobj($_, $bin, intent => "bin") } @{$unified_info{sources}->{$bin}};
		}

		# dobin is responsible for building scripts from templates. It will
		# call in2script.
		sub doscript {
		my $script = shift;
		$OUT .= in2script(script => $script,
		sources => $unified_info{sources}->{$script});
		}

		# Build all known libraries, engines, programs and scripts.
		# Everything else will be handled as a consequence.
		map { dolib($_) } @{$unified_info{libraries}};
		map { doengine($_) } @{$unified_info{engines}};
		map { dobin($_) } @{$unified_info{programs}};
		map { doscript($_) } @{$unified_info{scripts}};

		# Finally, should there be any applicable BEGINRAW/ENDRAW sections,
		# they are added here.
		$OUT .= $_."\n" foreach(@{$unified_info{rawlines}});
		-}

Configure

+41 −12

Original line number	Diff line number	Diff line
		@@ -844,6 +844,9 @@ $target{nm} = "nm";
		$target{build_scheme} = [ $target{build_scheme} ]
		if ref($target{build_scheme}) ne "ARRAY";

		my ($builder, $builder_platform, @builder_opts) =
		@{$target{build_scheme}};

		# if $config{prefix}/lib$target{multilib} is not an existing directory, then
		# assume that it's not searched by linker automatically, in
		# which case adding $target{multilib} suffix causes more grief than
		@@ -976,7 +979,7 @@ if (!$config{no_shared})
		}
		}

		if ($target{build_scheme}->[0] ne "mk1mf")
		if ($builder ne "mk1mf")
		{
		# add {no-}static-engine to options to allow mkdef.pl to work without extra arguments
		if ($config{no_shared})
		@@ -1163,7 +1166,17 @@ if ($strict_warnings)
		# If we use the unified build, collect information from build.info files
		my %unified_info = ();

		if ($target{build_scheme}->[0] eq "unified") {
		if ($builder eq "unified") {
		# Store the name of the template file we will build the build file from
		# in %config. This may be useful for the build file itself.
		my $build_file_template =
		catfile($srcdir, "Configurations",
		$builder_platform."-".$target{build_file}.".tmpl");
		$build_file_template =
		catfile($srcdir, "Configurations", $target{build_file}.".tmpl")
		if (! -f $build_file_template);
		$config{build_file_template} = $build_file_template;

		use lib catdir(dirname(__FILE__),"util");
		use with_fallback qw(Text::Template);

		@@ -1323,7 +1336,7 @@ if ($target{build_scheme}->[0] eq "unified") {
		next if @skip && $skip[$#skip] <= 0;
		push @rawlines, $_
		if ($target_kind eq $target{build_file}
		\|\| $target_kind eq $target{build_file}."(".$target{build_scheme}->[1].")");
		\|\| $target_kind eq $target{build_file}."(".$builder_platform.")");
		}
		},
		qr/^(?:#.\|\s)$/ => sub { },
		@@ -1601,7 +1614,7 @@ print OUT <<"EOF";
		);

		EOF
		if ($target{build_scheme}->[0] eq "unified") {
		if ($builder eq "unified") {
		my $recurse;
		$recurse = sub {
		my $indent = shift;
		@@ -1646,7 +1659,7 @@ EOF
		print OUT "1;\n";
		close(OUT);

		die <<"EOF" if $target{build_scheme}->[0] ne "unified" && $srcdir ne $blddir;
		die <<"EOF" if $builder ne "unified" && $srcdir ne $blddir;

		***** Trying building anywhere else than in the source tree will not
		***** work for target $config{target}. To make it possible, it needs
		@@ -1654,7 +1667,7 @@ die <<"EOF" if $target{build_scheme}->[0] ne "unified" && $srcdir ne $blddir;

		EOF

		print "IsMK1MF =", ($target{build_scheme}->[0] eq "mk1mf" ? "yes" : "no"), "\n";
		print "IsMK1MF =", ($builder eq "mk1mf" ? "yes" : "no"), "\n";
		print "CC =$target{cc}\n";
		print "CFLAG =$config{cflags}\n";
		print "LFLAG =$config{lflags}\n";
		@@ -1735,7 +1748,20 @@ sub build_Makefile {

		my %builders = (
		unified => sub {
		die "unified build currently does nothing";
		run_dofile(catfile($blddir, $target{build_file}),
		$config{build_file_template},
		catfile($srcdir, "Configurations", "common.tmpl"));

		my $make_command = "$make PERL=\'$config{perl}\'";
		my $make_targets = "";
		$make_targets .= " depend"
		if $config{depflags} ne $default_depflags && $make_depend;
		(system $make_command.$make_targets) == 0
		or die "make $make_targets failed"
		if $make_targets ne "";
		if ($config{depflags} ne $default_depflags && !$make_depend) {
		$warn_make_depend++;
		}
		},
		unixmake => sub {
		build_Makefile();
		@@ -1745,14 +1771,18 @@ my %builders = (

		my $make_command = "$make PERL=\'$config{perl}\'";
		my $make_targets = "";
		$make_targets .= " depend" if $config{depflags} ne $default_depflags && $make_depend;
		(system $make_command.$make_targets) == 0 or die "make $make_targets failed"
		$make_targets .= " depend"
		if $config{depflags} ne $default_depflags && $make_depend;
		(system $make_command.$make_targets) == 0
		or die "make $make_targets failed"
		if $make_targets ne "";

		if ($config{depflags} ne $default_depflags && !$make_depend) {
		$warn_make_depend++;
		}
		},
		mk1mf => sub {
		my $platform = shift;
		# The only reason we do this is to have something to build MINFO from
		build_Makefile();

		@@ -1769,7 +1799,7 @@ EOF
		close(OUT);

		# create the ms/version32.rc file if needed
		if (! grep /^netware/, @{$target{build_scheme}}) {
		if ($platform eq "netware") {
		my ($v1, $v2, $v3, $v4);
		if ($config{version_num} =~ /^0x([0-9a-f]{1})([0-9a-f]{2})([0-9a-f]{2})([0-9a-f]{2})([0-9a-f]{1})L$/i) {
		$v1=hex $1;
		@@ -1832,8 +1862,7 @@ EOF
		},
		);

		my ($builder, @builder_opts) = @{$target{build_scheme}};
		$builders{$builder}->(@builder_opts);
		$builders{$builder}->($builder_platform, @builder_opts);

		print <<"EOF";