Commit bcb3bdc6 authored by Eric Covener's avatar Eric Covener
Browse files

backport HTTP strict processing from 2.4.x. 

This backport is hand-constructed from many commits of HTTP strict,
subsequent fixes, as well as dependencies that hadn't been backported
to 2.2.x. 

The bulk is merged from httpd-2.4.x-merge-http-strict branch r1767941 - r1775671
Further details on httpd-2.2.x-merge-http-strict

Submitted By: sf, wrowe
Reviewed By: covener, ylavic, wrowe



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x@1777405 13f79535-47bb-0310-9956-ffa450edef68
parent 75d69581

CHANGES

+35 −0
Original line number Diff line number Diff line 
                                                         -*- coding: utf-8 -*-

Changes with Apache 2.2.32



  *) SECURITY: CVE-2016-8743 (cve.mitre.org)

     Enforce HTTP request grammar corresponding to RFC7230 for request lines

     and request headers, to prevent response splitting and cache pollution by

     malicious clients or downstream proxies. [William Rowe, Stefan Fritsch]



  *) core: CVE-2016-5387: Mitigate [f]cgi "httpoxy" issues.

     [Dominic Scheirlinck <dominic vendhq.com>, Yann Ylavic]



  *) Validate HTTP response header grammar defined by RFC7230, resulting

     in a 500 error in the event that invalid response header contents are

     detected when serving the response, to avoid response splitting and cache

     pollution by malicious clients, upstream servers or faulty modules.

     [Stefan Fritsch, Eric Covener, Yann Ylavic]



  *) core: Avoid a possible truncation of the faulty header included in the

     HTML response when LimitRequestFieldSize is reached.  [Yann Ylavic]



  *) core: Enforce LimitRequestFieldSize after multiple headers with the same

     name have been merged. [Stefan Fritsch]



  *) core: Drop Content-Length header and message-body from HTTP 204 responses.

     PR 51350 [Luca Toscano]



  *) core: Permit unencoded ';' characters to appear in proxy requests and

     Location: response headers. Corresponds to modern browser behavior.

     [William Rowe]



  *) core: ap_rgetline_core now pulls from r->proto_input_filters.



  *) core: Correctly parse an IPv6 literal host specification in an absolute

     URL in the request line. [Stefan Fritsch]



  *) core: New directive RegisterHttpMethod for registering non-standard

     HTTP methods. [Stefan Fritsch]



  *) Fix potential rejection of valid MaxMemFree and ThreadStackSize

     directives.  [Mike Rumph <mike.rumph oracle.com>]
@@ -45,6 +77,9 @@ Changes with Apache 2.2.32 
     failures under Visual Studio 2015 and other mismatched MSVCRT flavors.

     PR59630 [Jan Ehrhardt <phpdev ehrhardt.nl>]



  *) core: Support custom ErrorDocuments for HTTP 501 and 414 status codes.

     PR 57167 [Edward Lu <Chaosed0 gmail.com>]



Changes with Apache 2.2.31



  *) Correct win32 build issues for mod_proxy exports, OpenSSL 1.0.x headers.

docs/manual/mod/core.xml

+90 −0
Original line number Diff line number Diff line
@@ -1440,6 +1440,82 @@ MIME content-type</description> 
</usage>

</directivesynopsis>



<directivesynopsis>

<name>HttpProtocolOptions</name>

<description>Modify restrictions on HTTP Request Messages</description>

<syntax>HttpProtocolOptions [Strict|Unsafe] [RegisteredMethods|LenientMethods]

 [Allow0.9|Require1.0]</syntax>

<default>HttpProtocolOptions Strict LenientMethods Allow0.9</default>

<contextlist><context>server config</context>

<context>virtual host</context></contextlist>

<compatibility>2.2.32 or 2.4.24 and later</compatibility>



<usage>

    <p>This directive changes the rules applied to the HTTP Request Line

    (<a href="https://tools.ietf.org/html/rfc7230#section-3.1.1"

      >RFC 7230 &sect;3.1.1</a>) and the HTTP Request Header Fields

    (<a href="https://tools.ietf.org/html/rfc7230#section-3.2"

      >RFC 7230 &sect;3.2</a>), which are now applied by default or using

    the <code>Strict</code> option. Due to legacy modules, applications or

    custom user-agents which must be deperecated the <code>Unsafe</code>

    option has been added to revert to the legacy behaviors. These rules

    are applied prior to request processing, so must be configured at the

    global or default (first) matching virtual host section, by IP/port

    interface (and not by name) to be honored.</p>



    <p>Prior to the introduction of this directive, the Apache HTTP Server

    request message parsers were tolerant of a number of forms of input

    which did not conform to the protocol.

    <a href="https://tools.ietf.org/html/rfc7230#section-9.4"

      >RFC 7230 &sect;9.4 Request Splitting</a> and

    <a href="https://tools.ietf.org/html/rfc7230#section-9.5"

      >&sect;9.5 Response Smuggling</a> call out only two of the potential

    risks of accepting non-conformant request messages, while

    <a href="https://tools.ietf.org/html/rfc7230#section-3.5"

         >RFC 7230 &sect;3.5</a> "Message Parsing Robustness" identify the

    risks of accepting obscure whitespace and request message formatting. 

    As of the introduction of this directive, all grammer rules of the

    specification are enforced in the default <code>Strict</code> operating

    mode, and the strict whitespace suggested by section 3.5 is enforced

    and cannot be relaxed.</p>



    <p>Users are strongly cautioned against toggling the <code>Unsafe</code>

    mode of operation, particularly on outward-facing, publicly accessible

    server deployments.  If an interface is required for faulty monitoring

    or other custom service consumers running on an intranet, users should

    toggle the Unsafe option only on a specific virtual host configured

    to service their internal private network.</p>



    <p>Reviewing the messages logged to the <directive>ErrorLog</directive>,

    configured with <directive>LogLevel</directive> <code>debug</code> level,

    can help identify such faulty requests along with their origin.

    Users should pay particular attention to the 400 responses in the access

    log for invalid requests which were unexpectedly rejected.</p>



    <p><a href="https://tools.ietf.org/html/rfc7231#section-4.1"

         >RFC 7231 &sect;4.1</a> "Request Methods" "Overview" requires that

    origin servers shall respond with an error when an unsupported method

    is encountered in the request line. This already happens when the

    <code>LenientMethods</code> option is used, but administrators may wish

    to toggle the <code>RegisteredMethods</code> option and register any

    non-standard methods using the <directive>RegisterHttpMethod</directive>

    directive, particularly if the <code>Unsafe</code> option has been toggled.

    The <code>RegisteredMethods</code> option should <strong>not</strong>

    be toggled for forward proxy hosts, as the methods supported by the

    origin servers are unknown to the proxy server.</p>



    <p><a href="https://tools.ietf.org/html/rfc2616#section-19.6"

         >RFC 2616 &sect;19.6</a> "Compatibility With Previous Versions" had

    encouraged HTTP servers to support legacy HTTP/0.9 requests. RFC 7230

    superceeds this with "The expectation to support HTTP/0.9 requests has

    been removed" and offers additional comments in 

    <a href="https://tools.ietf.org/html/rfc7230#appendix-A"

      >RFC 7230 Appendix A</a>. The <code>Require1.0</code> option allows

    the user to remove support of the default <code>Allow0.9</code> option's

    behavior.</p>

</usage>

</directivesynopsis>



<directivesynopsis type="section">

<name>IfDefine</name>

<description>Encloses directives that will be processed only
@@ -3681,5 +3757,19 @@ hostname or IP address</description> 
</usage>

</directivesynopsis>



<directivesynopsis>

<name>RegisterHttpMethod</name>

<description>Register non-standard HTTP methods</description>

<syntax>RegisterHttpMethod <var>method</var> [<var>method</var> [...]]</syntax>

<contextlist><context>server config</context></contextlist>

<usage>

<p>HTTP Methods that are not conforming to the relvant RFCs are normally

rejected by request processing in Apache HTTPD. To avoid this, modules

can register non-standard HTTP methods they support.

The <directive>RegisterHttpMethod</directive> allows to register such

methods manually. This can be useful for if such methods are forwared

for external processing, e.g. to a CGI script.</p>

</usage>

</directivesynopsis>



</modulesynopsis>

include/ap_mmn.h

+8 −1
Original line number Diff line number Diff line
@@ -160,6 +160,13 @@ 
 * 20051115.40 (2.2.30) Add ap_map_http_request_error()

 * 20051115.41 (2.2.32) Add s member to proxy_server_conf struct and server

 *                      member to proxy_worker struct.

 * 20051115.42 (2.2.32) Add http09_enable, http_conformance, and

 *                      http_methods to core_server_config

 *                      Add ap_scan_http_field_token(),

 *                      ap_scan_http_field_content(),

 *                      and ap_scan_vchar_obstext()

 *                      Replaced fold boolean with with multiple bit flags

 *                      to ap_[r]getline()

 */



#define MODULE_MAGIC_COOKIE 0x41503232UL /* "AP22" */
@@ -167,7 +174,7 @@ 
#ifndef MODULE_MAGIC_NUMBER_MAJOR

#define MODULE_MAGIC_NUMBER_MAJOR 20051115

#endif

#define MODULE_MAGIC_NUMBER_MINOR 41                    /* 0...n */

#define MODULE_MAGIC_NUMBER_MINOR 42                    /* 0...n */



/**

 * Determine if the server's current MODULE_MAGIC_NUMBER is at least a

include/http_core.h

+15 −0
Original line number Diff line number Diff line
@@ -627,6 +627,21 @@ typedef struct { 
#define AP_MERGE_TRAILERS_DISABLE  2

    int merge_trailers;



#define AP_HTTP09_UNSET   0

#define AP_HTTP09_ENABLE  1

#define AP_HTTP09_DISABLE 2

    char http09_enable;



#define AP_HTTP_CONFORMANCE_UNSET     0

#define AP_HTTP_CONFORMANCE_UNSAFE    1

#define AP_HTTP_CONFORMANCE_STRICT    2

    char http_conformance;



#define AP_HTTP_METHODS_UNSET         0

#define AP_HTTP_METHODS_LENIENT       1

#define AP_HTTP_METHODS_REGISTERED    2

    char http_methods;



} core_server_config;



/* for AddOutputFiltersByType in core.c */

include/http_protocol.h

+12 −5
Original line number Diff line number Diff line
@@ -510,17 +510,22 @@ AP_DECLARE(int) ap_get_basic_auth_pw(request_rec *r, const char **pw); 
 */

AP_CORE_DECLARE(void) ap_parse_uri(request_rec *r, const char *uri);



#define AP_GETLINE_FOLD 1 /* Whether to merge continuation lines */

#define AP_GETLINE_CRLF 2 /*Whether line ends must be in the form CR LF */



/**

 * Get the next line of input for the request

 * @param s The buffer into which to read the line

 * @param n The size of the buffer

 * @param r The request

 * @param fold Whether to merge continuation lines

 * @param flags Bit flag of multiple parsing options

 *              AP_GETLINE_FOLD Whether to merge continuation lines

 *              AP_GETLINE_CRLF Whether line ends must be in the form CR LF

 * @return The length of the line, if successful

 *         n, if the line is too big to fit in the buffer

 *         -1 for miscellaneous errors

 */

AP_DECLARE(int) ap_getline(char *s, int n, request_rec *r, int fold);

AP_DECLARE(int) ap_getline(char *s, int n, request_rec *r, int flags);



/**

 * Get the next line of input for the request
@@ -538,7 +543,9 @@ AP_DECLARE(int) ap_getline(char *s, int n, request_rec *r, int fold); 
 * @param n The size of the buffer

 * @param read The length of the line.

 * @param r The request

 * @param fold Whether to merge continuation lines

 * @param flags Bit flag of multiple parsing options

 *              AP_GETLINE_FOLD Whether to merge continuation lines

 *              AP_GETLINE_CRLF Whether line ends must be in the form CR LF

 * @param bb Working brigade to use when reading buckets

 * @return APR_SUCCESS, if successful

 *         APR_ENOSPC, if the line is too big to fit in the buffer
@@ -547,7 +554,7 @@ AP_DECLARE(int) ap_getline(char *s, int n, request_rec *r, int fold); 
#if APR_CHARSET_EBCDIC

AP_DECLARE(apr_status_t) ap_rgetline(char **s, apr_size_t n, 

                                     apr_size_t *read,

                                     request_rec *r, int fold,

                                     request_rec *r, int flags,

                                     apr_bucket_brigade *bb);

#else /* ASCII box */

#define ap_rgetline(s, n, read, r, fold, bb) \
@@ -557,7 +564,7 @@ AP_DECLARE(apr_status_t) ap_rgetline(char **s, apr_size_t n, 
/** @see ap_rgetline */

AP_DECLARE(apr_status_t) ap_rgetline_core(char **s, apr_size_t n, 

                                          apr_size_t *read,

                                          request_rec *r, int fold,

                                          request_rec *r, int flags,

                                          apr_bucket_brigade *bb);



/**