Commit a4c7dcce authored by Jim Jagielski's avatar Jim Jagielski
Browse files

Merge r1780328, r1780329, r1781329, r1782164, r1782166, r1782193, r1778350,... 

Merge r1780328, r1780329, r1781329, r1782164, r1782166, r1782193, r1778350, r1781329, r1782194, r1782323, r1782418, r1782419, r1782482, r1782532, r1788040 from trunk:

Adjust as needed

debugging

add ProxyFCGISetEnvIf


Logging update

Follow up to r1782164: fix typo (closing double-quote).

Remove trailing whitespace : no functional change

PR60576: php-fpm broken w/ per-dir rewrites

Attempt to dig out of well-meaning fixes for generic fcgi backends
that negatively affected some FPM configs.

Adds ProxyFCGIBackendType 


add ProxyFCGISetEnvIf


trailing whitespace

Allow final admin-level fine-tuning

mod_proxy_fcgi: fix spelling in APLOG_INFO message

mod_proxy_fcgi: allow setting empty variables in ProxyFCGISetEnvIf

support unsetting vars



ProxyFCGISetEnvIf: reject invalid invocations, streamline unsets

Neither of the following makes sense:
- ProxyFCGISetEnvIf cond !VARIABLE value
- ProxyFCGISetEnvIf cond !

Error out in these cases. Also, don't execute the (unused) replacement
expression when unsetting.

drop the longhand version


Submitted by: jim, covener, jim, ylavic, jim, covener, covener, jim, jim, jchampion, jchampion, covener, jchampion, covener
Reviewed by: jim, covener, jchampion


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1788445 13f79535-47bb-0310-9956-ffa450edef68
parent 5485c367

CHANGES

+15 −0
Original line number Diff line number Diff line
@@ -2,6 +2,21 @@ 


Changes with Apache 2.4.26



  *) mod_proxy_fcgi: Add ProxyFCGISetEnvIf to fixup CGI environment

     variables just before invoking the FastCGI. [Eric Covener,

     Jacob Champion]



  *) mod_proxy: Allow the per-request environment variable "no-proxy" to

     be used as an alternative to ProxyPass /path !. This is primarily

     to set exceptions for ProxyPass specified in <Location> context.

    Use SetEnvIf, not SetEnv. [Eric Covener]



  *) mod_proxy_fcgi: Return to 2.4.20-and-earlier behavior of leaving

     a "proxy:fcgi://" prefix in the SCRIPT_FILENAME environment variable by

     default.  Add ProxyFCGIBackendType to allow the type of backend to be

     specified so these kinds of fixups can be restored without impacting

     FPM. PR60576 [Eric Covener, Jim Jagielski]



  *) mod_ssl: work around leaks on (graceful) restart. [Yann Ylavic]



  *) mod_ssl: Add support for OpenSSL 1.1.0. [Rainer Jung]

STATUS

+0 −25
Original line number Diff line number Diff line
@@ -137,31 +137,6 @@ PATCHES ACCEPTED TO BACKPORT FROM TRUNK: 
     2.4.x patch: http://people.apache.org/~covener/patches/ebcdic-interim.diff

     +1: covener, wrowe, ylavic (by inspection)



  *) PR 60576: 2.4.21 broke PHP-FPM with the patch to strip the bogus "proxy://"

     prefix from SCRIPT_FILENAME. We need to revert to the previous behavior

     ASAP to avoid any further hurdles with FCGI implementations while we figure

     this out.

     trunk patch: http://svn.apache.org/r1780328

                  http://svn.apache.org/r1780329

                  http://svn.apache.org/r1781329

                  http://svn.apache.org/r1782164

                  http://svn.apache.org/r1782166

                  http://svn.apache.org/r1782193

                  http://svn.apache.org/r1778350

                  http://svn.apache.org/r1781329

                  http://svn.apache.org/r1782194

                  http://svn.apache.org/r1782323

                  http://svn.apache.org/r1782418

                  http://svn.apache.org/r1782419

                  http://svn.apache.org/r1782482

                  http://svn.apache.org/r1782532

                  http://svn.apache.org/r1788040

     2.4.x patch: http://home.apache.org/~jim/patches/mod_proxy_fcgi-v3.patch

                 + http://svn.apache.org/r1788040

     +1: jim, covener, jchampion

     jchampion: holding off on promotion for an answer to jailletc36's ML

                question regarding `ProxyFCGIBackendType PHP-FPM`

     covener: added http://svn.apache.org/r1788040



PATCHES PROPOSED TO BACKPORT FROM TRUNK:

  [ New proposals should be added at the end of the list ]

docs/manual/mod/mod_proxy_fcgi.xml

+114 −0
Original line number Diff line number Diff line
@@ -195,4 +195,118 @@ ProxyPass "/myapp/" "balancer://myappcluster/" 
    </dl>

</section>



<directivesynopsis>

<name>ProxyFCGIBackendType</name>

<description>Specify the type of backend FastCGI application</description>

<syntax>ProxyFCGIBackendType FPM|GENERIC</syntax>

<default>ProxyFCGIBackendType FPM</default>

<contextlist><context>server config</context>

<context>virtual host</context><context>directory</context>

<context>.htaccess</context></contextlist>

<compatibility>Available in version 2.4.26 and later</compatibility>



<usage>

<p>This directive allows the type of backend FastCGI application to be

specified. Some FastCGI servers, such as PHP-FPM,  use historical quirks of

environment variables to identify the type of proxy server being used.  Set

this directive to "GENERIC" if your non PHP-FPM application has trouble

interpreting environment variables such as SCRIPT_FILENAME or PATH_TRANSLATED

as set by the server.</p>



<p>One example of values that change based on the setting of this directive is

SCRIPT_FILENAME. When using <module>mod_proxy_fcgi</module> historically,

SCRIPT_FILENAME was prefixed with the string "proxy:fcgi://". This variable is

what some generic FastCGI applications would read as their script input, but

PHP-FPM would strip the prefix then remember it was talking to Apache.  In

2.4.21 through 2.4.25, this prefix was automatically stripped by the server,

breaking the ability of PHP-FPM to detect and interoperate with Apache in some

scenarios.</p>

</usage>

</directivesynopsis>



<directivesynopsis>

<name>ProxyFCGISetEnvIf</name>

<description>Allow variables sent to FastCGI servers to be fixed up</description>

<syntax>ProxyFCGISetEnvIf <var>conditional-expression</var>

    [!]<var>environment-variable-name</var>

    [<var>value-expression</var>]</syntax>

<contextlist><context>server config</context>

<context>virtual host</context><context>directory</context>

<context>.htaccess</context></contextlist>

<compatibility>Available in version 2.4.26 and later</compatibility>



<usage>

<p>Just before passing a request to the configured FastCGI server, the core of

the web server sets a number of environment variables based on details of the

current request. FastCGI programs often uses these environment variables

as inputs that determine what underlying scripts they will process, or what

output they directly produce.</p>

<p>Examples of noteworthy environment variables are:</p>

<ul>

  <li>SCRIPT_NAME</li>

  <li>SCRIPT_FILENAME</li>

  <li>REQUEST_URI</li>

  <li>PATH_INFO</li>

  <li>PATH_TRANSLATED</li>

</ul>



<p>This directive allows the environment variables above, or any others of

interest, to be overridden.  This directive is evaluated after the initial

values for these variables are set, so they can be used as input into both

the condition expressions and value expressions.</p>

<p>Parameter syntax:</p>

<dl>

<dt>conditional-expression</dt>

<dd>Specifies an expression that controls whether the environment variable that

   follows will be modified.  For information on the expression syntax, see

   the examples that follow or the full specification at the

   <a href="../expr.html">ap_expr</a> documentation.

   </dd>

<dt>environment-variable-name</dt>

<dd> Specifies the CGI environment variable to change,

   such as PATH_INFO. If preceded by an exclamation point, the variable 

   will be unset.</dd>

<dt>value-expression</dt>

<dd>Specifies the replacement value for the preceding environment variable.

   Backreferences, such as "$1", can be included from regular expression

   captures in <var>conditional-expression</var>. If omitted, the variable is

   set (or overridden) to an empty string &mdash; but see the Note below.</dd>

</dl>



<example>

   <highlight language="config">

# A basic, unconditional override

ProxyFCGISetEnvIf "true" PATH_INFO "/example"



# Use an environment variable in the value

ProxyFCGISetEnvIf "true" PATH_INFO "%{reqenv:SCRIPT_NAME}"



# Use captures in the conditions and backreferences in the replacement

ProxyFCGISetEnvIf "reqenv('PATH_TRANSLATED') =~ m|(/.*prefix)(\d+)(.*)|" PATH_TRANSLATED "$1$3"

   </highlight>

</example>



<note><title>Note: Unset vs. Empty</title>

  The following will unset <code>VARIABLE</code>, preventing it from being sent

  to the FastCGI server:



    <highlight language="config">ProxyFCGISetEnvIf true !VARIABLE</highlight>



  Whereas the following will erase any existing <em>value</em> of

  <code>VARIABLE</code> (by setting it to the empty string), but the empty

  <code>VARIABLE</code> will still be sent to the server:



    <highlight language="config">ProxyFCGISetEnvIf true VARIABLE</highlight>



  The CGI/1.1 specification

  <a href="https://tools.ietf.org/html/rfc3875#section-4.1">does not

  distinguish</a> between a variable with an empty value and a variable that

  does not exist. However, many CGI and FastCGI implementations distinguish (or

  allow scripts to distinguish) between the two. The choice of which to use is

  dependent upon your implementation and your reason for modifying the variable.

</note>



</usage>

</directivesynopsis>



</modulesynopsis>

modules/mappers/mod_actions.c

+5 −1
Original line number Diff line number Diff line
@@ -186,7 +186,8 @@ static int action_handler(request_rec *r) 
        ap_field_noparam(r->pool, r->content_type);



    if (action && (t = apr_table_get(conf->action_types, action))) {

        if (*t++ == '0' && r->finfo.filetype == APR_NOFILE) {

        int virtual = (*t++ == '0' ? 0 : 1);

        if (!virtual && r->finfo.filetype == APR_NOFILE) {

            ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO(00652)

                          "File does not exist: %s", r->filename);

            return HTTP_NOT_FOUND;
@@ -197,6 +198,9 @@ static int action_handler(request_rec *r) 
         * (will be REDIRECT_HANDLER there)

         */

        apr_table_setn(r->subprocess_env, "HANDLER", action);

        if (virtual) {

            apr_table_setn(r->notes, "virtual_script", "1");

        }

    }



    if (script == NULL)

modules/proxy/mod_proxy_fcgi.c

+226 −13
Original line number Diff line number Diff line
@@ -17,13 +17,38 @@ 
#include "mod_proxy.h"

#include "util_fcgi.h"

#include "util_script.h"

#include "ap_expr.h"



module AP_MODULE_DECLARE_DATA proxy_fcgi_module;



typedef struct {

    ap_expr_info_t *cond;

    ap_expr_info_t *subst;

    const char *envname;

} sei_entry;



typedef struct {

    int need_dirwalk;

} fcgi_req_config_t;



/* We will assume FPM, but still differentiate */

typedef enum {

    BACKEND_DEFAULT_UNKNOWN = 0,

    BACKEND_FPM,

    BACKEND_GENERIC,

} fcgi_backend_t;





#define FCGI_MAY_BE_FPM(dconf)                              \

        (dconf &&                                           \

        ((dconf->backend_type == BACKEND_DEFAULT_UNKNOWN) || \

        (dconf->backend_type == BACKEND_FPM)))



typedef struct {

    fcgi_backend_t backend_type;

    apr_array_header_t *env_fixups;

} fcgi_dirconf_t;



/*

 * Canonicalise http-like URLs.

 * scheme is the scheme for the URL
@@ -39,7 +64,7 @@ static int proxy_fcgi_canon(request_rec *r, char *url) 
    fcgi_req_config_t *rconf = NULL;

    const char *pathinfo_type = NULL;



    if (strncasecmp(url, "fcgi:", 5) == 0) {

    if (ap_cstr_casecmpn(url, "fcgi:", 5) == 0) {

        url += 5;

    }

    else {
@@ -133,6 +158,49 @@ static int proxy_fcgi_canon(request_rec *r, char *url) 
    return OK;

}





/*

  ProxyFCGISetEnvIf "reqenv('PATH_INFO') =~ m#/foo(\d+)\.php$#" COVENV1 "$1"

  ProxyFCGISetEnvIf "reqenv('PATH_INFO') =~ m#/foo(\d+)\.php$#" PATH_INFO "/foo.php"

  ProxyFCGISetEnvIf "reqenv('PATH_TRANSLATED') =~ m#(/.*foo)(\d+)(.*)#" PATH_TRANSLATED "$1$3"

*/

static void fix_cgivars(request_rec *r, fcgi_dirconf_t *dconf)

{

    sei_entry *entries;

    const char *err, *src;

    int i = 0, rc = 0;

    ap_regmatch_t regm[AP_MAX_REG_MATCH];



    entries = (sei_entry *) dconf->env_fixups->elts;

    for (i = 0; i < dconf->env_fixups->nelts; i++) {

        sei_entry *entry = &entries[i];



        if (entry->envname[0] == '!') {

            apr_table_unset(r->subprocess_env, entry->envname+1);

        }

        else if (0 < (rc = ap_expr_exec_re(r, entry->cond, AP_MAX_REG_MATCH, regm, &src, &err)))  {

            const char *val = ap_expr_str_exec_re(r, entry->subst, AP_MAX_REG_MATCH, regm, &src, &err);

            if (err) {

                ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO(03514)

                              "Error evaluating expression for replacement of %s: '%s'",

                               entry->envname, err);

                continue;

            }

            if (APLOGrtrace4(r)) {

                const char *oldval = apr_table_get(r->subprocess_env, entry->envname);

                ap_log_rerror(APLOG_MARK, APLOG_TRACE4, 0, r,

                              "fix_cgivars: override %s from '%s' to '%s'",

                              entry->envname, oldval, val);



            }

            apr_table_setn(r->subprocess_env, entry->envname, val);

        }

        else {

            ap_log_rerror(APLOG_MARK, APLOG_TRACE8, 0, r, "fix_cgivars: Condition returned %d", rc);

        }

    }

}



/* Wrapper for apr_socket_sendv that handles updating the worker stats. */

static apr_status_t send_data(proxy_conn_rec *conn,

                              struct iovec *vec,
@@ -253,7 +321,9 @@ static apr_status_t send_environment(proxy_conn_rec *conn, request_rec *r, 
    apr_status_t rv;

    apr_size_t avail_len, len, required_len;

    int next_elem, starting_elem;

    int fpm = 0;

    fcgi_req_config_t *rconf = ap_get_module_config(r->request_config, &proxy_fcgi_module);

    fcgi_dirconf_t *dconf = ap_get_module_config(r->per_dir_config, &proxy_fcgi_module);



    if (rconf) {

       if (rconf->need_dirwalk) {
@@ -268,7 +338,13 @@ static apr_status_t send_environment(proxy_conn_rec *conn, request_rec *r, 
        if (!strncmp(r->filename, "proxy:balancer://", 17)) {

            newfname = apr_pstrdup(r->pool, r->filename+17);

        }

        else if (!strncmp(r->filename, "proxy:fcgi://", 13)) {



        if (!FCGI_MAY_BE_FPM(dconf))  {

            if (!strncmp(r->filename, "proxy:fcgi://", 13)) {

                /* If we strip this under FPM, and any internal redirect occurs

                 * on PATH_INFO, FPM may use PATH_TRANSLATED instead of

                 * SCRIPT_FILENAME (a la mod_fastcgi + Action).

                 */

                newfname = apr_pstrdup(r->pool, r->filename+13);

            }

            /* Query string in environment only */
@@ -278,6 +354,9 @@ static apr_status_t send_environment(proxy_conn_rec *conn, request_rec *r, 
                    *qs = '\0';

                }

            }

        } else {

            fpm = 1;

        }



        if (newfname) {

            newfname = ap_strchr(newfname, '/');
@@ -285,11 +364,43 @@ static apr_status_t send_environment(proxy_conn_rec *conn, request_rec *r, 
        }

    }



#if 0

    ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(09999)

                  "r->filename: %s", (r->filename ? r->filename : "nil"));

    ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(09999)

                  "r->uri: %s", (r->uri ? r->uri : "nil"));

    ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(09999)

                  "r->path_info: %s", (r->path_info ? r->path_info : "nil"));

#endif



    ap_add_common_vars(r);

    ap_add_cgi_vars(r);



    if (fpm || apr_table_get(r->notes, "virtual_script")) {

        /*

         * Adjust SCRIPT_NAME, PATH_INFO and PATH_TRANSLATED for PHP-FPM

         * TODO: Right now, PATH_INFO and PATH_TRANSLATED look OK...

         */

        const char *pend;

        const char *script_name = apr_table_get(r->subprocess_env, "SCRIPT_NAME");

        pend = script_name + strlen(script_name);

        if (r->path_info && *r->path_info) {

            pend = script_name + ap_find_path_info(script_name, r->path_info) - 1;

        }

        while (pend != script_name && *pend != '/') {

            pend--;

        }

        apr_table_setn(r->subprocess_env, "SCRIPT_NAME", pend);

        ap_log_rerror(APLOG_MARK, APLOG_TRACE4, 0, r,

                      "fpm:virtual_script: Modified SCRIPT_NAME to: %s",

                      pend);

    }



    /* XXX are there any FastCGI specific env vars we need to send? */



    /* Give admins final option to fine-tune env vars */

    fix_cgivars(r, dconf);



    /* XXX mod_cgi/mod_cgid use ap_create_environment here, which fills in

     *     the TZ value specially.  We could use that, but it would mean

     *     parsing the key/value pairs back OUT of the allocated env array,
@@ -915,7 +1026,7 @@ static int proxy_fcgi_handler(request_rec *r, proxy_worker *worker, 
                  "url: %s proxyname: %s proxyport: %d",

                  url, proxyname, proxyport);



    if (strncasecmp(url, "fcgi:", 5) != 0) {

    if (ap_cstr_casecmpn(url, "fcgi:", 5) != 0) {

        ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(01077) "declining URL %s", url);

        return DECLINED;

    }
@@ -976,18 +1087,120 @@ cleanup: 
    return status;

}



static void *fcgi_create_dconf(apr_pool_t *p, char *path)

{

    fcgi_dirconf_t *a;



    a = (fcgi_dirconf_t *)apr_pcalloc(p, sizeof(fcgi_dirconf_t));

    a->backend_type = BACKEND_DEFAULT_UNKNOWN;

    a->env_fixups = apr_array_make(p, 20, sizeof(sei_entry));



    return a;

}



static void *fcgi_merge_dconf(apr_pool_t *p, void *basev, void *overridesv)

{

    fcgi_dirconf_t *a, *base, *over;



    a     = (fcgi_dirconf_t *)apr_pcalloc(p, sizeof(fcgi_dirconf_t));

    base  = (fcgi_dirconf_t *)basev;

    over  = (fcgi_dirconf_t *)overridesv;



    a->backend_type = (over->backend_type != BACKEND_DEFAULT_UNKNOWN)

                      ? over->backend_type

                      : base->backend_type;

    a->env_fixups = apr_array_append(p, base->env_fixups, over->env_fixups);

    return a;

}



static const char *cmd_servertype(cmd_parms *cmd, void *in_dconf,

                                   const char *val)

{

    fcgi_dirconf_t *dconf = in_dconf;



    if (!strcasecmp(val, "GENERIC")) {

       dconf->backend_type = BACKEND_GENERIC;

    }

    else if (!strcasecmp(val, "FPM")) {

       dconf->backend_type = BACKEND_FPM;

    }

    else {

        return "ProxyFCGIBackendType requires one of the following arguments: "

               "'GENERIC', 'FPM'";

    }



    return NULL;

}





static const char *cmd_setenv(cmd_parms *cmd, void *in_dconf,

                              const char *arg1, const char *arg2,

                              const char *arg3)

{

    fcgi_dirconf_t *dconf = in_dconf;

    const char *err;

    sei_entry *new;

    const char *envvar = arg2;



    new = apr_array_push(dconf->env_fixups);

    new->cond = ap_expr_parse_cmd(cmd, arg1, 0, &err, NULL);

    if (err) {

        return apr_psprintf(cmd->pool, "Could not parse expression \"%s\": %s",

                            arg1, err);

    }



    if (envvar[0] == '!') {

        /* Unset mode. */

        if (arg3) {

            return apr_psprintf(cmd->pool, "Third argument (\"%s\") is not "

                                "allowed when using ProxyFCGISetEnvIf's unset "

                                "mode (%s)", arg3, envvar);

        }

        else if (!envvar[1]) {

            /* i.e. someone tried to give us a name of just "!" */

            return "ProxyFCGISetEnvIf: \"!\" is not a valid variable name";

        }



        new->subst = NULL;

    }

    else {

        /* Set mode. */

        if (!arg3) {

            /* A missing expr-value should be treated as empty. */

            arg3 = "";

        }



        new->subst = ap_expr_parse_cmd(cmd, arg3, AP_EXPR_FLAG_STRING_RESULT, &err, NULL);

        if (err) {

            return apr_psprintf(cmd->pool, "Could not parse expression \"%s\": %s",

                                arg3, err);

        }

    }



    new->envname = envvar;



    return NULL;

}

static void register_hooks(apr_pool_t *p)

{

    proxy_hook_scheme_handler(proxy_fcgi_handler, NULL, NULL, APR_HOOK_FIRST);

    proxy_hook_canon_handler(proxy_fcgi_canon, NULL, NULL, APR_HOOK_FIRST);

}



static const command_rec command_table[] = {

    AP_INIT_TAKE1("ProxyFCGIBackendType", cmd_servertype, NULL, OR_FILEINFO,

                  "Specify the type of FastCGI server: 'Generic', 'FPM'"),

    AP_INIT_TAKE23("ProxyFCGISetEnvIf", cmd_setenv, NULL, OR_FILEINFO,

                  "expr-condition env-name expr-value"),

    { NULL }

};



AP_DECLARE_MODULE(proxy_fcgi) = {

    STANDARD20_MODULE_STUFF,

    NULL,                       /* create per-directory config structure */

    NULL,                       /* merge per-directory config structures */

    fcgi_create_dconf,          /* create per-directory config structure */

    fcgi_merge_dconf,           /* merge per-directory config structures */

    NULL,                       /* create per-server config structure */

    NULL,                       /* merge per-server config structures */

    NULL,                       /* command apr_table_t */

    command_table,              /* command apr_table_t */

    register_hooks              /* register hooks */

};