mod_ratelimit: Don't interfere with "chunked" encoding.

By the time ap_http_header_filter() sends the header brigade and adds the
"CHUNK" filter, we need to garantee that the header went through all the
filters' stack, and more specifically above ap_http_chunk_filter() which
assumes that all it receives is content data.
Since rate_limit_filter() may retain the header brigade, make it run after
ap_http_chunk_filter(), just before AP_FTYPE_CONNECTION filters.

Also, ap_http_header_filter() shouldn't eat the EOS for HEAD/no-body responses.
For instance mod_ratelimit depends on it since r1835168, but any next request
filter may as well to flush and/or bail out approprietely.

This fixes the regression introduced in 2.4.34 (r1835168).
PR 62568.

Submitted by: ylavic
Reviewed by: covener, ylavic, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1839497 13f79535-47bb-0310-9956-ffa450edef68

Merge r1418761, r1418765, r1510295, r1757147, r1805163, r1818924, r1827374, r1831772, r1832351, r1832951, r1815004 from trunk:

Don't claim "BIO dump follows" if it is not logged due to log level config.


make ssl_io_data_dump respect per-conn loglevel


add high trace level log messages for debugging buffering and write completion


* modules/ssl/ssl_engine_kernel.c (ssl_callback_SessionTicket): Fail
  if RAND_bytes() fails; possible per API, although not in practice
  with the OpenSSL implementation.


Fix typo in log message.


ap_add_common_vars(): use apr_pstrmemdup().

This avoids a transient replacement/restore of '?' by '\0' in r->filename.


Use 'ap_request_has_body()' instead of duplicating its implemenation.

The logic in 'ap_request_has_body()' is:
    has_body = (!r->header_only
                && (r->kept_body
                    || apr_table_get(r->headers_in, "Transfer-Encoding")
                    || ( (cls = apr_table_get(r->headers_in, "Content-Length"))
                        && (apr_strtoff(&cl, cls, &estr, 10) == APR_SUCCESS)
                        && (!*estr)
                        && (cl > 0) )
                    )
                );
So the test is slighly different from the original code. (but this looks fine to me)

This also has the advantage to avoid a redundant call to 'apr_table_get()' and to improve readability.

While at it, move the test '!r->expecting_100' a few lines above because it is cheap.

PR62368: Print the unparsed URI in AH03454

... to include r->args and get otherwise get as close to possible to
what came in over the wire.

Submitted By: Hank Ibell <hwibell gmail.com>
Committed By: covener




All error handling paths of this function call 'apr_brigade_destroy()' , except this one.
So add it here too.

Probably spotted with the help of the Coccinelle software (Thx Julia for the patch and for Coccinelle)

See PR 53016

* modules/proxy/proxy_util.c (ap_proxy_share_worker): Skip creating subpool
  for debugging unless debug-level logging is enabled.  No functional change.


mod_watchdog: Correct some log messages and fix
compiler warning
"'rv' may be used uninitialized in this function".

Follow up to r1722154.

Submitted by: sf, jorton, jorton, ylavic, jailletc36, covener, jailletc36, jorton, rjung
Reviewed by: jailletc36, jim, jorton


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1838103 13f79535-47bb-0310-9956-ffa450edef68

allow quoted paths in <IfFile>

The boilerplate code for config sections conflicts with TAKE1
because of the trailing stuff to terminate the opening tag.

Change from TAKE1 to RAW_ARGS and call ap_getword_conf()
directly.





git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1838053 13f79535-47bb-0310-9956-ffa450edef68

backport of r1837357 from trunk.
  *) mod_md: When the last domain name from an MD is moved to another one,
     that now empty MD gets moved to the store archive. PR 62572.



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1837358 13f79535-47bb-0310-9956-ffa450edef68

Fix config merging of SSLOCSPEnable and SSLOCSPOverrideResponder.

Submitted by: trawick, Frank Meier <frank.meier ergon.ch>
Reviewed by: rpluem, ylavic, jorton


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1836472 13f79535-47bb-0310-9956-ffa450edef68

  Fix APR 1.4 compatibility by avoiding use of apr_escape.h.

Submitted by: jorton
Reviewed by: jorton, rpluem, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1836471 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1836159 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835553 13f79535-47bb-0310-9956-ffa450edef68

Review by popcorner with further edits for clarity by both CodeingBoy
and popcorner, their discussion at https://github.com/popcorner/chect/pull/1

Added CONTENT_LANGUAGE variable for both translations.

Note user-visible update in CHANGES



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835289 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835189 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835180 13f79535-47bb-0310-9956-ffa450edef68

In 'ap_proxy_cookie_reverse_map', iterate over each token of the 'Set-Cookie' header field in order to avoid updating the wrong one.

This could happen if the header field has something like 'fakepath=foo;path=bar". In this case fakepath would be updated instead of path.

We don't need regex anymore in order to parse the field values and 'ap_proxy_strmatch_domain' and 'ap_proxy_strmatch_path' are now useless. (and should be axed IMHO)

PR 61560


Submitted by: jailletc36
Reviewed by: jailletc36, rpluem, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835171 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835169 13f79535-47bb-0310-9956-ffa450edef68

mod_ratelimit: fix behavior with proxied content

mod_ratelimit works by splitting data in "chunks"
to send to the client, sleeping a predefined amount
of time between them (200ms). So for example,
a rate-limit 40 value would correspond to a chunk size
of 8192 bytes, flushed to the client every 200ms.

The idea works fine when httpd directly serves the
content, since the filter will be called once with
a single bucket brigade. In the context of a proxied
content though the filter is likely to be called multiple
times, with a bucket brigade size that corresponds to
the maximum allowed buffer size. If this value is lower
or higher than the chunk size, the filter will not
properly rate limit the data going to the client.

This patch solves the problem with two fix:
1) do_sleep is now stored in the ctx context struct,
   so if the filter is invoked multiple times it
   will still sleep when needed. For example, say
   that the chunk_size is 8192 and the bucket brigate
   len is 10240: the filter will flush 8192 bytes
   on the first invocation, sleep 200ms, flush the
   remaining bytes and then finish. The next invocation
   will do the same, clearly not leading to the
   correct "sleeping pattern".
2) The example above highlights also another issue:
   mod_ratelimit should  flush only chunk_size bytes
   at the time (I am now excluding the burst calculation
   from the picture), and buffer between invocations
   unless the brigade contains EOS.

The change has been tested with various scenarios and
it looks working as expected, but of course more
feedback/testing is welcome.

The original patch was written by me and then Yann
refactored the code to be more precise and efficient,
basically transforming an axe in a wonderful Japanese
katana sword, so credits to him for this work.

PR: 62362


Submitted by: elukey
Reviewed by: elukey, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835168 13f79535-47bb-0310-9956-ffa450edef68

Re-allow '_' (underscore) in hostnames.

'_' was not permitted in hostnames since 2.4.25's "HTTP Strict"
changes.




git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834895 13f79535-47bb-0310-9956-ffa450edef68

  *) mod_slotmem_shm: Add generation number to shm filename to fix races with
                      graceful restarts.  PRs 62044 and 62308.
     trunk patch: https://svn.apache.org/r1831868
                  https://svn.apache.org/r1831869
                  https://svn.apache.org/r1831870
                  https://svn.apache.org/r1831871
                  https://svn.apache.org/r1831872
                  https://svn.apache.org/r1831935
                  https://svn.apache.org/r1831938
                  https://svn.apache.org/r1832479
     2.4.x patch: http://home.apache.org/~ylavic/patches/httpd-2.4.x-balancer_slotmem-v2.patch
     +1: ylavic, jim, druggeri

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834887 13f79535-47bb-0310-9956-ffa450edef68

Russian translations of errordocs.
Contributed by Alexander Gaganashvili.


Error page translations submitted for zh-cn and zh-tw, contributed
by CodeingBoy <codeingboy gmail.com>.

Note that zh-tw is not a native translation; each translation aught
be reviewed by a native reader before backporting to 2.4.x branch.


Submitted by: niq, wrowe
Reviewed by: jailletc36, covener, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834843 13f79535-47bb-0310-9956-ffa450edef68

backport of current mod_md version and documentation.



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834671 13f79535-47bb-0310-9956-ffa450edef68

Add default schema ports for websockets

Nore userland/PR change

PR: 62480
Submitted by: Lubos Uhliarik <luhliari redhat.com>
Reviewed by: jim, rpluem, covener


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834422 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834323 13f79535-47bb-0310-9956-ffa450edef68

header value RFC compliant like the case of an
Expires time in the past: allow to overwrite the
non-caching decision using CacheStoreExpired and
respect Cache-Control "max-age" and "s-maxage".

Backport of r1833876 from trunk.

Submitted by: rjung
Reviewed by: rjung, ylavic, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834241 13f79535-47bb-0310-9956-ffa450edef68

mod_xml2enc: Fix forwarding of error metadata/responses. PR 62180.

All meta buckets are now aggregated (besides FLUSH) and forwarded down
the chain, and the output filter bails out on EOS.

Proposed by: Micha Lenk <micha lenk.info>


mod_xml2enc: follow up to r1829038.

If any, pass pending meta bucket down the chain before leaving.


mod_xml2enc: follow up to r1829038 and r1829039.

Use below pending_data logic for EOS bucket.
This closes #48


Submitted by: ylavic
Reviewed by: ylavic, jim, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834104 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834103 13f79535-47bb-0310-9956-ffa450edef68

core: ap_getline_core() reads nothing for n == 0.

PR62199: add worker parameter ResponseFieldSize to mod_proxy

Submitted By: Hank Ibell
Committed By: covener


add log id for r1828926


core: Add and handle AP_GETLINE_NOSPC_EOL flag in ap_rgetline_core().

This tells the ap_getline() family of functions to consume the end of line
when the buffer is exhausted.

PR 62198.


mod_proxy_http: make use of AP_GETLINE_NOSPC_EOL in ap_proxygetline().

Fixes response header thrown away after the previous one was considered too
large and truncated.

PR 62196.


core: forward flags to recursive/folding call to ap_rgetline_core().

We still need them when folding, other than AP_GETLINE_FOLD itself of course.


mod_proxy_http: follow up to r1829573: remain EBCDIC friendly.

Keep using ap_rgetline() as before r1829573, since ap_rgetline_core() is
EBCDIC agnostic.


Submitted by: ylavic, covener, covener, ylavic, ylavic, ylavic, ylavic
Reviewed by: ylavic, covener, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834093 13f79535-47bb-0310-9956-ffa450edef68

Extend SSLOCSPEnable with mode 'leaf' that only checks the leaf of a
certificate chain. PR62112 [Ricardo Martin Camarero <rickyepoderi yahoo.es>] 

Fixed OCSPEnable to keep accepting "off", not "none".

Submitted by: icing
Reviewedby: icing, ylavic, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834089 13f79535-47bb-0310-9956-ffa450edef68

* modules/http/http_request.c (ap_process_request_after_handler,
  ap_process_request): Cache and retrieve the brigade structure used
  to send EOR and FLUSH between requests in c->pool userdata, to avoid
  allocating a brigade structure per-request out of c->pool.

Submitted by: rpluem, jorton

Submitted by: jorton
Reviewed by: jorton, covener, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833841 13f79535-47bb-0310-9956-ffa450edef68

mod_proxy_html: Fix variable interpolation and memory allocation failure in ProxyHTMLURLMap.

Proposed by: Ewald Dieterich <ewald mailbox.org>
Reviewed by: ylavic


Submitted by: ylavic
Reviewed by: ylavic, jailletc36, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833840 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833471 13f79535-47bb-0310-9956-ffa450edef68

mod_remoteip: Fix RemoteIP{Trusted,Internal}ProxyList loading broken by 2.4.30.

Overwriting server config in pre_config hook breaks EXEC_ON_READ directives,
it's automatically created on purpose anyway.

PR 62220.


Follow up to r1832580: correct CHANGES entry.
Submitted by: ylavic
Reviewed by: ylavic, wrowe, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833070 13f79535-47bb-0310-9956-ffa450edef68

mod_remoteip: Set useragent port to zero PR59931

When overriding the useragent address from X-Forwarded-For,
zero out what had been initialized as the connection-level port.  

Submitted By: Hank Ibell <hwibell gmail.com>
Reviewed by: ylavic, covener, icing


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833007 13f79535-47bb-0310-9956-ffa450edef68

mpm_unix(es): cleanup properly on exit in one_process mode.

We can't destroy ap_pglobal in the MPMs because clean_child_exit() runs in
a DSO which would be unloaded under us.

So we defer an ap_terminate() with atexit() in ap_unixd_mpm_set_signals(),
all this is static/builtin code in "os/unix/unixd.c".


Follow up to r1822537: replace static variable with pool userdata.

Also adds a comment and a CHANGES entry.


Submitted by: ylavic
Reviewed by: ylavic, jim, icing


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833005 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832644 13f79535-47bb-0310-9956-ffa450edef68

add an <IfFile> config section like <IfDefine>

It allows a non httpd config file to be used as a marker directly in
httpd.conf without hiding logic in a script in front of apachectl
to do test -f and pass extra -D's.

This is something we've had in IBM's httpd distro for a little bit and
hadn't remembered to share. I've seen some questions/config files come
up in a few places lately that would benefit from this as an option.

Remove duplicate implementations of conditional section function.  No
functional change.

* server/core.c (start_cond_section): Factor out from start_if*.
  Adjust to use apr_strmemdup.
  (test_ifmod_section, test_iffile_section, test_ifdefine_section):
  Move container-specific tests into callbacks from start_if*.
  (core_cmds): Adjust <IfFile, <IfDefine, <IfModule to use
  start_cond_section with callbacks.

Add <IfDirective> and <IfSection>:

* server/core.c
  (test_ifdirective_section, test_ifsection_section): New callbacks.
  (core_cmds): Define new directives.

* include/http_config.h, server/config.c (ap_exists_directive):
  New function.

* include/ap_mmn.h: Bump MMN minor for above.

* docs/manual/mod/core.xml: Add docs.

* server/core.c (start_cond_section): Comment & variable name fixes,
  no functional change.

Submitted by: covener, jorton
Reviewed by: jorton, ylavic, covener


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832623 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832612 13f79535-47bb-0310-9956-ffa450edef68

mod_proxy_balancer: Add hot spare member type and corresponding flag (R). Hot spare members are
used as drop-in replacements for unusable workers in the same load balancer set. This differs
from hot standbys which are only used when all workers in a set are unusable. PR 61140.


mod_proxy_balancer: follow up to r1828890: indentation and 80 col.


Submitted by: jhriggs, ylavic
Reviewed by: jhriggs, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832609 13f79535-47bb-0310-9956-ffa450edef68

suexec: Add support for logging to syslog as an alternative to a
logfile.

* support/suexec.c (err_output) [AP_LOG_SYSLOG]: Log to syslog.
  (main): Close syslog fd if open, before execv.  Add -V output
  for AP_LOG_SYSLOG.

* configure.in: Add --with-suexec-syslog argument; allow
  --without-suexec-logfile to omit definition of AP_LOG_EXEC.


suexec: Support use of setgid/setuid capability bits on Linux, a
weaker set of privileges than the full setuid/setgid root binary.

* configure.in: Add --enable-suexec-capabilites flag.

* Makefile.in: If configured, use setcap instead of chmod 7555 on
  installed suexec binary.

* modules/arch/unix/mod_unixd.c (unixd_pre_config): Drop test for
  setuid bit if capability bits are used.

* docs/manual/: Add docs.


* docs/manual/suexec.html.en: Update for syslog logging.


* configure.in: Correct handling of --with-suexec-logfile in r1341905,
  thanks to rpluem.


* support/suexec.c: Define AP_LOG_FACILITY.
  (err_output): Use AP_LOG_FACILITY for syslog facility.

Suggested by: kbrand


* configure.in: Fix help text for --with-suexec-syslog.

Submitted by: jorton
Reviewed by: jorton, ylavic, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832565 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832499 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832498 13f79535-47bb-0310-9956-ffa450edef68

* modules/ssl/ssl_engine_init.c (ssl_init_Module): Don't enable SSL
  for a vhost if SSLEnable is not used and no certs are configured,
  even if the Listen protocol is "https".  Restores behaviour to that
  prior to r1809303 for configs which would now otherwise fail at
  startup.

Submitted by: jorton
Reviewed by: jorton, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832495 13f79535-47bb-0310-9956-ffa450edef68

if the ErrorLog directive is configured with the 'syslog' prefix. PR 62102

trunk patch: none, as far as I can see the code in trunk diverged too much
          due to the code in STALLED for ap_errorlog_provider.
 
Submitted by: elukey, jhriggs, jailletc36
Reviewed by: elukey, jhriggs, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832493 13f79535-47bb-0310-9956-ffa450edef68