mod_ratelimit: fix behavior with proxied content

mod_ratelimit works by splitting data in "chunks"
to send to the client, sleeping a predefined amount
of time between them (200ms). So for example,
a rate-limit 40 value would correspond to a chunk size
of 8192 bytes, flushed to the client every 200ms.

The idea works fine when httpd directly serves the
content, since the filter will be called once with
a single bucket brigade. In the context of a proxied
content though the filter is likely to be called multiple
times, with a bucket brigade size that corresponds to
the maximum allowed buffer size. If this value is lower
or higher than the chunk size, the filter will not
properly rate limit the data going to the client.

This patch solves the problem with two fix:
1) do_sleep is now stored in the ctx context struct,
   so if the filter is invoked multiple times it
   will still sleep when needed. For example, say
   that the chunk_size is 8192 and the bucket brigate
   len is 10240: the filter will flush 8192 bytes
   on the first invocation, sleep 200ms, flush the
   remaining bytes and then finish. The next invocation
   will do the same, clearly not leading to the
   correct "sleeping pattern".
2) The example above highlights also another issue:
   mod_ratelimit should  flush only chunk_size bytes
   at the time (I am now excluding the burst calculation
   from the picture), and buffer between invocations
   unless the brigade contains EOS.

The change has been tested with various scenarios and
it looks working as expected, but of course more
feedback/testing is welcome.

The original patch was written by me and then Yann
refactored the code to be more precise and efficient,
basically transforming an axe in a wonderful Japanese
katana sword, so credits to him for this work.

PR: 62362


Submitted by: elukey
Reviewed by: elukey, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1835168 13f79535-47bb-0310-9956-ffa450edef68

Re-allow '_' (underscore) in hostnames.

'_' was not permitted in hostnames since 2.4.25's "HTTP Strict"
changes.




git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834895 13f79535-47bb-0310-9956-ffa450edef68

  *) mod_slotmem_shm: Add generation number to shm filename to fix races with
                      graceful restarts.  PRs 62044 and 62308.
     trunk patch: https://svn.apache.org/r1831868
                  https://svn.apache.org/r1831869
                  https://svn.apache.org/r1831870
                  https://svn.apache.org/r1831871
                  https://svn.apache.org/r1831872
                  https://svn.apache.org/r1831935
                  https://svn.apache.org/r1831938
                  https://svn.apache.org/r1832479
     2.4.x patch: http://home.apache.org/~ylavic/patches/httpd-2.4.x-balancer_slotmem-v2.patch
     +1: ylavic, jim, druggeri

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834887 13f79535-47bb-0310-9956-ffa450edef68

Russian translations of errordocs.
Contributed by Alexander Gaganashvili.


Error page translations submitted for zh-cn and zh-tw, contributed
by CodeingBoy <codeingboy gmail.com>.

Note that zh-tw is not a native translation; each translation aught
be reviewed by a native reader before backporting to 2.4.x branch.


Submitted by: niq, wrowe
Reviewed by: jailletc36, covener, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834843 13f79535-47bb-0310-9956-ffa450edef68

backport of current mod_md version and documentation.



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834671 13f79535-47bb-0310-9956-ffa450edef68

Add default schema ports for websockets

Nore userland/PR change

PR: 62480
Submitted by: Lubos Uhliarik <luhliari redhat.com>
Reviewed by: jim, rpluem, covener


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834422 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834323 13f79535-47bb-0310-9956-ffa450edef68

header value RFC compliant like the case of an
Expires time in the past: allow to overwrite the
non-caching decision using CacheStoreExpired and
respect Cache-Control "max-age" and "s-maxage".

Backport of r1833876 from trunk.

Submitted by: rjung
Reviewed by: rjung, ylavic, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834241 13f79535-47bb-0310-9956-ffa450edef68

mod_xml2enc: Fix forwarding of error metadata/responses. PR 62180.

All meta buckets are now aggregated (besides FLUSH) and forwarded down
the chain, and the output filter bails out on EOS.

Proposed by: Micha Lenk <micha lenk.info>


mod_xml2enc: follow up to r1829038.

If any, pass pending meta bucket down the chain before leaving.


mod_xml2enc: follow up to r1829038 and r1829039.

Use below pending_data logic for EOS bucket.
This closes #48


Submitted by: ylavic
Reviewed by: ylavic, jim, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834104 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834103 13f79535-47bb-0310-9956-ffa450edef68

core: ap_getline_core() reads nothing for n == 0.

PR62199: add worker parameter ResponseFieldSize to mod_proxy

Submitted By: Hank Ibell
Committed By: covener


add log id for r1828926


core: Add and handle AP_GETLINE_NOSPC_EOL flag in ap_rgetline_core().

This tells the ap_getline() family of functions to consume the end of line
when the buffer is exhausted.

PR 62198.


mod_proxy_http: make use of AP_GETLINE_NOSPC_EOL in ap_proxygetline().

Fixes response header thrown away after the previous one was considered too
large and truncated.

PR 62196.


core: forward flags to recursive/folding call to ap_rgetline_core().

We still need them when folding, other than AP_GETLINE_FOLD itself of course.


mod_proxy_http: follow up to r1829573: remain EBCDIC friendly.

Keep using ap_rgetline() as before r1829573, since ap_rgetline_core() is
EBCDIC agnostic.


Submitted by: ylavic, covener, covener, ylavic, ylavic, ylavic, ylavic
Reviewed by: ylavic, covener, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834093 13f79535-47bb-0310-9956-ffa450edef68

Extend SSLOCSPEnable with mode 'leaf' that only checks the leaf of a
certificate chain. PR62112 [Ricardo Martin Camarero <rickyepoderi yahoo.es>] 

Fixed OCSPEnable to keep accepting "off", not "none".

Submitted by: icing
Reviewedby: icing, ylavic, rpluem


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1834089 13f79535-47bb-0310-9956-ffa450edef68

* modules/http/http_request.c (ap_process_request_after_handler,
  ap_process_request): Cache and retrieve the brigade structure used
  to send EOR and FLUSH between requests in c->pool userdata, to avoid
  allocating a brigade structure per-request out of c->pool.

Submitted by: rpluem, jorton

Submitted by: jorton
Reviewed by: jorton, covener, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833841 13f79535-47bb-0310-9956-ffa450edef68

mod_proxy_html: Fix variable interpolation and memory allocation failure in ProxyHTMLURLMap.

Proposed by: Ewald Dieterich <ewald mailbox.org>
Reviewed by: ylavic


Submitted by: ylavic
Reviewed by: ylavic, jailletc36, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833840 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833471 13f79535-47bb-0310-9956-ffa450edef68

mod_remoteip: Fix RemoteIP{Trusted,Internal}ProxyList loading broken by 2.4.30.

Overwriting server config in pre_config hook breaks EXEC_ON_READ directives,
it's automatically created on purpose anyway.

PR 62220.


Follow up to r1832580: correct CHANGES entry.
Submitted by: ylavic
Reviewed by: ylavic, wrowe, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833070 13f79535-47bb-0310-9956-ffa450edef68

mod_remoteip: Set useragent port to zero PR59931

When overriding the useragent address from X-Forwarded-For,
zero out what had been initialized as the connection-level port.  

Submitted By: Hank Ibell <hwibell gmail.com>
Reviewed by: ylavic, covener, icing


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833007 13f79535-47bb-0310-9956-ffa450edef68

mpm_unix(es): cleanup properly on exit in one_process mode.

We can't destroy ap_pglobal in the MPMs because clean_child_exit() runs in
a DSO which would be unloaded under us.

So we defer an ap_terminate() with atexit() in ap_unixd_mpm_set_signals(),
all this is static/builtin code in "os/unix/unixd.c".


Follow up to r1822537: replace static variable with pool userdata.

Also adds a comment and a CHANGES entry.


Submitted by: ylavic
Reviewed by: ylavic, jim, icing


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1833005 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832644 13f79535-47bb-0310-9956-ffa450edef68

add an <IfFile> config section like <IfDefine>

It allows a non httpd config file to be used as a marker directly in
httpd.conf without hiding logic in a script in front of apachectl
to do test -f and pass extra -D's.

This is something we've had in IBM's httpd distro for a little bit and
hadn't remembered to share. I've seen some questions/config files come
up in a few places lately that would benefit from this as an option.

Remove duplicate implementations of conditional section function.  No
functional change.

* server/core.c (start_cond_section): Factor out from start_if*.
  Adjust to use apr_strmemdup.
  (test_ifmod_section, test_iffile_section, test_ifdefine_section):
  Move container-specific tests into callbacks from start_if*.
  (core_cmds): Adjust <IfFile, <IfDefine, <IfModule to use
  start_cond_section with callbacks.

Add <IfDirective> and <IfSection>:

* server/core.c
  (test_ifdirective_section, test_ifsection_section): New callbacks.
  (core_cmds): Define new directives.

* include/http_config.h, server/config.c (ap_exists_directive):
  New function.

* include/ap_mmn.h: Bump MMN minor for above.

* docs/manual/mod/core.xml: Add docs.

* server/core.c (start_cond_section): Comment & variable name fixes,
  no functional change.

Submitted by: covener, jorton
Reviewed by: jorton, ylavic, covener


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832623 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832612 13f79535-47bb-0310-9956-ffa450edef68

mod_proxy_balancer: Add hot spare member type and corresponding flag (R). Hot spare members are
used as drop-in replacements for unusable workers in the same load balancer set. This differs
from hot standbys which are only used when all workers in a set are unusable. PR 61140.


mod_proxy_balancer: follow up to r1828890: indentation and 80 col.


Submitted by: jhriggs, ylavic
Reviewed by: jhriggs, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832609 13f79535-47bb-0310-9956-ffa450edef68

suexec: Add support for logging to syslog as an alternative to a
logfile.

* support/suexec.c (err_output) [AP_LOG_SYSLOG]: Log to syslog.
  (main): Close syslog fd if open, before execv.  Add -V output
  for AP_LOG_SYSLOG.

* configure.in: Add --with-suexec-syslog argument; allow
  --without-suexec-logfile to omit definition of AP_LOG_EXEC.


suexec: Support use of setgid/setuid capability bits on Linux, a
weaker set of privileges than the full setuid/setgid root binary.

* configure.in: Add --enable-suexec-capabilites flag.

* Makefile.in: If configured, use setcap instead of chmod 7555 on
  installed suexec binary.

* modules/arch/unix/mod_unixd.c (unixd_pre_config): Drop test for
  setuid bit if capability bits are used.

* docs/manual/: Add docs.


* docs/manual/suexec.html.en: Update for syslog logging.


* configure.in: Correct handling of --with-suexec-logfile in r1341905,
  thanks to rpluem.


* support/suexec.c: Define AP_LOG_FACILITY.
  (err_output): Use AP_LOG_FACILITY for syslog facility.

Suggested by: kbrand


* configure.in: Fix help text for --with-suexec-syslog.

Submitted by: jorton
Reviewed by: jorton, ylavic, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832565 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832499 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832498 13f79535-47bb-0310-9956-ffa450edef68

* modules/ssl/ssl_engine_init.c (ssl_init_Module): Don't enable SSL
  for a vhost if SSLEnable is not used and no certs are configured,
  even if the Listen protocol is "https".  Restores behaviour to that
  prior to r1809303 for configs which would now otherwise fail at
  startup.

Submitted by: jorton
Reviewed by: jorton, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832495 13f79535-47bb-0310-9956-ffa450edef68

if the ErrorLog directive is configured with the 'syslog' prefix. PR 62102

trunk patch: none, as far as I can see the code in trunk diverged too much
          due to the code in STALLED for ap_errorlog_provider.
 
Submitted by: elukey, jhriggs, jailletc36
Reviewed by: elukey, jhriggs, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832493 13f79535-47bb-0310-9956-ffa450edef68

Instrument 'bbout'


mod_http2: use proper ARP defined for formatting apr_off_t


On the trunk:

mod_http2: on level trace2, log any unsuccessful HTTP/2 direct connection upgrade
     with base64 encoding to unify its appearance in possible bug reports.


On the trunk:

* mod_http2: calculate unencrypted connection sniffing base64 only when log level is at required height. [Ruediger Pluem]


On the trunk:

mod_http2: accurate reporting of h2 data input/output per request via mod_logio. Fixes
     an issue where output sizes where counted n-times on reused slave connections. See
     gituhub issue: https://github.com/icing/mod_h2/issues/158


Submitted by: jailletc36, icing, icing, icing, icing
Reviewed by: icing, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832489 13f79535-47bb-0310-9956-ffa450edef68

On the trunk:

mod_http2: always wake up any conditional waits when streams are aborted. Fixes race conditions where timeouts would trigger instead.

Submitted by: icing
Reviewed by: icing, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832487 13f79535-47bb-0310-9956-ffa450edef68

* When mod_http2 is loaded more then ThreadsPerChild backend connections can
  be useful as mod_http2 has an additional thread pool on top of
  ThreadsPerChild.
  But leave the default with ThreadsPerChild.


* Add some some comment why we do not limit hmax any longer


mod_proxy: follow up to r1822849.

Get the help(er) of mod_http2 to determine how much connections should
be handled in the reslist by default (i.e. max_threads).


mod_proxy: follow up to r1822849 and r1822878.

Does r1822878's "static" APR_RETRIEVE_OPTIONAL_FN work if, say, mod_proxy is
builtin but mod_http2 isn't?

Not worth taking the risk here since it's not a fast path...

Note: if this is an issue, I'm afraid it applies elsewhere too.


mod_proxy: follow up to r1822849 and r1822879.

Fix my maths, thanks Stefan and Rüdiger!


needs mod_http2.h


* Add missing CHANGES entry for revisions 1822849,1822858,1822878,1822879,1822883,1828485


Submitted by: rpluem, ylavic, ylavic, ylavic, gsmith, rpluem
Reviewed by: rpluem, jim, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1832485 13f79535-47bb-0310-9956-ffa450edef68

PR62186: preserve %<m for ErrorDocument internal redirects

  *) core: Preserve the original HTTP request method in the '%<m' LogFormat
     when an path-based ErrorDocument is used.  PR 62186.
     [Micha Lenk <micha lenk.info>]

Submitted By: Micha Lenk
Committed By: covener

Submitted by: covener
Reviewed by: covener, jhriggs, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1830248 13f79535-47bb-0310-9956-ffa450edef68

     trunk patch: http://svn.apache.org/r1827196
     2.4.x patch: svn merge -c 1827196 ^/httpd/httpd/trunk .
     +1: icing, ylavic, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1829486 13f79535-47bb-0310-9956-ffa450edef68

mod_dumpio: do nothing below log level TRACE7.

For instance, depending on EnableMMAP/Sendfile configuration, don't split
file brigades to 8K heap buckets upon reading.



mod_dumpio: follow up to r1818802.

Negate APLOGctrace7(c) test!

Also, return DECLINED when nothing is to be done, same result as OK but
possibly more semantically correct.


Submitted by: ylavic
Reviewed by: jailletc36, jorton, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1828743 13f79535-47bb-0310-9956-ffa450edef68

mod_md: Fix compilation with OpenSSL before version 1.0.2.

Symbol ASN1_TIME_diff is only available for 1.0.2+,
but luckily alternative code we can use is already
available, originally written for the LibreSSL case.

Submitted by: rjung
Reviewed by: rjung, ylavic, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1828741 13f79535-47bb-0310-9956-ffa450edef68

mod_ssl: Fix merging of proxy SSL context outside <Proxy> sections.

Regression introduced in 2.4.30. PR 62232.

The proxy SSL_CTX was not inherited from the vhost (the only available in
2.4.29) in/for any directory context besides <Proxy>...

Mostly debugged and fixed by Rainer, thanks!


Submitted by: ylavic
Reviewed by: ylavic, rpluem, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1828735 13f79535-47bb-0310-9956-ffa450edef68

copy apr_sockaddr_is_wildcard to maintain 1.4.x support.




CHANGES for r1827654

Submitted by: covener
Reviewed by: covener, ylavic, rpluem, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1828734 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1828672 13f79535-47bb-0310-9956-ffa450edef68

modules/md/mod_md.h is now a new public API
also used by other modules (currently mod_ssl),
so it must be in the include path. It was
missing for cmake builds.

Note that this change does not yet enable building
mod_md itself using cmake. That part is still
missing.

CTR (cmake builds only).

Backport of r1828669 from trunk.


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1828670 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1827782 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1827651 13f79535-47bb-0310-9956-ffa450edef68