mod_macro: fix usability of globally defined macros in .htaccess files.
PR 57525.

Reverts pre_config hook from r1656669 (happens too late for EXEC_ON_READ), and
ensures ap_macros is reset on restart with a pconf cleanup.

Proposed by: Jose Kahan <jose w3.org>
Reviewed by: ylavic

Submitted by: ylavic
Reviewed by: ylavic, icing, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1815101 13f79535-47bb-0310-9956-ffa450edef68

core, mod_rewrite: introduce the 'redirect-keeps-vary' note
                   to allow proper Vary header insertion when
                   dealing with a RewriteRule in a directory
                   context.

This change is an attempt to fix a long standing problem,
brought up while working on PR 58231. Our documentation clearly
states the following:

"If a HTTP header is used in a condition this header is added
to the Vary header of the response in case the condition
evaluates to true for the request."

This is currently not true for RewriteCond/Rules working in
a directory context, since when an internal redirect happens
all the outstanding response headers get dropped.

There might be a better solution so I am looking forward to
hear more opinions and comments. My goal for a delicate change
like this one would be to affect the least amount of configurations
possible, without triggering unwanted side effects.

If the solution is good for everybody tests will be written
in the suite asap.


Submitted by: elukey
Reviewed by: elukey, icing, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1815100 13f79535-47bb-0310-9956-ffa450edef68

Merged /httpd/httpd/trunk:r1811649,1811664,1814118

  *) ab: Make the TLS layer aware that the underlying socket is nonblocking,
     and use/handle POLLOUT where needed to avoid busy IOs and recover write
     errors when appropriate.  [Yann Ylavic]

  *) ab: Keep reading nonblocking to exhaust TCP or SSL buffers when previous
     read was incomplete (the SSL case can cause the next poll() to timeout
     since data are buffered already).  PR 61301 [Luca Toscano, Yann Ylavic]




git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1814468 13f79535-47bb-0310-9956-ffa450edef68

Merge of r1813767 from trunk.

mod_http2: avoid unnecessary data retrieval for a trace log. Allow certain
     information retrievals on null bucket beams where it makes sense.



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1814420 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1814136 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1813980 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1812442 13f79535-47bb-0310-9956-ffa450edef68

* modules/metadata/mod_unique_id.c: Replace use of hostname + pid with
  PRNG output.

Submitted by: jkaluza
Reviewed by: jorton, wrowe, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1812267 13f79535-47bb-0310-9956-ffa450edef68

mod_rewrite/core: avoid the 'Vary: Host' header

In PR 58231 is was brought up that httpd adds the
Vary: Host header whenever a condition is set to true
in mod_rewrite or in an <If> block.

The https://tools.ietf.org/html/rfc7231#section-7.1.4
section seems to disallow this use case:

"The "Vary" header field in a response describes "
"what parts of a request message, "
"aside from the method, Host header field, [...]"

I had a chat with the folks in #traffic-server and
they don't see much point in having a Vary: Host header,
plus it was reported that Varnish doesn't like it very
much (namely it does not cache the response when
it sees the header, links of the report in the PR).

I don't see much value in this behavior of httpd so
I am inclined to remove this response header value,
but I'd be glad to get a more experienced opinion.



mod_rewrite,core: avoid Vary:Host (part 2)

This is a follow up of r1808746 after a chat
with Yann on dev@:

- the HTTP:Host variable suffers from the same problem
- the strcasecmp should be used to allow case-sensitive
  comparisons.
- in mod_rewrite is less cumbersome and more clean to just
  make the Host header check in lookup_header, so it will
  be automatically picked up by every part of the code
  that uses it. It shouldn't be a relevant overhead for
  mod_rewrite.


Submitted by: elukey
Reviewed by: elukey, ylavic, wrowe


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1812083 13f79535-47bb-0310-9956-ffa450edef68

bumping version, removing some unused code, fixes in base64url from mod_md

On the trunk:

  *) mod_http2: DoS flow control protection is less agressive as long as active tasks stay
     below worker capacity. Intended to fix problems with media streaming. 


On the trunk:
mod_http2: v0.10.12, removed optimization for mutex handling in bucket beams that could lead to assertion failure in edge cases.


reverting r1807238 bc not addressing the issue https://github.com/icing/mod_h2/issues/120

mod_http2: non-dev 1.10.12 for backport


Submitted by: icing
Reviewed by: icing, steffenal, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1812081 13f79535-47bb-0310-9956-ffa450edef68

Update offsets

Entry for 2.4.28 regression (r1808855 missing r1805195).

Submitted by: jim, ylavic
Reviewed/backported by: ylavic (RTC per miss in the original merge)


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1812074 13f79535-47bb-0310-9956-ffa450edef68

Fix a segmentation fault if AuthzDBDQuery is not set.

PR: 61546
Submitted by: Lubos Uhliarik <luhliari redhat.com>
Reviewed by: jailletc36, ylavic, elukey


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1811749 13f79535-47bb-0310-9956-ffa450edef68

*) mod_rewrite: Add support for starting External Rewriting Programs
   as non-root user on UNIX systems by specifying username and group name
   as third argument of RewriteMap directive.

Submitted by: jkaluza
Reviewed by: jorton, wrowe, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1811748 13f79535-47bb-0310-9956-ffa450edef68

* server/protocol.c (ap_content_length_filter): Rewrite the content
  length filter to avoid arbitrary memory consumption for streaming
  responses (e.g. large CGI script output).  Ensures C-L is still
  generated in common cases (static content, small CGI script output),
  but this DOES change behaviour and some responses will end up
  chunked rather than C-L computed.

PR: 61222
Submitted by: jorton, rpluem
Reviewed by: jorton, wrowe, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1811746 13f79535-47bb-0310-9956-ffa450edef68

mod_ssl: return non ambiguous value in ssl_callback_SessionTicket() for
encryption mode (we used to return 0, OpenSSL documents returning 1 instead).

Practically this does not change anything since OpenSSL will only check for
>= 0 return value (non error) for encryption mode (the other possible return
values are only relevant for decryption mode).

However the OpenSSL documentation for SSL_CTX_set_tlsext_ticket_key_cb()
states:
"
The return value of the cb function is used by OpenSSL to determine what
further processing will occur. The following return values have meaning:

2
    This indicates that the ctx and hctx have been set and the session can
    continue on those parameters. Additionally it indicates that the session
    ticket is in a renewal period and should be replaced. The OpenSSL library
    will call cb again with an enc argument of 1 to set the new ticket (see
    RFC5077 3.3 paragraph 2).

1
    This indicates that the ctx and hctx have been set and the session can
    continue on those parameters.

0
    This indicates that it was not possible to set/retrieve a session ticket
    and the SSL/TLS session will continue by by negotiating a set of
    cryptographic parameters or using the alternate SSL/TLS resumption
    mechanism, session ids.
    If called with enc equal to 0 the library will call the cb again to get a
    new set of parameters.

less than 0
    This indicates an error.
"

So 0 is not appropriate in our code, 1 is what we really want (and it won't
break if OpenSSL later changes its checks on the callback return value).

Reported/Proposed by: oknet on github, pull request #18.
Reviewed by: jorton, ylavic, wrowe
[Closes #18]


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1811742 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1809610 13f79535-47bb-0310-9956-ffa450edef68

event: Avoid possible blocking in the listener thread when shutting down
connections. PR 60956.

start_lingering_close_nonblocking() now puts connections in defer_linger_chain
which is emptied by any worker thread (all atomically) after its usual work,
hence any possibly blocking flush and lingering close run outside the listener.

The listener may create a dedicated worker if it fills defer_linger_chain or
while it's not empty, calling push2worker with a NULL cs.

The state machine in process_socket() is slighly modified to be able to enter
with CONN_STATE_LINGER directly w/o clogging_input_filters to interfer.

New abort_socket_nonblocking() allows to reset connections when nonblocking is
required and we can't do much about the connection anymore, nor we want the
system to linger on its own after close().

Many thanks to Stefan Priebe for his heavy testing on many event's changes!


Submitted by: ylavic
Reviewed by: ylavic, jim, icing


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1809299 13f79535-47bb-0310-9956-ffa450edef68

mod_speling/PR 38923: don't embed Referer in link in error page.

Submitted by: niq
Reviewed by: niq, ylavic, elukey


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1808856 13f79535-47bb-0310-9956-ffa450edef68

loadfactor now decimal

Umm. This is C :)

Fix remaining lint from ms capability for hchecks.

format typo

Reviewed by: jim, humbedooh, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1808855 13f79535-47bb-0310-9956-ffa450edef68

htdigest: prevent buffer overflow when strings in lines are too long.

Reported by: Hanno Böck
PR: 61511



Update CHANGES after r1808008
Submitted by: elukey
Reviewed by: elukey, icing, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1808853 13f79535-47bb-0310-9956-ffa450edef68

https://svn.apache.org/r1807754



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1808787 13f79535-47bb-0310-9956-ffa450edef68

Add logic to read the Upgrade header and use it in the response.
Use we you are proxying to a server that has multiple upgrade on the same IP/Port.
PR 61142

Submitted by: jfclere
Reviewed by: jfclere, jim, covener


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1808698 13f79535-47bb-0310-9956-ffa450edef68

Allow WatchdogInterval to be sub 1 second

Allow finer control over hcheck intervals... minimum is whatever
the watchdog slice is.

Fix remaining lint from ms capability for hchecks.

userland change = we now are OK w/ ms


Submitted by: jim
Reviewed by: jim, covener, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1807772 13f79535-47bb-0310-9956-ffa450edef68

mod_http2: version bump, partial fix for stream response getting stuck, see https://github.com/icing/mod_h2/issues/143

mod_http2: signalling produce IO before waiting on beam buffer to drain.

mod_http2: non-dev version for backport

Submitted by: icing
Reviewed/backported by: icing, steffenal, ylavic


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1807771 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1807762 13f79535-47bb-0310-9956-ffa450edef68

core: Disallow Methods' registration at run time (.htaccess), they may be
used only if registered at init time (httpd.conf).

Calling ap_method_register() in children processes is not the right scope
since it won't be shared for all requests.

Reviewed by: ylavic, covener, icing


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1807754 13f79535-47bb-0310-9956-ffa450edef68

mod_ssl: we can't use SSL_COMP_free_compression_methods() if OPENSSL_NO_COMP
is defined.  PR 61206.

Submitted by: Michael Schlenker <msc contact.de>


mod_ssl, ab: compatibility with LibreSSL.  PR 61184.

LibreSSL defines OPENSSL_VERSION_NUMBER = 2.0, but is not compatible with
all of the latest OpenSSL 1.1 API.

Address this by defining MODSSL_USE_OPENSSL_PRE_1_1_API which is true for
anything but OpenSSL >= 1.1 (for now).

Proposed by: Bernard Spil <brnrd freebsd.org>
Reviewed by: ylavic


Follow up to r1803396: CHANGES entry.


Reviewed by: ylavic, jim, covener


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1807734 13f79535-47bb-0310-9956-ffa450edef68

 Support use of optional "tag" in syslog entries. Streamline the patch.
 PR 60525.

Submitted by: jim
Reviewed by: rpluem, jim, mrumph


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1807707 13f79535-47bb-0310-9956-ffa450edef68

Fix ProxyAddHeaders merging.

* modules/proxy/mod_proxy.h:
  Add add_forwarded_headers_set field to proxy_dir_conf.

* modules/proxy/mod_proxy.c (create_proxy_dir_config):
  Initialize add_forwarded_headers_set.
  (add_proxy_http_headers): Set it.
  (merge_proxy_dir_config): Merge add_forwarded_headers correctly.

Reviewed by: jorton, rpluem, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1805390 13f79535-47bb-0310-9956-ffa450edef68

core: Disallow multiple Listen on the same IP:port when listener buckets
are configured (ListenCoresBucketsRatio > 0), consistently with the single
bucket case (default), thus fixing the leak of the corresponding socket
descriptors on graceful restart.



Follow up to r1789220.
Document the implicit behaviour of ListenCoresBucketsRatio when multiple
Listen-ers are configured on the same IP:port.


Submitted by: ylavic
Reviewed by: ylavic, jim, jorton


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1805221 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1804229 13f79535-47bb-0310-9956-ffa450edef68

Merge r1762580, r1762701, r1762702, r1762718, r1762723, r1762742, r1762743, r1774538, r1779354 from trunk:

event: use atomics for *timeout_queue->total since it's updated concurrently,
and move TO_QUEUE_*() macros to functions.


event: add/remove from/to the pollset outside of the critical sections.

We don't need external locking since it's created with APR_POLLSET_THREADSAFE,
hence reduce those sections to the lowest cycles possible.

A spinlock may be interesting instead of the mutex now, we won't block and the
TO_QUEUE_*() and process_timeout_queue() operations are fast...



event: follow up to r1762701: update log tag.

event: avoid unnecessary listener/polling wake ups (context switches) by using
apr_pollset_wakeup(), when implemented, to signal the listener according to the
next timers or timeout queues expiry (updated at insert and maintenance time).



Follow up to r1762718: CHANGES entry.

event: follow up to r1762718.
We still need to kill kept-alive connections in normal/expiry processing if
the workers are busy or dying.



event: follow up to r1762718 and r1762742: put de condition where it belongs.



event: follow up to r1762718.

On graceful shutdown/restart, kill kept-alive connections before poll()ing
again, avoiding to wait for their "normal" timers (before being woken up)
when they remain the last handled connections.



event: follow up to r1762701.
Keep QUEUE_APPEND()+pollset_add() or QUEUE_REMOVE()+pollset_remove() atomic.

Otherwise when a worker adds an entry in some queue (e.g. KA, lingering), it
might race with the listener in the time between the mutex is released and the
pollset is updated; meanwhile the listener might process the queue and find an
entry no yet in its pollset.

For the lingering queue, the entry could then have been used after its pool
destroyed.

Submitted by: ylavic
Reviewed by: ylavic, icing, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1802146 13f79535-47bb-0310-9956-ffa450edef68

mod_proxy_wstunnel: we want to detect whether some response was sent to
the client when forwarding data from the backend to the client, not the
reverse.


Follow up to r1754164: CHANGES entry.

Follow up to r1801994: CHANGES' PR reference.
Submitted by: ylavic
Reviewed by: ylavic, jchampion, jim


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1802144 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1802129 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1802121 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1801291 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1801086 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1801080 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1801079 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1801074 13f79535-47bb-0310-9956-ffa450edef68