Skip to content
GitLab
  1. 14 Nov, 2016 2 commits
    • William A. Rowe Jr's avatar
      · 2cffd4c4
      William A. Rowe Jr authored 
      Rename the previously undocumented HTTPProtocol directive
to EnforceHTTPProtocol, and invert the default behavior
to strictly observe RFC 7230 unless otherwise configured.
And Document This.

The relaxation option is renamed 'Unsafe'. 'Strict' is no
longer case sensitive. 'min=0.9|1.0' is now the verbose
'Allow0.9' or 'Require1.0' case-insenstive grammer. The
exclusivity tests have been modified to detect conflicts.

The 'strict,log' option failed to enforce strict conformance,
and has been removed. Unsafe, informational logging is possible
in any loadable module, after the request data is unsafely
accepted.

This triggers a group of failures in t/apache/headers.t as
expected since those patterns violated RFC 7230 section 3.2.4.
Submitted by: wrowe
Backport: r1756540

Correct AP_HTTP_CONFORMANCE_ flags
Submitted by: wrowe
Backport: r1756555

Renaming this directive to HttpProtocolOptions after discussion on dev@
Submitted by: wrowe
Backport: r1756649

Perform correct, strict parsing of the request line, handling the
http protocol tag, url and method appropriately, and attempting 
to extract values even in the presence of unusual whitespace in
keeping with section 3.5, prior to responding with whatever
error reply is needed. Conforms to RFC7230 in all respects,
the section 3.5 optional behavior can be disabled by the user
with a new HttpProtocolOptions StrictWhitespace flag. In all
cases, the_request is regenerated from the parsed components
with exactly two space characters.

Shift sf's 'strict' method check from the Strict behavior because
it violates forward proxy logic, adding a new RegisteredMethods
flag, as it will certainly be useful to some.
Submitted by: wrowe
Backport: r1756729



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x-merge-http-strict@1769662 13f79535-47bb-0310-9956-ffa450edef68
      2cffd4c4
    • William A. Rowe Jr's avatar
      Improve legibility of reviewing the generated table, using hex rather than dec · 80281ca0
      William A. Rowe Jr authored 
      Submitted by: wrowe
Backport: r1754536

Correct T_HTTP_TOKEN_STOP per RFC2068 (2.2) - RFC7230 (3.2.6),
which has always defined 'token' as CHAR or VCHAR - visible USASCII only.
NUL char is also a stop, end of parsing.
Submitted by: wrowe
Backport: r1754538

Be more explicit about NUL in case iscntrl is inconsistent
Submitted by: wrowe
Backport: r1754539

Introduce T_HTTP_CTRLS for efficiently finding non-text chars
Submitted by: wrowe
Backport: r1754540

Introduce ap_scan_http_field_content, ap_scan_http_token
and ap_get_http_token [later reverted] for more efficient
string handling.
Submitted by: wrowe
Backport: r1754541

With NUL as a TOKEN_STOP, this code is more efficient
Submitted by: wrowe
Backport: r1754544

We arrive here for more than one cause; offer a more general statement
Submitted by: wrowe
Backport: r1754547

Strictly observe spec on obs-fold
Submitted by: wrowe
Backport: r1754548

Leave an emphatic TODO per Jeff's observations
Submitted by: trawick
Backport: r1754555

Introduce ap_scan_http_token / ap_scan_http_field_content for a much
more efficient pass through the header text; rather than reparsing
the strings over and over under the HTTP_CONFORMANCE_STRICT fules.

Improve logic and legibility by eliminating multiple repetitive tests
of the STRICT flag, and simply reorder 'classic' behavior first and
this new parser second to simplify the diff. Because of the whitespace
change (which I had wished to dodge), reading this --ignore-all-space
is a whole lot easier. Particularly against 2.4.x branch, which is now
identical in the 'classic' logic flow. Both of which I'll share with dev@
Submitted by: wrowe
Backport: r1754556

Friendly catch by Rüdiger, restore line mis-removed by the previous commit
Submitted by: rpluem
Backport: r1754568

Clean up doubled-'{'
Correct usage for ap_scan_http_token (had used _get_ syntax)
Correct logic, detect no 'token' chars, or missing ':'
Submitted by: wrowe, rpluem
Backport: r1754569,r1754570,r1754577

Replacement solution to identify VCHAR/ASCII symbols, even in EBCDIC.
Looking for someone with an EBCDIC environment to post the output of
the test_char.h generated file for verification.
Submitted by: wrowe
Backport: r1754579

Clean up an edge case where obs-fold continuation preceeds the first header,
as with r1755098, but this time ensure the previous header processing logic 
ensures there was a previous header as identified by jchampion.

This patch restructures the loop for legibility with a loop continuation,
allowing us to flatten all of this hard-to-follow code. The subsequent
patch will be a whitespace-only change for formatting.

Testing len > 0 is redundant when *field is a "\0" and mismatches here,
folded flag was a no-op, unused once we added continue; logic.
Fix these as initially attempted in r1755114.

Improve comments and reflow whitespace.
Submitted by: wrowe
Backport: r1755123,r1755124,r1755125,r1755126

As promised, reduce this logic by net 9 code lines, shifting the burden 
of killing trailing whitespace to the purpose-agnostic read logic.

Whitespace before or after an obs-fold, and before or after a field value
have no semantic purpose at all. Because we are building a buffer for all
folded values, reducing the size of the newly allocated buffer is always
to our advantage.
Submitted by: wrowe
Backport: r1755233

Treat empty obs-fold line as a noop, eliminate all intra-obs-fold excess
whitespace, and observe the 1 SP per obs-folding per spec.
Submitted by: wrowe
Backport: r1755234,r1755235,r1755236

Treat empty obs-fold line as abusive traffic.
Submitted by: wrowe
Backport: r1755263

Stop reflecting irrelevant data to the request error notes, particularly
for abusive and malformed traffic the non-technical consumer of a user-agent
has no control over.

Simply take note where the administrator-configured limits have been exceeded,
that administrator can find details in the error log if desired.
Submitted by: wrowe
Backport: r1755264

Follow up to r1755264.
Don't crash when ap_rgetline() returns a NULL field on ENOSPC.
Submitted by: ylavic
Backport: r1755343

Follow on to r1755264, for the case of merged header length exceptions,
and ensure the field header name is truncated to a sane log width.
Submitted by: wrowe
Backport: r1755744



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x-merge-http-strict@1769649 13f79535-47bb-0310-9956-ffa450edef68
      80281ca0
  3. 12 Nov, 2016 1 commit
  5. 09 Nov, 2016 5 commits
  7. 04 Nov, 2016 4 commits
    • William A. Rowe Jr's avatar
      ap_rgetline_core() now pulls from r->proto_input_filters · 1ddf815e
      William A. Rowe Jr authored 
      for better input filtering behavior during chunked trailer
processing by ap_http_filter().
Backports: r1446421
Submitted by: joes



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x-merge-http-strict@1768093 13f79535-47bb-0310-9956-ffa450edef68
      1ddf815e
    • William A. Rowe Jr's avatar
      Stupid CodeWarrior compiler cant take vars with struct inits. · bd6a3d15
      William A. Rowe Jr authored 
      Ensure that is_v6literal is always initialized

Backports: r1428145, r1436457
Submitted by: fuankg, rpluem



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x-merge-http-strict@1768090 13f79535-47bb-0310-9956-ffa450edef68
      bd6a3d15
    • William A. Rowe Jr's avatar
      Add an option to enforce stricter HTTP conformance · c8e1f5ae
      William A. Rowe Jr authored 
      This is a first stab, the checks will likely have to be revised.
For now, we check

 * if the request line contains control characters
 * if the request uri has fragment or username/password
 * that the request method is standard or registered with RegisterHttpMethod
 * that the request protocol is of the form HTTP/[1-9]+.[0-9]+,
   or missing for 0.9
 * if there is garbage in the request line after the protocol
 * if any request header contains control characters
 * if any request header has an empty name
 * for the host name in the URL or Host header:
   - if an IPv4 dotted decimal address: Reject octal or hex values, require
     exactly four parts
   - if a DNS host name: Reject non-alphanumeric characters besides '.' and
     '-'. As a side effect, this rejects multiple Host headers.
 * if any response header contains control characters
 * if any response header has an empty name
 * that the Location response header (if present) has a valid scheme and is
   absolute

If we have a host name both from the URL and the Host header, we replace the
Host header with the value from the URL to enforce RFC conformance.

There is a log-only mode, but the loglevels of the logged messages need some
thought/work. Currently, the  checks for incoming data log for 'core' and the
checks for outgoing data log for 'http'. Maybe we need a way to configure the
loglevels separately from the core/http loglevels.

change protocol number parsing in strict mode according to HTTPbis draft
- only accept single digit version components
- don't accept white-space after protocol specification

Clean up comment, fix log tags.
Submitted by: sf
Backports: r1426877, r1426879, r1426988, r1426992



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x-merge-http-strict@1768036 13f79535-47bb-0310-9956-ffa450edef68
      c8e1f5ae
    • William A. Rowe Jr's avatar
      Correctly parse an IPv6 literal host specification in an absolute URL · 6dbeba9d
      William A. Rowe Jr authored 
      in the request line.

- Fix handling of brackets [ ] surrounding the IPv6 address.
- Skip parsing r->hostname again if not necessary.
- Do some checks that the IPv6 address is sane. This is not done by
  apr_parse_addr_port().

log client error at level debug, log broken Host header value

Backports: r1407006, r1426827
Submitted by: sf



git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x-merge-http-strict@1768035 13f79535-47bb-0310-9956-ffa450edef68
      6dbeba9d
  9. 03 Nov, 2016 5 commits
  11. 02 Nov, 2016 6 commits
  13. 01 Nov, 2016 5 commits
  15. 28 Oct, 2016 1 commit
  17. 27 Oct, 2016 3 commits
  19. 26 Oct, 2016 1 commit
    • Jim Jagielski's avatar
      Merge r1764040 from trunk: · 6d57c7e3
      Jim Jagielski authored 
      mod_dav: Fix a potential cause of unbounded memory usage or incorrect
behavior in a routine that sends <DAV:response>'s to the output filters.

The dav_send_one_response() function accepts the current head of the output
filter list as an argument, but the actual head can change between calls to
ap_pass_brigade().  This can happen with self-removing filters, e.g., with
the filter from mod_headers or mod_deflate.  Consequently, executing an
already removed filter can either cause unwanted memory usage or incorrect
behavior.

This patch changes the signature of the existing mod_dav's public API,
dav_send_one_response(), because this API is not yet a part of any 2.4.x
release.

* modules/dav/main/mod_dav.c
  (dav_send_one_response): Accept a request_rec instead of an ap_filter_t.
   Write the response to r->output_filters.
  (dav_send_multistatus, dav_stream_response): Update these calling sites
   of dav_send_one_response().

* modules/dav/main/mod_dav.h
  (dav_send_one_response): ...
      6d57c7e3
  21. 24 Oct, 2016 6 commits
  23. 23 Oct, 2016 1 commit