mod_firehose
Multiplexes all I/O to a given file or pipe.
Extension
mod_firehose.c
firehose_module

`mod_firehose` provides a mechanism to record data being passed between the httpd server and the client at the raw connection level to either a file or a pipe in such a way that the data can be analysed or played back to the server at a future date. It can be thought of as "tcpdump for httpd".

Connections are recorded after the SSL has been stripped, and can be used for forensic debugging.

The firehose tool can be used to demultiplex the recorded stream back into individual files for analysis, or playback using a tool like `netcat`.

# WARNING

This module IGNORES all request level mechanisms to keep data private. It is the responsibility of the administrator to ensure that private data is not inadvertently exposed using this module.

firehose

# Enabling a Firehose

To enable the module, it should be compiled and loaded in to your running Apache configuration, and the directives below used to record the data you are interested in.

It is possible to record both incoming and outgoing data to the same filename if desired, as the direction of flow is recorded within each fragment.

It is possible to write to both normal files and fifos (pipes). In the case of fifos, mod_firehose ensures that the packet size is no larger than PIPE_BUF to ensure writes are atomic.

If a pipe is being used, something must be reading from the pipe before httpd is started for the pipe to be successfully opened for write. If the request to open the pipe fails, mod_firehose will silently stand down and not record anything, and the server will keep running as normal.

By default, all attempts to write will block the server. If the webserver has been built against APR v2.0 or later, and an optional "nonblock" parameter is specified all file writes will be non blocking, and buffer overflows will cause debugging data to be lost. In this case it is possible to prioritise the running of the server over the recording of firehose data.

# Stream Format

The server typically serves multiple connections simultaneously, and as a result requests and responses need to be multiplexed before being written to the firehose.

The fragment format is designed as clear text, so that a firehose can be opened with and inspected by a normal text editor. Alternatively, the firehose tool can be used to demultiplex the firehose back into individual requests or connections.

The size of the multiplexed fragments is governed by PIPE_BUF, the maximum size of write the system is prepared to perform atomically. By keeping the multiplexed fragments below PIPE_BUF in size, the module guarantees that data from different fragments does not interleave. The size of PIPE_BUF varies on different operating systems.

The BNF for the fragment format is as follows:

     stream = 0*(fragment)

     fragment = header CRLF body CRLF

     header = length SPC timestamp SPC ( request | response ) SPC uuid SPC count

     length = <up to 16 byte hex fragment length>
     timestamp = <up to 16 byte hex timestamp microseconds since 1970>
     request = "<"
     response = ">"
     uuid = <formatted uuid of the connection>
     count = <hex fragment number in the connection>

     body = <the binary content of the fragment>

     SPC = <a single space>
     CRLF = <a carriage return, followed by a line feed>

All fragments for a connection or a request will share the same UUID, depending on whether connections or requests are being recorded. If connections are being recorded, multiple requests may appear within a connection. A fragment with a zero length indicates the end of the connection.

Fragments may go missing or be dropped if the process reading the fragments is too slow. If this happens, gaps will exist in the connection counter numbering. A warning will be logged in the error log to indicate the UUID and counter of the dropped fragment, so it can be confirmed the fragment was dropped.

It is possible that the terminating empty fragment may not appear, caused by the httpd process crashing, or being terminated ungracefully. The terminating fragment may be dropped if the process reading the fragments is not fast enough.

FirehoseConnectionInput
Capture traffic coming into the server on each connection
FirehoseConnectionInput [ block | nonblock ] filename
none
server config
FirehoseConnectionInput is only available in Apache 2.5.0 and later.

Capture traffic coming into the server on each connection. Multiple requests will be captured within the same connection if keepalive is present.

# Example

FirehoseConnectionInput connection-input.firehose
FirehoseConnectionOutput
Capture traffic going out of the server on each connection
FirehoseConnectionOutput [ block | nonblock ] filename
none
server config
FirehoseConnectionOutput is only available in Apache 2.5.0 and later.

Capture traffic going out of the server on each connection. Multiple requests will be captured within the same connection if keepalive is present.

# Example

FirehoseConnectionOutput connection-output.firehose
FirehoseRequestInput
Capture traffic coming into the server on each request
FirehoseRequestInput [ block | nonblock ] filename
none
server config
FirehoseRequestInput is only available in Apache 2.5.0 and later.

Capture traffic coming into the server on each request. Requests will be captured separately, regardless of the presence of keepalive.

# Example

FirehoseRequestInput request-input.firehose
FirehoseRequestOutput
Capture traffic going out of the server on each request
FirehoseRequestOutput [ block | nonblock ] filename
none
server config
FirehoseRequestOutput is only available in Apache 2.5.0 and later.

Capture traffic going out of the server on each request. Requests will be captured separately, regardless of the presence of keepalive.

# Example

FirehoseRequestOutput request-output.firehose
FirehoseProxyConnectionInput
Capture traffic coming into the back of mod_proxy
FirehoseProxyConnectionInput [ block | nonblock ] filename
none
server config
FirehoseProxyConnectionInput is only available in Apache 2.5.0 and later.

Capture traffic being received by mod_proxy.

# Example

FirehoseProxyConnectionInput proxy-input.firehose
FirehoseProxyConnectionOutput
Capture traffic sent out from the back of mod_proxy
FirehoseProxyConnectionOutput [ block | nonblock ] filename
none
server config
FirehoseProxyConnectionOutput is only available in Apache 2.5.0 and later.

Capture traffic being sent out by mod_proxy.

# Example

FirehoseProxyConnectionOutput proxy-output.firehose