mod_proxy_hcheck
Dynamic health check of Balancer members (workers) for mod_proxy
Extension
mod_proxy_hcheck.c
proxy_hcheck_module
Available in Apache 2.4.21 and later

This module provides for dynamic health checking of balancer members (workers). This can be enabled on a worker-by-worker basis. The health check is done independently of the actual reverse proxy requests.

This module requires the service of mod_watchdog.

# Parameters

The health check mechanism is enabled via the use of additional BalancerMember parameters, which are configured in the standard way via ProxyPass:

A new BalancerMember status state (flag) is defined via this module: "`C`". When the worker is taken offline due to failures as determined by the health check module, this flag is set, and can be seen (and modified) via the `balancer-manager`.

Parameter | Default | Description
--- | --- | ---

hcmethod

None

No dynamic health check performed. Choices are:

Method | Description | Note
--- | --- | ---
None | No dynamic health checking done |
TCP | Check that a socket to the backend can be created: e.g. "are you up" |
OPTIONS | Send an `HTTP OPTIONS` request to the backend | *
HEAD | Send an `HTTP HEAD` request to the backend | *
GET | Send an `HTTP GET` request to the backend | *
*: Unless `hcexpr` is used, a 2xx or 3xx HTTP status will be interpreted as passing the health check

hcpasses | 1 | Number of successful health check tests before worker is re-enabled
hcfails | 1 | Number of failed health check tests before worker is disabled
hcinterval | 30 | Period of health checks in seconds (e.g. performed every 30 seconds)
hcuri |  | Additional URI to be appended to the worker URL for the health check.
hctemplate |  | Name of template, created via ProxyHCTemplate, to use for setting health check parameters for this worker
hcexpr |  | Name of expression, created via ProxyHCExpr, used to check response headers for health. If not used, 2xx thru 3xx status codes imply success

mod_proxy

# Usage examples

The following example shows how one might configured health checking for various backend servers:

ProxyHCExpr ok234 {%{REQUEST_STATUS} =~ /^[234]/} ProxyHCExpr gdown {%{REQUEST_STATUS} =~ /^[5]/} ProxyHCExpr in_maint {hc('body') !~ /Under maintenance/}

<Proxy balancer://foo> BalancerMember http://www.example.com/ hcmethod=GET hcexpr=in_maint hcuri=/status.php BalancerMember http://www2.example.com/ hcmethod=HEAD hcexpr=ok234 hcinterval=10 BalancerMember http://www3.example.com/ hcmethod=TCP hcinterval=5 hcpasses=2 hcfails=3 BalancerMember http://www4.example.com/ </Proxy>

ProxyPass "/" "balancer://foo" ProxyPassReverse "/" "balancer://foo"

In this scenario, `http://www.example.com/` is health checked by sending a `GET /status.php` request to that server and seeing that the returned page does not include the string Under maintenance. If it does, that server is put in health-check fail mode, and disabled. This dynamic check is performed every 30 seconds, which is the default.

`http://www2.example.com/` is checked by sending a simple `HEAD` request every 10 seconds and making sure that the response status is 2xx, 3xx or 4xx. `http://www3.example.com/` is checked every 5 seconds by simply ensuring that the socket to that server is up. If the backend is marked as "down" and it passes 2 health check, it will be re-enabled and added back into the load balancer. It takes 3 back-to-back health check failures to disable the server and move it out of rotation. Finally, `http://www4.example.com/` is not dynamically checked at all.

ProxyHCExpr
Creates a named condition expression to use to determine health of the backend based on its response
ProxyHCExpr name {ap_expr expression}
server config
virtual host

The ProxyHCExpr directive allows for creating a named condition expression that checks the response headers of the backend server to determine its health. This named condition can then be assigned to balancer members via the `hcexpr` parameter.

# ProxyHCExpr: Allow for 2xx/3xx/4xx as passing

ProxyHCExpr ok234 {%{REQUEST_STATUS} =~ /^[234]/} ProxyPass "/apps" "balancer://foo"

<Proxy balancer://foo> BalancerMember http://www2.example.com/ hcmethod=HEAD hcexpr=ok234 hcinterval=10 </Proxy>

The expression can use curly-parens ("{}") as quoting deliminators in addition to normal quotes.

If using a health check method (eg: `GET`) which results in a response body, that body itself can be checked via `ap_expr` using the `hc()` expression function, which is unique to this module.

In the following example, we send the backend a `GET` request and if the response body contains the phrase Under maintenance, we want to disable the backend.

# ProxyHCExpr: Checking response body

ProxyHCExpr in_maint {hc('body') !~ /Under maintenance/} ProxyPass "/apps" "balancer://foo"

<Proxy balancer://foo> BalancerMember http://www.example.com/ hcexpr=in_maint hcmethod=get hcuri=/status.php </Proxy>

NOTE: Since response body can quite large, it is best if used against specific status pages.

ProxyHCTemplate
Creates a named template for setting various health check parameters
ProxyHCTemplate name parameter=setting [...]
server config
virtual host

The ProxyHCTemplate directive allows for creating a named set (template) of health check parameters that can then be assigned to balancer members via the `hctemplate` parameter.

# ProxyHCTemplate

ProxyHCTemplate tcp5 hcmethod=tcp hcinterval=5 ProxyPass "/apps" "balancer://foo"

<Proxy balancer://foo> BalancerMember http://www2.example.com/ hctemplate=tcp5 </Proxy>
ProxyHCTPsize
Sets the total server-wide size of the threadpool used for the health check workers
ProxyHCTPsize size
ProxyHCTPsize 16
server config

If Apache httpd and APR are built with thread support, the health check module will offload the work of the actual checking to a threadpool associated with the Watchdog process, allowing for parallel checks. The ProxyHCTPsize directive determines the size of this threadpool. If set to `0`, no threadpool is used at all, resulting in serialized health checks.

# ProxyHCTPsize

ProxyHCTPsize 32