HAProxy Operator is a Kubernetes-native solution designed to automate the deployment, configuration, and management of HAProxy instances using Custom Resources to abstract the key components such as backends, frontends, and listens.
helm repo add six-group https://six-group.github.io/haproxy-operatorhelm install haproxy-operator six-group/haproxy-operatorThis example will guide you through the process of setting up a basic HAProxy instance, configuring a frontend for receiving traffic, inspecting the generated HAProxy configuration, and making a sample request to demonstrate its functionality.
Create a simple instance of the HAProxy by applying the following YAML manifest:
apiVersion: proxy.haproxy.com/v1alpha1kind: Instancemetadata: name: examplenamespace: defaultspec: configuration: defaults: {}global: {}selector: matchLabels: proxy.haproxy.com/instance: examplenetwork: service: enabled: true
To define the port at which HAProxy should receive traffic, create a basic frontend configuration by applying the following YAML manifest:
apiVersion: config.haproxy.com/v1alpha1kind: Frontendmetadata: name: examplenamespace: defaultlabels: proxy.haproxy.com/instance: examplespec: mode: httpbinds: - name: hello-worldport: 8080defaultBackend: {}
Check the generated
haproxy.cfgstored in theSecretexample-haproxy-config:defaults haproxy_defaults mode http timeout connect 5000 timeout client 5000 timeout server 10000 frontend example mode http bind :8080 name hello-worldThe HAProxy pod is now listening on port 8080 exposed by a
Servicecalledexample-haproxy. If you make a request usingcurlexecuted from a pod within the same namespace, you’ll get back a response:$ curl http://example-haproxy:8080 <html><body><h1>503 Service Unavailable</h1> No server is available to handle this request. </body></html>
Granted, there’s no reply from a server since we haven't configured any backend servers yet. Nevertheless, you can see that HAProxy is functional.
For a more in-depth understanding of the HAProxy Operator and to explore complex use cases, refer to the upcoming sections in this documentation. These sections will provide detailed explanations, advanced examples, and configuration options to help you tailor the HAProxy solution to your specific requirements.
An HAProxy instance refers to a single running instance of the HAProxy service. This service can be configured to manage the load balancing and distribution of network traffic among a set of servers or backends within or external to a Kubernetes cluster.
Each HAProxy instance has its own configuration file, named haproxy.cfg and stored as a Secret, which defines all the settings for that instance, including defaults, frontends, and backends. This configuration file specifies how incoming connections are handled, which algorithms are used for load balancing, and how to monitor the health of the backends. Multiple HAProxy instances can be run on the same namespace, each with its own configuration and each listening on different ports.
Example:
This is a configuration for an HAProxy instance with two sections: global and defaults. The global section sets process-wide parameters, including the number of threads, maximum concurrent connections, stats socket configuration, buffer sizes, SSL parameters, and logging settings. The defaults section sets default parameters for all other sections. It sets the mode to TCP, enables logging, and sets various timeout values for different types of connections and requests.
global
nbthread 4
stats socket /var/lib/haproxy/run/haproxy.sock expose-fd listeners level admin mode 600
stats timeout 300000
tune.bufsize 32768
tune.maxrewrite 8192
tune.ssl.default-dh-param 2048
ssl-default-bind-options ssl-min-ver TLSv1.2 ssl-default-bind-ciphers SHA256
ssl-default-bind-ciphersuites TLS_SHA256
log /var/lib/rsyslog/rsyslog.sock local0
log-send-hostname
defaults haproxy_defaults
mode tcp
log global
option tcplog
timeout http-request 10000
timeout connect 5000
timeout client 30000
timeout client-fin 1000
timeout server 30000
timeout server-fin 1000
timeout tunnel 3600000
timeout http-keep-alive 300000
apiVersion: proxy.haproxy.com/v1alpha1kind: Instancemetadata:
name: examplenamespace: defaultspec:
configuration:
defaults:
logging:
enabled: truetcpLog: truemode: tcptimeouts:
client: 30sclient-fin: 1sconnect: 5shttp-keep-alive: 5m0shttp-request: 10sserver: 600sserver-fin: 1stunnel: 1h0m0sselector:
matchLabels:
proxy.haproxy.com/instance: exampleglobal:
logging:
address: /var/lib/rsyslog/rsyslog.sockenabled: truefacility: local0ssl:
defaultBindCipherSuites:
- TLS_SHA256defaultBindCiphers:
- SHA256defaultBindOptions:
minVersion: TLSv1.2statsTimeout: 5m0stune:
bufsize: 32768maxrewrite: 8192ssl:
defaultDHParam: 2048nbthread: 4reload: trueimage: 'haproxy:2.8.0'replicas: 2network:
route:
enabled: falseservice:
enabled: falseAPI Reference Instance defines all the features that can be configured in an HAProxy instance.
For the dynamic configuration of HAProxy instances, custom resources have been created for each configuration section, i.e., listen, frontend, backend, and resolver.
These configuration resources are associated with particular instances by the use of label selectors. A label selector is specified within the Instance configuration, and the corresponding label is applied to each configuration resource to establish a relation.
An example of a label selector used within an Instance to match a specific HAProxy instance is provided below:
selector:
matchLabels:
proxy.haproxy.com/instance: exampleThis approach allows HAProxy instances to be configured dynamically, with a focus on modularity and ease of management.
Frontend defines how incoming connections are handled based on the rules defined. It specifies the IP addresses and ports that HAProxy listens on and sets rules for what to do with connections once they are received. These rules can include Access Control Lists (ACLs), which allow you to route traffic based on various factors such as the client's IP address, the requested URL, or the type of protocol used. The HAProxy Operator allows you to define frontends in a declarative manner, specifying things like the port number and the default backend.
Example 1:
The HAProxy frontend 'example-1' operates in HTTP mode and listens for incoming connections on a Unix socket at /var/lib/haproxy/run/local.sock:9443. It has a certificate file configured, which is used to terminate TLS connections. It also has a default backend configured, which is used when no other rules match an incoming request.
frontend example-1
mode http
bind unix@/var/lib/haproxy/run/local.sock:9443 name https crt /usr/local/etc/haproxy/ssl-certs.crt ssl accept-proxy crt-list /usr/local/etc/haproxy/cert_list.map
errorfile 403 /usr/local/etc/haproxy/error-403.http
use_backend %[base,map_reg(/usr/local/etc/haproxy/edge.map)] if { base,map_reg(/usr/local/etc/haproxy/edge.map) -m found }
use_backend %[base,map_reg(/usr/local/etc/haproxy/reencrypt.map)] if { base,map_reg(/usr/local/etc/haproxy/reencrypt.map) -m found }
default_backend default-namespace
apiVersion: config.haproxy.com/v1alpha1kind: Frontendmetadata:
name: example-1namespace: defaultlabels:
proxy.haproxy.com/instance: examplespec:
backendSwitching:
- backend:
regexMapping:
name: edgeparameter: basecondition: '{ base,map_reg(/usr/local/etc/haproxy/edge.map) -m found }'conditionType: if
- backend:
regexMapping:
name: reencryptparameter: basecondition: '{ base,map_reg(/usr/local/etc/haproxy/reencrypt.map) -m found }'conditionType: ifbinds:
- acceptProxy: trueaddress: unix@/var/lib/haproxy/run/local.sockhidden: truename: httpsport: 9443ssl:
certificate:
name: ssl-certsvalueFrom:
- secretKeyRef:
key: tls.crtname: ssl-certs
- secretKeyRef:
key: tls.keyname: ssl-certsenabled: truesslCertificateList:
name: cert_listdefaultBackend:
name: default-namespaceerrorFiles:
- code: 403file:
name: error-403value: |- HTTP/1.0 403 Forbidden Pragma: no-cache Cache-Control: private, max-age=0, no-cache, no-store Connection: close Content-Type: text/html <!DOCTYPE html> <html lang="en"> <head> <title>403 Forbidden</title> </head> </html>valueFrom: {}mode: httpExample 2:
This is a HAProxy frontend configuration named 'example-2'. It operates in TCP mode, binds to a specific IP and port, and inspects TCP requests with a delay. It accepts requests with a specific SSL hello type.
frontend example-2
mode tcp
bind ${BIND_ADDRESS}:443 name public-ssl
tcp-request inspect-delay 5000
tcp-request content accept if { req_ssl_hello_type 1 }
default_backend default-namespace
apiVersion: config.haproxy.com/v1alpha1kind: Frontendmetadata:
name: example-2namespace: defaultlabels:
proxy.haproxy.com/instance: examplespec:
binds:
- address: '${BIND_ADDRESS}'name: public-sslport: 443defaultBackend:
name: default-namespacemode: tcptcpRequest:
- timeout: 5stype: inspect-delay
- action: acceptcondition: '{ req_ssl_hello_type 1 }'conditionType: iftype: contentExample 3:
This is a HAProxy frontend configuration named 'example-3'. It operates in HTTP mode and binds to a specific IP and port. For every HTTP request, it immediately returns a HTTP 200 OK status with a JSON response indicating a successful health check.
frontend example-3
mode http
bind ${BIND_ADDRESS}:50055 name health
http-request return status 200 content-type application/json string "{\"status\":\"OK\"}"
apiVersion: config.haproxy.com/v1alpha1kind: Frontendmetadata:
name: example-3namespace: defaultlabels:
proxy.haproxy.com/instance: examplespec:
binds:
- address: '${BIND_ADDRESS}'name: healthport: 50055defaultBackend: {}httpRequest:
return:
content:
format: stringtype: application/jsonvalue: '{\"status\":\"OK\"}'status: 200mode: httpAPI Reference Frontend defines all the features that can be configured in an HAProxy frontend.
Backend refers to a set of servers that will receive the forwarded requests. The backend section defines how to reach the server, how to check its health, and how to balance the load among the servers. It can contain one or more servers, each server representing an application server in your infrastructure. With the HAProxy Operator, you can define the desired state for your backends in OpenShift, and the operator will ensure that the actual state matches the desired state.
Example 1:
This is a HAProxy backend configuration named 'example-1'. It operates in TCP mode and defines an Access Control List (ACL) for a specific source IP. It enables connection redispatching with a maximum of 3 retries per request. It rejects TCP requests not matching the ACL. It defines a server with specific health check settings, initial address resolution disabled, a specific check interval, and specified resolvers for hostname resolution.
backend example-1
mode tcp
acl whitelist src 0.0.0.0
option redispatch 3
tcp-request content reject if !whitelist
server web web.namespace.svc.cluster.local:443 check init-addr none inter 500 resolvers dns-namespace
apiVersion: config.haproxy.com/v1alpha1kind: Backendmetadata:
name: example-1namespace: defaultlabels:
proxy.haproxy.com/instance: examplespec:
acl:
- criterion: srcname: whitelistvalues:
- 0.0.0.0mode: tcpredispatch: trueservers:
- address: web.namespace.svc.cluster.localcheck:
enabled: trueinter: 500msinitAddr: nonename: webport: 443resolvers:
name: dns-namespacetcpRequest:
- action: rejectcondition: '!whitelist'conditionType: iftype: contentExample 2:
The HAProxy backend 'example-2' operates in HTTP mode. It has an Access Control List (ACL) named "whitelist" that matches when the source IP of the request is 0.0.0.0. It adds the X-Forwarded-For header to preserve the client's IP address, redistributes sessions in case of failure, and sets a health check timeout of 5 seconds. If a TCP request doesn't match the "whitelist" ACL, it's rejected. Various X-Forwarded-* and Forwarded headers are added to the HTTP request to convey information about the original request. A server named "web" is defined within this backend, with health checks enabled and an interval of 500 milliseconds between checks. The server's hostname resolution uses the "dns-namespace" resolvers. SSL/TLS configuration and certificate verification are also specified for this server and its weight is set to 256.
backend example-2
mode http
acl whitelist src 0.0.0.0
option forwardfor
option redispatch 3
timeout check 5000
tcp-request content reject if !whitelist
http-request add-header X-Forwarded-Host %[req.hdr(host)]
http-request add-header X-Forwarded-Port %[dst_port]
http-request add-header X-Forwarded-Proto http if !{ ssl_fc }
http-request add-header X-Forwarded-Proto-Version h2 if { ssl_fc_alpn -i h2 }
http-request add-header Forwarded for=%[src];host=%[req.hdr(host)];proto=%[req.hdr(X-Forwarded-Proto)]
cookie e76a2f0f39106e5e833f1323866171d4 attr SameSite=None httponly indirect nocache insert secure
server web web.namespace.svc.cluster.local:443 check ssl alpn http/1.1,h2 ca-file /usr/local/etc/haproxy/service-ca.crt cookie 4b24b04d486a91808d248592b93d2293 init-addr none inter 500 resolvers dns-namespace verify required verifyhost web.namespace.svc weight 256
apiVersion: config.haproxy.com/v1alpha1kind: Backendmetadata:
name: example-2namespace: defaultlabels:
proxy.haproxy.com/instance: examplespec:
mode: httpcookie:
attribute:
- SameSite=NonehttpOnly: trueindirect: truemode:
insert: trueprefix: falserewrite: falsename: appnoCache: truesecure: trueforwardFor:
enabled: truehttpRequest:
addHeader:
- name: X-Forwarded-Hostvalue:
str: '%[req.hdr(host)]'
- name: X-Forwarded-Portvalue:
str: '%[dst_port]'
- condition: '!{ ssl_fc }'conditionType: ifname: X-Forwarded-Protovalue:
str: http
- condition: '{ ssl_fc_alpn -i h2 }'conditionType: ifname: X-Forwarded-Proto-Versionvalue:
str: h2
- name: Forwardedvalue:
str: 'for=%[src];host=%[req.hdr(host)];proto=%[req.hdr(X-Forwarded-Proto)]'acl:
- criterion: srcname: whitelistvalues:
- 0.0.0.0redispatch: truetcpRequest:
- action: rejectcondition: '!whitelist'conditionType: iftype: contentservers:
- port: 443initAddr: noneverifyHost: web.namespace.svccookie: truecheck:
enabled: trueinter: 500msname: webssl:
alpn:
- http/1.1
- h2caCertificate:
name: service-ca.crtvalueFrom:
- configMapKeyRef:
key: service-ca.crtname: openshift-service-ca.crtenabled: trueverify: requiredresolvers:
name: dns-namespaceaddress: web.namespace.svc.cluster.localweight: 256timeouts:
check: 5sAPI Reference Backend defines all the features that can be configured in an HAProxy backend.