Configure HAProxy transparent proxy on OpenWrt 22.03

Hello the community,

I want to configure transparent proxy on HAProxy (which is installed on my router) in order to forward HTTP packet to my web server with the public IP address and not the internal private address.

I does not seems to work:
HAProxy return the message curl -vv -4 website.com:8888

503 Service Unavailable
No server is available to handle this request

Below the configuration.
If I remove the following lines of my haproxy config file the web page behind is accessible but not with the real IP address.

        option transparent
        source 0.0.0.0 usesrc clientip
  • Installed kernel modules for nft:
opkg install kmod-nft-tproxy
opkg install kmod-nft-socket
  • Validate the following sysctl variable are setted:
sysctl net.ipv4.ip_nonlocal_bind
sysctl net.ipv4.conf.all.forwarding
sysctl net.ipv4.ip_forward

sysctl -w net.ipv4.ip_nonlocal_bind=1
sysctl -w net.ipv4.conf.all.forwarding = 1
sysctl -w net.ipv4.ip_forward = 1
  • Configure the route:
ip rule add fwmark 1088 table 100
ip route add local 0.0.0.0/0 dev lo table 100
  • Configure the firewall
    I have a luci firewall rule to authorize TCP from wan on port 8888 on this device:
config rule
        option name 'http8080'
        list proto 'tcp'
        option src 'wan'
        option target 'ACCEPT'
        option dest_port '8888'

I tried to load that rule with nft -f /etc/nftables.d/10-custom-filter-chains.nft:

table inet mangle {
    set byp4 {
        typeof ip daddr
        flags interval
        elements = { 0.0.0.0/8, 10.0.0.0/8,
                 127.0.0.0/8, 169.254.0.0/16,
                 172.16.0.0/12, 192.0.0.0/24,
                 192.0.2.0/24, 192.88.99.0/24,
                 192.168.0.0/16, 198.18.0.0/15,
                 198.51.100.0/24, 203.0.113.0/24,
                 224.0.0.0/4, 240.0.0.0-255.255.255.255 }
    }

    set byp6 {
        typeof ip6 daddr
        flags interval
        elements = { ::,
                 ::1,
                 ::ffff:0:0:0/96,
                 64:ff9b::/96,
                 100::/64,
                 2001::/32,
                 2001:20::/28,
                 2001:db8::/32,
                 2002::/16,
                 fc00::/7,
                 fe80::/10,
                 ff00::-ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff }
    }

    chain prerouting {
        type filter hook prerouting priority mangle; policy accept;
        ip daddr @byp4 return
        ip6 daddr @byp6 return
        tcp dport 8888 tproxy to :1088 meta mark set 0x00000440 accept
        #tcp dport { 0-65535 } tproxy to :1088 meta mark set 0x00000440 accept
        #udp dport { 0-65535 } tproxy to :1088 meta mark set 0x00000440 accept
    }

    # Only for local mode
    chain output {
        type route hook output priority mangle; policy accept;
        ip daddr @byp4 return
        ip6 daddr @byp6 return
        tcp dport 8888 meta mark set 0x00000440
        #tcp dport { 0-65535 } meta mark set 0x00000440
        #udp dport { 0-65535 } meta mark set 0x00000440
    }
}
  • HAProxy configuration
global
        daemon
defaults
        timeout connect 10s
        timeout client 60s
        timeout server 60s
frontend http8888
        mode http
        bind :80  name frontend-http8888
        default_backend http8888

backend http8888
        mode http
        option transparent
        source 0.0.0.0 usesrc clientip
        server 192.168.0.18:8888

weird, I have a similar working configuration, only not using tproxy but just

  1. special routing table and rule to route marked traffic with:
ip rule add fwmark 1088 table haproxy
ip route add local 0.0.0.0/0 dev lo table haproxy

(once you have declared the haproxy route table in /etc/iproute2/rt_tables)

  1. haproxy
    source 0.0.00 usesrc clientip
  2. add marking rule to prerouting chain:
    nft insert rule inet fw4 prerouting meta l4proto tcp socket transparent 1 socket wildcard 0 meta mark set 0x00000001 accept comment \"!haproxy: transparent proxy mode \"

note : it does not even seem to require net.ipv4.ip_nonlocal_bind=1 which is kind of weird (I will need to double check this one)
load balancing also works fine in this mode .

Thank you very much @gbordier :slight_smile: , I've tried the following:

opkg install kmod-nft-socket
ip rule add fwmark 1088 table 100
ip route add local 0.0.0.0/0 dev lo table 100
nft insert rule inet fw4 prerouting meta l4proto tcp socket transparent 1 socket wildcard 0 meta mark set 0x00000440 accept comment \"!haproxy: transparent proxy mode \"

Editing and reload the haproxy.cfg

backend http
        mode http
        option forwardfor                      
        source 0.0.0.0 usesrc clientip                
        server NAS_HTTP         192.168.0.18:80

But without any success.

Actually it seems everything is wrong in what I want to do and how I do it it.
I will let go the subject.

One note , my config intent was to use TCP mode hence all this complex mess.
if you are using http mode then you usually just need the X-Forwarded-For header to be inserted for the backend http server to understand the source IP.
otherwise did you check your config with
ip rule list
ip route list table 100
ntft list rule inet fw4 prerouting

would you by chance have a lingering port forwarding rule in openwrt that would bypass your haproxy (it happened to me a few times)