Pass docker container network through host iptables tproxy
by xeptore from LinuxQuestions.org on (#6JWXW)
I have a TPROXY-supported proxy server running on my machine (listening on 127.0.0.1:8080). I configured iptables to redirect traffic to this proxy server using `TPROXY` target as below:
Code:ip route add local default dev lo table 100
ip rule add fwmark 1 table 100
iptables -t mangle -N PROXY
# Ignore private IPs
iptables -t mangle -A PROXY -d 10.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY -d 100.64.0.0/10 -j RETURN
iptables -t mangle -A PROXY -d 127.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY -d 169.254.0.0/16 -j RETURN
iptables -t mangle -A PROXY -d 172.16.0.0/12 -j RETURN
iptables -t mangle -A PROXY -d 192.0.0.0/24 -j RETURN
iptables -t mangle -A PROXY -d 224.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY -d 240.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY -d 255.255.255.255/32 -j RETURN
iptables -t mangle -A PROXY -d 192.168.0.0/16 -j RETURN
iptables -t mangle -A PROXY -p tcp -j TPROXY --on-port 8080 --on-ip 127.0.0.1 --tproxy-mark 1
iptables -t mangle -A PROXY -p udp -j TPROXY --on-port 8080 --on-ip 127.0.0.1 --tproxy-mark 1
iptables -t mangle -A PREROUTING -j PROXY
iptables -t mangle -N PROXY_SELF
# Ignore private IPs
iptables -t mangle -A PROXY_SELF -d 10.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY_SELF -d 100.64.0.0/10 -j RETURN
iptables -t mangle -A PROXY_SELF -d 127.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY_SELF -d 169.254.0.0/16 -j RETURN
iptables -t mangle -A PROXY_SELF -d 172.16.0.0/12 -j RETURN
iptables -t mangle -A PROXY_SELF -d 192.0.0.0/24 -j RETURN
iptables -t mangle -A PROXY_SELF -d 224.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY_SELF -d 240.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY_SELF -d 255.255.255.255/32 -j RETURN
iptables -t mangle -A PROXY_SELF -d 192.168.0.0/16 -j RETURN
iptables -t mangle -A PROXY_SELF -m mark --mark 2 -j RETURN
iptables -t mangle -A PROXY_SELF -p tcp -j MARK --set-mark 1
iptables -t mangle -A PROXY_SELF -p udp -j MARK --set-mark 1
iptables -t mangle -A OUTPUT -j PROXY_SELFIt works as expected for locally-generated network. However, there's a problem with Docker container networks. As I didn't change any of its default network/iptables configuration, it uses internet-connected network device (`eth0`) as gateway to pass outgoing networks from. When I enable iptables tproxy to redirect traffic to the proxy server, all traffic originated from my machine is properly passed through the proxy server, but the traffic originated from inside Docker container does not reach the proxy server, and fails to access the internet. How can I resolve it?
I also used `LOG` iptables target for every chain in every table to see how packets traverse through, and to debug (or guess) what the issue is. From what I understood, for locally-generated packets as they pass through `mangle` table `OUTPUT` chain, they are marked with fwmark 1, then routed to `lo` device (instead of the default internet-connected device `eth0`) because of the `ip route`, and `ip rule` commands, and they will then received on `lo` device, which get redirected to the proxy server by `tproxy` target defined in `mangle` table's `PREROUTING` chain, and the rest of the proxying process. On the other hand, for packets sent from Docker container, they are first received on `docker0` (default Docker bridge network device), and once they are redirected by `tproxy` target, the packet [I]hangs_/I] in `nat` table `INPUT` chain, which AFAIK is just before they are handed over to the proxy server process. As I don't see any log message from proxy server, I guess these packets are being dropped (?) somehow. I tested the followings as well:
- Running the proxy server bounded to `0.0.0.0`
- Enabling `net.ipv4.ip_forward` kernel option
- Letting `docker0` packets to pass through the `mangle` table `PREROUTING` chain first by just marking them with fwmark 1 (without jumping to `tproxy`), and let `ip route` to re-route them back to `lo` (similar to what happens for locally-generated packets), and then apply tproxy redirection for the associated packet that is received later on on the `lo` device. This works as expected for the forwared packet, but the corresponding packet that is received on `lo` device hangs right after the `mangle` table's `PREROUTING` chain and does no proceed.
But with no results...
Any help is appreciated.
As I'm new to iptables advanced networking on Linux (started completely since 3 days ago), please also let me know if there is any more information that needs to be provided.
Cheers
Code:ip route add local default dev lo table 100
ip rule add fwmark 1 table 100
iptables -t mangle -N PROXY
# Ignore private IPs
iptables -t mangle -A PROXY -d 10.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY -d 100.64.0.0/10 -j RETURN
iptables -t mangle -A PROXY -d 127.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY -d 169.254.0.0/16 -j RETURN
iptables -t mangle -A PROXY -d 172.16.0.0/12 -j RETURN
iptables -t mangle -A PROXY -d 192.0.0.0/24 -j RETURN
iptables -t mangle -A PROXY -d 224.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY -d 240.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY -d 255.255.255.255/32 -j RETURN
iptables -t mangle -A PROXY -d 192.168.0.0/16 -j RETURN
iptables -t mangle -A PROXY -p tcp -j TPROXY --on-port 8080 --on-ip 127.0.0.1 --tproxy-mark 1
iptables -t mangle -A PROXY -p udp -j TPROXY --on-port 8080 --on-ip 127.0.0.1 --tproxy-mark 1
iptables -t mangle -A PREROUTING -j PROXY
iptables -t mangle -N PROXY_SELF
# Ignore private IPs
iptables -t mangle -A PROXY_SELF -d 10.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY_SELF -d 100.64.0.0/10 -j RETURN
iptables -t mangle -A PROXY_SELF -d 127.0.0.0/8 -j RETURN
iptables -t mangle -A PROXY_SELF -d 169.254.0.0/16 -j RETURN
iptables -t mangle -A PROXY_SELF -d 172.16.0.0/12 -j RETURN
iptables -t mangle -A PROXY_SELF -d 192.0.0.0/24 -j RETURN
iptables -t mangle -A PROXY_SELF -d 224.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY_SELF -d 240.0.0.0/4 -j RETURN
iptables -t mangle -A PROXY_SELF -d 255.255.255.255/32 -j RETURN
iptables -t mangle -A PROXY_SELF -d 192.168.0.0/16 -j RETURN
iptables -t mangle -A PROXY_SELF -m mark --mark 2 -j RETURN
iptables -t mangle -A PROXY_SELF -p tcp -j MARK --set-mark 1
iptables -t mangle -A PROXY_SELF -p udp -j MARK --set-mark 1
iptables -t mangle -A OUTPUT -j PROXY_SELFIt works as expected for locally-generated network. However, there's a problem with Docker container networks. As I didn't change any of its default network/iptables configuration, it uses internet-connected network device (`eth0`) as gateway to pass outgoing networks from. When I enable iptables tproxy to redirect traffic to the proxy server, all traffic originated from my machine is properly passed through the proxy server, but the traffic originated from inside Docker container does not reach the proxy server, and fails to access the internet. How can I resolve it?
I also used `LOG` iptables target for every chain in every table to see how packets traverse through, and to debug (or guess) what the issue is. From what I understood, for locally-generated packets as they pass through `mangle` table `OUTPUT` chain, they are marked with fwmark 1, then routed to `lo` device (instead of the default internet-connected device `eth0`) because of the `ip route`, and `ip rule` commands, and they will then received on `lo` device, which get redirected to the proxy server by `tproxy` target defined in `mangle` table's `PREROUTING` chain, and the rest of the proxying process. On the other hand, for packets sent from Docker container, they are first received on `docker0` (default Docker bridge network device), and once they are redirected by `tproxy` target, the packet [I]hangs_/I] in `nat` table `INPUT` chain, which AFAIK is just before they are handed over to the proxy server process. As I don't see any log message from proxy server, I guess these packets are being dropped (?) somehow. I tested the followings as well:
- Running the proxy server bounded to `0.0.0.0`
- Enabling `net.ipv4.ip_forward` kernel option
- Letting `docker0` packets to pass through the `mangle` table `PREROUTING` chain first by just marking them with fwmark 1 (without jumping to `tproxy`), and let `ip route` to re-route them back to `lo` (similar to what happens for locally-generated packets), and then apply tproxy redirection for the associated packet that is received later on on the `lo` device. This works as expected for the forwared packet, but the corresponding packet that is received on `lo` device hangs right after the `mangle` table's `PREROUTING` chain and does no proceed.
But with no results...
Any help is appreciated.
As I'm new to iptables advanced networking on Linux (started completely since 3 days ago), please also let me know if there is any more information that needs to be provided.
Cheers