Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Uses docker network instead of real network for Whitelist #134

Open
eirsik opened this issue Apr 28, 2024 · 8 comments
Open

[BUG] Uses docker network instead of real network for Whitelist #134

eirsik opened this issue Apr 28, 2024 · 8 comments
Labels
bug Something isn't working

Comments

@eirsik
Copy link

eirsik commented Apr 28, 2024

Describe the bug
When using whitelist to allow only spesific local networks, zoraxy seems to interpet the traffic coming from the docker swarm network instead of the actual local network.

Setting up zoraxy in docker swarm and enabling whitelist for client subnet (192.168..) does not work as Zoraxy sees the traffic as if its coming from docker network (10.0..). So I have to use 10.0.. in whitelisting to allow local clients to access, but this allows everyone on the local network to access the resource and not just one specific local network.
https://img.sikkylab.me/image/msedge-casfdrbxzk.vOJo

Expected behavior
Zoraxy should see that the requests comes from the actual client. X-Real-IP works fine and the real visitor IP is forwarded to the en resource as seen here: https://img.sikkylab.me/image/msedge-6nitrrqey4.vFxN

So as the end resource can see the real IP from the client, zoraxy should too.

Blocking for public access works just fine as expected. It is just the local ACL that's not working as expected.

@eirsik eirsik added the bug Something isn't working label Apr 28, 2024
@tobychui
Copy link
Owner

I don't personally use Docker swamp so I am not quite sure about your issue here. But Zoraxy whitelist check with priority X-Real-IP > X-Forward-For > Remote Address, and you didn't include the client request (the header the client sent to Zoraxy) so I can't tell which field Zoraxy is used to determine your client IP.

And for the out-going request (Zoraxy -> your web server), this is how the X-Forward-For header is being added.

func addXForwardedForHeader(req *http.Request) {
	if clientIP, _, err := net.SplitHostPort(req.RemoteAddr); err == nil {
		// If we aren't the first proxy retain prior
		// X-Forwarded-For information as a comma+space
		// separated list and fold multiple headers into one.
		if prior, ok := req.Header["X-Forwarded-For"]; ok {
			clientIP = strings.Join(prior, ", ") + ", " + clientIP
		}
		req.Header.Set("X-Forwarded-For", clientIP)
		if req.TLS != nil {
			req.Header.Set("X-Forwarded-Proto", "https")
		} else {
			req.Header.Set("X-Forwarded-Proto", "http")
		}

		if req.Header.Get("X-Real-Ip") == "" {
			//Check if CF-Connecting-IP header exists
			CF_Connecting_IP := req.Header.Get("CF-Connecting-IP")
			if CF_Connecting_IP != "" {
				//Use CF Connecting IP
				req.Header.Set("X-Real-Ip", CF_Connecting_IP)
			} else {
				// Not exists. Fill it in with first entry in X-Forwarded-For
				ips := strings.Split(clientIP, ",")
				if len(ips) > 0 {
					req.Header.Set("X-Real-Ip", strings.TrimSpace(ips[0]))
				}
			}

		}
	}
}

It means that if the X-Forward-For is not empty, you will see something like {original_ip},{forwarder_ip} reaching your server. That way, in your screenshot 2, X-Forwarded-For should be 192.168.50.188,10.0.x.x instead of 192.168.50.188.

So given you two screenshots, I guess the problem should be either the docker container is not forwarding the request origin ip in the correct header field or Zoraxy ip extraction logic in private IP range have some small issues. I guess you might need to provide more info regarding your request headers (before it reaches zoraxy and after it passes through) in order for me to check what might be the issue here.

Btw, this PHP script might help :)
debug.zip

@eirsik
Copy link
Author

eirsik commented Apr 28, 2024

Hello and thank you for quick reply.

Interesting enough after flushing some cache and the DNS cache on my DNS server (adguard) the X-Real-IP now shows the docker network IP instead.

Here is an output of your debug script.

REQUEST HEADERS
Content-Type:
Content-Length: 0
X-Real-Ip: 10.0.0.8
X-Forwarded-Server: zoraxy-358cc7ae-ae24-45c1-b830-4ce26f51e9f1
X-Forwarded-Proto: https
X-Forwarded-Host: test-web.sikkylab.cloud
X-Forwarded-For: 10.0.0.8
Upgrade-Insecure-Requests: 1
Sec-Fetch-User: ?1
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-Dest: document
Sec-Ch-Ua-Platform: "Windows"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua: "Chromium";v="124", "Microsoft Edge";v="124", "Not-A.Brand";v="99"
Priority: u=0, i
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0
Host: test-web.sikkylab.cloud
Accept-Language: no,en-GB;q=0.9,en;q=0.8,en-US;q=0.7,pt;q=0.6,fr;q=0.5,de;q=0.4
Accept-Encoding: gzip, deflate, br, zstd
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Pragma: no-cache
Cache-Control: no-cache
APACHE VARIABLES
HTTP HEADERS
HTTP_ACCEPT : text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
HTTP_COOKIE :
HTTP_FORWARDED :
HTTP_HOST : test-web.sikkylab.cloud
HTTP_PROXY_CONNECTION :
HTTP_REFERER :
HTTP_USER_AGENT : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0
CONNECTION & REQUEST
AUTH_TYPE :
CONN_REMOTE_ADDR :
CONTEXT_PREFIX :
CONTEXT_DOCUMENT_ROOT :
IPV6 :
PATH_INFO :
QUERY_STRING :
REMOTE_ADDR : 192.168.10.11
REMOTE_HOST : 192.168.10.11
REMOTE_IDENT :
REMOTE_PORT : 34396
REMOTE_USER :
REQUEST_METHOD : GET
SCRIPT_FILENAME : C:\inetpub\wwwroot\test-web.sikkylab.cloud\zoraxy\debug.php
SERVER INTERNALS
DOCUMENT_ROOT : C:\inetpub\wwwroot\test-web.sikkylab.cloud
SCRIPT_GROUP :
SCRIPT_USER :
SERVER_ADDR :
SERVER_ADMIN :
SERVER_NAME : test-web.sikkylab.cloud
SERVER_PORT : 80
SERVER_PROTOCOL : HTTP/1.1
SERVER_SOFTWARE : Microsoft-IIS/10.0
DATE & TIME
TIME_YEAR :
TIME_MON :
TIME_DAY :
TIME_HOUR :
TIME_MIN :
TIME_SEC :
TIME_WDAY :
TIME :
SPECIALS
API_VERSION :
CONN_REMOTE_ADDR :
HTTPS : off
IS_SUBREQ :
REMOTE_ADDR : 192.168.10.11
REQUEST_FILENAME :
REQUEST_SCHEME :
REQUEST_URI : /zoraxy/debug.php
THE_REQUEST :

@eirsik
Copy link
Author

eirsik commented Apr 28, 2024

After some digging is seems it is because of the overlay network in docker swarm.
And it can be fixed by having zoraxy using the host network instead of the overlay network. But that's last resort as it kinda defeats the purpose of swarm.

But it can also be solved by using the "send-proxy" directive in haproxy which basically forwards zoraxy the headers to use but it requires zoraxy to trust reverse proxy headers from haproxy which it seams it does not do at the moment, as when i enable send-proxy in haproxy Zoraxy refuses the traffic.

@eirsik
Copy link
Author

eirsik commented Apr 28, 2024

Setting zoraxy to use host network is working.
As you can see X-Forwarded-For keeps both client IP and the haproxy IP (which is a HA for the swarm).

It is not an elegant solution, but it is an workaround. I really liked having zoraxy inside the ovarlay network as it is intended to with swarm, but then it requires Zoraxy to trust proxy headers from Haproxy.

REQUEST HEADERS
Content-Type:
Content-Length: 0
X-Real-Ip: 192.168.50.188
X-Forwarded-Server: zoraxy-358cc7ae-ae24-45c1-b830-4ce26f51e9f1
X-Forwarded-Proto: https
X-Forwarded-Host: test-web.sikkylab.cloud
X-Forwarded-For: 192.168.50.188, 192.168.10.63
Upgrade-Insecure-Requests: 1
Sec-Fetch-User: ?1
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-Dest: document
Sec-Ch-Ua-Platform: "Windows"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua: "Chromium";v="124", "Microsoft Edge";v="124", "Not-A.Brand";v="99"
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0
Host: test-web.sikkylab.cloud
Accept-Language: no,en-GB;q=0.9,en;q=0.8,en-US;q=0.7,pt;q=0.6,fr;q=0.5,de;q=0.4
Accept-Encoding: gzip, deflate, br, zstd
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Connection: close
Cache-Control: max-age=0
APACHE VARIABLES
HTTP HEADERS
HTTP_ACCEPT : text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
HTTP_COOKIE :
HTTP_FORWARDED :
HTTP_HOST : test-web.sikkylab.cloud
HTTP_PROXY_CONNECTION :
HTTP_REFERER :
HTTP_USER_AGENT : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/124.0.0.0
CONNECTION & REQUEST
AUTH_TYPE :
CONN_REMOTE_ADDR :
CONTEXT_PREFIX :
CONTEXT_DOCUMENT_ROOT :
IPV6 :
PATH_INFO :
QUERY_STRING :
REMOTE_ADDR : 192.168.10.13
REMOTE_HOST : 192.168.10.13
REMOTE_IDENT :
REMOTE_PORT : 55338
REMOTE_USER :
REQUEST_METHOD : GET
SCRIPT_FILENAME : C:\inetpub\wwwroot\test-web.sikkylab.cloud\zoraxy\debug.php
SERVER INTERNALS
DOCUMENT_ROOT : C:\inetpub\wwwroot\test-web.sikkylab.cloud
SCRIPT_GROUP :
SCRIPT_USER :
SERVER_ADDR :
SERVER_ADMIN :
SERVER_NAME : test-web.sikkylab.cloud
SERVER_PORT : 80
SERVER_PROTOCOL : HTTP/1.1
SERVER_SOFTWARE : Microsoft-IIS/10.0
DATE & TIME
TIME_YEAR :
TIME_MON :
TIME_DAY :
TIME_HOUR :
TIME_MIN :
TIME_SEC :
TIME_WDAY :
TIME :
SPECIALS
API_VERSION :
CONN_REMOTE_ADDR :
HTTPS : off
IS_SUBREQ :
REMOTE_ADDR : 192.168.10.13
REQUEST_FILENAME :
REQUEST_SCHEME :
REQUEST_URI : /zoraxy/debug.php
THE_REQUEST :

@tobychui
Copy link
Owner

tobychui commented Apr 29, 2024

@eirsik I am not sure what do you mean by

requires Zoraxy to trust proxy headers from Haproxy.

Given that in your first debug.php output, we can see the followings.

X-Real-Ip: 10.0.0.8
...
X-Forwarded-For: 10.0.0.8

Then it is expected behavior for Zoraxy to use 10.0.0.8 as the access checking ip address. To fix your issue, either

  1. Use another proxy outside of the container / swamp to unset the X-Real-IP and append the remote address into X-Forward-For according to HTTP standards (i.e. 192.168.10.13, 10.0.0.8)
  2. Figure out a way to make docker swarm set the correct request origin IP address in the X-Real-IP field.
  3. Just use Zoraxy in host network

@tobychui
Copy link
Owner

tobychui commented May 3, 2024

@eirsik any updates regarding this bug report? Or can you validate this is actually a bug from Zoraxy?
If no, I will close this issue after next release.

@eirsik
Copy link
Author

eirsik commented May 3, 2024

Hello

What I mean is that HAProxy can use the PROXY protocol to keep the origin IP regardless of the network setup.
https://docs.haproxy.org/2.5/configuration.html#5.2-send-proxy

But it requires that the receiver accepts and supports it. Which it seems that Zoraxy does not.

I really want to use Zoraxy as it looks really cool and I love everything about it, but I have reverted back to Nginx where everything works by just adding one option to the configuration: https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/

Once Zoraxy supports the Proxy protocol, I will return :)

@tobychui
Copy link
Owner

tobychui commented May 4, 2024

@eirsik Ok, so it sounds like you are reporting a bug from a feature that Zoray not support. I guess it would be better to change this to an enhancement (feature) request for PROXY PROTOCOL support instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants