Recently I started interesting in Go language. First impressions of programming in Go are very good. In short brief, I like simplicity of this language, that you can not complicate the code too much. Goroutines and channels looks promising as solution for concurrency, and it seems simple to use. Static compiled binaries are easy to deploy. Performance is good.
I would like to share description and simple implementation in Go of fully transparent reverse or forward http proxy.
Go standard libraries – net/http and net/http/httputil provides everything needed to implement it.
Below the simplest implementation of http proxy:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
package main import ( "log" "net/http" "net/http/httputil" "net/url" ) func main() { log.Printf("Starting...") http.HandleFunc("/", ProxyFunc) log.Fatal(http.ListenAndServe(":8888", nil)) } func ProxyFunc(w http.ResponseWriter, r *http.Request) { u, _ := url.Parse(r.URL.Scheme + "://" + r.URL.Host) proxy := httputil.NewSingleHostReverseProxy(u) proxy.ServeHTTP(w, r) } |
Now, we can just set http proxy in browser and it will be working.
Transparent http proxy
But what if we want to to setup fully transparent proxy? For example: when we don’t want to configure manually browser on clients, but all outgoing http traffic should be pass by proxy for some reasons – logging, caching, make security scan for viruses etc. In this scenario transparent proxy is located between the client and the internet. Another use case of transparent http proxy is to set up it inline in communication between services in data center and performing some operations like traffic filtering, checking authorization etc.
System Configuration (routing table, tproxy)
I will describing scenario where http transparent proxy is acting on router which is the default gateway for my local network.
My network looks as follows:
|
1 2 3 4 5 6 7 8 9 |
+------------------------------------------+ +--------------------------------------------+ | | | | | +-----------------+---+----------------+ | | Local network | Router, gateway on Linux | Wan Network | | 192.168.1.1/24 -->> | http proxy in Go | 37.247.61.7 -->> | | eth1 | | eth0 | | +-----------------+---+----------------+ | | | | | +------------------------------------------+ +--------------------------------------------+ |
Tproxy will be use to redirect traffic.
Tproxy allows as to redirect traffic designated to remote location to the local process.
How tproxy works in details is described here:
https://www.kernel.org/doc/Documentation/networking/tproxy.txt
https://powerdns.org/tproxydoc/tproxy.md.html
https://people.netfilter.org/hidden/nfws/nfws-2008-tproxy_slides.pdf
Configuration of routing table and tproxy:
|
1 2 3 4 5 6 7 8 |
# create new routing table and tell that 0.0.0.0/0 addresses range is a local addresses... ip route add local 0.0.0.0/0 dev lo table 100 # redirect marked packets to table created above ip rule add fwmark 1 lookup 100 # mark packets with dst port = 80 (and use route table 100) nad redirect to Go http proxy listening on 127.0.0.1:8888 iptables -t mangle -A PREROUTING -s 192.166.1.0/24 -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 8888 --on-ip 127.0.0.1 |
Http proxy in Go
I had to wait for Go 1.11 to be able to create custom socket with IP_TRANSPARENT param. From Go 1.11 there is possible to pass socket option before start listening or dialing. ListenConfig provide this.
https://go-review.googlesource.com/c/go/+/72810
https://golang.org/pkg/net/#ListenConfig
The key in implementation is to create custom listener for http.Serve and use LocalAddrContextKey to get destinetion address to which client want to connect. In fact address:port values from http.LocalAddrContextKey, are the values from local socket dynamicly created by tproxy.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
package main import ( "context" "fmt" "log" "net" "net/http" "net/http/httputil" "syscall" ) //SetSocketOptions functions sets IP_TRANSPARENT flag on given socket (c syscall.RawConn) func SetSocketOptions(network string, address string, c syscall.RawConn) error { var fn = func(s uintptr) { var setErr error var getErr error setErr = syscall.SetsockoptInt(int(s), syscall.SOL_IP, syscall.IP_TRANSPARENT, 1) if setErr != nil { log.Fatal(setErr) } val, getErr := syscall.GetsockoptInt(int(s), syscall.SOL_IP, syscall.IP_TRANSPARENT) if getErr != nil { log.Fatal(getErr) } log.Printf("value of IP_TRANSPARENT option is: %d", int(val)) } if err := c.Control(fn); err != nil { return err } return nil } func main() { http.HandleFunc("/", TransparentHttpProxy) // here we are creating custom listener with transparent socket, possible with Go 1.11+ lc := net.ListenConfig{Control: SetSocketOptions} listener, _ := lc.Listen(context.Background(), "tcp", ":8888") log.Printf("Starting http proxy") log.Fatal(http.Serve(listener, nil)) } func TransparentHttpProxy(w http.ResponseWriter, r *http.Request) { director := func(target *http.Request) { target.URL.Scheme = "http" target.URL.Path = r.URL.Path target.Header.Set("Pass-Via-Go-Proxy", "1") /* Line below of this comment this is the quite tricky part of the configuration, necessary to make transparent proxy working. From http.LocalAddrContextKey we can get address:port destination of client requst. In fact address:port values from http.LocalAddrContextKey, are the values from socket dynamicly created by tproxy. This will be used to create a connection between the proxy and the destination, to which the client request will be pass. */ target.URL.Host = fmt.Sprint(r.Context().Value(http.LocalAddrContextKey)) } proxy := &httputil.ReverseProxy{Director: director} proxy.ServeHTTP(w, r) } |
Starting proxy:
|
1 2 3 |
router:/golang_proxy # go run proxy.go 2018/09/24 21:23:15 value of IP_TRANSPARENT option is: 1 2018/09/24 21:23:15 Starting http proxy |
Client from local network (192.168.1.6) is connecting to remote site on port 217.73.181.197:80. This connection is handled through the proxy.
Nestat is showing one very interesting thing:
|
1 2 3 |
router:~ # netstat -tpna | grep proxy | grep ESTABLISHED tcp 0 0 37.247.61.7:57554 217.73.181.197:80 ESTABLISHED 12777/proxy tcp 0 0 217.73.181.197:80 192.168.1.6:59752 ESTABLISHED 12777/proxy |
Tproxy created tcp socket with remote site address (217.73.181.197:80) on my local machine. My router has only 192.168.1.1 and 37.247.61.7 addresses, routing table 100 does the job.
Go http proxy after receive request from client (192.168.1.6), made a connection to exactly the same address:port as it received. MAGIC! 🙂
3 Comments
hi, I just wonder, what “-j TPROXY –on-port 8888” does in the following rule:
# mark packets with dst port = 80 (and use route table 100) nad redirect to Go http proxy listening on 127.0.0.1:8888
iptables -t mangle -A PREROUTING -s 192.166.1.0/24 -p tcp –dport 80 -j TPROXY –tproxy-mark 0x1/0x1 –on-port 8888 –on-ip 127.0.0.1
Does it tell linux kernel to deriver match package to local process(:8888)? If so, since local process get the package already, why need to change route tables by:
ip route add local 0.0.0.0/0 dev lo table 100
ip rule add fwmark 1 lookup 100
hi, I just wonder, what “-j TPROXY –on-port 8888” does in the following rule:
# mark packets with dst port = 80 (and use route table 100) nad redirect to Go http proxy listening on 127.0.0.1:8888
iptables -t mangle -A PREROUTING -s 192.166.1.0/24 -p tcp –dport 80 -j TPROXY –tproxy-mark 0x1/0x1 –on-port 8888 –on-ip 127.0.0.1
Does it tell linux kernel to deriver match package to local process(:8888)? If so, since local process get the package already, why need to change route tables by:
ip route add local 0.0.0.0/0 dev lo table 100
ip rule add fwmark 1 lookup 100
Hi 🙂 sorry for late response.
From what I remember additional route table is needed to catch all traffic with any DST addresses which are not match to any assigned address on router.
Good explanation is here: https://powerdns.org/tproxydoc/tproxy.md.html
Regards