I have using TP-Link TL-WDR4300 router with LEDE software. Recently, thanks to fast-classifier and shortcut-fe modules the router got a second life 🙂 To my surprise after loading fast-classifier modules it can be able to pass 500Mb/s over NAT, which is absolutely great result 🙂
But after that I noticed that my site-to-site IPsec tunnel, based on Strongswan stopped working properly… Ping was working, tcp connections over tunnel could be established, but after passing some tcp packets the connection freezes. I suspected a problem with MTU, but that was not it.
After unload kmod-fast-classifier and kmod-shortcut-fe tunnel was working properly. I’m started to reading source code of fast-classifier and shortcut-fe modules. I found that:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
/* * Don't process packets that aren't being tracked by conntrack. */ ct = nf_ct_get(skb, &ctinfo); if (unlikely(!ct)) { fast_classifier_incr_exceptions(FAST_CL_EXCEPTION_NO_CT); DEBUG_TRACE("no conntrack connection, ignoring\n"); return NF_ACCEPT; } /* * Don't process untracked connections. */ if (unlikely(nf_ct_is_untracked(ct))) { fast_classifier_incr_exceptions(FAST_CL_EXCEPTION_CT_NO_TRACK); DEBUG_TRACE("untracked connection\n"); return NF_ACCEPT; } |
I realized that the offloading which is doing by fast-classifier and shortcut-fe is basing on conntrack table. The next thought was that conntrack is needed to realize NAT. Only connection between my LAN and internet (WAN) should be tracked and should be in conntrack table. I don’t need to track connection beetween my local nets connected through site-to-site vpn! Connections between my local nets can be realized only based on routing table. I decided to disable conntrack for my local nets and see if it solved my problem.
My network looks as follows:
|
1 |
(router with LEDE) [192.168.0.0/24 | WAN IP] <<--IPsec tunnel over internet-->> [WAN IP | 192.168.1.0/24] (remote network) |
Strongswan configuration: /etc/ipsec.conf (192.168.0.0/24)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
conn site_to_site_ipsec_tunnel dpdaction=clear dpddelay=20s authby=secret auto=add keyexchange=ikev2 fragmentation=yes left=%any leftid=%any leftsubnet=192.168.0.0/24 right=%any rightsubnet=192.168.1.0/24 rightid=%any |
Strongswan configuration: /etc/ipsec.conf (192.168.1.0/24)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
conn site_to_site_ipsec_tunnel authby=secret auto=start closeaction=restart dpddelay=20 dpdaction=restart keyexchange=ikev2 keyingtries=%forever fragmentation=yes left=%defaultroute leftsubnet=192.168.1.0/24 right=<IP address of my IPsec gateway> rightid=%any rightsubnet=192.168.0.0/24 |
File with secrets is the same on booth sides. /etc/ipsec.secrets
|
1 2 |
# /etc/ipsec.secrets - strongSwan IPsec secrets file %any %any : PSK "mypassword" |
Excluding particular connections from conntrack can be done by iptables with raw and conntrack module – I had to install them earlier.
|
1 2 |
opkg update opkg install iptables-mod-conntrack-extra kmod-ipt-conntrack kmod-ipt-conntrack-extra kmod-ipt-raw |
I excluded whole local network class – 192.168.0/0/12
|
1 |
iptables -t raw -A PREROUTING -s 192.168.0.0/12 -d 192.168.0.0/12 -j CT --notrack |
Now my IPsec tunnel started passing traffic properly! 🙂 And there is no connections from my localnets in conntrack! (so fast-classifier not offloading this connections).
The only issue was that I was unable to connect to LEDE router (192.168.0.1) from my remote network behind IPsec tunnel. I resolved that by exclude IP of router from iptables rule:
|
1 |
iptables -t raw -A PREROUTING -m iprange --src-range 192.168.0.2-192.168.1.254 --dst-range 192.168.0.2-192.168.1.254 -j CT --notrack |
I had to install iprange module before:
|
1 |
opkg install iptables-mod-iprange |
Now I can enjoy very fast internet connection with fast-classifier and fully working strongswan IPsec vpn connections 🙂
5 Comments
I have a similar setup, but I cannot benchmark more than 60 Mbit/s when I’m connected with IPsec. What results do you get? Otherwise I have the same speed as you.
Hi,
On what hardware are you running IPsec?
I have two different devices on both sides of my tunnel.
The first is an old router Tp-link TL-WDR4300 with weak Atheros AR9344 (560 MHz) CPU. On this device I can reach to about 25Mb/s through IPsec tunnel with default ciphers. After change hash algorithm from default sha256 to md5 (esp=aes128-md5!) I can reach about 30Mb/s on this device.
The second device is an mini computer with Intel NUC DCP847SKE board with dual core x86 cpu clocked 1,1GHz. On this device I can acheive up to 130Mb/s through IPsec tunnel with default ciphers.
For tests purpose you can establish tunnel with null encryption, for example esp=null-sha256! and check if you have problem with cpu power to encrypt more traffic, or is it another problem.
Thank you for your reply! Without any encryption I can reach 85 Mbps, so it’s slightly better. I think I will use a PC with pfSense instead, and use the router for port forwarding.
This is the device I have been using for IPsec (with pfSense on the other side).
Gonna keep an eye on your blog! 🙂
i can’t seem to make this work.
I use strongswan ipsec for a certificate based vpn between my mobile devices (iOS + MacOS).
i had a working setup of ar71xx 4.9 kernel + Shortcut Forwarding Engine driver.
i switched to the new ath79 (device tree) + flow offloading and everything works as expected except for the vpn.
logs did not help out as the connection is simply deleted…
Hi,
I haven’t tried yet to setup this with with ath79, I’m still using ar71xx 4.9 kernel + Shortcut Forwarding Engine driver on my TP-Link.
On Lede forum there is a thread about software flow offloading added to kernel 4.14 netfilter-flow-offload-hw-nat and I can see that people complains about the problems with working together – offloading and IPsec. For example: netfilter-flow-offload-hw-nat/10237/130
Regards