[Vpn-help] Linux communications issues ...

[Matthew Grooms]

All,

I have been troubleshooting an issue on Linux which has been reported by 
several people. The problem was difficult to identify since the client 
can establish a connection and negotiate IPSec SAs, but return traffic 
never makes it to the userland applications. For example, ping displays 
the following stalled output ...

mgrooms at ubuntu8-64:~$ ping 10.1.2.100
PING 10.1.2.100 (10.1.2.100) 56(84) bytes of data.

... even though you can see response packets using tcpdump ...

mgrooms at ubuntu8-64:~$ sudo tcpdump -n icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
12:46:45.102547 IP 10.1.2.100 > 10.2.20.5: ICMP echo reply, id 32537, 
seq 1, length 64
12:46:46.102905 IP 10.1.2.100 > 10.2.20.5: ICMP echo reply, id 32537, 
seq 2, length 64

After a lot of tinkering, I was able to determine that the following 
sysctl setting will cause the kernel to drop packets received on one 
interface when the destination address is owned by another interface.

net.ipv4.conf.all.rp_filter

The documentation states the following ...

rp_filter - INTEGER
         2 - do source validation by reversed path, as specified in RFC1812
             Recommended option for single homed hosts and stub network
             routers. Could cause troubles for complicated (not loop free)
             networks running a slow unreliable protocol (sort of RIP),
             or using static routes.

         1 - (DEFAULT) Weaker form of RP filtering: drop all the packets
             that look as sourced at a directly connected interface, but
             were input from another interface.

         0 - No source validation.

For the client to work properly, this option value must be set to 0. 
Making the value stick on some systems can be a bit challenging. For 
example, the 8.10 Ubuntu host I used for testing has the following 
references to this sysctl option under etc ...

mgrooms at ubuntu8-64:/etc$ grep -r rp_filter *
sysctl.conf:#net.ipv4.conf.default.rp_filter=1
sysctl.conf:#net.ipv4.conf.all.rp_filter=1
sysctl.d/10-network-security.conf:net.ipv4.conf.default.rp_filter=1
sysctl.d/10-network-security.conf:net.ipv4.conf.all.rp_filter=1
ufw/sysctl.conf:net/ipv4/conf/all/rp_filter=1
ufw/sysctl.conf:net/ipv4/conf/default/rp_filter=1

I'm not sure which one should be set under what circumstances, but I was 
able to comment out all but the 10-network-security.conf line and set it 
to 0. After rebooting the host, the sysctl values looked like this ...

mgrooms at ubuntu8-64:/etc$ sysctl -a | grep rp_filter | grep -v arp
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.eth0.rp_filter = 0
net.ipv4.conf.pan0.rp_filter = 0
net.ipv4.conf.tap0.rp_filter = 0

And finally, everything works as expected using the VPN client ...

mgrooms at ubuntu8-64:/etc$ ping 10.1.2.100
PING 10.1.2.100 (10.1.2.100) 56(84) bytes of data.
64 bytes from 10.1.2.100: icmp_seq=1 ttl=64 time=1.55 ms
64 bytes from 10.1.2.100: icmp_seq=2 ttl=64 time=0.873 ms

Hope this helps,

-Matthew