[vpn-help] Packet loss when using 2.2.2 Windows x64 client on cable Internet connections

[Jim Harle]

Updating my own thread again...we tried the Shrew client on a different cable site today (Cox being the carrier), and it didn't have any of these problems.  It's using a similar cable modem to one of the Suddenlink sites, which we discovered today are using completely different modems (one Motorola and one Arris).

At this point we believe this issue is related to the carriers' configuration somehow.  We punted and put in Cisco 871 routers at both Suddenlink sites, and now they're solid.

From: Jim Harle [mailto:vpn at technicolor.com]
Sent: Monday, July 29, 2013 1:15 PM
To: vpn-help at lists.shrew.net
Subject: RE: Packet loss when using 2.2.2 Windows x64 client on cable Internet connections

I have some updated information for this problem.  I haven't received any responses for it, which is expected, as it's a hard one :|.  I'm re-pasting my "symptoms" section below, with the new information called out:

The Shrew Soft client works on both of these PCs, with some caveats:

-          Packet loss of up to 30% is introduced on the public connection (and likewise the tunnel) while the VPN tunnel is active.  This can be verified by using the "line quality" test at http://dslreports.com/pingtest.  When no tunnel is established, there is no packet loss.

-          The packet loss through the tunnel seems to degrade over time, as does the tunnel connectivity itself.  After an average of five hours, the VPN tunnel will establish and pass traffic, but only for about 30 seconds before the tunnel is dropped.  A reboot of the PC makes things "better" again (connection stays up, but with much packet loss).  [UPDATE] - when the Shrew client gets into this state, the connection drops after about 30 seconds with a "connection terminated by gateway" message.  If I stop/start the two Shrew daemon services, the VPN connection starts working again, albeit with the packet loss.

-          The VPN tunnel will only work with no NAT traversal (IP-to-IP ESP).  If we force the Shrew client to use NAT traversal, the tunnel will establish, but no traffic will pass through it (kinda like the Cisco client problem).  [UPDATE] - the Shrew client does pass traffic when the NAT traversal setting is 'force-rfc.'  Interestingly, we have other sites where the Cisco client will not pass traffic regardless of NAT-T setting, and the Shrew client will only pass traffic using this 'force-rfc' NAT-T.

We are installing Cisco 871 VPN routers at these two sites today and tomorrow, so this problem may become academic for now.  Still, I can reproduce the problem at my house if anyone has any ideas.  Installing 871s throws off our cost-model for this project, so we only want to use them when absolutely necessary, and I'm sure more problematic Cable sites will turn up.

From: Jim Harle
Sent: Friday, July 26, 2013 4:47 PM
To: 'vpn-help at lists.shrew.net'
Subject: Packet loss when using 2.2.2 Windows x64 client on cable Internet connections

Greetings, this is my first post to this list.  It is quite long, so if you have no interest in reading the context, you can skip to the last sentence at the end.

We are in the midst of a project involving Windows 7 x64 PCs which are "directly" connected to the Internet (public IP resides on a NIC in the PC), as opposed to behind a NAT device/hardware firewall as is typical.  These Windows PCs are using the Cisco VPN client (IPsec with NAT traversal, split-tunneled) to connect to a Cisco ASA gateway in our datacenter.  This ASA terminates many hundred VPN tunnels, mostly from Cisco 871 routers.  The Internet connections for the PCs are mixture of "commercial grade" DSL or cable (mostly DSL)...using various carriers.

We've had intermittent issues with the Cisco client, where it will establish the VPN tunnel, but not pass private traffic through the tunnel.  This is nearly always cleared up by power-cycling the DSL modem.  We have two chronic sites in Texas, both using Suddenlink cable Internet, which are having the Cisco-connects-but-doesn't-pass traffic problem.  However, power-cycling the cable modem at these sites doesn't always fix it.  So, we decided to try the Shrew Soft 2.2.2 client on these two PCs.

The Shrew Soft client works on both of these PCs, with some caveats:

-          Packet loss of up to 30% is introduced on the public connection (and likewise the tunnel) while the VPN tunnel is active.  This can be verified by using the "line quality" test at http://dslreports.com/pingtest.  When no tunnel is established, there is no packet loss.

-          The packet loss through the tunnel seems to degrade over time, as does the tunnel connectivity itself.  After an average of five hours, the VPN tunnel will establish and pass traffic, but only for about 30 seconds before the tunnel is dropped.  A reboot of the PC makes things "better" again (connection stays up, but with much packet loss).

-          The VPN tunnel will only work with no NAT traversal (IP-to-IP ESP).  If we force the Shrew client to use NAT traversal, the tunnel will establish, but no traffic will pass through it (kinda like the Cisco client problem).

I've attempted to analyze what is happening using Wireshark, although I'm not gleaning any useful information from the packet captures.  I've also tried various MTU settings, with the same results as above.

A colleague and I have also tried testing the Shrew client on one of these PCs, while directly connected to our cable modems (we both use Comcast).  We experience the identical symptoms as I've listed above, although we have more success with the Cisco client working than our Suddenlink sites (but the Cisco client doesn't always pass traffic).  Even weirder, I've confirmed the same symptoms using completely different hardware/NICs, and also different Windows versions (7 and 8), connecting to two different Cisco ASA gateways, all with the same results.  My colleague installed Ubuntu on one of our PCs and tried that with the Shrew client, and that one worked just fine - no packet loss or problems.  Additionally, we have tried the Shrew client on DSL and fiber-connected Internet sites, using the same PCs (identical hardware and OS image), and those have been solid.   It is truly a mystery why the Windows PCs have a problem with the Shrew client on four different cable connections.

So finally, I simply ask the question, has anyone else seen a packet loss issue when using the Shrew x64 client on a Windows PC, using a "direct" cable Internet connection (no NAT device between the PC and bridged cable modem)?

Many thanks,

Jim

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.shrew.net/pipermail/vpn-help/attachments/20130730/e9b51216/attachment.html>