[vpn-devel] SHA2 Inteoperability issues?

[Mark Larwill]

I'm a developer at WatchGuard and I'm working on adding SHA2 support for
SHA2-256, SHA2-384, and SHA2-512 to our firewall. I have noticed that when
testing with NCP's mobile VPN client, everything works. When I test with
Shrew's client it does not work. Here's what I know:

1. Phase1 and Phase2 both complete successfully
2. If I send ping traffic, Shrew does generate ESP packets
3. ESP packets are dropped at the firewall and a log message is generated
saying that "Integrity check failed"

May 20 20:45:30 2013 Firebox local0.warn kernel: [ 1553.257472] WG: Packet
rejected. Gateways (src 4.4.4.1 dst 8.8.8.1) Selector (src 77.0.0.1 dst
0.0.0.0) spi 0xb5d723ce proto 50 reason "Integrity check failed"

I understand that the problem could be on either side, so at this point it
is not clear if the Shrew client is doing something wrong, or if the
firewall is. However if I had to make an educated guess, I would speculate
that maybe the shrew client is sending the full SHA2 hash, instead of the
truncated value (per RFC4868). This could cause the integrity check to
fail. For example if using SHA2-256 and sending a 100 byte packet, with a
full 32 bytes of hash, for a total of 132 bytes, the firewall might assume
it's a 116 byte packet with 16 bytes of truncated hash.

I have two separate packet captures from the same setup, one showing NCP
working, and another showing Shrew not working. I can provide them if
desired, but don't want to spam the whole mailing list.

Since the ESPs are encrypted it is difficult to tell much about what's
wrong form the packets themselves. Curiously however with the NCP packet
capture I see 146 byte ESPs going both ways --- It's just ping traffic, but
with the Shrew capture I see various packet sizes for ESPs, 126 bytes, 142
bytes, 254 bytes.

Has anyone had any success with SHA2?
Do any shrew developers know what kinds of interoperability testing has or
has not been done with SHA2?
Would anyone like more information? I have a local setup and can reproduce
issues easily.

PS. I understand it is not Shrew's responsibility to help me troubleshoot
my code, but at this point I cannot tell whether the problem is with Shrew
or not, so I figure developers may be interested at least up to the point
where they can rule out bugs in the Shrew code. This is a new feature so
perhaps bugs haven't been exposed yet.

--Mark Larwill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.shrew.net/pipermail/vpn-devel/attachments/20130520/10c98aa0/attachment.html>