Cutlass Developers' Journal

So, we’re going to revise the handshaking protocol. It was under revision already, to support negotiation of crypto primitives, but I realized we had some additional requirements for NAT, and some people at Recon mentioned DoS resiliance as something that should be addressed up front. So, I figure we should take a step back, look at what other requirements we should address, and hopefully get the handshaking protocol stable so we can focus on other things, rather than have it be a moving target.

In some situations where we are NAT punching, we’ll be issuing initiation packets from both sides. If we’re doing that, we need a way to clean up the state so that we can sync up and complete handshaking by determining who becomes the initiator and who becomes the initiatee.

While we’re at it, we ought to harden handshaking to DoS attacks, and make the protocol capable of cipher primitive negotiation. And, hey, let’s clarify how the protocol tears down cleanly as well, so we can lose the quick-disconnect-restart class of bugs as well.

So, the following is a proposed set of requirements for the handshaking protocol. Feel free to propose additions or deletions to them:

- Cutlass must offer the ability to not answer to remote scanners that do not have a preshared secret
- Cutlass must not allow network observers to distinguish between an initial key exhange (where both sides do not know the other’s public keys), and subsequent key exchanges (where both sides are merely verifying the other’s public keys).
- Cutlass must offer the ability to exchange previously unknown public keys, offering usability no worse than that of ssh.
- Cutlass must require that initiators perform expensive computations before initiatees perform expensive computations
- Cutlass must require that the initiator prove receipt of initiatee information before the initiatee performs and expensive computations
- Cutlass must allow both sides to initiate connections simultaneously, over identical port pairs, and still be able to select disparate roles for each side (”initiator” and “initiatee”).
- Cutlass must allow one side to drop, and reconnect over identical port pairs, and still be able to take sensible action (Do we want to require the ability to renegotiate? Can we better define “sensible?”)

Now, I’m thinking that a great way to detect a potential drop-reattach situation is to have an undersized initiation packet. Since it’s hard right now to detect the difference between an invalid checksum and a reconnection attempt, if we detect a short packet, we could interpret that as a reconnection attempt.

How to prevent people from turning that into a DoS? We could start holding incomplete connections in a kind of kex purgatory, rather than in the connection hashtable proper. (This also makes it easier to allocate smaller amounts of memory until full handshake completion).

So once we detect a short packet of the proper format (whether or not we have an open connection), we can open a purgatory entry. From then on, incoming packets on that port pair get checked first to see if they kex properly. If they do, then we kill the purgatory connection.
If not, then we drop them down to the handshaking code to see if it can make sense of it. If we renegotiate a connection over the same port pair with the same public key, then we update the ephemeral keys and go forward from there. If it’s a different public key, tell the other side to kindly shove off.