Requirements & Kicking Off

For this tutorial you will ZeroMQ v4.0, CZMQ v2.0, and Sodium (the crypto library). We will build from source, since we're going to be coding anyhow. You can grab these either as download packages (here's Sodium, ZeroMQ, and CZMQ), or from GitHub (here's Sodium, ZeroMQ, and CZMQ).

Sodium is a packaging of the NaCl crypto library, which implements the Curve25519 elliptic curve cryptography (ECC) algorithm. It doesn't matter for this tutorial, but when you use ECC in real work, people will ask you (a) isn't ECC backdoored by the NSA?, and (b) how secure is this, and (c) how fast is it, and (d) who else uses it?

The answers are, at least today, (a) some ECC curves may be, and older crypto like RSA probably is, but Curve25519 is not, (b) it provides 256 bit ECC keys which is as strong as 3072 RSA keys, and (c) it's one of the fastest elliptic curves out there. Also, (d) it's not exactly mainstream, but is gaining popularity and Google is using it for their Quik protocol.

I'll assume you're working on Linux or OS/X and have gcc or similar installed, and some idea of how to compile and link C programs. The API I'll use is that provided by CZMQ. If you are working in another language, it may or may not provide the same level of abstraction. I'll explain what CZMQ is doing, in any case.

Here's how to build from GitHub (building from packages is very similar, you don't clone a repo but unpack a tarball):

git clone git://github.com/jedisct1/libsodium.git cd libsodium ./autogen.sh ./configure && make check sudo make install sudo ldconfig cd .. git clone git://github.com/zeromq/libzmq.git cd libzmq ./autogen.sh ./configure && make check sudo make install sudo ldconfig cd .. git clone git://github.com/zeromq/czmq.git cd czmq ./autogen.sh ./configure && make check sudo make install sudo ldconfig cd ..

Next, let's check that things are working, with a minimal example:

Here's how I build and run this:

gcc -o hello hello.c -lczmq -lzmq -lsodium ./hello

If that prints "Hello, Curve", then you're on the road to success. If not, check you have the necessary packages. On Debian/Ubuntu, you'll need build-essential, for instance.

What We'll Make

We'll make a minimal server-to-client PUSH-PULL flow with one server and one client. To make things really simple, the server will bind, the client will connect, and the server will send one message to the client:

Figure 1 - Minimal Server to Client

I'm going to make this in five steps, so you can see the different security patterns that are possible with today's ZeroMQ. To make it easy to remember these five patterns, I've given them mnemonic names:

• Grasslands, which is plain text with no security at all. All connections are accepted, there is no authentication, and no privacy. This is how ZeroMQ always worked until we build security into the wire protocol in early 2013. Internally, it uses a security mechanism called "NULL", but you don't even see that.

• Strawhouse, which is plain text with filtering on IP addresses. We still use the NULL mechanism, but we install an authentication hook (I'll explain this in a second) that checks the IP address against a whitelist or blacklist and allows or denies it accordingly.

• Woodhouse, which extends Strawhouse with a name and password check. For this, we use a mechanism called "PLAIN" (which does plain-text username and password authentication). It's still really not secure, and anyone sniffing the network (trivial with WiFi) can capture passwords and then login as if they were the real Joe.

• Stonehouse, which switches to the "CURVE" security mechanism, giving us strong encryption on data, and (as far as we know) unbreakable authentication. Stonehouse is the minimum you would use over public networks, and assures clients that they are speaking to an authentic server, while allowing any client to connect.

• Ironhouse, which extends Stonehouse with client public key authentication. This is the strongest security model we have today, protecting against every attack we know about, except end-point attacks (where an attacker plants spyware on a machine to capture data before it's encrypted, or after it's decrypted).

As I said in my last article, privacy is about cost, and these patterns get increasingly expensive to use. Stonehouse and Ironhouse are not as easy to deploy as TLS, for instance. But they are also much more expensive to break, and for most ZeroMQ users that is a good tradeoff.

The Grasslands Pattern

We'll kick off with the simplest possible example, which looks just like classic ZeroMQ code, because it is:

To keep the code short and focus on the API calls, I'm not handling errors here. This is fun, but dangerous, like driving a motorbike without a seatbelt. In practice, since your code will fail in lots of unexpected places, it's wise to catch at least the more basic problems early on.

When you start to use the ZeroMQ and CZMQ security APIs, the main symptom of confusion will be "nothing happens", in other words, there's no connection when you expect one. To make sure it's not a basic bind/connect error, at least check the return codes from those two calls. For example:

int rc = zsocket_bind (server, "tcp://*:9000"); // If the bind fails, don't even attempt to recover assert (rc != -1);

There are people who hate asserts, but they're an extremely useful and brutal tool for forcing your code through a single sane path, and giving you accurate and early proof of errors.

The Strawhouse Pattern

Our next model takes the same code and adds address checking. I'll explain what we're doing and how it works in detail, first here is the application code:

When you use CZMQ, all authentication happens through a zauth object, aka an "authenticator". In the Strawhouse code we create an authenticator, we enable verbose tracing (during development, at least), and we whitelist one address. Then we do the same work as Grasslands, and finally we destroy the authenticator:

zauth_t *auth = zauth_new (ctx); zauth_set_verbose (auth, true); zauth_allow (auth, "127.0.0.1"); ... zauth_destroy (&auth);

Internally, the authenticator talks to libzmq across a protocol called ZAP, aka RFC 27. Every single time a client tries to connect to a server, libzmq asks the ZAP handler (the authenticator), if there is one installed, to allow or deny the connection.

We tell libzmq that the server socket is in fact a "server" in the sense that it authenticates clients, by using this call:

zsocket_set_zap_domain (server, "global");

The ZAP domain concept lets you define authentication groups. For instance a set of server sockets could say, "We're in a domain called "test" and we only accept localhost IP addresses". CZMQ doesn't implement domains yet, and the "global" domain we use here has no significance at all. Remember that in this example we're using the NULL security mechanism, and by default libzmq just accepts NULL connections, period. But the simple fact of telling libzmq, "Please set domain X on this socket (with the default NULL security mechanism)" will cause it to act as a server, and call the ZAP handler for every client request.

You can mix different kinds of security in one application, by using different sockets, and then configuring each security mechanism separately in the authenticator. So it will handle PLAIN in one way, CURVE another way, and so on.

Now, to CZMQ's IP address filtering. This uses whitelisting and blacklisting, and works thus:

• You can whitelist one or more addresses, or blacklist one or more addresses. We do this through the zauth_allow() and zauth_deny() calls, rather than loading external files.

• The whitelist or blacklist applies to all requests that the authenticator gets from libzmq. That is, any NULL socket with a ZAP domain set, any PLAIN server socket, or any CURVE server socket, on that ZeroMQ context.

• If you have a whitelist (one or more addresses), that means, "reject any address that's not in the whitelist". Think of a private party, a very large bouncer, and a sheet of paper with names on it. "Sorry, buddy, you're not on the list. Please move out of the way."

• If you have a blacklist (one or more addresses), that means, "reject any address that's in the blacklist". Think of a public function with a couple of very large bouncers and a sheet of paper with names on it. "Hey buddy, didn't we tell you to never show your face here again?" You won't use both a whitelist and a blacklist, unless you want to make the bouncers confused, and angry.

• If not denied, and the security mechanism is NULL, then the connection is allowed.

• If not denied, and the mechanism is PLAIN or CURVE, then the authenticator carries on with its checking. So perhaps you're on the whitelist, or not on the blacklist, but now we need to see some identification, please.

So in our example, where we allow "127.0.0.1", we effectively deny every other client address.

The Woodhouse Pattern

Next, let's see how to do username and password authentication. Here is the full example:

Setting up the authenticator and the whitelist is the same as in Strawhouse, and I won't explain that again. The new API calls happen in three places in the code:

// Tell the authenticator how to handle PLAIN requests zauth_configure_plain (auth, "*", "passwords"); ... // Before binding the server zsocket_set_plain_server (server, 1); ... // Before connecting the client zsocket_set_plain_username (client, "admin"); zsocket_set_plain_password (client, "secret");

Bear this in mind: you have to configure a socket before doing a _bind or _connect on it. If you run the Woodhouse example without a password file, it'll never complete. Since we enabled tracing, you'll see it print out this:

I: PASSED (whitelist) address=127.0.0.1 I: DENIED (PLAIN) username=admin password=secret

The message in fact repeats because ZeroMQ re-tries the connection over and over. We mught change this behavior but it's not significant here. Let's create a password file, "passwords", in the current directory, with some entries, one per line:

guest=guest tourist=1234 admin=secret

Then run the Woodhouse example again, and it'll print this:

I: PASSED (whitelist) address=127.0.0.1 I: ALLOWED (PLAIN) username=admin password=secret Woodhouse test OK

The Stonehouse Pattern

Plain text is what it is. Anyone with half a malicious intent can sniff traffic and impersonate real clients, and make a mockery of your so-called security. That may be fine, or it may be a problem. The next level up is Stonehouse, which changes the game somewhat. Here's the example code:

Stonehouse starts with a single call to the authenticator that switches on the CURVE_ALLOW_ANY feature:

zauth_configure_curve (auth, "*", CURVE_ALLOW_ANY);

Here we do that for the domain "*", meaning all domains. Recall that CZMQ doesn't deal with domains yet, so if you use Stonehouse on one server socket, it will apply to all server sockets in the same context.

Now, let's talk about certificates. For any CURVE security you need a "long-term public / secret keypair" for the client, and for the server. CZMQ wraps up a keypair along with metadata (a name, email address, etc.) as a "certificate", which can be an in-memory object, or a disk file. Here is the explanation from the zcert class:

The zcert class provides a way to create and work with security certificates for the ZMQ CURVE mechanism. A certificate contains a public + secret key pair, plus metadata. It can be used as a temporary object in memory, or persisted to disk. On disk, a certificate is stored as two files. One is public and contains only the public key. The second is secret and contains both keys. The two have the same filename, with the secret file adding "_secret". To exchange certificates, send the public file via some secure route. Certificates are not signed but are text files that can be verified by eye.

Certificates are stored in the ZPL (ZMQ RFC 4) format. They have two sections, "metadata" and "curve". The first contains a list of 'name = value' pairs, one per line. Values may be enclosed in quotes. The curve section has a 'public-key = keyvalue' and, for secret certificates, a 'secret-key = keyvalue' line. The keyvalue is a Z85-encoded CURVE key."

What you would do in practice is generate certificates up-front (and CZMQ provides a simple tool to do this, in addons/makecert), and then put them carefully on each node in your network. It's like handing out neat little membership cards for an exclusive club, up front. Makes life easier for guests and security alike.

In the Ironhouse example we'll see how to work with "real" certificates on disk, using a so-called "certificate store". However for this example, we can generate certificates on-the-fly, in memory, and pretend we got them from disk.

We need a client certificate, a server certificate, and then we also have to pass the server's public key to the client. This is the total setup to use the Stonehouse pattern. CZMQ provides a zcert class that generates a new certificate, and which we can "apply" to a socket in order to enable it for CURVE security. Finally, we have to tell the client socket what our server public key is. Here are the relevant API calls:

// Generate certificates at runtime, since this is a test zcert_t *client_cert = zcert_new (); zcert_t *server_cert = zcert_new (); char *server_key = zcert_public_txt (server_cert); ... // Before binding the server zcert_apply (server_cert, server); zsocket_set_curve_server (server, 1); ... // Before connecting the client zcert_apply (client_cert, client); zsocket_set_curve_serverkey (client, server_key); ... // Destroy these in-memory objects when done zcert_destroy (&client_cert); zcert_destroy (&server_cert);

The zcert_new() call generates a public/secret key pair. We can ornament that with metadata, e.g. name, email, and organization, which can be helpful when managing certificates. Metadata doesn't play any role in authentication however.

The Ironhouse Pattern

Now, let's explore full authentication of clients. I call this the Ironhouse pattern, just to remind us that security is never total or perfect. It's always about cost. Even an iron house may have holes in the floor, or the roof, and your security is only as good as the systems it runs on. If you run Ironhouse on a box with an insecure or outdated operating system, don't expect privacy.

Here's the Ironhouse example:

Ironhouse is very similar to Stonehouse except it uses a "certificate store". That is a directory somewhere on the server's file system that contains a set of client public certificates. This is the API we use:

// Tell authenticator to use the certificate store in .curve zauth_configure_curve (auth, "*", ".curve"); ... // Stick the client public certificate into the directory zcert_t *client_cert = zcert_new (); zcert_save_public (client_cert, ".curve/testcert.pub");

For the client, nothing changes — it's the same code as for Stonehouse, as you'd expect. If this wasn't clear already, the zauth object belongs to the server, not the client.

A word about certificate stores. This is a CZMQ concept, and implemented by the zcertstore class. Normally you won't use that class directly, but like this example, allow the authenticator to manage it. All we do is (a) tell the authenticator what directory will hold certificates, and then make sure our clients' public certificates end up safely there.

In practice a server application will be a long-running process, so the certificate store reloads itself automatically if there is a change. This means we can copy new certificates into it at any time, to allow new clients, or remove certificates to reject clients.

Certificate filenames don't matter. Here we use "testcert.pub" but in practice I'd use a filename which is based of the certificate public key, for instance, to ensure it's unique. In our example we create the directory, and add some meta data, before saving the certificate:

// Create the directory zsys_dir_create (".curve"); zcert_set_meta (client_cert, "name", "Client test certificate"); zcert_save_public (client_cert, ".curve/testcert.pub");

When you save a certificate using the zcert_save_public (or zcert_save) method, it looks like this:

# **** Generated on 2013-09-19 21:25:34 by CZMQ **** # ZeroMQ CURVE Public Certificate # Exchange securely, or use a secure mechanism to verify the contents # of this file after exchange. Store public certificates in your home # directory, in the .curve subdirectory. metadata name = "Client test certificate" curve public-key = "I7[{YV4[}q[9a)]b&d>bisoT]UXa/7b$Tp:6yoyq"

In own applications I'm using $HOME/.curve to hold certificates, as zcert recommends, but you can put them anywhere. Some guidelines do apply:

• A certificate produced by makecert has a secret and a public file. Never share the secret file with anyone.

• When you transfer public certificates, use a secure route. If an attacker can modify this data without you knowing about it, they can construct a man-in-the-middle attack.

• For the same reason, make sure public certificates are not writeable by anyone except the current user.

The Ironhouse Pattern, Model 2

When you read the examples so far, you might be tempted to copy the code for your own applications. However it's not meant for that. It doesn't have a clean separation into client and server, and doesn't do any error checking. Here's a reworking of the Ironhouse example that is much more explicit. It's also twice the amount of code:

Here we see how to save and load client certificates from disk, to create truly persistent certificates:

zcert_t *client_cert = zcert_new (); zcert_save (client_cert, "client_cert.txt"); zcert_destroy (&client_cert);

And then, in a different place and time, or at least a separate thread:

// Load our persistent certificate from disk zcert_t *client_cert = zcert_load ("client_cert.txt");

Wrapping Up

OK, that's enough code. After half a dozen examples, we start to get code blindness. What is striking however is how short the code examples are, even in C. For me, this is success.

There's one last thing I want to discuss about security, which is the constant harping on about "server" and "client", which you may know are not ZeroMQ native concepts at all. The closest we get in ZeroMQ is having one side binding to an endpoint, and the other side connecting to it.

When using the PLAIN or CURVE security mechanisms (wood, stone, or iron), you can bind the client and connect the server. You can try this in stonehouse.c for example. Simply switch the bind and connect statements (change the socket arguments :) and observe that it still works. The server still authenticates the client.

You'll get the best results by typing the code, because muscle memory is a fantastic thing for programming. But if you're in a hurry, it is in the CZMQ project, in the examples/security directory. These examples are licensed under MIT/X11 so you can reuse the code in your own applications.

Let me know how you get on. Comment below, or come to the zeromq-dev mailing list. If you need commercial support in using these new security tools, or you're an open source project that wants to use them, drop me an email at moc.xitami|hp#moc.xitami|hp and tell me your use case.

by pieterh

The Case for Privacy

The summer of 2013 was when privacy on the Internet died. Or rather, when our belief in privacy died, because the surveillance of our phone calls, emails, web searches, and chats has been going on for many years, if not since the start of the telegraph era.

Privacy is a cost issue. An organization can secure its data centers from surveillance (by spies of all colors, whether they are foreign intelligence services, competitors, or criminals) by building its own physical networks and isolating these from the public Internet. However that is extraordinarily costly. More typically, we'll use virtual private networks (VPNs) to create secure lines that cross public networks.

VPNs are not reliably secure, however. They use long term keys that can be broken. They are complex to configure and to upgrade. The often use aging algorithms like RC4 that are considered crackable.

Even SSL/TLS are not immune from attack. The NSA themselves point out that the mainstream cryptography is over 20 years old, and state that "the best assured group of new public key techniques is built on the arithmetic of elliptic curves."

Without the shield of cheap, trustable privacy, an organization cannot safely use the public Internet. What the NSA can break today, any competitor or foreign intelligence service can break tomorrow. In our connected world, this throws a pool of ice cold water over the growth of low-friction Internet commerce.

Goals for ZeroMQ's Security Layers

Just as ZeroMQ brought us cheap, standardized connectivity for distributed systems, our goals with the security layers (which I'll explain in a minute) are to bring cheap, standardized privacy and authentication for distributed systems. The focus here is on cost. Cryptography has not stood still since the 1970s and there are many excellent, strong security options. However they remain costly to learn, understand, and safely use.

Here is our shortlist of goals:

• It must be very simple to use, and impossible to get wrong. Complexity is the number one risk with cryptography and the number one vulnerability. Every additional option is a way to get it wrong and end up with an insecure system.

• It must be fast enough for real work. If security makes a system too slow to use, people will switch it off, because the pragmatic need of being able to work today beats the risk of intrusion tomorrow.

• It must be based on standardized protocols, so that it can be re-implemented by any team, verified independently, and improved outside of the software stack.

• It must have free and open source implementations, so that the code can be inspected and fixed. One cannot trust a closed-source security stack, given the risk of secret backdoors.

• It must be free of patent risk, so that it cannot be blocked on grounds of intellectual property violations.

• It must work with all ZeroMQ socket types, and more generally, be interoperable with clear text messaging. A typical architecture will use clear text for internal messages, and security on external traffic.

• It must be built-in to the core libzmq library, so that it is available to all language bindings. Full-native stacks like JeroMQ should be able to re-implement the security protocols.

We started working on this in March of 2013, and the first formal publication will be in the forthcoming ZeroMQ/4.0.0 release. You can always get the latest version from the ZeroMQ GitHub project.

How ZeroMQ Security Works

The security architecture is broken into several layers:

• A new wire protocol, ZMTP 3.0, that adds a security handshake to all ZeroMQ connections.

• A new security protocol, CurveZMQ, that implements "perfect forward security" between two ZeroMQ peers over a TCP connection. I'll explain CurveZMQ below.

• A set of security mechanisms for ZMTP: NULL, PLAIN, and CURVE, each described by their own RFCs. NULL is essentially what we had before. PLAIN allows simple username and password authentication. CURVE implements the CurveZMQ protocol.

• An extended API for libzmq that lets you choose and configure the security mechanism for any socket. The default is NULL. You can then upgrade the socket to PLAIN or CURVE, either as a client (that connects securely to one server), or a server (which authenticates any number of clients).

• A protocol (ZAP) that lets you install arbitrary authentication servers. ZeroMQ lets you choose how you authenticate clients, be it via configuration files, LDAP, databases, etc.

• Finally, a separate security library, Curve, that implements CurveZMQ on top of ZeroMQ. Curve is a reference implementation for the CurveZMQ protocol, and a way to get that functionality on older versions of ZeroMQ. It will work with ZeroMQ/2.x and ZeroMQ/3.x (but not interoperate with the ZeroMQ/4.x CURVE mechanism, that works at a different level).

Let's walk through how we do secure PUB-SUB from one publisher to a number of subscribers, using the CURVE mechanism:

• We generate a long term keypair for the publisher. ZeroMQ provides a tool (curve_keygen) to do this. We store the resulting keypair safely.

• We manually distribute the public part of the keypair. This is a short text string. ZeroMQ does not do key exchange.

• If we want to authenticate subscribers, we generate a keypair for each subscriber, store the public+secret keypair safely at each subscriber, and collect all public subscriber keys on the publisher.

• Now, we can create a PUB socket and configure it with the long term keypair, and tell ZeroMQ that it is a server. This requires three zmq_setsockopt calls.

• We can then create any number of SUB sockets, and configure each one with its own keypair, and the server's public key. Again, three calls to zmq_setsockopt.

• We then bind the PUB socket, and connect the SUB sockets as usual.

• The ZeroMQ messaging API is unchanged. We send and recv messages as before.

One aspect that surprises some people is that one socket can have exactly one security level. You cannot have, for instance, PLAIN and CURVE subscribers connected to the same publisher. This would be technically feasible but would add extra work to applications to check the security level of every incoming message. It would make it easy to "get it wrong".

The Curve Story

Elliptic curve cryptography is a controversial topic. It has the reputation of being "backdoored" by the NSA, and heavily patented. There is no single curve, but an infinite number, and different curves have different strengths and weaknesses. Some of the "official" curves proposed by the NIST are suspect.

We use Curve25519 as designed by Daniel J. Berstein, which is the basis for the NaCl cryptographic library, more widely used as libsodium.

Curve25519 together with the other algorithms used in NaCl has some attractive features:

• It is fast.
• It has no known patent issues.
• It is considered strong.

Curve25519 provides encryption and authentication with 256-bit keys, equivalent to 3072-bit RSA keys.

This isn't sufficient guard against the simplest threat, where an attacker records encrypted data today, and then decrypts that tomorrow, after getting the keys by force, subterfuge, or warrant.

To prevent that, we need "perfect forward security", in which the two peers create temporary session keys and exchange those safely using the long-term keys. When the session is over, they discard their keys, and the encrypted data is then unreadable by anyone.

D. J. Bernstein developed the CurveCP protocol for secure networking over UDP, part of which was a secure handshake that provides perfect forward security using Curve25519.

The ZeroMQ CurveZMQ protocol is essentially a fork of CurveCP, adapted for use over TCP, and written as a single RFC.

Clients and servers have long-term permanent keys, and for each connection, they create and securely exchange short-term transient keys. Each key is a public/secret keypair, following the elliptic curve security model.

To start a secure connection the client needs the server permanent public key. It then generates a transient key pair and sends a HELLO command to the server that contains its short term public key. The HELLO command is worthless to an attacker; it doesn't identify the client.

The server, when it gets a HELLO, generates its own short term key pair (one connection uses four keys in total), and encodes this new private key in a "cookie", which it sends back to the client as a WELCOME command. It also sends its short term public key, encrypted so only the client can read it. It then discards this short term key pair.

At this stage, the server hasn't stored any state for the client. It has generated a keypair, sent that back to the client in a way only the client can read, and thrown it away.

The client comes back with an INITIATE command that provides the server with its cookie back, and the client permanent public key, encrypted as a "vouch" so only the server can read it. As far as the client is concerned, the server is now authenticated, so it can also send back metadata in the command.

The server reads the INITIATE and can now authenticate the client permanent public key. It also unpacks the cookie and gets its short term key pair for the connection. As far as the server is now concerned, the client is now authenticated, so the server can send its metadata safely. Both sides can then send messages and commands.

This handshake provides a number of protections but mainly, perfect forward security (you cannot crack data encrypted using the short term keys even if you record it, and then later get access to the permanent keys) and protection of client identity (the client permanent public key is not sent in clear-text).

Multilayer Security Using Libcurve

ZeroMQ's built-in Curve security can protect only one hop. For a B-to-C use-case this is ideal, and acts as a replacement for TLS/SSL, without the certificate authorities, but with the added cost of pre-distribution of public keys.

For C-to-B-to-C cases however, it is not sufficient, just as TLS/SSL is not sufficient. The cloud in the middle gets unencrypted traffic, and surveillance at that point is trivial.

To solve this we need security at a higher layer (instead of, or in addition to, the transport-layer security).

In effect this means encrypting messages before passing them to ZeroMQ, and then decrypting them after receiving them from ZeroMQ. The same requirement for perfect forward security applies, otherwise the data is vulnerable to key theft attacks.

With our security architecture, we use the Curve library (libcurve) to handle this higher layer. Curve provides both low-level codecs (to encrypt and decrypt) messages, and high-level "virtual sockets" (that act like ZeroMQ sockets but do security behind the scenes).

Coming Up

Security is complex, but if done right, will protect your business secrets from attackers. In my next article I'll start show example code that does secure messaging, over ZeroMQ. If you want to contact me directly about using ZeroMQ's security in your architectures, email me at moc.xitami|hp#moc.xitami|hp.

by pieterh

Preamble

Let me start by stating for the record that I'm not a security expert. I've got many… talents, such as knowing immediately if someone has sipped from my glass of beer, or making too many friends in too many places. But a deep and confident knowledge of how to stop the NSA reading my most private communications isn't one of them. However, the goal of this exercise is indeed to give us secure communications using ZeroMQ.

What is "security"? It's not one thing but a lot of answers to a lot of problems. At the core if A says something confidential to B, then A wants to know that C can't read it, can't modify it, can't pretend to be B, and can't imitate A either. But it goes further than this. Sometimes C will simply steal B's laptop, or kidnap B and torture him to give up his secret passwords.

If you study this long enough, you realize no-one is really a security expert. There are cryptographers and mathematicians, protocol designers and malware specialists, operating system security experts, code breakers and forensic cryptographers and on and on. I don't think any single person or team can build "security" or can even really know all the angles. What use is the best security library in the world if the application allows trivial SQL injections? Or if the operating system has known vulnerabilities? Or if the system admin is corrupt?

So this brings us back to something I repeat often in my writing and talks: it takes a big, diverse community to solve the hardest problems decently and cheaply. Luckily ZeroMQ is just that, which means given the "what" and the "who", I can segue into the "how".

How to solve a Very Difficult Problem

To solve a VDP, one starts by building a Minimal Plausible Solution, or MPS, and throwing this into the circus where the crowd can gather to watch it being torn apart by wild animals and religious fanatics. The crowd then realizes they can do better, so they make improvements and changes to the MPS and in turn throw these better versions into the ring. Over time the sport develops into a wild and furious game where the crowd split into factions each cheering different variations, and the noise attracts yet more wild animals and fanatics.

It's not quite that simple, but more or less, if you can throw a MPS into the ring and if people can improve this freely, you have a decent chance of seeing really good solutions emerge over time.

What are the characteristics of a good MPS? First, it must not take itself too seriously. You're designing Steak Tartar, remember. It has to be provocative and cheerfully cheap and simple. You're aiming for plausibility, not certainty.

Second, it has to at least work, so it can walk under its own power into the center of the ring, ready to be torn apart. We want the spectacle, the bloodshed, the drama, for it's this that pushes others to get involved. No crowd ever rioted after being shown a 50-slide PowerPoint.

Third, it has to be remixable in all directions. In the old days, when we ran punched card FORTRAN-77 on UNIVAC mainframes, we built security systems via committees of gray bearded elders who signed contracts defining who was allowed to make improvements to their Codes. These days we push everything under the (L)GPL and the use GitHub fork+pull request model. Old men are still welcome but only if they cut off their beards.

One of my talents is, I think, designing great MPSs and understanding how the circus works.

So this is what we've done for security for ZeroMQ. As you ask, "why didn't you implement SSL or TLS?", that makes you one of the wild animals. Enjoy the Steak haché tartare, it was meant for you.

Collecting the Problems

Diving down into the real grist of a minimal plausible solution, we need to collect some valid problems. It's no use producing an MPS for the wrong problems, that does nothing except waste everyone's time.

So what are the problems we need to solve first? You'd think it was something like "how do we implement a TLS handshake?" or some-such. But it turns out to be more meta than that. I started my research by looking at various security models, and making some quite elegant prototypes, but all I really learned was that choosing any particular model would be a mistake.

This is the first problem to solve: our own guaranteed incompetency when it comes to security. Even if we could build a perfect security model today (which I really doubt), it would become broken over time. It is not smart to build an expiration date into your software, especially if you are trying to make general purpose, long-lasting infrastructure, which is our goal with ZeroMQ.

There is, luckily, a known way to avoid having to choose a single security model, exemplified by SASL, an Internet security standard that's been used in quite a few protocols now. What SASL does, which is neat, is to delegate all security to a "mechanism" that client and server can implement independently of the rest of the protocol.

I find SASL overly complex, so would not implement it as-such unless someone was paying me to be inefficient. However it's quite doable to get the same effect with a simpler design. We stick a "mechanism name" into our protocol, and then depending on the mechanism the client and server propose, we call some plug-in library to handle the actual handshake.

The second problem is how to codify this so that we can get multiple independent implementations. It would be a big mistake to start by writing code (except as throw-away experiments). We need some formality that different stacks can aim at. Thus, standardization, and more specifically, in the ZeroMQ Message Transport Protocol (ZMTP) we use to tie our networks together.

Next, we need to prove our design by making a few different security mechanisms. It would be poor taste to force everyone into using full encryption, whether they need it or not. Some protocols then offer an on/off choice (e.g. insecure or secure HTTP). But it's smarter, I believe, to offer N choices, which can cover a spectrum of options. It fits with notion of extensible security and it also seems to fit what people really want, which is a sliding scale of trade-offs (usually performance and simplicity versus security).

So, let's recap our first set of problems:

• Making an extensible security model.
• Writing this up as a proper protocol specification.
• Creating a first set of security mechanisms.

We've been evolving ZMTP for some years from a minimal adolescent framing protocol to a more adult industrial-strength protocol. For example, adding a version header made it possible for newer versions of libzmq to talk safely to older ones, something that was impossible before.

ZMTP 3.0 basically breaks down into four layers or phases:

+-------------------------------+ | Version detection | +-------------------------------+ | Security handshake | +-------------------------------+ | Metadata exchange | +-------------------------------+ | Messages and commands | +-------------------------------+

The two peers exchange signatures and agree on a version; they then exchange credentials and agree on their security; they then exchange meta-data; and they then exchange commands and messages. The security handshake happens as early as possible to avoid leaking any information (the metadata) about peers.

I'm not going to explain ZMTP in much detail since there is a detailed RFC. But I can explain some of the design decisions. The mechanism is a single string. The two peers have to agree on this, or they can't work together. This is one big deviation from SASL, which allows peers to negotiate security dynamically. That could work in ZeroMQ but would force us to make the message API more complex (to indicate the level of security of the sender). We can't prove we need SASL's extra flexibility. So we allow just a single security mechanism.

ZMTP then specifies three default security mechanisms, NULL, PLAIN, and CURVE. The last two are specified by their own RFCs. They all use a somewhat similar handshake. The libzmq library implements the three mechanisms as separate classes.

So extensibility is now simple: create a new RFC that documents a new mechanism, and add the necessary class to libzmq. It's probably a week or two of work to add a new security mechanism like DTLS.

I'll briefly explain the three mechanisms. NULL means the client doesn't provide any credentials. We can check things like the IP address. PLAIN means the client provides a name and password, without encryption. It's meant to stop us doing stupid things like sending test data to production servers. Finally, CURVE implements an elliptic curve handshake called CurveZMQ, which is based on CurveCP. I'll explain this next.

Elliptic Curve Security, CurveCP, and CurveZMQ

CurveZMQ is a MPS for "full security" that was a lot more fun to make, and plausibly much more valuable, than an older asymmetric encryption model. I'm not a security expert, but elliptic curve cryptography, or ECC, works with shorter keys and uses less CPU than conventional cryptography.

However that's almost incidental. The real reason I chose to make CurveZMQ is because of something called CurveCP, a protocol and stack designed by mathematician and cryptographer Daniel J. Bernstein, author of the NaCl library. CurveCP was announced at the 27th Chaos Communication Congress on 28 December 2010 and CurveCP implementations are still considered experimental. I'm not going to claim CurveCP has any magical properties, is superior, or even works as-such. Indeed, the CurveCP protocol and software really could not work with ZeroMQ in any easy way, since it's designed to work over UDP.

Nonetheless, CurveCP starts with a handshake between client and server, and this handshake turns out, after we chop and cook it in various ways, to work very well both over TCP and over ZeroMQ DEALER-ROUTER, and to fit into our SASL-lite security model. There's a detailed RFC for CurveZMQ which explains the precise chopping and cooking I had to do.

CurveZMQ is essentially the CurveCP handshake, trimmed and cleaned-up to work over any bidirectional message transport, and documented as an RFC. I'm not yet sure how good or bad this is, but recall that we can't afford to take any particular mechanism too seriously. This is a minimal plausible solution for encryption, no more or less.

What immediately caught my attention was DJB's stated goal of making his APIs simple. The NaCl API is beautifully simple. I've worked with OpenSSL and it uses the industry standard approach of "expose every possible choice upfront with zero hints as to what is really vital, and what you will never ever use".

Whereas NaCl hides everything it can, on the theory that offering choices to people like me about what what key sizes or hashing algorithms to use is just asking for trouble. And I totally agree. NaCl is really like ZeroMQ, it hides choices we usually only get wrong.

Love at first sight! Whether or not the thing actually works, it was fun to learn and use and that's a powerful indicator. NaCl is in fact wrapped up as libsodium, which is the library we use.

So I wrote up CurveZMQ as a proper RFC and made a reference implementation. This library serves a double purpose. It acts as a guide to people implementing CurveZMQ and it can also be used by applications to do CurveZMQ encoding above ZeroMQ, to get end-to-end encryption. This is a little complex, and I'll make a real example app sometime to demonstrate it.

Helicopter View of CurveZMQ

To start a secure connection the client needs the server long term key. It then generates a short term key pair and sends a HELLO command to the server that contains its short term public key. The HELLO command is worthless to an attacker; it doesn't identify the client.

The server, when it gets a HELLO, generates its own short term key pair (one connection uses four keys in total), and encodes this new private key in a "cookie", which it sends back to the client as a WELCOME command. It also sends its short term public key, encrypted so only the client can read it. It then discards this short term key pair.

At this stage, the server hasn't stored any state for the client. It's generated a keypair, sent that back to the client in a way only the client can read, and thrown it away.

The client comes back with an INITIATE command that provides the server with its cookie back, and the client long term public key, encrypted so only the server can read it. As far as the client is concerned, the server is now authenticated, so it can also send back metadata in the command.

The server reads the INITIATE and can now authenticate the client long term key. It also unpacks the cookie and gets its short term key pair for the connection. As far as the server is now concerned, the client is now authenticated, so the server can send its metadata safely. Both sides can then send messages and commands.

This handshake provides a number of protections but mainly, perfect forward security (you cannot crack data encrypted using the short term keys even if you record it, and then later get access to the long term keys) and protection of client identity (the client long term key is not sent in clear-text).

Security Goals for CurveZMQ

Despite treating CurveZMQ as a disposable MPS, I want it to be solid against at least these attacks (this list is based on the CurveCP documentation):

• Eavesdropping, in which an attacker sitting between the client and server monitors the data. We send all in boxes that only the authentic recipient, knowing the necessary secret key, can open.

• Fraudulent data, in which an attacker creates packets that claim to come from the server or client. Only the authentic sender, knowing the necessary secret key, can create a valid box, and thus a valid packet.

• Altering data, in which an attacker sitting between client and server manipulates packets in specific ways. If a packet is modified in any way, the box it contains will not open, so the recipient knows there has been an attack and discards the packet.

• Replaying data, in which an attacker captures packets and re-sends them later to imitate a valid peer. We encrypt every box with a unique nonce. An attacker cannot replay any packet except a Hello packet, which has no impact on the server or client. Any other replayed packet will be discarded by its recipient.

• Amplification attacks, in which an attacker makes many small unauthenticated requests but uses a fake origin address so large responses are sent to that innocent party, overwhelming it. Only a Hello packets are unauthenticated, and they are padded to be larger than Cookie packets.

• Man-in-the-middle attacks, in which an attacker sitting between client and server substitutes its own keys so it can play "server" to the client and play "client" to the server, and thus be able to open all boxes. Since long-term keys are known beforehand, an attacker cannot successfully imitate a server, nor a client if the server does client authentication.

• Key theft attacks, in which an attacker records encrypted data, then seizes the secret keys at some later stage, perhaps by physically attacking the client or server computer. Client and server use connection keys that they throw away when they close the connection.

• Identifying the client, in which an attacker traces the identity of a client by extracting its public long-term key from packets. The client sends this only in an Initiate packet, protected by the connection keys.

• Denial-of-Service attacks, in which an attacker forces the server to perform expensive operations before authentication, thus exhausting the server. CurveCP uses high-speed high-security elliptic-curve cryptography so that a typical CPU can perform public-key operations faster than a typical Internet connection can ask for those operations. The server does not allocate memory until a client sends the Initiate packet.

Client Authentication

When we get a new connection into a server, we may have to check whether the client is allowed to connect or not. It's one of those problems that gets complex and messy quite quickly. Do we support password files, key files, databases, LDAP, PAM, GSSAPI, etc.?

As usual my favorite way to solve complex problems is to cheat and make this "someone else's problem". For authentication, what I wanted was a way to not implement any of the above list in the libzmq library, and still make it simple to support any of the above.

The answer in this case is to use ZeroMQ itself. The library, getting a new connection, sends a message requesting a yes/no decision, and someone gets the message, does the work, and sends a reply:

+----------+ | | | Client | | | +----------+ | | TCP connection V +----------+ | | | Server | | | +----------+ | REQ | +----------+ inproc:// | REP | +----------+ | | | Handler | | | +----------+

We can use the REQ-REP pattern over inproc://. If the process is built without any handler, the server won't authenticate clients (will allow them all). A handler can also proxy requests to out-of-process handlers, over local networking or ipc://.

I wrote this up as an RFC, called the ZeroMQ Authentication Protocol, or ZAP. Here's an example of a ZAP authentication request sent by a server to a handler, for a client using the PLAIN mechanism:

+-+ | | Empty delimiter frame +-+---+ | 1.0 | Version number +-----++ | 0001 | Request ID, for example "0001" +------+ | test | Domain, empty in this case +------+-------+ | 192.168.55.1 | Address +-------+------+ | PLAIN | Mechanism +-------+ | admin | Username +-------++ | secret | Password +--------+

This shows an example reply from the handler to the server:

+-+ | | Empty delimiter frame +-+---+ | 1.0 | Version number +-----++ | 0001 | Request ID echoed from request +-----++ | 200 | Status code +----++ | OK | Status text +----++ | joe | User id +-+---+ | | Metadata, empty +-+

It's a neat design because it works entirely out-of-band. That is, authentication doesn't affect message flow, can take as long as it has to, and can be handled by components that are fully independent of the application.

The ZeroMQ Security API

The security API is a set of new options that you apply to client or server sockets.

Here's how we configure a server socket for PLAIN security:

as_server = 1; zmq_setsockopt (server, ZMQ_PLAIN_SERVER, &as_server, sizeof (int));

And here's how we configure a client socket, which needs to know a username and password to send to the server:

strcpy (username, "admin"); strcpy (password, "password"); zmq_setsockopt (client, ZMQ_PLAIN_USERNAME, username, strlen (username)); zmq_setsockopt (client, ZMQ_PLAIN_PASSWORD, password, strlen (password));

Here's how we configure a server socket for CURVE security:

// Test keys from the zmq_curve man page char server_secret [] = "JTKVSB%%)wK0E.X)V>+}o?pNmC{O&4W4b!Ni{Lh6"; as_server = 1; zmq_setsockopt (server, ZMQ_CURVE_SERVER, &as_server, sizeof (int)); zmq_setsockopt (server, ZMQ_CURVE_SECRETKEY, server_secret, 40);

And here's how we configure a client socket, which needs to know three keys:

// Test keys from the zmq_curve man page char server_public [] = "rq:rM>}U?@Lns47E1%kR.o@n%FcmmsL/@{H8]yf7"; char client_public [] = "Yne@$w-vo<fVvi]a<NY6T1ed:M$fCG*[IaLV{hID"; char client_secret [] = "D:)Q[IlAW!ahhC2ac:9*A}h:p?([4%wOTJ%JR%cs"; zmq_setsockopt (client, ZMQ_CURVE_SERVERKEY, server_public, 40); zmq_setsockopt (client, ZMQ_CURVE_PUBLICKEY, client_public, 40); zmq_setsockopt (client, ZMQ_CURVE_SECRETKEY, client_secret, 40);

The application has to handle key exchange. For now the simplest model is that clients are given the server public key, and servers do not authenticate clients.

The Z85 Encoding Standard

The nice thing about standards is that you have so much choice. Wikipedia lists more than a dozen Base64 variations of dubious interoperability.

Why am I talking about Base64? Because exchanging binary keys is a royal pain. Almost as soon as we tried using CURVE security we hit the problem of how to represent keys in source code. For sure, you can encode binary as escape sequences but that's not portable between languages and doesn't work in configuration files, on the command line, or in emails.

Base64 is the common "standard" but it has several things about it that are nasty. First, there is no single standard so there's no real guarantee that Base64 implementations will work together. Second, it is a clumsy fit for binary data, and this is one reason there are so many variations. Base64 encodes 3 bytes of binary data as 4 characters. Yet most chunks of binary data are multiples of 4 or 8 bytes. That "divide by 3" aspect makes Base64 implementations more complex and fragile than such a simple problem deserves.

So I took an existing little-used encoding (Ascii85) and fixed it to make it safe to use in strings (so, also in XML, in JSON, on the command line, etc.) The result is Z85, which is "more compact than Base16, more reliable than Base64, and more usable than Ascii85".

I like Z85, because the code turned out to be simple and clean. That's always a good sign. The codec I wrote uses a precomputed table to rapidly convert from Base85 (Z85) to Base256 (binary).

Some people of course didn't like the introduction of Yet Another RFC, and they're right in some respects. I don't think it matters too much: I enjoyed making this little package; if it becomes a problem to others, someone will make a Base 64 encoding for the libzmq API.

The trick is when you do a setsockopt, if the key length is 32, it assumes binary. If the length is 40, it assumes Z85. If the key length is 44 (I think… or 43 perhaps?) you could assume Base 64.

Current Status

We got the security test cases running on June 21/22 at our ZeroMQ Developers' Meeting in Brussels, thanks to Ian Barber and Martin Hurton (who did the bulk of the work in libzmq). It's all on libzmq master today, and supported in the CZMQ binding. Other bindings will start to implement the security API options soon.

This all works, which is somewhat shocking.

It's also remarkably simple, which is also a pleasant surprise. We have, for applications:

• Three new options for PLAIN security.
• Five new options for CURVE security.
• One inproc:// plug-in endpoint for authentication handlers.
• One tool that generates new keypairs (in tools/curve_keygen).

And for those who want to get their hands dirty:

• Extensible classes in libzmq so anyone can try their hand at new mechanisms.
• A whole bunch of new RFCs along with reference implementations.

Conclusions

Of course no story is over until the main characters have been slaughtered in an orgy of bloodletting that leaves the audience shocked and traumatized and asking "why?" I'm sure CurveZMQ and ZAP and Z85 will get their Red Wedding in time. When it happens, I'd enjoy wielding the knife but it really doesn't matter as long as we make space for further interesting characters over time.

I'll be using the new security mechanisms in FileMQ, Zyre, and some other applications. There are a few other things still to fix in libzmq before this is ready for Internet use, and perhaps we'll print this as a new major version. I think it deserves it. But that's another story, for another day.

by pieterh

Real Protocols are Worth It

ZeroMQ used to have a sweet but unrealistic story: "our API looks like plain sockets, and we have practically no protocol". As it turns out, this was great marketing but not great technical design. The plain BSD socket API doesn't work for messaging (Contexts? Multipart messages? Zero-copy? Events? Threads?), and a non-protocol doesn't provide the number one guarantee users want, which is interoperability (you need version numbers).

ZMTP v1.0 was simple: each peer connected and then started blasting messages to each other. This is the protocol that libzmq v2.x speaks. The later release, v3.1 speaks a dialect of this protocol where subscribers send subscriptions upstream to publishers. You can't mix v2.x and v3.1 pub-sub on the same network.

We designed and implemented ZMTP v2.0 about a year ago. It added version numbering, and fixed the framing to be more sensible. This is the protocol the current libzmq v3.2 speaks (it will also downgrade to speak to older 2.x peers). It was a minimal change, and didn't address a bunch of other problems in ZMTP.

By 2013 it was clear that the number one problem with ZMTP is the lack of security. As I've shown in previous articles, it's not technically so difficult. It just needs to be done. At the same time, since adding security means big changes, we (Martin Hurton and myself) decided to address a few of the other long-standing problems in the protocol. These are mostly just questions the protocol should answer, but didn't.

As you read these changes, understand the "Cheap and Nasty" pattern. We connect once, and then exchange millions of messages over hours or days. The message transfer is asynchronous, changes very little over time, and has to be as efficient as possible. But creating a connection is synchronous, changes a lot over time, and efficiency is utterly irrelevant.

When you get this, you begin to see how to design what I'd consider a real messaging protocol. It breaks into two parts: a chatty negotiation at the start, and a high-speed asynchronous data stream thereafter. The negotiation needs to define commands with structure, and extensibility in the form of dictionaries (binary, or text, it doesn't really matter). ZMTP v2.0 didn't even start to go into this direction. In fact it broke that pattern by for instance sending meta data (identities and subscriptions) as messages.

Goals

A good way to understand ZMTP is to start with a plain TCP connection and then see what problems we need to solve in order to get a working message transport:

• TCP carries a stream of bytes with no delimiters, but we want to send and receive discrete messages. Thus, ZMTP reads and writes frames consisting of a size and a body.

• We need to carry metadata on each frame (such as, whether the frame is part of a multipart message, or not). ZMTP provides a flags field in each frame for metadata.

• We need to be able to talk to older implementations, so that our framing can evolve without breaking existing implementations. ZMTP defines a greeting that announces the implemented version number, and specifies a method for version negotiation.

• We need security so that peers can be sure of the identity of the peers they talk to, and so that messages cannot be tampered with, nor inspected, by third parties. ZMTP defines a security handshake that allows peers to create a secure connection.

• We need a range of security protocols, from clear text (no security, but fast) to fully authenticated and encrypted (secure, but slow). Further, we need the freedom to add new security protocols over time. ZMTP defines a way for peers to agree on an extensible security mechanism.

• We need a way to carry metadata about the connection, after the security handshake. ZMTP defines a standard set of metadata properties (socket type, identity, etc.) that peers exchange after the security mechanism.

• We want to allow multiple tasks to share a single external unique interface and port, to decrease system administration costs. ZMTP defines a resource metadata property that lets any number of tasks share a single interface and port.

Overall Behavior

A ZMTP connection goes through these main stages:

• The two peers agree on the version and security mechanism of the connection by sending each other data and either continuing the discussion, or closing the connection.
• The two peers handshake the security mechanism by exchanging zero or more commands. If the security handshake is successful, the peers continue the discussion, otherwise one or both peers closes the connection.
• Each peer then sends the other metadata about the connection as a final command. The peers may check the metadata and each peer decides either to continue, or to close the connection.
• Each peer is then able to send the other messages. Either peer may at any moment close the connection.

Version Negotiation

ZMTP v3.0 makes it much clearer how to do version negotiation. Each peer sends part of the greeting, and the higher version number can decide to downgrade to the lower version number. Thus implementations can choose to be backwards compatible (as libzmq is), but do not need to try to be forwards compatible (which is kind of hard). If you're a v3.0 peer and you get a greeting from a v4.0 peer, you just continue. If the 4.0 peer doesn't want to downgrade to your protocol, it'll close the connection.

Commands, Messages and Frames

Following the greeting, which has a fixed size of 64 octets, all further data is sent as size-specified frames. Command frames are used in the security handshake, and message frames are used to carry clear text application data. Commands always consist of a single frame. Messages consist of one or more frames.

A ZMTP connection is either clear text, or encrypted, according to the security mechanism. On a clear text connection, message data is always sent as message frames, using a single consistent format, so that traffic analysis tools don't need to know the specific details of a clear text mechanism. On an encrypted connection, message data is always encoded as commands so that wire analysis is not possible. (You'll never see message frames on a secure connection, period.)

The frame has the same format as in v2.0 (like I said, the asynchronous high-volume part of a protocol rarely changes once you get it right). It has a 1 byte flags field, followed by one or eight bytes of size, followed by body data. The only change we made in v3.0 was to add another flag bit to mark command frames.

People have proposed various other flag fields in the past, e.g. marking the first and last frame of a message rather than "more". Or, adding a "label" bit to mark message envelopes separately from message content (this would get rid of the null delimiter frame that request-reply still uses).

Right now, ZMTP still doesn't know much about actual socket semantics, but we could start to add this. I'd like, for example, to be more explicit about XSUB/XPUB subscribe and unsubscribe commands, which are now sent as messages with a special first byte. We could also do heartbeating and more. Actually it's not hard to add features. The real cost is not even implementing them, it's ensuring existing applications can migrate safely onto new stacks.

Authentication and Confidentiality

Now we come to the meat of ZMTP v3.0, which is security. More specifically, do we know who we're talking to, and are our conversations fully private?

Making secure protocols is complex, dangerous, and thankless work unless you find a way to cheat and make it all Someone Else's Problem. For ZMTP I borrowed from the IETF's Simple Authentication and Security Layer (SASL), aka RFC 4422, and the NaCl CurveCP security handshake. (I've explained my proposal for creative abuse of CurveCP in previous articles.)

If you haven't studied SASL, and you care even a little about security protocols, you should go off and read my latest book, called "SASL Made Simple" (1,350 pages, from iMatix University Press, $95.50 plus S&H). Just kidding… we can boil SASL down to a few key design elements:

• The server proposes, and the client accepts, a "security mechanism", which is a string.
• The implementations use the security mechanism to lookup a library that implements it.
• The server and client exchange challenges and responses, which are binary blobs.
• These blobs are passed to the security library, which digests them and possibly produces a new challenge or response.
• Eventually the security library at each side says, "OK, ready", and the server and client can now communicate.

It's a little more subtle than this, but that's the basic idea. After the two peers shake hands, they can send each other messages encrypted by the security library. Message I/O is still handled by the same code, with the security library acting as a filter (it encodes and decodes messages as needed).

What SASL does is pretty magic. First, we can add security mechanisms arbitrarily. The mechanism I want to use for ZMTP is CurveZMQ, but perhaps someone else will make a DTLS mechanism. Remember, each mechanism is a plug in library that doesn't need to know how data gets passed around. It takes blobs, produces blobs, and holds state about the connection. In my previous article I showed how a such a filter works.

Splitting the security work off makes it much easier to solve. We can test the mechanisms in isolation, before we plug them into real messaging. It also leaves the protocol totally naive about security. That means that as we switch to newer, faster security models, we don't have to change the protocol.

A peer announces precisely one security mechanism, unlike SASL, which lets servers announce multiple security mechanisms. Security in ZMTP is assertive in that all peers on a given socket have the same, required level of security. This prevents downgrade attacks and simplifies implementations.

When we use this approach, every connection always uses a security mechanism. Of course for most traffic will still be clear text. So ZMTP comes with three mechanisms: NULL (no security), PLAIN (clear text authentication) and CURVE (full security). PLAIN is there for two reasons. One, it provides a decent model for other mechanisms. I'll explain that model in a tick. Two, clear text authentication is pretty useful on internal networks. You don't want that test client connecting to the production server and sending 1M test messages by accident.

The NULL Security Mechanism

The NULL mechanism is easiest to understand - it gives us pretty much what we already have with ZMTP v2.0. The big difference is that there's an explicit READY command to send metadata (what you'd call "headers" in HTTP). Metadata is one of those things you really, really want to send in an extensible dictionary, since this is where 80% of the experimentation and profit comes from in protocols.

In the NULL mechanism, each peer sends off a READY to the other, and can then send or receive messages. They can check each others' metadata, e.g. to validate socket types. If either peer doesn't want to continue it simply closes the connection.

There's something about designing protocols to work like real human interactions that pleases me, and feels right. So there's no error reporting in ZMTP. If the peer closes the connection during handshaking, that's a strong rejection (take the hint, and don't connect again!), whereas if the peer closes the connection later, that's a temporary blip (sorry, got distracted, please connect again).

Here's the grammar for the NULL mechanism:

null = ready *message ready = command-size "READY " *property property = name value name = OCTET *name-char name-char = ALPHA | DIGIT | "-" | "_" | "." | "+" value = 4OCTET *OCTET ; Size in network byte order

Commands always start with a fixed-space 8 character name, and then descend into whatever form of madness makes the command work. For READY, we are sending a metadata dictionary. I decided to go with short text names and long binary values. 2GB should be enough for anyone. This format seems easy to create or parse in any language.

The PLAIN Mechanism

I'll explain how ZMTP's security works, because if we're going to use this for real work, we need criticism. I've some ideas for encouraging people to try to break the CURVE mechanism, but that's for later.

Since I've not finished documenting CURVE, we'll look at PLAIN. These two mechanisms have RFCs already, and you'll see that they use the precise same command grammar:

C:hello S:welcome C:initiate S:ready *( C:message | S:message )

This was deliberate. The handshake is synchronous and consists of two requests from the client and two responses from the server. For PLAIN, we could collapse this to one request and one reply. For CURVE it has to be split up, since the initiate-ready step depends on security already established by hello-welcome. (Note that in CurveCP, which is the original design for the CURVE mechanism, the command is called COOKIE, not WELCOME).

So conceptually, we have three steps:

• Establish security (HELLO, WELCOME).
• Send meta data (INITIATE, READY).
• Send messages.

While the first four commands are synchronous, messages are async. That is, the client can start sending messages as soon as it's received READY, and the server can send messages right after it's sent READY. INITIATE and READY carry metadata, like the NULL mechanism's READY command, and using the same dictionary format.

The CURVE Mechanism

Roughly, the CURVE mechanism works like PLAIN except that it exchanges encryption keys. CURVE in fact implements CurveZMQ, which is a generic security handshake I'm designing that is taken from CurveCP, a UDP-based security and traffic control protocol designed by cryptographer and mathematician D. J. Bernstein. CurveCP is based on NaCl, the security library from the same djb.

The security handshake in CurveCP is elegant and relatively simple (compared to other security designs), and comes with a very good pedigree. Whether it really is as robust as djb claims we won't know for a while. However as well as security, CurveCP aims to replace TCP entirely, doing its own traffic control over UDP.

To be honest, I think CurveCP (over UDP) would make a great mechanism in itself for ZMTP but that's a serious challenge. I've tried to keep ZMTP semantically compatible with that idea. In the more pragmatic short term, it turns out that stripping down CurveCP to just the security aspects, assuming TCP is going to be here for some time yet, is feasible.

This is CurveZMQ: the CurveCP handshake taken out and stripped down a little. There's a website, curvezmq.org, but it's still empty. My plan is to take material from curvecp.org, and remix that into a single clean RFC.

It turns out that it's very useful to do end-to-end security on top of 0MQ, as an alternative to doing security in the library. It means you can route over untrusted peers. Imagine I want to make a secure chat application. Two phones, talking to a server in the middle. We don't want the server to ever get clear text data. So the phones do end-to-end security. If we use security in ZMTP, that won't work.

End-to-end security only works with a ROUTER-DEALER combo (or another mix of sockets that can imitate it), since it needs that chatty request-reply handshake. But this is how we build most large-scale services anyhow.

Meaning, at some point I'll write a CurveZMQ mechanism in CZMQ, and show how to use this in a protocol like http://rfc.zeromq.org/spec:19 rfc.zeromq.org/spec:19/FILEMQ or rfc.zeromq.org/spec:20/ZRE.

Pluggable Authentication

A protocol shouldn't have an opinion on implementations except when it affects how they work together. But as ZeroMQ users we should be asking, "how does my application plug into this security model?"

The answer is a little subtle. Clearly as peers come and go, we have to invoke code that authenticates them. ZeroMQ can't do this itself in a realistic way. For instance even the PLAIN mechanism could do a remote username / password lookup to some other server.

Our idea is that libzmq provides an inproc socket for authentication and other events. Presumably this would happen on a per-context basis. You then write your authentication logic as a classic ZeroMQ task: connect to a socket, get the events, process them, and send back OK / NOT-OK replies. Meanwhile the "real" application logic continues to run ignorant of the ongoing authentication. It just gets messages and sends messages like today.

Connection Metadata

Adding a dictionary to the security handshake gives us extensible metadata. Right off, this let me move the socket type and identity out of limbo into a safer home. It also provides a place for more information about the connection.

There's really only one reason to put information upfront in the greeting, and that is when we absolutely need it for security negotiation. You'll see one flag ("as-server") that tells the mechanism whether the peer is acting as client or server. It means we don't need to rely on the bind/connect order. But all other metadata sits in this dictionary.

As well as socket type and identity, I defined "resource", which lets us do virtual addressing.

Virtual Addressing

One of the missing features in ZMTP was virtual addresses. Physical addresses (TCP endpoints) can be costly in some architectures, e.g. when crossing a firewall. If the protocol provides a place for a peer to request a virtual address, during connection negotiation, we can quite easily allow multiple tasks to bind to the same TCP endpoint.

In ZMTP we use the concept of resource, because it fits the schema we already use for zmq_bind and zmq_connect endpoints. Here's how a peer makes a resource available by binding to it:

zmq_bind (publisher, "tcp://*:9001/my/resource/name");

Here's how a peer connects to a specific resource on another peer:

zmq_connect (subscriber, "tcp://192.168.55.23:9001/my/resource/name");

The implementation will only create one listening TCP socket, but will keep a list of resources available for that listening socket. When a peer connects, the resource it specifies is then used to route messages to the right task.

In ZMTP the resource name comes after the security handshake, so it's encrypted and tamper-proof when we use secure mechanisms (I'll come to mechanisms in a second). Putting the resource name after security does have some implications. It means we cannot use the resource to select credentials. In effect, all connections on a physical endpoint must share the same credentials, e.g. the same server keys.

This isn't really a limitation since TCP in any case does not allow multiple listeners on a port. It's quite plausible to make a mulitiprocess design in which arbitrary processes on a box share a single physical endpoint, by each specifying a different resource. However, there is going to be a single process that does all the zmq_bind commands, acting as proxy for the other processes. And it's sensible that one process has one set of security credentials.

Security Considerations

I'll collect possible attacks and strategies to defend against them (if you have more suggestions, let's hear them!) Even attacks we can't defend against are worth documenting. So ZMTP makes some recommendations to implementors:

• Attackers may overwhelm a peer by repeated connection attempts. Thus, a peer MAY log failed accesses, and MAY detect and block repeated failed connections from specific originating IP addresses.

• Attackers may try to cause memory exhaustion in a peer by holding open connections. Thus, a peer MAY allocate memory to a connection only after the security handshake is complete, and MAY limit the number and cost of in-progress handshakes.

• Attackers may try to make small requests that generate large responses back to faked originating addresses (the real target of the attack). This is called an "amplification attack". Thus, peers MAY limit the number of in-progress handshakes per originating IP address.

• Attackers may try to uncover vulnerable versions of an implementation from its metadata. Thus, ZMTP sends metadata after security handshaking.

• Attackers can use the size of messages (even without breaking encryption) to collect information about the meaning. Thus, on an encrypted connection, message data SHOULD be padded to a randomized minimum size.

• Attackers can use the simple presence or absence of traffic to collect information about the peer ("Person X has come on line"). Thus, on an encrypted connection, the peer SHOULD send random garbage data ("noise") when there is no other traffic. To fully mask traffic, a peer MAY throttle messages so there is no visible difference between noise and real data.

Conclusions

We're still some way off from a secure ZeroMQ, but we're seeing the mountain clearly now. The next step is to make simple reference implementations of the core protocol and each mechanism. Then, we can implement the new protocol in libzmq, with some minimal authentication hooks. One step at a time…

So what do you think? Have you tried NaCl or libsodium, and did you like it? Are you more of a DTLS guy, and if so, what do you need to start designing your plugin mechanism for ZMTP/3.0? Are you one of those crazy geniuses who has reimplemented ZMTP/2.0 in Java, C#, Erlang, or Bash? How does this new protocol feel to you? Come and discuss on zeromq-dev.

by pieterh

Overview of CurveZMQ

The CurveZMQ code uses the CLASS conventions which make the C code look like something nicer. Here's the self-test code, which sends "Hello" and expects "World" back. CurveZMQ uses a single module, zcurve:

zmq_msg_t *msg; // Start server peer zcurve_t *server = zcurve_new (); msg = zcurve_start (server, NULL); assert (msg == NULL); // Start client peer and ping/pong through the handshake zcurve_t *client = zcurve_new (); msg = zcurve_start (client, zcurve_public_key (server)); while (msg) { // Send encoded message from client to server // ... // Receive encoded message in server msg = zcurve_accept (server, msg); if (msg) // Send encoded message from server to client // ... // Receive encoded message in client msg = zcurve_accept (client, msg); } msg = zcurve_encode (client, (byte *) "Hello", 5); assert (msg); // Send encoded message over the wire // ... // Receive encoded message from the wire msg = zcurve_accept (server, msg); assert (msg); assert (memcmp (zmq_msg_data (msg), "Hello", 5) == 0); msg = zcurve_encode (server, (byte *) "World", 5); assert (msg); // Send encoded message over the wire // ... // Receive encoded message from the wire msg = zcurve_accept (client, msg); assert (msg); assert (memcmp (zmq_msg_data (msg), "World", 5) == 0); zcurve_destroy (&server); zcurve_destroy (&client);

We work with two zcurve instances; one acts as client and one as server. In practice you'd create a zcurve instance for each 1-to-1 connection. The security protocol, which I'll explain, assumes that the underlying transport is TCP. There's an easy tweak to make this work over disconnected protocols (PGM, UDP, or even non-network protocols like shared disk). But since that isn't our use case, I'll explain the tweak and then forget about it.

The zcurve API reminds us a little of SASL, with its "I'll consume and produce opaque blobs until I'm satisfied". The blobs in our case are 0MQ messages. In the test code above we don't send them anywhere, we simply pass them between two zcurve instances until the handshaking is done. There are three critical methods in the API:

• zcurve_start () - which starts the state machine and may issue a challenge command.
• zcurve_accept () - which accepts a command and may issue a response.
• zcurve_encode () - which turns a plain text payload into an encoded blob.

In fact the API still needs work: right now the accept method's response message does double duty. If we're still handshaking, it has to go back to the peer. If we're finished hand-shaking, it's a decoded payload. I'll fix this in the next prototype.

But one step at a time. Don't make stuff you don't need. This brings me to the CurveZMQ protocol, which is derived from the CurveCP handshake and uses the same concepts like cookies, nonces, and vouches, but is simpler. The original CurveCP handshake is designed for UDP and includes a variety of non-security related features like virtual hosts. Trimming off this and other unnecessary fields gives us a minimal Curve-derived protocol.

The zcurve module uses a state machine to decide what's valid and what's not. In this prototype, anything invalid causes an assertion failure. It is not meant to be production code, yet. In practice invalid messages will cause the connection to be dropped.

Building and Testing

To build this you first want to install and build libsodium. Use the standard commands:

./configure make && make check && make install

You also want libzmq and CZMQ. Then take the CurveZMQ code from the gist and compile and link zcurve_test:

git clone https://gist.github.com/5224958.git CurveZMQ cd CurveZMQ gcc -g -o zcurve_test zcurve_test.c zcurve.c -lczmq -lzmq -lsodium

And run the test program:

$ ./zcurve_test Running self tests... * zcurve: C:HELLO: OK S:COOKIE: OK C:INITIATE: OK -:MESSAGE: OK -:MESSAGE: OK Tests passed OK

Security Models

CurveZMQ allows three possible security models:

• Anonymous clients, known server: where the server key is known to clients, but the server does not validate client keys.
• Pre-shared clients, known server: where the server key is known to clients, and all clients share a single key.
• Fully authenticated: where the full set of allowed clients is known to the server.

We don't yet have a hook in zcurve to authenticate a client; it would be trivial to add, e.g. using a callback that checks a given client long-term public key.

Differences between CurveZMQ and the CurveCP Handshake

To understand this section you should at least briefly read the CurveCP website and protocol description.

While CurveCP runs over UDP and uses a number of techniques to harden the server (such as not allocating memory until the client has successfully echoed a cookie), CurveZMQ runs over a connected transport (TCP or IPC) and cannot use, and does not need all the functionality of CurveCP:

• We do not use a minute key for cookies; instead, each connection creates a unique cookie key which is valid until the client sends an INITIATE command. The server could also check that INITIATE comes within 60 seconds of HELLO.

• We do not send the client short-term key in MESSAGE commands. This would be useful over disconnected transports but for our use case, TCP and IPC, it's pointless. If we added that, we'd add it both for client and server MESSAGE commands, so that clients could work with multiple servers.

• For simplicity, INITIATE does not contain message data.

• Thus, server and client MESSAGE commands have the same structure and are the only carriers for payloads.

• We don't have a hostname in the INITIATE command.

And we don't use any of the CurveCP traffic control protocol fields. CurveZMQ assumes a reliable connected transport.

The current CurveZMQ prototype still uses a fixed message size, maximum of 100 bytes. The next iteration will need variable message encoding. There are a number of other simplifications that are more or less safe:

• Long term server nonces are not segregated but we'd want to do this in practice: start the server nonce at a random number and then increment by 1.

• Short term nonces are always generated from random numbers. The chance of a collision is low enough that this is robust for production use.

CurveZMQ Internals

Here are the command structures we use. I'll document the detailed specification of each field later; the CurveCP website does explain them already:

typedef struct { byte id [8]; // Command ID byte cn_client [32]; // Client short-term public key C' byte cn_nonce [8]; // Client short-term nonce byte box [80]; // Box [64 * %x0](C'->S) byte padding [64]; // Anti-amplification padding } c_hello_t; typedef struct { byte id [8]; // Command ID byte nonce [16]; // Server long-term nonce byte box [144]; // Box [S' + cookie](C'->S) } s_cookie_t; typedef struct { byte id [8]; // Command ID byte cn_client [32]; // Client short-term public key C' byte cn_cookie [96]; // Server connection cookie byte cn_nonce [8]; // Client short-term nonce byte box [112]; // Box [C + nonce + vouch](C'->S') } c_initiate_t; typedef struct { byte id [8]; // Command ID byte cn_nonce [8]; // Server short-term nonce byte box [116]; // Box [M](A'->B') (from A to B) } cs_message_t;

The flow is quite simple:

• Client sends a HELLO command to server
• Server responds with a COOKIE command
• Client responds with an INITIATE command
• Client can now send MESSAGE commands
• When server receives the INITIATE command, it can send messages.

Next Steps

As next step we'll implement CurveZMQ in the libzmq core and document the protocol properly as an RFC. The C implementation will act as a reference implementation for the protocol.

by pieterh

The short story is: Sodium delivers. This is the library I want to build ØMQ security on.

There's so much to like about Sodium and only one downside that I'll come to. The project lives on GitHub so I forked and cloned it. It uses autotools, so builds just like all our projects do:

./autogen.sh ./configure make && make check && make install

This will make it easy to integrate into FileMQ or CZMQ when the time comes. When I recall how difficult it used to be to build OpenSSL, this is gratifying. Autotools may be one of the least friendly toolsets ever but it's established a standard for C library projects that just works.

So I cleaned out my NaCl installation and installed Sodium. Sodium uses the same approach as CZMQ and libzmq, which is to provide a single header file (sodium.h) for the whole library. Awesome!

Allow me a little rant. When you make a library, please do like Sodium does, and export a single header file. Why give the user extra work with no benefit? The old technique of "if you want functionality X, include header file Y" was a great idea in 1970 when compilers took minutes per file, but is pathological in 2013.

When I include czmq.h in a program, that pulls in several dozens of the commonly-used system header files as well as the whole CZMQ interface. It adds about 0.1 seconds to a compilation. Even in 1990 when this trick added 2-3 seconds, this was a good trade-off because it gave me simpler, more consistent code.

It's very poor practice to start a C source file with a bunch of includes. Libzmq does this and it's been a source of error and confusion. "Oh yes, you have to include winsock.h but after that other file, otherwise you get a bad definition for the maximum sockets." Bleh. Create a project header file that sets-up the whole compile environment for you and then include that in each source.

And export your API as one header file, please. </rant>

So here's a minimal Sodium example:

#include <sodium.h> int main (void) { unsigned char public_key [crypto_box_PUBLICKEYBYTES]; unsigned char secret_key [crypto_box_SECRETKEYBYTES]; crypto_box_keypair (public_key, secret_key); return 0; }

Here's the next thing I love about Sodium: it is 100% compatible with the NaCl API. Not 80%, or 99%, but literally one hundred percent. It's hard to express the joy this gave me as a developer. NaCl's API is one of those "love at first sight" things. This is the gold standard for everyone building APIs: stick to the contracts, resist the temptation to screw over your users in the name of "doing it better this time". Try designing the API right the first time around, maybe?

Now, language bindings. Sodium reports two bindings, PyNaCl and RbNaCl. This means any design I make using Sodium will be easy to re-implement in the Python and Ruby bindings for libzmq (PyZMQ and rbzmq). Yay!

It was literally 30 seconds' work to change, recompile, and retest my zcurve proof-of-concept. It helps that I use a helper script called "c" that does smart things like searching /usr/local/lib for all libraries:

~/work/zcurve$ c -l zcurve-poc Compiling zcurve-poc... Linking zcurve-poc... ~/work/zcurve$ ./zcurve-poc C:HELLO: OK S:COOKIE: OK C:INITIATE: (host=localhost) OK C:MESSAGE: (request=Hello) OK S:MESSAGE: (reply=World) OK -> 'Hello World' test successful

Now, performance testing. Here's the performance I was getting from NaCl, compiled specifically for my AMD64 CPU:

One-step box: Encryptions: 6400/second Decryptions: 6400/second Two-step box: Encryptions: 1449300/second Decryptions: 762600/second

The one-step box does a hashed key exchange and then does an encryption using the resulting key. The key exchange is slow, so NaCl lets you do this in two steps: do the key exchange once, and the encryption repeatedly (using a different nonce each time). That gives the second result.

Now here are the figures for Sodium:

One-step box: Encryptions: 300/second Decryptions: 300/second Two-step box: Encryptions: 733400/second Decryptions: 488800/second

The lesson is clear: don't use single-step encryption unless you're really doing a one-off (as in the start of the CurveCP handshake). The two-step encryption is about half the speed of NaCl's optimized code. The Sodium team know that performance is a concern, and they say:

Optimized implementations (including NEON optimizations) will eventually be supported, but tests and benchmarks will be performed at run-time, so that the same binary package can still run everywhere.

They've gotten it right: clean packaging and portability is more important than raw performance. It'll take a few years for Sodium to replace OpenSSL as the security library of choice and by that time, there will be enough people using it to make optimizations worthwhile.

by pieterh

What's Special About CurveCP and NaCl?

NaCl, the lower-level library, is the ØMQ of security libraries. It has two outstanding features. One, it's very fast. Two, it has a simple API that hides everything except literally what you need to encrypt and decrypt data. Security is not an easy subject but NaCl makes it as easy as I've ever seen. Hiding the algorithms means as a developer I can't get it wrong. The site says:

The following report specifies NaCl's default mechanism for public-key authenticated encryption, and along the way specifies NaCl's default mechanisms for scalar multiplication (Curve25519), secret-key authenticated encryption, secret-key encryption (Salsa20), and one-time authentication (Poly1305): (PDF) Daniel J. Bernstein, "Cryptography in NaCl", 45pp.

A simple encryption/decryption test shows I can do 1.4M encryptions or 750K decryptions per second. This is of a short text but it's still impressive.

Now CurveCP, which is packaged with NaCl. This is quite the beast! It's not just a security layer, it's a full replacement for TCP and includes a full protocol client and server using UDP. The interesting thing for us ØMQ programmers is how CurveCP steps aside from TCP's connection model, and creates its own logical, secured connections.

There is one "gotcha" with CurveCP, you have to handle your own key distribution. Peers have to know each others' public keys in advance. This is a challenge we'll solve later. CurveCP is asymmetric, so a peer is either a "client" or a "server". Nothing unusual here. The freaky part (which I appreciate) is how carefully Daniel J.Bernstein, the mathematician, cryptologist, and programmer behind this animal has designed it to stand up to a range of attacks.

CurveCP authenticates servers and clients, stops replay and man-in-the-middle attacks, and provides active and passive forward secrecy, which means even if you capture traffic, you can't do anything with it. The client's long-term public key is safe from snoopers: connections use short-term keys.

The list of goodies goes on: CurveCP is a fully disconnected protocol. So a client can move around and switch IP addresses and continue to work. It's route independent: any connection, any transport, any intermediaries. If you're a ØMQ developer you should be drooling by now. This is sexy stuff!

So CurveCP looks really attractive. Next question, can we use it for ØMQ dialogues, and if so, how?

Taking CurveCP Apart

As provided, CurveCP is really two things. One, a set of specifications for the protocol, and second, an implementation of the client and server. The protocol isn't a single document like an RFC but a set of web pages. It's immediately clear that DJB has bitten off rather a lot with CurveCP. There's no layering. My mind reels as I read the protocol description and then the source code. It's C, but barely comprehensible.

There's no way I can see to use the CurveCP implementation, not any of the source code, nor the protocol as-is, for anything ØMQ-related.

Luckily Grant Rodgers, who works at GitHub (I think), wrote a Ruby implementation of just the CurveCP handshake. This is a great tutorial for the inner workings of CurveCP's security. The Ruby code uses some functions from RbNaCl so I ended up reading that code too.

ØMQ has many echoes of UDP, with its disconnected message passing so it seemed natural to build a CurveCP handshake above ØMQ. The handshake is pretty simple (at the surface):

• Client sends HELLO
• Server sends COOKIE
• Client sends INITIATE
• Server or Client sends 0 or more MESSAGEs

The hard work is in the details of each command but to be honest, except for the boring business of pushing bytes around and managing various sets of keys, and making some good choices, it's quite straight-forward. The code is about 450 lines and I've made no attempt to create abstractions. This is a straight unfolded step-by-step implementation.

When I run this, it does a handshake and a "Hello World" exchange:

C:HELLO: OK S:COOKIE: OK C:INITIATE: (host=localhost) OK C:MESSAGE: (request=Hello) OK S:MESSAGE: (reply=World) OK -> 'Hello World' test successful

Installing NaCl

If you want to try the PoC you'll have to install NaCl. Here's how I did this on my Linux box. First we build the library:

wget http://hyperelliptic.org/nacl/nacl-20110221.tar.bz2 bunzip2 < nacl-20110221.tar.bz2 | tar -xf - cd nacl-20110221 ./do

Then we install into /usr/local:

cd build/myboxname sudo cp include/amd64/*.h /usr/local/include cd lib/amd64 ar r libnacl.a randombytes.o sudo cp libnacl.a /usr/local/lib cd ../.. sudo cp bin/amd64/bin/curve* /usr/local/bin cd ../..

Details of the Proof-of-Concept

Though CurveCP hides the dirty choices of what primitives to use, we need to choose good "nonces". A nonce is a "number used once" and it's the caller that has to provide these. If you reuse the same nonce, you're effectively opening a crack that will leak data and/or public keys.

CurveCP uses two kinds of nonces:

• Long-term nonces (16 bytes), used with the client and server long-term keys. The website describes a few strategies for choosing these. In the proof-of-concept I used random bytes from /dev/random. This is the simplest option and the chance of duplicate nonces is so low that at this stage, that's the choice I'll keep for real code.

• Short-term nonces (8 bytes), used with short-term keys (CurveCP uses a new key set for each connection). These are just incrementing 64-bit integers, which even I can produce.

The CurveCP INITIATE command in theory can carry a message (that is, user data). I didn't implement this for the example. Instead the client sends INITIATE and then immediately a MESSAGE carrying the "Hello" string.

The PoC also sends fixed-sized MESSAGE commands but this is just to keep it simple. In reality we'd want variable length MESSAGE commands.

The PoC is filled with magic numbers like the sizes of keys (32 bytes), nonces (24 bytes) and so on. This sucks but it seems unavoidable at least for now.

Next Steps

The code needs packaging; there's a lot of repetition and I'd like to turn this into a CZMQ-like module. The CurveCP handshake can be fully hidden. It could maybe look like SASL, that is, a client and server exchanging opaque blobs until they're satisfied (or they break the connection).

In the FileMQ project I used SASL for security, and CurveCP seems like a perfect fit into that. So that'll be my next step: secure file transfer.

At this stage the rest is just ideas. But I'd like to experiment with multiple routes between clients and servers. So, for instance, start a secure connection over DEALER-ROUTER, then continue it over PUB-SUB. If we can create a nice module, it could go into CZMQ so it becomes available to a wide set of developers.

Then, key distribution. I'm thinking, the Clone pattern from Chapter 5 of the Guide, combined with CurveCP security.

I'd also like to make a proper RFC for CurveCP-derived security over ØMQ. What I implemented in the PoC isn't the CurveCP protocol, it's a subset of that. For example there are no extensions, and the MESSAGE command lacks all the flow-control fields. This needs to be documented so that other people (e.g. Ruby programmers who are lucky enough to have RbNaCl) can make interoperable code.

Conclusions

ØMQ doesn't play nicely with the old-fashioned connected security model of TLS/SSL. Luckily there's something better in the shape of CurveCP, a fast peer-to-peer security layer that promises authentication and confidentiality, speed, and simplicity. I've written a simple proof-of-concept that demonstrates the CurveCP handshake running over ØMQ.

Edit: Marc Falzon points out that libsodium is a rather nicer, API compatible repackaging of NaCl which also runs on Windows.

by pieterh