Forward with Federated

It's no longer deniable that "cloud services" are a part of the future of the software industry. And free software proponents rightly fear this development: in a cloud service, you don't have any control over what source code runs on your behalf, and what it does with your data; neither can you change it. By and large, the free software answer to cloud services has come in one of two forms. In one, a free software version of the cloud service still has a client-server design, but the server is designed to be run on the user's hardware. In the other, the free software version is completely peer-to-peer.

Both these approaches are flawed. Free software client-server programs are rarely adopted due to the hassle of running your own server. In general, despite being a smart and sometimes-capable developer, I don't trust myself to run a mail server – the security implications, the bug tracking burden, and the time commitment are all too much. Instead, I ask Google to do the same. In the same way, I'd rather let a bank secure my money, than do it myself. I want to offload my sys-admin responsibilities to others; and if using your software requires me to administer a server, it has a very limited audience.

On the other hand, peer-to-peer programs have their own flaws. Peer-to-peer programs tend to require each client to perform work helpful to all others. I don't want to help store other people's files, record other people's transactions, route other people's messages. In a client-server design, my computer, and my network pipe, is responsible only for its own actions, and those of others that are directly relevant to me. Peer-to-peer programs also have the flaw that they require a large network to work. How many successful purely peer-to-peer programs are there, other than BitTorrent and Bitcoin?

Perhaps the simplest way to summarize my issues with both of the above designs is that they assume a very degenerate graph of communication between agents. Client-server programs assume that the graph is a wheel shape, and peer-to-peer programs assume that the graph is completely connected. But neither of these is realistic. In the real world, there are a few servers I trust to represent me in terms of email: Google, MIT, and the MX record stored in Namecheap's DNS. I do not trust all mail servers, and do I want to burden them all with receiving and routing my mail; but neither do I want a single point of failure.

I think a better model for free-software web services and protocols best assume a multi-master design, where each client communicates to several servers (but not all), and where the servers are assumed to be run by competent sys-admins[1]. This means that each user can trust the administration of a service to a third party; but paranoid users might use two or three third parties for safety's sake. I'm going to call this the "federated" architecture, since software of this form is a composition of relatively few large servers.

A few things to note about writing software with this architecture in mind:

1. Since third parties are run competently, you can generally assume that server failures will be rare. However, third parties can and do go out of business, so at least some method of backing up data is a significant boon to users.
2. Since a user might use two different servers, there ought to be a way for two third parties to assert that they share a user. This could be done by, for example, simply having each propose such a merge to the other. Or, for superior, cryptographic security, we might have the two servers produce user-signed certificates that they claim to be the use on the other domain.
3. Two users might be located on different servers, but still want to interact; thus some method of cross-server interaction is necessary. One way is to name each user by the server they are from and delegate authentication (for example, I might be user pavpanchekha on the example.com server, but pavpanchekha@example.com on all others, and I could log into any other server as that user, which would forward authentication to example.com. This also has the benefit of also solving the uniqueness problem every user has a unique name based on their "home" server.
4. A user may want to move from one server to another. Anyone who's ever changed their email address knows how painful this can be. It'd be best to allow this as a first-class action; though perhaps if users can delete accounts, that, combined with sharing accounts, can be enough.

There are many pieces of software successfully using this model already. Email is the most obvious: every email user has an email provider, all of which work with all other email providers. This is nice for users, too. It means that we can have competition among mail providers (witness the disruption that Google Mail was), that most users can be safe knowing that their email is backed up and secured for them, and that the most paranoid users can run their own mail servers successfully. In email, it is generally assumed that mail servers will not fail, certainly not for too long, so email providers simply retry a few times and then give up. Authenticating two servers as sharing a user is done with forwarding rules, which while not a part of the standard are nowadays standard. And users interact simply by sending email, which works over a standard protocol every mail server understands.

MIT's Kerberos system, which in other forms became LDAP and similar technologies has a similar mechanism for authenticating to another "realm". So my MIT credentials can be reused to authenticate on CMU's network, or within MIT's CSAIL lab (which uses its own Kerberos realm for historical reasons).

Jabber, too, has a similar federation design. It even includes explicit means to tell the server that a contact has multiple names, and the server can decide which names to try.

Here's a few idioms that can be found for federated software. First, each user ought to have a personal identifier that includes the name of some sort of home server. People tend to be OK with multiple names pointing to the same person, so if two servers share a user, that user will simply have two user names. However, note that for some users, user names are an important social signal; so if there is any way for a third party to verify that two usernames are the same user, there should be a verification step between a username is assigned to a user.

Another idiom is how to manage cross-server interactions. This is usually done (see: email, Kerberos, Jabber) by forwarding requests from one server to another. In particular, all of this routing is done through the server, not the client, presumably to make the clients less complicated to write. If you can assume that faults are rare, I think this is a good model, though of course you have to very careful about synchronization and concurrency.

Most existing federated designs do not tackle security and privacy. This is a very difficult area, since both are global properties of a system, and in a federated architecture you don't have global control. But many of the easy problems with security are solvable with public-private cryptography, so reasonably-secure systems can probably be made with a federated architecture.

Finally, almost all existing federated architectures come with their own protocol. Something needs to be shared between the various servers, and a protocol seems a reasonable place to put this common ground. Nowadays, it may be most reasonable to layer this protocol over HTTP. Not because HTTP is "cool", but because TCP is a good base on which to build, and because HTTP is universally unblocked. Or you could do TLS-encrypted communication over port 443, but this seems as ugly as layering another protocol atop HTTP. And, for many applications, especially those with a streaming or request/response design, HTTP provides a lot of nice features: content negotiation, caching, and a rich vocabulary for error messages.

I'm hoping that more open source projects adopt a federated design. It is in many ways exceptionally user-friendly, and allows for a large amount of user control. Perhaps anyone who has attempted to implement one of these federated services can weigh in on specific challenges? I'm happy that some new projects, like Tent, are adopting this architecture. Let's see if it spreads.

[1] Note that even some purely peer-to-peer services, like BitTorrent, end up setting up out-of-band communication based on this mechanism. For example, BitTorrent has each client connect to some set of trackers, which follow this multi-master setup: each torrent may be held by multiple trackers, and a client can connect to several (but surely it does not connect to all).