Open source projects to secure Internet traffic
Tuesday June 25th 2013, 9:39 am
Filed under: SOA/Web/Etc.
Filed under: SOA/Web/Etc.
In the beginning of the net the focus was mostly on getting packets safely from point A to point B. Anyone who knows the history of the early Internet knows how insanely hard that actually was (as a hint think: network of networks, oy). Later some thought was given to privacy and technologies like SSH and SSL show up. Now we are at the point where we need to think hard about traffic analysis. In this article I try to catalog what I think (based on little evidence) are the main types of open source projects trying to create traffic analysis resistant transports on the Internet.
1 Required knowledge
It is assumed that the reader is familiar with the following terms, ideas and technologies:
- traffic analysis,
- onion routing,
- mix networks,
- the difference between onion routing and mix networks,
- distributed hash tables,
- public key cryptography,
- forward secrecy,
- proof of work,
- sybil attack,
- friend to friend network,
- TLS/SSL and
Generally Wikipedia is your friend.
The data below is intended as my notes from a reconnaissance mission to see ’what’s out there’. I gathered this data by reading documents, websites, etc. I didn’t run the actual software or look at the source code. So I can only imagine the laughable errors I’ve made. You have been warned.
3 Onion Routers
Tor is a pure onion routing infrastructure. It supports arbitrary TCP connections and is primarily used to allow requesters to run their TCP requests to servers on the open Internet to be run through the onion routing infrastructure. Tor also supports hidden services which allow servers to route any requests to them through the onion routing infrastructure. Hidden services are identified by URIs in the ’.onion’ DNS space and hashes of their public keys. As a practical matter hidden services aren’t on the ’open Internet’ since they require a special handshake to establish a session. There do exist however proxies that accept .onion requests and will route to the hidden service.
Tor uses a centralized directory infrastructure to record and manage intermediaries available for onion routing as well as part of the hidden services communication infrastructure.
Tor supports ’hidden proxies’ that users can run on their own machines and get addresses for from the Tor administrators which can forward Tor requests. The point is to allow a user to hide the fact that they are using Tor by making connections that ’look’ like normal traffic to destinations that are not known to be part of the Tor network.
3.1.1 Writing apps for Tor
Tor provides a proxy that runs on all the major PC operating systems and Android. It allows one to pass TCP/IP traffic either outbound (for anonymous access) or inbound (through hidden services). Tor has its own project list and there is Anonymous Python Application Framework (APAF) which provides a framework for writing Python services that are intended to be exposed as hidden services.
I honestly don’t understand I2P. It claims to be both a mix network and an onion routing network. I can clearly see the onion routing part but I don’t see anything that indicates they have implemented the mix part.
From an onion routing perspective I2P consists of half duplex connections of two distinct types, inbound and outbound. It is up to the person setting up the tunnel to decide how many hops they want. Protocols like TCP are built on top of I2P’s native tunnels by binding together an inbound and outbound tunnel. In theory the intermediaries using in the routing could perform mix activities but I haven’t seen any sign that they actually do so in the documentation.
I2P uses a completely distributed system called netDB for discovering potential intermediaries. netDB is an implementation of Kademlia and all members of I2P can potentially become part of netDB if they determine that their services are needed. netDB stores two types of data, routerInfo and leastSet. Routerinfo identifies potential intermediaries. leaseSet identifies all types of services that are available for use. There are also encrypted leaseSets that can only be read by those who have the appropriate key, this is used to advertise services to a limited subset of I2P users (i.e. ones with the key).
All services in I2P are identified by keys and routes to them can be built using netDB to discover availability. There is an address book service that tries to build up an address book of names/services/keys based both on the local app and on the address books of any identified friends.
There do exist both in and out proxies to make I2P services available to the open Internet.
I2P has a catalog of services built on top of it including bit torrent, websites, chat, etc.
It does not appear to be a goal for I2P to hide that someone is using I2P, only to hide what they are doing with I2P.
3.2.1 Writing apps for I2P
I2P provides a rich set of options for development. They have custom libraries, tunnels, etc. The predominant language seems to be Java.
4 Friend to Friend Networks
Freenet is a distributed storage service. A user creates a file, encrypts it, generates a hash that identifies it and uploads it to Freenet. The file is broken into pieces and multiple copies of those pieces are then distributed amongst all the members of Freenet. To get a particular file one needs to know its key and ask for it via the DHT built across all the Freenet members. Freenet actually supports a variety of discovery scenarios. One can upload keys that are signed with a public key (for validation) and include human readable strings. One can then issue queries against those keys in order to find them on Freenet. Freenet also supports a key type that supports updating content by identifying a key that is allowed to do the updating.
There is no direct way to delete content posted to Freenet. Instead content is kept distributed in multiple copies across multiple user’s caches. When a particular user’s cache fills up it will discard the least recently used content. So if no one asks for content for long enough it will eventually be flushed from the system.
Note that even with the ’update’ capability previously mentioned there is still no direct delete feature. The update basically creates a pointer that can be changed but the content that was previously pointed to will remain in the system until flushed.
Freenet runs what amounts to an almost accidental onion routing. Freenet consists at its heart of a huge distributed hash table. When inserting or retrieving data one does a search through the DHT across many machines looking for the right places to read/write the data. The nodes the search goes through act like an onion route thus providing privacy for the writer/reader.
What makes Freenet a friend to friend network is that it has a ’darknet’ mode where one will only connect to previously identified friends.
4.1.1 Writing apps for Freenet
Freenet has a Java based server that manages relationship with the Freenet infrastructure. There is a developer API for plugins as well as implementations of the FreeNet Client Protocol v2 in various languages.
OneSwarm is an extension of BitTorrent. One can connect to friends and see all of their content and can download the content through friends. There is a Kadmelia type DHT however to discover where one’s friends are at the moment. When dealing with strangers OneSwarm’s file discovery and posting behavior looks a lot like FreeNet.
4.2.1 Writing apps for OneSwarm
I’m sure there is an API in there somewhere but honestly OneSwarm feels more like an app than a platform.
This has a similar vibe to OneSwarm and Freenet’s darknets in the sense that when dealing with people who are friends there is no privacy or traffic analysis protection. There is a distributed hash table which is used instead of DNS so one can publish ones locations for friends to find. They currently use BitTorrent’s DHT. There is a relay capability for both searches and data transfers that gives an onion router type effect but it seems to work through common friends and while it’s supposed to provide end to end anonymity it doesn’t seem to provide hop by hop anonymity. This lack of hop by hop anonymity is a signature of friend to friend networks.
I found it difficult to understand RetroShare’s over all architecture because I failed to find a doc that explained it. But it seems like when files are uploaded they can either be pushed directly to friends or made anonymously available. It also seems like when someone downloads a file its existence on their machine is anonymously announced (just like Freenet or OneSwarm) and thus is available for downloading. RetroShare then uses a BitTorrent like strategy for pulling parts of desired files from multiple machines. So the more people like a file the faster it will be to get the file because it will be available from more sources.
4.3.1 Writing apps for RetroShare
There are a number of existing apps such as chat, group chat and general file sharing. There is a site with libraries and such for RetroShare.
5 Mix Networks
Note: Mixminion seems fairly dead. Its dev archive has no posts since 2011 and the last github update was 2 years ago. I’m including it mostly because it’s really interesting.
This is a third generation of remailers used to allow for traffic analysis resistant e-mail, its explicitly a successor to Mixmaster. Mixminion runs over SMTP where all connections are encrypted with TLS over TCP. Ephemeral keys are used to provide forward secrecy. Senders can use onion routing to determine which mixes will process their messages. A mixer receives groups of message blocks (which are reasonably small, so large e-mails are broken up into multiple blocks) and then mixes them together within the mix and sends them out in a different order when a minimum number of pieces have been received.
Mixminion has an explicit goal of allowing non-anonymous senders to send to anonymous receivers therefore Mixminion supports exposing itself as an incoming SMTP relay.
Mixminion also wants to allow anonymous senders to send to non-anonymous receivers. So Mixminion will act as an outgoing SMTP relay.
Mixminion also wants to support both sender and receiver anonymity. This appears (I think) to require the use of Nym servers which haven’t actually been implemented yet.
What Mixminion has implemented are SURBs or single-use rely blocks. These are partial encrypted paths that need to ’somehow’ be transmitted to those who want to send messages where they don’t know who you are. The SURB allows them to create messages that will eventually be transmitted to you without actually knowing who or where you are. But how the sender got the SURBs is currently undefined.
Mixminion has a directory server infrastructure to discover what mixes are available.
5.1.1 Writing apps for Mixminion
It has a command line client and server so presumably one could use the client via the command line interface to send and receive Mixminion mail.
Based on a Kadmilla style distributed hash table to store files. It supports searching by files in the system via file name, file name in a namespaces, a general search query or even looking for files on a particular machine (if you know its ID). One can choose to route requests via multiple nodes (onion style) and also specify how much traffic the node has to handle before forwarding requests (mix style). There is an internal metric system to measure ’good’ versus ’bad’ contributors and give them differential service. But the measurements are made locally not via a global watcher so they can be gamed. They use a proof of work system (which apparently can take a few days to generate on a standard machine) in order to establish identity and prevent Sybil attacks.
5.2.1 Writing apps for GNUnet
The main APIs are all in C although there apparently is a Java library available.
6 Some applications built on top of these networks
This is a distributed forum application with its own forum protocol. What’s interesting is that it’s designed to hook into a multitude of underlying traffic analysis resistant networks including: Tor, I2P, Mixminion/Mixmaster and Freenet. In looking at the instructions I only saw a command for Freenet but I didn’t spend too much time searching.
An Android client that has its own distribute anonymous forum protocol build on top of BitTorrent run on top of I2P.
An IM client with its own protocol that runs over Tor.