EGTP

Introduction

EGTP -- Evil Geniuses Transport Protocol -- is a communications library originally developed by Evil Geniuses For A Better Tomorrow for use in their Mojo Nation product. The Evil Geniuses released the library as open source (LGPL) and it has since been maintained by volunteer hackers from around the world. EGTP serves as the comms engine for the Mnet project.

EGTP uses one or more underlying communications protocols to deliver messages between nodes, but it does not depend on the underlying protocol to provide any guarantees with respect to reliability, ordering, bidirectionality, or quality of service. Because EGTP does not need such guarantees, and because EGTP can dynamically select among the available underlying protocols on a per-message basis at runtime, it provides a very flexible communications abstraction which can be deployed atop a variety of communications systems.

Reliability, streaming, and other features are implemented by the higher-level code in Mnet, using sophisticated techniques including swarming distribution and the "self-healing" property of the Mnet peer to peer network.

EGTP serves several purposes:

Message Formats

Wire Encoding

At the lowest level, EGTP messages are all s-expressions, as defined in Internet Draft draft-rivest-sexp-00.txt. This is a human-readable, 8-bit-clean, structured format which is simple and efficient to produce and to parse. It is therefore nearly as efficient in time and space as customized binary encodings or ASN.1, while being as flexible, and nearly as human-readable, as XML.

Message Header

Each message header contains the unique id of the public key of the sender and recipient, and the type of message.

The EGTP comms library maintains an in-memory "routing table" of which underlying communication protocol, and which address within the context of that protocol, is used to send messages to the counterparty which holds the identified public key. (This combination of an underlying communication protocol and an address is called a "comm strategy". Please see messages_overview.html. In addition, if the attempt to deliver the message fails, or if the routing table doesn't contain an entry for that public key, the EGTP library queries a "MetaTracker" in order to find out the current comm strategy for that recipient.)

Note that in future versions of EGTP, the MetaTracker will be replaced with a fully decentralized Distributed Hash Table, a la Chord or Kademlia.)

Transaction Header

Each message header contains a unique nonce which serves to identify which transaction (that is: which query/response pair) includes the message.

Each response message contains the cryptographic hash (using the SHA1 algorithm) of the initiating message to which it is a response. Since a cryptographic hash is collision-free, this guarantees that the recipient can identify which request any given response message answers, even if some of the nodes on the network were acting maliciously and attempting to perform "active attacks" against other nodes.

Note that this technique can be used by the node which initiates the transaction, to protect itself from active attacks, but it does cannot be used by the other node -- the one which receives the initial message. Some of the Mnet hackers have designed a variation on the protocol which could protect both sides, but it has not yet been implemented.

Message Body

Each message contains arbitrary data determined by the specific message type (e.g. "do you have blocks", "request block", "pass this along", etc.). The format of each type of message is documented in messages_overview.html and specified in concrete detail in OurMessages.py. (Despite the ".py" extension, you can read that file without knowing Python, as it contains only a set of templates specifying the names and types of all of the components of each kind of message body.)


Zooko, mnet-devel mailing list
Last modified: Thu Jul 25 03:14:01 ADT 2002