This document is current as of Mnet v0.6. There are some small inaccuracies about the structure of some messages, but nothing that really changes the meaning.

Mnet Messages Overview

Note: the purpose of this document is to inform about the contents of Mnet network messages and their rationale at a high level of abstraction. It is a technical document and contains much technical fact; some of the details are glossed over or altered for clarity and simplicity. Do not use this document for an accurate specification of the actual messages down to the last bit. However, once you have understood the general idea from this document, it should be easy to understand the specific details from the well-documented reference source code, or from other more precise specification documents which have not yet been written.

See "OurMessage.py", "OurMessagesTokens.py", and "OurMessagesPublicKey.py" files for more detailed specifications. Those files contain templates used to validate the contents of the messages. They also provide a readable specification of the message formats and contents.

The messages in the Mnet system are grouped according to the major feature of the system that they implement: network maintenance, block search and retrieval, message relay service, and content tracking.

Conventions used in this document:

Representing data:

{ }: a table of "key: value" pairs separated by commas
[ ]: a list of items separated by commas
' ': a string literal
2358203 and other numbers: integers of arbitrary size
true and false: boolean values
'01CDEFGHIJKLMNOPQRSTUVWXYZA': a string literal that contains a 160-bit unique ID
"7H8J9K0L"...65536 bytes total..."A2D3G5H7": a string literal that contains a 65536 bytes

Message descriptions:

Each message description is in this format:

Message: message name from sender description to receiver description
Query: description of body of initiating message
Example: example of body of initiating message
Response: description of body of response message
Example: example of body of response message

Introductions: Building and Maintaining the Network

Introducing One Node To Another

contact info

Contact info contains an RSA public key, a list of one or more "comm strategies", and a list of zero or more services offered by the node. It also includes an incrementing counter (called a "sequence num") so that newer contact info can be used in preference to older. The version number of the node is included for debugging purposes.

Contact info is not actually a stand-alone message, it can be appended onto other messages. The "hello" message is a message that gets sent from nodes to metatrackers which contains nothing but contact info. In addition, any other message between any two nodes may also contain contact info for zero or more other nodes. (This piggybacking of contact info on other messages enables the network to adapt and heal during normal operaton without sending separate messages.)

Example:
{ 'pubkey': { 'key header': { 'cryptosystem': 'RSA', 'type': 'public', 'usage': 'only for communication security' }, 'key values': { 'public exponent': 3, 'public modulus': "8vJcUz7w"...128 bytes total..."JhJho1Wi" } } 'comm strategies': [ { 'comm strategy type': 'TCP', 'IP address': '140.184.83.25', 'port number': 20035 }, { 'comm strategy type': 'Relay', 'relay server ID: '098765432112345678901234567' } ], 'sequence num': 16, 'services': { 'block server': { 'centerpoint': 9249037, 'stddev': 2358203 }, 'relay server': true, 'content tracker': true } 'node version': '0.5.1.19-STABLE' }

Original Introduction

bootpage
The bootpage is a tiny text file served via HTTP used to bootstrap a node onto the system by providing it with the contact info for one or more meta trackers. Example: http://www.haknam.de/bootpage.txt

Message: "hello" from any node to a meta tracker
Query: contact info: information about how to send Mnet messages to the sender.
Example: A detailed example is shown above in the "Introductions" section.
Response: "success" or "failure" describes if the submission was successful or not.

Message: "goodbye" from any node to a meta tracker
Query: empty -- this is an advisory message which signals that the node is going off-line
Response: None.

Message: "lookup contact info" from any node to a meta tracker
Query: node Id -- the ID of the node which the sender wishes to contact.
Example: '01CDEFGHIJKLMNOPQRSTUVWXYZA'
Response: contact info: information about how to send Mnet messages to the node identrified in the request
Example: A detailed example is shown above in the "Introductions" section.

Message: "list block servers" from any node to a meta tracker
Query: empty -- a node that does not yet know about enough other nodes will send this query to find out about several other nodes to begin communicating with.
Response: a list of contact information for some block servers
Example: A detailed example of contact information is shown above in the "Introductions" section.

Block Search and Retrieval: Delivering Data

Message: "do you have blocks" from any node to a block server
Query: a list of block IDs that the client wants to locate
Example: [ '01CDEFGHIJKLMNOPQRSTUVWXYZA', 'aBcDeF00000L00O0Q0S0U0W0Y0A', '1B1D1F1H1J1L1N1P1R1T1V1X1Z1' ]
Response:: a list of block IDs that the server has available -- a subset of the block IDs included in the query
Example: [ '01CDEFGHIJKLMNOPQRSTUVWXYZA', '1B1D1F1H1J1L1N1P1R1T1V1X1Z1' ]

Message: "request block" from any node to a block server
Query: a single block ID that the client wants to download
Example: '01CDEFGHIJKLMNOPQRSTUVWXYZA'
Response: the actual data contents of the requested block
Example: { 'data': "7H8J9K0L"...65536 bytes total..."A2D3G5H7" }

Block Publication: Storing Data

Message: "put block" from any node to a block server
Query: data contents of a single block for the server to store
Example: { 'data': "7H8J9K0L"...65536 bytes total..."A2D3G5H7" }
Response: "success" or "failure" indicating if the server accepted the block

Relay: Routing Around Obstacles to Communication

Message: "pass this along" from any node to a relay server
Query: a recipient node Id and an encrypted message -- the relay server will forward the encrypted message (which is encrypted so that only the recipient can decrypt it) to the recipient
Example: { 'recipient': '1001001HIJKLMNOPQRSTUVWXYZA', 'message': '\000\000\000\001lgAG9a_S_\253@#^'...500 bytes...'lkahgAT_G\072HG@#' }
Response: a success or fail signal
Example: { 'result': 'success'}

Message: "are there messages" from any node to a relay server
Query: empty request -- This query is an empty message sent from a node to relay server to indicate that the node would like to have that relay server accept and deliver messages for it.
Response: a list of queued messages for relay -- This response acknowledges acceptance of the request for relay service. It also includes a list of previously queued up messages that could not be delivered directly for the node to pick up (this can happen if a node using a relay server goes offline for a couple of minutes).

Message: "message for you" from a relay server to any node
Query: an encrypted message -- intended for the receiving node and encrypted so that he can decrypt it. Relay servers will send messages that they received in "pass this along" messages to the destination node using this message if they have a way to communicate directly with them (otherwise they will hold the messages temporarily for pickup). The message can only be decrypted by the recipient.
Example: { 'message': '\000\000\000\001lgAG9a_S_\253@#^'...500 bytes...'lkahgAT_G\072HG@#' }
Response: a success or fail signal
Example: { 'result': 'success'}

Content Tracking: Providing a Directory of Available Content

Message: "content tracker submit" from any node to content tracker
Query: XML-encoded metadata describing content. This metadata includes the mnetId, which is the canonical key used to retrieve the data. The content tracker stores the entry in its databases for future lookup.
Response: a success or fail signal

Message: "content tracker lookup" from any node to content tracker
Query: an XML description of a database query on metadata
Response: a list of XML-formatted metadata for content entries that match the query, including the retrieval key (the mnetId)

Zooko, mnet-devel mailing list

Last modified: Mon Oct 6 08:16:09 EDT 2003