This document is current as of 2003-10-07.

Mnet Roadmap

Sometimes our goals are aren't readily apparent to those who don't check in with us often. We've put this document together to map out where we are and where we want to go.

what is in the current branch

Currently the HEAD branch has get and put functionality, but there is no integrated search mechanism. There are two separate networks in the HEAD branch -- a traditional MetaTracker-based one, and a Kademlia-based one. The MetaTracker network uses a set of "MetaTrackers" to tell one where other nodes are. This was a degree of centralization that we didn't like, so we created the ent network, which is a Kademlia-based distributed hash table. The ent network is implemented enough so that it should pass the "put and get a random block" unit test, but it doesn't pass yet due to some subtle bugs.

what we are shooting for in version 0.7

new EGTP code

Zooko has cleaned up the EGTP code, and will soon be replacing the old code with the new and cleaner code. Part of this clean up was removing the old mojo based micropayment system from the EGTP, now hooks for micropayments must be added at a higher level.

unit testing and documentation

This has come a long way. We now use pydoc to automatically generate documentation and unit tests.

going "headless" with Distributed Hash Tables

One of the key problems with our current architecture is that Mnet is dependent on metatrackers. You can think of MetaTracker as the great phone book in the sky, who knows how to get in contact with all the Mnet nodes. If these nodes where shutdown (there are two of them at the time of this writing) Mnet would cease to function.

Several papers have been written, like Chord and Kademlia, that suggest possible solutions to the problem of a distributed lookup system.

We can use EGTP's (our communication protocol stack's) new interface for lookup and discovery services, so that different systems can be used with minimum of fuss.

We currently have one of these new systems in the works. The ent network is based on the Kademlia DHT, and is currently being worked on quite a bit.

Another solution is Achord, which is attractive because it minimizes how much one node knows about other nodes in the system. This is yet a twinkle in icepick's eye.

NAT traversal

There are two possibilites for NAT traversal, using the new ent network to route around firewalls, and using UPnP to open up ports on them. And why choose one when we can do both?

Twisted

The Twisted networking infastructure offers many intriguing features such as: trial (for unit testing), PB (for remote procedure calls), and integrated SSL. Also, their reactor core should make unit tests easier too. All in all moving would be a "good thing" and there is a branch underway to do just that.

CP2PC/Tristero interface

In the next version the GUI will run in another process than the network code. In this way the Mnet node can run in the background all the time, offering whatever services it wants. The GUI will communicate to the node via XML-RPC (with similar Twisted's Perspective Broker interface later) using the CP2PC/Tristero interface. This will also allow command line script to be easily created (no more web ui parsing). Hopeful other p2p systems will also create Tristero type interfaces.

browser integration

We'd like web browsers to understand Mnet URIs.

info on mozilla

for later versions of Mnet

replacing the configuration system

We'd like to replace our home-grown confman system with the Python Standard Library's ConfigParser, or perhaps with Twisted's equivalent config file tool.

new optional micropayment system

Previously we were hoping to work with the OpenDBS people to use their system for payments. But it seems that OpenDBS is vaporware, so we are now going to have to roll our own. This is a bit tricky since there are some patent issues around this. But never fear, this is everyone's ultimate goal, so we'll figure something out. [It's not my ultimate goal, although my ultimate goal and this one have some things in common and might help each other. --Zooko]

Ideally the software that handles withdrawing and storing tokens (the wallet) will be a separate piece of software running in another process. Those not interested in micropayments will simply not install the wallet software. Mnet will keep the Paris Metro Pricing system, meaning those that pay for their blocks/content searches will get priority over free transactions at servers configured to take payments.

search system

The search system will be reworked to be based on RDF triples. Each database subject will have a limited number of attributes associated with it, but what those attributes are will be flexible. This will allow the apps that are written to use Mnet a lot more flexibility about what can be searched.

Two attributes that will be mandated to be on all content tracker submissions: size (in bytes) and the file's bitprint.

We should make use of a metadata extractor lib like mmpython rather than our own.

for more information

See the issue tracker, which is where we keep track of specific tasks, their status and priority, and who is responsible for them.

See also the mailing list, which is where we discuss more general strategy, current events, and design issues.

See also the IRC channel, which is where we settle all of the age-old questions of theology, sex, and politics.