Mnet coding standards

This document is current as of Mnet Mnet v0.7.

CVS revision: $Id: coding_standards.html,v 1.124 2004/07/17 11:33:25 arnowa Exp $

Here are some Python code style guidelines. We also use some other official Python guidelines by reference (see official Python standards below). Our Mnet-specific standards override the official standards whenever there is a conflict.

Mnet-specific guidelines:

basic standards

compatibility

Mnet requires Python v2.3 or greater. No effort should be made to offer compatibility with versions of Python older than 2.3. Effort should be made to work with the most recent release of Python, and with every release between v2.3 and the most recent release.

naming and layout

comments, idioms, miscellany, license

Here is a useful template for starting new Python files:

# Copyright (c) 2003 Bryce "Zooko" Wilcox-O'Hearn
# mailto:zooko@zooko.com
# See the end of this file for the free software, open source license (BSD-style).

"""
XXX doc string describing the module here
"""

__version__ = "$Revision: 1.124 $"
# $Source: /cvsroot/mnet/www/coding_standards.html,v $

# XXX import Python Standard Library modules here

from pyutil.assertutil import _assert, precondition, postcondition
from pyutil.debugprint import debugprint, debugstream
from pyutil.humanreadable import hr
# XXX import other pyutil modules here

# XXX import from other libraries, with a blank line between each library

# XXX your code here

# Copyright (c) 2003 Bryce "Zooko" Wilcox-O'Hearn
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software to deal in this software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of this software, and to permit
# persons to whom this software is furnished to do so, subject to the following
# condition:
#
# THIS SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THIS SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THIS SOFTWARE.

truths and falsehoods

modules

advanced idioms

preconditions and assertions

basic preconditions and assertions

Make sure you have from pyutil.assertutil import _assert, precondition, postcondition in your imports (as shown in the template above). Now design preconditions for your methods and functions, and assert them like this:

def oaep(m, emLen, p=""):
    precondition(emLen >= (2 * SIZE_OF_UNIQS) + 1, "emLen is required to be big enough.", emLen=emLen, SIZE_OF_UNIQS=SIZE_OF_UNIQS)
    ...

Notice how you pass in any values that ought to be printed out in the error message if the assertion fails -- in the example, we pass the values emLen and SIZE_OF_UNIQS. You can pass these as normal args or keyword args. If you use keyword args then the name of the argument will also appear in the error message, which can be helpful. For example, if the assertion above fails, then a debug message will appear at the end of the stack trace, like this:

>>> oaep("some secret thingie", 20)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 2, in oaep
  File "/home/zooko/playground/pyutil_new/pyutil/assertutil.py", line 47, in precondition
    raise preconditionfailureexception
AssertionError: precondition: emLen is required to be big enough. -- emLen: 20 :: <type 'int'>, 'SIZE_OF_UNIQS': 20 :: <type 'int'>

The "error message" that will accompany a failed expression should be a statement of what is required for correct operation. Don't write something like "Spam isn't firm.", because that is ambiguous: the error could be that the spam is supposed to be firm and it isn't, or the error could be that spam isn't supposed to be firm and it is! A good practice is to always use the words "required to" in your message, for example "Spam is required to be firm.".

class invariants

If your class has internal state which is complicated enough that a bug in the class's implementation could lead to garbled internal state which could in turn lead to misbehavior, then you should have a class invariant. A class invariant is a method like this (an actual example from BlockWrangler, but truncated for space):

def _assert_consistency(self):
    # All of the keys in all of these dicts must be ids.
    for d in (self.bId2chunkobj, self.bId2peers, self.Idsofwantedblocks, self.Idsoflocatedblocks,):
        _assert(not [key for key in d.keys() if not idlib.is_id(key)], "All of the keys in these dicts are required to be ids.", listofnonIds=[key for key in d.keys() if not idlib.is_id(key)])

    # For each (peer, blockId,) tuple in peerclaimedblock, if the peer *has*
    # claimed the block, then the blockId must appear in bId2peers[blockId],
    # and if the peer has claimed *not* to have the block then the blockId
    # must *not* appear in bId2peers[blockId].
    for ((peer, blockId,), claim,) in self.peerclaimedblock.items():
        _assert((claim == 1) == (peer in self.bId2peers.get(blockId, ())), "The blockId must appear in bId2peers if and only if the peer has claimed the block.", claim=claim, peer=peer, bId2peersentry=self.bId2peers.get(blockId, ()))

Now you can put assert self._assert_consistency() everywhere in your class where the class ought to be in an internally consistent state. For example, at the beginning of every externally-callable method. This technique can be very valuable in developing a complex class -- it catches bugs early while isolating them into specific code paths and it disambiguates the internal structure of the class so that other developers can hack on it without subtle misunderstandings.

list comprehensions

If you are writing code to generate a list, consider using Python's list comprehension syntax. Consider writing resultlist = [x for x in inputlist if x > 3] instead of

resultlist = []
for x in inputlist:
    if x > 3:
        resultlist.append(x)
.

configuration

minimizing configuration

how to implement configuration

Whether in application code or in library code, never pass configuration values via a "conf dict". Instead use Python parameters. For example, do not write

class BlockStore:
    def __init__(self, confdict={}, recoverdb=True, name='*unnamed*'):
        if confdict.has_key('MAX_MEGABYTES'):
            self.maxspace = (2**20) * int(confdict.get('MAX_MEGABYTES'))
        else:
            self.maxspace = None
        self.basepath = os.path.abspath(confdict.get("PATH", ""))
        self.maintainertype = confdict.get("MAINTAINER", "rnd").lower()
        self.backendtype = confdict.get("BACKEND", "flat").lower()

blockstore = BlockStore(confdict)
, but instead
class BlockStore:
    def __init__(self, maxspace=None, path="", maintainertype="rnd", backendtype="flat", recoverdb=True, name='*unnamed*'):
        self.basepath = os.path.abspath(path)
        self.maintainertype = maintainertype
        self.backendtype = backendtype

maxspace = confdict.get('MAX_MEGABYTES')
if maxspace is not None:
    maxspace = int(maxspace) * 2**20
blockstore = BlockStore(maxspace, confdict.get('PATH', ""), confdict.get('MAINTAINER', 'rnd').lower(), confdict.get('BACKEND', 'flat').lower())
.

official Python standards

These are listed in decreasing order of priority, so if a point in one of the latter guidelines contradicts a point in one of the earlier ones, then go with the earlier. The Mnet-specific guidelines above override all else, of course.

PEP 290

PEP 290: Code Migration and Modernization

PEP 8

PEP 8: Style Guide for Python Code

PEP 257

PEP 257: Docstring Conventions


Zooko
Last modified: Sun Mar 7 12:33:46 EST 2004