I ran into a bit of a conundrum this week when dealing with a view that required me to pass a very large Python dictionary to a template for navigation and toolbar purposes.  Because I’m working with a designer who is using AJAX GETs for entire portions of interactive content, this dictionary had a mixture of navigation, subnavigation and actual object JSON and was constructed via a fairly complex function.

Once the designer I was working with explained just how complex this dictionary (JSON in his terms) was going to be, I started to worry about the view — knowing that millions of simultaneous requests to a view like this would leave my server badly bruised.

Enter cPickle!  I realized that this navigation was not going to change after launch, so I could use cPickle to store it in the project’s data directory.  This allowed me to circumvent the open database connections and it sped up my view immensely.

def pop_nav():
    #code that populates dict
    fn = os.path.join(os.path.dirname(__file__), 'data/nav')
    f = open(fn,'w')

As you can see, I can run this in iPython anytime to update the nav, in case something should change. Here’s the section of the view that handles it.

def view(request,element):
    #code for various other pieces of view
    fn = os.path.join(os.path.dirname(__file__), 'data/nav')
    f = open(fn,'r')
    nav = pickle.load(simplejson.dumps(content_list),f)

Now it’s ready to send to the template. :)

Although I’m sure this is not quite as speedy as it would be if I were using a NoSQL DB to store all my objects, I highly recommend it for those projects that have large queries that will remain mainly static after launch.