开发者

how to speed up my couchdb views written with python

开发者 https://www.devze.com 2023-01-22 22:29 出处:网络
I\'m writing my couchdb views with python. I think it\'s not very ... pythonic to do something like def fun(doc):

I'm writing my couchdb views with python. I think it's not very ... pythonic to do something like

def fun(doc):
    import re
    # do something with re
    yield 1, 1

because re is for every document imported. putting import re a开发者_如何学运维t the beginning gives me an error (string should compile to a proper function), a

 del re

at the view's end makes re unavailable within the function.

so how can I avoid importing re again and again?


I got into the same trouble recently and solved it like this:

import re

def foo(doc, re_xx=re.compile("required-pattern")):
    # use re_xx
    yield 1, 1

del re


View server handles many documents with same context. So import re really imports module only while handles first document. All other map calls will only lookup sys.modules.


If you are using the re module, there may be things that could be done with your regular expressions and how they are compiled that would speed things up more than removing the module lookup would (dictionaries in python are extremely fast). I don't understand why you would want to del re at the end of the function, that will probably slow things down.

Do you have a way of timing this so that you can measure the improvements/regressions when you tune the view? Getting some timing established will make it much easier to tune.

0

精彩评论

暂无评论...
验证码 换一张
取 消