Can't get Beaker cache working

I'm trying to use Beaker's caching library but I can't get it working.

Here's my test code.

class IndexHandler():
    @cache.cache('search_func', expire=300)
    def get_results(self, query):
        results = get_results(query)
        return results

    def get(self, query):
        results = self.get_results(query)
        return render_index(results=results)

I've tried the examples in Beaker's documentation but all I see is

<type 'ex开发者_运维知识库ceptions.TypeError'> at /
can't pickle generator objects

Clearly I'm missing something but I couldn't find a solution.

By the way this problem occurs if the cache type is set to "file".

---

If you configure beaker to save to the filesystem, you can easily see that each argument is being pickled as well. Example:

tp3
sS'tags <myapp.controllers.tags.TagsController object at 0x103363c10> <MySQLdb.cursors.Cursor object at 0x103363dd0> apple'
p4

Notice that the cache "key" contains more than just my keyword, "apple," but instance-specific information. This is pretty bad, because especially the 'self' won't be the same across invocations. The cache will result in a miss every single time (and will get filled up with useless keys.)

The method with the cache annotation should only have the arguments to correspond to whatever "key" you have in mind. To paraphrase this, let's say that you want to store the fact that "John" corresponds to value 555-1212 and you want to cache this. Your function should not take anything except a string as an argument. Any arguments you pass in should stay constant from invocation to invocation, so something like "self" would be bad.

One easy way to make this work is to inline the function so that you don't need to pass anything else beyond the key. For example:

def index(self):

    # some code here

    # suppose 'place' is a string that you're using as a key. maybe
    # you're caching a description for cities and 'place' would be "New York"
    # in one instance

    @cache_region('long_term', 'place_desc')
    def getDescriptionForPlace(place):
      # perform expensive operation here
      description = ...
      return description

    # this will either fetch the data or just load it from the cache
    description = getDescriptionForPlace(place)

Your cache file should resemble the following. Notice that only 'place_desc' and 'John' were saved as a key.

tp3 
sS'place_desc John' 
p4

---

I see that the beaker docs do not mention this explicitly, but, clearly, the decorate function must pickle the arguments it's called with (to use as part of the key into the cache, to check if the entry is present and to add it later otherwise) -- and, generator objects are not pickleable, as the error message is telling you. This implies that query is a generator object, of course.

What you should be doing in order to use beaker or any other kind of cache is to pass around, instead of a query generator object, the (pickleable) parameters from which that query can be built -- strings, numbers, dicts, lists, tuples, etc, etc, composed in any way that is easy to for you to arrange and easy to build the query from "just in time" only within the function body of get_results. This way, the arguments will be pickleable and caching will work.

If convenient, you could build a simple pickleable class whose instances "stand for" queries, emulating whatever initialization and parameter-setting you require, and performing the just-in-time instantiation only when some method requiring an actual query object is called. But that's just a "convenience" idea, and does not alter the underlying concept as explained in the previous paragraph.

---

Try return list(results) instead of return results and see if it helps.

The beaker file cache needs to be able to pickle both cache keys and values; most iterators and generators are unpickleable.