Rails 3 and Memcached - Intelligent caching without expiration

I am implementing caching into my Rails project via Memcached and particularly trying to cache side column blocks (most recent photos, blogs, etc), and currently I have them expiring the cache every 15 minutes or so. Which works, but if I can do it more up-to-date like whenever new content is added, updated or whatnot, that would be better.

I was watching the episode of the Scaling Rails screencasts on Memcached http://content.newrelic.com/railslab/videos/08-ScalingRails-Memcached-fixed.mp4, and at 8:27 in the video, Gregg Pollack talks about intelligent caching in Memcached in a way where intelligent keys (in this example, the updated_at timestamp) are used to replace previously cached items without having to expire the cache. So whenever the timestamp is updated, the cache would refresh as it seeks a new timestamp, I would presume.

I am using 开发者_运维知识库my "Recent Photos" sideblock for this example, and this is how it's set up...

_side-column.html.erb:

<div id="photos"">
   <p class="header">Photos</p>
   <%= render :partial => 'shared/photos', :collection => @recent_photos %>     
</div>

_photos.html.erb

<% cache(photos) do %>
  <div class="row">
    <%= image_tag photos.thumbnail.url(:thumb) %>
    <h3><%= link_to photos.title, photos %></h3>
    <p><%= photos.photos_count %> Photos</p>
  </div>
</div>

<% end %>

On the first run, Memcached caches the block as views/photos/1-20110308040600 and will reload that cached fragment when the page is refreshed, so far so good. Then I add an additional photo to that particular row in the backend and reload, but the photo count is not updated. The log shows that it's still loading from views/photos/1-20110308040600 and not grabbing an updated timestamp. Everything I'm doing appears to be the same as what the video is doing, what am I doing wrong above?

In addition, there is a part two to this question. As you see in the partial above, @recent_photos query is called for the collection (out of a module in my lib folder). However, I noticed that even when the block is cached, this SELECT query is still being called. I attempted to wrap the entire partial in a block at first as <% cache(@recent_photos) do %>, but obviously this doesn't work - especially as there is no real timestamp on the whole collection, just it's individual items of course. How can I prevent this query from being made if the results are cached already?

UPDATE In reference to the second question, I found that unless Rails.cache.exist? may just be my ticket, but what's tricky is the wildcard nature of using the timestamp...

UPDATE 2 Disregard my first question entirely, I figured out exactly why the cache wasn't refreshing. That's because the updated_at field wasn't being updated. Reason for that is that I was adding/deleting an item that is a nested resource in a parent, and I probably need to implement a "touch" on that in order to update the updated_at field in the parent.

But my second question still stands...the main @recent_photos query is still being called even if the fragment is cached...is there a way using cache.exists? to target a cache that is named something like /views/photos/1-2011random ?

---

One of the major flaws with Rails caching is that you cannot reliably separate the controller and the view for cached components. The only solution I've found is to embed the query in the cached block directly, but preferably through a helper method.

For instance, you probably have something like this:

class PhotosController < ApplicationController
  def index
    # ...

    @recent_photos = Photos.where(...).all

    # ...
  end
end

The first instinct would be to only run that query if it will be required by the view, such as testing for the presence of the cached content. Unfortunately there is a small chance that the content will expire in the interval between you testing for it being cached and actually rendering the page, something that will lead to a template rendering error when the nil-value @recent_photos is used.

Here's a simpler approach:

<%= render :partial => 'shared/photos', :collection => recent_photos %>

Instead of using an instance variable, use a helper method. Define your helper method as you would've the load inside the controller:

module PhotosHelper
  def recent_photos 
    @recent_photos ||= Photos.where(...).all
  end
end

In this case the value is saved so that multiple calls to the same helper method only triggers the query once. This may not be necessary in your application and can be omitted. All the method is obligated to do is return a list of "recent photos", after all.

A lot of this mess could be eliminated if Rails supported sub-controllers with their own associated views, which is a variation on the pattern employed here.

---

As I've been working further with caching since asking this question, I think I'm starting to understand exactly the value of this kind of caching technique.

For example, I have an article and through a variety of things I need for the page which include querying other tables, maybe I need to do five-seven different queries per article. However, caching the article in this way reduces all those queries to one.

I am assuming that with this technique, there always needs to have at least "one" query, as there needs to be "some" way to tell whether the timestamp has been updated or not.