*scottstuff*

Time-limited caching for Rails

Posted by Scott Laird Fri, 20 Jan 2006 17:25:02 GMT

I’m finally getting back into Typo hacking after too long away. I tried to apply a few patches last weekend, but I was traveling and my network access was too spotty. So, I spent my time adding a bit of new functionality. Since then, I’ve been debating whether to commit it to Typo or not. I decided I’d write about it here and see which way the comments go.

The code in question is a time-limited cache for Rails. I’d like to be able to say “cache this page, but only for three hours. After three hours, re-render the page.” This sort of thing comes up in Typo all the time. The most obvious example is the sidebar–some of the sidebar components display information with a short lifetime, and it’d be dumb to keep pages in the cache for weeks when they include sidebar data that’s only good for hours. This isn’t really a problem on busy sites, because the current cache sweeper usually resorts to sweeping the entire cache every time a new article is posted, but it’s a pain on slower sites.

There are certainly other ways to fix the sidebar problem (AJAX sidebars are the obvious example), but the same basic pattern comes up all over Typo. A few examples:

1. Users keep requesting the ability to create articles with a publication date in the future. The article won’t appear on the site until after the publication date. This is common CMS feature, and apparently other blog engines have it as well, but it really doesn’t mesh with Typo’s current cache, because there’s no way to say “sweep the cache at 7:30 today” short of adding a cron job for every article that’s posted this way.

2. We have a bunch of aggregation classes that suck data off of other sites, like Flickr, Upcoming.org, and so on. These usually end up as sidebars, but we need to cache the back-end data somewhere. An expiring fragment cache would work perfectly for this.

3. On really busy sites, we could use something like this to avoid rebuilding comment pages on every comment–we could drop the sweep-on-new-comment code and swap for expire-after-5-minutes. If you’re getting more then 1 comment every 5 minutes, this would be a win. If you’re getting a comment every few seconds (think Slashdot or Curt Hibb’s “hammer my comments” post), this would be a major win.

To accomplish this, I added two new features. First, I added a set of “meta-fragment cache” methods, building on Rails’ existing fragment cache. The fragment cache stores (key, value) pairs, while the meta-fragment code stores (key, value, metadata_hash) triples. This is simply implemented as two fragment cache entries, one for the data and one for the serialized metadata hash.

Then, on top of that, I re-implemented my caches_action_with_params code. This is a variant of Rails’ native action cache with a number of cleanups and bugfixes.

When all is said and done, you’re left with a controller that looks something like this:

class ArticleController < ApplicationController
    caches_action_with_params :read

    def read
      response.lifetime = 3600 # 1 hour
      ...
    end
end

That’s it–the read action will now be cached with a 1 hour lifespan. After an hour, the cached version will expire. If response.lifetime isn’t set, then the page won’t expire on its own, and it’ll need to be swept as usual.

So here’s the big question–should this go into Typo? I can see good arguments on each side.

Pro:

• It solves a lot of cache-with-parameter problems that we’ve had.
• Switching to some variant of the action cache means that switching between production and development mode doesn’t leave cache problem. This is a major cause of bug reports from new users.
• It’ll let us implement future posting easily.
• It’ll make it easy for sidebars to stay current.
• It’ll let us move the aggregation backends behind the sidebars to a more reasonable architecture. For example, we’ll be able to use the Flickr class for the Flickr sidebar instead of (mis-)parsing their RSS feed.
• It’ll make us less dependent on web server configuration and weird rewrite rules.

Con:

• It’s slower then the page cache. I haven’t benchmarked my new code yet, but the last time I checked, on my box I could handle almost 2400 page cache requests per second, while the action cache was good for *10* hits per second. That exposed a couple major Typo performance bugs; I suspect that retesting with the new code would give us 100-200 hits/second, which is pretty busy for a blog. Still, this may be an issue for shared hosting providers.
• The action cache serves cached pages via Rails, while the page cache serves the same pages directly from the webserver without invoking Rails at all. Because of this, I suspect that a lot of sites will want to increase the number of FastCGI Typo processes that they run. With the page cache, running with one FCGI process was usually okay; with the action cache, it might be better to use a second process.

Those are the only two major problems that I see. Basically, if we switch to using the action cache (in any form), we’re going to be harder on big hosting companies like TextDrive and Planet Argon, and they’ve been very supportive of Typo in the past.

Does anyone feel strongly about this one way or another?

Tags caching, ruby, rubyonrails, typo | 11 comments

Benchmarking Typo

Posted by Scott Laird Mon, 17 Oct 2005 16:51:13 GMT

I finally had a bit of time to do some Typo benchmarking over the weekend and (as usual) found that my instincts were all wrong.

I was specifically interested in the performance difference between the page cache and the action cache–my guess was that the action cache was a 10x performance hit.

So I set up a test environment under Xen, running Typo r683, PostgreSQL, Apache 2, FastCGI, Ruby 1.8.2, and Rails 0.13.1. I didn’t do any Apache or Postgres tuning–I just ran them out of the box.

Then I ran ab against a snapshot of scottstuff.net from a couple weeks ago. I used the index page for my testing, as it’s a fairly large page and I wanted to give Typo a real workout.

Here’s what I found:

That really wasn’t what I’d expected. The action cache underperformed my expectations by a factor of 20.

I then did a bit of experimentation. I created a new uncached action in my test Typo setup that did nothing but render :text => 'foo', :layout => false, just to see if the caching system was slowing things down. Result? 10 requests/second. Then I created a new Rails project from scratch and added a new controller with the same action, and saw the same results. Still 10 requests/second.

However, in Rails’s logs, it said that it handled the request in 2 ms, and I should be seeing 500 hits/sec for the “foo” page. So something is adding an extra 98 ms to each request. I’m still hunting for this–I don’t know if it’s something Xen-related on my system, an artifact of my Apache config, or what.

I’ve tried upgrading to Rails 0.14.0, but that’s a whole other article.

Conclusions:

1. The page cache is really, really fast.
2. The action cache is a substantial improvement over the uncached case–about 10x on this system–but can’t touch the performance of the page cache.
3. Changing the concurrency settings on ab and/or the number of FastCGI backends in use didn’t make a substantial performance difference. Settings 2-15 gave roughly the same results.
4. My caches_action_with_params is slightly faster then the stock action cache.
5. Moving the action/fragment cache from the FileStore (Typo default) to MemoryStore gives no real performance boost. Moving to the MemCacheStore is a substantial performance hit (~2 hits/sec vs 10 hits/sec).
6. Adding a new uncached action in ArticlesController that simply returns a fixed string (render :layout => false, :text => 'foo') is no faster then the action cache.
7. Moving the new action from the previous step to a Controller of its own doesn’t help.
8. Removing all of the routes except the default :controller/:action/:id route doesn’t help, either.

Frankly, on this hardware, I don’t seem to be able to get more then 10 requests/sec out of Rails no matter what I do. I’m pretty sure that this is a mistake, so I’ll post a followup when I figure out what’s wrong.

Update 1: Another datapoint. Running an example Ruby FCGI on this box gives me 832 hits per second. So whatever the problem is, it’s not fundamental to Ruby FCGI on this box. So I need to look into Rails and see what’s happening.

Update 2: Making some progress. Apparently I screwed up when I tested a new, standalone Rails FCGI app before (forgot to restart FCGI?). This time, I got 130 req/sec, which is a vast improvement over the 10 req/sec that I was seeing before. Most of that speed hit seems to come from using Postgres for session storage. Unfortunately, even after making that change, my null controller is still only getting 40 req/sec with Typo. Transporting the same controller to a blank Rails project gives me 130 req/sec with the same code. Watching strace, it looks like something is forcing Typo to reload the digest/md5 module for every hit. Unfortunately, I can’t figure out how that’s happening–my environment.rb is identical between the two trees, as is my database.yml. I’ll get back to this later; I have other things that I need to finish today.

Posted in Typo | Tags apache, benchmark, caching, rubyonrails, typo | 9 comments

Rails caches_action_with_params

Posted by Scott Laird Tue, 04 Oct 2005 15:54:24 GMT

One of the big problems with caching in Rails is the way that Rails’s caching systems handles query parameters. Page caching completely screws this up–the page cache will turn /articles/read?id=100 into /articles/read.html, and Apache will then hand all future hits on /articles/read off to that static HTML file, even if the user was looking for /articles/read?id=99. You can mostly get around this by making sure that you always use named parameters via Rails’s routes, but even then a malicious user can do weird things to your cache by feeding query parameters via ?.

The action cache is slightly better, but it’ll still misbehave with the examples above. What we really need is a caching system that pays attention to all parameters, not one that ignores all of them that aren’t part of a route.

Towards that end, I’ve created caches_action_with_params. It’s a minor derivative of caches_action with a different fragment cache key; instead of using the URL (as generated by url_for), it ignores URLs completely and uses ACTION_PARAM/<host>/<controller>/<<action>/<params>. This way caching isn’t dependent on routing, which will help with some of the stranger problems that Typo has seen. On the downside, if your actions explicitly check the URL that the user used, then caches_action_with_params won’t work for you.

Once I’m off the train and sitting somewhere with usable IP, I’ll post some sample code to the Typo bug tracker and generate a couple benchmarks. I expect this to about about 10% as fast as the page cache, but it should still be faster then 100 hits/second, which is my personal definition of “fast enough” this week. Then, if no one has any big complaints, I’ll commit this and switch off the page cache.

Once that is done, it’ll be fairly easy to add a lifespan to cached pages, so we can say “this page is only good for 2 hours” and have it regenerate automatically after that.

Tags caching, ruby, rubyonrails, typo | 2 comments

Rails caching presentation

Posted by Scott Laird Wed, 28 Sep 2005 15:53:57 GMT

I gave a short talk on caching with Rails last night at the Seattle.rb meeting. The short version is “the page cache is going to hurt a lot worse then you’d expect,” but anyone who read my previous article on caching should already know that. I did the slides for the talk with S5, which was new to me–I had planned on using Keynote, but it seems to have died in the year and a half since I last had a use for it. S5 worked well enough, although there were some formatting issues that kept popping up as the browser window size changed. By and large it was easy to use, and it’s nice to have a HTML version of the talk that doesn’t look like a nasty afterthought.

About halfway through preparing for the talk, I realized that I really need to add a new action cache option, something like caches_action_with_params, so we can explicitly say how query strings and other parameters affect the cache. Here’s a bit of sample code:

class ArticlesController < ApplicationController
  caches_action :index
  caches_action_with_params :read, :id
  caches_action_with_params :permalink, :year, :month, :day, :title

  def index
    @pages, @articles = paginate(
      :article, 
      :per_page => config[:limit_article_display], 
      :conditions => 'published != 0', 
      :order_by => "created_at DESC"
  end

  def read  
    @article = Article.find(
      params[:id], 
      :conditions => "published != 0", 
      :include => [:categories])    
  end

  def permalink
    @article = Article.find_by_permalink(
      params[:year], 
      params[:month], 
      params[:day], 
      params[:title])
  end

  ...
end

At least as of Rails 0.13.1, calling /articles/read?id=10 will create a cache entry for /articles/read, which is wrong, and then asking for /articles/read?id=20 will return the cached entry for id=10. Yes, the user is supposed to use routes for this, but explicit query params still work, and there are times when you really need to use them. Fortunately, this is really only 20 lines of code, so it shouldn’t be too hard to write.

Tags caching, ruby, rubyonrails, typo | 2 comments