*scottstuff*

The Great Typo Memory Leak

Posted by Scott Laird Mon, 24 Oct 2005 19:30:55 GMT

A number of users complained this weekend that Typo was using way too much memory, with reports of 100+ MB per FastCGI dispatcher. Typo usually uses around 20 MB, and even that’s too much; 100 MB is enough to cause big problems with hosting providers like TextDrive.

The first step that I took was to verify that the problem actually exists outside of TextDrive. I set up a test Apache/FastCGI/Typo server, disabled caching, and then pounded on it using curl:

# while true; do curl http://typo1/ > /dev/null; done

I let that run for a few seconds and watched while my dispatch.fcgi processes grew from 22 MB to 80 MB. I then did a bit of experimenting:

• The main index page leaked
• RSS feeds didn’t leak
• Individual article pages leak
• Static pages, like /pages/about leak
• Error pages even leak

Disabling the layout for a leaking page and then re-testing it showed that the leak followed the layout. Turning layouts back on and removing the sidebar block fixed the leak.

Entertainingly enough, disabling the sidebar from inside of the sidebar infrastructure didn’t fix the leak. The mere act of calling render_component to generate the sidebars seemed to be causing the memory leak. Since Typo is one of the very few users of Rails components, this suggests that render_component may have a leak that no one else has noticed, so I created a new test Rails app with only two files. First, app/controllers/foo_controller.rb:

class FooController < ApplicationController
  def bar
    render_component :layout => false, 
      :controller => 'sidebars/sidebar', :action => 'index'
  end
end

Then components/sidebars/sidebar_controller.rb:

module Sidebars
  class SidebarController < ApplicationController
    def index
      render :text => 'test', :layout => false
    end
  end
end

This is about as minimal as a Rails app can get. Then I set up a FastCGI server running this project, and ran curl against /foo/bar, and watched the process size climb. So the leak is part of Rails, not really part of Typo.

Unfortunately, I’m not sure where the leak is coming from. I read component.rb and made a few small changes, but the leak hasn’t stopped. So I’m going to file this as a Rails bug and see if we can get it fixed before 1.0.

Update: Rails bug 2589.

Update: Thanks to Scott Barron, the bug has been fixed. Users with memory problems should probably install the patch, although a bit of testing would obviously be recommended first. The next release of Rails (either 1.0rc3 or 1.0; I’m not sure what they’re planning) should include this fix.