The Great Typo Memory Leak
A number of users complained this weekend that Typo was using way too much memory, with reports of 100+ MB per FastCGI dispatcher. Typo usually uses around 20 MB, and even that’s too much; 100 MB is enough to cause big problems with hosting providers like TextDrive.
The first step that I took was to verify that the problem actually exists outside of TextDrive. I set up a test Apache/FastCGI/Typo server, disabled caching, and then pounded on it using curl
:
# while true; do curl http://typo1/ > /dev/null; done
I let that run for a few seconds and watched while my dispatch.fcgi
processes grew from 22 MB to 80 MB. I then did a bit of experimenting:
- The main index page leaked
- RSS feeds didn’t leak
- Individual article pages leak
- Static pages, like
/pages/about
leak - Error pages even leak
Disabling the layout for a leaking page and then re-testing it showed that the leak followed the layout. Turning layouts back on and removing the sidebar block fixed the leak.
Entertainingly enough, disabling the sidebar from inside of the sidebar infrastructure didn’t fix the leak. The mere act of calling render_component
to generate the sidebars seemed to be causing the memory leak. Since Typo is one of the very few users of Rails components, this suggests that render_component
may have a leak that no one else has noticed, so I created a new test Rails app with only two files. First, app/controllers/foo_controller.rb
:
class FooController < ApplicationController
def bar
render_component :layout => false,
:controller => 'sidebars/sidebar', :action => 'index'
end
end
Then components/sidebars/sidebar_controller.rb
:
module Sidebars
class SidebarController < ApplicationController
def index
render :text => 'test', :layout => false
end
end
end
This is about as minimal as a Rails app can get. Then I set up a FastCGI server running this project, and ran curl
against /foo/bar
, and watched the process size climb. So the leak is part of Rails, not really part of Typo.
Unfortunately, I’m not sure where the leak is coming from. I read component.rb
and made a few small changes, but the leak hasn’t stopped. So I’m going to file this as a Rails bug and see if we can get it fixed before 1.0.
Update: Rails bug 2589.
Update: Thanks to Scott Barron, the bug has been fixed. Users with memory problems should probably install the patch, although a bit of testing would obviously be recommended first. The next release of Rails (either 1.0rc3 or 1.0; I’m not sure what they’re planning) should include this fix.