Apache tuning for Rails and FastCGI

Posted by Scott Laird Wed, 20 Jul 2005 08:23:36 GMT

There’s a surprisingly small amount of documentation out there on tuning Apache for optimum Rails performance. Almost everyone mentions the first step (use FastCGI, not regular CGI), but that’s such a huge performance boost that it’s really obvious–waiting 2-3 seconds per hit for Rails to start up is an indicator that you’re doing something wrong.

Once you get past that, there’s not a lot of documentation. There are examples from place to place, but no one seems to discuss what they mean or why they should be used.

Ever since I switched to Typo, I’ve been seeing occasional HTTP 500 errors from Apache, suggesting that Apache was unable to talk to Typo. Looking in the logs shows that Apache was usually in the middle of restarting a FastCGI instance whenever the errors occurred. Digging through the mod_fastcgi shows that FastCGI can work in three different modes with Apache:

  1. Static. FastCGI servers are started when Apache is reloaded and remain running.
  2. Dynamic. FastCGI server processes are started whenever a FastCGI URL is hit. Excess processes are killed off when there’s no traffic.
  3. External. Apache and your FastCGI app talk via TCP sockets.

Dynamic mode is the default, but that’s not a good fit for Rails, because of its slow startup time. Switching to static mode really helps. To do that, I added this line to /etc/apache2/apache.conf on my Debian server:

FastCgiServer /var/web/typo/public/dispatch.fcgi -idle-timeout 120 \
       -initial-env RAILS_ENV=production -processes 2

Notice that I had to list the full path to Rails’s dispatch.fcgi file; on some systems you may be able to get away with only listing public/dispatch.fcgi, but that will almost certainly not work if you’re using virtual hosting.

By default, FastCGI assumes that your server will respond to queries within 30 seconds. I added the -idle-timeout 120 parameter just so I can deal with really slow responses better. Typo’s article admin page currently tries to list all 466 articles on one page, and that can take over 30 seconds to process.

The -processes parameter tells Apache how many FastCGI processes should run for this application. For 95% of users, 1 or 2 will be best. If you get a lot of traffic, then raising this to 3-4x the number of CPUs in your system might get you slightly better performance.

Finally, the -initial-env bit makes sure that Rails runs in production mode, talking to my production DB and not returning error backtraces to the user.

Posted in , ,  | Tags , , , , ,  | 40 comments

Comments

  1. David Heinemeier Hansson said about 1 hour later:

    Good suggestions, Scott. There’s more on this topic in the Deployment chapter of the Rails book.

  2. Scott Laird said about 4 hours later:

    Blink.

    You know, you’d think I would have checked there, since I had the PDF open on my desktop. Sigh. There it is, too. Nice directions.

  3. Jason said 26 days later:

    So for each rails app I am running on the server I need to call that FastCGI line?

    like:

    FastCgiServer /rails_app_1/dispatch.fcgi -idle-timeout 120 -initial-env RAILS_ENV=production -processes 10

FastCgiServer /rails_app_2/dispatch.fcgi -idle-timeout 120 -initial-env RAILS_ENV=production -processes 10

    Thanks

  4. Scott Laird said 26 days later:

    Right

  5. Fabrice said about 1 month later:

    I have no apache.conf file on my Tiger System. Any idea where to put this line ?

    Thanx

    Fabrice

  6. Scott Laird said about 1 month later:

    Look in /etc/httpd/httpd.conf

  7. Rik said 3 months later:

    Installing Typo and Ruby/Rails on a Debian system with limited memory (around 180MB), using the static configuration above did give memory problems, so be careful.

    Users with limited memory and low traffic may want to opt for dynamic startup rather than static, or to reduce the number of instances, as Scott mentions.

    My Xen-based Linux server ground to a halt with 10 static processes. It consumed the 180MB of RAM in addition to its allocated swap.

    I must admit that I’m surprised that what is essentially a fancy CGI requires so much memory just to sit there waiting for connections.

  8. Scott Laird said 3 months later:

    You can safely reduce the number of processes down to 2 in almost all cases, and even 1 will be fine most of the time. I’ll edit the article and lower the example.

  9. will-c said 3 months later:

    thanks for the tips! I was running my production site for a day with the default dynamic setup and on my ancient 300mhz server it didnt seem to work too well.. was quite easy to get the HTTP 500.

    Now with two static processes things seem pretty sweet :)

  10. Chris Nolan.ca said 3 months later:

    I think this migth finally solve the problems I’ve been having the last few days. Thanks!

  11. Sebastian Gräßl said 4 months later:

    What about Vhosts? Setup for every Vhost a FastCGI Server?

  12. Scott Laird said 4 months later:

    You need one line for each distinct FCGI process that you want to run. So if you’re using 5 vhosts, each of which runs one instance of Typo (or any other Rails app), then you’ll need a total of five FastCgiServer lines.

  13. Kees said 4 months later:

    Hi Scott,

    I’m trying to run typo on MacOSX Tiger, Apache1.3. I’ve added the entries you suggested to the httpd.conf file hoping this would solve the problem I have getting typo up and running. However I still get the error: FastCGI: incomplete headers (35 bytes) received from server “/path/to/typo/public/dispatch.fcgi.

    Do you have any idea what could cause this?

  14. Kees said 4 months later:

    Hi Scott,

    Solved it. Problem had to do with the database configuration.

  15. matt said 4 months later:

    Kees, im also getting the same error as you

    FastCGI: incomplete headers (35 bytes) received from server “/path/to/typo/public/dispatch.fcgi.

    How’d you fix it ?

  16. matt said 5 months later:

    Has anyone had any success with the following;

    Windows XP box Apache 2.0 with FCGI and virtual hosts Rails 1.0 Latest Typo Install

    Ive followed all the advice I can on the net and it just doesnt seem to work. I still get this error;

    FastCGI: incomplete headers (35 bytes) received from server

    Using regular CGI with Apache 2.0 I have this error -

    Premature end of script headers: dispatch.cgi

    Any help at all would be appreciated; ?

    Matt

  17. Daniele said 5 months later:

    Not doing it on MacOS but on Linux. Anyway, rails seems to run fine with the WebBrick server.

    When using fastcgi I also get

    FastCGI: incomplete headers (35 bytes) received from server “/path/to/typo/public/dispatch.fcgi”

    And I can’t figure out why…

  18. Daniele said 5 months later:

    Ok, figured out why! It is the problem with sessions and session cookies described here (and in other places): http://wrath.rubyonrails.org/pipermail/rails/2005-December/005727.html

  19. matt said 5 months later:

    Hi

    Yes I got it to work - I un-installed both ruby and rails on my Windows system, then reinstalled (latest versions) Then edited my Apache - http.conf file to include the following lines for my typo installation;

    # matthewHutchinson (BloG)
# Configure FASTCGI for development
FastCgiServer "c:/rails/matthewhutchinson.net/typo/public/dispatch.fcgi" -idle-timeout 60 -initial-env RAILS_ENV=development -processes 1
<VirtualHost *:80>
    ServerName matt
    DocumentRoot c:/rails/matthewhutchinson.net/typo/public
    ErrorLog c:/rails/matthewhutchinson.net/typo/server.log
     <Directory c:/rails/matthewhutchinson.net/typo/public/>
      Options ExecCGI FollowSymLinks
      AllowOverride all
      Allow from all
      Order allow,deny
    </Directory>
</VirtualHost>

EnableSendfile Off
EnableMMAP Off
Win32DisableAcceptEx
FastCgiConfig -maxClassProcesses 1 -maxProcesses 1 -minProcesses 1 -processSlack 1

    I am of course using NamedVirtualHosting and I have modrewrite enabled and the following for modfastcgi

    # hanlders and modules for FASTCGI
LoadModule fastcgi_module modules/mod_fastcgi.so
<IfModule mod_fastcgi.c>
  AddHandler fastcgi-script .fcgi
</IfModule>

    My Typo, public/dispatch.fcgi has the ruby path set as;

    #!c:/ruby/bin/rubyw

    (but this will depend on where rubyw was installed on your system)

    Hope this all helps,

    Matt

  20. shanghaichris@gmail.com said 5 months later:

    Watching it..

  21. shanghaichris@gmail.com said 5 months later:

    the website returns 500 error why

  22. jim said 6 months later:

    Can anyone think of a reason why this would cause the server to always return the default index.html instead of calling the controllers?

  23. LarryK said 7 months later:

    This error message can also be caused by session cookie conflicts with another Ruby application from the same server.

    To check: Assumption: you have two apps, rubyapp1 and rubyapp2 on the same server.

    On a client machine: 1) quit your browser (all windows). 2) restart browser and try rubyapp1 If it now works then you may have the conflict in session id’s (restarting the browser clears the browser-session cookie cache) 3) now try rubyapp2 4) now try rubyapp1 again and be sure to reload from the server, not the browser’s page cache. On IE, use shift-reload. If it no longer works then you probably have the conflict

    The problem is that both apps are trying to store/restore the ruby session and both are using the default ruby cookie of sessionid

    SOLUTION Add ActionController::CgiRequest::DEFAULTSESSIONOPTIONS[:sessionkey] = ’rubyapp1’

    to end of environment.rb

  24. henning said 7 months later:

    and one more reason:

    not having installed the ruby fcgi bindings

    on Ubuntu, and probably debian:

    wajig install libfcgi-ruby1.8

  25. bmgz said 8 months later:

    You saved me from throwing in the towel and continuing having to code in PHP, Now my 400mhz server runs rails apps really quick!

  26. null said 8 months later:

    Some notes:

    1. modfcgid is better than modfastcgi for Apache 2.x for a number of reasons.

    2. each rails dispatch.fcgi instance will consume around 20MB RAM or more

    3. use only 1 dispatch.fcgi instance per physical CPU or CORE (unless you have plenty of extra RAM)

  27. Chris said 8 months later:

    If I use FastCGI and have one static process, I upload a larges files, I think that one FastCGI process is completely blocked while the file is uploading, so if someone else came to the site, they wouldn’t get anything until file finished uploading, is this correct?

    Is there a way to use dynamic with a min number of processes already started?

  28. Tyler Broadbent said 10 months later:

    Great find Daniele. This fixed the problem I was getting as well. (35 bytes). Before the 35 byte error I was getting a file not found execle() error for FastCGI in the apache logs: the solution to this problem was to create a new rails app on the server, then copy the public/.htaccess, dispatch.cgi and dispatch.fcgi to your existing rails app. Make sure to change the rewrite line in .htaccess from dispatch.cgi to dispatch.fcgi.

  29. Tyler Broadbent said 10 months later:

    UPDATE: After looking into it further, you DO NOT need to copy the dispatch.cgi/.fcgi files from a new rails app. The problem is that the path to ruby is not the same on the current system as the system the app was original developed on. To fix this, edit your dispatch.fcgi/.cgi files and change the first line to point to the correct ruby path: ie: #!/usr/bin/ruby18 (on gentoo)

  30. Rajesh S said 10 months later:

    Answering question for above issue. There are bunch of reasons why this could happen:

    • Open your dispatch.cgi/.rb file and make sure shebang is right. it should be #!c:/ruby/bin/rubyw for windows and #!/usr/bin/ruby for Mac OSX

    • Another reason, your /public directory needs to have chmod 750 directory access, do not make it as world writable

    • If nothing works, take the rails app that works, go to the public directory copy the complete public directory on your non working rails app public dir (offcourse take a backup first)

  31. Bob C said 10 months later:

    if anyone could help, my problem is outlined here:

    http://www.sitepoint.com/forums/showthread.php?t=383191

  32. jum said 11 months later:

    Because you need to delete index.html. And of course, have a map.connect ”, blabla in routes.rb. Hey, RTFM and the welcome screen!

  33. David Nitzsche-Bell said 11 months later:

    Hi,

    I had RoR working beautifully but because of an OS update snafu, had to reinstall OS and RoR. Piece of cake, I thought. Nope.

    I have installed the bare minimum according to Hivelogic: ruby 1.8.4 and rails, via gems.

    If I use script/server, the (new) Welcome to Rails is displayed and the Ajax-y link “About your apps environment” works.

    I can’t get it to work with Apache and virtual hosts.

    Anyone else been able to do that? I think it’s a conflict between Document Root and absolute/relative paths.

    Help?

  34. Philip said 11 months later:

    Excellent website! Many thanks! Congratulations! renesans

  35. ' AND 1=1; said about 1 year later:

    ‘AND 1=1;

  36. Emily said about 1 year later:

    Here is how I setup Apache 2.2, fastcgi and RoR.

    http://hack.emilykwan.com/node/95

  37. Fred said about 1 year later:

    It is so much faster now! It was slow before I added these lines, and I mean very slow… 20-40 seconds to render the page. Now at the most 5 seconds

  38. Fred said about 1 year later:

    Finally! Steve Longdo, found a great way to reduce mamory usage… from 60m to 25mb on my server.

    go here and try it yourself:

    http://www.stevelongdo.com/articles/2006/08/04/typo-4-0-and-memory-reduction

  39. Alex said about 1 year later:

    I installed typo 4 I don’t have the .htaccess file and I can’t run on apache2 + FastCGI

    Can do somebody show me yours config files?

    I’m sorry, my engish is poor

  40. Ashok Modi said about 1 year later:

    While this solution works very well, one thing I must mention is if you’re using Ruby to send files (may be done if each file has access to be downloaded on a user’s permissions). If you’re running with 2 fastCGI processes, then it basically means that if 2 people are downloading a large file (let’s say 100 megs), the fastCGI processes will be tied up and thus anyone else that tries to access the server will be granted a 404 error. As suggested above, you can add move fastcgi processes to the mix. Otherwise, you can have apache serve the files if there is no requirement for checking file permissions. I’m trying to figure out if there is a way to server files via apache while still keeping a way to grant restrictions on viewing the file. Any ideas?

RSS feed for this post

Comments are disabled