Posted by Scott Laird Thu, 20 Oct 2005 23:30:21 GMT
Dave Thomas has a nice little announcement, complete with screenshot: Agile Web Development with Rails is the best-selling programming book on Amazon.com. Even better, Programming Ruby has the number two spot. Corgratulations.
Posted by Scott Laird Tue, 09 Aug 2005 04:23:41 GMT
One of the biggest improvements in Typo 2.5 is page caching. By using Rails’s built-in page cache, we can get 100x the performance on many benchmarks without doing more then a few lines of work. This lets us serve high-volume weblogs (like weblog.rubyonrails.com) without requiring heroic measures like clustering.
Unfortunately, there are a number of hidden problems with Rail’s 0.13.1’s page cache implementation. We’ve had to work around a number of them in order to get Typo 2.5 out the door.
Enabling Rails’s page cache is amazingly simple–just add caches_page :actionname to the top of your controller class and the :actionname action will spit out page cache files automatically. A couple small tweaks to Apache’s .htaccess file, and Apache will now serve cached files all on its own without involving Rails. If a client asks for http://blog.example.com/articles/2005/08/08/foo, Apache will first check for a articles/2005/08/08/foo.html file in Typo’s public directory. If that file exists, then it’s sent off to the client without touching Rails at all.
That part of caching is easy. It’s the other end that’s hard: sweeping the cache to remove stale cache entries. Rails provides a simple cache sweeper that can remove specified pages, but that’s not really good enough for us. With Typo, there are a number of events that end up touching a huge number of cached files. Adding a comment, for example, touches the cached article page, but it also changes the comment counter on the main index (if the article is still on the front page), the day, month, and year indexes, some number of category indexes, tag indexes, and potentially paginated versions of all of the above. The code to track these all down was trouble-prone and frequently missed one of the pages that needed to be changed; this led to stale caches. Even worse, some actions, like changing themes, need to invalidate all pages. Rails’s page cache doesn’t keep a list of cached pages, so there’s no clean way to sweep them all.
What we ended up doing was adding a page_caches table to the database and adding hooks to insert a new PageCache entry every time a page was cached. We also added a hook to remove entries from the page cache table whenever a page was manually swept, and then added a PageCache.sweep_all method to flush the entire page cache. For now, we’ve simply ripped out all of our old “smart” sweeping code and force a full sweep of the entire cache whenever anything substantial changes. Sooner or later we’ll start adding smart cache sweeping back in, but for now this works surprisingly well.
Another shortcoming of Rails’s page cache implementation shows up when you start using query strings. Asking for http://blog.example.com/articles?page=2 ends up handing the ?page=2 parameter to the static .html cache page if it exists instead of calling Rails to ask for page 2. Even worse–if this cached page doesn’t exist, then Rails will generate it and store it for future access, even though it’s the second page of the index, not the first.
Finally, and worst of all, in Typo http://blog.example.com/articles is actually equivalent to http://blog.example.com/, because the article index view is the default index page. This means that the cached page for http://blog.example.com/articles?page=2 is actually /index.html, so anyone visiting page 2 of the article index screws up the front page of the blog. There’s no easy way around this with Rails 0.13.1; for now we’ve had to do work to keep ?page= from paginating anything. There’s one point that we could interrupt the page cache process from inside of Typo, but it doesn’t have any way to see the @request object or any of the query strings.
Long-term, we’re going to need to patch Rails to add a cachable property to @request that gets set to false when there’s a query string present, and also tweak Apache’s rewrite rules to skip static files if a query string is present. That assumes that Apache is even able to do that–every time I read the mod_rewrite documentation I end up with a headache. Since Typo officially supports lighttpd as well as Apache, we’ll need to get both of them to do the right thing, which is far from trivial.
Finally, Rails screws up cached filenames when the URL has non-ASCII characters. So any URL with accented characters or any non-ASCII script is totally uncachable. At least with Apache and Webrick, Rails sees non-ASCII characters in the URL encoded using the usual %XX URL-encoding scheme. Unfortunately, both servers actually look for unencoded filenames. So Rails writes out the cache file for /foö as public/fo%C3%B6.html (assuming UTF-8 encoding), but Apache actually looks for public/fo<C3><B6>.html (where <C3> is a byte with the value of C3 in hex). This is actually not all that hard to fix–just add a URI::Util.decode to the right place inside of Rails–but it’s not clear what the security implications of this are.
Given all of these problems, I’ve been tempted to try using Rail’s action cache instead of the page cache–the action cache doesn’t let Apache serve the cached files directly, so Typo would have a brief chance to block the cache from handling specific files, and we could approach sweeping from the opposite direction. It’s not clear how big of a speedup the action cache would actually give us, though, compared to the massive win that we get from the page cache. We’d really like to keep using the page cache and fix all of its bugs to its usable by other Rails users.
Posted by Scott Laird Fri, 05 Aug 2005 21:42:57 GMT
I missed this when it first came out, but Why has a pointer to a WebDAV server plugin for Webrick, Ruby’s native web server. I wonder how easy this would be to port out of Webrick and into a generic FCGI controller, so we could use it with Rails? That would let Rails act as a file server, which would be useful for some sorts of user interaction. I wonder if we could make use of it in Typo?
Posted by Scott Laird Fri, 05 Aug 2005 15:23:00 GMT
It’s OSCON season, and most of the open-source world has descended on Portland for the week. This includes most of the leaders of the Ruby community, so the Portland Ruby Group held their own “Free OSCON” night at FreeGeek. Four of the people giving Ruby talks at OSCON gave their talks for free at FOSCON, and most of the rest of the Ruby speakers were there lurking in the back of the room, including Dave Thomas and Matz.
I dragged my trusty D60 along and took a few pictures, along with a few notes.
The first speaker was David Heinemeier Hansson, the creator of Ruby on Rails and O’Reilly’s “Hacker of the Year” for 2005. He basically gave us his 15-minute OSCON keynote on Rails, which was both a brief introduction and a marketing talk for Rails. His big theme was “flexibility is overrated”–by reducing the number of ways that you can approach web development, Rails makes it enormously easier to actually get things done.
When David’s talk was complete, our host Phil (Tomson, I presume) presented him with a vintage copy of How to Win Friends and Influence People.
This was deeply cool. Rich Kilmer gave a brief presentation on his current project, ActionStep, a port of OS X’s Cocoa API to Flash. Right now, there are free tools that can create Flash .swf files, but Macromedia’s licensing keeps them from legally using any of Flash’s windowing tools. So Rich decided to write his own windowing toolkit, using NextStep/Cocoa API. They’re over halfway done, and expect to release the first complete version before the end of 2005.
I’m not particularly fond of flash, so I wasn’t paying a lot of attention until Rich started showing off his ideas for Rails integration. He’s building a layer that will glue his Flash front end to a Rails back end, and the demo code that he presented made it look even easier then creating HTML forms for user interaction. I’m not sure that it’ll be something that I’ll ever end up using, but it looked deeply cool. Here are a couple screenshots:
Next up, Glenn Vanderburg gave a talk on metaprogramming in Ruby. He showed how Ruby itself uses metaprogramming to implement things like attr_accessor, and then showed how people have used and extended Ruby over the years. One theme was the continuing development of metaprogramming idioms in Ruby; he showed how things have changed over the years, starting with an X protocol wrapper that someone wrote years ago, through Rich Kilmer’s Java debug protocol system, and up through Rails. Ruby’s metaprogramming ability is one of the things that makes Rails so successful–the ability to extend the language to let you say things like:
class Article < ActiveRecord::Base
has_and_belongs_to_many :categories
has_many :comments
belongs_to :user
...
endThis is one of the things that makes Rails so useful and so much fun to program.
Finally, Why The Lucky Stiff came on stage. Why (or sometimes _why) is sort of the rock star of the Ruby world. His real identity is a closely guarded secret. He’s the author of why’s (poignant) guide to Ruby, which is easily the strangest programming book that I’ve ever seen.
His FOSCON talk was sort of a performance-art interpretation of his book.
We knew that interesting things were afoot when he showed up with a backup band, and we had to stop for a break while they set up on stage.
Once that bit was done, they launched into song. Why had a nice little piece that was essentially the Ruby lexer set to music. “A symbol starts with a colon and is followed by lowercase letters and numbers! A constant is composed of capital letters and underscores!”
This sort of thing went on for a while. He alternated between animated segments on the projector, demonstrations of distributed Ruby programming (with audience participation), singing about Ruby, and utter non-sequiturs.
Why’s presentation ended with the immortal words “stop her, she’s stealing our eigenclasses.”
Thanks to Phil and everyone else from the Portland Ruby group, and all of the presenters for giving us a very memorable and enlightening night. I have a few more pictures on Flickr, if anyone’s interested.
Posted by Scott Laird Tue, 02 Aug 2005 19:00:43 GMT
Work isn’t paying for OSCON this year, but one advantage of living in Seattle is that it’s close enough to Portland to be able to drive down for the day. So, I’m going to head down Wednesday afternoon for the Portland Ruby group’s FOSCON talks, and then hang around for a few hours Thursday morning before driving back for work.
It looks like several of the Ruby people with OSCON talks will be giving their talks for FOSCON as well, so it should be interesting. I suspect it’ll turn into a big Ruby BoF; that’s reason enough to drive down.
Posted by Scott Laird Wed, 20 Jul 2005 08:23:36 GMT
There’s a surprisingly small amount of documentation out there on tuning Apache for optimum Rails performance. Almost everyone mentions the first step (use FastCGI, not regular CGI), but that’s such a huge performance boost that it’s really obvious–waiting 2-3 seconds per hit for Rails to start up is an indicator that you’re doing something wrong.
Once you get past that, there’s not a lot of documentation. There are examples from place to place, but no one seems to discuss what they mean or why they should be used.
Ever since I switched to Typo, I’ve been seeing occasional HTTP 500 errors from Apache, suggesting that Apache was unable to talk to Typo. Looking in the logs shows that Apache was usually in the middle of restarting a FastCGI instance whenever the errors occurred. Digging through the mod_fastcgi shows that FastCGI can work in three different modes with Apache:
Dynamic mode is the default, but that’s not a good fit for Rails, because of its slow startup time. Switching to static mode really helps. To do that, I added this line to /etc/apache2/apache.conf on my Debian server:
FastCgiServer /var/web/typo/public/dispatch.fcgi -idle-timeout 120 \
-initial-env RAILS_ENV=production -processes 2Notice that I had to list the full path to Rails’s dispatch.fcgi file; on some systems you may be able to get away with only listing public/dispatch.fcgi, but that will almost certainly not work if you’re using virtual hosting.
By default, FastCGI assumes that your server will respond to queries within 30 seconds. I added the -idle-timeout 120 parameter just so I can deal with really slow responses better. Typo’s article admin page currently tries to list all 466 articles on one page, and that can take over 30 seconds to process.
The -processes parameter tells Apache how many FastCGI processes should run for this application. For 95% of users, 1 or 2 will be best. If you get a lot of traffic, then raising this to 3-4x the number of CPUs in your system might get you slightly better performance.
Finally, the -initial-env bit makes sure that Rails runs in production mode, talking to my production DB and not returning error backtraces to the user.
Posted by Scott Laird Wed, 13 Jul 2005 18:37:59 GMT
Dave Thomas just posted that Agile Web Development with Rails is complete and has been released for printing.
This is the first book on Rails, and it’s been a fascinating process watching them develop it. Dave claims that their beta book process has made this one of the most Agile book development processes ever. I ordered it on the first day that it was available, and I’ve now sorted through three or four different PDFs and reported a couple different errors. I think I posted the very first error on their public errata-tracking system, but it’s hard to get it to sort by time now that there are zillions of entries from different reviewers.
I’m looking forward to receiving my final PDF and (eventually) the final printed copy of the book.
Congratulations on a job well done, guys.
Posted by Scott Laird Thu, 07 Jul 2005 03:28:00 GMT
It looks like Rails 0.13 has been released. There’s a lot of nifty-looking stuff in there, but the one that really grabs my eye is the new migration facility. Rails now has the ability to make schema changes when you upgrade your app. So, when version 1.1 of your app comes along, and it needs to add an eye_color field to the user table, you can now handle it entirely within Rails:
class AddEyeColor < ActiveRecord::Migration
def self.up
add_column :users, :eye_color, :string
end
def self.down
remove_column :users, :eye_color
end
endThe nice thing is that this is completely database-independent. It’ll work on both MySQL and PostgreSQL now, and other databases as soon as their Rails drivers are updated.
One possible shortcoming with migrations: I don’t see any obvious way to enforce the correct order of migrations–if version 1.2 adds a pets table, and version 1.3 adds a nickname field to pets, then I don’t see how the upgrade process from 1.1 to 1.3 will know which order to apply the two migration scripts.
Also, the migration code seems to be a nice first start on database-independent schemas. I suspect that a near-future Rails revision will allow developers to specify the schema as a set of migrations, and then the migration mechanism will create all of the tables needed automatically. That’ll be another huge step for Rails usability.
Posted by Scott Laird Thu, 14 Apr 2005 21:59:54 GMT
The Rails app that I was working on a couple weeks ago for work is finally complete and in the hands of users. A couple higher-priority tasks kept me from getting back to it until yesterday, and the final push towards release was fantastically ugly.
For legacy reasons, I need two of my app’s controllers to use HTTP basic authentication, not form-based authentication. HTTP authentication with Rails isn’t well documented, but the ”teenage mutant ninja hero coders” have an example that works just fine with webrick. Unfortunately, it completely fails with Apache for reasons that weren’t particularly clear. It’s fairly well-known that Apache blocks CGIs from seeing HTTP authentication headers, but none of the workarounds that should have worked with Apache 1.3 (mod_rewrite, fastcgi, etc) actually worked.
In the end, I ended up upgrading to Apache 2 with FastCGI, and it worked flawlessly. Er, except the CGI variable that I was looking for went from being called HTTP_AUTHORIZATION to Authorization, so I had to tweak application.rb a bit.
Posted by Scott Laird Thu, 31 Mar 2005 01:48:38 GMT
My little Rails project at work is starting to wind down finally. I’ve been heads-down on this thing for almost a week, but it seems to be worth it–I have a nice UI around my database, along with a clean schema to replace the hacked-up one from the previous design. I have 8 models, 9 controllers, and 40-some views. Including tests and documentation, I currently have *90* files open in xemacs, a personal record.
Of the 4 days I’ve spent on this, at least one full day was lost to bugs in Rails that I had to find workarounds for. Probably another day was spent searching for examples and documentation on specific Rails features and generally learning how the system fits together. The remaining two days were divided between database/code design (there are some weird controller issues for this application) and HTML design. It’s been years since I last threw together anything more complex then a Movable Type theme, so my web-design skills were years out of date.
All things considered, 4 days doesn’t strike me as amazingly fast for this project, but I doubt that I could have been much faster using any other framework. The big thing that’s impressed me with Rails is the amount of polish that I’ve been able to put into this in a very small amount of time–things like pagination and live searching were nearly trivial to implement. With a bit of practice, I suspect that I could churn out similarly-complex applications in a little over a day, and that would be just astounding.
Posted by Scott Laird Fri, 25 Mar 2005 08:53:41 GMT
I’ve started to pick up Rails a few times, always to be interrupted by something–usually either a missing feature that I really needed, or a change in project priorities at work. I finally sat down with it today in an attempt to finish a little project that I’ve been avoiding for months at work.
It was all going well until suddenly one of my classes stopped working right. My Host class belongs_to my Group class. At one point this afternoon, host.group stopped working from inside of my web app. It worked perfectly with the unit tests, but I got a method undefined exception whenever I tried to access the group method on a Host object. Here’s a snippet of the code involved:
class Host < ActiveRecord::Base
belongs_to :group
belongs_to :customer
has_and_belongs_to_many :messages, :order=>"id"
end
class Group < ActiveRecord::Base
has_and_belongs_to_many :packages, :order=>"pkgorder"
has_many :hosts
validates_length_of :name, :maximum=>40
validates_format_of :name, :with => /^[-0-9a-zA-Z.]+$/,
message=>"may only contain letters, numbers, ., and -"
endThese two classes were in their own files, as generated by Rail’s generator script. Can you see what’s wrong? The line right above end in the Group definition should start with :message, not message. That missing colon in group.rb broke the Host class, but only when it was used after Group was defined. So the unit tests worked, because they tested each class separately, but it failed in a bizarre way when the two were used together.
Things like this make me wonder if maybe Rails isn’t getting just a wee bit too clever for its own good.
Other then that, though, it’s been working great. I’m spending too much time searching the Rails docs for help, but I’m moving right along.
Posted by Scott Laird Fri, 28 Jan 2005 19:11:52 GMT
The Seattle Ruby users group is meeting tonight. David Heinemeier Hansson and several of the other Ruby on Rails folks will be present. If you aren’t familiar with Rails, it’s an up-and-coming web framework that has been getting a lot of attention lately, even from outside of the traditional Perl/Python/Ruby communities. Oreilly’s ONLamp has a nice introduction.
If you’re interested in Ruby, this will almost certainly be worth attending.
We’re meeting at The Omni Group’s offices, near University Village. The Seattle.rb page has directions.
Posted by Scott Laird Tue, 14 Sep 2004 20:22:21 GMT
The second edition of Programming Ruby is now available for pre-order. The first edition was easily my favorite Ruby book, but it’s a bit dated now. The second edition covers Ruby 1.8.2, and includes an extra couple hundred pages of reference material.
In addition to the printed version (available via Amazon, etc), they’re also selling PDF copies of the book. If you order them together, then you pay $10 extra for the PDF, which seems really reasonable to me. I’d much rather search through the PDF then deal with the index of a printed book. I’d like to see other programming books sold with the same model.
Posted by Scott Laird Fri, 13 Aug 2004 11:38:48 GMT
So, I’ve started playing with Rails, a Ruby MVC web framework. So far, I’ve just barely dipped my toes into it, but I’m really impressed with what I’ve seen so far.
My first stop was ActiveRecord, the database layer for Rails. It’s clearly young, but it’s almost a perfect match for Ruby’s dynamic nature. It uses reflection and database metadata to build table classes on the fly. Here’s a brief example:
#!/usr/bin/ruby
require 'active_record'
ActiveRecord::Base.establish_connection(
:adapter => "postgresql",
:host => "localhost",
:username => "user",
:password => "password",
:database => "asterisk")
class Cdr < ActiveRecord::Base
def self.table_name() "cdr"; end
end
Cdr.find_all().each do |cdr|
puts "#{cdr.uniqueid}: #{cdr.channel} -> #{cdr.dstchannel}"
endThis dinky block of code is enough to connect to my Asterisk database and extract call details from the cdr table. I didn’t need to tell ActiveRecord anything about the structure of the table; it does the right thing on its own.
It’s actually quite a bit more capable then this example shows; it understands relationships and keys, and it has a full set of searching and updating methods. I suspect that I’ll be getting a lot of use out of it.
Posted by Scott Laird Tue, 13 Apr 2004 03:16:26 GMT
I’ve been playing with Borges, a Ruby web application server for the past week or so at work. It’s an interesting beast; it’s continuation-based, which means the flow of control in Borges apps is much closer to traditional programming then it is to normal web applications. I don’t have a good example handy, but it’s kind of mind-twisting, because you can make non-event-driven web applications with it. I’ll probably move some of my Asterisk stuff over to Borges, partly because Borges could use a good example, and partly because the big pile of Asterisk stuff that I’ve been accumulating could use a better web framework.
Older posts: 1 2
