*scottstuff*

This site actually lives on a box under my desk at home and talks to the world via DSL with a static IP address. For the past three years or so, it’s been running on a 700 MHz Athlon with 768 MB of RAM and a 5 GB hard drive (from 1997!). The old box has been working hard to keep up, but there’s just too much going on between mail, Asterisk, multiple blogs, and a zillion other services, and I have a few tools that I’d love to write but there’s just no way the old box could keep up.

So, it’s time for new hardware. I have an Athlon 64 sitting in my home office mostly unused, so I’m going to swap it into service next month, after I finish travelling. The new box will be an Athlon X2 3800+ with 3 GB of RAM and a pair of 250 GB drives. That’s probably an 8x jump in performance over the old hardware. I’m going to try to use it with Xen, so I can spawn sites off onto their own virtual server for better isolation, but I’m not sure how well Asterisk will cope. It’s basically a realtime app that needs direct access to a pair of PCI telephony cards, so it’s not all that easy to virtualize.

I’m in Vancouver for Bar Camp this weekend. We started off this afternoon with a barbecue at Bryght’s new offices and then moved on to Workspace. It’s been a couple years since I was last in Vancouver, and I’m not sure why it’s taken me so long to make it back–considering how active Vancouver’s startup and open-source scene is, I really should drive up more frequently.

Anyway, if you’re at Barcamp and interested in Typo (or Rails, or Google, or whatever), then stop by and say hi.

I took this last month when I was on vacation. It’s not perfect, but I’m happy with how it turned out.

I just realized that today marks three years since I first started blogging here. In previous years, I’ve provided statistics, but I only have 3 minutes left today, so I’m not going to dig them up right now. Maybe later. FWIW, here’s the first post.

One of my long-running problems with Rails (and Ruby in general) is that it’s difficult to debug memory leaks. I’ve had a number of cases where I’ve stuck something into a long-lived array or hash and discovered much later that my Ruby process was eating over 100 MB of RAM. While ps makes it easy to see when Ruby’s using lots of RAM, actually figuring out where it went is a lot harder.

Several people have been working on memory leak debuggers for Rails, and for Typo in general, including Steve Longdo, but I didn’t have a lot of luck actually finding leaks with their tools. I asked the Seattle Ruby Group for help, and Ryan Davis gave me a quick little memory leak spotter that he uses. I made a few additions to it, and it helped me discover that my Typo development tree was leaking 1-3 strings per hit, but it didn’t help me figure out where the leak was happening. After playing with a few options, I settled on dumping all strings to a file once per memory profiler loop, and then I diffed the files that showed my problem. It took about 15 seconds to discover a bug in my route cache code, and about 30 seconds more to fix it.

I’ll package this up as a Rails plugin eventually, but I thought it might be worth sharing here for now. Just load this code and then call MemoryProfiler.start. By default it logs a record of the 20 classes with the biggest changes over the last 10 seconds; you can change the cycle speed by adding :delay => 20 to the start command, and you can dump all strings to a file on each loop by adding :string_debug => true. Don’t leave string debugging on for too long; it’ll eat a ton of disk space.

class MemoryProfiler
  DEFAULTS = {:delay => 10, :string_debug => false}

  def self.start(opt={})
    opt = DEFAULTS.dup.merge(opt)

    Thread.new do
      prev = Hash.new(0)
      curr = Hash.new(0)
      curr_strings = []
      delta = Hash.new(0)

      file = File.open('log/memory_profiler.log','w')

      loop do
        begin
          GC.start
          curr.clear

          curr_strings = [] if opt[:string_debug]

          ObjectSpace.each_object do |o|
            curr[o.class] += 1 #Marshal.dump(o).size rescue 1
            if opt[:string_debug] and o.class == String
              curr_strings.push o
            end
          end

          if opt[:string_debug]
            File.open("log/memory_profiler_strings.log.#{Time.now.to_i}",'w') do |f|
              curr_strings.sort.each do |s|
                f.puts s
              end
            end
            curr_strings.clear
          end

          delta.clear
          (curr.keys + delta.keys).uniq.each do |k,v|
            delta[k] = curr[k]-prev[k]
          end

          file.puts "Top 20"
          delta.sort_by { |k,v| -v.abs }[0..19].sort_by { |k,v| -v}.each do |k,v|
            file.printf "%+5d: %s (%d)\n", v, k.name, curr[k] unless v == 0
          end
          file.flush

          delta.clear
          prev.clear
          prev.update curr
          GC.start
        rescue Exception => err
          STDERR.puts "** memory_profiler error: #{err}"
        end
        sleep opt[:delay]
      end
    end
  end
end

As usual, the good bits are Ryan’s, and the bad bits are mine.

I just commited a big patch to the Typo trunk. This is the first part of my big Typo 4.1 cleanup project. It includes a whole bunch of cleanups, including:

• URLs are now generated via blog.url_for using blog.canonical_server_url. This lets us generate URLs directly from models without needing a controller or a request.
• Removal of all of the text filter controller mess.
• Moved text filters out of components/ and into vendor/plugins
• Massive helper cleanup, deprecating dozens of redundant helpers and model methods.
• Generic deprecation code to make it easier to flag methods as obsolete.
• Memory profiler.
• body_html/extended_html content fields moved into fragment cache .
• Total audit of all url_for calls to fix weird stuff.

This is probably a bit unstable, so don’t use it on a production blog until it’s had a chance to settle out.

Typo 4.0.3. It’s available as a .gem as well as tar and zip files. This is a bugfix release, fixing several small bugs that have been reported since 4.0.2.

I’m not quite done releasing Typo 4.0.x builds yet, but I’ve already started working on code for the next major Typo release. My big goals for 4.1 are speed and cleanliness. Typo still has some cruft buried in it from Rails 0.12.x, and there are a number of subsystems that have been partially refactored and rewritten several times. I want to clean all of that up and make Typo as fast as possible while reducing its memory footprint.

One of the problems that we have in Typo is text filters–our text filtering system includes several components that need to generate URLs that point back to the current blog using url_for, but url_for requires a request object to allow it to find the current base URL. Part of the run-up to Typo 4.0 included the addition of a canonical_server_url configuration field that is auto-populated on blog creation, but we weren’t really using it for anything yet.

Starting with 4.1, we’re going to be cheating and using canonical_server_url to generate most of our URLs. This has a lot of big advantages. First, we can get rid of the whole text-filters-are-controllers problem, because we can use this_blog.url_for to generate URLs. Second, I’ve added permalink_url methods to most models, using Blog.url_for. This has let me deprecate dozens of helpers and remove cruft from all of the tree. The third advantage is that generated URLs will be stable–it doesn’t matter if people go to scottstuff.net or www.scottstuff.net, all of the links will point to http://scottstuff.net either way. Finally, I’m actually caching calls to Blog.url_for–if you ask for the same page more then once during the lifetime of your dispatch process, then the second call should be nearly instant. Stefan Kaes keeps pointing out that routes are one of the slowest parts of Rails; hopefully this will help our performance.

I’ve also deprecated boatloads of helpers. I think we had 6 or 7 different ways of generating an article permalink. They’re all gone now, replaced by article.permalink_url. I created my own deprecation tool–just add a call to typo_deprecated at the top of each deprecated method and it’ll print a warning the first time it’s called in production or development mode. In test mode, deprecated methods will throw exceptions every time they’re called.

My current patch touches 117 files an has over 1000 lines of changes. I’ve made a lot of progress in cleaning up Typo, but there’s still a lot of work left to go.

I’m planning on releasing Typo 4.0.3 later this week. Once that’s out, I’m going to create a 4.0.x branch and start adding my new code to the trunk.

I mentioned a few weeks ago that I was trying to add generic database backups to my installer, so I can do database backups before each upgrade. I got backups working a couple releases ago, and they’re useful enough that I’ve extracted them so everyone can use them.

The latest installer release includes a pair of command-line tools that make it easy to use the installer’s backup and restore code without using the rest of the installer. Run rails-backup and it’ll produce a .yml file containing all of your data. Then run rails-restore BACKUPFILENAME and it’ll restore the backup. The dump and restore formats are database-independent, so it should be possible to use these tools to migrate between database engines, but this hasn’t been heavily tested yet.

One warning about the restore–rails-restore doesn’t restore the schema, just the data. So you’ll need a way to build a database with the right schema revision. Look in the .yml file to see which schema version you need (search for schema_info), and then run rake migrate VERSION=xx to move your database to the right schema revision.

To use these, just install the rails-app-installer gem, and they should appear in your path automagically. Report bugs to the rails-app-installer bug tracker.

I just released version 0.2.0 of my rails-app-installer tool.

This is the installer that I created for Typo, extracted into its own package so other Rails apps can use it as well. The installer lets users install Typo with only two commands:

  $ sudo gem install typo
  $ typo install /some/path

With a little bit of work, you can get your app to be just as easy to install. New in 0.2.0 is a rails-app-installer-setup command that will do most of the work for you. Just cd to your project and run rails-app-installer-setup my-app, and it’ll create a bin/my-app installer for you, along with some config files and a lib/tasks/release.rake file that knows how to build a .gem for you. Just follow the directions that rails-app-installer-setup gives you and you’ll have a .gem in no time at all.

Changes for version 0.2.0:

• Added a rails-app-installer-setup command to help set up new apps.
• Made the installer fetch its default version of Rails from the application .gem dependency, instead of making developers repeat themselves.
• Fixed a restore bug that kept IDs from being restored correctly.
• Added a command-line backup and restore tool that can be used with any Rails app. I’ll talk about it more in another post.

At this point, it’s nearly complete. It needs better documentation, and I’d like to get a bit of feedback from other apps before I call it 1.0, but my to-do list is getting pretty short.

I just released Typo 4.0.2. This is mostly a security upgrade; it’s designed to work with Rails 1.1.6, but it also includes workarounds for all of the known Rails 1.1.x security bugs, thanks to Piers Cawley.

In addition, we’ve fixed several bugs in the installer. MySQL users should be able to upgrade from 4.0.0 or 4.0.1 without problems now, fixing a problem with the 4.0.1 upgrader.

Upgrade directions:

Repeat however you installed Typo in the first place. If you downloaded the .tgz or .zip files, then you’ll need to download them again and install them over the top of your existing install. Then run rake migrate and restart your Typo processes.

If you’re using the .gem installer, just run gem install typo and then typo install /some/path to upgrade.

Apparently Rails 1.1.5 is still broken. It fixed one attack but left a couple other holes open. Piers has a workaround that should work for any Rails app.

Now would be a great time to fix this, because the exploit is fairly obvious right now.

I just released Typo 4.0.1. By and large, this is a bug-fix release. Please make sure that you use Typo 4.0.1 with Rails 1.1.5, as earlier versions (1.1.4 especially) have substantial security problems.

A number of changes were made to the installer for this release:

• It has been spun off into its own .gem and lives in its own source repository, so other projects can use it as well.
• It now supports Postgres as well as SQLite3. I’ll write about this soon.
• It performs database-agnostic backups to a .yml file.
• A number of bugs have been squashed.

If you installed Typo 4.0.0 via the new installer, then upgrading is easy–just gem install typo and then typo install /some/path to upgrade. If you’re using the tar or zip files from Rubyforge, then unpack them over the top of your existing install, run ‘gem install rails’ to make sure that you have Rails 1.1.5, run ‘rake migrate’, and then restart your FastCGI processes.

If you’re installing for the first time, then you’ll probably find the .gem installer to be easier. If you can install things as root (or your hosting provider has already installed the Typo .gem), then do this:

  $ sudo gem install typo
  $ typo install /some/path

If you don’t have the ability to run gem install as root, then do this:

  $ export GEM_PATH=~/gems
  $ gem install -i ~/gems typo
  $ ~/gems/bin/typo install /some/path

As usual, let me know if you hit problems.

The Rails team just released Rails 1.1.5, a “mandatory security” upgrade. So, if you’re running Rails, now would be a great time to upgrade.

I’m going to release Typo 4.0.1 tonight, including Rails 1.1.5 and a bunch of bug fixes.