Posted by Scott Laird
Sat, 10 Dec 2005 18:14:12 GMT
I just released a new version of my Rails Schema Generator on Rubyforge. The schema generator takes a collection of Rails database migration scripts and assembles a complete set of SQL schema files using only the information from the migrations. This release supports MySQL, PostgreSQL, and SQLite; it can generate schemas for all three DB types even if the databases aren’t installed on the system.
We’ve found that the schema generator drastically lowers the work needed to keep Typo working correctly with multiple database types.
This release fixes a number of bugs and should finally work correctly with all common migration operations, including field renaming. This has only been tested with Rails 0.14.4, and at least one API has changed recently, so this may not work with earlier 0.14.x releases.
To use the schema generator, run gem install schema_generator, and then (from your Rails project directory) ./script/generate schema. This will create several schema files in the db/ directory, prompting you before overwriting existing files.
Tags database, ruby, rubyonrails, schema, typo | 4 comments | no trackbacks
Posted by Scott Laird
Sat, 03 Dec 2005 06:45:00 GMT
I just committed a big patch to Typo that adds the first part of a notification framework. This piece adds the ability to send email or Jabber messages whenever certain events happen. The events currently supported are new article creation and new comment posts.
This is still a work-in-progress, but it should be safe for a few brave people to use. You’ll have to edit your user settings in Typo to configure your notification settings and then edit the general settings to configure which email address Typo should use. After that, it should just work.
Tags notification, ruby, rubyonrails, typo | 2 comments
Posted by Scott Laird
Thu, 01 Dec 2005 04:24:03 GMT
Just to be clear, I’m finally back at work enhancing Typo. I was working on email notification before my entire schedule disintegrated last week; I’m now able to devote at least some time to Typo hacking. Right now, I have email notification mostly working in my tree, but I need to do a bit more work (and write more tests, as usual) before it’s ready to commit. One way or another, I should have it committed before the end of the weekend, and possibly sooner.
Tags typo | 5 comments | no trackbacks
Posted by Scott Laird
Wed, 30 Nov 2005 06:37:53 GMT
I just wanted to say that I’m still here, I’ve just been really busy the past week or so dealing with Thanksgiving, flying to California, and starting a new job. I think tomorrow will be a relatively light day, so hopefully I’ll be able to get back to Typo hacking, so I can get my notification code into the tree sooner rather then later. Once that’s committed, I can start closing the big pile of bugs that has piled up over the last week.
Tags personal, typo | 1 comment | no trackbacks
Posted by Scott Laird
Thu, 17 Nov 2005 20:54:14 GMT
I just committed a minor new feature for Typo–there is now a route-of-last-resort that can generate HTTP redirects out of a database table. This is something that people have asked for for months, because it’s of the easier ways to keep old URLs working when moving to Typo.
At the moment, none of the converters know anything about it, so if you want to use it, then you’re going to have to do a bit of coding. I suspect that we’ll see patches within a few days that will help people make the jump. The basic code for adding a redirect is pretty simple:
Redirect.create(
:from_path => 'archives/000001.html',
:to => '/articles/2005/11/17/first-post')
This would be a bit cleaner if from wasn’t a reserved word in SQL. The from_path bit should be relative to Typo’s root, and shouldn’t start with a slash.
Unfortunately, none of Typo’s URL helpers are available inside of converters, but it shouldn’t be too hard to code this up. Send me mail if you need help.
Tags migrating, rubyonrails, typo | 5 comments
Posted by Scott Laird
Thu, 10 Nov 2005 16:19:32 GMT
Typo 2.6.0 is out. Tobi suggested that we bump the version number from 2.5.9 to 2.6.0; I probably should have done this for 2.5.7, but better late then never.
This is largely a bug-fix release–it fixes one brown-paper-bag bug in 2.5.7/2.5.8 (renderfix.rb was in the wrong directory), as well as adding a couple additional cleanups and 0.14.x compatibility fixes.
There are two fixes in here that theme contest users need to pay attention to. First, as part of the live search fix for 0.14.x, I moved the observer for the search field from the layout (default.rhtml) into the search partial (_search.rhtml). This matches the current trunk, so it’ll make themes more portable, but if your theme’s default.rhtml still has the search observer in it, then weird and wonderful things will happen when users try to search. So either nuke the observer in default.rhtml or override _search.rhtml to make it match. I also back-ported a second change from the trunk–each sidebar item is now wrapped in <div class="sidebar-node">. I’d be amazed if this managed to break anyone’s CSS, but theme builders might wish to use it to style the sidebar.
At this point, I have no plans for any further releases between now and November 28th, the last day of the theme contest. If any major bugs pop up, I’ll put out a new release to fix them, but I’m not planning on adding any additional features to 2.6.x, *or* releasing the trunk (as Typo 4.0–it’s a long story) before the end of the theme contest. Good luck.
Tags rubyonrails, typo | no comments
Posted by Scott Laird
Wed, 09 Nov 2005 01:44:36 GMT
Well, that was fun. Typo 2.5.7 lasted 80 minutes, only to be replaced by 2.5.8.
This is a security release, fixing a bug that has been in Typo since at least 2.5.0. All Typo users should upgrade to either 2.5.8 or the current Subversion trunk.
Tags rubyonrails, typo | 3 comments
Posted by Scott Laird
Wed, 09 Nov 2005 00:46:00 GMT
Typo has had theme support since before Typo 2.5.0 was released, but the first version of Typo’s theme engine didn’t include the ability for themes to change any of the HTML generated by Typo’s views. Themes could change CSS, graphics, and the top-level layout that generated the HTML for the site, but individual HTML blocks, like comments or article bodies, couldn’t easily be changed.
The latest release, Typo 2.5.7, fixes this shortcoming. It’s now possible for themes to completely override any bit of HTML or XML that Typo generates. Here’s how it works:
Like all Rails apps, Typo stores all of its HTML views in app/views/<controller>/<viewname>. For example, the view that generates individual article pages is app/views/articles/read.rhtml. The theme system in Typo 2.5.7 and newer lets themes override views by putting a replacement file into themes/<themename>/views/<controller>/<viewname>. So if I created a new theme called scottstuff and wanted to override the read.rhtml view from above, I’d put the new view into themes/scottstuff/views/articles/read.rhtml. Typo would then read this view from the theme instead of app/views.
It’s also possible to replace the HTML generated by sidebar plugins; just put the replacement view into themes/<themename>/views/plugins/sidebars/<sidebarname>/content.rhtml.
Typo should pick up on the new views immediately, even in production mode, but you’ll have to flush the page cache before the views really take effect.
Read more...
Tags rubyonrails, themes, typo
Posted by Scott Laird
Wed, 09 Nov 2005 00:22:12 GMT
I just released Typo 2.5.7. This is a relatively minor upgrade over 2.5.6, but it adds two important features:
- Support for Rails 0.14.x. Due to one little change, Typo 2.5.6 worked fine with Postgres and the new Rails, but MySQL failed horribly. This should now be fixed.
- Support for overriding views from within themes. The theme contest folks really want this, and it’s been available in the trunk for months, so I backported it.
Posted in Typo | Tags release, rubyonrails, typo | 1 comment
Posted by Scott Laird
Tue, 08 Nov 2005 17:11:00 GMT
I’d love to release Typo 2.5.7 today, but I’d like a few people to verify that the current 2.5.x Subversion repository actually works correctly in their environment first. If you’re interested, please grab it from svn://leetsoft.com/typo/branches/branch_2_5_x and kick the tires. I’ll take feedback any way I can get it–via IRC, direct mail, the Typo mailing list, or comments here. I’d prefer not to clutter up the Typo trac with this, though, as there’s no good way to say that the report is about the 2.5.x SVN tree.
Tags typo | 7 comments
Posted by Scott Laird
Tue, 08 Nov 2005 04:42:18 GMT
I’m just about to start in on Typo 2.5.7 development. I have two goals for this update to Typo 2.5.6:
- Make Typo work correctly with Rails 0.14.x.
- Backport the new theme updates from the Typo trunk to the 2.5 series so Typo Theme Contest developers can use them.
I’d like to have this on RubyForge before I leave for the airport on Wednesday.
Tags rubyonrails, typo | no comments
Posted by Scott Laird
Tue, 08 Nov 2005 03:43:42 GMT
One of the great problems with RSS is that it’s really hard to know how many readers you have. Feedburner is supposed to be able to help with that, but I’m reluctant to outsource my RSS feeds to them–I’m not really sure how I’d get them back if I decided to can Feedburner. So, while I know that I’m averaging around 1,300 JavaScript-enabled page hits per day on my blog, I have *no* idea how many people are reading via RSS. On one level, it doesn’t really matter, but I find that I’m more willing to write when I know that people are reading, and the more readers I have, the more time I’m willing to spend writing.
The problem is that there isn’t a 1:1 correspondence between RSS downloads and readers, like there is for normal web pages (modulo caching and a few other issues). Bloglines is helpful enough to tell me that it has around 60 subscribers, and I know that I’ve served up around 24,000 RSS and Atom feeds so far this month, but I have no easy way to know if that’s 1,000 people with a slow refresh set or 11 people refreshing every 5 minutes, or even 50,000 people all reading via a portal. Plus, there are at least three “planet” sites syndicating one feed or another (PlanetRubyOnRails, PlanetTypo, and Planet Foo), and I have no clue how many readers they have, either via HTML or RSS.
I’ve been tempted to integrate a 1-pixel “web bug” into Typo’s RSS feeds more then once, but I don’t really like the privacy implications. Fortunately or unfortunately, I get the same effect any time I post an image here. The Flickr montage that I posted almost 8 hours ago has resulted in 347 image hits. Of those, 150 have no referrer, so they’re probably from standalone RSS readers, like NetNewsWire. Another 95 are from scottstuff.net, followed by 42 from Planet Ruby On Rails, then 24 from Bloglines, 16 from Planet Typo, 3 from Planet Foo, 3 from Google Reader, and a couple that are either comment spammers or internal feeds from stealth companies.
Does anyone have any good leads on how to track this sort of thing on a more regular basis? While we’re at it, does Feedburner just play session cookie games, or are they doing something clever? Finally, it seems clear that embedding images into RSS feeds works most of the time, but I’ve never heard of anyone using web bugs with RSS–did I just miss the discussion, or are people avoiding them?
Tags rss, traffic, typo, users | 5 comments
Posted by Scott Laird
Tue, 08 Nov 2005 02:53:16 GMT
As a reminder, the Typo Theme Contest is still running. The deadline for entry is November 28th, so there’s still time to start, but you don’t want to wait much longer. The prize pool has grown again–the top prizes are a new 15” PowerBook, a 12” iBook, an iPod Nano, and a year’s free hosting. Although it pales in comparison to a new PowerBook, I’d love to bundle the top couple themes with future Typo releases, so you can count on thousands of Typo users enjoying your work.
Posted in Typo | Tags rubyonrails, themes, typo | 2 comments
Posted by Scott Laird
Fri, 04 Nov 2005 19:06:20 GMT
As of this morning, the Typo trunk requires Rails 0.14.2. I ran rails . and then spent a couple hours cleaning up Rakefile and environment.rb, and then updated all of the tests to work without instantiated fixtures. As advertised, this makes tests run quite a bit quicker; since Typo’s test suite had been flirting with the 5 minute mark on my laptop, anything that we can do to speed up tests is welcome.
Now that we’re safely requiring 0.14.x versions of Rails, I’ve started adding session :off all over the place. My sessions table for scottstuff.net has around 70,000 sessions in it right now, and I doubt that more then 15 or 20 of them were ever useful. We use sessions for user authentication, but that really only matters for admin pages plus a few other cases, like comment posting. In essence, any page that can be cached will never need access to a session. So, I made a little change to the Article controller. Instead of
class ArticlesController < ApplicationController
caches_page :index, :read, :permalink, :category,
:find_by_date, :archives, :view_page, :tag
...
end
I’m now using
class ArticlesController < ApplicationController
cached_pages = [:index, :read, :permalink, :category,
:find_by_date, :archives, :view_page, :tag]
caches_page *cached_pages
session :off, :only => cached_pages
...
end
This is a really common pattern, because pages that can use the page cache really shouldn’t depend on the session in any way. If I was doing this in more then one place, I’d probably want to extract it into a nice little plugin, but I’m just not feeling the motivation right now.
Posted in Typo | Tags ruby, rubyonrails, typo | no comments
Posted by Scott Laird
Mon, 31 Oct 2005 16:32:42 GMT
This seems to be the season for talking about Rails migrations. A lot of people are finally discovering them and finding that they’re very useful for maintaining your database schema over time. I’m a big fan of Rails migrations; we’ve been using them with Typo since the middle of July, when they were all new and shiny. We’re currently up to 24 migrations in the Typo source tree. We’re even using migrations to create our initial database, via my Schema Generator. I haven’t done a formal survey, but I suspect that Typo is the biggest open-source user of migrations, and may actually be the biggest user overall.
The big problem is that we’ve been using migrations wrong the whole time, and we just realized it.
There are probably a dozen bugs in Typo’s bug tracker that boil down to “I fell behind the trunk and now rake migrate throws exceptions and I can’t upgrade anymore.” The problem is that migrations are designed to run against an earlier version of your database, but they use the current version of your code. The first time that this caused problems was with the migration from Typo 2.0 to 2.5–we’d added two new fields to articles. Migration number 7 added the permalink field and a before_save hook to make sure that all saved articles have permalinks. Then migration number 9 added GUIDs and a second before_save hook to fill the guid field. Both migrations did Articles.find(:all).each { |a| a.save } to update each Article and populate the new fields.
This worked great for developers who frequently upgraded. A few days after the GUID migration went in, though, we started getting weird bug reports–users who tried to do both upgrades at the same time found that migration number 7 was dying. What was happening was that migration number 7 added the new permalink field to articles, but when it went to run the save loop both before_save hooks ran, and Typo tried to add a GUID to each article. However, the guid field didn’t exist yet, so the migration threw a bunch of exceptions and died.
This caused a bunch of grumbling on the Typo IRC channel. We threw around a bunch of possible fixes. Our favorite was separating migrations into two parts–a schema change part and a data change part. First we’d run all of the schema changes, and then update all of the data. As a work-around, we added a hack that checked the current schema version and disabled specific before_save filters for older versions.
We managed to keep this little bandaid working until a couple weeks ago, when a huge set of new migrations went it; they renamed the articles table and merged several other tables into the new contents table using STI. And, again, we found that older migrations broke when users tried to upgrade from Typo 2.5.6 to the current dev tree. Unlink the permalink/guid case, this time there was no simple workaround. We couldn’t just add a couple if statements in a filter and make it all go away.
The fundamental problem is that we were using the wrong mental model for migrations. I saw migrations as a one-dimensional thing–a list of steps for migrating old data into the new format. In this view, the migration for going from schema version 6 to schema version 7 is constant–once it’s been written, the only reason to change it is if a bug turns up in the logic for that migration. Otherwise, the migration code should remain unchanged over time.
And that’s the problem–migrations aren’t one-dimensional. They are (and need to be) two dimensional–the schema version is one dimension and the code version is the other. Individual migrations exist to migrate from a specific old schema version to the current version, using the current code. Each migration should change over time to adapt to the changes in the code. So, the right fix for the permalink migration that caused so many problems wasn’t to add a bunch of logic to before_save. Instead, we should have deleted the entire save loop from the migration, and trusted the GUID migration to update both fields. If that wasn’t good enough, then we should have added a new migration at the end to do permalink cleanup after the GUIDs were added.
Once I came to grips with this, the migration changes needed to allow 2.5.x users to upgrade to the current trunk were pretty simple, and took about 5 minutes to write and test.
Or was I the only person in the Rails universe who thought about migrations this way?
Tags database, migrations, ruby, rubyonrails, typo | 5 comments
