After posting yesterday’s typo filter announcement, I started to have a few misgivings about the way that I was planning on configuring filters. I asked a few questions on the Typo IRC channel and got a number of wildly different suggestions and opinions, but one line from cDlm stuck with me:

it’s gonne be a nightmare to keep all those filters orthogonal

There was also a comment that every blog was going to end up with its own markup language, incompatible with everyone else. I thought about it a bit, and I think I’m approaching filters wrong. Or, rather, I’m approaching them like a programmer, not like an end user. The filter interface that I’ve been planning is too generic, and we’ll probably be better-served if we remove some of the genericness, at least on the front end.

As I see it, filters fall into 4 basic categories:

  1. Markup languages–they convert from some non-HTML markup language into HTML. Examples include Markdown and Textile.
  2. HTML post-processors–they convert generic HTML elements into other generic HTML elements. Example: SmartyPants, and sort of the Amazon filter that I discussed earlier.
  3. Typo macro tags. This includes the <flickr> and <sparkline> filters from yesterday.

Looking at this list, I see 2 very specific things:

  1. There’s really no reason to run two markup languages on the same post. You want at most one of them, and possibly 0 if you’re writing raw HTML.
  2. As long as the Typo macro tags fit into a clean namespace and they don’t have side-effects when they aren’t used, there’s no reason to ever turn them off. If a macro tag filter is installed, then it should be used on all articles.

This really simplifies things, because the filter configuration no longer requires the user to set up an arbitrary set of filters. Now it just needs to know:

  1. Which markup language
  2. Which post-processing filters.

I think we can call the Amazon filter a post-processing filter without breaking anything. Furthermore, we can probably pre-order the post-processing filters by having a priority built into them. This way, the user just needs to click checkboxes. This is a lot less complex then dragging a half-dozen filters around in a script.aculo.us Sortable like we have to use for the sidebar.

This won’t handle 100% of the cases that people want, but it’s almost certainly well over 90%, and I think it’ll have about 10% of the complexity. The common cases, like “I want Markdown with SmartyPants” or “I want Textile with SmartyPants and WikiWords” will be really simple, and that’s much more important then the ability to stack filters in arbitrary orders. Anyway, there will be a way to get around the remaining cases by hand-creating TextFilter database entries; if someone really wants to do sufficiently weird things, like applying Markdown and Textile to the same article, then I don’t think asking them to do something like this is excessive:

$ ./script/console production
>> TextFilter.create(:name => 'weird filter', :description => 'My Weird Filter', :filters => [:markdown, :textile, :macros, :smartypants, :textile, :piglatin])

I’m going to start re-working my filter code to fit into this framework; I should have something to show in a few days.