Typo is live

Posted by Scott Laird Sat, 02 Jul 2005 02:57:13 GMT

As of about 5 minutes ago, the Typo version of this blog is now live, and the old Movable Type version is disabled. All old URLs should still work–I spent quite a bit of time generating a list of redirects to send each old page to the correct URL in Typo.

I still have a lot of template work to do–this site looks really plain to me, and I’ve been meaning to redesign the site for a while now, anyway. That should happen over the next week or so. I’d also like to implement threaded comments in Typo. The display end of threaded comments looks trivial to implement, but I’m not so sure about the other end of things.

Update–Wow, Typo publishes new articles way faster then MT did. From clicking the ‘Publish’ button in Ecto to seeing the new article appear in NetNewsWire is only a couple seconds. MT 2.6 took around a minute before the article was ready. On the downside, categories in Ecto seem to be broken.

If anyone sees problems with the new site, either leave a comment here or send me mail. Thanks.

Posted in ,  | Tags , ,  | 1 comment

Movable Type upgrade, Part 2

Posted by Scott Laird Wed, 29 Jun 2005 14:31:58 GMT

Last night’s upgrade to Movable Type 3.17 went fairly well. The upgrade from 2.6 to 3.17 was trivial–it installed and ran without problems. I didn’t really expect that, but I wasn’t about to complain. Once I verified that 3.17 was working, I started the process of moving Movable Type from using the Berkeley DB onto PostgreSQL, and then things started breaking.

Movable Type, like most open-source software, was really designed to use MySQL, with Postgres support added later. I’ve found a lot of software that’s supposed to work with Postgres, but actually fails in a number of fundamental ways, simply because there are a number of MySQL-related assumptions lurking in the code, and no one actually tested the system against Postgres.

In Movable Type’s case, the BDB-to-SQL conversion script is able to create the right set of tables in Postgres, and it’s even able to populate the tables with all of the right data. Unfortunately, it fails to update the sequence counters for any of the tables, which causes new articles and comments to show up as ID #1. If you already have a #1, then posting new articles and comments will silently fail.

It took me most of an hour to track that back and figure out why new comments weren’t being posted. Then it took about 15 seconds to fix–just do a select max(comment_id) from mt_comment to determine the maximum comment ID, then do select setval('mt_comment_id',9999), replacing 9999 with the number from the previous select. Then repeat for every table in the DB.

So, once that was done, everything worked, right? Ha. I was now able to post new comments and articles, but they didn’t show up on my site. Hitting ‘rebuild’ didn’t actually rebuild anything. So what happened this time?

Ah, the conversion script failed to populate the template table, so MT didn’t know which pages to build. Fortunately, I keep all of my templates in text files, not in the DB, so it only took 15 minutes to re-enter all of the template data; once that was done I was able to add comments and articles without problems, and hitting ‘rebuild’ correctly rebuilt the whole site.

So, everything appears to be working now. On the plus side, comment posting is way faster–it used to take around 30 seconds from the time users hit ‘publish’ until the browser reloaded. Now it’s down to 5 seconds or so. Hopefully that’ll cut down on the spate of duplicate comments that I’ve seen, because people won’t be so tempted to keep hitting ‘publish’ every ten seconds until it works.

Now that that’s done, I’m going to move on to testing out Typo. I noticed that the MT-to-Typo conversion script has a couple obvious MySQL-isms in it, but the script is so short that it shouldn’t be hard to work around.

Posted in ,  | Tags , , , , ,  | no comments

Upgrade to Movable Type 3.17

Posted by Scott Laird Tue, 28 Jun 2005 23:52:20 GMT

I’ve been promising to upgrade this blog to something more modern and interesting for over a year, and still haven’t done anything about it. I looked at Drupal for a while, but it just isn’t what I’m looking for.

So, for now, I’m going to upgrade from MT 2.6 (using BDB) to MT 3.17 (on Postgres). Then, once I have a bit of time, I’ll probably move to Typo, largely because it’s written in Ruby and looks easy to hack. It’s lacking a few things that I really wish I had (like threaded comments), but the code is so clean and simple that it shouldn’t take long to add them.

I’d jump straight to Typo, except that the MT-to-Typo conversion script requires MT 3.x to work. So I’m going to upgrade MT first, and then later this week play with Typo and see how it goes.

Once I start the MT upgrade, I expect everything here to break for a few hours. I don’t see an easy way around that, though.

Posted in  | Tags , ,  | 1 comment

My first trackback blog spam

Posted by Scott Laird Sat, 12 Feb 2005 15:58:27 GMT

I received my first three pieces of trackback spam overnight. By blog receives tons of comment spam, most of which are blocked by MT-Blacklist, but I haven’t seen any trackback spam before, even though I’ve been expecting it for over a year. Since trackback is designed to be automated, it seems like it’d be easier to abuse then comment spam. Fortunately, a few relatively simple steps should help stop trackback spam. A quick verification step would probably stop most of it–if you hand me a trackback URL, and I go fetch it, is there a link to my page anywhere on the trackback page?

One quick-and-easy suggestion for stopping comment spam–add </a> to your blacklist. MT 2.6 allows HTML in comments, even though it’s not supposed to, and comment spammers always try to stick links in there. Since my comment page isn’t supposed to allow HTML anyway, this does a great job of blocking spam.

Posted in  | Tags , , ,  | no comments

Comment spam explodes

Posted by Scott Laird Tue, 30 Nov 2004 23:50:16 GMT

I don’t know if it’s just me or if everyone is seeing this, but the amount of blog comment spam that I receive has exploded lately.

Spam Trend

So far this month, MT-Blacklist has blocked over 2,400 comment spam attempts. That doesn’t count the number that it missed–that has to be at least 200 more, including *12* so far while I’ve been writing this message. The latest couple batches don’t even seem to be obvious spam–they include a fake email address and some text, but no web pages, either in the URL field of the comment or in the body, and the text is generic. If I wasn’t receiving a few dozen per hour from different IP addresses with the same basic text, I’d assume it was just a deranged poster or two. As it is, I can only assume that it’s an attempt to pollute a Bayes table with bogus text, except MT-Blacklist doesn’t use Bayes–it’s just keyword matching.

At the present rate, I think I’m actually seeing more comment spam attempts then legitimate page views some days. I think I’m getting more blog spam then email spam, too, but it’s a close race.

I swear, I need to move off of MovableType 2 one of these days, but the last time I tried, I just couldn’t find anything that I was willing to spend the effort on. Drupal is nifty, but it’s not really what I’m looking for. MT 3.1 would probably work, but it’s not exactly what I want, either. I keep waiting for one of the Rails-based blog systems to become usable, but I don’t think we’re quite there yet.

Posted in  | Tags , ,  | 1 comment

The Blog Upgrade question strikes again

Posted by Scott Laird Wed, 01 Sep 2004 16:27:00 GMT

I installed Movable Type when I first started this blog, but I’ve been itching to change for months. A small part of that itch is Movable Type’s new pricing model, but it’s really more then that. I have a number of needs that MT isn’t really filling, and I’d like to move to something that works better for me.

The big problem is that I can’t find anything that’s quite right. I looked at Drupal for a while, but there are a few things with it that I just couldn’t cope with:

  • It’s a much bigger system then I really need, with a lot of complexity.
  • It’s written in PHP. If I could treat it as a black box, I wouldn’t really care, but I couldn’t because…
  • It’s essentially hard-coded to need MySQL. In theory, it’ll work with PostgreSQL, but I fought with it for days without actually getting it to work. There were a number of deeply-embedded MySQLisms in the code that I just couldn’t fix, even after digging into the code for a while.
  • PHP’s SQL code is too scary to look at. While the core of Drupal goes to great lengths to prevent SQL injection attacks, a number of add-in modules looked pretty clueless. In addition, all of the SQL code is built up using command = 'insert into foo (a,b) values ("'+value1+'","'+value2'");'-style commands, which are inherently ugly and prone to problems. I really prefer the Perl (and Ruby) DBI version: insert into foo (a,b) values (?,?), where you provide value1 and value2 as parameters to the DBI execute function.
  • Template modifications are a royal pain compared to MT. Out of the box, all of the templates used HTML tables, unlike MT’s clean CSS-only templates.

Now, if I was setting up a big community site, none of these would really matter to me. I could spend a couple weeks on templates. Heck, I’d expect to spend a while tweaking things until they worked right for me. If I was doing this from a corporate perspective, I could just hire someone with experience in Drupal, like Bryght. But I’m not building a big community site, and I’m not willing to pay someone to do it for me. I’m largely doing this for the fun of it, and Drupal doesn’t seem to be a lot of fun.

So I’m back looking again. MT has cleaned up their prices, so I could just install MT 3.1 and be done with it. It wouldn’t be a lot of work to upgrade, and there’d be a handful of benefits, but it still wouldn’t give me an HTML photo gallery or a decent interface for static non-blog pages.

I’m fighting off the urge to use Rails to write a blogging system for myself. Hopefully, if I fight off the urge long enough, then someone else will do it for me, and I can just take their framework and adapt it to my needs. One can always hope :-).

Posted in  | Tags , , , , ,

Uhm, who stole my formatting?

Posted by Scott Laird Tue, 15 Jun 2004 13:44:33 GMT

That was weird. For at least a few hours, none of the Markdown-formatted entries on this site were formatted properly. MT refused that Markdown was even installed, even though Markdown.pl was correctly installed in the plugins directory. I grabbed a newer version of Markdown and re-installed it, and suddenly everything works.

I hate software some days.

Posted in  | Tags , ,  | no comments

Wow, it looks like a bad day to be Six Apart

Posted by Scott Laird Fri, 14 May 2004 16:14:25 GMT

The new pricing for Movable Type (the software that runs this blog, along with a zillion others) is out. The previous release was free for non-commercial use. The new release is free–if you only have one author and fewer then 3 blogs. Any more then that, and you need to pay. The cheapest license is $99, marked down to $69 for the moment, and that only covers 3 authors. For commercial users, pricing starts at $299 (on sale now–only $199) for 5 authors, and goes up to $699 ($599) for 20 authors/15 blogs.

Needless to say, this is causing a bit of an uproar, and a lot of people are looking at switching from MT to other systems.

I guess I’m probably one of them. I’ve been half-heartedly looking for a different system for a while, but my needs are kind of unusual (as usual :-). Here’s a short list of what I’m looking for:

  • Simple, customizable blog engine, supports RSS and Atom, as well as at least one API supported by Ecto and NetNewsWire’s editor. Atom API support would be nice, but not all that critical, since I don’t have an Atom-aware editor yet.
  • Trackback and comment support. Preferably threaded. I actually like the concept behind Six Apart’s TypeKey, but that’s too much to ask, probably.
  • Support for non-blog pages. Take a look at http://svn.scottstuff.net for an example. Most of the pages are auto-generated, but I’d like to be able to share the template with my blog, and it’d be nice to be able to use the same comment engine.
  • Support for the Markdown markup language. I’ve found it to me vastly easier to work with then writing raw XHTML. That’s not to say that HTML is hard, but Markdown really lowers the amount of effort required.
  • Decent comment-spam tools. Admittedly, most comment spam is keyed to MT’s comment system, but that’ll change.
  • Tools for converting from MT. I don’t mind spending a bit of time on this, and I only have 190-ish posts here, but I’m not throwing them away, and it’d be nice to save the comments and trackbacks, too.
  • A photo gallery system that doesn’t suck. Since I haven’t found one that doesn’t suck yet, this is a difficult requirement. My goal is to be able to maintain one big master index in iView MediaPro on my Mac, and then sync the pictures and metadata onto my server from time to time, mostly using rsync and xml. Then, I want an automated script to pre-render thumbnails (on-demand thumbnails of 6 MP images are too slow for my poor server) and lay everything out. I’m currently using Album, but I’m not particularly fond of it. It just works better then anything else I’ve used. Systems that require manual, non-scriptable uploading of individual images need not apply.
  • A semi-integrated Wiki’d be nice, but I doubt I’d use it any time soon.
  • It needs to be scriptable and easy to enhance. Ideally, it’d be written in a language that I’m comfortable with; Ruby’d be best, and Perl’s okay. I can cope with Python and PHP, but I don’t really like either. A decent XML RPC/SOAP/REST interface would be nice, too.

If anyone has any suggestions, please leave comments. I suspect I’ll hear at least one recommendation for Drupal, but it’d be nice to hear other suggestions too. Can Drupal handle semi-static non-blog pages easily?

Posted in , ,  | Tags , , ,  | 1 comment