One of the dirty little secrets of Typo development is that this blog has been running the same Typo snapshot for almost 9 months. I’ve added a lot of code to Typo, but I haven’t been eating my own dogfood. There are several reasons for this, but the biggest was my threaded comment patch from early last year. It wasn’t merged into the trunk, and over time my blog and the Typo trunk diverged to the point where it wasn’t really practical to merge them anymore.
I finally gave up today and ditched my threaded comment code. I’ll rewrite it for Typo 4.1.
The migration to the current Typo trunk was reasonably painless, but it took most of an hour. I have more content then most users, and I’m running on slower hardware, so an hour is probably the worst-case scenario.
Now that I’m on the trunk, it’ll be a lot easier for me to test out spam filtering techniques here before committing them for general consumption.
Let me know if you see anything broken.
I’m starting to look at expanding Typo’s comment system. I have a few goals:
Give Typo the best comment system of any weblog engine.
Make it easier to handle large numbers of comments. This includes some sort of comment threading as well as comment pagination. Yes, those conflict with each other. No, I’m not sure how I’ll resolve it.
Make the comment system more resistant to spam. I’m not really sure which approaches I’m going to use for this either, but this is a well-explored problem, and I have a few ideas. Fortunately, Typo has always had a few features that discourage comment spam, and we’d like to add to that a bit.
Add support for authenticated comments, ideally using one or more external user identity systems, like TypeKey or OpenID. Personally, I’m really excited about OpenID, and I’d like to have Typo become both an OpenID client and server.
Increase the “socialness” of the comment system. Make it easier to conduct discussions in comments. Allow frequent users to track responses to their comments via RSS or email. Allow “community building” via comments. Consider allowing some forum-like features, like allowing users to re-edit their comments after they’re posted. This obviously depends on having a reasonable identity and authentication system.
Make it faster while we’re at it. Posting new comments currently invalidates Typo’s entire page cache; there has to be a way around that so we can keep up with Slashdot when they link to a Typo site.
Keep it easy to run. One of my basic goals with Typo is to minimize the number of configuration options in the system, following DHH’s mantra: ”flexibility is overrated.” At the same time, different people *do* have different needs; we need to find the right balance between having 50,000 little configuration options and force-feeding the One True Comment System down people’s throats.
My goal is to have something to show for this in the the next couple weeks; that depends on my consulting schedule and a few other things that are hard to predict, but I’m pretty optimistic about this.
Before I dig deeply into design, does anyone have any particularly good comment systems that we should look at? My current favorite is Dunstan Orchard’s on 1976design.com, but I’m open for suggestions. I’d love to hear about needs that I’m overlooking, too.
Last year, I posted a bunch of statistics about how the first year went. Unfortunately, my log rotator went wild for a while earlier this summer, so I only have the most recent 4 months worth of log data, but a few comparisons are still in order.
My "one year of blog" post is post number 259 here; this one will be number 514, so I guess I've fallen slightly behind last year's posting rate--I should have had 4 more articles this year. That's amazingly consistent. Last year I had a total of 110 comments, this year the number's up to 890, largely thanks to two articles.
I've clearly received a lot more traffic this year then I did last year. The Canon 5D article that I posted a couple weeks ago has received more hits this month then the top 4 articles from last year did over the entire year.
Unfortunately, switching from Movable Type to Typo a couple months ago makes it hard to calculate which pages were the most popular, because the URLs for pages changed. I have a conversion table somewhere, but I really don't feel like spending the morning doing programming just to build up half-useless statistics. So instead, I'll give a couple simple statistics and then move on.
Since April 18th (the oldest logs that I still have), I've logged a total of 1.25 million hits. That includes search engines, spam, graphics, and other floatsam, but it's still a lot more traffic then I served up the year before. So far this month, awstats reports that I've taken a total of 482,000 hits, with almost 400,000 of those being from real visitors. Counting search engines, I've served over 9 GB so far this month, which is just amazing. I've served 20,784 unique visitors a total of 75,238 pages this month.
Hopefully next year will be better. At this rate I'll have to pay someone for hosting within a year or so, because my little DSL pipe won't be able to keep up with the traffic.
It looks like Ecto has a hard time with my previous posting. Ecto completely locked up on me when I tried to edit the article after adding a few
<typo:code> blocks. After that, Ecto wouldn't even start up right--it only showed a single article on the blog and refused to let me click anywhere on the list screen, although the menu worked.
I had to completely remove all of Ecto's application settings and cached blog entries, lower my saved article count back down to 20, and re-add my blog, and it's still acting weird. I sense a bug report in my near future.
Update: Fixed. Adriaan spent most of an hour digging through logs and found that Typo was sending the article date in the wrong format--it was using a string when it should have been a date field. That changed in Typo a couple days ago in an attempt to work around a bug with another editor; I reverted it back to its original behavior and now everything's fine. I take back everything bad that I ever implied about Ecto--it's working perfectly for me now, and it's great to be back in Ecto instead of MarsEdit (sorry Brent).
I’ve never been happy with my web stats software for this blog. I’ve been limping along with Awstats for a while, because it’s better then nothing, but only slightly. It’s never been able to tell me what I really want to know–things like “where are the visitors to this specific page coming from” or “what pages are especially popular today”. I’ve poked around at other stats packages, but none of the free ones seem any better (more secure, perhaps, but not much more useful). I’ve always had a nagging sense that I was missing something, but I assumed that if I waited long enough, a generic package would show up that was good enough and would do what I wanted.
Today, I finally figured out what I was missing–I want a report that shows the most popular categories on my blog. I want to know that I’ve had 900 Asterisk hits this month but only 252 PDA hits. Since there’s no way that a generic stats package can know that this page belongs to my “Ruby” category, it’s pretty clear that I’m going to need a stats package that knows about my blog software. And since no stats packages know anything about Typo, I’m going to have to write one.
I’m going to play with this a bit in my spare time over the next week or so.
For now, what I’d really like to see from people is a list of questions that they’re like their blog stats software to be able to answer. I’ll start the list here by giving some of the basics and repeating a couple from above:
- How many hits am I getting per day?
- Where are the visitors coming from?
- What search terms are people using?
- What categories generate the most hits?
- Which pages are getting a larger-then-normal number of hits today?
Feel free to leave more in the comments and I’ll add them to my list.
I upgraded this site to Typo revision 351 last night. It was a bigger pain then I anticipated–there were conflicts with some of my local patches that took a while to find and fix. Now that it’s working, I’m really impresses–Typo now caches the HTML that it generates and gets Apache to serve it up directly without involving Typo at all whenever possible. This gives a huge speed increase, but it’s taken Tobias several revisions to get it all working correctly.
Unfortunately, I found another static caching bug one I put it up on my test site: caching only works when the Typo site lives in the root of a website. I have mine in a subdirectory (http://scottstuff.net/blog/), and that did entertaining things to the caching system. It would work right for the first hit, but once the cache was generated then Apache would throw 400 errors every time the page was hit. That kinda sucked.
Fortunately, the fix wasn’t too hard, and now everything seems to be working perfectly. As usual, let me know if you see any bugs.
I’m basically finished with my last block of changes to Typo, and most of them have been merged upstream. At this point, I have most of the features that I care about, but there are still a few things left to do:
- Implement the rest of the Movable Type API, including the DB fields behind all of the useful stuff. Basically, everything that’s available in common blog editors should be available in Typo.
- Optionally link article keywords to Technorati tags.
- Look into turning filters and sidebar items into plugins, where all that’s needed to add them is to drop their files into a directory and then enabling/disabling/reordering them from the admin pages. (sidebars are done: #157)
- Add a Flickr text-formating plugin that will let me say something like
[[flickr:scottlaird/24727421 "Some guy on a bicycle"]]and have that turned into a clickable image (with size tags) and a caption. I’m not sure what the right syntax will be for this–I know what Markdown uses, so I can avoid running into it, but I’m not all that familiar with Textile. It might be worth looking at Unicode-only brackets, like «» or 「」. Of course, adding untypable brackets sort of cuts down on the utility of shortcuts like this.
- Add an Amazon text-formatting plugin that allows
amazon:<ASIN>URLs and transforms them into links, optionally with affiliate ID attached. This is a lot less complex the the Flickr filter, because it can just search for
- Add a better way of mapping stacks of text-formatting filters to names. The Admin UI and the MT API both want symbolic names for filters; right now, this is hard-coded. It’d be nice to make this dynamically managed, but it looks like a total pain.
These used to be up above, but they’re done now.
- Link to author’s email address if provided. (done, #156)
- Add GeoURL support, with a tag in the headers for it and a config option for providing your latitude and longitude. (done, #154)
- Add Flickr config parameters so adding Flickr doesn’t require editing the source. (done, #155)
- Fix the Flickr sidebar so it doesn’t do weird things with portrait images–as it is, the border around the images fits landscape images perfectly but leaves a big gap with portrait images. Alternately, just use the square layout that flickr likes. (done, I’m using square images)
Update: Pretty much everything here is complete as of August 26th. See the Typo Wishlist wiki page for details on where we’re going from here.