Posted by Scott Laird
Tue, 11 Jul 2006 18:00:21 GMT
One of the dirty little secrets of Typo development is that this blog has been running the same Typo snapshot for almost 9 months. I’ve added a lot of code to Typo, but I haven’t been eating my own dogfood. There are several reasons for this, but the biggest was my threaded comment patch from early last year. It wasn’t merged into the trunk, and over time my blog and the Typo trunk diverged to the point where it wasn’t really practical to merge them anymore.
I finally gave up today and ditched my threaded comment code. I’ll rewrite it for Typo 4.1.
The migration to the current Typo trunk was reasonably painless, but it took most of an hour. I have more content then most users, and I’m running on slower hardware, so an hour is probably the worst-case scenario.
Now that I’m on the trunk, it’ll be a lot easier for me to test out spam filtering techniques here before committing them for general consumption.
Let me know if you see anything broken.
Tags blog, typo | 3 comments
Posted by Scott Laird
Fri, 16 Sep 2005 02:10:00 GMT
I’m starting to look at expanding Typo’s comment system. I have a few goals:
Give Typo the best comment system of any weblog engine.
Make it easier to handle large numbers of comments. This includes some sort of comment threading as well as comment pagination. Yes, those conflict with each other. No, I’m not sure how I’ll resolve it.
Make the comment system more resistant to spam. I’m not really sure which approaches I’m going to use for this either, but this is a well-explored problem, and I have a few ideas. Fortunately, Typo has always had a few features that discourage comment spam, and we’d like to add to that a bit.
Add support for authenticated comments, ideally using one or more external user identity systems, like TypeKey or OpenID. Personally, I’m really excited about OpenID, and I’d like to have Typo become both an OpenID client and server.
Increase the “socialness” of the comment system. Make it easier to conduct discussions in comments. Allow frequent users to track responses to their comments via RSS or email. Allow “community building” via comments. Consider allowing some forum-like features, like allowing users to re-edit their comments after they’re posted. This obviously depends on having a reasonable identity and authentication system.
Make it faster while we’re at it. Posting new comments currently invalidates Typo’s entire page cache; there has to be a way around that so we can keep up with Slashdot when they link to a Typo site.
Keep it easy to run. One of my basic goals with Typo is to minimize the number of configuration options in the system, following DHH’s mantra: ”flexibility is overrated.” At the same time, different people *do* have different needs; we need to find the right balance between having 50,000 little configuration options and force-feeding the One True Comment System down people’s throats.
My goal is to have something to show for this in the the next couple weeks; that depends on my consulting schedule and a few other things that are hard to predict, but I’m pretty optimistic about this.
Before I dig deeply into design, does anyone have any particularly good comment systems that we should look at? My current favorite is Dunstan Orchard’s on 1976design.com, but I’m open for suggestions. I’d love to hear about needs that I’m overlooking, too.
Tags blog, comments, rubyonrails, typo | 7 comments
Posted by Scott Laird
Sat, 27 Aug 2005 03:43:07 GMT
I misses posting it by a day, but scottstuff.net is now two years old. The first post was August 25, 2003.
Last year, I posted a bunch of statistics about how the first year went. Unfortunately, my log rotator went wild for a while earlier this summer, so I only have the most recent 4 months worth of log data, but a few comparisons are still in order.
My "one year of blog" post is post number 259 here; this one will be number 514, so I guess I've fallen slightly behind last year's posting rate--I should have had 4 more articles this year. That's amazingly consistent. Last year I had a total of 110 comments, this year the number's up to 890, largely thanks to two articles.
I've clearly received a lot more traffic this year then I did last year. The Canon 5D article that I posted a couple weeks ago has received more hits this month then the top 4 articles from last year did over the entire year.
Unfortunately, switching from Movable Type to Typo a couple months ago makes it hard to calculate which pages were the most popular, because the URLs for pages changed. I have a conversion table somewhere, but I really don't feel like spending the morning doing programming just to build up half-useless statistics. So instead, I'll give a couple simple statistics and then move on.
Since April 18th (the oldest logs that I still have), I've logged a total of 1.25 million hits. That includes search engines, spam, graphics, and other floatsam, but it's still a lot more traffic then I served up the year before. So far this month, awstats reports that I've taken a total of 482,000 hits, with almost 400,000 of those being from real visitors. Counting search engines, I've served over 9 GB so far this month, which is just amazing. I've served 20,784 unique visitors a total of 75,238 pages this month.
Hopefully next year will be better. At this rate I'll have to pay someone for hosting within a year or so, because my little DSL pipe won't be able to keep up with the traffic.
Tags anniversary, blog, statistics | 1 comment
Posted by Scott Laird
Thu, 25 Aug 2005 11:21:26 GMT
It looks like Ecto has a hard time with my previous posting. Ecto completely locked up on me when I tried to edit the article after adding a few <typo:code> blocks. After that, Ecto wouldn't even start up right--it only showed a single article on the blog and refused to let me click anywhere on the list screen, although the menu worked.
I had to completely remove all of Ecto's application settings and cached blog entries, lower my saved article count back down to 20, and re-add my blog, and it's still acting weird. I sense a bug report in my near future.
Update: Fixed. Adriaan spent most of an hour digging through logs and found that Typo was sending the article date in the wrong format--it was using a string when it should have been a date field. That changed in Typo a couple days ago in an attempt to work around a bug with another editor; I reverted it back to its original behavior and now everything's fine. I take back everything bad that I ever implied about Ecto--it's working perfectly for me now, and it's great to be back in Ecto instead of MarsEdit (sorry Brent).
Tags blog, bug, ecto, osx | 1 comment
Posted by Scott Laird
Tue, 09 Aug 2005 20:54:25 GMT
I updated my Typo to-do list this morning and uploaded it to the Typo wiki.
Hopefully we can get most of those features into the next major Typo release.
Posted in Typo | Tags blog, programming, rubyonrails, todo, typo | 2 comments
Posted by Scott Laird
Fri, 05 Aug 2005 19:59:02 GMT
Version 2.5.0 of Typo was just released. The Typo web site hasn’t been updated yet, but you can download it from RubyForge.
Typo is the software that runs this website. It’s written in Ruby using Rails.
Posted in Typo | Tags blog, ruby, typo | 1 comment
Posted by Scott Laird
Fri, 29 Jul 2005 21:37:47 GMT
I’ve never been happy with my web stats software for this blog. I’ve been limping along with Awstats for a while, because it’s better then nothing, but only slightly. It’s never been able to tell me what I really want to know–things like “where are the visitors to this specific page coming from” or “what pages are especially popular today”. I’ve poked around at other stats packages, but none of the free ones seem any better (more secure, perhaps, but not much more useful). I’ve always had a nagging sense that I was missing something, but I assumed that if I waited long enough, a generic package would show up that was good enough and would do what I wanted.
Today, I finally figured out what I was missing–I want a report that shows the most popular categories on my blog. I want to know that I’ve had 900 Asterisk hits this month but only 252 PDA hits. Since there’s no way that a generic stats package can know that this page belongs to my “Ruby” category, it’s pretty clear that I’m going to need a stats package that knows about my blog software. And since no stats packages know anything about Typo, I’m going to have to write one.
I’m going to play with this a bit in my spare time over the next week or so.
For now, what I’d really like to see from people is a list of questions that they’re like their blog stats software to be able to answer. I’ll start the list here by giving some of the basics and repeating a couple from above:
- How many hits am I getting per day?
- Where are the visitors coming from?
- What search terms are people using?
- What categories generate the most hits?
- Which pages are getting a larger-then-normal number of hits today?
Feel free to leave more in the comments and I’ll add them to my list.
Posted in Typo, Blog stuff | Tags accesslog, blog, blogstats, stats, typo | 3 comments
Posted by Scott Laird
Wed, 20 Jul 2005 14:51:07 GMT
I upgraded this site to Typo revision 351 last night. It was a bigger pain then I anticipated–there were conflicts with some of my local patches that took a while to find and fix. Now that it’s working, I’m really impresses–Typo now caches the HTML that it generates and gets Apache to serve it up directly without involving Typo at all whenever possible. This gives a huge speed increase, but it’s taken Tobias several revisions to get it all working correctly.
Unfortunately, I found another static caching bug one I put it up on my test site: caching only works when the Typo site lives in the root of a website. I have mine in a subdirectory (http://scottstuff.net/blog/), and that did entertaining things to the caching system. It would work right for the first hit, but once the cache was generated then Apache would throw 400 errors every time the page was hit. That kinda sucked.
Fortunately, the fix wasn’t too hard, and now everything seems to be working perfectly. As usual, let me know if you see any bugs.
Posted in Typo, Blog stuff | Tags blog, typo | no comments
Posted by Scott Laird
Thu, 14 Jul 2005 04:25:55 GMT
I just applied about 30 changesets worth of Typo changes all at once. If anyone sees any problems, please let me know.
Now that I’m up to date again, I can start back in on my big list of Typo changes.
Posted in Typo, Blog stuff | Tags blog, typo | no comments
Posted by Scott Laird
Sun, 10 Jul 2005 21:43:57 GMT
I’m basically finished with my last block of changes to Typo, and most of them have been merged upstream. At this point, I have most of the features that I care about, but there are still a few things left to do:
- Implement the rest of the Movable Type API, including the DB fields behind all of the useful stuff. Basically, everything that’s available in common blog editors should be available in Typo.
- Optionally link article keywords to Technorati tags.
- Look into turning filters and sidebar items into plugins, where all that’s needed to add them is to drop their files into a directory and then enabling/disabling/reordering them from the admin pages. (sidebars are done: #157)
- Add a Flickr text-formating plugin that will let me say something like
[[flickr:scottlaird/24727421 "Some guy on a bicycle"]] and have that turned into a clickable image (with size tags) and a caption. I’m not sure what the right syntax will be for this–I know what Markdown uses, so I can avoid running into it, but I’m not all that familiar with Textile. It might be worth looking at Unicode-only brackets, like «» or 「」. Of course, adding untypable brackets sort of cuts down on the utility of shortcuts like this.
- Add an Amazon text-formatting plugin that allows
amazon:<ASIN> URLs and transforms them into links, optionally with affiliate ID attached. This is a lot less complex the the Flickr filter, because it can just search for <a href="amazon:...">.
- Add a better way of mapping stacks of text-formatting filters to names. The Admin UI and the MT API both want symbolic names for filters; right now, this is hard-coded. It’d be nice to make this dynamically managed, but it looks like a total pain.
Done
These used to be up above, but they’re done now.
- Link to author’s email address if provided. (done, #156)
- Add GeoURL support, with a tag in the headers for it and a config option for providing your latitude and longitude. (done, #154)
- Add Flickr config parameters so adding Flickr doesn’t require editing the source. (done, #155)
- Fix the Flickr sidebar so it doesn’t do weird things with portrait images–as it is, the border around the images fits landscape images perfectly but leaves a big gap with portrait images. Alternately, just use the square layout that flickr likes. (done, I’m using square images)
Update: Pretty much everything here is complete as of August 26th. See the Typo Wishlist wiki page for details on where we’re going from here.
Posted in Blog stuff | Tags blog, ruby, todo, typo | 3 comments
Posted by Scott Laird
Fri, 08 Jul 2005 15:07:29 GMT
I’ve updated this blog again, using my latest Typo patches. It’s currently running Rails 0.13 (less then 2 days old), Typo SVN r280 (the most recent), and every change that I’ve submitted to the Typo Trac system. This includes:
Posted in Typo, Blog stuff | Tags blog, typo | no comments
Posted by Scott Laird
Wed, 06 Jul 2005 15:36:00 GMT
I just upgraded this blog. It was running the latest released version of Typo (version 2.0.6). It’s now running a development version of Typo from Typo’s Subversion tree, plus the patches that I posted here earlier.
One upshot of this is that threaded comments should now work. Feel free to play with them here. If you have any problems with them, then please send me mail. Thanks.
One other change is that comments are now formatted with Markdown, so it’s easy to use bold, italics, lists, and links. See the Markdown page for details; eventually I’ll add a blurb to the comment form explaining the simple stuff.
Posted in Typo, Blog stuff | Tags blog, markdown, typo | 2 comments
Posted by Scott Laird
Sat, 02 Jul 2005 02:57:13 GMT
As of about 5 minutes ago, the Typo version of this blog is now live, and the old Movable Type version is disabled. All old URLs should still work–I spent quite a bit of time generating a list of redirects to send each old page to the correct URL in Typo.
I still have a lot of template work to do–this site looks really plain to me, and I’ve been meaning to redesign the site for a while now, anyway. That should happen over the next week or so. I’d also like to implement threaded comments in Typo. The display end of threaded comments looks trivial to implement, but I’m not so sure about the other end of things.
Update–Wow, Typo publishes new articles way faster then MT did. From clicking the ‘Publish’ button in Ecto to seeing the new article appear in NetNewsWire is only a couple seconds. MT 2.6 took around a minute before the article was ready. On the downside, categories in Ecto seem to be broken.
If anyone sees problems with the new site, either leave a comment here or send me mail. Thanks.
Posted in Typo, Blog stuff | Tags blog, movabletype, typo | 1 comment
Posted by Scott Laird
Thu, 30 Jun 2005 06:14:12 GMT
I’ve spent most of the evening playing with Typo, and I believe I’ve fixed all of the problems that I was seeing this morning. Most of the problems that I saw were caused by the import script; adding a bunch of ORDER BY clauses suddenly made it import items in the same order that they were originally created.
At this point, I’m nearly ready to make it live. I just have three issues left:
Typo doesn’t ask for email addresses for comments, even though there’s a spot in the DB for them. I’ve found that it’s a lot easier to respond to people when the can provide me with an email address, even if it’s never published.
Similarly, there’s no way for it to display *my* email address. This should just be a quick template hack.
Finally, I really dislike Typo’s default date format. Instead of providing an exact date, it says things like “Posted by Scott 672 days ago.” It looks cool for recent entries, but it’s pointless for anything more then a couple weeks old.
That’s pretty much it. Once those are done, I’ll make Typo my primary blog and start turning Movable Type off. If anyone’s interested, they can see my work-in-progress Typo blog, although I don’t guarantee that it’ll always be running.
Posted in Typo, Blog stuff | Tags blog, todo, typo | 1 comment
Posted by Scott Laird
Wed, 29 Jun 2005 17:17:17 GMT
My test installation of Typo is up and running. First impressions:
It’s easy to install, once you have Rails working. I’d never actually installed Rails on my home web server, so I had to do the whole install gems, install rails, hunt down dependancies bit. I had an early version of Active Record installed, and it was being loaded instead of the current version that comes with rails, so I had to hunt it down and kill it.
The MT-to-typo conversion script worked okay, once I fixed the MySQL-isms in it. Specifically, it assumed that it could access the MT database by using databasename.tablename; that syntax gets you into different namespaces in Postgres, not different databases. I worked around it by copying the MT tables into the Typo database and then removing the databasename. part of the queries. It probably would have been easier to just pass public as the database name, though.
The script also assumes that all articles are formatted as HTML. Mine are (almost) all Markdown. To fix this, I had to install BlueCloth (gem install bluecloth) and then rebuild all of the cached versions of each of my articles. There’s no obvious way to do this in the Typo UI, but Typo’s internals makes it easy. All I had to do was run the Rails script/console command and then run Article.find_all.each { |article| article.text_filter='markdown'; article.save }. For the Ruby-phobic, this iterates through each Article in Typo’s DB, changing the formatting type and then saving the results back to the DB. Typo’s Article model hooks the save method to verify that text format conversions are handled correctly, so that’s all I really had to do. Nice and simple.
It’s kind of slow. I’m running Typo in debugging mode right now, and Rails is always slow in Debugging mode. I suspect that Typo will still be a bit pokey, even in production mode, though. I need to look into how it handles caching.
For some reason, not everything is sorted correctly. The ‘latest comments’ sidebar seems to be sorted randomly. This is probably an artifact of the conversion script–it looks like entity IDs aren’t in the same order as they were in MT, and that screws stuff up. That’s just a one-line fix to the conversion script.
Comments are listed newest-first. I’d rather have them appear in the order that they were written. This should be a one-line change.
The admin interface is sort of sparse. Rails makes it really easy to make calls directly into Typo’s API from the command-line, so I’m not too concerned with the lack of things to click on.
Somehow, some of the Article-Category associations were lost during the conversion. Most of them seem okay, but not all of them. This might take a bit of work to fix.
The categories list on the sidebar seems to be in ID order; it should be sorted by category name. This is easy to fix.
Nothing here looks fatally broken. I need to spend 15 minutes updating the conversion script and another few minutes fixing little issues with Typo, but the system seems to work well enough, at least on initial inspection. I’ll re-import the MT data again later today using the fixed conversion script, and then see what it takes to get it working with Apache in production mode. If that works out okay, then I’ll spend some time tweaking the CSS to make it look a bit more like my current site and then make it live.
Oh–one more thing–I need to have the conversion script save the old URLs from each entry to a file so I can write a bit redirect list that will point each MT article’s URL to the new Typo URL. I get too much traffic from Google to lose all of the links.
Posted in Typo, Blog stuff | Tags blog, rails, ruby, rubyonrails, typo | 4 comments
