Posted by Scott Laird
Tue, 08 Nov 2005 03:43:42 GMT
One of the great problems with RSS is that it’s really hard to know how many readers you have. Feedburner is supposed to be able to help with that, but I’m reluctant to outsource my RSS feeds to them–I’m not really sure how I’d get them back if I decided to can Feedburner. So, while I know that I’m averaging around 1,300 JavaScript-enabled page hits per day on my blog, I have *no* idea how many people are reading via RSS. On one level, it doesn’t really matter, but I find that I’m more willing to write when I know that people are reading, and the more readers I have, the more time I’m willing to spend writing.
The problem is that there isn’t a 1:1 correspondence between RSS downloads and readers, like there is for normal web pages (modulo caching and a few other issues). Bloglines is helpful enough to tell me that it has around 60 subscribers, and I know that I’ve served up around 24,000 RSS and Atom feeds so far this month, but I have no easy way to know if that’s 1,000 people with a slow refresh set or 11 people refreshing every 5 minutes, or even 50,000 people all reading via a portal. Plus, there are at least three “planet” sites syndicating one feed or another (PlanetRubyOnRails, PlanetTypo, and Planet Foo), and I have no clue how many readers they have, either via HTML or RSS.
I’ve been tempted to integrate a 1-pixel “web bug” into Typo’s RSS feeds more then once, but I don’t really like the privacy implications. Fortunately or unfortunately, I get the same effect any time I post an image here. The Flickr montage that I posted almost 8 hours ago has resulted in 347 image hits. Of those, 150 have no referrer, so they’re probably from standalone RSS readers, like NetNewsWire. Another 95 are from scottstuff.net, followed by 42 from Planet Ruby On Rails, then 24 from Bloglines, 16 from Planet Typo, 3 from Planet Foo, 3 from Google Reader, and a couple that are either comment spammers or internal feeds from stealth companies.
Does anyone have any good leads on how to track this sort of thing on a more regular basis? While we’re at it, does Feedburner just play session cookie games, or are they doing something clever? Finally, it seems clear that embedding images into RSS feeds works most of the time, but I’ve never heard of anyone using web bugs with RSS–did I just miss the discussion, or are people avoiding them?
Tags rss, traffic, typo, users | 5 comments
Posted by Scott Laird
Wed, 31 Aug 2005 05:13:00 GMT
Last week’s filter project is now a standard part of Typo, so I’ve been spending time on a new Typo project: unified RSS feeds. Typo has had support for RSS 2.0 and Atom 0.3 for months, as well as RSS feeds for site-wide comments, site-wide trackbacks, and per-article comments. Each feed was generated by its own distinct view code. This wasn’t scaling very well–every time we add a new feed type (like per-category or per-tag feeds), we have to write a bunch of new code and copy existing templates, and replicate that work once for each feed format that we want to support. That’s why most of the feed types were only available in RSS–creating Atom feeds for them was too much work and too error-prone.
So I decided to rip all of the feed code out and replace it with a new system. Instead of having one template per feed type, I have one master template per feed format (RSS 2.0, Atom 0.3, and now Atom 1.0), plus a per-item template for articles, comments, and trackbacks in each format. That’s a total of 12 templates for 3 different formats. My controller then generates a list of items based on some set of queries and then asks the view to assemble itself based on the type of items that it finds in @items. So the recent articles RSS feed produces a list of the 15 most recent articles, shoves that into @items, and asks the view to take care of it. The trackbacks Atom feed works the same way–@items gets a list of the 15 most recent trackbacks and the same basic view logic does the work. Mixed-type feeds like the article+comments feed work too. This will make adding new feed types mostly trivial.
While I was at it, I added Atom 1.0 support. NetNewsWire 2.0.1 has Atom 1.0 support, so it’s time for us to add it too. Because of our nice new template infrastructure, every feed type that was available in RSS before is now available in Atom 1.0 as well. I also added per-category and per-tag feeds, although they aren’t linked in anywhere yet, so users have to type in the feed URL by hand in order to use them.
The big problem with having so many different feed types (6 types times 3 different formats) is keeping them all standards-compliant. Ruby doesn’t have a feed validator class yet, so I decided to cheat. The Python code from feedvalidator.com is available from SourceForge, so I grabbed it and installed it locally. Then I took their demo wrapper, copied it into /usr/local/bin/feedvalidator, and told Typo’s feed tests to test each feed type using the Python validator code. This helped immensely, and I was able to get all three feed formats back into compliance in no time at all. Frankly, this was the coolest application of unit testing that I’ve ever done, and it was massively and immediately useful. I’ve been trying to get into unit tests for a long time, and this is the first time that I feel like I’m really getting things right.
Hopefully I’ll be able to commit this to the Typo trunk later tonight, and then I can move on to my next little Typo project. I’m not really sure which one to pick up next–either podcasts, context-sensitive sidebars, action caching, statistics, or schema generation from migrations. So many choices.
Tags atom, feed, rss, typo | 1 comment
Posted by Scott Laird
Wed, 22 Sep 2004 20:11:21 GMT
Once upon a time, I loved localfeeds.com. It was a great little RSS service that collected blog feeds, sorted them by latitude and longitude, and sent you a list of your neighbors’ blog postings.
Unfortunately, while it was very useful, it wasn’t very well managed, and it was apparently destroyed in a disk crash. I’ve been pining for it since February. The author has been promising to resurrect it for months, but it’s been a long, slow process without any obvious progress.
Until today, that is, when I noticed that Localfeeds is collecting data again. Ross has a post on the topic. He started the crawler back up in August. According to my logs, it started reading my blog in earnest on September 13th. At this point, NYC and Harrisburg are working; hopefully the Seattle area will return to life shortly.
Posted in Blog stuff | Tags localfeeds, rss, seattle | 2 comments
Posted by Scott Laird
Thu, 15 Apr 2004 19:46:58 GMT
Localfeeds is still down. That’s not exactly breaking news; it’s been down since February. I was really starting to enjoy Localfeeds–it did a great job of showing me what other people in Seattle were talking about. Sooner or later it’ll pop back to life, so NetNewsWire keeps polling it every hour, hoping for something to turn up.
The thing that amazes me is that no one else has jumped in and implemented this themselves. The basic data is all available; it’s just a matter of database coding. And a lot of bandwidth. Still, there has to be a way to make money at this, and I’m amazed that no one has stepped into the void.
Posted in Blog stuff | Tags localfeeds, rss, seattle | 1 comment
