My favorite hack

Something reminded me recently of project that I worked on around 10 years ago. I was a simple, clean solution to a problem that had remained unsolved for over a year. It was also one of the most horrible hacks that I’ve ever put into production.

At the time, I was working for a startup that was suffering from explosive growth. We’d began rolling out new sales and engineering offices all over the US faster than we could build the infrastructure that we needed to manage the offices. We built each office with two small Linux servers; one acted as a file server (with Samba) and the other did email for the office. The problem that we were facing is that each office was completely separate from the others–we had technical and political issues that made rolling out a VPN very difficult, so there was no easy way for salespeople to share documents between offices other than email. Which they did with great abandon, and it was killing our servers in larger offices.

The Great And Glorious VPN Project had tried to fix the company-wide filesharing problem repeatedly without success. There were too many requirements, too little money, and the technology wasn’t mature enough. I assume that they managed to deploy something eventually, but not for years after I left the place.

So, we needed a solution for distributed file sharing. It needed to be relatively quick, compatible with Linux and Windows computers, encrypted when passing over the Internet at large, able to deal with 100+ ms latencies between servers, and ideally not require a full mesh, because we were adding 1-2 new offices per month and having to reconfigure all of them every time would have killed us.

We played around with distributed filesystems like Coda and AFS. None of them fit our needs. We considered some sort of read-only replication with rsync copying each office’s content to other locations, but it was too messy to explain to people. We considered deploying Novell Netware (into a Linux shop) just for this, but it wouldn’t have helped enough. We would have rolled out Windows servers, but Microsoft didn’t have anything useful at the time. We looked at other commercial solutions, but nothing that we found was more than a partial solution.

Until one day, when I was sitting in a meeting about some other topic and the solution just popped into my head fully formed. It was really, really evil, and it would all Just Work. Either that, or it wouldn’t work at all, and it’d only take a few minutes to determine the answer. I think I excused myself (“I think I just solved the file sharing problem”) and walked out in the middle of the meeting to test it.

Remember, each office had 2 servers: one acting as a file server, running Samba, the other a mail server, running sendmail and an IMAP server. Here’s all it took: I configured Samba on each mail server, and had it join the same workgroup (I don’t think we were using domains yet) as the existing file server. So each Windows computer now saw two Samba file servers instead of one. The Samba file server software actually consists of two different programs: nmbd, which handles naming and network browsing, and smbd, which actually serves files. I fired up nmbd on each mail server, but instead of running smbd, I ran stunnel and had it encrypt any traffic that was headed to smbd and ship it off to a single file server in our headquarters, where it was decrypted and handed off to a real smbd process.

So, any Windows computer would browse the network and see two servers. One named something like “” and the other named something like “global-nfs”. When they tried to connect to “global-nfs,” their packets were shipped off to a completely different computer in a different city. And everything just worked. It was dead simple. We’d been looking and spending hundreds of thousands of dollars to solve this problem and having to install new hardware in every office; in the end we spent a couple thousand on a single new file server and rolled it all out overnight with a minor software change.

The ugly part is that Samba shares state between nmbd and smbd, and none of that was actually being shared here. You aren’t supposed to run the two processes separately. It gets confused. Except it doesn’t actually hurt anything in this case. None of the problems actually impact anything that we cared about.

While I was at it, I used the same mechanism to standardize printing across the company, but that’s a story for another day.

Posted by Scott Laird Sat, 25 Dec 2010 08:19:00 GMT

Pancakes, with a side-order of problem-solving

So, lo and behold, it’s Saturday morning. It’s the day before Mother’s Day, and we’re going to be driving most of the day tomorrow, so it seemed like the perfect time to cook something a bit special for breakfast. After thinking about it a bit, I decided that what I really wanted was German Pancakes.

That posed a bit of a problem. You see, German Pancakes want to be cooked in the oven. And we’re in the middle of remodeling.

Our Kitchen, being remodeled

The oven goes about where the white bucket is sitting. Until the remodel is done, we’re stuck cooking with just an induction pad and a convection microwave. There’s no way the microwave will heat up to 450°, which is what the recipe calls for. What’s a hungry programmer to do? Why, improvise, of course. There’s one more cooking appliance available, although it’s not widely used for baking pancakes:

Our gas grill

Technically, there’s no reason you can’t bake in an outdoor gas grill, but getting the temperature control right is tricky. I let it heat up for 15 minutes, and then tweaked the settings to get it to stay around 450°. Then I put a stainless steel frying pan onto the grill (all of the cast iron is packed up for now) and let it heat up for another 10m, and checked the temperature:

Looks like it’ll hold 450° just fine. The recipe calls for 20-25m of cooking time, but that’s always seemed high to me; our old oven usually took 15-20m. After 17m I checked the pancake, and it looked perfect:

There’s nothing like grilled pancake for breakfast, now is there?

Posted by Scott Laird Sat, 08 May 2010 18:57:00 GMT

I need iPad blinders

These iPad reviews and App Store links are starting to get hazardous to my (financial) health. I mean, I really don’t need one. I have 2 Nexus One phones in my pocket, a Kindle DX, and I rarely go anywhere without a Macbook Pro. The biggest thing that I’d use a small tablet for would be to-do/calendar management, and I don’t think the iPad is particularly good at that–it’s not really possible to integrate information from multiple silos (Calendar, To-Do, Email, etc) given the iPhone/iPad architecture. My general opinion hasn’t changed much since the last time I wrote about it.

I’m not convinced that it matters though. Much like Stross, I have a low saving throw versus Shiny!

Posted by Scott Laird Thu, 01 Apr 2010 23:13:00 GMT

iPad: meh

Back in 2004 or so, I desperately wanted a 8-10” tablet from Apple. Something that would let me browse the web and check my mail while I was away from my desk, without forcing me to lug my laptop around. Something that would let me play with new ideas over lunch. Something that could use to build cool stuff.

Somewhere along the way, I decided that it was never going to happen. There just wasn’t enough of a market for it, and the form-factor requires UI that is drastically different from desktop OS X. They’d have to start from the ground up to build it. It wasn’t going to happen.

So, every 6 months or so, when the Apple Tablet rumor popped back up, I just ignored them–they were obviously bunk–and they went away on their own.

That is, until a few months ago, when the rumors shifted dramatically, and it became clear that Apple really was building a tablet. So yesterday morning, I was watching 3 live streams of the launch to see what magical thing Steve and company were working on. And now the rumors and the hype have cleared, and we’re left holding the iPad. Which is more or less exactly what I was looking for in 2004.

And I’m amazed to discover that I don’t want one. Not now, not in 2010.

It’s not like I’m immune to iProducts. I mean, I was literally in a car, on the way to the Apple store halfway through the iPhone announcement. We had to turn around when he announced that it wasn’t going to ship for 6 months. As a family, we’ve bought 9 iPods and iPhones over the years, including the very first model.

So why doesn’t the iPad work for me?

  1. It doesn’t do anything that I can’t already do. I have a laptop. I have a phone. I have a Kindle. The iPad can’t replace the phone or the laptop, and my Kindle doesn’t need replaced right now. Sure, it’s a better web-browsing platform than the phone, but not really all that much better.
  2. It’s too closed. I switched from an iPhone 3GS to a Nexus One a month and a half ago, and I don’t really want to go back. The iPhone is prettier, and Apple has more useful apps, but the N1 is much more useful for me. It has a dramatically better interface to Gmail and Google Calendar. It multitasks, so IM works right. When the browser is too slow loading a page, I can go check my mail and then flip back to check on the page loading later. Random third-party apps sync in the background on their own. Apps can extend the core OS experience trivially; install a Flickr uploading app, and the ‘Share’ button in the existing camera app suddenly knows how to upload to Flickr. Heck, I even replaced the default home page/app browser with Slidescreen, just because it fits what I want out of my phone better. None of this is possible with the iPhone, and none of it will be possible with the iPad.
  3. I think this is really the killer for me: the iPad is really just a media consumption device. I agree with almost everything Tim Bray has to say on the topic–the iPad is 98% oriented towards consuming existing content. Which is nice, sometimes. I mean, that’s all the Kindle is good for, and I love my Kindle. But it’s not what I want in a portable, always-with-me computer.

So what *do* I want in tablet-like device, or really any mobile computer, PDA, tablet, or phone? I want it to make my life better in some meaningful way. I want to build things with it. I want to look at how I do things in my life and create tools to make myself more powerful. And I don’t believe I can do that with the iPhone or iPad. They’re too tied to Steve’s View Of The World, and I’m not Steve. Almost everything that I’d like to have a device like this automate for me requires some sort of background processing, syncing with services online, and the iPad can’t do any of that right now. Maybe that’ll change in the future. Maybe iPhone/iPad OS 4.0 will fix that. Maybe the second-generation iPad will be able to do some of that. Maybe they’ll be able to convince me to fork over $100 just to write software for my own device. Maybe.

For now, though, I’m planning on sticking with my Nexus One, and I’ll probably keep an eye open for mid-sized Android tablets with high resolution screens. Android absolutely lacks the polish and shine of Apple’s products, but it’s moving amazingly fast, and it finally seems to be Good Enough for me. It’s reached a sort of critical mass, and it’s developing into a viable ecosystem of its own. At this point, the only thing that I really miss from my iPhone is the Kindle reader app, and I assume that Amazon is working on one for Android. Android’s not perfect, but it’s getting quite good. And while it lacks Apple’s polish and shine, it doesn’t have the same shrink-wrapped, pre-packaged, plastic feeling that so many interactions with Apple have these days.

Posted by Scott Laird Thu, 28 Jan 2010 18:12:00 GMT

Finally signed up for Facebook

I’m currently filled with self-loathing, as I failed in my attempt to be the last person in North America without a Facebook account. It’s still entirely unclear why I’d want to share a single social network with my grandmother, my neices’ friends, and a bunch of current and former co-workers. I mean, really–what could I possibly say that’s appropriate and interesting for all of them?


Posted by Scott Laird Sat, 12 Dec 2009 09:13:00 GMT

Around the World in 365 Days

I know that I’ve been on the road a lot lately, but TripIt just made it a bit more real when it told me that I’ve travelled over 21,000 miles so far this year. The Earth’s circumference seems to be around 24,900 miles, so I’m only 3900 miles short of making a round-the-world trip this year. And that’s without the 3 trips left on my travel schedule–Mexico next week, Vegas the week after, and Dublin a few weeks after that. It looks like I’m going to end up covering around 30,000 miles and will have spent over 60 days on the road in total.

And, frankly, I doubt that next year will be any different. If anything, it might be busier.

Posted by Scott Laird Thu, 05 Nov 2009 01:47:00 GMT

New Stuff Day

It looks like today’s shaping up to be really a really entertaining day for product announcements.

First, Canon just announced the 1D mk IV, which may finally close the lead that Nikon opened with the D3 a couple years ago. The big news with the mk IV is ISO 100-12800, with the ability to go up to ISO 102,400 in a pinch, which is more or less the same as the D3s that Nikon announced last week. If I was in the market for a new camera (and I’m not completely convinced that I’m not), the mk IV is interesting. It doesn’t have a full-frame sensor, but in all other respects it looks just about perfect.

Next, Apple’s due to announce a boatload of stuff at any minute. Current rumors suggest new iMacs, a new Mini, a new Airport base station (with multiple antennas, and maybe multiple radios), a new mouse with iPhone-like touch scroller, and some sort of desktop touchpad.

Finally, for some reason Barnes and Noble decided that today’d be the perfect quiet news day to get loads of press coverage for their new ebook reader. We’ll see how that works for them.

Posted by Scott Laird Tue, 20 Oct 2009 14:43:00 GMT

New iPhone Time

I wasn’t exactly one of the first people in line to buy an iPhone when they first came out, but I did own and use a first-generation iPhone for about a year and a half. It was a great little phone, but 3 things bugged me:

  • It’s slow. Oh, man, so slow. Apps open slowly. Things download slowly.
  • It doesn’t have all that much storage space. 8 GB just isn’t enough. It was always full.
  • The camera sucked.

Earlier this week, I gave in and bought a shiny new iPhone 3GS. So far I’m really happy. It was cheaper than the first-generation iPhone was, and the camera seems better. Plus the reception is slightly better in my new office. And it’s actually kind of zippy, while the old iPhone had felt sluggish since I upgraded it to 2.0 or so.

I’m headed off to New York next week, and I hate dragging a real camera along on business trips, so I’ll see if the 3GS is enough to make me happy on that front.

Posted by Scott Laird Thu, 24 Sep 2009 22:37:00 GMT

Back from the dead

The past two months have been nerve-wracking at work–I’ve been working on a big project that should be completely invisible to users, but should make life a lot easier for those of us who run things behind the scenes. The potential for failure was huge, and I did everything I could to prepare and test and tweak the process. Fortunately, it looks like it’s all paid off; things went off almost perfectly yesterday, and unless something catastrophic happens this week, I’m going to be able to move on to a pile of new, more interesting, and hopefully less stressful projects. That’s the penalty for success, right?

I kind of regret missing Gnomedex this year, as looks like a lot of old friends were there, but I just couldn’t afford the time away from work in the middle of things. Maybe next year. In fact, I’m not quite sure where my summer went–I’ve been concentrating on this since early July; there are a huge pile of home improvement projects that didn’t get done last year and I’d put off until spring or summer this year, and I haven’t even touched them yet. Hopefully the weather will hold through the end of September, and I’ll be able to get a bit of yard work in.

Posted by Scott Laird Sun, 06 Sep 2009 15:50:00 GMT

Kindle DX Review

I’ve been too busy traveling and reading my Kindle DX to get around to writing up the review that I promised last month. Since it’s been out for a month now and people have been reviewing it the whole time, I’m just going to cover the things that have made me especially happy and unhappy.

In general, the DX is a nice step up from the Kindle 1. The larger size is, as expected, a mixed blessing. The bigger screen is great, especially when reading PDFs, but the extra size and weight makes long reading sessions slightly less comfortable than the Kindle 1. I almost exclusively used the Kindle 1’s left-side page changing buttons, and I’m amazed that Amazon left them off of the DX.

I took my DX to Boulder with me last week and read 4 books along the way. The DX may be larger than my old Kindle 1, but it’s so much thinner that it’s easier to travel with. It’s easy to slip it into airline seat-back pockets and it fits into my laptop bag much better than the Kindle 1 did.

A few gripes:

  • It’s crashed a couple times on me. Holding down the power button forced a reboot.
  • With the wireless on, the battery life’s kind of short. Don’t expect more than a few days between charges, even if you aren’t using the DX every day.
  • The Kindle DX really needs a set of left-hand next/previous buttons.
  • The Kindle 2/DX software is a nice improvement over the Kindle 1, but it still needs work. It’s great for reading books, but once you give it more than 100 titles, you’re going to be doing a lot of scrolling. Amazon needs a better way to manage your library.
  • Wikipedia is ugly and slow on the Kindle. Amazon should spend some time building a Kindle-optimized version of Wikipedia. A bit of CSS and then optimizing the HTML a bit would go a long ways towards making it more usable.
  • PDFs with lots of small text are hard to read. Beautiful Security from O’Reilly is rendered in a tiny little font that’s readable on the DX, but it’s so small that it’s uncomfortable and it kills my reading speed. Removing the whitespace around the outside of the PDF would help with this; I’m going to look for a PDF post-processor to do it for me, but the Kindlle should really be able to handle this on its own.
  • The page tilt sensor is dumb. It’s prone to rotating your book for you while you’re busy reading. Fortunately, it’s easy to turn off.

All in all, I’m happy with my DX. I’d buy it again tomorrow if I had to (and if it didn’t have a ~1 month waiting list at the moment). If you’re looking for an ebook reader that can read PDFs and handle complex documents, then it’s the best thing on the market, *IF* you can live with the price and you don’t want to put more than 100 documents at a time on it. If you need more than that, then expect to spend a lot of time scrolling through the list of available books on the Kindle.

If you don’t care about PDFs, then I’d recommend that you just buy the Kindle 2. It’s way cheaper ($299 instead of $489), lighter, and smaller, and it’s more or less the same size as a paperback book. If you can handle cheap paperbacks, then you can live with the Kindle 2’s screen.

Posted by Scott Laird Thu, 09 Jul 2009 16:49:00 GMT

Kindles, Kindles everywhere

I’ve owned an Amazon Kindle for almost a year now, and I’m still really happy with it. A few highlights of the year:

  • I’ve read at least 80 Kindle books. 68 of those were purchased from Amazon, a few were free books from Tor, and a few were purchased from Baen.
  • Of those, probably 10 were read via the Kindle app on my iPhone. It’s a lot less enjoyable than reading on my Kindle, so given the choice I’ll always grab the Kindle first, but I can keep my iPhone in my pocket and pull it out at lunch or while I’m stuck waiting in line. Since it syncs with the Kindle over the air, I’m able to hop back and forth between the two devices and still have the right page sitting and waiting for me.
  • By and large, I’ve found that I enjoy reading books on the Kindle substantially more than I enjoy them on paper. I’ve probably only read 5 paper books over the past year.
  • My original Kindle broke in February. I came home from work and picked it up, and half of the screen was dead. I called Amazon’s 800 number and they overnighted a replacement to me. Everything synced over to the new Kindle and I was back in business.

I’m about 95% happy with my original Kindle. The button placement is still kind of annoying, but over time I’ve gotten better at not pressing the forward and back buttons accidentally. It’s slow startup is kind of annoying, and mine crashes about once per month and needs me to pull the battery to get it working again. Over all, though, I’m happy enough with it that I’d willingly buy it all over again.

Although, if I was going to buy one over again, I’d buy a Kindle DX. When I originally bought my Kindle, I’d carry it with me everywhere to read when I had spare time. Now I mostly only bring the Kindle when I’m traveling and keep my iPhone in my pocket, so the DX’s larger size isn’t a liability. At the same time, the DX’s larger screen will make it usable for reading technical books in PDF form, which never really worked right on the Kindle 1. The DX starts shipping next Wednesday. Mine’s due to arrive on Friday. I’ll let everyone know how it goes :-).

Posted by Scott Laird Fri, 05 Jun 2009 04:33:00 GMT

Off to California

One of the things that made last year so interesting at work was the amount of travel that I did. For the previous 8 years, I flew maybe once per year for work, either to visit remote offices or for conferences. Last year was a bit crazier–I think I had eight business trips between April and October.

This year’s been a bit tamer so far, with only one trip to New York and one to California, but the slow stretch is over now–I’m spending all week next week in Mountain View, and then heading to Boulder at the end of the month for three days. I’ll be back in California for almost two weeks late in July and early in August, and then I’ll probably be in New York in early September.

This will be my first trip to California in a while where I’ll have some free time; if anything cool is happening next week, please let me know :-).

Posted by Scott Laird Fri, 05 Jun 2009 04:05:00 GMT

Best HTPC media-center platform?

At some point in the next couple months, I’d like to buy a new media-center system to go with my new 50” living room TV. Right now, I have a slightly older MythTV install in the basement, and I’ve been watching video via a pair of Xbox 360s. They both have a few issues, though, and I’d like to find something better. I’ve considered customizing a newer MythTV build, and I’ve played with Boxee on a Mac, but before I invest a bunch of time and money into this, I figured I should look for advice. Here’s what I’m looking for:

I need the ability to:

  • Talk to a SMB or NFS file server.
  • Play video in .avi, MP4, VOB, MKV, or OGM files with common codecs.
  • Play back content up to 1080p without stuttering.
  • Change audio tracks and subtitles for files that have multiple tracks.
  • Control the whole thing via a remote control.
  • Play video via HDMI, or DVI plus optical audio out.

It’d be nice if it could:

  • Play YouTube and Netflix videos.
  • Play DVDs or maybe Blu-ray disks.
  • Be controlled via HDMI-CEC.

I don’t care about:

  • Recording or playing OTA or cable HDTV signals.
  • Ripping disks
  • RSS feeds, weather, etc.

I don’t really care about Linux vs Mac vs Windows in this case. I’d be okay with some sort of appliance, but I’ve never found one that actually works right. I have a Popcorn Hour A-100, and it’s about an order of magnitude slower than what I’m looking for, plus it tends to crash or lose audio sync on a lot of the more obscure files that I’ve thrown at it.

Any suggestions?

Posted by Scott Laird Tue, 26 May 2009 21:41:00 GMT


All I can say is that right now, I really wish I hadn’t given away my last copy of To Serve Spammers, if you know what I mean.

Posted by Scott Laird Sun, 19 Apr 2009 15:37:00 GMT


As mentioned a few days ago, I’m using Vyatta for my home router software. Vyatta is amazing; it’s a complete open-source router platform based on Linux. It’s something that I’ve been looking for for the last decade. You just boot up the CD image and wham–it’s a router, with a Juniper-ish command shell. Type configure and you’re in router configuration mode, with context-sensitive editing. Type a couple more commands and it’ll copy itself off of the CD and onto your hard drive or USB drive. It doesn’t get much easier than that.

At the same time, it’s impressively powerful. It still lacks a few features that upper-end Cisco or Juniper routers have–no MPLS, no policy routing, and IPv6 support is weak. But it’s a huge step above any of the Linksys or D-Link routers that I’ve seen. It supports BGP and OSPF, plus reasonably flexible NAT and ACL settings. I’ve never benchmarked my router, but after 2 weeks of uptime it claims that it’s spent 99.9% of its time idle while copying almost 750 GB of data between interfaces. Vyatta claims that a 4x2.66 GHz Intel CPU can route 3 Gbps of 512 byte packets, and I see no reason to doubt that.

Vyatta is open source, but it has a company behind it (also named Vyatta), selling support to anyone who will pay. I’m always conflicted when I run into projects like this. I’m happy that they’re available, and that they’re making progress forward, but they only rarely develop any sort of community around them. Maybe Vyatta will prove me wrong.

Posted by Scott Laird Sun, 19 Apr 2009 05:15:00 GMT