Posted by Scott Laird
Mon, 22 Oct 2007 21:39:47 GMT
I mentioned that I bought a Gigabyte GC-RAMDISK (a.k.a. i-RAM) to go in my new home file server, largely to see if using it as a solid-state log device would improve ZFS performance.
Unfortunately, I’ve been completely and totally unable to get the card to do anything at all. I’m not sure if I have a defective card or if Gigabyte’s SATA implementation is just really buggy. When I plugged it into the motherboard’s ICH9R SATA ports, the BIOS didn’t even show it on the boot-up scan and Solaris reported it as failing to initialize correctly. When I plugged it into the Supermicro AOC-SAT2-MV8 8-port SATA card, the BIOS could see it but Solaris gave a similar error. Connecting it to the motherboard’s Marvell eSATA ports made the Marvell hang at bootup and made Solaris really unhappy, spewing drive failure messages all over the console.
I can’t find a single review that suggests that anyone has got this to work with a recent motherboard. Digging through the Linux kernel mailing list suggests that it has a really spotty SATA implementation. Apparently they developed it using a couple Windows drivers as a comparison, instead of actually paying attention to the SATA spec.
So, it’s going back to Amazon today. It was a nice idea, but it just doesn’t appear to work. I don’t know if it’s broken or just poorly designed, but either way it’s not useful to me.
Tags broken, gc, gigabyte, i, ram, ramdisk, ssd, zfs | 8 comments
Posted by Scott Laird
Sun, 21 Oct 2007 03:33:00 GMT
I now have Solaris up and running and reasonably stable-looking, after only 12 hours of work. A number of things turned out to be bigger issues than I’d anticipated, largely because it’s been years since I last used Solaris and, frankly, Solaris’s disk partitioning and formatting tools suck.
- My first problem is still unresolved: my BIOS refuses to boot from the IDE DVD drive that I installed. Once the system boots, it works just fine, so I’m not sure what’s up. Maybe a BIOS bug. Fortunately, the system’s perfectly happy booting off a USB DVD drive, and (amazingly) Solaris is happy installing from it.
- The GC-RAMDISK card that I was looking forward to testing is a complete failure so far. I don’t know if I have a bad card or if it’s simply incompatible with both SATA chips in the system, but the BIOS completely ignores it if its plugged into the motherboard, and Solaris fails to talk to it on either bus. If it’s plugged into the MB, then I get a
device failed initialization error; if it’s plugged into the PCI-X SATA card then I get device on port 5 still busy after reset. I’ve swapped cables and RAM. I’d really like to get it to work, so I’m going to try it with an older system before RMAing it.
- Actually getting a working Solaris install took me 3 tries. The first time I installed it to the wrong drive (the first disk on the PCI-X card, not the first disk on the motherboard), and it was unable to mount the root partition after rebooting. Next, I managed to install it onto an EFI partition, and that wouldn’t boot either. Finally, I installed it onto the second drive on the right bus, and that worked.
- Since Solaris’s installer doesn’t support ZFS yet, I had to manually copy the root filesystem onto a newly created ZFS filesystem mirrored across a pair of drives. The directions are helpful, but I kept screwing things up. First, I accidentally created the ZFS pool using the entire disk, which made ZFS re-label the drives with EFI, which makes them unbootable. Then, I missed the line in the directions that says to run
format -e instead of using format; that left me with a pair of nicely partitioned drives that still used EFI. The third try worked, and the system is now booting off of ZFS via GRUB without problems. Er…
- Well, one problem–I can’t change the GRUB
menu.lst file for some reason. I don’t know where GRUB is looking for it, but it’s not in /boot/grub/menu.lst on my boot array. My changes are being completely ignored. I can live with this for the weekend.
- OpenSolaris doesn’t ship with drivers for the ASUS P5K WS’s onboard Ethernet chips. I had to grab them from Marvell, but it was easy enough to install.
- Creating an 8-drive ZFS filesystem is trivial. One command takes care of RAID, logical volume management, creates the filesystem, and mounts it:
zpool create -f space raidz2 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c3t2d0 c3t3d0 c3t4d0 c3t5d0.
- ZFS performance is decent. Here’s a bonnie with and without ZFS compression, using 10 GB of data on a box with 2 GB of RAM:
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec %CPU
zfs 10 105.0 55.3 163.3 27.3 121.0 30.4 119.2 88.4 287.1 36.2 169 1.8
zfs+c 10 112.9 59.7 181.5 30.3 127.8 29.1 118.1 86.0 424.9 52.2 198 2.1
- 163 MB/sec writing and 287 MB/sec reading is good enough for me. I was expecting slightly higher numbers, but there’s nothing here to complain about. Adding compression improves writing a bit and makes a big difference reading. It’s quite a bit faster then GigE, which was my goal.
Tags gs, opensolaris, ramdisk, server, zfs | 6 comments
Posted by Scott Laird
Tue, 28 Jun 2005 02:28:37 GMT
Somehow I missed this earlier this month–Gigabyte has announced a $50-ish PCI card that takes up to 4 DDR DIMMs and acts like a SATA RAMDISK. It has a battery that supposedly lasts 12-16 hours and will recharge via the PCI standby power line.
I’ve seen a bunch of people excited about using this as a boot disk or a Windows paging disk, but personally I’d love to see this used as an external journal for EXT3 filesystems. For some workloads, this would result in huge performance boosts for an amazingly small amount of cash. It’d be nice to have more battery life (36-72 hours would be ideal)–my personal record for a home power outage is 13 hours. All of my work-related outages have been brief, except for the facility in Manhattan that was dark for about a week in September of 2001.
via Ambient Irony
Update: I’ve been thinking about this, and the whole thing would be massively more useful with a couple small additions. First, add a Compact Flash socket to the board, and then update the ASIC that runs the board so it will copy the contents of the RAM onto the CF card after an hour or two without power. Then copy it all back when the power comes back up. You should be able to buy 1 GB of DRAM and 1 GB of CF flash for around $150; adding $100 for the PCI card give you 1 GB of seriously non-volatile memory for $250. I’d probably make them a standard feature in every server that I bought, just for the performance boost.
Posted in Computer Hardware | Tags ddr, gigabyte, journal, nonvolatile, raid, ramdisk, sata | 1 comment
