Posted by Scott Laird
Mon, 11 Apr 2005 18:12:25 GMT
Newsforge is running an interview with the three main participants in The Great Linux SCM Saga, Linux, Larry McVoy, and Tridge. By and large, it’s a good article, but I suspect that someone who didn’t know the people involved would assume that the whole mess was Tridge’s fault–he’s the one that was working on cloning BitKeeper, even though any sane person would know that it would really piss Larry off. Even after people pointed this fact out to him, he kept working on his BitKeeper tools.
I’d be remiss if I didn’t point out that Tridge has a history of doing this sort of thing. I’m aware of two other cases where he’s dug in and reverse-engineered similar sets of protocols and file formats. The first time, the result was Samba, which was (and still is) really one of Linux’s first killer apps. The second time, he decoded TiVo’s on-disk media format. Pretty much any tool on the net that knows how to extract video from TiVos (except for TiVo’s recent TiVo-to-Go release) is based on Tridge’s work.
That’s not to say that reverse-engineering is all that he does–rsync is his too.
I remember people questioning his ethics during his TiVo work–besides just downloading video from TiVos, his would could (in theory at least) allow someone to buy a TiVo and feed it program guide information without paying TiVo’s monthly subscription. Without that, TiVo’s revenue model falls apart, and the company would be forced to either sue their own users or go out of business. The BitKeeper folks might have paid attention to how he handled the TiVo issue–as I recall, he released the video download code, but kept the programming guide code to himself. In some ways, that actually helped TiVo–I had no qualms about buying a second TiVo, even when their financial footing was shaky. Without Tridge’s programming guide code, a TiVo box without TiVo, Inc would just be a big paperweight. Just knowing that the program guide code existed was enough to ensure that my TiVo would continue to be useful, because someone would pick up the torch if TiVo fell.
I don’t know what Tridge was planning to do with his BitKeeper tool, but based on his past record, I really doubt that he would have used it to sabotage BitMover. Or, at least not to do anything that he saw as sabotage. Clearly Larry McVoy (and to some extent Linus)
saw things differently.
Getting off of BitKeeper is probably best for Linux in the long run. It’s a pity we couldn’t have waited for another year or so for open-source SCM software to mature more, though. There are a number of promising contenders, but they all have issues that keep them from being usable for the Linux kernel today.
Posted in Linux | Tags bitkeeper, linus, linux, samba, scm, tridge | no comments
Posted by Scott Laird
Sun, 20 Mar 2005 13:24:46 GMT
Playing with my new PC:
# apt-get install xemacs21
...
The following NEW packages will be installed:
xemacs21 xemacs21-basesupport xemacs21-bin xemacs21-mule
xemacs21-mulesupport xemacs21-support
0 upgraded, 6 newly installed, 0 to remove and 21 not upgraded
Need to get 33.6MB of archives.
After unpacking 106MB of additional disk space will be used
Choke, choke. xemacs needs 106 MB of disk space?
Posted in Linux | Tags debian, sysadmin, xemacs | 1 comment
Posted by Scott Laird
Thu, 17 Mar 2005 19:32:08 GMT
The parts for my latest home PC arrived yesterday afternoon and Gabe and I spent a couple hours assembling them into a workable system. It’s amazing how much help a 4-year-old can provide, even around delicate PC parts. I now have a working Athlon 64 3000+ (S939) system with 1 GB of ram and a reasonably large amount of disk space sitting on my desk. I’m going to be using this for Xen testing, but I’ll write more about that later. For now, I want to concentrate on the hardware.
I went out on a limb a bit when ordering this system, because the motherboard I picked, MSI’s RS480M2-IL, is the first motherboard on the market with ATI’s first Athlon 64 chipset (the Radeon Xpress 200), and Google doesn’t give any clear Linux success stories for the motherboard or chipset. However, this is the only socket 939 board that I could find with on-board video, and I really like on-board video for servers. It was also quite a bit cheaper then buying a comparable board plus an AGP video card. A bit of poking around suggested that the SATA ports might be trouble, and it was unclear how well X supports the on-board video, but I don’t really care about either of those for this system. The parallel IDE ports and Ethernet are the only really important parts for me.
So, after installing all of the hardware, I burned a new Ubuntu install CD and gave it a spin. It booted up okay, found the network, found the IDE hard drives, and installed without any serious problems. Ubuntu’s install CD doesn’t seem to have a driver for ATI’s IDE chipset, so I was stuck in slow PIO mode, but it still worked. Once the install finished, I rebooted and watched Ubuntu try to add all of Gnome and OpenOffice to my nice little server system–yikes. After stopping that, I installed gcc, downloaded the source for Linux 2.6.11.4, and build a new kernel.
After booting the new kernel, almost everything looks okay. Here are the drivers needed for this hardware:
- IDE: ATI IXP (in stock 2.6.11)
- SATA: libata’s sata_sil driver detects 4 SATA ports. I have no SATA drives to use for testing, though.
- Ethernet: 8193too (in stock 2.6.11)
- IEEE1394/firewire: OHCI1394 (in stock 2.6.11). Only lightly tested, but able to mount disks connected to FW DVD burner.
- USB: EHCI (8 ports USB 2.0)/OHCI (4 ports USB 1.1). Looks okay, but untested.
I’m currently fighting two problems:
- Massive clock skew–the system clock is running twice as fast as it should. This is usually a power-management issue or a BIOS bug. A lot of new systems suffer from this, and it shouldn’t take too long to fix.
- The system won’t reboot cleanly. Linux shuts down okay, but the system hangs and I need to hit ‘reset’ before it’ll reboot. This is probably related to problem #1.
Update (3/18/2005): Disabling the APIC fixed the clock problem, but not the reboot problem. I tried changing a number of power management settings without success. Most likely, the APIC will start working with a future BIOS revision. This problem seems to be preventing me from booting Xen right now, but that’ll probably be fixed by a new version of Xen in the fairly short term.
Posted in Linux, Computer Hardware | Tags athlon, motherboard, msi, review, rs480 | 155 comments
Posted by Scott Laird
Mon, 14 Mar 2005 19:17:29 GMT
Debian is easily my favorite Linux distribution. It has its issues (horrific installer, tends to value ideology over technology, glacial release schedule), but its core is fantastic, and I’ve grown used to all of its quirks over the years. I think I installed my first Debian box in 1996 starting with either the buzz or rex release (Debian names all of its releases after characters from Toy Story) and I’ve been running at least one Debian system ever since.
One thing about Debian is that it has historically tried to support every platform under the sun. At last count, there are 11 supported Debian platforms, from PCs to PDA-like devices to IBM mainframes. According to The Register, this is going to change, with Debian dropping 7 of their 11 platforms, largely in an attempt to speed up the Debian release process. There are examples where security bugfixes have been delayed for months because the fix won’t build properly on an uncommon platform. In other cases, it appears that there just isn’t enough CPU power to keep up with the build load on older platforms, so the build just keeps falling further and further behind.
The plan isn’t really to totally discontinue support for these 7 architectures, but rather to move them to a new Debian “second-class citizen” support system and not include them new releases.
The platforms that will continue to be supported are:
- i386
- amd64 (Athlon64/Opteron/new Intel 64-bit x86)
- powerpc
- ia64 (Itanium)
Posted in Linux | Tags debian, platforms, sysadmin | no comments
Posted by Scott Laird
Tue, 08 Mar 2005 00:36:14 GMT
I’ve been watching Xen for a while now, and I’m nearly ready to take the jump and do some testing with it. I’m thinking about ordering a cheap Athlon 64 box for home to use as a testbed for the lightweight server concept that I’ve been kicking around for years. In the 18 months that have passed since I last talked about it, virtualization on the PC has advanced by leaps and bounds; at the time, I was looking at UML, which wasn’t really fast or stable enough. Xen looks to be both fast and stable, and it has a clear migration path onto the virtualization hardware offered by the next generation of PC hardware. That makes it nearly ideal for my purposes.
Posted in Linux, Xen, Computer System Administration, LWVS | Tags sysadmin, xen | no comments
Posted by Scott Laird
Wed, 23 Feb 2005 03:08:59 GMT
There’s an interesting thread going on right now on the Linux netdev mailing list, speculating about the network accelerator technology that Intel’s been talking about recently. No one’s quite sure what Intel is planning on adding, but for the past several years “network accelerator” has usually meant TCP offload engines (ToE), and Linux’s core networking guys are almost famously anti-ToE. Even though no one really knows what Intel’s up to, there’s a feeling that it’s not just ToE this time.
Several people have pointed out other technologies that can make a huge difference without requiring the sorts of compromises that ToE needs to work. For instance, this post by Lennert Buytenhek suggests that PCI and memory system latency is a big problem, but fixing it can have huge payoffs:
The reason a 1.4GHz IXP2800 processes 15Mpps while a high-end PC hardly
does 1Mpps is exactly because the PC spends all of its cycles stalling on
memory and PCI reads (i.e. ‘latency’), and the IXP2800 has various ways
of mitigating this cost that the PC doesn’t have. First of all, the IXP
has 16 cores which are 8-way ‘hyperthreaded’ each (128 threads total.)
I haven’t paid much attention to Intel’s IXP network processor family in the past, and that may be a mistake–from the description here, the IXP2800 sounds like a cross between Tera’s multithreaded CPU and IBM’s new Cell processor. Tera’s CPU, which was designed to support tons of threads, automatically switches between threads whenever one thread blocked due to I/O or memory access. The goal with Tera was to be able to remain efficient while the gap between CPU and memory speeds continued to grow. The IXP2800 isn’t as ambitious as the Tera, but the fundamental concept looks similar–support lots of threads in hardware, and switch when latency gets in the way. The IXP2800’s threaded CPUs aren’t full-blown processors, though–like the Cell, the IXP2800 contains one main CPU and a cluster of smaller domain-specific processors that are specialized for one specific task.
It’s unlikely that Intel will roll something like this into their Xeon CPUs anytime soon, though. It’s certainly not a quick fix–it’d require major changes in any OS that wanted to make use of it, and would probably take 3-6 years before it was really fully utilized.
Massively-multithreaded CPUs aren’t the only approach that has paid off for dedicated network processors, though. Some of FreeScale and Broadcom’s chips know how to pre-populate the CPU’s cache with headers from recently-received packets. This drastically cuts latency, but it seems to require that the CPU and network interface be very tightly coupled. Reducing the overhead needed to talk to the NIC can help, too–apparently some of Intel’s 865 and 875 motherboards use a version of their GigE chip that is connected directly to the north bridge, bypassing the PCI bus entirely, and some benchmarks show substantial improvements.
Reading the thread suggests that most of the effort going into Linux network optimization in the next few years will be happening on the receive end of things. Over the past several years, most higher-end NICs have added limited support for checksum generation and TCP segmentation offloading (TSO), where the CPU can hand the NIC a block of data and a TCP header template, and then have the NIC produce a stream of TCP packets without requiring the CPU to touch the data at all. Relatively little has happened on the receive side, but this seems to be changing. For example, Neterion’s newest card can separate headers from data, and is nearly able to re-assemble TCP streams on its own, sort of the inverse of transmit-time TSO. It’s not clear how many streams the card can handle at a time, though–even my little web server at home is currently maintaining 384 simultaneous TCP connections, and a busy system could easily have tens or hundreds of thousands of open streams. Odds are, throwing 100,000 steams at the card would run it out of RAM and completely negate any benefit that receive offloading would have. Unless it’s bright enough to be able to handle the 1,000 or so fastest streams and then let the main CPU handle the 99,000 that are dribbling data at 28k modem speeds.
This is a fascinating topic, and I can’t wait to see how this will turn out.
Posted in Linux, Computer Networking | Tags intel, linux, networking, sysadmin, toe | no comments
Posted by Scott Laird
Wed, 08 Dec 2004 20:50:17 GMT
If you’ve been living under a rock, then you might not have noticed that PalmSource has announced that they’re going to be building a version of the Palm operating system that runs on top of Linux. It’s not completely clear what this means–are they replacing the kernel in PalmOS 6 with Linux, or is this a parallel project, intended to fit into new niches? PalmSource released an open letter to the Linux community that provides a few details:
- Existing 68k-based Palm apps will work fine.
- Apps based on the new Cobalt API will need to be recompiled.
- ARM-based apps for PalmOS 5 aren’t mentioned, it’s probably safe to assume that most of them will break.
- They’re going to enhance the Linux kernel as needed and contribute their changes back to the community.
- It’ll be possible to run Linux apps underneath their UI, but if you want a user interface, you’ll need to use their API. In other words, it’ll be possible to run things like Apache and MySQL on PalmOS for Linux, but not X applications.
- Their licensing model for PalmOS itself isn’t changing–they’re still licensing the whole package to hardware manufacturers and expecting them to port it to their hardware. Presumably, this will become easier when using Linux, because it comes with more drivers and Linux driver programming is a easier skill to hire then PalmOS driver programming.
Of course, that glosses over most of the important issues. Particularly, is any vendor actually going to ship this? Ever? PalmOS 6 (“Cobalt”) was released to manufacturers at the end of 2003, and not only is there no PalmOS 6 hardware available, there aren’t even any rumors of any on the horizon. It’s unclear if PalmOne will ever ship a PalmOS 6 device. It’s entirely possible that the only PalmOS 6 hardware to ship in 2005 will be from afleet of small asian contract manufacturers building for local markets, although Samsung may have something up their sleeves.
Given the glacial rate of PalmOS 6’s adoption, PalmSource will probably be best off focusing all of their attention onto PalmOS for Linux and calling it PalmOS 7, because there’s no way they can carry three software lines–PalmOS 5, PalmOS 6, and PalmOS for Linux. Since current PalmOS 6 applications won’t be binary-compatible with PalmOS for Linux, there’s no way they can call it PalmOS 6.2 and pretend that it’s an extension of the current 6.x line. If they’re going to push a Linux product at all, then they need to push it hard, and they can’t push two “next generation” products that are mutually incompatible.
Which brings up the big question: when will it be ready? After reading their press releases, I don’t thing they’ve been working on this for very long. They certainly aren’t ready to ship anything, and I’d be surprised if they actually have much more then a proof-of-concept port in-house. On the other hand, they have a solid, well-known base to work from, so it’s not like they have to fight with alpha-grade build tools, flaky OSes, and all of the other moving targets that they presumably had to deal with when building PalmOS 6. Porting the current PalmOS to run on top of the Linux framebuffer device shouldn’t be very hard. Adding support for Linux’s network stack might be interesting–as I recall, PalmOS 5’s TCP stack was entirely located in user space, so it the API might not be very close to the traditional BSD socket API, but I don’t really know. Porting 68k apps will be easy; they already have an emulator that runs on Linux and has for years. Adapting it to the new framework shouldn’t require a whole lot of work.
Unfortunately, the one thing that will probably be hardest is the thing that makes PalmOS so unique–it’s filesystem, or rather the lack of one. Traditionally, PalmOS applications don’t really have the notion of saving or multitasking–everything lived in RAM, and switching between programs didn’t involve a whole lot of extra effort. Applications kept their data organized into databases, not files, and they edited the databases directly, without any sort of “save” step. This meant that switching between apps is fast and gives a good user experience for simple applications, but it hasn’t scaled well because it doesn’t provide an easy way to manage block-based storage, like external flash cards or internal hard drives. Instead, PalmOS has had to add an whole extra API for accessing filesystem-based devices, and this has left us in a state where some applications won’t run off of flash cards, and many applications are unable to access data saved on flash cards.
With a virtual-memory based OS like Linux, it’s possible to fake a lot of this with mmap, but that isn’t ideal when you’re dealing with flash cards–it’s easy to wear out most flash cards today by sending them thousands of small writes, and that’s what I’d expect to see when changing a mmaped database. Also, what happens when a flash card is ejected while an application has a file mapped? Linux is never happy when removable devices go away, but causing applications to crash just because the card was removed is seriously user-unfriendly. If mmap won’t work, the big alternative is to copy things to RAM transparently and then copy them back out when done, but that will push the memory requirements up, which will push up costs and limit battery life.
Given all of this, I’d be surprised to see a PalmOS for Linux device before mid-2006, and that’s a long ways away. It’s not clear that the Palm world can wait for another year and a half, falling further and further behind the networking and multitasking abilities of their PocketPC-based competitors. Given that, PalmSource must be feeling a lot of pressure from their licensees to switch to Linux, or they wouldn’t have made this announcement at all.
Posted in Linux, Handheld and PDA | Tags linux, palm, palmos, palmos6 | no comments
Posted by Scott Laird
Thu, 14 Oct 2004 22:43:44 GMT
I just noticed that Busybox 1.00 has been released. If you aren’t familiar with it, Busybox is a software package that provides about 90% of what you need to build a simple Linux system (init, shell, cp/mv/ls/rm, ifconfig, dhcp, etc). It’s commonly used in embedded systems and on install disks.
I’ve found it fantastically useful on a number of occasions, just because it makes it *so* easy to build your own miniature Linux distribution for things like recovery disks, install disks, appliances, and so forth. Busybox, a kernel, a couple empty directories, a few devices in /dev, and a bootloader, and you’re set to go.
Posted in Linux | Tags busybox, linux | no comments
Posted by Scott Laird
Mon, 11 Oct 2004 20:03:37 GMT
So, suppose you’re setting up a test environment, and you want a server to be able to handle some improbably large number of IP addresses, like a /16 or even larger. You could just write a script to add them all one at a time, or you could use this little shortcut and add the entire netblock at once:
# ip route add local 10.0.0.0/8 dev eth0 proto kernel scope host table local
That’ll make the entire 10/8 block local to the box. It’ll answer pings sent to any address in the 10.x.x.x space, and you can bind to any address in the netblock and use it as the source address for packets. For all intents and purposes, it’ll act just like the addresses were added with ip addr add or ifconfig.
I used this last week when I needed to simulate a network with 4,000+ hosts all sending UDP traffic to a specific server. It was a piece of cake.
I’ve seen this a few places online, but it’s never been very easy to find. To the best of my knowledge, it’s worked with Linux back to at least 2.0 or so; I’ve used it with 2.4 and 2.6 recently.
Posted in Linux, Computer Networking | Tags howto, linux, router | no comments
Posted by Scott Laird
Wed, 22 Sep 2004 21:17:17 GMT
Chandler, the open-source PIM suite directed by Mitch Kapor (of Lotus 1-2-3 fame) is making progress. According to Oren Sreebny:
… there will be a 0.5 release in the first quarter of 2005 that will be designed to be usable for basic individual and collaborative tasks in a small workgroup - OSAF intends to adopt that release for their own uses, hence the nickname “dogfood” (as in “eat our own…”) for this release.
There’s also some discussion about collaborative calendaring:
We had a lot of discussion about whether it would make sense for OSAF to elevate the priority of adding calendaring client functionality (using CalDAV) earlier rather than later (that of course begged the question of who was going to build a CalDAV server). The prevailing sentiment in the group was that sounded like a good idea, but we wanted some more detail on the level of effort required and which other features were likely to be deferred to make that happen.
IMHO, workgroup collaboration, and especially calendaring, is the single biggest issue that the open source desktop people face. In most areas (Networking, email transport, email access, file transfer), the industry has adopted open protocols and open standards, and the result has been a huge amount of consumer choice. Calendaring has been the big exception–the only real standard there is Microsoft Exchange, and it leaves users with little to no choice at all. Strangely, the Internet community has failed to come up with a single solid calendar exchange protocol after over a decade of trying; hopefully this attempt will finally be successful.
Posted in Linux | Tags chandler, groupware | 1 comment
Posted by Scott Laird
Wed, 25 Aug 2004 20:45:29 GMT
Re-reading my ”One Year of Blog” post reminded me of another anniversary that I’d forgotten: I’ve now been using Linux for over 12 years. I started in mid-August 1992, with Linux 0.97. It’s grown a bit since then; 0.97’s tar.gz file was around 288 kB, while Linux 2.6.8 is over 44 MB. That’s a factor of 154 growth in 12 years, or a compounded rate of around 50% per year. At that rate, the compressed kernel source will be too big to fit onto a CD by 2011.
Posted in Linux, Personal | Tags history, linux | no comments
Posted by Scott Laird
Fri, 23 Jul 2004 20:45:26 GMT
According to GrokLaw, BayStar, one of SCO’s PIPE investors, is suing SCO. From the press release:
SAN FRANCISCO, July 23 /PRNewswire-FirstCall/ – BayStar Capital today announced that, despite a prior announcement by The SCO Group, Inc. (Nasdaq: SCOX - News) to the contrary, the transactions contemplated by the Stock Repurchase Agreement by and between BayStar and SCO, dated as of May 31, 2004, have not closed due to an unresolved dispute between the parties. BayStar intends to file an action requesting a declaratory judgment with respect to its rights under the Stock Repurchase Agreement. Until a final determination is made by the court, BayStar maintains its position as a Series A-1 Preferred stockholder of SCO.
Posted in Linux | Tags baystar, lawsuits, sco | no comments
Posted by Scott Laird
Wed, 21 Jul 2004 17:46:11 GMT
Groklaw has a preliminary report saying that the judge dismissed SCO’s suit against DaimlerChrysler.
This shouldn’t be too surprising, because the case was a complete mess. SCO was basically suing DaimlerChrysler because DCC hasn’t responded to an audit request that SCO sent. Of course, SCO sent it to the wrong address, to a company with the wrong name. After waiting a few weeks for a response, SCO sued. After the suit was filed, DCC responded, saying something to the effect of “we have no idea who you are, we have no contract with you. You claim to be the successor-in-interest to a contract that we had with AT&T years ago, but we weren’t notified when it changed hands, as required in the contract. It doesn’t matter anyways, we haven’t used the software in question in years. Want it back?”
That wasn’t good enough for SCO, but it looks like it was enough for the judge.
Posted in Linux | Tags lawsuits, sco | no comments
Posted by Scott Laird
Tue, 20 Jul 2004 21:45:00 GMT
Gotta love these guys. According to Slashdot, SCO is claiming that they own the ELF executable format, and Linux is infringing. According to LinuxWorld:
SCOsource chief Chris Sontag, the SCO VP in charge of the company’s hate-inducing IP push, claims TISC, which folded immediately after the spec was published, exceeded its rights even though both Novell and the old SCO - as well as Microsoft, IBM and Intel - were on the committee.
Sontag also says that any entities that ignore SCO’s ELF copyrights are infringing. Such a claim is likely to put SCO on a war footing, if it isn’t already, with the Free Software Foundation, whose GNU operating environment makes broad use of ELF.
Can someone please take SCO aside and explain what copyright means? I mean, this is just too much–copyright doesn’t protect processes or abstract information. That’s a patent or trade secret. Unless Linux copied SCO’s ELF code or is illegally copying copyrighted standards documents, then Linux isn’t infringing on any of their ELF copyrights. Assuming that they even have any.
Since SCO’s suit against DaimlerChrysler is probably going to get hammered in court tomorrow, this is probably just another attempt to prop up their stock price by making wild and unsupportable claims in court. I really need to stop getting all worked up about SCO.
Posted in Linux | Tags insane, sco | no comments
Posted by Scott Laird
Tue, 11 May 2004 01:10:27 GMT
Groklaw mentions that SCO is closing their office in Poland, and that their former manager is starting a Linux company.
The best part is what’s left of www.sco.pl. It’s just classic.
Update: I grabbed a copy, just in case.
Posted in Linux | Tags sco | no comments
