Posted by Scott Laird
Mon, 18 Apr 2005 18:25:32 GMT
It looks like Cisco is finally starting to push their new, modular IOS code down from their uber-expensive CRS-1 router into their merely amazingly expensive routers. Network World is reporting that they’re almost ready to release a version of IOS XR for Cisco 12000-series routers. So now you’ll be able to run a semi-modern operating system that implements things like memory protection between processes on routers that cost under $500k. When they get under $100k, this might start to be interesting.
Cisco’s also going to release a line of “shared port adapters” that can be used in routers from the 7300, 7600, 12000, and CRS-1 product families. You need a SPA Interface Processor card for your router type, and then you plus SPAs into the SIP card. According to their picture, the SIP for the 7600 family can hold 4 SPAs, which means that the SPAs themselves must be fairly small–almost certainly smaller then the PAs that 7200/7400/7500 routers use. There are a pile of different SPAs on their list, from 8x cT1 to 10x GigE to 1x 10GigE, to OC 192.
Of course, in typical Cisco fashion, the “shared port adapters” aren’t really all that shared. There are 3 different SPA carrier cards for the 7600 series; one model is only good with the VPN SPA, and the other two overlap a bit–both are good with 2x or 4x OC-3 SPAs, but one supports 1x OC-12 while the other supports T1 and T3 SPAs. None of the three models support the GigE SPAs. Ther 12000-series has 2 different SIP models; one is good for T1/T3 use, while the other one is good for GigE and OC-192s. The CRS-1 SIP is even more fun–it supports POS OC-3s, POS OC-192, and GigE. No OC-12 or OC-48 support, apparently.
So, even though the cards are called “shared port adapters,” there are some real limits on which chassis will work with which cards. The DS3 SPA will apparently only work on the 7600 and 12000, and not on the CRS-1 or 7304. I suspect that a lot of this is mostly a driver support problem, but it shows how screwed up and fractured Cisco’s product lineup is.
Posted in Computer Networking | Tags bfr, cisco, crs1, ios, networking, router | no comments
Posted by Scott Laird
Tue, 08 Mar 2005 07:23:21 GMT
It’s sort of an axiom of programming that features that aren’t continually used or tested won’t actually work. A similar rule holds for system administration–any feature that hasn’t been tested since the last upgrade is probably broken. An obvious corollary suggests that systems get more reliable as their user load increases–more users means more features are used more frequently, and broken features will be spotted sooner. And the corollary to that is that any server wedged under a desk in someone’s home office is probably flakier then hell because it’s probably just sitting there collecting dust and not getting used.
I’m not convinced that that applies to my home gateway box. It’s a busy little beaver:
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes prot opt in out source destination
234M 75G all -- dsl0 * 0.0.0.0/0 0.0.0.0/0
47M 1001G all -- eth0 * 0.0.0.0/0 0.0.0.0/0
In the 25.75 days since I last rebooted this system, it’s received over 75 GB via its DSL link and around 1 TB over its main Ethernet link. If my math is right, that’s an average of 3.6 Mbps on the Ethernet link and around 270 kbps over DSL. I wasn’t keeping outgoing traffic stats when I first booted this box, but more recent estimates make it look like there’s almost as much outgoing traffic on dsl0 as there is incoming.
CPU load is similarly heavy–the box has averaged 51.9% idle since it was rebooted. My rule of thumb for years was that any production box that was under 80% idle was due to be upgraded soon, because it was probably pegging the CPU during peak times during the day. If the box was under 70% idle, then it was time to start scrounging for an immediate upgrade. By those metrics, this box is way overdue for a major upgrade. Fortunately for my wallet, those metrics don’t really apply to this box–it’s spending a lot of its CPU time on tasks that aren’t particularly critical. Also, Linux 2.6 made some changes to /proc/stat that procinfo doesn’t seem to have picked up on; once you factor those into the equation, the box is really closer to 75% idle. Subtract off the non-critical usage, and the system is probably only 10% busy. I’ll probably upgrade it later this year if my virtual-server project works out, but that’s more for security and reliability then pure performance.
Posted in Computer System Administration | Tags dsl, home, linux, networking, router | no comments
Posted by Scott Laird
Tue, 09 Nov 2004 17:42:27 GMT
Slashdot has an article this morning on the OpenBSD people’s new BGP daemon, OpenBGPD. In essence, the OpenBSD people did the same thing that they’ve done repeatedly before, and taken a protocol that didn’t have an open, secure implementation and provided a clean, minimalistic, BSD-licensed tool.
Personally, I find OpenBGPD kind of fascinating, because I’ve worked with router jockeys for years, and I get dragged into “can we run a BGP daemon on this PC” discussions with surprising frequency.
OpenBGPD’s stated goals include this fun little snippet:
Provide a lean implementation, sufficient for a majority. Don’t try to support each and every obscure usage case, but cover the typical ones
And that’s where my problem lies. I don’t think I’ve ever been asked for a “lean implementation” of BGP. Every time I’ve been dragged into a BGP discussion, it’s been because network engineers have been trying to do something bizarre and creative with BGP, and the tools that they’re used to using aren’t sufficient. For instance, at Internap, we wanted to add per-prefix, per-peer prepending for a huge number of prefixes, and we wanted to change the path selection algorithm to include a bunch of extra information that we had on reachability and performance. In other cases, I’ve been asked for simulators and BGP loggers that could feed BGP prefix reachability information into a database. Inevitably, every time someone needed just a “lean implementation,” they’d already have a Cisco box handy and they’d use it instead of monkeying with BGP on a PC.
That’s not to say the PCs make lousy routers or anything like that–the price/performance is impossible to match with anything from Cisco–but that the totals costs involved in any BGP peering that I’ve seen make the cost of the router little more then noise in the equation. If you’re paying tens of thousands of dollars per month for multiple pipes to providers, then what does saving $20k on a router buy you, besides maintenance and reliability headaches and a hard time finding network engineers familiar with your setup? Most of the time, it’s cheaper to spend $20k on hardware and make it up on productivity and reduced downtimes.
So, while OpenBGPD is cool, I’m not sure how useful it really is outside of test labs and maybe small ISPs, if there are any of them left. On the other hand, I’d love a good OpenBGPD-ish OSPF implementation. I’ve played with Zebra, and the whole design of the thing just rubs me wrong (although Quagga might be better). I need to remember to actually give Xorp a try, too. OSPF is more useful inside of existing networks, and it makes a lot more sense on a LAN then BGP does.
When it gets down to it, I suppose my real point is this: it’s largely pointless to scale PC-based routers up to make them compete toe-to-toe against Cisco’s big WAN routers, because the network
costs and the maintenance costs of doing one-off routers works against us. It’s also really hard to get reliable, well-tested WAN interface cards for anything faster then a T1. Try finding a PCI OC-12 POS card with Linux drivers sometime.
On the other hand, other alternatives make a huge amount of sense:
- Scale them down. You can build a cheap Linux router for almost no money these days–look at the Linksys WRT54G.
- Scale them out. Imagine a medium sized company replacing all of their assorted branch office routers with PCs talking to DSL and providing QoS, routing, firewalling, VPNs, VoIP, etc. It’s expensive to do it once, but you can replicate the work onto a hundred devices for very little additional cost.
- Push them into niches. There are cases where the fantastic flexibility of PCs can make them much more useful then an equivalent Cisco. Linux, for example, has no problem running multiple routing tables and a fantastic number of firewall rules. You can do amazingly creative things with just the stock tools, if you can figure out how to use them.
Posted in Computer Networking | Tags bgp, cisco, linux, openbgpd, openbsd, router | 1 comment
Posted by Scott Laird
Mon, 11 Oct 2004 20:03:37 GMT
So, suppose you’re setting up a test environment, and you want a server to be able to handle some improbably large number of IP addresses, like a /16 or even larger. You could just write a script to add them all one at a time, or you could use this little shortcut and add the entire netblock at once:
# ip route add local 10.0.0.0/8 dev eth0 proto kernel scope host table local
That’ll make the entire 10/8 block local to the box. It’ll answer pings sent to any address in the 10.x.x.x space, and you can bind to any address in the netblock and use it as the source address for packets. For all intents and purposes, it’ll act just like the addresses were added with ip addr add or ifconfig.
I used this last week when I needed to simulate a network with 4,000+ hosts all sending UDP traffic to a specific server. It was a piece of cake.
I’ve seen this a few places online, but it’s never been very easy to find. To the best of my knowledge, it’s worked with Linux back to at least 2.0 or so; I’ve used it with 2.4 and 2.6 recently.
Posted in Linux, Computer Networking | Tags howto, linux, router | no comments
Posted by Scott Laird
Wed, 09 Jun 2004 18:55:21 GMT
Network World is reporting that Juniper is about to ship a pile of new low-ish end routers designed to compete with Cisco’s NM/WIC-based router lineup. They don’t have a ton of details, but there seem to be 3 models, the 2300 with two FE ports and two WAN ports, the 4300 with 6 WAN ports, and the 6300 with a faster processor and 6 WAN ports.
The quoted throughput rates are really low–the 2300 is only rated for 8 Mbps, which puts it in the same territory as an ancient Cisco 2500, which has less horsepower then most modern cell phones. Given that, I suspect that Network World is actually quoting VPN throughput. We’ll see next week.
Unless I missed something, this is Juniper’s first foray into this area of the market. It’ll be interesting to see how well they can compete, especially since Cisco is aggressively rolling VoIP and Ethernet switching abilities into the higher end of their competing products.
Posted in Computer Networking | Tags cisco, juniper, router | no comments
Posted by Scott Laird
Mon, 19 Apr 2004 19:00:54 GMT
CNET is running a blurb on Xorp, the eXtensible Open Router Platform. According to the article, Xorp’s authors are hailing it as “the Linux of routing.” Since open-source router platforms are one of my interests, and I’d never really heard of Xorp, I just took a quick look and was pleasantly surprised.
It’s based on the Click modular router from MIT, which I’ve been fascinated with for years. Click is largely a replacement for Linux’s (and potentially other free OS’s) networking stack. It slides in between the hardware network interface drivers and moves the kernel’s native packet handling off to the side. Click’s packet processing is completely programmable; you can write network switches, routers, VPN hardware, mesh routers, programmable network test hardware, or nearly anything else with Click, all without fighting against the kernel’s native packet handling. More importantly, Click is freakishly fast; one of the demos I read about a couple years ago was handling roughly 1 million packets per second on a normal dual-CPU PC.
The big problem with Click was that it was difficult to configure. You had to understand IP networking at a very low level in order to make heads or tails of it, and even a simple 2-port router took most of a page to configure. Multiport routers or switches were doable, but you wouldn’t want to set them up by hand.
Unless I’m mis-reading things, this is where Xorp comes in–it’s a full wrapper for Click. It looks like it provides a CLI, dynamic routing support, and a simple configuration language. Xorp’s backend then sets Click up and off it goes, flinging packets around at freakish speed (er, freakish for a PC, that is–I wouldn’t put it up against a Catalyst 6000 or faster).
Xorp 1.0 is due out soon. I haven’t downloaded it yet, or even looked through more then one whitepaper, but I’m looking forward to playing with it. Assuming that someone goes to the trouble of gluing this together with a small Linux distribution (I think it’s Linux-based; their website and whitepaper don’t say, but Click only works on Linux and FreeBSD, and the FreeBSD layer has been broken for a while), it should be an easy way to build cheap mid-speed routers.
Posted in Computer Networking | Tags bsd, click, linux, router, xorp | no comments
