scottstuff.net

Pricing for Hugo on Google Cloud

Jun 8, 2022

As I mentioned a week and a half ago, I’ve shifted this blog from being self-hosted to living on Google Cloud, because I didn’t want to spend time maintaining a web-serving infrastructure at home.

Obviously, there were many ways that I could have solved this problem. I decided on Google Cloud for a couple simple reasons. First, I work for them, but have very little experience as a customer (anti-disclaimer: I pay retail prices without a discount). Second, I already have an account and billing set up, even though all I’ve really been using it for is DNS hosting. And third, it looked about as simple as any of my options.

My one concern was cost. Almost nothing in Google Cloud (or Amazon, or Azure, or anyone else) is free. The full price list is available, which is nice, but it’s hard to figure out exactly what this was going to end up costing me. I’ve only been paying a few dollars per month for Google Cloud DNS hosting. I figured my price would go up, but there were too many unknowns to be able to figure it out from the price list alone. I was expecting Netflix-ish monthly prices (spoiler: yep), and knew that I wasn’t going to end up with a giant bill, but beyond that it was a bit of a mystery.

So, it’s been over a week since the last change now, and the daily billing numbers from Google Cloud are pretty steady. I’m going to give prices in US$ per week, just because I have a week’s worth of actual data. The bulk of the bill is for Cloud Load Balancing, which is costing me $4.20/week. Beyond that, the numbers are tiny. I splurged for Google’s CDN when setting this up; the last week has cost $0.19 in CDN bandwidth and $0.12 in CDN lookups. My Cloud Storage bill for the whole blog, including images, is a whopping $0.01 for the past week. A bit under $20/month in total.

These numbers will vary with load, of course. The Load Balancing bill is (unfortunately) just the minimum charge for having a load balancer at all. The Cloud Storage bill is proportional to the total size of your content. And the network numbers will obviously go up with traffic. Of which I have approximately none. Adding 10x as much traffic would probably cost me an extra $2/month.

So far, I’m reasonably happy. It took me an hour or so to set up, including SSL (using an auto-generated SSL cert, presumably from Let’s Encrypt), redirections from HTTP to HTTPS, redirections from www.scottstuff.net to scottstuff.net, and a redirection from an ancient RSS URL to the current version. Could I do this for less money? Absolutely. Could I do this, on my own domain, with auto-managed SSL certs, with custom redirect rules, via IPv4 and IPv6, via a global CDN, with monitoring, for less effort and less money? Probably, but not a whole lot less work or much less money.

SAS counters on linux

Jun 4, 2022

There isn’t a lot of documentation on debugging SAS-specific storage problems under Linux, unfortunately. I’ve had a couple issue recently that were tied to flaky SAS cables and mostly had to debug them through swapping cables and logic to figure out which device was causing issues. Along the way I’ve learned a few helpful things.

First, Linux actually has counters for tracking SAS-level problems. They’re just really hard to find, and very few tools expose them. The kernel would love to tell you that the connection between your SAS expander and drive 23 is flaky, but it’s not really going to go out of its way to tell you. You need to go looking for it.

Juniper NFX250, part 1

May 29, 2022

As mentioned earlier, I’ve been playing with using a Juniper NFX250 as a backup home router for the past month or two. The NFX line (and the NFX250 specifically) is kind of a weird beast. It has most of the capabilities of an SRX, while adding the ability to run high-bandwidth virtual machines directly on the router. It looks like it’s supposed to sort of sit next to the SRX1500 or maybe SRX380 in Juniper’s lineup, while being much cheaper than either. The NFX250 lists for $4,700–$8,500 depending on the configuration, vs $11,000 for the SRX1500 or $14,295 for the SRX380. At those prices, none of these would make a very interesting home router, much less a home backup router. What makes the NFX250 interesting to me is that it’s available on eBay, frequently for under $400. That’s not too shabby for a 2x10GbE + 12xGbE router that can supposedly handle around 10 Gbps of traffic.

In fact, the Juniper MX150 is really the same hardware with different software; Juniper claims that it’s good for 20 Gbps.

My NFX250 next to a few Unifi switches.

Under the hood, the NFX250 is mostly just a PC with a Xeon-D 15xx CPU. The NFX250-S1, -S1E, and S2 have a 6-core CPU, while the down-specced NFX250-LS1 only has 4 cores. Other than that, the only difference between the models is the amount of RAM and SSD included. Since the NFX250 uses DDR4 RAM and M.2 2260 SATA SSDs, it’s not particularly difficult or expensive to upgrade any of the models beyond the NFX250-S2 spec of 32GB RAM/400 GB SSD, if you need it.

Note that I said “mostly just a PC” above; the entertaining thing about the NFX250 (and, frankly, the most frustrating part about it) is all of the front panel Ethernet ports (except the management port) are actually connected to a managed Ethernet switch chip, and the switch chip is then wired into the PC inside with 2x10GbE links.

The NFX250 runs Linux, and dumps both internal 10GbE interfaces into a software switch (OVS, specifically). Then the NFX software lets you run VMs and connect them to the software switch in various ways. Managment of the NFX is done via a Junos VM that provides pretty much all of the abilities of a Juniper SRX. By and large, you wouldn’t know that you’re talking to a FreeBSD/Junos VM on top of Linux, right up to the point where you start trying to configure interfaces. Then things get strange.

The front panel interfaces are named ge-0/0/0 through ge-0/0/11 plus xe-0/0/12 and xe-0/0/13, following Juniper’s usual naming conventions. If this was an SRX and you wanted to assign an IP address of 1.1.1.1/24 to ge-0/0/0, then you’d just run something like

> set interfaces ge-0/0/0 unit 0 family inet address 1.1.1.1/24

With the NFX, you rapidly discover that family inet isn’t supported on any of the internal interfaces. Only family ethernet-switching. So how do you use this as a router? Easy… ish.

> set vlans my-vlan vlan-id 100
> set interfaces ge-0/0/0 unit 0 interface-mode access
> set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members my-vlan
> set interfaces sxe-0/0/0 unit 0 family ethernet-switching vlan members my-vlan
> set interfaces ge-1/0/0 vlan-tagging
> set interfaces ge-1/0/0 unit 100 vlan-id 100
> set interfaces ge-1/0/0 unit 100 family inet address 1.1.1.1/24

In short, you need to create a VLAN, map your ge-0/0/x port onto that VLAN, make sure that sxe-0/0/0 includes the VLAN as well, and then add a new unit to ge-1/0/0 to handle that VLAN’s traffic.

Where does ge-1/0/0 come from? It’s a purely virtual beast that hides inside of the NFX250. Entertainingly, even though Junos calls it ge-1/0/0 (which implies that it’s a regular Gigabit Ethernet port), show interface ge-1/0/0 thinks that it’s really a 10GbE link (which Junos usually calls xe-x/x/x, not ge-x/x/x). A half-duplex 10G link, for some reason:

> show interfaces ge-1/0/0
Physical interface: ge-1/0/0, Enabled, Physical link is Up
  Interface index: 170, SNMP ifIndex: 540
  Link-level type: Ethernet, ... Link-mode: Half-duplex, Speed: 10Gbps, ...
  ...

Whatever, it’s virtual.

This all seems less odd if you realize that it’s really just a “Router On A Stick” config internally. If you’d like a more traditional Router On A Stick setup, then Junos will happily let you set up VLAN trunks (... family ethernet-switching interface-mode trunk) and share VLANs across interfaces just like a perfectly normal L2 switch.

In Part 2, I’ll jump into some of the weirdness around NFXs, where they behave differently from SRXes. Then I’ll move on to discuss Virtual Network Functions, which is why Juniper made NFXes weird.

Useful links:

Routers I Have Known

May 28, 2022

Let’s just agree on this up front: my home network is overly complicated. There’s a nice, boring LAN with WiFi APs and TVs and Playstations and so forth. There are family members trying to conduct business and school and social lives, and they mostly depend on things working.

And then… let’s call it The Dark Side of the Network. Where things get complicated. Where servers live and talk OSPF to multiple L3 switches, so everything keeps working even when I reboot a switch or move cables. Where service IP addresses are announced from multiple servers via BGP, and fail over when servers go down. Where there are redundant WAN links and redundant routers, and things get messy. To put a more acceptable spin on it, let’s call it “lab space.”

Reviving

May 28, 2022

Another year, another lack of updates.

I’ve been running this site on a computer in my garage, but keeping it up and running hasn’t been a priority to me, and were too many moving parts to actually keep it stable without work. So I’ve punted and moved the serving part of this to Google Cloud, and updated Hugo to be current along the way.

I’ve had a number of entertaining network adventures over the past couple years, and they’re really too long for Twitter, so I’ll be adding them here as time allows.

Linux on a Beckhoff CX5020

Sep 4, 2019

This is likely of no use to anyone but me, ever, but I’ll post it anyway. I’m trying to install Linux on a 10-year old Beckhoff CX5020 industrial computer, and everything works fine until about somewhere in the middle of the user-mode startup process when the screen goes blank and never comes back. I tried disabling X and that didn’t matter. Even booting in single-user mode didn’t help. It still went blank while booting. I could SSH in over the network to debug, but couldn’t get a working monitor.

The problem seems to be the Kernel’s video driver mis-understanding the number and/or type of video ports that the CX5020 has. The solution is easy: add the video=LVDS-1:d option to the kernel’s command line. After that everything seems to work fine. It boots up and runs X as well as could be expected for a 10 year old compact PC. Which is to say not tremendously quickly.

Measuring Backlash, Part 2

May 21, 2019

(See part 1 for the beginning of the process)

Yesterday, I set out to measure backlash in the Z axis of Beaver HDZero CNC. I concluded that it had about 15 microns (0.015mm, or roughly 0.0005") of backlash in the Z axis.

Today I’m going to measure the X and Y axes. They’re each a bit different:

  • They use larger ballscrews than the Z axis (1610 vs 1605) with twice the pitch. That means that one turn of the stepper motor goes twice as far. A single step should be 12.5 microns, and the stepper’s encoder should be able to measure about 5 microns.
  • The Y axis has a pair of steppers driven in parallel, one on either side of the gantry. In theory, there’s no reason why this should matter, but I could see extra inaccuracy creeping in.
  • I had a difficult time getting the ball nuts assembled correctly when building this; I repacked each of the Y nuts at least once, but the X nut seemed to work okay so I didn’t repack it. It lost at least a few ball bearings, though, so I probably should have repacked it. We’ll see if that matters.

The process here is exactly the same as the Z axis from yesterday. I fed the CNC a program that tells it to move from 0 to 0.200 mm, in 0.003mm steps. It paused after each step so I could take a measurement using a Mitutoyo 543-302b digital dial indicator. The dial indicator is accurate to 3 microns, which is why I’m moving the CNC in 0.003mm steps. The dial indicator is wired into my laptop via Mitutoyo’s USB cable. The cable makes the indicator act like a USB keyboard; every time I press the button in the middle of the USB cable it “types” the current measurement into the computer.

Here’s the Y axis’s motion:

Y Axis: Intended vs Measured Position.

Compared to the Z axis from yesterday, the “stairsteps” on this are very pronounced, as expected. Comparing the delta between the two lines gives this:

Y Axis: Normalized difference between intended and measured location.

This looks like around 15 microns of backlash again, very similar to the Z axis.

Now, on to the X axis. It’s a bit more interesting.

X axis Intended vs Measured Position.

Notice that it moved differently on the first vs second and third iterations?

X Axis: Normalized difference between intended and measured location.

Uhm, yeah. There’s way more backlash, and it seems to have been drifting at least a bit over time. Repacking that ball nut would probably be a good idea.

Here are all three axes on one graph; it makes it clearer that Y and Z are very similar, but X is an outlier.

Intended vs Measured Position.

Measuring Backlash, Part 1

May 20, 2019

I wanted to measure how much backlash my CNC has. For today, I’m measuring the backlash in the Z axis, because it’s the easiest, and also because it came fully assembled and there wasn’t much of an opportunity for me to mis-assemble it and screw things up.