Vacation Update!

I’ve been on vacation up in the great frozen cool north, in Calgary to visit my mom. My brother cashed in some of his frequent flier points to come visit as well. The story so far:

Thursday, 6/26: Arrived in Canada. Finished a novel. Got eaten by mosquitoes.

Friday, 6/27 : Went to the University to get some books for the girls and see my mom’s office. Took the girls to the wading pool. Bought some new hiking/work shoes. Got eaten by mosquitoes.

Saturday, 6/28 : Went hiking in Kananaskis Country, up Ribbon Creek. about 9km round trip, Faith walked the whole way. Went with a group of seniors from my mom’s church. Got left in the dust. These people go hiking nearly every week. Finished another novel. Got eaten by mosquitoes.

Sunday, 6/29 : Went to church with my mom. Met some of the hikers again. Finished another novel. Picked up my brother at the airport. Got eaten by mosquitoes.

Monday, 6/30: Made pilgrimage to IKEA. Lunch with my mom’s friends. Music night at my mom’s house. Finished another novel. Got eaten by mosquitoes.

Tuesday, 7/1 : Canada Day! Went downtown to the festivities on Prince’s Island. Went to see fireworks afterwards. Got eaten by mosquitoes.

Wednesday, 7/2 : Exhausted. Down day. Went to the city library to get some videos for the girls. Got eaten by mosquitoes.

Thursday, 7/3 : Went to the Royal Tyrell Museum in Drumheller. Went out for dinner and drinks with Andrea and my brother. Got eaten by mosquitoes.

Friday, 7/4 : Rested at home. Took the girls to the wading pool. Got eaten by mosquitoes. Introduced Andrea to Canadian Tire.

Saturday, 7/5: Spent the day at the Calgary Zoo Introduced Andrea to poutine. Caught evening service at Centre Street Church. Introduced girls to Tim Horton’s. Got eaten by mosquitoes.

Sunday, 7/6 plans: Stampede Breakfast at my mom’s church. Maybe get together with my godmother, as I haven’t seen her in years. Probably will get eaten by mosquitoes.

Monday, 7/7 plans: Meeting with Gregg, the IT Director at CSC to extend the hospitable hand of CITRT. Going to stampede events. Probably will get eaten by mosquitoes.

Tuesday, 7/8: Fly home. No mosquitoes, I hope.

Wednesday, 7/9: Back to the daily grind to try and fix everything that broke the second I stepped on the plane to Calgary. Rejoice in the lack of mosquitoes.

Canadian things to do:

  • Tim Horton’s
  • Poutine
  • Canadian Tire
  • Canada Day
  • IKEA (OK, not really Canadian, but we don’t have IKEA in KC)
  • Hockey (wrong season 🙁 )
  • Mountains
  • Curling (wrong season 🙁 )

Day at the Farm!

Shatto Milk was having an open house today at their farm up in Osborn, MO, and we thought it would be a fun thing to do with the girls. While Andrea was up at Troost Elementary for COR’s Bless The School missions project, I took the kids up to the farm.

They had a bounce house, hot dogs, beans, samples of their milk (and requisite cookies to go with, provided by Midwest Airlines). The girls got to see the bottling plant, the bottle washer, and the milking parlor, and got to pet some cows and some of the new calves (a few weeks old!). A good time was had by all and the weather was absolutely perfect: low 80s, sunny, with a breeze.

It was nice enough to go up in my car instead of the minivan (My car has been referred to by some as my “Dave Ramsey Car”). My car’s air conditioner is currently out of commission, so summer trips in it with the kids are rare. Luckily, it’s highway most of the way, so it’s easy to get a good breeze going in the car even with the windows shut. I filled up before I left and when I got back. Total round trip: 158.7 miles. Total fuel consumed: 3.8 gallons. Yes, you saw that right. My $1700, 16-year-old Corolla got 41.7 mpg on that trip. Take that, Prius!

I’ll edit this post later to add linky goodness and pictures!

An interesting look at the Canadian economy

Take a look at this data coverage map from Sprint. Orange is EV-DO, Yellow is 1xRTT:

Wireless Data Coverage to the north

That spot with lots and lots of EV-DO, disproportionate to the rest of the country? That’s Alberta. The entire province is a mere 3.5 million people.

The isolated (but large) blob in the northeastern part of the province is centered on Fort McMurray, population 47,000. The EVDO coverage area is probably several thousand square miles.

Three guesses where all the oil is 🙂

Hardware redundancy? Hah!

I thought it rather ironic that as I was installing my HA firewall cluster that I hadn’t planned the whole hardware redundancy thing all the way through.

In order to install the new machine and the NICs, I had brought a screwdriver to mount rack rails and such. This particular screwdriver was one of the ratcheting kind, and it’s been a poorly functioning department fixture since before I arrived. Today, it decided to completely and catastrophically fail. One moment, I’m turning a screw, the next finds my hand holding about half a dozen pieces of the ratcheting mechanism, and the screwdriver shaft spinning freely and uselessly.

… and in my planning to build the HA cluster for the firewall, I’d neglected to brnig a spare screwdriver in case that hardware failed (which we’d expected it to do long ago). Luckily, one of our “neighbours” happened to have one with him and let me borrow it.

Moral of the story, make sure you have full hardware redundancy, including your screwdrivers.

High-Availability firewall on the cheap!

I now have a profound appreciation for BSD.

Yesterday, our pfSense firewall at 1102 Grand suddenly went silent and the panel on WhatsUp for that site went all red. Not good. I went down to the cage after small group last night and found the machine to have just simply locked up cold. I suspect hardware, since pfSense/BSD didn’t log a thing about it going dark.

It became quite clear that this setup was… suboptimal. Clif‘s shooting for 99.99% on this new setup. I can’t be racing off to the datacenter every time the firewall machine decides to take a holiday from reality. Brian and I quickly determined that we needed not only a remote power control unit, but some sort of high-availability solution that wasn’t going to empty our wallet like a pair of NSA 4500s would. (sorry Mark, we simply don’t have that kind of money) We already had a spare, identical machine at the datacenter doing duty as a hardware spare and development server, and another one just like it in inventory at the Central Campus. I grabbed the extra machine and went back 1102 Grand for the second time in 12 hours, with a quick stop at Micro Center for some cheap NICs and a red crossover cable.

Fortunately, pfSense has high availability capability built in, thanks to BSD’s CARP and pfSync. CARP allows me to set up virtual IPs on both firewalls and synchronize between them with pfSync. The extra NICs were for a dedicated sync/heartbeat link between the two boxes. I’m still a little fuzzy on the technical details of how this works, but it works… Convergence/Failover time is 3 seconds or less, and everything is synchronized between the two machines, including state and session information. I had it all set up, and hit the reset button on the primary firewall… and nary a ping was dropped.
The setup involves giving each machine its own LAN and WAN addresses (and a unique address on the other zones as well) and then creating a CARP virtual IP on that interface. The virtual IP is the one used as the gateway and as the NAT address. All rules and configs (including IPSEC Tunnels!) is mirrored on the second box.

Reference material:

Excellent documentation on the process from the folks at Countersiege.

Some background from the OpenBSD folks.

A few good tips here. These proved to be crucial.

Happy Birthday to Me!

Had a wonderful 35th birthday today, thanks to my family and a bunch of friends, mostly from the department, with a few others thrown in for good measure.

We moved our usual end-of-the-month game night up a week to have a birthday party at the same time, which was the third event of the week at our house, and the fourth gathering I’d been to this week…

Monday: Small Group at our house.
Tuesday: Arena Survival Party at Brian’s place. Perfect day for it. Got some great pics of the girls.
Wednesday: night off.
Thursday: Après-Party for Jeremy’s wife following dinner at Llywelyn’s (a nice celtic pub within walking distance of our place, with good food, good beer, and good music.)
Tonight: Game Night/Birthday.

Janelle and Brandon (who looks like he could be Terry Storch‘s little brother) joined us. Janelle is our staff Adminstrel/mercenary and goes around filling in for folks on vacation. This week, she’s in HR. Brandon is a web designer for a local advertising agency. Janelle said she loved to play Settlers of Catan, so I invited her and her hubby along (since he’s a geek too). There were enough of us (11 plus the kids) to play two boards of Settlers, which amazingly finished up within minutes of each other. Brian and my wife were the victors.

Before gaming, there was munching. A ridiculous amount of dessert and munchies materialized, in addition to the “official” birthday cake from 3 women and an oven (which is dangerously within walking distance of our house):

(in the background is one of my rapidly-becoming-legendary margaritas, and a bit of Janelle in her KU swag)

Matt was duly impressed by the candle arrangement. This was after all a party attended by geeks. The carrot cake was heavenly. It was the first time we’d outsourced birthday cake production due to Andrea starting full-time work at the church this week and our busy social calendar.

Wait, what? I have a social life? How did that happen? OK, so it’s mostly with other geeks. Demented and sad, but social.

The night was capped off by Brandon, Matt, and Philip playing on Matt’s Wii. Thanks to all my peeps and homies for making it such a fun birthday.

It’s all about the pixels!

Yesterday I went down and picked up some some servers and workstations that were donated to us. This company runs a trading floor, and I started salivating when I walked in to the place. This is what a typical desk looks like:

Yep, that’s *sixteen* monitors per machine. A typical desk has just shy of 21 million pixels covering nearly 20 square feet to play in. There are no cube walls, because they’re all made of monitors.

The upside to this was that the workstations we were donated ran these pixel rigs in their former lives. And they all were stuffed full of PCI video cards (GeForce FX5500 for the curious). Naturally, I snagged a couple out of one of the stations to put in my WhatsUp monitoring station. I was disappointed to find that it only has 2 PCI slots.

I expanded on my own pixel rig, which I had recently reconfigured by mounting the 24″ screens to the top of my cube walls in order to reclaim some much-needed desk space. Here is the result:

Left to right, top to bottom:

  • Acer X241W, 24″ 1920×1200, running WhatsUp Gold and keeping an eye on the facility map and my rack
  • Acer X241W, 24″ 1920×1200, running WhatsUp Gold with 2 IE panes. The one on the left is monitoring network traffic and response, the one on the right is monitoring disk usage,.
  • Dell E153FP, 15″ 1024×768, running Google Earth with Area 51 radar overlays from WeatherTap (probably gonna replace this with a 17″)
  • Dell E177FPb, 17″ 1280×1024, running VirtualCenter console
  • Dell E207WFPc, 20″ 1680×1050, running WhatsUp Gold, with the wireless bridge workspace. This is a dual-input monitor that is also hooked to my laptop
  • Dell D820, 15.4″, 1920×1200, main workstation laptop

The brains of the monitoring station run on a Dell Optiplex GX620 with 3.2GHz P4, 3GB RAM, Radeon X600 primary (AGP) and two GeForce FX5500 secondaries (PCI). The WhatsUp backend runs on a Dell PowerEdge 2650 with 4GB RAM and dual 2GHz Xeon processors.

Total pixels: 10,773,152

I still have one additional monitor output I can use. If I put a 17″ on that one and replace the 15″, I’ll be cruising at 13,918,880 pixels.

I love my job 🙂

Starbucks goes retro (and a little naughty) !

Over here, on $* changing the logo on their cup to the pre-1987 logo as part of a promotion of their new house brew:

Here’s my question, Starbucks fans: Does the logo, be it be green, white and conservative, or brown, white and slightly pornographic, have anything to do with why you line up around the block for the Seattle company’s coffee — new blend or old?

Reminds me of the joke about the origins of the Canadian flag… They wanted to create a flag that represented both male and female Canadians… But that would be pornographic, so they covered it up with a maple leaf 🙂