Enhancing the public Wi-Fi experience

March 30, 2017 |   |  11 Comments  |  Church IT, Hardware, IT, networking, Wireless

Recently, there was an excellent blog post from WLAN Pros about “Rules for successful hotel wi-fi“. While it is aimed primarily at Wi-Fi in the hotel business (where there is an overabundance of Bad-Fi), many of the tips presented also apply to a wide variety of large-scale public venue wifi installations. Lots of great information in the post, and well worth a read.

At the 2016 WLPC there was an interesting TENTalk from Mike Liebovitz at Extreme Networks about the pop-up wifi at Super Bowl City in San Francisco, where analytics pointed to a significant portion of the traffic being headed to Apple.

Meanwhile, a few months later at the 2016 National Church IT Network conference, I heard a TENTalk about Apple’s MacOS Server, where I first heard about this incredibly useful feature (sadly, it wasn’t recorded, that I know of, so I can’t give credit…)

With most of the LPV installations I’ve worked on, I’ve found the typical client mix includes about 60% Apple devices (mostly iOS). For example, this is at a large church whose wireless network I installed. (Note that Windows machines make up less than 10% of the client mix on wifi!)

Client mix from Ruckus ZoneDirector

OK, So what?

This provides an opportunity to make the wifi experience even better for your (Apple-toting) guests. Whenever possible, as part of the “WiFi System” I will install an Apple Mac Mini loaded with MacOS Server. This allows me to turn on caching. This is not just plain old web caching like you would get with a proxy server such as Squid, but rather a cache for all things Apple. What does this do for your fruited guests? It speeds up the download of software distributed by Apple through the Internet. It caches all software and app updates, App Store purchases, iBook downloads, iTunes U downloads (apps and books purchases only), and Internet Recovery software that local Mac and iOS devices download.

Why is this of interest and importance? Let me give you an example: A few years ago, we were hosting a national Church IT Round Table conference at Resurrection on a day when Apple released major updates to MacOS, iOS, and their iWork suite. In addition to the 50 or so staff Mac machines on the network, there were another hundred or two Mac laptops and iThings among the conference attendees. The 200MB internet pipe melted almost instantly under the load of 250 devices each requesting 3-5GB of updates. That would have melted even a gigabit pipe, and probably given a 10Gbps pipe a solid run for its money (not to mention bogging down some of the uplinks on the internal network!. Having a caching server would have mitigated this. It didn’t do great things to the access points in the conference venue either, all of which were not only struggling for airtime, but also for backhaul.

Just by way of an example, Facebook updates their app every two weeks and its current incarnation (86.0, March 30, 2017) weighs in at 320MB (the previous one was about half that!), and its close pal Messenger clocked in at 261MB. Almost everyone has those to apps, so they’re going to find itself in your cache almost instantly, along with numerous other popular apps. Apple’s iWork suite apps and Microsoft Office apps all weigh in around 300-500MB apiece as well. This has potential to murder your network when you least expect it. (A few years back, the church where I was working hosted the national Church IT conference that happened to coincide with Apple’s release of OSX Mavericks, and a major iWork update for both iOS and MacOS. The conference Wi-Fi and the church’s 200Mbps WAN pipe melted under the onslaught of a couple hundred Apple devices belonging to the guest nerds and media staff dutifully downloading the updates.)

In any case, check out the network usage analytics from either your wireless controller or your firewall. If Apple.com is anywhere near the top of the list (or on it at all), you owe it to yourself and your guests to implement this type of solution.Network Statistics from Ubiquiti UniFi

The Technical Mumbo-Jumbo

Hardware

As mentioned previously, a Mac Mini will do the job nicely. If you’re looking to do this on the cheap, it will happily run on a 2011-vintage Mini (you can find used Mac Minis on Craigslist or eBay all day long for cheap), just make sure you add some extra RAM and a storage drive that doesn’t suck (the stock 5400rpm spinning disks on the pre-2012 era Mac Mini and iMacs were terrible.) Fortunately, 2.5″ SSDs are pretty cheap these days. Newer Minis will have SSD baked in already.

If you’re wanting to put the Mac Mini in the datacenter, you might want to consider using a Sonnet RackMac Mini (which is available on Amazon for about $139) and can hold one or two machines.

Sonnet RackMac Mini

You can also happily run this off of one of the 2008-era “cheese grater” Mac Pros that has beefier processing and storage (and also fits in a rack, albeit not in the svelte 1U space the Sonnet box uses). If you have money to burn, then by all means use the “trash can” Mac Pro (Sonnet also makes a rack chassis for that model!).

This is a great opportunity to re-purpose some of those Macs sitting on the shelf after your users have upgraded to something faster and shinier.

Naturally, if you’re running a REALLY big guest network, you’ll want to look at something beefy, or a small farm of them Minis with SSD storage (the MacOS Server caching system makes it quite easy to deploy multiple machines to support the caching.)

The Software

MacOS Server (Mac App Store, $19.99)

Since most of your iOS guests will have updates turned on, one of the first things an iOS device does when it sees a big fat internet pipe that isn’t from a cell tower is check for app updates. If you have lots of guests, you will need to fortify your network against the onslaught of app update requests that will inevitably hit whenever you have lots of guests in the building.

The way it works is this: When an Apple device makes a request to the CDN, Apple looks at the IP you’re coming from and says, “You have a local server on your LAN, get your content from there, here’s its IP.” The result being that your Apple users will get their updates and whatnot at LAN speeds without thrashing your WAN pipe every time anyone pushes out a fat update to an app or the OS, which is then consumed by several hundred people using your guest wifi over the course of a week. You’ve effectively just added an edge node to Apple’s CDN within your network.

Content will get cached the first time a client requests it, and it does not need to completely download to the cache before starting to send it to the client. For that first request, it will perform just as if they were downloading it directly from Apple’s servers. If your server starts running low on disk space, the cache server will purge older content that hasn’t been used recently in order to maintain at least 25GB of free disk space.

MacOS Caching Server Configuration

The configuration

If you have multiple subnets and multiple external IPs that you want to do this for, you can either do multiple caching servers (they can share cache between them), or you can configure the Mini to listen on multiple VLANs:

Mac OS network preferences panel

Once you have the machine listening on multiple VLANs, you can tell the caching server which ones to pay attention to, and which public IPs. The Mac itself only needs Internet access from one of those subnets.

MacOS Server Caching Preferences

The first dropdown will give you the option of “All Networks”, “Only Local Subnets”, and “Only Some Networks”. Choosing the last one opens an additional properties box that allows you to define those networks:

Mac OS Server Cache Network Settings

The second one gives you the options of “Matching this server’s network” or “On other networks”. As with the first options, an additional properties box is displayed.

In both cases, hit the plus sign to create a network object:

Mac OS Server Create a New Network

It should be noted here that this only tells the server about existing networks, but it won’t actually create them on the network interface. You’ll still need to do that through the system network preferences mentioned previously. If you don’t want to have the server listen on multiple VLANs, you can just make sure its address is routable from the subnets you wish to have the cache server available, define the external and internal networks it provides service to, and you should be off to the races. This will provide caching for subnet A that NATs to the internet via public IP A, and B to B, and so on. Defining a range of external IPs also has you covered if you use NAT pooling.

There’s also some DNS SRV trickery that may need to happen depending on your environment. There are some additional caveats if your DNS servers are Active Directory read-only domain controllers. This post elaborates on it.

 

Is it working?

Click the stats link near the top left of the server management window. At the bottom is a dropdown where you can see your cache stats. The red bar shows bytes served from the origin, and green shows from the cache. If you only have one server doing this, you won’t see any blue bars, which are for cache from peer servers. Downside is that you can only go back 7 days.

On this graph, 3/28 was when there were both a major MacOS and iOS update released, hence the huge spike from the origin servers on Apple’s CDN. Nobody has updated from the network yet… But guest traffic at this site is pretty light during the week. I’ll update the image early next week.

MacOS Server Cache Stats

Other useful features

A side benefit of this is that you can also use this to provide a network recovery boot image on the network, in case someone’s OS install ate itself – on the newer Macs with no optical drive, this boots a recovery image from the internet by default. This requires some additional configuration, and the instructions to set up NetInstall are readily available with a quick Google search.

If you want, you can also make this machine the DHCP and local DNS server for your guest network. With some third-party applications, you can also serve up AirPrint to your wireless guests if they need it.

Conclusion

From a guest experience perspective, your guests see their updates downloading really fast and think your WiFi is awesome, and it’s shockingly easy to set up (the longest and most difficult part is probably the actual acquisition of the Mac Mini) It will even cache iCloud data (and encrypts it in the cache storage so nobody’s data is exposed). Even if you have a fat internet pipe, you should really consider doing this, as the transfers at LAN speed will reduce the amount of airtime consumed on the wireless and the overall load on your wireless network. (Side note, if you’re a Wireless ISP, this sort of setup is just the sort of thing you ought to put between your customer edge network and your IP transit)

Of course, you could also firewall off Apple iCloud and Updates instead, but why would you do that to your guests? Are you punishing them for something?

Android/Windows users: So sad, Google and Microsoft don’t give you this option (Although Microsoft sort of does in a corporate environment with WSUS, but it’s not nearly as easy to pull off, nor is it set up for casual and transient users). I would love it if Google would set up something like this for play store, Chromebook, etc, as about half of the client mix that isn’t from Apple is running on Android. You can sort of do it by installing a transparent proxy like squid.

Now, if only we could do the same for Netflix’s CDN. The bandwidth savings would be immense.

Update

(Added November 16, 2017)

As of the release of MacOS High Sierra and MacOS Server 5.4 (release notes), the caching service is now integrated into the core of MacOS, so any Mac on the network can do it, without even needing to install Server. The new settings are under System Preferences > Sharing:

 

 

Controlling Audio With ProPresenter

November 20, 2016 |   |  2 Comments  |  Church IT, Hardware, Misc

Our church is a small one. So its not always especially easy to fully staff our tech booth, and sometimes, one must fly solo, which adds to the workload, and sometimes stuff gets forgotten, like unmuting microphones for the choir or the person reading the scripture.

Fortunately, there is some tech than can help us in this regard. We use ProPresenter for our graphics presentation, and an Allen & Heath QU-24 console for our audio. The Qu-24 is connected to the Mac that runs ProPresenter via a USB cable, which shows up in the Mac as a 32 in/32 out audio device, as well as a MIDI device. This is primarily to be able to use the console as a multitrack and DAW interface, but it also lets us play back audio from ProPresenter media cues without ever leaving the digital domain, and saving us a couple of inputs on the board (although there’s no shortage of those). But because it’s also a MIDI device, this gives us some options with ProPresenter’s $99 MIDI module add-on. The Qu series boards can also do MIDI over IP (in fact, the Qu-Pad remote control app for iPad uses MIDI over IP to work its magic). If you’re using MIDI over IP with a Mac, you’ll need a special driver for the Mac. No driver is needed for USB.

First, a few resources we’ll need:

In the Qu Series, mutes and mute groups are controlled by a sequence of a Note On/Off message. The specific note determines the channel or mute group being controlled, and a the velocity value determines if it’s being turned on (Muted) or off (Unmuted). Velocity values below 64 turn the mute off, and above turn it on.

Meanwhile, over in ProPresenter, since Version 6, we have the ability to add MIDI Note On/Off cues to a slide. See where this is going? Unfortunately, ProPresenter doesn’t have the ability to do anything other than MIDI notes in a slide at the moment, so we can’t get really crazy with starting recordings or anything else requiring non-note MIDI messages.

So how do we know what notes emulate button presses? The documentation provides this handy method:

OK, this requires thinking and math. Not so helpful. This is where the MIDI monitor comes in. Download it and run it, and it shows everything coming across the MIDI interface. Push the button you’re interested in, and lo, MIDI Monitor helpfully shows you what note you’re interested in:

In this case, G#4 is the mute group for our choir. A4 is the mute group for the speaking mics on the chancel. A1 is the lectern mic.

Screenshot 2016-11-20 13.51.30So now, to be able to add a cue at the beginning of a song the choir is singing, I simply have to add two cues to the first slide to turn on the choir microphones:

  • NOTE ON, G#4(80), 63
  • NOTE OFF, G#4(80)

Then I can add a slide at the end of the playlist entry that then turns them back off, or add these to the beginning of the next playlist entry:

  • NOTE ON, G#4(80), 127
  • NOTE OFF, G#4(80)

Likewise, when someone is at the lectern reading scripture, I can unmute that channel automatically using the corresponding note number, and mute them again when they’re done.

On the flip side, you can also use note on/off commands to control ProPresenter. So you *could* also use the Mute, SEL, and PAFL buttons on unused channels to trigger things in ProPresenter (you also want to make sure that you don’t overlap these with the mutes and mute groups that you are actively using so as not to inadvertently advance a slide when hurriedly muting a channel). ProPresenter also conveniently tells you what the last note sent was, so you can actively push the button you want to use, make a note of its number, and put it in the action you wish.

 

Another approach you can take is to create a presentation in ProPresenter containing blank slides with the various functions you wish to use. Then you can copy these slides into presentations and add a Go To Next timer to them to automatically advance to the next slide. I would also recommend using slide labels and colors to clearly identify what each slide is doing:

Screenshot 2016-11-20 13.47.55

 

If you have controllable lighting and your lighting console also has MIDI capability, This comes in handy as well. And if you’re really a one-man band, and like to do things like pads underneath certain worship elements, you can use this to trigger those as well. But if you get to that point, you may want to look into QLab to control all of them at the same time.

So there you have it: a quick and easy way to automate some of your workload with the Qu series boards. If you’ve got another board that you use, let me know in the comments if you do (or would like to do) something like this. Would also love to hear if anyone is using hardware MIDI controllers like the Novation LaunchPad and how you have it set up.

Additional Info:

Summary of MIDI Messages (midi.org)

In the wild: EGO cordless electric mower

July 9, 2014 |   |  5 Comments  |  Hardware, Misc

It’s a clean, green, mowing machine!

mower_largeI’m going to veer off my usual topics here to give you my thoughts on a recently acquired tech toy of a different flavor: My lawnmower. This is NOT a sponsored post.

When we moved into our previous house almost 11 years ago, we went on a coupon-fueled shopping spree at Home Depot, and picked up your standard 4-stroke gas-powered mower. It worked well, but after a while, the tedium of dealing with oil and gas and all those moving parts makes you think “there’s got to be a better way”. We’d been eyeing electric mowers for a while, but either they were corded (obnoxious – chance of mowing the cord is high!), or had enough battery life to make you need a wagon full of spare batteries, and in neither case did they have enough oomph to cut grass that had gone more than about 12 hours since the last mowing.

When the Toro died this spring (I think it busted a rod, or something else pretty major in the engine), I started looking at battery-powered options again. At our new place, the lot is nearly half an acre. Anyone in their right mind would have bought a small riding mower. Consumer Reports narrowed me down to two options: The Black & Decker CM1936, with a 19″ deck and self-propelled version for $439 at Amazon (now $379), or the EGO LM2001, with a 20″ deck for $499 at Home Depot. I was initially leaning toward the B&D’s lower price, but was eventually won over by the EGO’s slightly wider cutting deck, its 5-year warranty, and the 30-minute charger (which would have been another $130 for the B&D). That the EGO is 50 pounds lighter was a big plus as well.

Since Home Depot had it in stock, I headed over there and picked it up. It comes in a large cardboard box that easily fit in the back seat of my full-size Toyota. Because the mower handle folds down and collapses, there was no assembly to speak of. It’s pretty much a matter of taking it out of the box, removing the requisite bits of tape and protective film, and a few plastic bags, and putting the box on the curb for the recyclers to pick up. Virtually all of the packaging is recyclable, which is a plus.

EGO Battery Pack

This is alien technology. Or at least from the future.

The battery pack and the charger look like something right out of a sci-fi movie. The battery is the heart of the system, and battery technology has made significant progress in the last few years. Power tool manufacturers love to tout the voltage of their battery system, under the idea that “More Volts = Better”. Being a geek with a background in electronics, I know that this is mostly bunk, but when it comes to battery systems, there’s some validity to it, because virtually all power tool batteries consist of a serial/parallel array of 1.2V rechargeable cells. More cells = more oomph. EGO says the motor in this mower is 600 watts, which works out to a little over 10 amps. The battery pack is 4Ah, so at full load, one should expect about 25 minutes of use. Since in the real world, the motor isn’t under full load the whole time, it gets pretty close to EGO’s claimed run time of about 45 minutes. I’ve found that this isn’t enough to do my whole yard, but after the battery runs out, I’m usually ready to go sit inside for half an hour, and cool off with a cold beverage and some air conditioning, while the battery charges back up (and yes, it DOES only take half an hour!).

Performance-wise, it does OK with normal mowing, but heavy grass is something best approached in phases at different cutting heights (which are adjustable to 5 levels with a single lever somewhat reminiscent of the shifter in my minivan). On my gas mower, I’d usually deal with heavy grass by putting the discharge chute on and letting it eject the cuttings rather than mulch them. On the EGO mower, this plan is no good. While it comes with a discharge chute that attaches in the same place as the bag, it clogs easily, and is generally useless. Bagging works pretty well, though.

So easy a kid can mow!

Where this mower really shines is how easy it is to use. Much of the body is molded polypropylene, so at 40-odd pounds, it weighs about half what my old gas mower did (the B&D unit is actually 15 pounds HEAVIER than my Toro), and is so quiet that it won’t bother the neighbors. When sitting inside while someone is mowing, it sounds like a gas mower several blocks away. A typical gas mower is usually over 90dB, where permissible exposure levels are not much more than an hour. I don’t know offhand what this unit is, but it’s a LOT quieter. There’s no fuel to mess with, or oil changes, or any of that. When you’re done, it folds up neatly and can be stood on end, taking up no more than 2 square feet of your garage. Between the light weight, the quiet, and the lack of fumes, my 10-year-old daughter is actually willing to mow the lawn. (Another major motivating factor is that mowing the grass pays for her cell phone service).

Other than keeping it clean, there’s really no long-term maintenance to worry about. It comes with a 5-year warranty, although the blade isn’t covered, as it’s considered a wear part. There are no moving parts other than the motor itself (compare to a 4-stroke gasoline engine, which even with a single cylinder is a very complex piece of machinery).

As for energy consumption, a full charge is 224 watts of electricity. A full charge will run you somewhere between 2 and 4 cents worth of electricity, depending on where you live. The EPA defines a gallon of gasoline as equivalent to about 33.4kW of electrical energy. My old Toro would go through about a quart of gas to mow the yard. At nearly 4 bucks a gallon for the non-ethanol stuff that won’t wreck the engine, that’s a buck a mow. With 2 charges, that’s also about 20x the amount of energy that the EGO uses to achieve the same job. Oddly enough, the “fuel” cost of the electric mower is also about 1/20 that of the gasoline.

EGO in storage mode

If carbon footprint is something you care about, 1 full mow is about half a pound of CO2 if your electricity is from natural gas, and about a full pound if it’s from coal. Zero if it’s from nuclear, solar, or wind. The gasoline mower belches out about 4.5 pounds per mow, along with a whole bunch of other nasty stuff that your car has the decency to clean up first.

As for quirks, there are a couple. The handle has a couple of different safety interlocks, and must be fully extended and locked for the mower to operate. There’s a dead-man switch on the push bar, much like virtually any other mower. There’s also a removable safety key that must be pushed in to start the mower. I’ve found that when mowing close to bushes that the slide lock tends to come undone, allowing the handle to slide in just enough to cut the mower out. The first few times, you find yourself standing there wondering why the mower won’t work, until you notice the bright green latch on the handle hanging open.

When going through heavy grass, if the current draw on the motor becomes too much, the green power light will start flashing yellow. If you don’t ease up on it, it will stop the mower. Once the battery gets low, the indicator light will turn red, at which point you’ve got about 3-5 minutes until break time. The battery is really good at delivering a fairly flat and constant amount of power, so there’s not really much decline in power until the battery decides to go completely dead.

EGO also has a blower, edger, and hedge trimmer that work with the same battery (they also make a smaller and lighter 2Ah battery for those devices, but all batteries in the EGO tool family are interchangeable, so if you find yourself about 10 minutes short on the mower, get the trimmer or blower with its battery, run the mower on the small battery for 20 minutes, and then switch to the big battery to finish the lawn while you recharge the small battery, which will be ready for trimming or blowing by the time you get done mowing. If you already have batteries and chargers, you can order just the tools without batteries directly from EGO for less money. Similarly, you can order extra batteries from them (the big one is $199, the little one is $129).

Bottom line, It may be a spendy piece of equipment, but not having to deal with gasoline, fumes, noise, maintenance, and being able to send the kid out to mow instead of doing it myself is well worth the price of admission. Oh, and it also has an LED headlight, for those midnight mowing escapades.

My God, It’s Full Of Pixels!

June 19, 2014 |   |  Comment  |  Hardware

One of my desktop monitors gave it up last week, the backlight started flickering and randomly disconnecting from my Mac (which causes all attached displays to go dark for a second while it recomputes your workspace). Needless to say, it was sapping productivity even worse than Facebook. I’ve been eyeing 4K monitors for a while now, and both Dell and Lenovo have some decent low-cost options in 28″ sizes ($700ish), but that was still more than I was willing to cough up for a new monitor right now, even with the Lenovo dealer demo discount.

Then came last weekend, where TigerDirect ran a $70 rebate on a Seiki SE39UY04 39″ (yes, you read that right, thirty-nine inches) 4K TV with a whole mess of inputs (3 HDMI, 1 VGA, 1 Component, as well as a tuner). Base price before rebate: $400. Four hundred bucks. BEFORE the rebate. For 4K. That brings it down into the realm of even inexpensive 27″ monitors. Some googling found that the tech press actually has good things to say about this “off-brand” display (with the caveat that there are very few graphics chips currently out there that can drive this resolution, and that HDMI 1.4 is limited to 30Hz refresh at 4K). Since the rebate was a very limited-time offer, I jumped on it, figuring I was gonna have spend that much on a monitor anyway. For those who missed last weekend’s rebate from TD, It’s currently available on Amazon (with Prime!) for $299 (plus larger sizes too!).

Three days later, UPS shows up bearing goodies, and I hooked it up this morning. The unit is generally well built, comes with a solid but unobtrusive pedestal, and the bezel is not huge. It even comes with a decent HDMI cable. There’s been a lot of discussion online as to whether the first-generation Retina MacBook Pro can drive this monster at native resolutions at all. Let me put those to rest: it can, with Mavericks 10.9.3, at 30Hz. Because of the refresh rate, it’s not a great rig for gaming (but it’s still beautiful with X-Plane!!!), but it’s great for sheer pixel space to put my calendar, e-mail, a couple of network monitoring screens, multiple RDP sessions, and lots of other things that don’t require high refresh rates. Colors are quite good, as is brightness, and the built-in speakers are surprisingly loud (almost too loud to use with my computer even at the lowest volume setting) If you press “Menu” on the remote, followed by “0 0 0 0”, you can get into the factory menu which allows you to tweak the color temperature, and the “Warm” setting is shockingly close to my MacBook. The factory menu also lets you dial down the backlight (which I did – even so, this TV is already near the bottom of its category when it comes to energy consumption). I also dialed the default sharpness setting down to 0, as, like most TVs, the edge enhancement algorithms designed for making TV pictures look better really butcher computer signals.

Tweetdeck is 1440x900, RDP window is 1920x1080. No scaling. Just pixels.

Tweetdeck is 1440×900, RDP window is 1920×1080. No scaling. Just pixels.

Did I mention that 4K is an awful lot of pixels? It is a LOT of pixels. 8,294,400 of them . Holy cow. I’m a lifelong pixel junkie, and I’m loving this. The 39″ display is big enough to use on your desktop at native resolution at a comfortable distance of about 4′ (rather than driving it at a pixel-doubled 1920×1080 workspace). Drawing network maps in Visio and Ekahau at 4K resolution is something out of a dream.

Visio at 3840x2160.

Visio at 3840×2160.

Ekahau Site Survey

Ekahau Site Survey

My biggest problem? Losing my mouse cursor. Gotta use a solid color background. That’s OK, my GPU is probably just as glad it doesn’t have to deal with an 8MP image.

X-Plane in its full 39" 4K Glory

X-Plane in its full 39″ 4K Glory

Bottom line, for under $400, this is a surprisingly good piece of hardware. Seiki may be considered an “off-brand” label, but don’t forget that Vizio was in the same position when they started selling good HD televisions for dirt cheap. The only real downside I see right now is using a VGA port that is limited to 1920×1080 instead of putting a DisplayPort interface there instead. OK, Roku, When can I expect a 4K version of your box so I can watch Breaking Bad and House of Cards in their full 4K glory? Et tu, Chromecast?

This video looks utterly spectacular on this screen (Downloaded with YTD and played with QuickTime)

Mobile Internet in Haiti

November 4, 2013 |   |  1 Comment  |  Haiti, Hardware, IT, networking, Wireless

Note: Be sure to read my March 2015 update about this…

I’m back down in Haiti, as some of you already know, working on some of the wireless networks linking the different sites of the Église Méthodiste d’Haïti (EMH), which is the Haitian Methodist Church. Knowing that I was coming into an environment where the internet connection was not functioning properly, and that I was likely going to need internet access for troubleshooting, I armed myself with a 3G GSM hotspot that I picked up on eBay.

After parting with about 50 bucks (plus another 15 for a charger and 2 spare batteries), the Huawei E583C unit showed up via USPS on my doorstep 4 days later bearing a postmark from Hong Kong (color me impressed, I can’t even get postcards from Toronto that quickly!)

20131125_150332I opened it up and inside was a “T-Mobile Wireless Pointer” from the UK division of T-Mobile. I popped on down to the local T-Mobile store and get a SIM for testing, and fired it up. After much futzing around trying to get it to speak 3G to the network without any success, I go back to T-Mobile and pick a tech’s brains. Turns out this one operates on the 800/1800/1900 band, which T-Mobile has phased out 3G on to make room for more LTE. Meanwhile, Jay was in Haiti, so I asked him to pick up a NatCom SIM and bring it home with him.

I’ll pause briefly here to talk a bit about mobile in Haiti. There are two major players, Digicel (which has a thing for island nations all over the world) and NatCom, which is formed out of what was left of the national telephone company (Teleco) and the Vietnamese national telecom (VietTel) that bought up a 70% interest in Teleco not long after the earthquake. What little copper telecom infrastructure existed in the country has long since been destroyed by a number of different Screen Shot 2013-11-25 at 3.20.19 PMmeans, both natural and human. Since the earthquake, NatCom has been building out a LOT of fiber. Digicel operates the only direct fiber link out of the country to Columbus Networks‘ Fibralink fiber network that links the Caribbean up to the rest of the world. The other way out of Haiti to the internet is via microwave backhaul to the Dominican Republic which has 2 landings of the ARCOS fiber ring.

In the nearly 4 years since the quake, mobile internet in Haiti has gone nuts. It’s now quite reliable, and surprisingly cheap if you know how to do it. Monthly postpaid plans for data cost about a quarter what they do in the US – a 10GB plan on digicel will set you back 1000 HTG (about 25 bucks). The same plan on Verizon in the US by comparison is about $100! Digicel offers current-generation Android phones like the S4 (but be prepared to part with full unsubsidized price for it), and Apple recently started making unlocked SIM-less iPhones available on its own store. The smartphone revolution is coming to Haiti, and it’s going to be interesting to watch. There was someone at church on sunday using an iPad, and it wasn’t someone from our team.

When I got down to Haiti and put the SIM Jay obtained for me into the hotspot (erm, “Pointer”… can any Brits enlighten me as to the origin of that term?), and getting no joy. Realizing that the zillion config changes I’d made to try and get it to work on T-Mobile’s network were probably interfering, I hit the factory reset button, and as soon as it rebooted, it was speaking 3G on Natcom’s network. It was that easy.

Next step was to load up some funds on the card, since it was a basic card that came empty of funds. Normally you can do this from the phone, but since this was a hotspot, I didn’t have the ability to dial numbers (although the Huawei firmware does allow you to SMS, which turned out to be a critical component). Natcom partners with a third party called EzeTop which allows you to reload phone cards online (yours or anyone else’s). So I dropped 10 bucks onto it (which translates to 392 HTG, a fairly lousy exchange rate) plus a penny per 10 Goudes as a transaction fee, and off I go. No sign anywhere of what the per-MB cost is. NatCom’s website isn’t particularly helpful in that regard (I later find out that it’s 1.9HTG/MB, about 4 cents.)

Now that I had mobile internet, I fired up the iPad and did some testing on the drive to Petit-Goave, and was getting quite reasonable speeds around 1.5-2Mbps in both directions, very much capable of posting pictures to facebook and whatnot.

Once we got to the guest house where we were staying, we discovered that the wifi there was indeed out of service. I put the hotspot to good use downloading information I was going to need to fix it. In very short order, net access ceases, and I get a screen from NatCom saying that my card is empty, and provides a helpful list of plans and how to activate them. I then go find our hostess and borrow her laptop and internet access to load up some more funds on the card, and then try to activate one of the listed plans. It tells me I can’t do that because I have the wrong type of card.

Then, disaster. Within a matter of little more than an hour, 20 bucks worth of data on the card had vanished. After some digging, I discovered that my good buddy CrashPlan had stabbed me in the back and decided to start a big backup. I killed CrashPlan and reloaded the card (this is getting expensive, and I’m still not entirely sure how much data I’m burning through, especially now that the team is sharing in the internet joy — and the cost!)

Now that I’m back online, I start digging around the NatCom site again to figure out what plans I can access through the SIM I already have. Turns out that they have slightly different SIMs and plans for laptop/USB modems and for mobile phones. I had the latter, a “Nat-Mango” card, which can be had from any street vendor for 25 HTG. I finally found the list of mobile internet plans for the phones, and the correct number to SMS the plan change to. So I send off the text, only to get back “You don’t

Screen Shot 2013-11-05 at 8.03.55 AM

have enough funds for this plan”. I keep moving down the list until even the cheapest one kicks back the message… Uh-oh, I’m running on fumes again. Just as I go to top it up again, it shuts off. Fortunately, one of our Haitian team members had data on his Digicel phone, and I was able to get the account charged up, and switched over to the “Unlimited” plan. Unlimited in this case means 3.5GB at max HSPA+ speeds, then you’re rate-limited to 3.5 Mbps after that. Given that I never saw 3Mbps anywhere, this isn’t really a huge hindrance (that may be a factor of the device more than the network, too). By the time the week was out, our team had gobbled up nearly 25 gigabytes of data through the device.

So, in short, mobile internet from local carriers in Haiti is reliable and cheap (if you know the trick to not paying out the nose per MB), and can be done on a fairly inexpensive piece of hardware. If you’re so inclined, you can also get USB sticks from NatCom for about 1500 HTG. My next step is going to be to see if a device from Cradlepoint can handle the Natcom USB sticks, since they don’t have such a tight limitation on clients.

Hands On: Vizio Tablet (Part 1: Initial Impressions)

October 14, 2011 |   |  Comment  |  Hardware

Over the last few weeks, I’ve noticed that Costco is selling an 8″ Android tablet from Vizio (Model VTAB1008) for the very attractive price of $234.99. Unfortunately, their merchandising is somewhat lacking as this is a Wi-Fi only tablet, and there’s no Wi-Fi to speak of at Costco (and, as it turns out, the demo mode on them won’t allow Wi-Fi anyway!). I remembered today that Costco has a very generous 90-day return policy, which should give me ample opportunity to put one of these units through its paces, and picked one up.

The specs:

  • 802.11n Wifi
  • Bluetooth
  • GPS
  • HDMI Out
  • MicroSD slot
  • 4GB onboard storage (about 2.4 GB is available to the user)
  • 1 GHz Processor
  • Front facing camera (VGA)
  • Universal Remote App
  • 1024×768 screen
  • Lifetime tech support

What’s inside:

  • The tablet
  • Wall charger and cable (standard USB-microUSB)
  • Screen cloth
  • Quick Start Manual (full manual preloaded on the device)

Hardware:

  • Power button on the top, in the middle, flanked with a pair of speakers
  • Volume buttons on the right side
  • MicroUSB and HDMI on the bottom, in the middle, with the SD slot to the left and another speaker to the right – this is so that your audio still sounds right in landscape orientation.
  • There’s a back on it that looks like the battery may be user-accessible.

Initial impressions:

  • This thing seems a little dense for its size, but it’s not bad. The unit weighs in at 20.6 ounces (584 grams). The screen is reasonably bright.
  • When you fire up the tablet (and it came with a more than reasonable 80% battery charge), you go through a simple 5-step setup process that instructs you on general UI usage, connecting to Wi-Fi, and linking up to your Google account. Almost immediately after the setup, the device informed me of an available firmware update that added Netflix and some Android updates. Unfortunately, the device comes with Gingerbread and not Honeycomb.
  • There are 3 soft buttons on the bottom of the screen, the “return” button, a vizio logo, and the “menu” button. The Vizio logo is meant to be the home button, but that’s not particularly obvious (I purposely skipped the UI intro to see how intuitive things would be to someone who uses an Android phone and who has used an iPad)

Preloaded Apps:

  • The usual set of Googly stuff
  • A universal remote control app
  • Barnes & Noble’s Nook app
  • Adobe Flash Player 11
  • Netflix (with the OTA update)
  • Widget Board
The Android market comes up with a UI that looks suspiciously like Microsoft Windows Phone. I installed my usual set of apps, and they work quite nicely.

Weird quirks:

There’s a set of buttons across the bottom on the UI labeled “Browser”, “Market”, “Email”, “Gallery”, and “Music”. Oddly enough, even if you’ve set up your google account, the “Email” button runs you through another setup, asking if the account is IMAP, POP3, or Exchange. Has nothing to do with the gMail app.
Connecting to USB presents you with a file structure that isn’t really clearly documented. Tried copying a video over, but had to use a file browser app to actually locate it.

Really cool stuff:

I dropped an XVID-encoded AVI file onto the storage, and once I was able to locate it (see above), It played. No fuss, no muss, no conversions. This is a pretty standard format for the sorts of video you’d find on the internet (so I’ve heard), so this is really convenient if you have lots of content in that format.

The on-screen keyboard is nice to use, especially coming from a phone-sized android keyboard.

Kindle on this screen is NICE. This could make a pretty good alternative to the Fire.

The size on this unit is about perfect… A little bigger than the 7-inch “mini tablets”, but not as obnoxiously big as a 10″ tablet like a Xoom or iPad.

Screen at full brightness is actually too bright to use indoors.

The soft buttons are also present on the side of the unit. The correct set lights up based on orientation

Audio is excellent.

In combination with CoPilot Live, this could make a kick-butt car GPS.

Meh stuff:

The camera sucks. This is unsurprising. It’ll work fine for video chat, which is its primary purpose.

UI Animations are a little sluggish. They can be turned off, though.

Viewing angles from top and right (portrait) / top and left (landscape) leave something to be desired.

tl;dr version

Decent tablet for the money, probably usable by your grandmother. Onboard storage is adequate for someone not using it as a media device. Expandable storage solves that problem. Good support for multiple media formats. Makes a good e-reader.

Windows Updates, To Go!

July 7, 2011 |   |  1 Comment  |  Haiti, Hardware, IT

When I leave for my trip to Haiti in a few weeks, one of the things I’ll be doing is bringing multiple computers up to current patches. There are a few ways to do that:

One is to bring some sort of removable media (optical or flash stick) down and apply them manually. The problem with this is that once I leave, the machines stay in their current state until the next geek can come down and apply the next batch of patches. Downloading patches for multiple machines over developing-world internet connections can easily run into daily bandwidth caps, and Windows Update doesn’t cache very well through a normal proxy server such as Squid.

Another is to use Windows Server Update Services (WSUS). I initially considered setting up a Windows Server VM on my laptop, syncing up the updates stateside and temporarily configuring the machines down there to pull from my impromptu update server. Then I got the idea that a lightweight appliance-type server that lived down there permanently would be a useful solution that would download the patches once and distribute them over the LAN. Since we’re planning on using Microsoft Security Essentials for anti-malware, this solves the problem of definition updates. Daily patch sync would happen in the wee hours of the morning when the oversubscribed connections in Haiti are generally pretty clear.

I rummaged around the office and found a Dell FX160 thin client that we got as a demo unit from Dell (I have a number of blog posts on the topic of this device). It has been gathering dust for some time as it’s hobbled with a 1GB SATA flash disk and limited RAM. After checking on hardware requirements for both Windows Server and WSUS, I went out and picked up a 120GB SSD and a pair of 2GB RAM sticks and put them in. The choice of an SSD wasn’t so much for performance reasons (although it can’t hurt), but for the machine to be entirely solid-state. It’s going to live in a fairly harsh environment where mechanical failures are likely.

Once I got the hardware put together, I hooked up a USB optical drive and loaded Windows Server 2003 R2, and then installed WSUS and performed an update sync. The whole process went mostly smoothly.

Here are a few of the gotchas in installing Windows 2003 on an FX160 thin client, a job it was NEVER meant to do:

  • SATA controller needs to be in ATA mode. If it’s in AHCI mode, Windows 2003 will not recognize the disk.
  • When using a storage device that the BIOS recognizes as a hard drive, it expects to see a fan plugged into the motherboard. This fan is part of the hard drive bracket kit (Dell P/N H224H). When a fan is not detected, each boot will require a manual intervention during POST to press F1.
  • Stock Windows 2003 media does not include video drivers or network drivers for the FX160 (Broadcom NetXTreme 57XX).
  • Dell’s support site doesn’t have the most recent drivers for the Broadcom.
  • It’s virtually impossible to find a 6″ SATA extension connector, either for data, power, or both. I was finally able to find a power extension, but used a standard SATA cable to connect to the other SATA port on the motherboard.

The SSD I used for this is an OCZ Agility 3, 120GB. Disk performance on large writes is almost 100MB/sec, which is about twice as fast as my 7200RPM spindle drive in my laptop. Windows performs very well with 4GB, a SSD, and a 1.6GHz Atom processor.

The next step was to configure the clients to update from the server for testing. I still have one of the Asus netbooks that we deployed to Haiti in a previous trip. This is where I discovered that Windows Home and Windows Starter don’t include the policy editor (gpedit.msc) that I’m used to finding on Pro/Enterprise/Ultimate versions of windows. This is understandable, your average home user doesn’t (and shouldn’t) normally jack with system policy. Fortunately, all the policy editor does is manipulate registry keys, and the process of configuring Windows Update via the registry is well documented. This actually simplifies things, since all I have to do is create a .reg file that I can import on all the target machines.

Next post: Installing Squid. Not content to use this box for mere update caching, we’re gonna have it be our web proxy as well.

Sony VISCA RS-422 Control

December 20, 2010 |   |  81 Comments  |  Hardware, Internet Campus, streaming

Update – January 2014: Wow, 3 years later this is still one of the most popular posts on this blog! I’ve had some questions about using this with the EVI-HD1, which has only RS-232 DIN ports. Theoretically, it should work, but you might need to alter some pinouts in the breakout box, and I would highly recommend using shielded/grounded cable, as RS-232 is an unbalanced signal. A reader is going to give it a try, and if it’s successful, I’ll update the post with some pictures.

Update – October 2021: It blows my mind that nearly 11 years later, this is still one of the most frequently visited posts on the site…

Now, back to our regularly scheduled blog post!

We recently acquired a few more of Sony’s excellent EVI-D70 cameras for use in our chapel for streaming weddings, funerals, and other events in our smaller worship space.

When we remodeled the space a few years back, it was originally designed with these cameras in mind.  The original intent was to provide some additional angles for videographers to use, but the idea never really took off. Due to lack of use, the cameras originally installed were re-purposed for Resurrection Online in the main sanctuary. Things have come full circle now, and the ability to stream services and events from this space is being requested. As a result, we acquired some more cameras, and are in the process of updating the camera system in that room.

The original design used an AMX touchscreen/joystick controller and a custom integration over RS-232, with each camera homerun to the control rack. There were numerous difficulties with the cameras randomly freezing up and not responding to controls, requiring someone to get on a ladder and power-cycle the unit.

As part of the updated system, we’ve ditched the AMX controller and are using Sony’s RM-BR300 control unit which is designed for this particular camera. We also have user familiarity, since we already have one of these controllers in our main sanctuary for the BRC-H700 remote camera mounted on the catwalks (aka, the “SkyCam”). The controller can do Sony’s VISCA protocol over RS-232 (via a Mini-DIN) or RS-422 (via a Phoenix connector).

This is where it got sticky. We have an 8-conductor homerun cable from each camera position, but the Sony controller is designed to daisy-chain the VISCA ports. Each camera has two RS-232 Mini-DIN ports (one in, one out). Fortunately, both RS-422 and RS-232 in this application only require four wires, so we can loop out and back on the same cable.

Due to the annoyance factor of having to re-terminate Mini-DIN connectors, I opted for the RS-422 port which uses a Phoenix screw terminal (Part # 1840434 in case you need to order one – Sony wants an obscene amount of money for them, they’re dramatically cheaper at an electronics supplier like Mouser). RS-422 also has the advantage of much longer signal path due to its balanced signal. Since we’re also adding a new location, I wanted to be able to wire it up with standard Category 6 twisted-pair cabling. This cable also has eight conductors, making it ideal for the task. In terms of flexibility, RJ-45 is king in the twisted-pair world, so I had do design a means of daisy-chaining my VISCA ports via ordinary patch cords.

At first, I was a little baffled by the wiring of VISCA, since the RM-BR300 connector pinout is exactly backwards from that of the one on the cameras, and the documentation provided is a little confusing. Fortunately, the Sony POSC was quick to help and they e-mailed me a wiring diagram for this specific application (and were kind enough to allow me to post it. I translated that into two main components, a breakout box and a standard cable, that would work on either the controller or the cameras.

To make the cable, I simply took a patch cord off the shelf, lopped one end off, and terminated it on the Phoenix connector:

The wiring is as follows:

Now, you’ll notice my wiring diagram shows the orange pair on the first two, and the picture shows green. This is because I found out (after much frustration of tracing signals) that the patch cord I grabbed happened to be wired for 568A rather than the more common 568B. Simply swap orange and green if this is the case.

Once I got the cables sorted out, I then replicated Sony’s wiring diagram with a handful of data jacks. The connections go like this:

I used bits I had on the shelf, but I would recommend using a different jack color for the control input so you don’t get it confused. Once I got it wired up, this is what I had (I colored the control jack black with a Sharpie):

Even if this install only has three cameras, I wired it up for five, to fill a six-way biscuit box that I had on the shelf (these are Lucent/Avaya components):

.. and put the lid on it with some labels:

As for the hookup, set the DIP switches on the bottom of the controller and the cameras to use RS-422 and either 9600 or 38400 bps, and hook them up. Note that they must be in sequence, or the whole chain will be broken if you skip a slot. Plug a camera into #1, it will be #1 on the controller after they self-enumerate on startup, in order of closest to farthest on the chain. Connecting a camera will cause the controller to re-initialize.

Action Shot:

I used a biscuit box, but you could also use a modular patch panel to do the same thing. I hope to use a second category 6 run with an S-Video termination on it (2 pairs) and power (other 2 pairs) so that the whole system can run off a standard 2-cable pull.

Review: EcoSmart A19 LED bulb (40W Equivalent)

December 3, 2010 |   |  1 Comment  |  Hardware

EcoSmart A19 LED BulbI’ve posted before about LED lighting, and consider myself an early adopter.

Recently, Home Depot significantly expanded their selection of LED bulbs beyond the Philips brand that I’ve been using for a few years, mostly in my kitchen track lights (GU10 formfactor). I’ve been quite happy with those, and I only have two more halogens left in the track that seem to be hanging on, knowing their fate should they burn out.

The EcoSmart line appears to be a Home Depot house brand, and as such is only available there. The brand encompasses both CFL and LED lighting technologies. The bulbs are less expensive than the Philips bulbs.

I picked up eight of the 40W-equivalent A19-style bulbs. These have a form factor that matches that of a traditional incandescent bulb, so they’ll fit anywhere a regular bulb will fit, unlike the bulbs I blogged about previously. I bought these to use in our master bath above the vanity, and the  master bedroom ceiling fan, where solid-state bulbs are a much better option due to the vibrations of the fan which drastically shorten the life of filament bulbs. Because of this application, these bulbs are also dimabble.

The specs on the bulb are as follows:

While this bulb is listed as a 40W equivalent, because of the directionality of the bulb, it’s roughly equivalent to the light you’d get from a 75W bulb in directional applications. I tried it in my regular bathroom fixture, which assumes a standard bulb emitting light in all directions, and it’s not up to the task. Side-by-side comparisons in a reading lamp, on the other hand, give a result that looks almost identical to the 75W incandescent bulb that was in it previously.

In the ceiling fan application (four bulbs), it lights up the large master bedroom quite nicely and dimming is not only smooth and flicker-free, but quiet. The overall consumption of  35W is a dramatic improvement over the set of 75W halogens that used to be in there.

This is the packaging for eight bulbs. The tiles are 12" across.

My only real gripe about these bulbs is the logical disconnect between the “eco-smart” name and the ridiculous amount of packaging they come in. The bulb itself is in a plastic clamshell that is not coded for recylcing (but is likely PET), which is then encased in a fairly convoluted cardboard box big enough for a PAR38. These then come four each in a corrugated cardboard tray for retail display. I haven’t been able to confirm it, but the boxes look like they may use the same box die for all their bulbs, and swap out the plastic shell for the appropriate one. At least it’s not your typical wound-inducing plastic retail shell. If I needed to return these to the store, it looks simple enough to repackage them such that they can be resold.

Home Depot also offers EcoSmart LED bulbs in PAR30, PAR38, PAR20, G25, MR16, and E26 formats, as well as a whole host of holiday lighting options.

Time will tell how well these hold up. Next up are likely going to be the G25 bulbs in the dining room fixture.