Trying again on the Wireless LAN bridge

It's been quite some time since I gave an update on our experience with the wireless LAN bridge that connects our staff offices at Southcreek (SC) with our Central Campus (CC), 1.7 miles away.  After putting the pair of 80 GHz radios from Bridgewave into service, I posted about how we were suffering from link outages due to rain fade.  This is a problem because the bridge is the only connection to our 40 staff at SC, carrying voice, LAN access to our data center, and Internet access.

The signal budget takes into account the clear-air RSL (received signal level), an allowance for rain fade according to weather models, and a small margin before link drop.  The link will definitely drop at -72 dBm, but link quality gets very poor with high Ethernet frame errors at around -69 dBm.

Here's the timeline:

Apr. 10, 2007: Link is live just before staff move-in.  RSL is -45 dBm on both ends, giving a fade margin of 24 dBm.  Link down for 1 minute in a brief, heavy rain.

Apr. 16, 2007: Based on some rain events during the test period, we decide to move the CC radio to a more favorable location.  We also replace both radios after observing link drop at -66 dBm, which was 3 dB higher than the specification. RSL is -41 dBm CC and -44 dBm SC.  For the first time we notice a significant difference in RSL on the two ends.

Apr. 25, 2007: Afternoon storms result in 23 minutes of outages.  Staff at SC are inconvenienced, but understanding.

May 6, 2007: A large storm brings flooding rains to much of Kansas.  Link is down for a total of 76 minutes.  It's a Sunday when staff are not in the office, so no impact.

May-July 2007:  Link drops every time there is a decent rain.  Some of these are during office hours, causing user impact.  Staff at SC are getting restless.  We continue to document each incident, providing our vendor RSL data captured by What's Up Gold and rain gauge data from the official city gauge near SC.

Aug. 2007:  We continue to experience outages during relatively routine rains.  Our vendor begins to discuss the possibility of putting high-gain (2' dish) antennas on one or both ends of the link.  They cost around $2,000 a piece and are designed to increase signal strength by approximately 6 dB each.  We press our vendor to help us with this since the link isn't meeting our availability requirement.

Sept. 20, 2007:  SC end RSL drops from -44 dBm to -54 dBm.  What's Up Gold records the drop, but the link stays up so we don't notice.

Nov. 20, 2007:  Issues are resolved with the vendor.  Bridgewave provides the high-gain antenna at cost.  Our vendor installs it on CC end.  When looking at RSL values post installation, we discover the 10 dB drop that happened two months earlier.  RSL increases from -42 to -36 dBm CC, and from -54 to -48 dBm SC.  Winters in Kansas City are relatively dry, so outages are rare and we still have plenty of time to resolve the issues.

Feb. 15, 2008:  After further discussion with Bridgewave, they agree something is wrong with our SC radio and ship us a replacement along with a high-gain antenna.  RSL increases from -36 to -31 dBm CC, and from -48 to -32 dBm SC.  So the SC end got the 10 dB back, plus the additional 6 dB from the high-gain antenna.  Photos of the installation are below.  You can see how much larger the high-gain antenna is.

Old radio being uninstalled

New radio with 2', high-gain antenna being installed

Clear air signal strength of -32 dBm means we now have 37 dB of fade margin.  With that much margin, we estimate that the link will stay up during rain rates up to around 1.5 inch/hour.  According to the weather data model used by our vendor, that should give 4 nines of availability in an average Kansas City year (56 min. total down time).  Of course, no year is actually average, so we'll just have to wait and see what happens.

Over the weekend we had a storm that started as moderate to heavy rain and ended as warm, wet snow.  During the heaviest period of rain, the signal dropped briefly to as low as -59 dBm.  That's encouraging because we still had more than 10 dB of fade margin at that rain rate.  That event would have caused a link drop if it happened before the high-gain antennas were installed.  Big improvement.

When it started snowing yesterday, the signal dropped steadily from -37 to -67 dBm over a period of five hours.  Turns out the wind direction was directly into the the CC radio's face, causing the wet snow to accumulate on the antenna.  When Ian got up on the roof and cleared the snow, the signal immediately came back up to -37 dBm.  Yeah - a 30 dB signal drop from snow on the antenna.  Yikes!  Ian has full details, including pictures, on his blog.