There’s a box that every tech Journalist has to check in their career, writing the overhyped, shark-jumping, “The Internet is Doomed” article. People read it and shake their heads, and the Internet goes on. Not this time… this time we’re all doomed. Over the years there have been millions of support tickets complaining of slowness and lag. They’re all real, and they’re real because the internet’s buffers are bloated. It’s getting worse too. In this article are tools designed to detect, and combat bufferbloat. Someday soon those tickets, and indeed lag itself will be a thing of the past, or the Internet will die.
Bufferbloat, network-crippling latency spikes caused by large and misconfigured buffers in network equipment, is really nothing more than a traffic jam on the Internet. As traffic density increases, hardware buffering built into the devices on your network causes all of the traffic to slow down. This is a critical loss of performance for VoIP, games, and cloud applications.
So do you have bufferbloat? Solving it might be tricky, but there’s a quick easy way to find out!
DSL Reports free internet speed test now tests for bufferbloat as part of their speed test. You can access their test here.
We ran two tests in the Zenoss lab to see what we would get. The test machine for both tests is a “late ‘14” Macbook Air, with 802.11AC, and Apple’s Thunderbolt Ethernet connector. For the wireless test (top) we used the Linksys WRT 1900ACS from our previous open source networking blog article. The WRT is plugged directly into a Juniper SRX-220 running the latest JunOS firmware (12.3). For the wired test (bottom), we plugged the Macbook directly into the Juniper SRX.
Bufferbloat at Zenoss
So it appears that we have bufferbloat here at the Zenoss Lab. The test provides additional information to help us determine where it’s coming from. Pulling data doesn’t result in bufferbloat, but pushing it does. This means that the issue is likely to be on our test network rather than upstream.
While DSL Reports gave us a C, it’s unlikely a VoIP call would survive a prolonged bout of 240ms latency. Web Applications would become sluggish and unresponsive at the office, and productivity would most certainly be lost due to the slowdown.
So why did we have bufferbloat? After testing, we found that it was the DSL modem, or DSLAM causing the bloat. There was not much that could be done about that, sadly.
Not all bufferbloat is on the internet, in fact most of it isn’t, and more complex network topographies can result in bufferbloat on the LAN, and legacy applications in the corporate datacenter can suffer. VPN’s, and WiFi create bufferbloat too.
Testing across the corporate LAN/WAN environment can be easily accomplished with Netperf<http://www.netperf.org/netperf/>, and Flent<http://www.flent.org/>. Flent utilizes Netperf, the Netperf netserver daemon and Matplotlib to run, and chart performance tests across a LAN/WAN environment. Ideally you will want to install Netperf netservers throughout the network. Not only will Flent test for bufferbloat, it will also detect latency, and bottlenecks. Ideally, Flent should be run during peak network usage to ensure that your tests match worst case user experience.
How do you go about remedying bufferbloat? Most major routing hardware vendors have documentation, or at least release notes covering buffer tuning. If you identify a problem device, a support request to the vendor should get you what you need to resolve it.
So we’ve checked the box reader, we’ve written the article and doomed the Internet. Best of all, we’ve checked it together. The Internet is doomed to be crushed under the weight of bloated buffers. There is hope for the Internet now though. You know how to save your network from bufferbloat. You can monitor for bufferbloat on an ongoing basis with Zenoss by utilizing a custom Netperf or Flent command datasource<http://wiki.zenoss.org/Writing_Command_Data_Sources>. Remember that netperf, and flent run for an extended (specified) period of time, run them so that they’re non-blocking.
Interested in learning more? Enter your email address below to subscribe to our blog!
