Monitoring Nexus 9000 VXLAN

Zenoss recently announced support for Nexus 9000 VXLAN. VXLAN is a very important technology for anyone who wants to extend VLANs across their datacenter, across multiple datacenters, or is adding Cisco ACI or any software defined networking function. I thought this new function was worth more of an explanation than the 20 or so words in the ZenPack Catalog that covered the new function.

First, let’s revisit why VXLAN matters and provide a simplified explanation of how it works.

Why VXLAN Matters

Over the past few years, most data centers have been adding a layer of application security in the networks. The goal is to limit the attack surface of a security incident by limiting the internal network visibility. For example, the only way to gain access to a web application server is through specific ports from the web servers. If (oh, no!) a hacker gained root access on the web server, they wouldn’t be able to see our financial servers or manufacturing systems, or even do anything on the web application servers other than make application function calls.

Several network technologies work together to accomplish this, but the important one here is VLANs. A VLAN is a virtual network segment. Through software, we can create a VLAN within a switch or across several switches and connect specific hosts to a VLAN.  The networking devices enforce the security restrictions, and we have another layer of protection.

As organizations have made increasing use of VLANs, several issues have come up:

  • A single network switch can only support 4,000 VLANs. Sounds like a lot, but a typical three tier application can easily use five or six and in a big network pod it’s easy to run out.
  • A VLAN only operates easily within a single network pod. Connecting from one set of switches to another means adding routing function and additional complexity. And if the application actually requires layer 2 connectivity the routing configuration gets even more complex.
  • Cisco ACI uses VXLAN internally, and customers need a way to create private networks that include devices connected to an ACI pod with devices on legacy networks.

VXLAN directly addresses all of these issues. It can support millions of private networks and is designed to connect devices in one pod to another, even across geographically distributed data centers.

How VXLAN Works

To extend a VLAN across multiple network pods, you’ll need to add a switch supporting a VXLAN Tunnel EndPoint (VTEP) to each pod. The VTEP serves as a VLAN gateway, bridging VLAN traffic over VXLAN tunnels. A VXLAN tunnel is a logical connection between VTEPs that carries VLAN traffic over standard IP network equipment.

One of the major advantages of VXLAN is that you add to your standard network without making any changes to intervening switches and routers. All of the VTEPS automatically locate one another, so there’s no need for network changes and all the packets carried over the virtual tunnel are standard TCP/IP packets. VXLAN wraps the VLAN packets from one endpoint in an IP packet envelope, they swim across the network, and the target endpoint unwraps the packet and sends it along its way in the new network pod. Pretty slick!

VXLAN switches create a Virtual Network Interface (VNI, naturally) for each connection between VLANs in different pods. All the VNIs between two pods are carried in the same tunnel, and the switch tracks the amount of traffic for each VNI. And the best news – there can be millions of VNIs, so we are unlikely to run out during this decade’s career.

Monitoring VXLAN

So this is easy to add, and no core network changes are required. But there is no magic, and sometimes things can go wrong. What do you need to monitor with a VXLAN installation?

Here’s the three basic things you need to watch for with VXLAN monitoring:

  • Which Nexus 9000 switches are operating as VXLAN Tunnel EndPoints? And are they working properly? What’s the total throughput across the tunnel?
  • We want to track usage of individual VNIs, to know where all the bandwidth is disappearing. And we need to know the relationship between VLANs and VNIs.
  • We’ve got to watch for errors, to know if problems are affecting our smooth flow.

Those are some familiar concepts. How do you make it happen?

Zenoss Provides VXLAN Monitoring

And that’s exactly what the new Zenoss VXLAN support for Nexus 9000 provides.

First, it automatically discovers when a Nexus 9000 switch is operating as a tunnel endpoint, and picks up all the configured VLANs, VNIs and the relationships between VLAN and VNI. It identifies all the peer tunnel endpoint switches that this switch is aware of.

Figure 1: Zenoss Service Dynamics maintains an inventory of VXLAN interfaces, peers, VNIs, and VLAN-VNI relationships
Figure 1: Zenoss Service Dynamics maintains an inventory of VXLAN interfaces, peers, VNIs, and VLAN-VNI relationships

Second, it tracks availability for the switch itself, and collects any fault events.

And third, it tracks bandwidth usage for the tunnel as a whole and for individual VNIs. In case you’re worried about scale, the Zenoss user interface is designed to function well for up to 10,000 VNIs. While that’s a long way from the maximum possible number of VNIs, we’re pretty sure that it’s a lot more than any customer will be using for at least a couple of years.

Figure 2: VXLAN NVE performance tracks overall utilization of the VXLAN tunnel at this switch
Figure 2: VXLAN NVE performance tracks overall utilization of the VXLAN tunnel at this switch
Figure 3: Track send and receive bandwidth utilization for every VXLAN VNI
Figure 3: Track send and receive bandwidth utilization for every VXLAN VNI

VXLAN and Cisco ACI

You probably know that Zenoss is really excited about Cisco Application Centric Infrastructure. When you use ACI to define an application in the network, Zenoss can automatically extend monitoring of the compute, storage, guest OSs, and virtualization infrastructure that supports the application. The tight integration makes the transition from Day 1 setup to Day 2 operations practically seamless – especially since changes to the application, like adding a new web server VM to a load balanced end point group automatically become part of the Day 2 service assurance.

Figure 4: Zenoss impact graphs provide root cause analysis for the infrastructure supporting ACI LF-L7 network services
Figure 4: Zenoss impact graphs provide root cause analysis for the infrastructure supporting ACI LF-L7 network services

Our new VXLAN support works perfectly with Cisco ACI network pods. No surprise, really, since ACI is VNI all the way down. No VLANs inside of your ACI network – all the end point groups are enabled by VXLAN. This makes the job of the VXLAN Tunnel End Point switch connected to an ACI leaf node a bit easier – there’s no need to remove a VLAN packet from the VNI envelope.

Everything Zenoss tracks for a Nexus 9000 VXLAN Tunnel Endpoint switch looks identical whether that switch is connected to an ACI pod or classic Core-Aggregation-Access network structure.

Is VXLAN Coming to a Data Center Near You?

It’s fairly safe to say that VXLAN is coming soon to your data center, if it’s not already there. If you’re already a Zenoss customer, just download and install the updated Cisco Devices ZenPack from the customer support portal and you’re ready to go.

New to Zenoss?

If you want to learn more about Zenoss and what we do check out the following links, which provide more information about Zenoss and the unified monitoring capabilities Zenoss provides:

  • Watch Quick Tour: Unified Monitoring with Zenoss Service Dynamics Resource Manager to get a quick overview of Zenoss and how Zenoss can help your more efficiently and cost-effectively monitor your environment.
  • Read Unified Service Insight from Zenoss to get a quick overview of how Zenoss Service Dynamics provides Unified Service Insight into the end-to-end operation of IT infrastructures and supports service delivery, improves service quality, and reduces your IT operational costs.
  • Request a free trial! See how you can use Zenoss more in your environment to more effectively and cost-effectively monitor and manage your environment using a single, unified monitoring view and unified monitoring processes.


Enter your email address in the box below to subscribe to our blog.

Zenoss Cloud Product Overview: Intelligent Application & Service Monitoring
Analyst Report
451 Research: New Monitoring Needs Are Compounding Challenges Related to Tool Sprawl

Enabling IT to Move at the Speed of Business

Zenoss is built for modern IT infrastructures. Let's discuss how we can work together.

Schedule a Demo

Want to see us in action? Schedule a demo today.