Map Your Own IT Ecosystem
See how Zenoss fits into your IT ecosystem.
Start MAPPING NOW
Huntington Success Story
Learn how Huntington consolidated from 37 tools to a single monitoring solution.
Customer Support Portal
Zenoss Learning Center
Zenoss & Cisco: Changing IT with service assurance for hybrid data centers.
Zenoss Partner Portal
Become a Partner
BBC Success Story
The BBC chose Zenoss for three key reasons: scalability, flexibility and value for money.
Hybrid IT Monitoring
Zenoss provides complete visibility into physical, virtual, cloud and converged environments.
Request A Demo
This is an Open Source ZenPack developed by Zenoss, Inc. Enterprise support for this ZenPack is available to commercial customers with an active subscription.
The ZenPacks.zenoss.Hadoop ZenPack monitors Hadoop clusters via JMX web interface (JMXJsonServlet) and Python twisted.web.client asynchronous framework.
The features added by this ZenPack can be summarized as follows. They are each detailed further below.
The following components will be automatically discovered through zProperties you provide:
Note: Yarn components (Resource Manager, Node Manager, Job History) will be modeled for 2.x version of Hadoop and MapReduce components (Job Tracker, Task Tracker) for previous versions.
The following metrics will be collected and graphed every 5 minutes by default:
The following events will be triggered:
When combined with the Zenoss Service Dynamics product, this ZenPack adds built-in service impact capability for Hadoop. The following service impact relationships are automatically added. These will be included in any services containing one or more of the explicitly mentioned entities.
With a fully deployed Hadoop environment, the HBase Master server can move around to different Hadoop Nodes. This ZenPack provides optional discovery of HBase Master Device within Hadoop Data Nodes, which may be configured with the help of the following configuration properties:
If you checkmark zHBaseAutodiscover property, every 5 minutes by default each Hadoop Data Node will be probed for hosting HBase Master. In case HBase is found, the HBase device will be created in the Device Class set in zHBaseDeviceClass (usually /Server/Linux or /Server/Microsoft/Windows) with HBase modeler plugins bound. If the HBase device was already discovered, but HBase Master moves to a different Node, the device's IP address will be changed to a new one.
Use the following steps to start monitoring Hadoop cluster:
Alternatively you can use zenbatchload to add Hadoop monitoring servers from the command line. To do this, you must create a text file with hostname, username and password of all the servers you want to add. The devices should be added under the target machine device class (e.g. '/Devices/Server/Linux' in the example below).
/Devices/Server/Linux <HOST NAME or IP>
zCollectorPlugins=['HadoopDataNode', 'HadoopServiceNode'], zHadoopNameNodePort='50070' zHadoopPassword='', zHadoopUsername='', zHadoopScheme='<http(s)>'
You can then load the Hadoop monitoring servers into Zenoss Core or Resource Manager as devices with the following command.
$ zenbatchload <filename>
This ZenPack uses the NameNode WebUI port for Hadoop modeling and performance monitoring, therefore please make sure that the UI instance is running. For more information refer to HDFS Ports.
Please refer to the Zenoss Service Dynamics documentation if you run into any of the following problems:
If you cannot find the answer in the documentation, then Resource Manager (Service Dynamics) users should contact Zenoss Customer Support. Core users can use the #zenoss IRC channel or the community.zenoss.org forums.
Installing this ZenPack will add the following items to your Zenoss system.
$ sudo su - zenoss
$ zenpack --install ZenPacks.zenoss.Hadoop-*.egg
$ zenoss restart
In order to do a development mode installation you will want to clone the existing git repository, and then use the --link flag with the zenpack command:
$ git clone https://github.com/zenoss/ZenPacks.zenoss.Hadoop.git
$ zenpack --link --install ZenPacks.zenoss.Hadoop
View the discussion thread.
This ZenPack is developed and supported by Zenoss Inc. Contact Zenoss to request more information regarding this or any other ZenPacks. Click here to view all available Zenoss Open Source ZenPacks.