ZenPacks

EMC.base ZenPack

The EMC base ZenPack provides support for modeling, monitoring, event management, impact services and analytics for block and file storage for EMC VNX/VMAX devices.

Releases

Version 1.2.1- Download
Released on 2017/06/09
Requires WBEM ZenPack (>= 1.0.3), PythonCollector ZenPack (>=1.5.2), StorageBase ZenPack (>=1.3.0), ZenPackLib ZenPack (>=2.0.4)
Compatible with Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.0.x, Zenoss Resource Manager 5.1.x, Zenoss Resource Manager 5.2.x
Version 1.1.2- Download
Released on 2016/02/24
Requires WBEM ZenPack,PythonCollector ZenPack,StorageBase ZenPack
Compatible with Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.x
Version 1.0.5- Download
Released on 2015/09/11
Requires WBEM ZenPack ,PythonCollector ZenPack ,StorageBase ZenPack ,CalculatedPerformance ZenPack 
Compatible with Zenoss Resource Manager 4.2.x, Zenoss Resource Manager 5.x

Features

  • Support block storage via SMI-S for VNX
  • Support block storage for Symmetrix VMAX 10/20/40K Series
  • Support file storage via API for VNX
  • Initial discovery and continual update of relevant components.
  • Performance monitoring of both of statistics and health scores.
  • Event management integration of VNX/VMAX events into Zenoss event system.
  • Service impact and root cause analysis. (Requires Zenoss Service Dynamics)

EMC Prerequisites

  • EMC Solutions Enabler V7.6
  • SMI-S Provider 4.6

Block Storage for VNX/VMAX

For monitoring block storage, the EMC base ZenPack enables Resource Manager to use the Web Based Enterprise Management (WBEM) protocol to monitor and model VNX and VMAX devices under /Storage/EMC/VNX and /Storage/EMC/VMAX device classes by executing queries and parsing the output. Parsing of this information is performed on the Resource Manager server (if using a local collector) or on a distributed collector. Information is gathered via the EMC Storage Management Initiative Specification (SMI-S) Provider.

The following video demonstrates how to setup block storage monitoring via SMI-S on EMC.base versions 1.0.5 and 1.1.2.

Block VNX/VMAX Discovery

Block VNX Storage Model Diagram

The following components will be discovered when modeled.

Data Devices
Properties: OwningStorageProcessor, RAID, StoragePool, Array ID, WWN, Total Managed Space (MB), Total Managed Space (GB), StoragePoolID, StoragePoolGroup, Total Consumed MB, Total Consumed GB, WBEM Class Name, WBEM SP Instance ID, WBEM SV Creation Class Name, WBEM SV Device ID, WBEM SV System Name, WBEM SV System Creation Class
Relationships: Storage Pools, Storage Pool Groups
Batteries
Properties: Battery Status, Battery ID, Array ID, Enclosure, Manufacturer, WBEM Class Name, WBEM Tag
Relationships: Storage Enclosure
Hard Disks
Properties: Volume, Storage Pool ID, Disk Type, Array ID, Disk Drive Name, Storage Pool, WBEM Class Name, WBEM Device ID, WBEM System Name, WBEM System Creation Class Name
Relationships: Storage Pools, Storage Enclosure
Power Supplies
Properties: Power Supply Serial Number, Power Supply ID, Enclosure, Array ID, EMC Status, WBEM Class Name, WBEM Tag
Relationships: Storage Enclosure
Storage Pools
Properties: Pool ID, EMC Status, Total Managed Space, Thin Provisioned Space, Array ID, Space Limit, Element Name, Number of Disks, Number of Data Devices, Class Name, Tag
Relationships: Data Devices, Hard Disks
Link Control Cards
Properties: LCC Name, EMC Status, Version, Array ID, Card ID, Enclosure ID, WBEM Class Name, WBEM Tag
Relationships: Storage Enclosure
Storage Processors
Properties: SP IP, SP Name, Array ID, Manufacturer, Version, Serial Number, Number of Ports, Enclosure, WBEM Class Name, WBEM Tag
Relationships: Storage Enclosure, SP Ports
SP Ports
Properties: Port Name, Port Number, Array ID, Processor Card ID, Port Status, Processor Card, Enclosure, WBEM Class Name, WBEM Tag
Relationships: Storage Processor
Storage Enclosures
Properties: Enclosure State, Serial Number, Address, Hardware model, Manufacturer, Product ID, Logical Identifier, Firmware Revision, Title, Array ID, Number of Batteries, Number of Disks, Number of Link Control Cards, Number of Power Supplies, Number of Storage Processors, WBEM Class Name, WBEM Tag
Relationships: Storage Processors, Link Control Cards, Batteries, Power Supplies, Disks
Arrays
Version, Name, Model, Serial Number, Block SW Version, File SW Version, WBEM Class Name, WBEM Identifying Number, WBEM Tag
Relationships: SMI-S Provider

Block VNX/VMAX Performance Monitoring

The following datapoints will be collected every 5 minutes by default.

Data Devices:
  • Datasources - Status, StorageVolume
  • Graphs - Time Utilization, Queueing, Operation Throughput, Data Throughput, Response Time
Batteries:
  • Datasources - Status
Hard Disks:
  • Datasources - Disk Drive, DiskExtent, Status
  • Graphs - Space Utilization, Time Utilization, Operation Throughput, Data Throughput
Power Supplies:
  • Datasources - Status
Storage Pools
  • Datasources - UnifiedStoragePool
  • Graphs - Space Utilization, Managed Space Usage, Raw Space Usage
  • Thresholds - Low Managed Space, Low Remaining Capacity, Over Subscribed, Projected Hard Disk
Link Control Cards
  • Datasources - Status
Storage Processors
  • Datasources - Status, StorageProcessorSystem
  • Graphs - Time Utilization, Flush Rate, Flush Throughput, Operation Throughput, Data Throughput
SP Ports
  • Datasources - FEPort, Status
Storage Enclosures
  • Datasources - Enclosure Chassis, Status
  • Graphs - Input Power, Inlet Temperature
Arrays
  • Datasources - Array, ArrayChassis, PrimordialStoragePool, Status
  • Graphs - Flush Rate, Operation Throughput, Data Throughput, Input Power
  • Thresholds - Low Managed Space, Low Total Capacity

Block VNX/VMAX Impact Services

When combined with the Zenoss Service Dynamics product, this ZenPack adds built-in service impact capability for EMC devices. The following service impact relationships are automatically added. These will be included in any services that contain one or more of the explicitly mentioned entities.

Service Impact Relationships
  • SP Ports failure affects related Storage Processors
  • Power Supplies, Batteries, Link Control Cards, Storages Processors, or Device failure affects related EMC Storage Enclosures
  • Storage Enclosures failure affects related Hard Disks
  • Hard Disks failure affects related Storage Pools
  • Storage Pools failure affects related Data Devices

File Storage for VNX

For monitoring file storage, the EMC base ZenPack enables Resource Manager to communicate with the VNX XML API Server under /Storage/EMC/VNXFile device class. The API queries are made directly to the VNX Control Station from the Resource Manager server (if using a local collector) or on a distributed collector. If the XML API Server is disabled, instructions on Starting the XML API Server  are detailed below in this page.

Reference: EMC API documents available here: https://community.emc.com/docs/DOC-19626

File VNX Discovery

File Storage Model Diagram

The following components will be discovered when modeled.


File VNX Overview

Overview
Properties: Product Number, Serial Name, File Version
Graphs: Top 5 File Systems Usage, Top 5 Storage Pools Usage

Graphs populate when collector collects File System and Storage Pool metrics

Control Station
Properties: Address, Hostname, Slot, Version, DnsServers, NtpServer, Netmask, TimeZone, Type
Relationships: CelerraSystem, Mover
Mover
Properties: Role, Host, Standbys, StandbyFors, I18NMode, FailoverPolicy
Relationships: MoverInterface, CifsServer, Volume, FileStoragePool, FileSystem, NfsExport, CifsShare, CelerraSystem, ControlStation
Interfaces
Properties: Device, MacAddr, IpAddress, Mtu, Up, Vlanid, IpVersion, NetMask, BroadcastAddr
Relationships: Mover, CifsServer
Volume
Properties: Type, Size, VirtualProvisioning, ClientVolumes (Used Volumes, Used by Volumes), ClientFileSystems
Relationships: Mover, FileStoragePool, FileSystem, UsedBy, Uses, CelerraSystem
Storage Pool
Properties: Description, DiskType, Size, MemberVolumes, Storage, MayContainSlicesDefault, AutoSize, VirtualProvisioning, IsHomogeneous, DataServicePolicies, TemplatePool, StripeCount, StripeSize
Relationships: Volume, FileSystem, CelerraSystem, Mover
Filesystem
Properties: Type, ContainsSlices, BasedOn, InternalUse, Storages
Relationships: NfsExport, CifsShare, CelerraSystem, Mover, Volume, FileStoragePool
NFS Export
Properties, Fil, Path, ReadOnly
Relationships: Mover, FileSystem
CIFS Server
Properties: Interfaces, Type, Workgroup, LocalUsers, MoverIdIsVdm
Relationships: MoverInterface, CifsShare, Mover
CIFS Share
Properties: Path, FileSystem, MoverIdIsVdm
Relationships: Mover, CifsServer, FileSystem

File VNX Performance Monitoring

The following datapoints are collected every 5 minutes by default.

Movers:
  • Datasources - Network-Ip, Network-Tcp, Network-Udp, ResourceUsage
  • Graphs - Utilization, IP Network, TCP Network, UDP Network
  • Thresholds - CPU Utlization (default at 90%), Memory Utlization (default at 90%)
Mover Interfaces
  • Datasources - Network-Devices
  • Graphs - Traffic
StoragePool
  • Datasources - FileStoragePool
  • Graphs - Capacity
  • Thresholds - Pool Capacity (default at 90%)
FileSystem
  • Datasources - FileSystemUsage
  • Graphs - File Usage, Space Usage
  • Thresholds - FileSystem Capacity (default at 90%)
NFS Export
  • Datasources - V2 (Calls/Time/Failure), V3 (Calls/Time/Failure), RPC
  • Graphs - V2 Calls, V3, RPC Requests
CIFS Share
  • Datasources - SMB (Calls/Time), Trans2 (Calls/Time), NT (Calls/Time), State
  • Graphs - Procedure Calls, Open State

File VNX Events


File Events

Zenoss periodically polls the Control Station for VNX events and creates Zenoss events when they occur. Events can be configured either through the VNX device or blocked/filtered through the unique eventClassKey: emc-vnx-file-events| in Zenoss Resource Manager. Events are collected every 5 minutes by default. 

EMC VNX File (Celerra) events are mapped to Zenoss events in the following way:

Components:
Mapping of Celerra Sources and Zenoss Components

Celerra Source

Zenoss Component

Celerra SourceZenoss Component

DART (Data Access in Real Time)

Mover

CS_CORE, XML_API, CS_PLATFORM

Control Station

Severities:
Mapping of Celerra Severities and Zenoss Severities

Celerra Severity

Zenoss Severity

Celerra SeverityZenoss Severity

Clear

Clear (0)

Debug

Debug (1)

Info

Info (2)

Notice

Info (2)

Warning

Warning (3)

Error

Error (4)

Critical

Critical (5)

Alert

Critical (5)

Emergency

Critical (5)

Zenoss Event description
Severity: Zenoss Severity mapped from Celerra severity
Component: Zenoss Component mapped from Celerra source
EventClass: Zenoss Event Class generated internally from eventClassKey rule
eventKey: Celerra Message code
eventClassKey: emc-vnx-file-events|
summary: Description of event or Celerra message code
message: An unparsed XML API response recevied from the VNX XML API Server
celerra-facility: Celerra Facility
celerra-target-logtime: Celerra logtime
EMC Celerra documents warn that message codes may vary with different models. If the collector receives an unknown message code, the Zenoss summary event field will display the celerra message code and the Zenoss message event field will display the full unparsed XML API response.

Reference: Mapping of Celerra message codes and Celerra descriptions are referenced  from EMC's Configuring Celerra Events and Notifications

File Service Impact and Root Cause Analysis

File Impact Services

When combined with Zenoss Service Dynamics, this ZenPack adds built-in service impact and root cause analysis capabilities for services running on VNX. The service impact relationships shown in the diagram and described below are automatically added and maintained. These will be included in any services that contain one or more of the discovered components listed above.

Impact Relationships with File components
  • CelerraSystem impact Control Stations, Movers, Storage Pools, Volumes and File Systems
  • Control Station impact Movers
  • Mover impact Interfaces, CIFS Servers, CIFS Shares, NFS Exports
  • Interface impact CIFS Servers
  • Volume impact File Systems, Storage Pools and other types of Volumes
  • Storage Pool impact File Systems
  • File System impact NFS Exports, CIFS Shares
  • CIFS Server impact CIFS Shares

Zenoss Block and File Analytics

This ZenPack provides additional support for Zenoss Analytics. Perform the following steps to install extra reporting resources into Zenoss Analytics after installing the ZenPack.

  1. Copy emc-analytics-bundle.zip for block or file-analytics-bundle for file from $ZENHOME/ZenPacks/ZenPacks.zenoss.EMC.base/ZenPacks/zenoss/EMC/base/analytics/ on your Zenoss server.
  2. Navigate to Zenoss Analytics in your browser.
  3. Login as superuser.
  4. Remove any existing EMC base ZenPack folder.
    1. Choose Repository from the View menu at the top of the page.
    2. Expand Public in the list of folders.
    3. Right-click on EMC base ZenPack folder and choose Delete.
    4. Confirm deletion by clicking OK.
  5. Add the new EMC base ZenPack folder.
    1. Choose Server Settings from the Manage menu at the top of the page.
    2. Choose Import in the left page.
    3. Remove checks from all check boxes.
    4. Click Choose File to import a data file.
    5. Choose the emc-analytics-bundle.zip or file-analytics-bundle file copied from your Zenoss server.
    6. Click Import.

You can now navigate back to the EMC base ZenPack folder in the repository to see the following resources added by the bundle.

Domains

  • EMC Domain (Imported from emc-analytics-bundle.zip for block only)
  • EMC File Domain (Imported from file-analytics-bundle.zip for file only)
Reports
  • (None)
Views
  • EMC_Storage_Pool (Imported from emc-analytics-bundle.zip for block only)
  • EMC_Storage_Pool_Subscribed_Usage (Imported from emc-analytics-bundle.zip for block only)
  • EMC_Storage_Pool_Usage_Bytes (Imported from emc-analytics-bundle.zip for block only)

Domains can be used to create ad hoc views using the following steps.

  1. Choose Ad Hoc View from the Create menu.
  2. Click Domains at the top of the data chooser dialog.
  3. Expand Public then EMC base ZenPack.
  4. Choose the EMC Domain for block or EMC File Domain for file.

Installed Items

Installing this ZenPack will add the following items to your Zenoss system.

Device Classes
  • /Storage/EMC/VNX
  • /Storage/EMC/VMAX
  • /Storage/EMC/VNXFile
Modeler Plugins
  • emc.wbem.VNX
  • emc.wbem.VMAX
  • emc.api.CelerraSystem
Monitoring Templates
  • Array (Storage/EMC)
  • Array (Storage/EMC/VNX)
  • Battery (Storage/EMC)
  • CelerraSystem (Storage/EMC/VNXFile)
  • CIFSShare (Storage/EMC/VNXFile)
  • DataMover (Storage/EMC/VNXFile)
  • Device Storage Pool (Storage/EMC)
  • FileSystem (Storage/EMC/VNXFile)
  • HardDisk (Storage/EMC)
  • Interface (Storage/EMC/VNXFile)
  • LUN (Data Device) (Storage/EMC)
  • NFSExport (Storage/EMC/VNXFile)
  • Power Supply (Storage/EMC)
  • Processor Port (Storage/EMC)
  • Storage Enclosure (Storage/EMC)
  • Storage Enclosure (Storage/EMC/VMAX)
  • StoragePool (Storage/EMC/VNXFile)
  • Storage Processor (Storage/EMC)
  • Unfied Storage Pool (Storage/EMC)
  • Virtual Prov Pool (Storage/EMC)

Usage

Configuring Block or File for Monitoring

To monitor a VNX or VMAX storage device, select the Add EMC Storage option.

Add EMC Storage Dropdown

This will present an Add EMC Device - Block or File window.

Adding Block Storage Devices


Add EMC Storage Window - Block

  • Select VNX (Block) or VMAX (Block) Device Type
  • Enter IP Address of the EMC SMI-S-Provider
  • Enter Port number. Default: 5989
  • If the SMI-S provider supports SSL on the port, the SSL box must be checked.
  • Enter Username and Password credentials of the SMI-S Provider. This may be different from UniSphere
  • Select collector from dropdown.

Alternatively you can use zenbatchload to add VNX/VMAX block storage from the command line. To do this, you must create a file with contents similar to the following. Replace all values in angle brackets with your values minus the brackets. Multiple endpoints can be added under the same /Devices/Storage/EMC/VNX or /Devices/Storage/EMC/VMAX section.

# For VNX (Block Storage) Devices
/Devices/Storage/EMC/VNX
vnx_block setManageIp='<smi-s_address>', zCommandUsername='<smi-s_username>', zCommandPassword='<smi-s_password>'

# For VMAX (Block Storage) Devices
/Devices/Storage/EMC/VMAX
vmax_block setManageIp='<smi-s_address>', zCommandUsername='<smi-s_username>', zCommandPassword='<smi-s_password>'

You can then load the endpoint(s) with the following command.

zenbatchload <filename>

SMI-S Provider Verification

To validate that data is being gathered correctly and that the SMI-S provider is configured correctly you can make direct calls via WBEM command line tools. On Linux you can install several tools via the local package manager to make these types of calls. One such tool is called wbemcli and below is a sample command you can execute to test data retrieval. Zenoss recommends that you **DO NOT** install this on the Zenoss server but onto a separate Linux installation that has a network path to the SMI-S provider.

Sample command:

 wbemcli -dx -noverify ei 'http://:@:5988/root/emc:EMC_VolumeView'

This command will enumerate instances (ei) of the Disk Drive class. Please note, command format depends on installed version of the wbemcli tool. Please check usage documentation.

Adding File Storage Devices


Add EMC Storage Window - File

  • Select VNX (File) Device Type
  • Enter IP Address of the Control Station of the EMC VNX Device
  • Enter Username and Password API credentials of the Control Station (Unisphere credentials)
  • Select collector from dropdown

Alternatively you can use zenbatchload to add VNX file storage from the command line. To do this, you must create a file with contents similar to the following. Replace all values in angle brackets with your values minus the brackets. Multiple endpoints can be added under the same /Devices/Storage/EMC/VNXFile.

# For VNX (File Storage) Devices
/Devices/Storage/EMC/VNXFile
vnx_file setManageIp='<control_station_address>', zCommandUsername='<api_username>', zCommandPassword='<api_password>'

You can then load the endpoint(s) with the following command.

zenbatchload <filename>

Starting the XML API Server

The following section is referenced from EMC API documents available from here: https://community.emc.com/docs/DOC-19626

From EMC® VNX™ for File 7.1 - XML API Programmers Guide (Pg. 180):

To start the XML API server, do the following:

As root, use a text editor to uncomment the following entry in /nas/sys/nas_mcd.cfg:

daemon XML API Server-
       executable /nas/sbin/start_xml_api_server
       optional yes
       canexit yes
       autorestart yes
       ioaccess no

Restart nas services with the following command:

# service nas start

The XML API is now started and is controlled by the master control daemon.

Changes

1.2.1
  • Enable adding VNX File devices from multi-device add wizard (ZPS-1440)
  • Fix "old_data_path" errors on Zenoss 5.2.3 and newer (ZPS-1392)
1.2.0
  • Add support for File Storage via API
  • Fix traceback KeyError: 'Serial Number' in VMAX modeler plugin (ZPS-184/ZPS-186)
  • Fix unit test errors due to vSphere ZenPack version (ZEN-24481)
  • Added addtional LUN Metrics - “Response Time” graph, updated “Data Throughput” and “Operation Throughput” graphs with reading and writing data (ZPS-414)
  • Fix event view from dependency view (ZPS-182)
  • Added migrate to remove duplicated graph threshold for 'Raw Space Usage' graph after the upgrade. (ZPS-179)
  • Fix Reports for block emc-vnx-bundle.zip in Analytics (ZPS-964)
1.1.2
  • Remove dependency on CalculatedPerformance ZenPack. (ZEN-19707)
1.1.1
  • Fix incorrect data device (LUN) impacts. (ZEN-21813)
1.1.0
  • Add DynamicView support for EMC storage dependencies. (ZEN-19844)
  • Fix scaling of time utilization values on LUN and hard disk graphs. (ZEN-20152)
1.0.6
  • Removed dependency on Calculated Performance (ZEN-19707)

Installation

Normal Installation (packaged egg)

  1. Download the appropriate egg file for the version of Zenoss you are running.
  2. Ensure you are logged in as the zenoss user:
    $ sudo su - zenoss
  3. Install the ZenPack:
    $ zenpack --install ZenPacks.zenoss.EMC.base-*.egg
  4. Restart these services:
    $ zenoss restart

Commercial

This ZenPack is developed and supported by Zenoss Inc. Commercial ZenPacks are available to Zenoss commercial customers only. Contact Zenoss to request more information regarding this or any other ZenPacks. Click here to view all available Zenoss Commercial ZenPacks.