Analytics architecture

Analytics combines extract, transform, and load (ETL) procedures with a data warehouse and reporting tools to process report data.

Analytics creates two databases:
  • zenoss_analytics (Analytics database): Stores report design, execution information, and the results of the reports, including PDFs. You can also access previous reports stored here.
  • reporting: Warehouses all data retrieved from Resource Manager. Also referred to as the "etl database".
Analytics uses the following functionality:
  • Jaspersoft
    • A robust, ad-hoc reporting feature to facilitate self-management
    • A built-in report scheduler
    • A wide range of report export formats, including PDF, CSV, Excel, Word, HTML, and JPG.
    • Analytics reports and features are accessed through an integrated interface. Reports are distributed by email.
  • Resource Manager
    • Out-of-the-box (OOTB) example domains, views, and "reports"
    • Additional capabilities installable from ZenPack Analytics bundles
  • Data Warehouse
    • A data warehouse used for enterprise reporting. The data warehouse schema is driven by meta data provided by the ZenETL ZenPack, providing a flexible platform for report creation.
    • The following three data sources are used:
      • Zope object database (ZODB)

      • MariaDB event database (EventDB)

      • Centralized performance data storage (HBase cluster)

ETL is not a continuous process; it extracts and processes data periodically (according to user-defined parameters) in batches. Model, event, and performance data are independently extracted.

As part of Resource Manager, three daemons run to export data from its sources of data to the Analytics server:

  • zeneventetl - Extracts event information from ZEP, which is then sent to the Analytics server for subsequent transform and load. There is no necessary configuration to be done. All event information is automatically ETLed to the data warehouse.

  • zenmodeletl - Extracts ZODB model information, which is then sent to the Analytics server for subsequent transform and load. There is no necessary configuration to be done. All event information is automatically ETLed to the data warehouse.

  • zenperfetl - Performance data for the subset of data points with aliases is sent to the Analytics server for subsequent transform and load. This daemon also runs on each collector. Typically, this is run on the Resource Manager server and in the collector instances in the various resource pools.

All data points are available for performance data extraction. You can select which data points are extracted by assigning them an alias.

Figure 1. Analytics components