Upgrading an Analytics 5.0.x instance

Patching Analytics 5.x, that is upgrading from Analytics 5.0.x to Analytics 5.0.y where y > x is a straightforward process requiring very little downtime (approximately 15 minutes downtime for Analytics/Resource Manager). There are no data warehouse "schema" changes affecting your data with this upgrade, so any existing reports are guaranteed to work without modification post-upgrade, which is done "in-place". If you are upgrading from Analytics 4.4.0, please skip this chapter and proceed to the following section, Upgrading to Analytics 5.x from Analytics 4.x

To perform an upgrade to Analytics 5.0.y from an earlier version of Analytics 5.0.x, perform the following process:
  1. Ensure that the /tmp directory has 50G of space or adjust your /etc/my.conf tmpdir setting appropriately.
  2. In the Control Center UI, stop the following services (found in Analytics ETL group):
    • zenmodeletl
    • zeneventetl
    • zenperfetl
  3. Stop the Analytics service by logging in to the Analytics server as the root user and enter the commands:
    sudo -s 
    service zenoss_analytics stop
  4. On the Analytics server, upgrade the RPM for Analytics.
    rpm -Uvh zenoss_analytics-5.0.y-<version>.noarch.rpm
    The output of this process suggests running the "/opt/zenoss_analytics/bin/upgrade_db.py" process. This message should be ignored for this type of upgrade.
  5. If you are upgrading to Analytics 5.0.5 or later, skip this step. If you are upgrading to Analytics 5.0.3 or an earlier 5.0.x version and you have previously connected Analytics to one or more Resource Manager 5.x installations, execute the following command as root on the Analytics server to work around a known issue (ZEN-24476) concerning renaming performance extractors prior to first service start:
    mysql -u root reporting -e
      "update meta_extractor set extractor_name =
        CONCAT(extractor_name,'/0'
      where extractor_type = 'PERFORMANCE'
      and zenoss_instance_key in
        (select zenoss_instance_key from dim_zenoss_instance
        where query_service_account is not null);"
  6. Start the Analytics service again:
    service zenoss_analytics start
  7. Immediately tail the Analytics application log (a Tomcat log) and watch it as Analytics starts up for the first time post upgrade.
    tail -F /opt/zenoss_analytics/logs/catalina.out
    On first startup post upgrade, the data warehouse schema will be adjusted as/if necessary. This process usually takes about 5 minutes, during which the Analytics application will be unavailable. Success of the process is confirmed by the following message in this log and you should wait for it to appear before proceeding further:
    INFO: Server successfully started in <time in ms>

    This completes the Analytics server upgrade.

  8. Upgrade the client side, by upgrading the ZenETL ZenPack following the normal procedure for installing a ZenPack and then restart services.

    For Resource Manager 4.x, perform the following:

    1. Ensure you are logged in as the zenoss user.
      sudo su - zenoss
    2. Install the ZenETL ZenPack:
      zenpack --install ZenPacks.zenoss.ZenETL<version>.py2.7.egg
    3. Stop Zenoss:
      zenoss stop
    4. Start Zenoss:
      zenoss start
    5. Update all hubs and collectors to push the new ZenPack code out using dc-admin.

    For Resource Manager 5.x, perform the following:

    1. Copy the ZenETL ZenPack egg file to a local directory on the Control Center master host, e.g., /tmp/zenpacks.
    2. Change the file permissions.
      chmod -R 777 /tmp/zenpacks
    3. Change directory to the directory in which the ZenPack egg file is located.
      cd /tmp/zenpacks
    4. Install the ZenETL ZenPack.
      serviced service run zope zenpack-manager install ZenPacks.zenoss.ZenETL<version>.py2.7.egg
    5. To speed up performance data extractions in Resource Manager 5.2.x or later (or any Resource Manager instance that has OpenTSDB v2.2 or later installed), the following change should be made. The configuration option, tsd.query.skip_unresolved_tags should be set to True. This option can be set in the /opt/zenoss/etc/opentsdb/opentsdb.conf file that is editable in the "opentsdb reader" service in Control Center.
    6. Stop and then start Resource Manager.
      serviced service stop Zenoss.resmgr
      # Wait for containers to stop
      serviced service start Zenoss.resmgr

      This stopping and starting is necessary to initialize the zenmodeletl, zeneventetl, and zenperfetl services.