ZenPacks

Google Cloud Platform ZenPack

The Google Cloud Platform ZenPack adds support for monitoring resources deployed in Google Cloud.

Currently the following services are supported.

  • Compute Engine (GCE)
  • Kubernetes Engine (GKE)

Releases

Version 1.0.0 Download
Released: 2018-06-15
Compatible with Zenoss 5.3 - 6.2
Requires: ZenPackLib ZenPack, PythonCollector ZenPack

Features

The following features are available to help you monitor resources deployed in Google Cloud Platform.

Discovery

The following component types will be automatically discovered through Google Cloud Platform APIs. The components, their properties, and their relationships will be periodically remodeled to provide automatically up-to-date monitoring.

Projects

Each Google Cloud Platform project to be monitored must be added to Zenoss. The resulting Zenoss "device" equates to this project, and all of the resources within that project become components of the project in Zenoss.

Project Overview
Project Overview
Projects
Properties: n/a
Relationships: Regions, Zones, Images, Snapshots, Instance Templates, Kubernetes Clusters, Quotas

All Compute Engine (GCE) components will have the following standard properties.

Properties: Name, Creation Timestamp, Kind, GCE ID, GCE Self-Link

All Kubernetes Engine (GKE) components will have the following standard properties.

Properties: Name, Creation Timestamp, Self-Link
 

Regions

Regions will only be discovered if the project contains instances in the region.

Regions
Regions
Regions
Properties: Description, Status
Relationships: Project, Zones, Instance Groups, Quotas
 

Zones

Zones will only be discovered if the project contains instances in the zone.

Zones
Zones
Zones
Properties: Description, Status
Relationships: Project, Region, Instance Groups, Machine Types, Disk Types, Disks, Instances
 

Images

Images will only be discovered if they're the source image for at least one disk within the project.

Images
Images
Images
Properties: Description, Family, Disk Size, Archive Size, Status
Relationships: Project, Disks
 

Snapshots

Snapshots
Snapshots
Snapshots
Properties: Disk Size, Archive Size, Storage Status, Status
Relationships: Project, Source Disk
 

Instance Templates

Instance templates will only be discovered if they were the template for an instance within the project, or if they're the template for a managed instance group in the project.

Instance Templates
Instance Templates
Instance Templates
Properties: Description, Machine Type, Minimum CPU Platform, Number of Disks, Number of Network Interfaces
Relationships: Project, Instances
 

Quotas

All project-level and region-level quotas with a limit set will be modeled.

Quotas
Quotas
Quotas
Properties: Metric, Usage, Limit
Relationships: Project or Region
 

Instance Groups

Instance groups will only be discovered if they contain at least one instance. This includes multi-zone (region) instance groups, single-zone instance groups, and managed and unmanaged instance groups.

Instance Groups
Instance Groups
Instance Groups
Properties: Description, Managed, Multi-Zone, Size, Target Size, Base Instance
Relationships: Region or Zone, Instance Template, Instances, Kubernetes Node Pool
 

Machine Types

Machine types will only be discovered if used by an instance within the project. Note that it is possible for a machine type of the same name to vary in capabilities from region to region. So you will see multiple machine types of the same name that only vary by their zone.

Machine Types
Machine Types
Machine Types
Properties: Description, Guest CPUs, Shared CPUs, Memory, Maximum Persistent Disks, Maximum Persistent Disk Size, Image Space
Relationships: Zone, Instances
 

Disk Types

Disk types will only be discovered if used by a disk within the project. Note that it is possible for a disk type of the same name to vary in capabilities from region to region. So you will see multiple disk types of the same name that only vary by their zone.

Disk Types
Disk Types
Disk Types
Properties: Description, Default Disk Size, Valid Disk Size
Relationships: Zone, Disks
 

Disks

Disks will only be discovered if they're attached to at least once instance within the project.

Disks
Disks
Disks
Properties: Size, Last Attach Timestamp, Last Detach Timestamp
Relationships: Zone, Disk Type, Source Image, Snapshots, Instances
 

Instances

All instances within the project will be discovered.

Instances
Instances
Instances
Properties: Description, Guest, CPU Platform, Management IP, Network IPs, NAT IPs, IP Forwarding, In Kubernetes, Status
Relationships: Zone, Machine Type, Disks, Network Interfaces, Instance Group, Instance Template
 

Network Interfaces

All instance network interfaces within the project will be discovered.

Network Interfaces
Network Interfaces
Network Interfaces
Properties: Network IP, NAT IPs, Access, Access Type
Relationships: Instance
 

Kubernetes Clusters

All Kubernetes clusters within the project will be discovered.

Kubernetes Clusters
Kubernetes Clusters
Kubernetes Clusters
Properties: Cluster Ipv4 CIDR, Node Ipv4 CIDR Size, Services Ipv4 CIDR, Current Master Version, Initial Cluster Version, Logging service, Monitoring service, Endpoint, Network, Subnetwork, Current Node Count, Status
Relationships: Project, Zones, Node Pools
 

Kubernetes Node Pools

All Kubernetes node pools within the project will be discovered.

Kubernetes Node Pools
Kubernetes Node Pools
Kubernetes Node Pools
Properties: Version, Initial Node Count, Node Disk Size, Node Image Type, Node Service Account, Node Machine Type, Status
Relationships: Cluster, Instance Groups

Status Monitoring

Many of the discovered resources have a status available via the Google Cloud Platform APIs. All of these are monitored by default by this ZenPack.

In addition to the per-resource-type status values listed below, each of these resource types can also have one of the following non-API status values.

  • PENDING: Zenoss has not yet (or recently) collected status for the resource.
  • MISSING: Resource no longer exists in Google Cloud, and will likely be removed the next time modeling occurs.
  • UNKNOWN: An unexpected status value not known to exist for the resource type.
Regions
Status: DOWN, UP
Event: DOWN, MISSING
Zones
Status: DOWN, UP
Event:: DOWN, MISSING
Images
Status: FAILED, PENDING, READY
Event: FAILED
Snapshots
Status: CREATING, DELETING, FAILED, READY, UPLOADING
Storage Status: UNKNOWN, UP_TO_DATE, UPDATING
Event: FAILED
Instances
Status: PROVISIONING, STAGING, RUNNING, STOPPED, SUSPENDING, SUSPENDED, TERMINATED
Kubernetes Clusters
Status: STATUS_UNSPECIFIED, PROVISIONING, RUNNING, RECONCILING, STOPPING, ERROR, DEGRADED
Event: ERROR, DEGRADED
Kubernetes Node Pools
Status: STATUS_UNSPECIFIED, PROVISIONING, RUNNING, RUNNING_WITH_ERROR, RECONCILING, STOPPING, ERROR
Event: RUNNING_WITH_ERROR, DEGRADED

Performance Monitoring

Performance (metric) monitoring is done via the Stackdriver Monitoring API. The following metrics are monitored by default.

Instances

  • CPU Utilization
    • Average: compute.googleapis.com/instance/cpu/utilization
  • Total Disk Throughput
    • Read: compute.googleapis.com/instance/disk/read_bytes_count
    • Write: compute.googleapis.com/instance/disk/write_bytes_count
  • Throttled Disk Throughput
    • Read: compute.googleapis.com/instance/disk/throttled_read_bytes_count
    • Write: compute.googleapis.com/instance/disk/throttled_write_bytes_count
  • Total Disk Operation Rate
    • Read: compute.googleapis.com/instance/disk/read_ops_count
    • Write: compute.googleapis.com/instance/disk/write_ops_count
  • Throttled Disk Operation Rate
    • Read: compute.googleapis.com/instance/disk/throttled_read_ops_count
    • Write: compute.googleapis.com/instance/disk/throttled_write_bytes_count
  • Total Network Throughput
    • Received: compute.googleapis.com/instance/network/received_bytes_count
    • Sent: compute.googleapis.com/instance/network/sent_bytes_count
  • Total Network Packet Rate
    • Received: compute.googleapis.com/instance/network/received_packets_count
    • Sent: compute.googleapis.com/instance/network/sent_packets_count
  • Firewall Throughput
    • Dropped: compute.googleapis.com/firewall/dropped_bytes_count
  • Firewall Packet Rate
    • Dropped: compute.googleapis.com/firewall/dropped_packets_count
  • Container CPU Usage (Kubernetes Nodes Only)
    • CPU Usage: container.googleapis.com/container/cpu/usage_time
  • Container Memory (Kubernetes Nodes Only)
    • Total: container.googleapis.com/container/memory/bytes_total
    • Used: container.googleapis.com/container/memory/bytes_used
  • Container Memory Faults (Kubernetes Nodes Only)
    • Major: container.googleapis.com/container/memory/page_fault_count
    • Minor: container.googleapis.com/container/memory/page_fault_count
Disks
  • Disk Throughput
    • Read: compute.googleapis.com/instance/disk/read_bytes_count
    • Write: compute.googleapis.com/instance/disk/write_bytes_count
  • Throttled Disk Throughput
    • Read: compute.googleapis.com/instance/disk/throttled_read_bytes_count
    • Write: compute.googleapis.com/instance/disk/throttled_write_bytes_count
  • Disk Operation Rate
    • Read: compute.googleapis.com/instance/disk/read_ops_count
    • Write: compute.googleapis.com/instance/disk/write_ops_count
  • Throttled Disk Operation Rate
    • Read: compute.googleapis.com/instance/disk/throttled_read_ops_count
    • Write: compute.googleapis.com/instance/disk/throttled_write_bytes_count

Dynamic View

Project Dynamic View
Project Dynamic View

Dynamic View support is enabled for projects, and all resources that have an impactful relationship to other resources. See the Service Impact and Root Cause Analysis section for more detail on what the specific impact relationships are.

The project-level Dynamic View shows a high-level logical overview of the discovered project's resources instead of the typical (what impacts the project) view because nothing would typically impact the project as a whole.

 

Service Impact and Root Cause Analysis

When combined with the Zenoss Service Dynamics product, this ZenPack adds built-in service impact and root cause analysis capabilities for services running on Google Cloud. The service impact relationships described below are automatically added. These will be included in any services that contain one or more of the explicitly mentioned components.

Impact Diagram
Impact Diagram

Impact Relationships

  • Kubernetes node pools impact their Kubernetes cluster.
  • Instance groups impact any associated Kubernetes node pool.
  • Instances impact their associated guest operating system devices.
  • Instances impact any instance group to which they belong.
  • Network interfaces impact their instance.
  • Disks impact any instance to which they're attached.
  • Zones impact any contained instances.
  • Regions impact all zones in the region.
  • Projects impact all regions.
 

Usage

Creating a Service Account

Adding a Google Cloud Platform Project to Zenoss requires a service account with the Project View role. This role allows Zenoss to view all resource information within the project, but not make any changes.

Use the following steps to setup a service account.

  1. Open the Google Cloud Console.
  2. Choose the project you want to monitor.
  3. Navigate to the IAM & admin service.
  4. Navigate to Service accounts.
  5. Click Create Service Account.
  6. Enter a name for the account.
  7. Choose the Project / Viewer role.
  8. Select Furnish a new private key.
  9. Select the JSON key type.
Be sure to save the downloaded JSON file to a safe place. Open this JSON file in any text editor. You'll be needing the full values from the project_id, client_email, and private_key keys to add the project to Zenoss.

Adding a Project

Use the following steps to add a Google Cloud Platform project to Zenoss.

Add Project
Add Project
  1. Navigate to the Infrastructure page.
  2. Click the Add Devices menu button above the devices grid.
  3. Choose the Add Google Cloud Project... menu option.
  4. Complete the form..
    • Device Name: Any valid Zenoss device name.
    • Project ID: project_id from the service account's JSON file.
    • Client Email Address: client_email from the service account's JSON file.
    • Private Key: private_key from the service account's JSON file.
    • Guest Devices Use External IP: Select this if you plan to monitor the operating systems running on instances via their NAT IP addresses. Deselect it if you will monitor them via their internal Google IP addresses.
    • Collector: The Zenoss collector responsible for monitoring the project.
  5. Click Add.
 

Alternatively you can use zenbatchload to add projects from the command line. To do this, you must create a file with contents similar to the following. Replace all values in angle brackets with your values minus the brackets. Multiple projects can be added under the same /Devices/GoogleCloudPlatform section.

/Devices/GoogleCloudPlatform
my-project-3 \
    zGoogleCloudPlatformProjectId='<project_id>', \
    zGoogleCloudPlatformClientEmail='<client_email>', \
    zGoogleCloudPlatformPrivateKey='<private_key>'

You can then load the endpoint(s) with the following command.

zenbatchload <filename>

Limitations

This ZenPack has the following known limitations.

Adding Project

Adding using the Add Multiple Devices or Add Multiple Devices requires that an IP address or resolvable name be entered for the device. Google Cloud Platform projects don't have an IP or resolvable name, so something arbitrary such as cloud.google.com must be used. It is recommended that the Add Google Cloud Project menu item be used to add projects instead because it doesn't require a resolvable name be entered.

Model Accuracy

Continuous modeling is not currently supported by this ZenPack. This means that the model seen in Zenoss may be up to 12 hours old because it is only updated every 12 hours by default.

Installed Items

This ZenPack installs the following items into the Zenoss system.

Configuration Properties

  • zGoogleCloudPlatformProjectId: Project ID to monitor. (project_id)
  • zGoogleCloudPlatformClientEmail: Client email address to use for monitoring. (client_email)
  • zGoogleCloudPlatformPrivateKey: Private key to use for monitoring. (private_key)
  • zGoogleCloudPlatformGuestUseExternalIP: Use external (NAT) IP for creating and linking guest devices.
  • zGoogleCloudPlatformStackDriverMonitoringInterval: How often (in seconds) to poll for Stackdriver Monitoring datasources.
  • zGoogleCloudPlatformQuotaInterval: How often (in seconds) to poll for quota datasources.
  • zGoogleCloudPlatformStatusInterval: How often (in seconds) to poll for status datasources.
  • zGoogleCloudPlatformActivityLogSubscriptionName: Subscription name that will be used to pull activity logs.
  • zGoogleCloudPlatformActivityLogInterval: How often (in seconds) to poll for activity logs.
  • zGoogleCloudPlatformComputeMaxResults: Maximum number of results per page for Compute Engine API requests.
  • zGoogleCloudPlatformMonitoringPageSize: Maximum number of points per page for Monitoring API requests.

Device Classes

  • /GoogleCloudPlatform: All Google Cloud Platform projects are added to this device class.

Modeler Plugins

  • GoogleCloudPlatform.Project: Models all components for a Google Cloud Platform project.

Datasource Types

  • Google Cloud Platform Activity Log Monitoring
  • Google Cloud Platform Quota
  • Google Cloud Platform Stackdriver Monitoring
  • Google Cloud Platform Status

Monitoring Templates

  • /GoogleCloudPlatform
    • ProjectDevice
    • ComputeRegion
    • ComputeImage
    • ComputeSnapshot
    • ComputeZone
    • ComputeInstance
    • KubernetesNode
    • ComputeDisk
    • KubernetesCluster
    • KubernetesNodePool

Event Classes

  • /GoolgleCloudPlatform

Event Class Mappings

  • /Status
    • gcp-status-api
    • gcp-status-region
    • gcp-status-image
    • gcp-status-snapshot
    • gcp-status-snapshot-storage
    • gcp-status-zone
    • gcp-status-instance
    • gcp-status-kubernetes-cluster
    • gcp-status-kubernetes-nodePool

Changes

1.0.0

  • Initial release.
Commercial

This ZenPack is developed and supported by Zenoss Inc. Commercial ZenPacks are available to Zenoss commercial customers only. Contact Zenoss to request more information regarding this or any other ZenPacks. Click here to view all available Zenoss Commercial ZenPacks.