Emergency shutdown of services

Control Center monitors each service's short-term storage usage trends and current usage levels. When Control Center predicts that a service is about to exhaust storage space, it initiates an automatic emergency shutdown of the service. By shutting down while enough space is available to perform recovery operations, Control Center minimizes the risk of data corruption.

Emergency shutdown is performed for services that are in a resource pool that has DFS access. Services that are in pools that do not have DFS permissions and do not write to the DFS continue running.

Control Center displays thin pool and DFS information in the following graphs in the browser interface:
  • On the Applications tab, the Thin Pool Usage graph shows used and available bytes for the thin pool.
  • On the Applications page for each service, the DFS Usage graph shows used and available bytes for the DFS.

By comparing usage to the available storage and the amount of space that must be reserved, Control Center determines when a service must be shut down before filling the thin pool or DFS, and initiates the emergency shutdown. The browser interface identifies services in the emergency shutdown state, as does issuing the serviced service status command in the command line.

To minimize data loss, Control Center shuts down services in the following order: databases; services that cannot be recovered; indices and services that are difficult to recover; any other services. Services in emergency shutdown status cannot be restarted until the underlying cause of the shutdown is resolved.

To resolve an emergency shutdown:
  1. Examine the service that was shut down to determine why it was using excessive storage and correct the issue. For example:
  2. Clear the emergency shutdown flags.
  3. Start the service by using the browser interface or command line interface. Control Center starts services in the reverse order of shutdown.