Configuring and Managing Alarms

By default, Exoprise automatically configures a set of alarms that the engineers consider to be critical for the sensor. When deploying a new sensor, you will be presented with the Alarm Setup section. Below is an explanation of each field.

Setup Recommended Alarms – This field will allow you to specify how long the sensor runs prior to the Alarms being configured. Once the sensor has ran for the specified times, the alarms will be generated based on the average of that time frame.

Alarm Types to Create – This field allows you to configure the Alarms to trigger on everything, only errors, or only performance issues.

Alarm Recipient Group – You can select who will be receiving the Alarms via EMail or SMS

Disable Sensor After – Here you can specify how long you want the sensor to run in an errored state prior to disabling itself. If a sensor disables itself, it can be manually enabled or the system will automatically re-enable the sensor after 24 hours.

You can add additional alarms, aggregate them together, or disable alarms for any sensor or sensor type with the following instructions:

Add Alarm

1. Go to the Add Alarm page
2. At the top, choose the type of sensor you want to create an alarm for.
3. Choose an element for that particular sensor (click the choose button). Each sensor exposes different metrics, sometimes referred to as elements.
4. When you choose an element and a sensor type, the table of matching, deployed sensors (below) is updated with recent values for each location where a sensor has been deployed. 8 hours worth of aggregate history (Min, Max, Avg) is displayed.
5. Using the values from the table below, choose a good comparison threshold value for your alarm. Typically, +/- 10% of the maximum value is a good place to start if the monitored service has been operating well and depending on the sensitivity of the alarm that is desired.
6. Update the recipients for the alarm. Different Alarm Recipient Groups can be configured.
7. Optionally create or choose Email hooks or Webhooks for the alarm
8. Check each of the sensors you would like to apply the alarm to in the table below.
9. Click ‘Create Alarm’ to finish adding the alarm. Or click ‘Update Alarm’ if you are editing an alarm.

Threshold Values

Ideal threshold values may be different for each of the different sites (locations) where you have deployed sensors. The best practice when configuring an element is to look below in the sensors table at Minimum, Maximum and Average values for each element. When you choose a different element, the table is updated below for each site. Choosing a value that is slightly (10% or similar) greater than the maximum is recommended if the service you are monitoring is operating correctly.

Operational Elements

Most sensors have two (2) operational elements that can be monitored:

• operations.SensorError
  This is for tracking sensor errors throughout the system and sensor.
• operations.SensorOffSchedule
  If the sensor stops running or the sensor is unable to communicate with Exoprise servers, this alarm will fire and notify the recipients.

Email Subject Template

The subject of alert emails can be formatted by including options from the following list surrounded by ‘$’:

• sensor_type
• element
• visible_element
• element_type
• operator
• operator_value
• consecutive_periods
• period
• affected_percent
• affected_threshold
• location
• name
• title
• ipaddress
• sensor_interval

For example, an email subject might be configured like this:

A $sensor_type$ sensor alarm has fired after $consecutive_periods$ periods.

Links in the Email Subject Template

We recently enhanced the alarm email subjects to include support for links. You can now substitute links to either the sensor or the alarm in the email subject:

• alarm_link
• sensor_link

For example, an email subject might be configured like this:

A $sensor_type$ has fired: $alarm_link$

Aggregate Alarms

Aggregate alarms are a very powerful mechanism for being notified when cloud or application conditions are affecting multiple different locations or reducing alarm generation for what might be transient network or application conditions.

Aggregate alarms generate alerts or notifications only when the condition or threshold is affecting multiple CloudReady synthetic sensors.

Example

For example, let’s say you deploy 10 Email sensors across 10 different branch offices. You can configure an alarm for a condition across all 10 sensors and set the aggregate percentage to 30%. The alarm will only be generated when 4 or more sensors are exceeding the threshold or generating an error.

This can be very helpful for knowing when there is a centralized cloud service error condition or performance slow down.

Configure Aggregate Alarms

Aggregate alarms are not automatically configured as sensors are deployed. We recommend first removing the existing alarms for the condition that you will be configuring and then setting up the aggregate alarm.

1. Go to the Alarm Dashboard
2. Filter by sensor type. Aggregate alarms can only be configured for the same sensor type. Choose the sensor type and click Filter in the upper right of the alarm dashboard.
3. Remove the existing alarm for the condition that you would like to configure across the sensors that you will be aggregating the alarm. Select the different alarms, click the Action dropdown, then Delete then click apply.
4. Go to Add Alarm from the left-hand menu
5. Select the sensor type from the drop-down list
6. Choose the element or condition that you will be alarming on. Configure the threshold, consecutive periods as you would for a normal alarm creation
7. Choose the percentage of sensors that you want to “vote” on the alarm condition. If you have 10 sensors and you only want to know about a problem when half of them are breaching their threshold or experiencing the condition, then choose 50%
8. Configure the recipient group, name, and notes for the alarm
9. Finally, in the bottom table, choose all of the sensors that the alarm will apply to. And click ‘Create Alarm’ at the bottom

This creates an aggregate alarm for an error condition or performance threshold applied to all of the sensors of a certain type. The alarm will only be triggered when the condition is breached across half or more of the sensors.

Disable Alarms

1. Go to the Alarm Dashboard under Alarms on the left-hand menu.
2. Using the filters in the top right, select the type of Sensor you want to disable the alarm for.
3. Select the Alarms by checking the checkbox on the left.
4. Under the Actions dropdown in the top left, select “Disable”.
5. If disabling only one Alarm, you can also uncheck the checkbox on the far right “Enabled”