What are recovery thresholds?
Recovery thresholds are additional thresholds added to a monitor that indicates an additional condition to a monitor’s recovery from alert or warning states.
When you set up a threshold metric monitor, you get alerted when a metric passes the alert threshold.
The recovery threshold adds a condition to the monitor’s recovery such that it only enters recovered state once it has passed the recovery threshold. If no recovery threshold is set, the monitor will recover whenever the alert conditions are no longer met.
Note: Your metric value needs to be strictly below/above the recovery threshold for the monitor to recover.
Recovery thresholds reduce noise in cases of a flapping monitor, and increase your confidence that upon recovery, the alerting metric has recovered and the issue has been resolved.
The following table shows you when a monitor recovers when you set a recovery threshold:
|Alert condition||Recovery condition|
|> alert threshold||<= alert recovery threshold|
|>= alert threshold||< alert recovery threshold|
|< alert threshold||>= alert recovery threshold|
|<= alert threshold||> alert recovery threshold|
How to set up recovery thresholds?
Via the UI
You can set up thresholds for alert and warning recovery:
Via the API
When you create/edit a monitor (Monitor API doc) use the attributes critical_recovery and warning_recovery attributes in the options.thresholds attribute of your JSON monitor: