From a03d63841dcf250d20d40ebd5a63dd388e1ebf71 Mon Sep 17 00:00:00 2001 From: "Dustin C. Hatch" Date: Sat, 21 Dec 2024 12:17:45 -0600 Subject: [PATCH] v-m/alerts: Fire paperless email alert after 12h We don't need a notification about paperless not scheduling email tasks every time there is a gap in the metric. This can happen in some innocuous situations like when the pod restarts or if there is a brief disruption of service. Using the `absent_over_time` function with a range vector, we can have the alert fire only if there have been no email tasks scheduled within the last 12 hours. --- victoria-metrics/alerts.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/victoria-metrics/alerts.yml b/victoria-metrics/alerts.yml index 7205472..d8c37ab 100644 --- a/victoria-metrics/alerts.yml +++ b/victoria-metrics/alerts.yml @@ -254,11 +254,11 @@ groups: for details about the task failures. - alert: Paperless email task not running expr: >- - absent( + absent_over_time( flower_events_total{ type="task-started", task="paperless_mail.tasks.process_mail_accounts" - } + }[12h] ) annotations: summary: Paperless task to process mail accounts has not run recently