---
title: "Prometheus Alertmanager Phone Call Alerts: Wake Up for Critical Only"
description: "Alertmanager has no voice receiver. Route Prometheus alerts to a phone call for critical severity only: webhook config, conditions, and the Watchdog trap."
date: 2026-09-04
author: Nooc
authorAvatarLink: /images/avatars/nooc.webp
authorLink: https://nooc.me
tags:
  - Prometheus
  - Alertmanager
  - phone call alerts
  - webhook notifications
  - Kubernetes
  - on-call
---

# Prometheus Alertmanager Phone Call Alerts: Wake Up for Critical Only

Alertmanager has no voice receiver. To get a phone call when a Prometheus alert fires, add a `webhook_configs` receiver pointing at an Echobell channel whose subscription type is **Calling**. This guide covers the exact YAML, the condition that stops resolved alerts from calling you, severity routing, and the Watchdog alert that will otherwise ring your phone every four hours forever.

Prometheus is the default metrics stack for most infrastructure built in the last decade, and [Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/) is genuinely good at the hard parts: deduplicating alerts, grouping them, silencing them during maintenance, and inhibiting the downstream noise when an upstream dependency fails.

What it will not do is wake anybody up.

## Why Alertmanager cannot ring your phone on its own

Alertmanager ships receivers for email, Slack, PagerDuty, OpsGenie, Discord, Telegram, Pushover, Webex, MS Teams, and a dozen more. Every one of them delivers a *message*, and messages are subject to the ringer switch, Do Not Disturb, and iOS Focus Modes. At 03:00 that means the alert arrives and nothing happens.

There is no `voice_configs`. The options people usually land on are:

- **PagerDuty / OpsGenie / Splunk On-Call** — these do place calls, and they are full incident-management platforms with per-seat pricing to match. The right answer if you need rotations and escalation trees; heavy if you need a phone to ring. (OpsGenie in particular is [winding down](/en/blog/opsgenie-end-of-life-alternatives), which is why so many teams are re-evaluating this layer right now.)
- **SMS bridges** like [Sachet](https://github.com/messagebird/sachet) — you run another service, you pay a gateway per message, and an SMS still lands as a message. On iOS a text does not break Focus Mode unless the sender is on your allow list.
- **Twilio-based glue** — write a small webhook receiver, buy a number, pay per call, and now you own a piece of production infrastructure whose only job is to make a phone ring.

The generic **webhook receiver** is the escape hatch. It posts a documented JSON payload to any URL, which is all you need.

## What you need

- A running Prometheus + Alertmanager setup, and access to edit `alertmanager.yml`
- Echobell installed ([App Store](https://apps.apple.com/app/apple-store/id6743597198?pt=128151925&ct=blog-alertmanager-phone-call-alerts-en&mt=8) / [Google Play](https://play.google.com/store/apps/details?id=one.echobell.echobellandroid))
- Ten minutes

This guide was written against Alertmanager 0.31. The webhook payload has been at `version: "4"` for years, so 0.2x releases behave identically.

Your Alertmanager needs outbound HTTPS to `hook.echobell.one`. It does **not** need to be reachable from the internet, so an Alertmanager inside a cluster, a VPC, or a homelab works fine.

## Step 1 — Create a channel that calls you

In Echobell, create a channel named something like `Prometheus Critical`. Set its subscription notification type to **Calling**. This is the setting that matters: Calling alerts arrive as an incoming call screen and ring through iOS Focus Mode and Do Not Disturb, which a push notification does not.

Set the templates to read the Alertmanager payload directly:

```
Title: 🔴 {{commonLabels.alertname}} on {{commonLabels.instance}}
Body: {{commonAnnotations.summary}}
{{commonAnnotations.description}}
```

And, under Advanced Settings, a **Link Template** so the notification record jumps straight to the graph:

```
{{alerts[0].generatorURL}}
```

Then copy the channel's **Webhook URL**:

```
https://hook.echobell.one/t/<channel-token>
```

Treat that URL as a secret — anyone holding it can make your phone ring.

## Step 2 — Add a webhook receiver

In `alertmanager.yml`:

```yaml
route:
  group_by: ["alertname", "cluster", "service"]
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  receiver: echobell-critical

receivers:
  - name: echobell-critical
    webhook_configs:
      - url: "https://hook.echobell.one/t/<channel-token>"
        send_resolved: false
```

Reload with `curl -X POST http://localhost:9093/-/reload` or `SIGHUP`.

Note `send_resolved: false`. **The default for a webhook receiver is `true`**, unlike most other Alertmanager receivers, so leaving it out means your phone rings when the service breaks *and* rings again when it fixes itself. The second call is the one that teaches people to ignore the first. Step 4 shows how to get the recovery notice back without the ring.

## Step 3 — Understand what actually arrives

Alertmanager groups alerts, then POSTs one payload per group:

```json
{
  "version": "4",
  "groupKey": "{}:{alertname=\"HighErrorRate\"}",
  "truncatedAlerts": 0,
  "status": "firing",
  "receiver": "echobell-critical",
  "groupLabels": { "alertname": "HighErrorRate" },
  "commonLabels": { "alertname": "HighErrorRate", "severity": "critical" },
  "commonAnnotations": { "summary": "Error rate above 5% for 10m" },
  "externalURL": "http://alertmanager.internal:9093",
  "alerts": [
    {
      "status": "firing",
      "labels": { "alertname": "HighErrorRate", "instance": "api-7d9f:8080" },
      "annotations": { "summary": "Error rate above 5% for 10m" },
      "startsAt": "2026-09-04T02:41:07.351Z",
      "endsAt": "0001-01-01T00:00:00Z",
      "generatorURL": "http://prometheus:9090/graph?g0.expr=...",
      "fingerprint": "a1b2c3d4e5f60718"
    }
  ]
}
```

Echobell reads the JSON body as-is, so every one of those fields is available in templates and conditions. Nested access works with either syntax — `{{commonLabels.severity}}` or `{{alerts[0].labels["instance"]}}`.

Two properties of this payload drive everything below:

**The top-level `status` is `firing` if *any* alert in the group is firing.** It only becomes `resolved` once every alert in the group has resolved. That makes it a clean thing to filter on.

**`commonLabels` holds only the labels shared by every alert in the group.** This is the single most common surprise. If `group_by` is broad enough that one webhook carries `HighErrorRate` on three different instances, then `commonLabels.instance` is absent and `{{commonLabels.instance}}` renders as an empty string. There is more on how to handle that below.

## Step 4 — Send recoveries as a quiet push

You still want to know when something recovers — you just do not want to be called about it. Add a second Echobell channel named `Prometheus Recovered`, set its notification type to **Normal**, give it these templates:

```
Title: ✅ {{commonLabels.alertname}} resolved
Body: {{commonAnnotations.summary}}
```

and, under Advanced Settings, this **condition**:

```
status == "resolved"
```

Conditions are expressions evaluated before anything is delivered. If the expression is false, Echobell accepts the request and sends nothing.

Then point the same receiver at both channels — one receiver can hold several `webhook_configs`:

```yaml
receivers:
  - name: echobell-critical
    webhook_configs:
      # Rings the phone. Firing only.
      - url: "https://hook.echobell.one/t/<calling-channel-token>"
        send_resolved: false
      # Quiet push. The channel condition drops the firing half.
      - url: "https://hook.echobell.one/t/<recovery-channel-token>"
        send_resolved: true
```

The recovery channel receives both firing and resolved payloads and discards the firing ones. The result: downtime rings, recovery arrives as a push you read in the morning.

## Step 5 — Route by severity, not by everything

A catch-all route that sends every alert to a calling channel is a machine for producing ignored phone calls. Split by severity in Alertmanager, where the routing tree belongs:

```yaml
route:
  group_by: ["alertname", "cluster", "service"]
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  receiver: echobell-warning

  routes:
    # Watchdog never reaches a human. See Step 6.
    - matchers:
        - alertname = "Watchdog"
      receiver: "null"

    - matchers:
        - severity = "critical"
      receiver: echobell-critical
      group_wait: 10s
      repeat_interval: 1h

receivers:
  - name: "null"

  - name: echobell-critical
    webhook_configs:
      - url: "https://hook.echobell.one/t/<calling-channel-token>"
        send_resolved: false

  - name: echobell-warning
    webhook_configs:
      - url: "https://hook.echobell.one/t/<normal-channel-token>"
        send_resolved: true
```

Routes are evaluated top to bottom and the **first match wins** — `continue` defaults to `false`. So order matters: the `Watchdog` route has to sit above anything that would otherwise swallow it.

If you would rather keep one channel and filter on the Echobell side, the equivalent condition is:

```
status == "firing" && commonLabels.severity == "critical"
```

Doing it in Alertmanager is usually better, because `severity` then also drives `group_wait` and `repeat_interval`. Doing it in Echobell is better when you cannot get a config change merged today.

## Step 6 — The Watchdog trap

If you run [kube-prometheus-stack](https://github.com/prometheus-operator/kube-prometheus), you have an alert called `Watchdog` whose expression is `vector(1)`. It is *designed* to fire forever — it exists so that an external system can notice when Prometheus itself has stopped. The default config routes it to a `null` receiver.

Point a catch-all route at a calling channel without excluding it and Watchdog will call your phone every `repeat_interval`, forever, starting immediately. This is the number-one way people conclude that phone alerts "do not work."

Keep the `null` route from Step 5. Then, optionally, do the useful thing with it: turn Watchdog into a real dead man's switch.

```yaml
    - matchers:
        - alertname = "Watchdog"
      receiver: deadmansswitch
      group_wait: 0s
      group_interval: 1m
      repeat_interval: 50s

receivers:
  - name: deadmansswitch
    webhook_configs:
      - url: "https://hc-ping.com/<your-check-uuid>"
        send_resolved: false
```

Echobell cannot be the dead man's switch itself — it alerts when a request *arrives*, not when one stops arriving. So send the Watchdog ping to a service built for silence detection ([Healthchecks.io](https://healthchecks.io), Cronitor, Dead Man's Snitch), and then point *that* service's "check went down" webhook at your Echobell calling channel. Now a phone call means "monitoring itself is dead," which is the one alert you most want to be woken by and the one nobody configures.

## Tuning so it does not cry wolf

Three Alertmanager settings do most of the work, and one Prometheus one:

| Setting | Where | What it does |
| --- | --- | --- |
| `for:` | Alert rule | How long the condition must hold before it fires at all. Your first line of defence against a two-second blip. |
| `group_wait` | Route | How long to wait for more alerts before the first notification. 30s default; drop to `10s` for critical. |
| `group_interval` | Route | Minimum gap before a notification about *new* alerts in an existing group. Default 5m. |
| `repeat_interval` | Route | How often an unresolved alert re-notifies. **Default 4h** — so a night-time outage calls you at 03:00 and again at 07:00. |

`repeat_interval` is the one worth thinking about. Four hours is a long time to leave something broken; twenty minutes is a machine for making you disable the channel. One hour on critical is a reasonable starting point.

If you want an unanswered call to retry immediately rather than waiting for the next `repeat_interval`, turn on **Retry Failed Call** in Echobell's app settings.

## Only ring outside working hours

During the workday you are probably already looking at a dashboard. Echobell's system time variables (all UTC) let a channel behave differently by hour, without a second Alertmanager route:

```
status == "firing" && (hour >= 17 || hour < 9)
```

That calls you only outside 09:00–17:00 UTC. Point a second, Normal-type channel at the inverse for daytime pushes:

```
status == "firing" && hour >= 9 && hour < 17
```

Add `dayOfWeek >= 1 && dayOfWeek <= 5` to treat weekends as out-of-hours too. Remember these are always computed in UTC — offset for your own timezone. There is a fuller treatment in [time window notifications using UTC conditions](/en/blog/time-window-notifications-using-utc-conditions).

## Handling the empty `commonLabels` problem

When a group contains alerts from several instances, `commonLabels.instance` disappears and your notification title reads `🔴 HighErrorRate on `.

Three ways out, in order of preference:

1. **Put the label in `group_by`.** If `group_by` includes `instance`, then every alert in a group shares it and `commonLabels.instance` is always present. The cost is more notifications — one per instance instead of one per alert name.
2. **Read the first alert instead.** `{{alerts[0].labels.instance}}` is always populated. It is only one of possibly many, so pair it with a count: `{{alerts[0].labels.instance}} (+{{alerts.length}} alerts)`.
3. **Design the label so empty still reads.** Echobell has no default-value operator — `{{a || "unknown"}}` renders the literal text `true`, not a fallback — so write `Instance: {{commonLabels.instance}}` on its own line, where a blank value is obviously blank rather than a broken sentence.

## Keeping the payload small

A group covering a hundred pods produces a large JSON body, and Echobell rejects trigger bodies over 1 MiB with HTTP 413. Cap it in Alertmanager:

```yaml
      - url: "https://hook.echobell.one/t/<channel-token>"
        send_resolved: false
        max_alerts: 20
```

Alertmanager then sends at most twenty alerts and sets `truncatedAlerts` to the number it dropped, which you can surface in the body:

```
Body: {{commonAnnotations.summary}}
Alerts: {{alerts.length}} (+{{truncatedAlerts}} truncated)
```

## Sharing the alert with your team

An Echobell channel can be shared via a subscription link, and every subscriber picks their own notification type. The same route can therefore ring the on-call engineer while landing as a normal push for everyone else — no per-seat pricing, and no extra routing rules in Alertmanager.

It also fits the reason many teams self-host Prometheus in the first place: your metrics and alert rules stay on your infrastructure, and Echobell keeps notification content and history on the device rather than on its servers.

## What this setup does not give you

Being honest about the boundary saves you a bad migration later. Echobell is a delivery layer, not an incident-management platform. It has no:

- On-call rotation schedules or follow-the-sun handoffs
- Escalation trees that page a second person when the first does not answer
- Incident timelines, acknowledgement tracking, or postmortem tooling

If your team needs those, you need PagerDuty, Grafana Cloud IRM, or similar. What this covers is the specific gap Alertmanager leaves open: converting a firing alert into a phone that actually rings. For solo operators, small teams, and homelabs, that is usually the entire requirement.

## Troubleshooting

**Nothing arrives at all.** Check Alertmanager's own logs first (`level=error component=dispatcher`), then confirm the route actually resolves to your receiver — `amtool config routes test severity=critical alertname=HighErrorRate` tells you which receiver an alert would land in without waiting for one to fire.

**Echobell returns HTTP 404.** The channel token is wrong or the channel was deleted. An unknown token is a 404, not a silent success.

**Echobell returns 200 with `"notificationTriggered": false`.** Your condition evaluated to false. The response body also carries `"conditionsMet": false`, which is the fastest way to tell "my condition is wrong" apart from "my webhook never arrived." Check `status == "firing"` against what Alertmanager actually sent — the top-level status, not `alerts[0].status`.

**HTTP 413.** The payload exceeded 1 MiB. Set `max_alerts` as above.

**HTTP 405.** The channel has **POST Only** enabled and something sent a GET. Alertmanager posts, so this usually means you tested the URL in a browser.

**The title has an empty gap in it.** `commonLabels` did not contain that label for this group. See the section above.

**Nothing rings, but the notification arrives.** The subscription's notification type is Normal or Time Sensitive, not Calling. Notification type is chosen per subscriber, so check it on the device that is not ringing.

**Testing without breaking production.** Add a rule with `expr: vector(1)`, a distinct `alertname`, and `severity: critical`, let it fire once, then delete it. Or fire one by hand:

```bash
curl -X POST http://localhost:9093/api/v2/alerts -H 'Content-Type: application/json' -d '[
  {"labels":{"alertname":"EchobellTest","severity":"critical"},
   "annotations":{"summary":"Testing the phone call path"}}
]'
```

## Frequently asked questions

### Can Prometheus Alertmanager make a phone call natively?

No. Alertmanager has receivers for email, Slack, PagerDuty, OpsGenie and many others, but there is no voice or SMS receiver. Phone calls require routing the generic webhook receiver to a service that can place one, such as Echobell, or paying for an incident-management platform.

### Will the phone call bypass Do Not Disturb?

Yes. Echobell's Calling notification type presents as an incoming call, which rings through iOS Focus Mode and Do Not Disturb. See [bypassing iOS Focus Mode for critical alerts](/en/blog/how-to-bypass-ios-focus-mode-for-critical-alerts) for the details and the settings involved.

### Does this work with Alertmanager behind a firewall or inside Kubernetes?

Yes. The webhook is an outbound HTTPS request from Alertmanager, so it only needs to reach `hook.echobell.one`. Your Alertmanager does not need a public address or an ingress.

### How do I stop getting called when an alert resolves?

Set `send_resolved: false` on the webhook config that points at your calling channel. The webhook receiver defaults to `true`, unlike most other Alertmanager receivers, so this is opt-out rather than opt-in. To still receive recoveries quietly, add a second channel with the condition `status == "resolved"`.

### Why does my phone ring every four hours for the same alert?

That is `repeat_interval`, which defaults to `4h`. Alertmanager re-notifies about a still-firing alert on that cadence. Set it per route — `1h` on critical is a common choice. If the calls started immediately after you added a catch-all route, the culprit is more likely the always-firing `Watchdog` alert; see Step 6.

### Can several people be called for the same alert?

Yes. Share the channel with your teammates and each subscriber chooses their own notification type. Everyone subscribed to the calling channel gets rung, at no per-seat cost.

### Should I filter severity in Alertmanager or in Echobell conditions?

Prefer Alertmanager: routing there also lets you set `group_wait` and `repeat_interval` per severity, and the routing tree stays in version control with the rest of your config. Use Echobell conditions when you cannot change the Alertmanager config, or for filters Alertmanager has no concept of — such as time-of-day.

## Wrap-up

The setup is one receiver, one `send_resolved: false`, and a routing tree that keeps everything except `severity: critical` away from the calling channel. It leaves your alert rules, grouping, silences, and inhibition exactly as they are, and it closes the gap between "Prometheus noticed" and "a human noticed."

[Download Echobell for iPhone](https://apps.apple.com/app/apple-store/id6743597198?pt=128151925&ct=blog-alertmanager-phone-call-alerts-en&mt=8) or [get it on Google Play](https://play.google.com/store/apps/details?id=one.echobell.echobellandroid), then fire the `EchobellTest` alert above before you rely on the path for anything real.

---

## Related

- [Prometheus integration docs](/en/docs/developer/prometheus)
- [Channel conditions reference](/en/docs/conditions)
- [Grafana call notifications](/en/blog/grafana-call-notification)
- [Uptime Kuma phone call alerts](/en/blog/uptime-kuma-phone-call-alerts)
- [A developer's guide to fixing alert fatigue](/en/blog/fix-alert-fatigue-developer-guide)