Integrations and alert intake
Enterprise feature (1.19.0,
helpdesk). See the Helpdesk overview.
Monitoring tools, scripts and bots can open tickets of their own. Desk settings → Integrations (Helpdesk settings or Email channel) adds an integration for each tool. Its tickets are automated: they dedupe repeats, solve themselves when the alert recovers, and never send mail, receipts or surveys to anyone.
Adding an integration
Section titled “Adding an integration”Add integration and pick the kind:
| Kind | For |
|---|---|
| Prometheus Alertmanager | Alertmanager’s webhook receiver |
| Grafana | A Grafana Alerting webhook contact point |
| Datadog | A Datadog webhook |
| Azure Monitor | An action group’s webhook, with the common alert schema |
| PagerDuty-style events | Any tool that speaks the PagerDuty Events API v2 |
| Generic webhook | Anything that can POST JSON |
| API or bot | Your own scripts, bots and agents |
| Automated email | Tools that can only send mail |
Then set:
- The team its tickets go to.
- The severity map: which priority each alert severity gets. Out of the box Critical is Urgent, High is High, Warning is Normal and Info is Low. The desk reads the tool’s own words (“CRITICAL”, “Sev1”, “warn”, “P2”).
- The dedup window (24 hours by default) and whether a repeat after solving reopens the ticket within it (on by default).
- Whether a resolved alert solves the ticket.
- A rate limit (60 requests a minute by default).
Saving shows the integration’s key once: copy it now. It is stored
only as a hash, so nobody can read it back; Make a new key replaces
it; the page shows the first characters of the current key. The page also shows the endpoint, the steps for that tool and a
curl example.
The endpoint is https://your-domain/api/helpdesk/intake/<id>, and the
key goes in Authorization: Bearer <key> or X-Api-Key: <key> (or, for
Azure Monitor only, ?key=<key> in the address). Your monitoring tools
must reach it over HTTPS.
What an alert does
Section titled “What an alert does”| The alert | The desk |
|---|---|
| Fires, new | Opens a ticket for the integration’s team, marked Automated · Grafana, with priority from the severity map. Assignment rules and agents’ notices run as for any ticket. |
| Fires again while its ticket is open | Adds “Fired again (×N)” to the ticket. No new ticket, no new notices. |
| Fires again after its ticket was solved | Within the window, with reopen on: reopens the ticket with a note. Otherwise opens a new ticket that notes the earlier one. |
| Resolves | Adds a note and, with auto-solve on, solves the ticket, unless a person has replied publicly on it (“Alert resolved; ticket left open because a person replied”). |
A tool’s retry of the same request has one effect, however close together the two arrive. Each alert’s dedup key ties its firings and its resolve together.
An automated ticket shows the alert’s links and fields beside the
conversation. Its requester is the integration (“Monitoring (Grafana)”),
so the composer opens on an internal note. Alert text can’t insert
images, embeds, videos or disguised links: the desk writes every link
itself, and a link’s target can’t break out of it. Recent deliveries on the
integration’s page lists what each request became: Created,
Updated, Reopened, Resolved, Ignored or Rejected.
Send a test alert opens a real ticket tagged test-alert.
Per tool
Section titled “Per tool”Prometheus Alertmanager
Section titled “Prometheus Alertmanager”Add a receiver to alertmanager.yml and route the alerts you want as
tickets to it. Keep send_resolved: true so a recovered alert solves its
ticket. Each alert’s fingerprint is its dedup key, and its severity label
picks the priority.
receivers: - name: compassdocs-helpdesk webhook_configs: - url: "https://your-domain/api/helpdesk/intake/<id>" send_resolved: true http_config: authorization: type: Bearer credentials: "<your key>"Grafana
Section titled “Grafana”- Alerting → Contact points → add a contact point of type Webhook with the endpoint as its URL.
- Under Optional Webhook settings, set Authorization Header
Scheme to
Bearerand the credentials to your key. - Point a notification policy at the contact point. Resolved notifications solve the ticket.
Datadog
Section titled “Datadog”-
Integrations → Webhooks: add a webhook with the endpoint as its URL and this payload:
{"title": "$EVENT_TITLE","text_only_msg": "$TEXT_ONLY_MSG","alert_transition": "$ALERT_TRANSITION","alert_type": "$ALERT_TYPE","alert_id": "$ALERT_ID","aggreg_key": "$AGGREG_KEY","priority": "$ALERT_PRIORITY","hostname": "$HOSTNAME","link": "$LINK","tags": "$TAGS"} -
Under Custom headers, add
{"Authorization": "Bearer <your key>"}. -
Mention
@webhook-<name>in a monitor’s message. The Recovered transition solves the ticket.
Azure Monitor
Section titled “Azure Monitor”- In an action group, add a Webhook action and turn on the common alert schema.
- Azure can’t send an Authorization header, so put the key in the
address:
https://your-domain/api/helpdesk/intake/<id>?key=<your key>. - Fired opens a ticket and Resolved solves it; Sev0 to Sev4 map to the priorities you set.
PagerDuty-style events
Section titled “PagerDuty-style events”For tools that speak the PagerDuty Events API v2: use the endpoint as the
events URL and send the key as a Bearer token. event_action: trigger
opens or repeats, resolve solves, and dedup_key ties them together.
routing_key is ignored.
Generic webhook
Section titled “Generic webhook”POST JSON with Authorization: Bearer <your key>:
curl -X POST "https://your-domain/api/helpdesk/intake/<id>" \ -H "Authorization: Bearer <your key>" \ -H "Content-Type: application/json" \ -d '{"title":"Disk almost full on fs-02","severity":"warning","status":"firing","dedup_key":"fs-02-disk","source":"fs-02"}'Fields: title (required), description, severity, status (firing
or resolved), dedup_key, source, links ([{ "label", "url" }])
and fields (an object of extra facts). Send the same dedup_key with
status: resolved to solve the ticket.
API or bot
Section titled “API or bot”The same fields as a generic webhook, from a script, bot or agent. Add
requester_email to open the ticket for a workspace member: when it is an
active account that may use Get help, the ticket is theirs, not automated,
and they get the usual updates; otherwise the address is ignored. Send an
Idempotency-Key header to make retries safe.
By email
Section titled “By email”For tools that can only send mail, add an Automated email integration
and list the tool’s sender addresses. Have the tool mail your help
address (see Email channel) from one of them. Its mail
is accepted even though it is automated, as long as your mail provider
confirms the sender (see
Sender authentication): results
the sender wrote into its own headers are ignored. Mail with no result at
all is refused unless the integration has Accept mail without SPF or
DKIM results on; a failed check is always refused. The subject is the ticket’s title, and a subject
tagged [RESOLVED] or [OK] solves the ticket with the same subject. A
subject pattern can pick the dedup key out of the subject. A pattern
that could backtrack without end is refused, and every match has a time
budget.
Answers
Section titled “Answers”| Code | Means |
|---|---|
200 / 201 |
Accepted; the body says what the alert became |
400 |
The payload isn’t one the integration understands (no title, bad JSON) |
401 |
No key, or a wrong one |
402 |
The helpdesk license is missing or expired |
403 |
The integration is turned off |
413 |
The body is over 256 KB |
429 |
Over the integration’s rate limit; retry later |
Every rejection is listed under Recent deliveries with its reason, once per reason a minute. Repeated wrong keys are rate limited before anything is written.
In the audit log
Section titled “In the audit log”| Action | Shown as |
|---|---|
helpdesk.integration.created / .updated / .deleted |
Connected / Edited / Removed a monitoring or API integration |
helpdesk.integration.key_rotated |
Issued a new integration API key |
helpdesk.intake.rejected |
Rejected an alert or API ticket |
