Major incidents

Enterprise feature (new in 1.19.2, helpdesk). See the Helpdesk overview.

When the VPN goes down, forty people raise forty tickets. A major incident gathers them: one ticket for the outage, with the related tickets linked to it. Each update you post reaches everyone affected at once, Get help shows a banner so nobody needs to raise another request, and when it’s fixed you solve the incident and its linked tickets in one go.

Who can do what

Two desk role capabilities, both in the Tickets section of Desk settings → Roles:

Capability	Out of the box	Lets you
Link tickets to a major incident	Agent and up	Link and unlink tickets of your teams to an open incident. Checked on each linked ticket’s team.
Run major incidents	Team lead and up	Declare one, post updates, choose its Get help banner, edit its rule and solve it with its linked tickets. Checked on the incident’s team.

Running an incident also needs Reply to the requester in the incident’s team, because each update is a public reply. Someone whose replies wait for review can’t run one: the incident’s ticket says “Your replies are reviewed before they’re sent, so you can’t post to everyone affected. Ask a team lead.”

Posting the incident’s updates to the Status page also needs the workspace permission Manage incidents (status.incident_manage), which the Approver and Administrator roles hold.

Declaring an incident

Declare one from:

• a ticket’s … menu: Declare a major incident… (that ticket is linked to it); or
• the queue: tick the related tickets, then Declare incident… in the bulk bar (the selected tickets are linked); or
• a Status page service your team follows (see Status services for teams).

The Declare a major incident dialog asks for:

• Title people will see, such as “VPN is down from home”. It is the incident ticket’s subject and the banner’s heading in Get help.
• Team, the team that runs it, and Priority (Urgent by default).
• Link: this ticket, or the selected tickets.
• Link new tickets automatically while it’s open: Subject or description contains: up to five words, separated by commas (see Linking new tickets automatically). Leave it empty to link tickets by hand.
• Show a banner in Get help with a “Me too” button (on by default).
• Tell the requesters of linked tickets by email (on by default).

Declare incident opens a new ticket for the outage, in that team. It has no requester, it is assigned to you if you’re in the team, and it starts with an internal note: “Priya Shah declared a major incident and linked #1052, #1051 and #1049.” Each ticket you link is checked on its own; if any can’t be linked, the dialog lists each one with its reason and nothing is declared.

The incident’s ticket

The incident’s ticket says Major incident at the top, who declared it and when, how many tickets are linked and whether it’s shown in Get help. Above the conversation it has:

• Updates to everyone affected: every update so far, with who posted it and “sent to 34 requesters and shown in Get help”, and the box to post the next one.
• Linked tickets: the linked tickets in teams you work, with their requester and when they were linked. Tickets in teams you don’t work are counted, not listed. Link tickets adds more, and the rule and Tell requesters when their ticket is linked are under the list.

In the side panel:

• Get help banner: who sees it (Shown to), a preview, the number of “Me too” clicks, and People affected: the requesters of linked tickets and everyone who said Me too, each counted once.
• Status page, when the Status page has an internal service (see On the Status page).

In the queue, the incident’s ticket carries a red Major incident chip, and each linked ticket a chip with the incident’s number. A Major incidents group in the queue’s left column lists each open incident whose ticket you work, with how many of its linked tickets are still open.

Linking tickets

A ticket can be linked to one incident at a time. To link more:

• on the incident’s ticket, Link tickets and type the Ticket numbers, separated by commas (up to 200 at a time); or
• on the ticket itself, … → Link to a major incident…, then pick the incident and Link to #1101.

Each ticket is checked on its own, as if you linked it by hand: you need Link tickets to a major incident in its team. Closed and merged tickets, workflow tasks and tickets already linked elsewhere can’t be linked. When some are skipped, the message says how many, and See why lists each with its reason.

A linked ticket shows a strip: “Linked to major incident #1101 VPN is down from home · its updates reach this requester”, with Unlink. Unlinking stops its requester hearing the updates, and solving the incident then leaves that ticket open.

Telling the requester. With Tell requesters when their ticket is linked on, linking a ticket adds a public reply, “We know about this problem: “VPN is down from home”. We’re working on it and will keep you posted here, so there’s no need to send anything else.“, and emails the requester. With it off, linking is quiet. Updates go out either way.

Linking new tickets automatically

While the incident is open, a new ticket whose subject or description contains one of the rule’s words links itself. Set the words when you declare, or with Edit the rule on the incident’s ticket.

• Up to five words or phrases, separated by commas, each three letters or more and up to 60 characters.
• Letter case doesn’t matter, and a word must start a word in the ticket: vpn matches “VPN down” and “the vpn’s slow”, not “openvpn”.
• Only new tickets of the incident’s team link themselves. A ticket in another team links only if the person who declared the incident is still active, works that ticket and may link tickets in its team.
• Workflow tasks never link themselves.

A ticket the rule links gets a line its requester can see on the timeline, not a reply, so it doesn’t count as the first reply or move the ticket from New to Open. The requester is still emailed when Tell requesters when their ticket is linked is on.

Posting an update

Write the update in Updates to everyone affected (up to 5,000 characters) and choose Post update, or press Ctrl+Enter (Cmd+Enter on a Mac). Each update:

• becomes a public reply on every open linked ticket you may answer. It stops the ticket’s first-reply clock and moves a New ticket to Open. Solved and closed linked tickets get nothing more;
• skips linked tickets you can’t reply on (not in your queues, or a team where you can’t reply to requesters). The message says “2 not reached”, and See why lists them;
• sends one email and one bell to each person affected: the requesters and CCs of the linked tickets, and everyone who said Me too. Each person hears once per update, however many linked tickets they are on. It replaces the ordinary reply email for those tickets, and it is never held for a digest.

The email is the Helpdesk: major incident update template, and people choose it under Major incident updates in their notifications (see Notification policies). Customer portal customers get the portal’s version, with a link to their request there. Followers of linked tickets get no desk notice for an update.

The Get help banner

While the incident is open and its banner is on, Get help shows it at the top: the title, the latest update (or “We know about this and we’re working on it. You’ll hear from us here.” before the first), the team, when it was last updated and how many people are affected.

On the incident’s ticket, Shown to chooses who sees it:

• Everyone (the default): everyone who may use Get help.
• The team’s requesters (shown as “IT’s requesters”): people who have asked the incident’s team for help before, and everyone on a linked ticket.
• Off: no banner. Linked tickets’ requesters still hear each update.

The banner shows only in Get help, not in the customer portal.

Me too

Me too on the banner counts you as affected without raising a request. You then hear each update by email and bell, for as long as the banner is on and you may use Get help. Press it again to take it back. Saying Me too is in the person’s data export.

Solving the incident

When it’s fixed, tick Solve the incident and its 34 linked tickets under the update box. With an update written, Post update sends it first; with the box empty, the button reads Solve all. You are asked to confirm.

The incident’s own ticket is checked first. If it can’t be solved (for example, a checklist item is still required), nothing is posted or solved, and you are told why. Then each linked ticket is solved as if you had solved it by hand: its requester gets the solved email and the survey, and time and checklist rules apply. Any you can’t solve stay open, and See why lists them. Tickets linked while the solve was running are solved too.

Requesters of linked tickets get the update email first, then the solved email.

When an incident ends. An incident is open while its ticket is open. Solving or closing its ticket by any route ends it, and so does merging it into another ticket or deleting it (from 1.19.4, moving it to the recycle bin; restoring it doesn’t reopen the incident): the banner goes and the rule stops linking. Reopen the incident’s ticket to post again. Linked tickets you solve on their own simply stop getting updates.

On the Status page

When the Status page has an internal (Manual) service, the incident’s side panel has a Status page card. With Manage incidents, turn on Post updates to the Status page too; the card then names the service (“Post updates to the Status page service VPN too”), and with more than one internal service you can pick another. Then:

• the Status page gets an incident on that service, and each update is posted to it;
• it is resolved there when the incident ends, whichever way: solved, closed, merged or deleted, or when you turn the card off or move it to another service;
• reopening the incident’s ticket opens a new Status page incident.

The Status page’s own “everyone” notification isn’t sent for these: the people affected already hear through the incident.

Limits

What	Limit
Title	200 characters
Update	5,000 characters
Rule	Five words or phrases, three to 60 characters each
Tickets linked in one go	200
Declaring	10 a minute per person
Linking	30 requests a minute per person
Updates	10 a minute per person
Me too	20 a minute per person

Privacy

Get help’s banner never shows ticket numbers or agents’ names. When the incident’s ticket is anonymised, its updates, title, rule and the declarer’s name are cleared too, and its audit entries keep only the length of each update.

In the audit log

Every entry is recorded against the incident’s ticket.

Action	Shown as
helpdesk.incident.declared	Declared a major incident
helpdesk.incident.linked	Linked tickets to a major incident
helpdesk.incident.unlinked	Unlinked a ticket from a major incident
helpdesk.incident.auto_linked	Linked a new ticket to a major incident by its rule
helpdesk.incident.update_posted	Posted a major incident update
helpdesk.incident.solved	Solved a major incident and its linked tickets
helpdesk.incident.settings_changed	Changed a major incident’s settings
helpdesk.incident.affected_added / .affected_removed	Said a major incident affects them / Took back “Me too” on a major incident