How Software Teams Keep Customer Trust During Incidents

1. First I tell the customers within 10 minutes what has happened, who is affected by it and what they need to do right now. The cause of a problem can wait. My first guess at the cause of a problem is usually wrong and by the time I have corrected it, we are already issuing a correction to the correction. All of my clients are traders who run Expert Advisors with live positions on their brokers. An outage costs them money by the minute, not by hours of their time. Therefore, saying 'we are investigating' is useless. Saying 'Servers in node X are unreachable. Open positions are still on your broker's server. Log in to your phone terminal and manage your risk there until we confirm recovery' is very actionable. Stating first that something is 'minor' until it has been fixed, may mean big losses for your customers, while to you they may have seemed minor.

2. Upstream network failure at a data center with VPS servers hosted there caused connectivity failure for a chunk of our VPS servers during a very volatile London session. It took us almost an hour to find the root cause, but a fixed update schedule of posts every 15 minutes (including updates of "no change" and the work with the upstream provider still ongoing) eventually saved the day. The fact that we hit every update deadline in time and communicated what was going on in a timely fashion even though we were down for a while prevented a much worse outcome. In the end, we even sent a simple postmortem with a timeline of events, root cause and fixes to the routing as well as to the underlying problem, and there were even fewer cancellations than in a "normal" month. People forgive downtime, but they will not forgive you for being unknown for a while and then coming up with the worst possible scenario in the meantime.

Customer communication decisions depend on the level of impact. Essential tasks becoming unavailable or customer information appearing at risk calls for immediate communication. A technical label should never delay a notice. The message should explain the affected behavior, confirm that an investigation is underway, and provide a clear time for the next update.

One incident involved a defect that showed outdated completion records after a change. The records appeared incorrect before their actual integrity could be confirmed. The notice clearly separated what was visible from what was known, explaining that records might appear delayed while validation continued. This prevented unnecessary alarm while respecting customer concerns, and a final update shared the findings and the safeguard introduced afterward.

Early in an outage, the urge is to explain the cause. Don't. Your first guess is wrong surprisingly often, and walking back "it was a database issue" two hours later costs you more trust than the outage did.

What customers actually need early on is three things. What's broken, in their terms, not ours. Whether there's a workaround. And when they'll hear from you next. Nobody cares that a processing queue is backed up. They care that "papers you upload right now won't turn into audio yet, but anything already in your library still plays fine." That second half matters a lot. It tells people what they don't need to worry about.

The practice that helped us most was the next-update time. We had a stretch where audio generation stalled, right around finals, which is about the worst timing possible for us. Our first message said: we're on it, here's what still works, next update at 3:00. We had no idea when the fix would land. But we could always promise the next update.

At 3:00 we didn't have much. Still broken, still digging. We posted anyway. "No fix yet, here's what we've ruled out, next update at 4:30." That update, the boring one with no good news, is the one people thanked us for. It showed someone was actually watching it.

I think that's the real lesson. During an incident you can't control when the fix arrives, but you can always control when the next update arrives. Hit every one of those times, even when there's nothing new to say, and people will cut you a surprising amount of slack.

I once told a machine-building client: "We have four hours' response time in the contract. But that's not the standard we hold ourselves to. If something is on fire, we want to help straight away." That is how I'd open any outage message before the cause is known: say you are on it now, then say what you know so far. After 20 years as founder of hagel IT, I think people forgive a missing root cause much sooner than a slow first reply. Jens Hagel, Founder & Owner, kjello (Hamburg, Germany)

We tell customers only what we know before the cause of an issue is clear. We begin with a simple, useful statement that we can confirm, such as a delay in a specific workflow without knowing its cause. Sharing these limits is better than using broad words that suggest everyone is affected. Clear communication prevents unnecessary worry and helps customers understand the actual situation.

We also explain what must happen before we consider an issue resolved, rather than only sharing investigation updates. This includes stable performance and confirmation that delayed activity has been processed. We build confidence by showing customers clear signs of recovery instead of making unsupported claims. This makes our updates more credible and helps customers trust the final resolution.

Software leaders often feel compelled to issue frequent updates during an outage to demonstrate care for their clients. However, providing meaningless updates can undermine trust more quickly than silence.

Research indicates that 78% of customers would prefer a clear communication stating that an outage will take four hours to resolve rather than receiving hollow updates every 30 minutes.

In these situations, it's essential to prioritize transparency over corporate apologies. Begin by conducting an initial investigation lasting around 30 minutes, after which a realistic estimate for resolution should be provided, expressed in a range that reflects uncertainty.

If a precise timeframe isn't available, share the next steps being taken, such as a database restoration, which typically requires a couple of hours.

Additionally, avoid relying on manual email updates, as this approach can lead to inefficiencies and miscommunication. Instead, automate data collection from all internal tickets pertaining to the incident.

Establish a system that can draft factual updates automatically, ensuring that information is timely and accurate. Furthermore, connect this communication automation system to the live engineering status.

This integration allows for real-time updates without distracting engineering teams, enabling them to focus on resolving the issue efficiently while keeping customers informed with accurate information.

The communication practice that helped protect trust during a real incident was assigning one person to manage external updates. During an account information incident, several people had different facts, and their draft messages began to conflict. The team stopped this process and appointed one incident lead to gather details and share all updates.

This approach worked because customers received consistent information across every channel. Each update explained what had changed instead of repeating general assurances. Internally, it ensured that every claim was checked before reaching customers. Externally, it showed that the team was organized and accountable, helping prevent further confusion while the incident was being resolved.

Early incident communication should be designed for the customer who must brief an executive, not the engineer watching dashboards. That means leading with business consequence, then stating what is known, unknown, and being done next. I avoid adjectives such as isolated or limited until boundaries have been measured, because those words can later become exhibits in a trust failure.

One incident practice was maintaining a sentence that changed only with evidence. It stated the core fact while later updates added detail. This stopped contradictory posts from suggesting information was withheld. Responders revised hypotheses internally, but public language stayed evidence-based. Customers saw investigation and fact kept distinct.

The first message names the symptom in the user's own words and says the cause is still unknown. It also gives a clock time for the next update. That's all of it. "Some scans are returning wrong results. I'm still finding out why. Next update at 3 PM." Then I post at 3 PM even if the update is "still looking."

I skip guessing at causes early. A guess that gets corrected an hour later makes every earlier sentence look shaky, and people remember the correction more than the fix.

I can't hand you a dated incident, and I won't make one up. What I can give you is the failure that shapes how I'd communicate as someone building a photo-based AI app. The worst defect there is a confident wrong answer, because the user can't see that it's wrong. Silence makes that worse. So the message has to say which results to hold off trusting, and what to do meanwhile. If the feature can be paused, I'd rather pause it and say so than leave it running with a quiet fix underway.

People forgive an outage fairly quickly. What they remember is whether the updates arrived when promised and whether the wording stayed plain. A kept promise to update at 3 PM does more for trust than a clever explanation at 6.

We build the first message around the possibility that our team may be wrong. Although this may sound pessimistic, it leads to clearer communication. We avoid absolute promises and make our next commitment specific. Instead of promising a quick fix, we commit to providing an update at a set time, even if the investigation is still underway.

One incident showed us the value of explaining what remained unchanged. After a disruption, customers wanted to know whether nearby workflows were also affected. Our updates clearly identified the areas that remained unaffected, offering more reassurance than general monitoring statements. This also prevented unnecessary workarounds by unaffected customers.

I give my customers three things up front: what we know, what we don't know, and when their next update will be.

The incident: In early spring, our site scans started to fail. There were false score readings, and we didn't know why.

What I did: I posted a quick update in 15 minutes on our status page and email blast. "Scans are failing; scores might be wrong. We're working on it. Our next update is at 3 PM." I didn't speculate on causes, nor did I offer a timeframe for a fix.

The habit that helped: I posted updates every 30 minutes, even without any new information. "Still working; no further info available." People hate silence. Even small amounts of information calm them.

Results: Fixing the problem took 3 hours. Customer support tickets were 60% lower than the previous outage. Two customers cancelled; most of the others wrote "Thanks for the honesty."

What I learned: Honesty trumps speculation. Set your next update time and stick to it. Once the problem was fixed, I gave a brief cause report.

Before we know why a service has failed, I tell customers which function is affected and what we've done to contain the problem. I separate what users can observe from what engineers are still investigating. An early notice shouldn't carry a suspected cause or a repair time that the team can't substantiate. It should let a user make a decision about the function right now.

A fintech app we built had a wallet function dependent on a bank API. When that API failed, our feature flags let us switch off the affected wallet function remotely. We sent users a push notification about the change. The pairing mattered: users received an account of what had changed, and the app no longer invited them into the broken function. We could state that action while the underlying API problem remained outside the app's control. I didn't need to turn an unfinished diagnosis into a customer-facing explanation to describe what the team had already done.

For another incident, I'd have the team confirm the user-facing symptom and take the safest available containment action before wording the notice. The message can say how the user's next action is affected and what the team changed. Engineers should keep hypotheses in their internal incident discussion until they can verify them. If the affected function changes state or returns, we can send another update with the newly confirmed facts. A useful early notice lets a customer decide what to do without borrowing certainty from the team's hypothesis.

At Your Team in India, our engineers work as dedicated teams for companies that ship software to their own users. So when an outage hits, two relationships are at risk at once: ours with the client, and the client's with their customers.

When the cause is still unclear, we decide what to say by asking what the customer needs to make their own decisions in the next hour. Usually that's plain facts about impact: which feature is down and who is affected. If data is safe, we say so early, since that's the first fear most people have. We share a workaround if there is one. We don't guess at causes in public. A wrong early theory is much harder to take back than a short "we're still investigating."

What has protected trust most for us is a fixed update schedule. Once an incident is declared, the client's status page or email thread gets an update every 30 minutes, even when nothing has changed. A note that says "no new findings, next update at 3:30 PM ET" looks thin. It still stops support tickets from piling up, and it keeps account managers from inventing their own explanations.

One incident shaped this. After a release, a client's checkout flow started failing for some users. Our first instinct was to stay quiet until we had a clear answer. Within 40 minutes, the client's support inbox was full of people asking if they'd been charged twice. We posted a short note confirming that no duplicate charges had gone through, naming the affected payment method, and committing to updates every 30 minutes. Ticket volume dropped almost right away, even though the fix took close to three more hours. It turned out to be a rollback plus a patch to a third-party SDK version.

We also keep the engineering conversation separate from the customer conversation. Engineers talk freely in the incident channel. One person, usually the delivery lead, turns findings into customer language and owns every external message. That keeps half-confirmed theories from leaking out. Within 48 hours of the fix, we send a written summary. It covers what happened, how long it lasted, what users saw, and what we've changed in testing or monitoring so it doesn't happen again.

Tell them the impact, the workaround, and the time of the next update. Withhold the cause until it is confirmed, and say that is what you are doing. Customers can handle 'we do not know yet' far better than a confident explanation that gets replaced on Thursday. The first message in a serious incident should contain only things that will still be true when the postmortem is written: which customers and functions are affected, since when, what we have already done, and what we are checking.

We use pre-agreed wording for the early messages, decided in calm times, because the middle of an incident is a bad place to negotiate with legal and leadership over adjectives. The templates are not evasive. They are a promise about the shape of communication: impact in plain terms, no speculation dressed as diagnosis, a named next update. Engineers resist templates until the first time one lets them send a message in five minutes instead of fifty.

Trust is protected by the corrections too. When our early understanding turns out wrong, we say so in the same channel, at the same prominence, without burying it in a longer update. A company that corrects itself in public gets believed the next time it says 'we do not know yet.' That belief is the whole asset, and it is built one incident at a time.

I run a paid trading community where DataBot, DAC's AI assistant, answers members around the clock. So I think about this in terms of what members need to hear while I'm still digging. They need to know what's broken and when the next update is coming. The cause can wait.

I'd post within minutes, even if the whole message says the bot is down or giving bad answers and I'm on it. Members who can't tell whether something is broken start guessing in public, usually at the worst version. A short message with a time on the next update gives them something to wait for.

I won't name a cause until I can show it. If I blame a data feed and it turns out to be my own code, I've handed people a second thing to be upset about. Saying what I've ruled out and what I'm checking is enough, and it's all true.

I don't have a dramatic outage story and I won't invent one. What I do practice comes from when DataBot launched. I told members on a livestream to flag any wrong answer so I could fix it, which made reporting a defect a normal thing to do. When something breaks now, they already know where to say so, and I can post the update in that same place.