"We had a chatroom incident and I want to learn from it rather than move on."
Recommended. 30 minutes per incident drives the biggest difference in operator learning + chatroom trust. This post covers a 4-section template + 5 core questions.
When to run a post-mortem
Incident priority matrix
P1 (immediate post-mortem) / P2 (quarterly batch) / P3 (ops stability), 7 categories.
Bot exposure
Trust impact
Ad-bot dialogue
Member churn risk
Refund / dispute
Legal exposure
Member report
Chatroom removal risk
Data loss
Continuity
Tone drift
Gradual naturalness loss
Response latency
Ops stability
All P1 → 30-min postmortem. P2-3 batched quarterly.
4-section template
Postmortem — [Date] [One-line incident summary]
## 1. Facts (What happened)
- Incident start time
- Full flow (5-min granularity)
- Affected chatrooms / members / responses
- Operator detection time (how late)
- Incident end time
## 2. Impact
- Member reactions (churn / dispute / reports)
- Trust metric shifts (engagement / new-join / churn)
- Operator time burden (response / follow-up)
- Legal / policy risk (if any)
## 3. Root cause
- Direct cause (e.g. persona responded to ad-bot message)
- Indirect cause (e.g. ad-bot block rules missing)
- System cause (e.g. operator absent at time of member report)
- Operator decision / rule / time-management impact
## 4. Prevention
- Persona / chatroom rule / operator policy changes
- Chatroom notice / member handling / post-action
- Automation rule reinforcement (triggers / forbidden / hard-banned)
- Periodic review schedule update
5 core questions
Sample incident timeline
Start → spread → operator detection → response → close. Detection lag (orange) heavily affects recovery time.
1. When did the incident start?
Clear start point. Detection usually trails start. Use Replyer's Activity / Logs to find the actual moment.
2. What went wrong?
Facts only (no operator evaluation / emotion). Distinguish personal opinion (one member) from operational-system failure.
3. Why did it happen?
Direct + indirect + system causes (3 levels). "5 Whys" - chain the same question 5 times.
5-Why trace tree
Trace ad-bot incident 5 levels deep to reach the system root cause.
4. What was the scope?
- Affected member count
- Affected chatroom count (multi-room ops)
- Affected time range (start → end)
- Metric shifts (engagement / churn etc.)
5. How do we prevent it next time?
Concrete / actionable rule changes. No "be more careful" abstract. e.g.:
- Add "refund" / "dispute" to persona hard-banned phrases
- Strengthen ad-bot block trigger regex (URL detect / English ratio 30%+)
- Add "ad-bot intrusion check" to operator periodic review
- Auto-fire operator webhook on incident-keyword match
Post-postmortem application
Mermaid: incident → postmortem → application flow
Operator decision tree from incident occurrence to 1-week re-check.
flowchart TD
A[Incident] --> B{P1?}
B -- Yes --> C[Start postmortem after 24h]
B -- No --> D[Queue for quarterly batch]
C --> E[Write 4 sections + 5 questions]
E --> F[Apply within 24h]
F --> G[Update persona prompt]
F --> H[Add hard-banned phrases]
F --> I[Update pinned message]
G --> J[48h: notify members]
H --> J
I --> J
J --> K[1-week re-check]
K --> L{Recovered?}
L -- Yes --> M[Preserve record]
L -- No --> N[More rule changes + re-postmortem]
N --> F
Immediate (within 24h)
- Update persona prompt (triggers / forbidden)
- Update pinned message (if needed)
- Update operator policy (operator notes)
Member announcement (within 48h)
- Honest disclosure (operator handles)
- Spell out post-actions
- Open member feedback / Q&A
1-week re-check
- Verify impact recovery (engagement / churn metrics)
- Check rule-change side effects (false positives etc.)
- Operator mental / time burden check
Operator self-retrospective
Postmortem is for system, but the operator also needs a self-retrospective:
- Why was detection late? (Missed review? No alert channel?)
- Operator's emotion during response (anger / avoidance / anxiety)
- Continue chatroom vs close consideration
Catch burnout signals. See operator burnout recovery.
5 postmortem traps
1. Right after the incident
Emotional bleed → not objective. Wait 24h. Stabilize first, then summarize fact / impact.
2. Blame-heavy
Specific member / operator blame → no learning. Focus on "why", avoid "who".
3. Abstract prevention
"Be more careful" / "next time better" don't work. Concrete / executable rules only.
4. Postmortem but no rule changes
Reflection without persona / rule updates → same incident repeats. Apply immediately.
5. No record kept
No record → 6 months later forgets old postmortem and same incident hits again. Record + periodic review essential.
FAQ
Q. Trivial incidents (single short member dispute etc.) too?
Trivial incidents can skip. Postmortem criteria:
- Chatroom trust impact (5+ members noticed)
- Legal / policy risk (refund / dispute / report)
- Heavy operator time (1h+ response follow-up)
Q. Postmortem getting too long?
30-min alarm. Operator's qualitative self-retrospective happens separately (a week later). Stick to facts / impact / cause / prevention in 30 min.
Q. Multi-operator postmortems?
Share in operator meetings. One operator drafts → others review + comment.
Q. Automate the postmortem template?
Partial. Replyer's Activity / Diagnostics auto-provide fact / impact data. Cause / prevention need operator qualitative analysis.
Q. Frequency climbing?
Operator-limit signal. 5+ incidents/month → consider closure / multi-operator / split. See solo vs multi-operator scaling.
Q. Heavy emotional weight after postmortem?
Operator emotion is natural. Set aside time for operator self-retrospective + rest / self-care. See operator self-care.
Next step
Grab the build for your OS from the Replyer download page and follow the usage manual for step-by-step setup.