"I can feel whether the chatroom mood is good or bad, but is there a quantitative metric I can show others?"
Yes. Moving chatroom ops from "feel" to numbers means operators / multi-operators / outsourced ops all judge by the same standard. This post covers 5 core metrics + normal range + anomaly signals + actions.
All 5 metrics at a glance - normal vs anomaly
The bar chart below shows the normal-range upper bound (blue) vs anomaly threshold (red), simulated for info chatrooms. If your measurements stay inside the blue bars you're safe; the red zone means immediate review.
Metric 1, active member ratio
Share of total members who sent at least 1 message / reaction in the last 7 days. Measure via the Telegram member list + Replyer Activity page. Normal: info 20-40%, social 40-60%, paid 50-70%.
Under 10% = chatroom dying. Use the revival flow or refresh chatroom identity.
Metric 2, daily message frequency
24-hour message count (operator + auto-reply + member messages). members × 0.5-2 is typical.
30-day trend simulation - frequency as a signal
The line chart below simulates 30 days of daily message frequency. The red threshold is 50%+ drop (subsidence), yellow is 50%+ surge (possible ad-bot / dispute). Operator review fires when the trend touches red / yellow.
Metric 3, engagement rate
Per auto-reply (or operator message), how many member follow-up reactions (emoji / replies / follow-ons). Replyer Diagnostics' response stats or direct chatroom count.
- Operator direct response - 2-5 per message
- Auto-reply (natural persona) - 1-3 per message
- Auto-reply (aging persona) - 0-1 per message
Under 0.5 = persona naturalness declining (persona aging). Revisit agents/*.yaml prompt + chatroom topics.
Metrics 4 + 5, new-join rate / churn rate
Weekly new joins / total members vs weekly leaves / total members. Reading them together gives you immediate net-add / net-loss judgment.
Per-type metric weights - heatmap
Not all metrics weigh equally. The SVG heatmap shows the priority of each metric per chatroom type. Darker cells = higher priority for review.
Info room = message frequency + engagement first. Social = active ratio + frequency. Paid = engagement + churn (value retention).
Monthly review flow
Operator monthly across all chatrooms:
□ Measure 5 metrics (15-30 min)
□ Compare with last month / 3-month avg (5 min)
□ Identify anomaly items (5 min)
□ Prioritize actions (10 min)
□ Apply quick fixes immediately (10 min)
□ Schedule big changes separately (5 min)
Under 1 hour total. Replyer Diagnostics' response / no-reply / tone charts auto-provide some metrics (/api/diagnostics/timeline, /api/diagnostics/quality) → cuts measurement time in half.
FAQ
Q. Normal ranges differ per chatroom - what's mine?
Use your room's last 3-month average as baseline. ±30% of that baseline is your normal range. Trend matters more than absolute values.
Q. Chatrooms under 100 members?
Small rooms have high metric variance + weak statistical signal. Qualitative assessment (operator personally observing mood) is more effective. 100+ members is when metrics start to help.
Q. Auto-measure metrics?
Partial. Replyer Diagnostics provides some (message frequency / engagement rate) auto (/api/diagnostics/timeline, /api/diagnostics/quality). Active ratio / new-join / churn need direct Telegram check.
Q. All metrics normal but chatroom mood feels off?
Qualitative signals (disputes / conflict / tone shift) can be missed by metrics. Quantitative + qualitative (operator personal observation) combined is safest. Don't decide on metrics alone.
Q. Compare metrics across multiple chatrooms?
Tabulate per-chatroom metrics. Anomaly in one room → that room is the priority. For multi-chatroom ops, see cross-chatroom info sync.
Q. Metric changes after automation introduction?
Re-baseline after 1 month of automation. Auto-reply shifts message frequency / engagement. Pre-introduction baseline vs post-introduction baseline measures automation impact. See first 30 days KPI for automation.
Q. Most effective way to reduce churn?
Keep operator direct response share (no 100% auto-reply) + 1:1 welcome new members (operator responds right after join) + explicit policy / vibe disclosure. See new member onboarding automation.
Next step
Grab the build for your OS from the Replyer download page and follow the usage manual for step-by-step setup.