"Just crossed 1000 members but actual active members are under 50. Which metrics actually help me decide what to do?"
Most chatroom analytics stops at member-count tracking, with zero influence on decisions. This guide separates 7 decision-grade metrics from 5 vanity ones.
Vanity vs decision metrics - impact map
Each metric plotted by [decision impact] vs [tracking cost] (simulation). Upper-left quadrant = high ROI.
Metric ROI scatter (simulation)
5 Vanity Metrics
| Vanity metric | Why it doesn't help | Decision-grade alternative |
|---|---|---|
| 1. Total member count | Many joined but inactive | Active ratio (DAU/MAU) |
| 2. Total message count | A few members may dominate | Posting-member ratio |
| 3. Months running | Measures operator endurance only | Member lifetime |
| 4. SNS/blog followers | Many didn't join the chatroom | Chatroom conversion rate |
| 5. Daily reply count | Measures automation activity only | Reply quality (👍 ratio) |
7 decision-grade metrics - healthy ranges
Healthy range (green) vs danger zone (red) for each metric. Map your numbers onto the gauge:
Metric 1, Active ratio (DAU/MAU)
Metric 2, Reply rate (within 30 min)
Metric 3, Revisit cycle (hours)
Metric 4, Member lifetime (days)
Metric 5, Content ROI (reactions per operator-hour)
Metric 6, Churn signal strength (0-100)
Metric 7, Operator time ROI
What Replyer activity log auto-extracts
# Pseudocode (parsing activity log JSONL)
# Metric 1: active ratio
mau = unique_users_with_message_last_30_days(chat_id)
dau = unique_users_with_message_today(chat_id)
active_ratio = dau / mau
# Metric 2: reply rate
operator_messages = filter(message, kind="operator")
replied_messages = filter(operator_messages, has_reply_within=30min)
reply_rate = len(replied_messages) / len(operator_messages)
# Metric 3: revisit cycle
member_intervals = []
for member in active_members:
intervals = compute_intervals(member.messages)
member_intervals.append(median(intervals))
revisit_cycle = median(member_intervals)
# Metric 4: lifetime
inactive = filter(members, no_activity_last=90days)
lifetimes = [m.last_active - m.joined for m in inactive]
avg_lifetime = mean(lifetimes)
Replyer's activity log is JSONL and persistent. Use Python pandas, Excel, Notion freely.
Monthly operator dashboard (live chart)
Monthly KPI trend - simulation drawn with Chart.js. Replace with your numbers for live ops decisions:
Monthly KPI trend (simulation)
Metrics → Decision Flow, 4 Steps
Step 1, Collect Metrics (Automated)
- Replyer activity log + external tools (Google Sheets, Notion DB) auto-populated
- Weekly·monthly auto-extracts
Step 2, Identify Changes (Operator, 5 min)
- Check each metric: this period vs last period
- Mark metrics with ±10%+ change
Step 3, Hypothesize Causes (Operator, 15 min)
- 3 possible causes per changed metric
- Verify by searching the chatroom activity log
Step 4, Decide + Execute (Operator, 30 min)
- 1-3 decisions per verified hypothesis
- Verify decision impact in next month's dashboard
What Not to Track
- Chatroom vs chatroom comparison (too many external variables)
- Single-message likes/reactions (noise > signal)
- Daily fluctuations (only weekly+ aggregates separate signal)
- External SNS followers vs chatroom members
- Revenue alone (content·relationship value lives outside revenue)
Conclusion
Most chatroom analytics is [feel-good-number] tracking. Tracking 7 decision-grade metrics monthly converts 80% of ops decisions to data-driven. The flow [metric → decision → execution → verification] matters more than the metric itself.