The 'Success Metric' That Kept Me Awake for Three Days
On day twenty of the 12W acceleration program, the team's third KPI was officially abandoned. The KPI itself was nothing special—daily new sign-ups, with a target of 15% weekly growth. At the time of setting it, the metric seemed completely reasonable: user base is the prerequisite for monetization, and without enough users, any business model is just talk. However, after three consecutive weeks of data tracking, the team discovered an unsettling phenomenon: registrations were indeed growing, but the seven-day retention rate remained stuck between 3% and 5%, far below the industry average of 20% to 30%.
This discovery forced us to confront a fundamental question: what are we actually optimizing for? While all resources were directed toward advertising and channel acquisition, the real product problems that needed solving were being pushed aside. According to Eric Ries, author of The Lean Startup, many startups fall into the trap of vanity metrics—numbers that look impressive but provide no actionable guidance.
"User registrations" is a textbook vanity metric. It measures acquisition capability rather than whether the product actually solves user problems. When retention is low, growing sign-ups only expands a silent user base that knows about the product but doesn't use it, providing no real contribution to business value.
Why This KPI Misled Resource Allocation
After deeper analysis, we identified three core reasons why this KPI failed. First, the metric definition was too narrow—it only measured "someone completed registration," while ignoring "who this person is," "why they came," and "whether they have actual use cases." These dimensions are often more important than raw volume in the early stages.
Second, the targets were set based on growth desire rather than actual user behavior patterns. The 15% weekly growth rate implies linear accumulation thinking, but early-stage user acquisition is often non-linear—a single feature improvement or word-of-mouth referral can trigger explosive growth, or the product can stall for weeks. Treating such volatile metrics as rewards and punishments for team effort creates unnecessary pressure and anxiety.
Third, and most critically: the causal chain between this KPI and the business model is too long. Registration → engagement → payment → referral—each step in this path involves massive attrition. If you only set metrics at the top of the funnel, teams will gravitate toward the cheapest way to inflate numbers—buying ads, gaming clicks, promo hunters—rather than solving the fundamental product-market fit problem.
According to Startup Genome's 2022 report, 74% of failed startups collapsed due to premature scaling rather than insufficient product value. Obsessive focus on vanity metrics is precisely one manifestation of this premature scaling.
After Abandoning It: Searching for "Value Density" Metrics
The decision to abandon "user registrations" wasn't about the team stopping to care about user growth—it was about replacing vague quantity tracking with a more precise framework. After internal discussion, we adopted "Value Density" as our new core metric framework. The specific definition: the ratio of core value action frequency observed on a single user, to the cost of acquiring that user.
Using a B2B SaaS product as an example, Value Density might be measured as "the average number of key tasks completed by paying users within their first two weeks, divided by CAC (Customer Acquisition Cost)." When this ratio starts rising steadily, it indicates the product is accumulating genuine moat—not users brought in through subsidies, but users who truly derive value from the product.
This adjustment brought immediate changes in resource allocation. Funds originally earmarked for scaling ad spend were redirected toward user interviews and behavioral analysis. Instead of tracking "how many people signed up today," the team started monitoring "what obstacles are these new users encountering that prevent them from completing core tasks."
This wasn't an easy transition. It required the team to abandon the intuition that "numbers equal progress," and to accept an uncomfortable fact: sometimes behavioral data from a small number of deeply engaged users is more valuable than aggregate data from a large number of shallow users.
An Adjustment You Can Execute Immediately
If your team is facing a similar KPI dilemma, here's an adjustment you can execute today: convert any of your existing quantity-based metrics into a "ratio + threshold" composite format.
The specific approach: change "100 daily new sign-ups" to "at least 30% of weekly new sign-ups complete their initial setup within 48 hours of registration, and those 30% users maintain a 7-day retention rate of no less than 25%." This composite metric tracks both user quality and behavioral depth, giving a more accurate picture of whether product value is being perceived by users.
One trap to watch for during execution: absolutely do not pile in every quality metric you can think of. A good composite metric should contain only three variables—one quantity baseline, one quality threshold, and one behavioral target. The more complex a metric, the higher the tracking cost, and the more dispersed the team's attention becomes.
The core purpose of this adjustment isn't to make the numbers look better—it's to make the data more actionable. When you find that a composite metric has missed its target in a given week, the direction of your follow-up questions becomes very clear: is the problem with the quality of acquired users, or are there obstacles in the user journey? Each question points to specific team actions.
Closing: KPIs Are Tools, Not Steering Wheels
In the third week of 12W, the most important lesson we learned was this: KPIs should be dashboards measuring your speed, not steering wheels deciding where you're going. The steering wheel is held by the team's understanding of product value and user problems. When there's a disconnect between dashboard and steering wheel—numbers growing but direction unclear—that's the time to stop and recalibrate what you're actually tracking.
The biggest risk in early-stage entrepreneurship isn't slow growth—it's putting your energy into the wrong things. A carefully considered "abandonment decision" is often more valuable than persisting with a KPI that was never questioned.
"The moment you see a metric as a target, it ceases to be a good metric."— Thomas Sowell
(當你把一個指標當作目標，它就不再是個好指標。)