Death by a thousand metrics
Death by a thousand metrics
Local vs. global maxima, why products die, and how to build things that work
A week ago, this screenshot of weather.com went viral on Twitter.
It’s a freeze-frame of a fight between growth, legal, revenue, the web team, and the app team. And it’s utterly unusable.
It’s death by a thousand metrics.
Companies that manage to pull off delightful products don’t make decisions based on metrics; they use metrics to reject or confirm their hypotheses in order to move toward a product vision.
At Square, metrics didn’t dictate the product roadmap—the goal was to create a new way of doing commerce, and no extrapolation of user behavior could have guided them there. As Keith Rabois explains:
“Square’s vision is driven by what people want to see in the world, what's a better product, what's a better experience, and then using metrics to measure if we're right or wrong.”
Metrics do play a role, but they should not derail the larger product direction which promises to create something truly extraordinary. Keith continues:
“Having a design-driven culture doesn't mean that you can't use metrics and empirical analysis too, but you need to decide whether you’re making a decision based upon optimizing a specific metric, or making a bet that over time you're rearranging how people think about a product or service which will eventually pay off in metrics.”
As companies grow larger, teams become siloed and metrics become the most efficient way to enforce accountability and measure progress. But the best companies avoid vision myopia by installing a benevolent product dictator to ensure progress on the only non-negotiable metric: product delightfulness.