A retail chain’s general manager hired two data analysts and a statistician. Over six months, the team processed sales data, built dashboards, and delivered presentation after presentation. Colorful charts circulated through board meetings. Yet not a single decision changed as a result. Inventory policy stayed the same. Pricing stayed the same. Store layouts stayed the same. The team was dissolved. The manager noted in his file that ‘data science doesn’t work.’ The problem, however, was not the team.
Data science projects most often fail for a single reason: technical capacity is built before the business question is defined. Companies set up a data warehouse, hire analysts, and then ask ‘so what do we do now?’ Until this sequence is reversed, no analytics investment will produce sustainable results. The correct order is straightforward: first, identify a specific, actionable business question that decision-makers genuinely need answered; then determine what data, what analysis, and what skill profile that question requires.
The distinction between a business question and a research question is critical. ‘What do we know about our customers?’ is a research question — it has no boundaries, no answer, and no action attached to it. ‘Which customer segment reduced its basket size over the past three months, and which campaign type has historically reversed that trend?’ is a business question. The second is answerable, bounded, and directly feeds a decision. Hand the first type of question to your analyst and six months later you will have many charts but no decisions made.
In practice, the most common symptom of this problem is when analysis reports are described as ‘interesting.’ Managers say ‘very interesting, let’s move forward’ and the meeting ends. Nothing moves forward, because an ‘interesting’ finding contains no action trigger. By contrast, analysis that emerges from a well-defined business question is not interesting — it is either uncomfortable or confirming, and both demand a decision. When a logistics firm asks which routes and which days concentrate delivery delays, the operations manager acts immediately, because the answer lands directly on his problem.
From a total cost of ownership perspective, the cost of analytics investment misdirected by the wrong question is not limited to analyst salaries. When data cleaning time, infrastructure costs, management hours, and opportunity cost are combined, a six-month analytics project pointed in the wrong direction costs far more than initial estimates suggest. Beyond direct cost, the ‘data science doesn’t work’ perception that forms during such projects closes the door on future investments that might actually be well-structured. Any honest ROI calculation must include this second layer of damage.
Another common trap is expecting the data scientist to generate the business question. The analytics team holds technical expertise; knowing which problem carries strategic priority for the company is not their responsibility. That responsibility belongs to the decision-maker. A data scientist may suggest candidate questions, but which question aligns with the company’s primary objective is a judgment call that only leadership can make. In organizations that fail to draw this line, analyst teams gradually begin producing their own agenda — one that tends to be technically satisfying but operationally marginal.
Before launching any analytics project, a decision-maker should be able to answer three questions. First: which decision will I make differently based on the outcome of this analysis? If there is no clear answer, the project is not yet ready. Second: do I know specifically what information is missing to make that decision, or am I simply asking for ‘more data’ in general? Third: when the analysis is complete, does the organization have the operational capacity and authority to act on the findings? Without satisfactory answers to all three, hiring a data scientist is like installing a roof on a building with no foundation.
This article was originally written in Turkish by Gökhan MERCANOĞLU on February 13, 2012 and has been automatically translated into English and other languages using machine translation.