The data generated by business processes hold insights both into the performance of those processes and often also into related (and sometimes not-so-related) business activities. Since Alfred Sloan invented the modern corporation (which largely depends on management’s ability to measure business performance), some level of data analysis has been a fixture in modern business. The advent of the spreadsheet (analytic tool) and the ERP (source of data) took this analysis to the next level. The latest wave of analytic tools (big data repositories, visualization, AI, etc.) have the potential to revolutionize the value that can be derived from analysis:
- Enhance the efficiency of business. I am not just talking about incremental improvements. I have some experience in the construction industry and my feeling is that improvements driven by data analytics could make for double digit improvements in the efficiency of that industry. And, when I outlined my reasoning for my assertion to industry players, I received no push-back;
- Improve customer satisfaction. Sophisticated players, like Amazon, don’t use the data that they have (and they have lots of it) effectively. Amazon regularly lets me down in terms of promised delivery dates and can’t even tell me what they can deliver to my address (which is a PO Box) until after I have agreed to make the purchase. They are obviously spending their Analytics dollars on generating more revenue (more on this in the next bullet) rather than keeping their customers happy. This is an uncharacteristically short-sighted of Amazon;
- Provide additional income streams (for relatively small investments) from businesses that already exist. Often the data generated processing the transactions associated with delivering product (value) to customers provides opportunities for additional revenue. This can come in two forms:
- Add-on income for products that already exist and whose data drives the new income stream. Amazon’s recommendation engine is a good example of this. They use data about your buying habits (or even what you are about to buy) to make recommendations about other things that you may want to buy. This has got to work better than the brick and mortar tactic of stocking “impulse buys” at the checkout counter as Amazon suggests purchases that make sense and are often purchased with what you are already buying, i.e. earphones for the music player you just bought.
- Completely new business opportunities that are made possible by your data, but not directly related to the business you are in. For instance, assume you are in the construction payments business and you suspect that data in your system is predictive of construction activity generally– https://www.chicagofed.org/publications/chicago-fed-letter/2016/366 . You might use this data to set up a business selling these predictions to financial institutions.
Given the potential, I am amazed by the slow adoption (even by what I consider to be sophisticated players) of data analytics. There are reasons for the slow adoption:
- Resistance to the acceptance of new methods, processes and technology. This is always an issue. Companies tend to do things the way they have always been done, just because they have always been done that way. I have personally seen this behavior at large and small, established and startup. I won’t embarrass any of them by naming names, but I have been employed by at least a couple of them;
- But, the problem is bigger than that. There is a serious lack of business people who understand and appreciate (beyond a very narrow scope) the power of the data that their business is generating. This is not inertia, it is a lack of understanding of the possibilities. I had a client in the 1990s who had a wealth of data that we believed had more revenue potential than their primary business. We couldn’t sell the idea and as far as I know, the business is still generating data (from their primary business of processing of transactions), but are not taking advantage of the potential for that additional revenue stream. It just isn’t on management’s radar. And, that is because they don’t have a mindset that says that the data is a product in itself;
- The paucity of technical people (in this case I am referring to data scientists, but this is, to a greater or lesser extent, an issue in all disciplines that provide technology support to business) who have a deep enough understanding of business issues so that they can productively (at a “change the game” level) collaborate with business people and exploit the potential in the data. This problem is subtler than a question of: Does the data scientist understand the business problem and are they able to articulate their position? It has to do with their ability to interact with business leaders and convince them that they are worth listening to.
Technical people not relating to business people is an old problem. We ran into it a Price Waterhouse in the 1980’s and 1990’s. We took the time to train our practitioners how to “play in the board room”. This training focused on both communications and relationship building. It was designed to improve our ability to sell technology related process improvement that could “make a difference”. And, it was successful. PW teams sold and implemented significant and successful process improvement (financial systems, manufacturing systems, supply chain systems) that were strategic to business and required buy-in from top management;
- The focus on tools rather than solving business problems. In the absence of discussions about business problems, data science practitioners, all too often focus on the tools—visualization, AI, “big data” repositories and do themselves a disservice. They either lose their clients entirely or settle to address the smaller issues rather than deal with the game changers;
- The lack complete and well-defined data analytics paradigm and even a widely (or well) understood of the current state of the practice:
- Approach to the work. If you are going to engineer an automobile or a piece of software there are agreed methodologies to accomplish these tasks. In fact, there are several generations of increasingly sophisticated methodologies in each of these disciplines. The methodology provides a process that ensures that you think about all the relevant issues in the design process and guides you to the use of tools that support successful outcomes. Without methodology, the work process is haphazard and likely to produce work products that are of uneven quality. And, the ability to deliver meaningful quality improvement, which requires manageable, repeatable processes, is very difficult if not impossible.
- Agreement on who the players are and who does what. I have long thought that a team has to be made up of members who have complementary skills and (among other things– https://hbr.org/2005/07/the-discipline-of-teams ) are all looking to achieve the same goal. The complementary skills part is particularly important. There are several roles that need to be played on a data analytics effort and my experience is that each of the potential players on the team think that they have all of the requisite skills necessary to deliver successful work product and they don’t. Lack of input (from the relevant players) and teamwork leads to the expected bad outcomes time and again.
- Analytic options (statistics, AI, visualization, etc.)—I am not talking about technologies here, I am talking about ways that people think about squeezing value out of data. And, in my experience, different disciplines (finance, operations, strategy, economics, etc.) tend to gravitate to different toolsets and ignore ones that might be potentially useful. This is where a methodology that enforces a discipline that would require a broad consideration of options would be helpful. For instance, finance tends to use visualization (probably driven from the long-term use of spreadsheets) while economists use statistical tools. Both groups might find AI (in terms of identifying interesting patterns that are not easily discovered by other tools) a productive and profitable toolset.
- The embryonic state of the tools necessary to perform the analytics. While the tools have gotten better (especially related to their outputs), it is not clear in many cases how they work or how they fit into a complete and well-defined view of a data analytics toolset. Further many of the current toolsets are both inflexible and require significant knowledge of their mechanics. These issues leave (when partnered with a lack of methodology as discussed above) many participants taking a very incomplete view of their options and what they can achieve with the data that they possess.
- A shortage of well-trained data scientists. I recently attended a data science conference and was both encouraged (by the number of participants) and disappointed by the lack of sophistication of the attendees, both the audiences and the presenters. I don’t think anyone would argue that the demand for data scientists far exceeds the demand. I would add that those who are currently in the profession and those who intend to join it will need better (depth and breadth) training and better tools (frameworks, methodologies and technology) to be able to deliver the good outcomes that analytics promise.
I close this post by acknowledging that my categorical statements about each of these areas does not properly acknowledge the current state of the art. I do however think that it does represent the general state of the practice and that even goes for players that I would think should be doing better—like Amazon.
So, what do we need to do? That will be the subject of my next post.