Thanks in large part to the technologies associated with the fourth industrial revolution, organisations are generating growing torrents of data.
Initially seen simply as a by-product of operations, it is now viewed as an asset, one that enables better, more timely decision-making. However, organisations often underestimate how profound the change to data-driven decision-making will be.
While there will always be a place for expert ability to generate counter-intuitive insights, having a handle on the data provides immeasurable value in a business environment that is increasingly competitive and staggeringly complex.
As a result, CIOs are committing more and more spend to business intelligence (BI) to provide business with the capability to support better decision-making based on facts and not gut feel.
According to Statista, revenue for the BI software market will reach $25.73 billion this year and $34.16 billion by 2028, due to a forward compound annual growth rate of 5.85%.
But these very BI projects fail at an astonishingly high rate. According to Gartner, 70%-80% of corporate BI projects fail, implying this mission-critical sector is marginally worse off than its software peers.
CIOs are committing more and more spend to BI to provide business with the capability to support better decision-making based on facts and not gut feel.
The Standish Group estimates over 70% of all software projects do not deliver value. Earlier research from McKinsey and the University of Oxford suggests half of all large IT projects run 45% over budget, 7% over time and deliver 56% less value than planned.
In short, despite its acknowledged importance to the future of the corporation, BI remains subject to the same ups and downs as any other software project.
To further complicate matters, BI is an area characterised by significant, ongoing change − the current disruptions caused by the emergence of artificial intelligence (AI) software based on large language models, such as ChatGPT, is the most recent case.
More change will come as generative AI, which more closely replicates the human brain, comes to maturity, and on into the future. The point is that change will continue to be a hallmark of the BI space, and companies will have to continue investing in new BI tools or upgrading existing tools to take advantage of the changes.
Because companies ultimately don't have the option to stop investing in new generations of BI tools, they will have to get better at making these important projects successful. Failed or partially successful BI projects are not just a waste of expenditure, they imperil the company's ability to compete.
This is particularly true in the fast-moving consumer goods space, which generates huge amounts of product and customer data and must also respond rapidly to changing consumer demands in a cutthroat market.
Finding a solution
BI projects fail for multiple reasons; however, when viewed in the overall context of big technology project failures, this is not a unique situation, to put it mildly.
Forbes reports Gartner findings that digital transformation projects fail 70% of the time, with McKinsey reporting that only 30% of digital transformation projects result in improved corporate performance. Forbes confirms technology projects fail at an astounding rate − at enormous cost to the companies. Possibly the harshest finding is 90% of such projects fail to deliver any measurable ROI.
It's hardly surprising BI is no exception to what appears to be the rule and difficult to get off the starting block when one major set of challenges in the BI project stakes relates to user buy-in and adoption.
If the people who are going to use the software do not understand why they should use it and what the benefits are, the BI investment will not be realised. A related issue is training: can employees actually use the new software?
Then there's the question of culture. Organisations are facing a mammoth culture change as they move towards making data-driven decisions rather than ones relying on instinct or habit.
An overarching requirement is for focused and repeated communication to ensure everyone understands the project goals. Communication is often sketchy at best.
Another issue is that BI projects are intrinsically disruptive, and thus pose substantial risks for the organisation.
These challenges can be overcome by an effective change management programme. User adoption and buy-in, overcoming resistance to change, driving change in organisational culture, and ensuring everybody gets the necessary training and uses it, will all benefit from a well-thought-through change programme. None of these things will happen organically − they need to be planned, executed and then, crucially, assessed.
Having a formal, well-resourced change management programme in place will also help with the second set of challenges related to business disruption and risk. Potential risks and disruptions can be identified proactively, and strategies developed to minimise the impact.
A caveat is that change management is resource-intensive and cannot be handled as an add-on to an existing workload. It also needs to be done professionally, particularly when it comes to assessing its effectiveness and rolling it out again and again for further projects.
For this reason, it makes sense to use a third-party change management firm to undertake this work. In this way, the process will be professionally executed using tried-and-trusted techniques − and a service provider can be held accountable in a way that internal, likely overstretched, resources cannot.
A new BI tool is an investment in the organisation's future − and given the rate of change, that will have to be repeated as new capabilities become available. It's vital all these investments deliver a proper return; professional change management is the best way of ensuring they do.