Why your predictive analytics and AI projects are failing – and how to transform your success

In a recent trial, Air Canada suggested that an AI-powered chatbot that misled a customer was a separate legal entity responsible for its own actions. The court didn’t buy this argument. But it begs the question: Do businesses understand the risk and implications of deploying artificial intelligence (AI) and large language model (LLM) technologies for customer experience (CX)?

Predictive analytics and generative AI promise to revolutionize business operations by providing insights that can improve decision-making, optimize processes, reduce costs, and enhance CX. Yet, many predictive analytics and AI projects don’t live up to expectations, waste resources and miss opportunities, and damage brand reputation.

According to a recent Harvard Business Review article, most AI projects fail. Some estimates place the failure rate as high as 80 percent – nearly double the rate of corporate IT project failures a decade ago.

Let’s look at the five reasons why these projects struggle and some remedies.

1. Lack of clear objectives and expectations

Many initiatives are driven by the coolness of the technology, initiated in the IT department, and greenlit by businesspeople who lack the savvy to ask the right questions. These projects start without a clear goal, are poorly scoped, and lack focus. In the 2023 Data Science Survey by REXER Analytics, only 34 percent of data scientists said objectives are usually well-defined before they get started.

These projects also result in unclear business outcomes. It’s hard to determine how to objectively measure fuzzy goals such as improved operational efficiency or higher brand value.

Recommendation

Before picking AI and LLM technologies, you must frame the business problem and define what short-term and long-term success will look like. 

Once you define the desired outcomes, plan the metrics to measure business value as part of your project setup.

Focus on translating the AI initiative’s performance to metrics that can be monitored by the business team in collaboration with the data scientists and technical team.

Set clear business goals, identify specific customer and employee pain points within your organization, and choose the technology that best addresses them. Many experts recommend starting with a well-defined, small, internally focused project with a clear set of business metrics to prove its value. This will help you assess the risk and feasibility of your CX AI project.

According to a recent Gartner poll, 38 percent of leaders see improving customer experience and retention as the primary purpose of initiatives to deploy applications trained on LLM.
 

Related Content: Nike’s Andrae Kirkland on the power of AI

2. Data availability and quality

Predictive analytics and AI rely heavily on data. However, many initiatives suffer from low-quality data that may be inaccurate, fragmented, scattered, contain inconsistent information collected from various customer touchpoints, and lack ongoing update mechanisms. This leads to the AI and LLM producing inaccurate results, or “hallucinating.”

The challenge of good data availability involves technical, business, and governance aspects. There are tools that address the technical aspects, but the real challenge lies in operationalizing customer data across the organization.

Operating with unreliable or incomplete data can result in operational inefficiencies and other significant issues. For example, LLMs trained on outdated knowledge base articles and missing company procedures can result in a customer service chatbot providing inaccurate information.

According to Deloitte Digital, only seven percent of the contact centers that offer multiple service channels are able to transition customers between channels seamlessly by providing data, history, and context to the next agent or system.

Recommendation

Invest in data management strategies and practices to ensure high-quality, reliable data is fed into your predictive analytics and AI systems. Create guardrails, frameworks, policies, and guidelines on how to maintain and update data and knowledge base content and fine-tune the models for accuracy while accounting for updates.

Here are some steps to consider:

1. Data strategy: Establish well-defined principles for data collection, storage, analysis, and alignment with business goals; address data governance; and ensure data quality, privacy, and security.
2. Conduct a data audit: Identify and address data quality issues before building your model.
3. Develop internal tools: Manage the entire data lifecycle to maintain data hygiene.

Related content: Why data is the most underappreciated asset in your company

3. Talent

Successful deployment of generative AI requires a diverse set of skills that may not be present within the CX department. They may lack “translators” who can bridge the business and analytics realms by identifying high-value use cases, communicating business needs to tech experts, and generating buy-in with business users. CX leaders should collaborate with counterparts in marketing, sales, operations, and IT to acquire missing skills through training, hiring, or sharing talent.

Key challenges include aligning the work of data scientists across the organization, especially when silos are deeply embedded in the culture, and overcoming change resistance from teams accustomed to existing processes. Other hurdles include finding talent with the right analytics and AI skills and identifying good business cases.

Recommendations

It’s crucial to foster a culture of collaboration, breaking down silos and encouraging open communication between data scientists and business stakeholders. Have management identify the necessary capabilities based on prioritized AI use cases, considering both technical and business functions. Then, enhance the understanding of AI, build internal capabilities, and upskill current employees by investing in education and training.

According to the REXER Analytics 2023 Data Science Survey, 50 percent of organizations are conducting internal training to increase data science skills, and 39 percent are working with academic institutions to offer internships and other work experiences. This comprehensive approach can help you navigate the challenges and reap the benefits of AI and analytics in enhancing CX.

4. Technology

Predictive and generative AI have become mainstream, but there’s still a lack of understanding about these complex technologies, including modeling techniques. Without this knowledge, you may struggle to implement these technologies effectively and realize their full potential for anticipating customer needs, streamlining operations, reducing costs, and driving innovation.

AI uses statistical algorithms and machine learning modeling techniques to analyze historical data and predict future outcomes. Due to this level of complexity, which uses millions of parameters for modeling, implementing AI may require significant upfront investments. These can include technology acquisition, infrastructure, integrating platforms with legacy systems, and talent recruitment.

Recommendations

1. Start with a technology stack inventory, including data stack, LLM tool, processes, cloud storage, and operations, to understand the gaps that require new investment.
2. After defining objective and scope, start with smaller AI project pilots to assess their potential impact before full-scale implementation.
3. Pick a pilot project that is primarily a back-office or employee-facing use case.
4. If you must pilot customer-facing use cases, they should be limited in scope and include human review of the output. Train the model to first build customer trust rather than trying to be first to market with advanced capabilities.

Related content: The state of data and analytics with Caroline Carruthers

5. Governance and oversight

A widespread misperception among business leaders is that generative AI is a plug-and-play technology that yields immediate returns. The reality is that many AI projects face challenges in running LLM with the organization’s own data, integrating AI with existing processes, and aligning AI projects with business objectives.

Using LLMs to generate content that is relevant to the business with little or no customization or fine-tuning will result in responses that are too generic or even irrelevant to the brand’s products or consumer needs.

Often, teams do not establish necessary processes prior to rollout, leading to rushed deployment under market pressure, without clear strategy or adequate oversight. They struggle to identify and advance projects that generate business value, while discontinuing underperforming ones.

Recommendations

For successful AI deployment, establish a governing body comprising business, IT, and analytics leaders. This team shares accountability for the AI and analytics deployment process within your organization. Their responsibilities include:

1.     Establishing a talent recruitment and training strategy

2.     Building partnerships with data and AI service and software providers.

3.     Developing AI standards, processes, and policies.

4.     Overseeing data architecture, data strategy, model development, procedure deployments, and adoption.

5.     Measuring performance and tracking updates.

6.     Maintaining and updating the data and knowledge base content.

7.     Promoting a culture of data-driven decision-making within the organization.

8.     Fine-tuning models for accuracy while accounting for updates.

Importantly, the team must ensure there are well-defined ethical considerations and guardrails against biases. While this article doesn’t address AI biases, privacy, security, and regulatory considerations, your team must include them in the discussion. Failure to adequately do so can lead to reputational damage and legal liabilities.

Implementing a new model often requires significant changes to existing processes. The governing body should manage this change carefully to ensure buy-in from all stakeholders and safely implement the new AI initiative.

Improve the outcomes of your AI projects

Adopting predictive analytics and generative AI for CX is both exciting and challenging for any business. The upside potential of using AI is tempered by the necessity to get it right or risk damage to your business reputation and customer relationships.

The success or failure of an AI project occurs well before launch, as you identify the right pilot use case, set clear goals, ensure initial and ongoing data quality, eliminate internal silos, and establish the governance and oversight to avoid common pitfalls.  Following the recommendations in this article will help you create better outcomes for your AI initiatives.

All Access: Voice of the Customer 2025

Join us to hear from industry leaders, innovators, and CX experts as they share insights, strategies, and tools for harnessing customer feedback to drive meaningful change. 

Register Now