Citi^® Gen AI Summit 2024 Takeaways – The Gen AI Customer Perspective: Hype vs. Reality

For this year’s Citi^® Gen AI Summit, I had the distinct pleasure of hosting a talk exploring which parts of the Gen AI revolution have and haven’t been overhyped in the almost two years since the launch of ChatGPT.

To get enterprise tech buyer perspectives from across industries, I spoke with Sean Wechter, the CIO of software integration and automation platform Boomi; Arni Raghvender, Director of GenAI GTM at AWS; Elena Alikhachkina, Chief Data & Analytics Officer at TE Connectivity, an industrial technology manufacturer; and Burak Gokturk, VP of Engineering at Google Cloud AI. The panelists discussed use cases they've been exploring, where enterprises are in their Gen AI adoption journeys, and strategies they can employ to ensure that LLMs produce factual outputs, among other topics. Below are a few of the insights that I found the most illuminating.

Some Gen AI solutions are overhyped, but the tech behind them is the real deal

While certain solutions on the Gen AI market appear to be more hype than reality, the unanimous opinion among the panelists is that Gen AI is a truly transformational technology. It is already showing promise in several use cases, and enterprises across industries are exploring numerous other applications, including:

• Aggregating and classifying engineering documents while remaining compliant with different countries’ trade laws
• Helping sales teams forecast revenue more accurately
• Helping talent recruiters make better hiring decisions by building interview plans, taking automated notes and monitoring equitable hiring practices
• Leveraging internal-facing chatbots with a human in the loop

However, LLM outputs aren’t 100% factual on their own — yet

Along with their incredible reasoning and creative capabilities that demonstrate real business value across various use cases, LLMs come with one serious challenge: hallucinations, or outputs that are erroneous, irrelevant or inconsistent with the input (aka prompt). The larger a model is, the better it is at reasoning but also the more likely it is to hallucinate. And this problem isn’t limited to one model or another, either: all large Gen AI models are prone to giving these confidently incorrect answers, which can have dire consequence for sectors where accuracy is paramount, from healthcare to finance.

To overcome this, enterprises are beginning to realize that LLMs must be grounded through processes such as retrieval-augmented generation (RAG), which optimizes a model’s output by having it draw from a knowledge base outside of its training data. Meanwhile, the big players in LLM development are increasingly focused on improving model accuracy.

Prepare to spend 90% of your company’s Gen AI resources on the last 10% of the adoption journey

The tremendous Gen AI enthusiasm of the last 18 months has led to an explosion of proofs of concept (POCs), with enterprises exploring what the tech can do and where it can add real business value. As companies begin moving the winning solutions into production, they might be tempted to think that most of the hard work of vetting them has been done. But POCs are almost always successful because they are focused on solving a finite problem, and (as discussed above) non-deterministic Gen AI systems carry significant levels of unpredictability.

So rather than trying to implement every use case at once, our panelists recommended that enterprises focus on getting one or two into production, making sure they’re absolutely clear about the business value and putting all the necessary pieces in place to reduce hallucinations — because around 90% of a company’s resources will likely be spent on the last 10% of the adoption journey.

Gen AI will change how businesses are built for decades to come

Though it’s still early days, the panelists displayed unambiguous optimism about Gen AI as a groundbreaking technology. While some solutions out there are indeed marketing ploys tied to the hype cycle, enterprise tech buyers are preparing for a paradigm shift in the way businesses are built and agree that Gen AI adoption could become a make-or-break factor in a company’s competitive advantage moving forward.

Are you a founder in the enterprise tech space? I’d love to get in touch. Please reach out to me at matt.carbonara@citi.com