Australian organisations are struggling to get value out of early AI deployments and could be walking into a cost trap, Gartner analysts warned on Monday, urging CIOs to study pricing models and implement a new ‘sandwich’ model to manage risks.
Governments and other public sector organisations in particular need to include additional “trust technologies” that operationalises AI safeguards and minimise a new wave of AI inspired shadow IT.
The recommendations come as Australia finalises outside regulation of AI and the government introduces its own guardrails and urges businesses to adopt voluntary safeguards immediately.
The potentially soaring costs of poorly planned generative AI deployments have quickly emerged as a barrier and in the near term. Nine in 10 CIOs report to Gartner that that managing costs is limiting their ability to get value from the technology.
Costs are now “as big as an AI risk” as security or hallucinations — the incorrect or misleading results AI can produce — Gartner analyst Mary Mesaglio told the Gartner Symposium at the Gold Coast on Monday.
“The first thing to know is that AI investments are not like conventional IT investments. With Gen AI, it’s really easy to waste money… On average in 2023, organisations that are already deploying Gen AI spent between US$300,000 and US$9 million, depending on company size, just in the proof of concept basis,” she said
“These costs are already going up. Vendors are raising prices by up to 30 per cent as they incorporate Gen AI.”
Not understanding the costs of generative AI could lead to a 500 to 1000 per cent error in estimates, according to one finding that dominated the talk at Gartner’s CFO conference in Sydney earlier this year.
The AI cost derailment comes from volatile or poorly understood costs for inference, data preparation and grounding, that typically operate under a token model, Ms Mesaglio said. Organisations need to mitigate them with preparation and, for sophisticated AI use, continuous cost monitoring.
Where possible, organisations can also potentially save millions of dollars by developing their own front end or interface with an AI API instead of procuring a full AI package, she said.
But there’s an even easier way to avoid an AI cost blowout: don’t use it.
“People are using gen AI foundation models to answer problems they should be using search for,” Ms Mesaglio said.
“You should not be using a Gen AI foundation model to be like, ‘What’s the capital of Costa Rica?’ You just cut down… 10 times more trees by asking a foundation model [rather] that than asking Google search!”
The rising costs of generative AI are on the Australian Parliament’s radar, with experts earlier this year warning of the intense energy requirements for data centres being driven by the technology and calling for more transparency.
The rising costs are also coming as AI struggles to deliver, with nearly half of CIOs reporting that the emerging technology is not meeting ROI expectations.
This is because of a mix of high costs, blunt implementation, time savings not necessarily translating to productivity and a host of new issues driven by the technology, Gartner analysts said at Monday’s conference.
But CEOs are more bullish than ever about AI, heaping pressure on tech leaders to deliver.
“For years, only around 20 per cent of CEOs said that AI was the tech that would most impact their industry,” Ms Mesaglio said.
“Then in 2023 that number leapt to 59 per cent and in 2024 it leapt again to 74 per cent.
Gartner is recommending a clear-eyed approach to AI capabilities and risks.
Organisations need to decide which of the main AI benefits — employee productivity, process improvement or business model innovation — they want out of the technology. They must also have a plan for how the benefits will translate into business outcomes.
“Employees here at [Australia and New Zealand] who have successfully integrated AI in their daily work say that they get a little over three hours of time back per week. That’s awesome, right? That’s 37 minutes saved per day,” Ms Mesaglio said.
But if the employee does not use that time saved elsewhere it becomes what Garner terms a “productivity leakage”.
“Gartner expects you to experience 10 to 30 per cent leakage, depending on the use case,” Ms Mesaglio said. “The bottom line is that time saved doesn’t necessarily need increased business benefits.”
To deliver results while managing costs and other risks, Gartner is urging CIOs to deploy an AI ‘sandwich’ model.
Unlike a bottom up tech stack, the sandwich model recognises the cross flow of data and AI applications, including an ability to pull information from unstructured data sources.
But the potential productivity savings come with additional risks like ‘BYOAI ‘ – a new wave of shadow IT that Gartner says reflects different parts of the organisation deploying their own AI software – and new behavioural challenges like ethics and privacy.
Public sector organisations are being urged to shore up their sandwich by limiting BYOAI and using the most trust technologies. The off the shelf applications can automatically stop AI use that may breach policies or ethical standards.
“Just because you wrote something down in an AI policy doesn’t make it true or enforced,” Ms Mesaglio said.
“[Trust technologies] make sure your sales forecast doesn’t get out or any other enterprise data. For that matter, they can also be output filters, looking to filter out violent language, swear words, or straight up hallucinations.”
Joseph Brookes attended Gartner IT Symposium/Xpo as a guest of Gartner
Do you know more? Contact James Riley via Email.