Enterprises have spent the past two years wrapping their heads around the basics of using large language models and AI agents. But ready or not, the need to manage AI agents at scale means another major upheaval is just around the corner.
As agents proliferate, risks rise and token costs strain budgets, business leaders will want greater control and more affordable alternatives to frontier AI models and hosted services.
That means the engineers already managing internal developer platforms will have to decide where a growing number of workloads will run. They’ll need to route each workload to the most cost-effective agentic model or service across public cloud and on-premises infrastructure – while preserving performance, resilience, security and governance, of course.
At some companies, platform engineering emerged ad hoc in response to the last decade’s cloud-native and DevOps trends. Departments and development teams ended up with separate platforms tailored to their preferred tools. That will come to an end as the risks of running agentic AI intensify and further thrust platform teams onto center stage within enterprises, said Torsten Volk, an analyst at Omdia, a division of Informa TechTarget.
“The biggest architectural challenge will be the consolidation of all the heterogeneous agents and shadow agents built by people at home and at work onto one centrally managed and governed runtime,” he said. “Platform engineering teams will start owning agentic AI … and the on-premises roadmap in many enterprises by the end of 2027.”
Here are the key trends that will lead to an expanded role for platform engineers throughout the year:
1. AI inference changes where and how workloads run.
The shift from training AI models to AI inference, in which trained models deliver fresh outputs based on new data, will dramatically change infrastructure design requirements in 2027.
AI model training workloads require consistent throughput among tightly synchronized sets of compute, storage and networking resources. This situation results in the growth of mammoth AI data centers where advanced hardware can be clustered closely together.
AI inference can be accomplished using more accessible hardware, but places more emphasis on keeping workloads close to users to minimize network latency, particularly for industrial and robotics applications.
Experts predict AI inference will be the biggest driver of self-managed AI infrastructure, on an increasingly diverse set of chips and at increasingly distributed locations in 2027. According to Forrester Research’s “Predictions 2027: Prepare For New Infrastructure Demands” report, a quarter of AI infrastructure RFPs will require non-Nvidia accelerator options next year, while infrastructure investments will shift by 20% to industry-specific edge devices for organizations in manufacturing, healthcare and retail industries.
2. AI risks intensify.
Growing risks and the need to protect sensitive proprietary data will prompt some CEOs and CIOs to bring AI workloads in-house and under their direct control. As a result, Forrester’s “Predictions 2027” report estimated that 30% of large enterprises will build private AI infrastructure for inference next year, putting AI workloads behind both brick-and-mortar and technical barriers.
The stakes for executives are financial, from whether AI investments pay off to what it would cost the company if an agent takes unauthorized action.
“In 2027, at least one CEO in the Fortune 500 will step down or be fired because they have failed to achieve a positive financial return from their AI investments,” predicted Omdia analyst Scott Sinclair. “2027 will also feature multiple news headlines of enterprise AI agents misbehaving in enterprise organizations, leading to material financial impacts and making AI governance and accountability a top three CEO priority.”
Especially for regulated enterprises, lacking the ability to prove to auditors that sensitive data will never be accessed by another company to train its AI “makes them extremely uncomfortable,” said Antonio Piraino, chief strategy officer at Ionic Digital, during a panel session at the Yotta conference September 30.
Data gravity is also a factor; regulated firms already tend to store their most sensitive data on-premises, and moving it to a hosted AI service would quickly become unworkable, Piraino said.
“You’re going to have to go back, find the data on-prem, yank it out, put it in some massive table with data that you collected from everywhere else, run the model, then test it, then yank it again, and eventually there will be drift,” he said.
3. Token costs and open models shift build-vs.-buy decisions.
Token costs have already begun to complicate AI build-versus-buy decisions for enterprises, but cost overruns will continue to rise as enterprises deploy more AI agents, according to Gartner’s “Top AI Trends in Infrastructure and IT Operations for 2027” report published in September.
“Through 2027, enterprises that implement AI agents in IT operations using uncapped variable pricing models tied to token usage will exceed initial cost expectations by over 50%, compared to those using predictable subscription and bundled models,” the report said.
One enterprise IT pro also predicted that cloud tokenomics will only get worse as AI companies are required to show profitability to investors as they mature, rather than just growth.
“I’ve lived this working at [companies] where investors suddenly want profitability, not scale or growth,” said Christopher Maher, director of AI/ML operations at Ashley Furniture Industries. “There’s going to be an upcharge. Things are going to get more expensive. So, this is the time to fully own that cost yourself.”
The rise of open source utilities such as llm-d, which can optimize the deployment of more powerful open-weight models, will also strengthen the case for running AI in-house, according to Gartner.
During a breakout session at KubeCon North America on November 11, Maher demonstrated another example, LLMKube, an open source project he created to bring Mac laptops into Kubernetes clusters to host AI models. Other companies have begun doing this as well to contend with server supply shortages.
“Most of my inference and all my home lab stuff is now happening on my hardware [rather than in the cloud], and that’s a recent shift over the last few months,” Maher said. He also expects a similar shift to happen in data centers.
“There’s going to be that inflection point, especially where these renewals are coming up for companies, where all of a sudden, it’s like, ‘Whoa! I was paying pennies for this. Now you need, like, 5X the money at scale.'”
4. Data center politics hinder infrastructure expansion.
However, the trends pushing AI workloads toward on-premises, self-managed infrastructure will also encounter major roadblocks. For example, the 2026 midterm elections reflected ongoing public opposition to large data center buildouts.
As new officials take office in January, data center operators – including large enterprises interested in bringing AI workloads in-house – will face continued uncertainty over project approvals and timelines. In October, Gartner predicted massive AI infrastructure write-downs and consolidation by 2030, particularly among neocloud providers. Forrester further predicted this month that data center backlash will also force at least two hyperscalers to cancel projects.
5. Hardware shortages constrain enterprise deployments.
Data storage teams have had to contend with rising memory prices and a scarcity of new storage capacity throughout 2026, and this will continue to affect pricing for IT buyers through 2027, according to Omdia analyst Simon Robinson.
“Critical memory and storage component shortages – especially DRAM and NAND – persist into 2027, continuing to throttle enterprise IT ambitions and limiting the extent of some enterprise on-prem AI infrastructure rollouts,” Robinson said.
6. Skills shortages complicate in-house AI plans.
As enterprises weigh insourcing AI infrastructure, the staff to support such efforts might also be in short supply. Bringing AI infrastructure in-house means investing in people who can run it, an expense some companies are better able to absorb than others, said Scott Robohn, CEO at technology consulting and education firm Solutional.
“The ones with money, and largely financial services, are the ones that will probably be the first to bite there,” he said. “It’s not impossible, and it becomes more and more tractable every month. But it does mean investing in human resources, and that’s not trivial.”
For companies that can’t hire or train enough people to run AI infrastructure themselves, public cloud may be the more practical choice. Even as some enterprises invest in-house, IDC expects public cloud infrastructure to continue growing faster than private cloud and on-premises infrastructure over the next five years, according to analyst Matt Flug.
“We have public cloud deployments growing in the 20% range over the next five years; private cloud and on-prem we have growing at around 6% or 7% CAGR,” Flug said. “Up until last year, we were projecting negative growth for private cloud and on-prem, and CAGR has actually come down or stayed flat for public cloud. So, public cloud is still growing much faster, but on-prem private cloud has sped up.”
7. AI gateways centralize management across environments.
Ultimately, the future of AI infrastructure will be decidedly distributed. Gartner forecasts that by 2029, 80% of large enterprises will operate production AI workloads across at least three infrastructure environments, including public cloud, neoclouds, on-premises data centers and edge locations.
In anticipation of this potential pattern, a new infrastructure layer has emerged over the past two years to centralize management and route AI workloads accordingly: the AI gateway. This infrastructure component will be the key element keeping AI agents in check for enterprises going forward, Flug said.
“In the past two and a half months, I have not had a conversation with a vendor or a user that has not mentioned in some way an AI gateway or control plane that sits outside of the hyperscalers and the big vendors,” he said.
8. AI governance and context requirements bring IT disciplines together.
As ‘rogue agents’ repeatedly proved in 2026, any governance mechanism agents can directly access within their applications, or orchestration harnesses, can be escaped or manipulated.
As a result, agentic security will increasingly be anchored in infrastructure, namely networks, according to Gartner, which predicted in its AI infrastructure trends report that by 2030, 25% of enterprises will merge their network operations and security operations centers, up from 5% in 2026.
It has also become clear that without a unified, well-maintained data context layer, AI agent output quality will not meet enterprises’ needs. Context engineering will bring together data storage and data management disciplines in 2027 and beyond, as enterprises stitch together corporate data and unify disparate data repositories, according to Rob Strechay, lead analyst at VB Intelligence, in an interview.
“The challenge becomes how do you manage islands of automation that we’re now recreating [with AI], and that’s going to be a big story next year,” Strechay said. “The big question is, where does context live? Not just the technical metadata, but all the [business] data metadata. Are you going to rely on a vendor who’s building it into their infrastructure?”
9. Centralized platform teams become critical to enterprise AI success.
All of the above trends add up to one thing: an increasingly crucial role for platform engineering teams in the next year and beyond. In the absence of centralized infrastructure, a centralized organization must serve as a means of controlling the chaos.
In fact, according to Gartner’s September report, “Through 2030, organizations that don’t have a strong Infrastructure & Operations (I&O) function that drives AI infrastructure design, architecture and support will be over twice as likely to fail in their AI initiatives.”
Enforcing enterprise governance and compliance, including FinOps, is increasingly reliant on the observability data owned by platform teams and will push large organizations to centralize what had often been disparate platforms, according to IDC’s Flug.
“With the need to govern agents [with] that observability layer where you can see everything and attribute it to a team or a person, teams need one application platform for the entire organization,” Flug said.
At companies such as CVS Health, where centralized platforms are already in place, the ability to assign prescriptive controls to workloads on the platform has been key to enabling agents to access it and, eventually, will allow the company to host its own AI agents there, too.
“Agents consuming [services] and able to create [code] on behalf of a human have a very deterministic set of options from which to choose,” said Jim Beyers, vice president of enterprise developer experience at CVS Health. “It’s a very, very limited list – it’s not the Cheesecake Factory menu.”
The platform team and security teams already work closely together at CVS Health, too, as was the principle behind the DevSecOps movement in the past. For many organizations, this collaboration still hasn’t quite come together, but AI governance will force it to happen, predicted Omdia analyst Melinda Marks.
“There are bits of this that give me flashbacks to the move to cloud-native [infrastructure], where devs and DevOps were getting into security, but then they lacked consistency across teams and hit walls when they needed to scale,” she said. “We’ve learned from that … and AI adoption depends on security, so I think teams are trying harder to collaborate and align with business goals.”
10. Costs and trust concerns slow AgenticOps adoption.
To keep pace with the rapid change in agentic environments, infrastructure vendors have begun offering AgenticOps products that claim to speed IT operations management.
But IT ops teams are usually among the most conservative in enterprise IT organizations, and the cost and complexity of ensuring AgenticOps agents perform well means adoption will continue to evolve slowly over the next year, Flug said.
“Just as important as building that trust over time is the humans learning what they need to protect against – we’ve never done this before,” he said.
It doesn’t help the outlook for AgenticOps that Gartner’s report identified IT Service Management (ITSM) workflows as among the most costly agentic workloads.
“Token consumption can become extremely high in tasks such as root-cause analysis, where the process is not completely defined, and a significant amount of data may need to be retrieved,” according to the report. “Additionally, AI ITSM consumes many tokens given the amount of tickets generated, and AI agents will likely read each ticket.”
As a result, a Forrester Research IT automation predictions report concluded that enterprises will revert 20% of agentic workflows to deterministic processes in 2027 to save money.
However, if platform teams can’t respond quickly enough to keep up with shifting infrastructure trends, shadow AI will proliferate, Gartner’s report warns.
“With the cloud, organizations bypassed I&O with shadow IT since I&O was seen as slow to respond, ineffective and even an obstacle,” according to the report. “Some have the same expectation with agentic AI, where organizations are looking to bypass I&O and look to the data and analytics, software engineering and AI teams or shadow AI to fill the void.”
Beth Pariseau, lead editor, infrastructure news for TechTarget, is an award-winning veteran of IT journalism. Have a tip? Email her or connect on LinkedIn.
