IT leaders risk saddling their organisations with 'silent and expensive' artificial intelligence (AI) agents unless they fundamentally restructure their data infrastructure and enterprise software stacks. That warning comes from Jeff Clarke, vice-chairman and chief operating officer of Dell Technologies, who recently laid out five structural and architectural imperatives to help enterprises successfully scale autonomous agentic workflows without losing control of their budgets or data security.

The rise of AI agents—autonomous systems that plan, execute, and iterate on complex tasks—has created a new set of challenges for enterprises. Unlike simple chatbots or large language models that generate text, agents interact with enterprise systems, modify databases, and trigger business processes. Without proper design, these agents can become costly black boxes that drain resources and introduce security risks.

1. Stop moving data to AI

Clarke's first imperative addresses the scattered state of enterprise data. He notes that while modern AI runs on data, most organisations are grappling with data silos and are unprepared for agentic use cases. 'In most enterprises, the data is scattered across dozens of systems,' Clarke said. 'Eighty to ninety percent of it is unstructured – none of it is connected in a way to effectively power agents at scale.'

Rather than copying and shifting massive datasets to external cloud-based models, Clarke argues that organisations must bring the models to their existing storage. 'To power agents at scale, you need a real-time connected knowledge line, and here’s the structural and architectural decision you’re going to have to make: don’t move the data to the AI, move AI to the data,' he said. 'That’s fundamentally a different approach and, quite frankly, a decision that we need to make now.'

This approach reduces data transfer costs, minimizes latency, and keeps sensitive data within the enterprise perimeter. It also aligns with the growing trend of federated learning and edge AI, where models run closer to where data resides. For many organisations, this means investing in on-premises or hybrid infrastructure that can host AI workloads alongside existing data stores.

2. Design for massive, multi-step inference workloads

While early corporate AI efforts focused on centralized model training, Clarke highlights that the exponential growth in AI inferencing demands an entirely different tier of compute. Unlike simple chatbots, agentic AI systems perform multi-step reasoning, frequently calling various models to plan, execute, and iterate on complex workflows.

'When I think about inference, you have reasoning models executing multi-step chains,' Clarke explained. 'These workloads are 10 to 100 times – some even say 1,000 times – more compute-intensive than what we were running just 18 weeks ago. An AI-native enterprise has to be built for both [training and inference].'

This imperative requires IT leaders to rethink their hardware and software architecture. Inference accelerators, high-bandwidth memory, and optimized orchestration layers are essential to handle the bursty, sequential nature of agentic reasoning. Moreover, organizations need to plan for capacity that can scale dynamically as agent usage grows. Failure to do so results in performance bottlenecks and skyrocketing cloud costs as agents wait for compute resources.

3. Demand a 'receipt' for every autonomous action

Because AI agents do not just retrieve information but actively execute business processes—such as modifying customer records, placing orders, or issuing financial transactions—security can no longer be treated as a passive layer. Clarke calls for enterprise systems to be redesigned to log, track, and reversibly commit every action taken by an AI agent.

'They don’t just call a model; they call your CRM, ERP and financial systems, and they call your customer databases,' he said. 'Every one of those touch points has to be secured, logged and reversible.' He notes that tracking agentic actions is key for business continuity: 'When an agent acts on your behalf – changes prices, updates a customer record or initiates a return workflow – you need to know what it did, why it did it and how to undo it if it got it wrong. In an AI workforce, every action needs a receipt. That’s not a compliance checkbox; that’s how you build trust into the system that will act on its own.'

Implementing this imperative involves integrating immutable audit trails, role-based access controls, and rollback mechanisms into the agent framework. Many enterprises are adopting 'agent observability' platforms that monitor every call and decision, providing a clear chain of accountability. This not only mitigates risk but also helps troubleshoot errors and refine agent behaviour over time.

4. Integrate the enterprise stack

Clarke warns that if an organisation’s existing software stack is not unified through application programming interface (API) orchestration, any deployed AI agents will become costly, isolated failures. 'The agent becomes the coordinator,' he said. 'Agents need to plan tasks, call tools, execute work and handle exceptions across your entire stack. That means an API code architecture, workflow orchestration, and an agent framework that can do multi-step execution.'

Without this level of deep, stack-wide integration, Clarke delivers a blunt warning for CIOs: 'If your stack can’t do that, your agents are going to be siloed and expensive, and that’s simply not a conversation you want to have with the board.'

To achieve integration, enterprises must standardize on modern APIs, event-driven architectures, and low-code integration platforms that allow agents to connect with legacy systems. This often involves breaking down departmental silos and investing in enterprise service buses or API gateways. The goal is to create a 'single plane of control' where agents can seamlessly interact with any system, from CRM to supply chain management, without manual data transfers or custom point-to-point integrations.

5. Master the economics of token routing

Finally, Clarke urges IT leaders to optimize their tokenomics rather than defaulting to the most advanced frontier models for every task. True efficiency lies in routing the right task to the most cost-appropriate model, whether it sits on-premises, at the edge, or in the cloud.

'The question isn’t whether consumption grows – it absolutely will. The question is, are you running the right tokens on the right infrastructure?' Clarke said, adding that generating a routine summary, for example, doesn’t require the use of a frontier model. 'If you do that correctly, you’re going to get optimal performance, privacy and cost efficiency. Run everything in one place because it’s easy, and you’d be ready for a surprise – a large bill that’s only going to get larger. The notion of token routing – where to put that token – is going to be one of the most important decisions we will make,' Clarke concluded.

Token routing involves using intelligent orchestration layers that classify tasks by complexity, latency requirements, and data sensitivity, then direct them to the most suitable model—be it a small specialized model running on edge hardware or a large frontier model in the cloud. This approach can reduce costs by 50% or more while maintaining acceptable performance. Organizations should also consider using model distillation to create smaller, task-specific models that run efficiently on commodity hardware.

The five imperatives from Dell’s vice-chairman provide a roadmap for enterprises looking to adopt AI agents without breaking the bank. By bringing AI to data, designing for heavy inference, demanding receipts for actions, integrating the stack, and mastering token routing, IT leaders can build a foundation for scalable, cost-effective, and trustworthy agentic AI. As AI continues to evolve, those who ignore these structural principles may find themselves struggling with runaway costs and brittle systems that fail to deliver on the promise of autonomous enterprise workflows.

Source: ComputerWeekly.com News