Agentic experiences often depend on dozens or hundreds of chained micro-inferences per session. Even small delays stack up, making experiences slow and brittle.
Training is computationally heavy and bursty, typically run in discrete cycles. Inference is continuous, driven by user interactions, and can involve multiple dependent calls per engagement.
It is an infrastructure approach designed for real-time, distributed, latency-sensitive inference at global scale, using highly distributed GPUs combined with edge-native decisioning.
Centralized AI factories for training, fine-tuning, and heavyweight or “one-shot” inference.
A distributed GPU layer near users for real-time, latency-sensitive inference.
An edge routing and security layer to evaluate, secure, and route requests before they reach GPUs.
It validates and classifies incoming requests, filters threats and bots, handles token security and privacy-sensitive traffic, and routes requests to the best GPU location based on latency, cost, and availability.
Placing GPUs near population centers reduces latency, increases concurrency, and minimizes long-haul network travel, which is critical for real-time inference and agentic orchestration.
Workloads that need real-time responsiveness and run close to users or data, including agentic workflows, multimodal applications, and demanding media/video intelligence scenarios.
Akamai platform analytics suggest that 10–15 ms of added delay can increase abandonment during critical retail workflows, which becomes more pronounced when micro-inferences are chained.
It outlines phases: distributed inference enablement first, then real-time multimodal intelligence, then fully agentic applications that can retrieve data, plan tasks, and collaborate with other agents.
