Where is the edge in AI networks?

Where is the edge?

In AI networks, the edge is where inference runs physically closer to where data is created. Inference is introducing new data to trained models to generate outputs in real-world situations.

Rather than relying entirely on a central data centre, inference can take place on or near end-user devices to reduce latency for real-time decisions and boost efficiency and reliability. In addition, using the edge for data processing keeps sensitive data local, supporting data residency and sovereignty requirements.

Since ChatGPT broke onto the scene in 2022, frontier AI labs have focused on training models in large, centralised data centres. Now, as AI becomes embedded in everyday life, inference is becoming the dominant AI workload. It’s forecast to account for around two-thirds of all AI compute in 2026, up from a third in 2023.

But don’t expect all inference to end up on your phone or laptop. The edge isn’t a single location. It’s a continuum between end user and cloud, with distributed inference running in the location best suited to each application.

Far edge

The far edge is the point closest to the end user where data is created or consumed. Think connected vehicles, industrial sensors, production lines, healthcare devices, mobile phones and smart glasses.

Why run inference at the far edge? Primarily for real-time decision-making, resilience and local data processing. Some applications need microsecond response times. Others need to keep operating if connectivity is disrupted. In some cases, privacy, security or data sovereignty requirements demand processing be kept local.

Typical workloads include:

• AI-enabled quality control on production lines
• Computer vision for hazard detection in connected vehicles
• Predictive maintenance on industrial equipment
• AI-assisted diagnostics on healthcare devices
• On-device AI assistants on mobile phones

However, many applications still rely on nearby compute resources for additional processing. That's where the network edge comes in.

Network edge

The network edge sits between far-edge devices and centralised cloud platforms. Examples include factory edge servers, branch office AI gateways and local platforms supporting smart cities, transport hubs and video analytics.

Here, organisations can aggregate data from multiple nearby sites or devices before sending selected traffic to the cloud.

Typical workloads include:

• Connected vehicles and transport hubs
• Smart CCTV aggregating video feeds from multiple cameras
• Retail analytics across multiple stores
• Smart city sensors feeding a local AI platform
• Inference running on an edge gateway for hundreds of IoT devices

With more compute and storage than end-user devices, the network edge can support larger AI workloads while maintaining lower latency than the cloud.

In some architectures, inference also runs in nearby metro data centres, adding compute capacity while remaining close to users and devices.

As AI deployments scale, organisations may run workloads across a mix of edge and cloud environments.

Regional cloud infrastructure

Beyond the edge, AI workloads also run in regional or country-specific cloud infrastructure, including:

• Private and sovereign clouds
• GPU as a Service (GPUaaS) platforms
• AI SaaS platforms
• Hybrid edge-cloud environments

For example, in a hybrid deployment, a manufacturer may run real-time quality-control AI at the network edge. Meanwhile, AI models are updated and performance is analysed in a regional cloud.

Why the need for regional infrastructure? As AI adoption accelerates, it can support growing demand for GPU capacity and larger inference workloads. Crucially for some organisations, it can also help meet data residency and data sovereignty requirements.

Central cloud

While some inference is moving to the edge, the central cloud remains the foundation of AI ecosystems. AI services such as ChatGPT and Claude, public cloud platforms like AWS, Microsoft Azure and Google Cloud, and private cloud environments all rely on large-scale data centre infrastructure.

They’re essential for:

• Foundation model training
• Large-scale AI research and development
• Enterprise AI platforms serving users worldwide

However, distributed AI is set to reshape network traffic, from the far edge to the cloud and core networks. As model distribution moves from the cloud to the edge, inference workloads increase and the data we create grows in both volume and criticality.

The challenge is providing the right connectivity across the whole AI journey.

Connecting AI across the UK

At Neos Networks, we’re using our high capacity, UK-wide fibre network to help hyperscalers, neoclouds and GPUaaS providers extend AI networks across the UK, from far edge to cloud to core networks.