CDNs are central to enhancing the delivery of AI-augmented customer experiences

Generative AI can improve customer service and experience, but it needs the support of edge infrastructure to truly deliver on that promise.

Generative AI is recognised as a key enabler for transforming the user experience. The introduction of natural-language interfaces, both text and voice, offers a new way for users to interact with digital applications and content services such as video-on-demand. Meanwhile, behind-the-scenes generative AI is capable of similarly transformative change, helping - for example - to answer the questions of users more quickly, or helping developers to code new feature releases and ship them much faster. 

Success or failure in this space hinges on both the efficacy of the large language model, and on the ease with which this AI-augmented experience can be delivered through to users - and meets their needs.

The current trend is to heavily rely on AI models served out of the public cloud. While this can lower technical barriers to entry, it is heavily dependent on a reliable and stable network connection being maintained at all times between the cloud and customers, to ensure quality of experience can be consistently delivered. 

As AI becomes an important and integrated part of engaging with audiences, adopters of the technology are starting to consider alternative or hybrid setups to deliver that consistent, intelligent, AI-augmented experience through to users.

One of those alternative setups is edge AI, which combines artificial intelligence algorithms with edge computing to enable instant, low-latency AI processing on local devices without the need for cloud communication. 

Leveraging a modern content delivery network (CDN) to put AI-enabled experiences into the hands of customers in a performant and secure way can have a transformational impact on such initiatives. 

An expanded remit for CDN infrastructure

The idea that edge infrastructure and CDNs have an elevated role to play in getting AI-augmented experiences to users should not come as a complete surprise, given the traditional role of CDNs in media and digital application delivery.

CDNs distribute content across a network of globally-located servers located near users, caching data at these edge locations and routing user requests to the nearest server to deliver content quickly and efficiently.

By moving and distributing applications and content closer to end users, edge computing and CDNs significantly reduce latency and enhance real-time application performance, which is critical for providing personalised experiences that are performant, engaging, and safe.

Edge infrastructure itself continues to be enhanced to create new experience optimisation opportunities. 

One of these opportunities is to leverage multiple CDNs, and seamlessly switch between CDN providers, even in the middle of a user streaming for example video on-demand, without any interruption to their service. A switch of CDN of this nature might be warranted to maintain the highest video quality for users at all times. 

Underpinning this is WebAssembly (WASM), which is emerging as a critical tool for edge application development. WASM acts as a unifying technology for edge platforms, enabling application owners and experience providers to write portable, high-performance code that can run across multiple platforms. It is supported by multiple CDN providers, including Fastly and Google Cloud.

While WASM is a powerful technology, CDN providers are not just limiting innovation to their own ecosystem. Leaders in the space are increasingly focused on helping application developers and digital experience owners to embrace and incorporate AI into their offerings, in a way that meets or exceeds end user expectations for the technology.

Semantic caching for the experience win

As businesses embrace GenAI to enhance user experience, being able to scale up and out those AI-enabled experiences brings challenges such as latency and increasing costs.

While AI is helping developers create many new experiences, too often today's AI platforms make users wait for responses to be returned to them or for the AI to trigger a certain action.

Semantic caching is one of the ways that edge computing is helping to increase the performance of interactions with AI tools within the digital services users are interacting with. 

Generally, every time a user has a question, they might type it into a GenAI chatbot or virtual assistant, from where it's passed through to the large language model (LLM), which processes it and returns an answer. 

However, depending on the context of the question being asked, this traffic round trip might not be necessary. Instead of constantly hitting the original LLM API, semantic caching breaks queries down into smaller, meaningful concepts, which can be used to understand and respond to similar questions in the future — even if the questions aren't identical, but are just semantically similar.

By caching responses, semantic caching amps up performance and reduces query time. This translates to less waiting time for users, and is an example of how edge computing technology is helping to streamline AI-augmented experience delivery.

The combination of edge computing and AI is unlocking new opportunities for businesses across industries. Innovations such as WASM portability and semantic caching are proving transformative. 

As companies continue experimenting, the edge will undoubtedly play an integral role in the future of secure, scalable, and intelligent digital experiences.