The size of the Deep Learning and AI models has increased substantially within the past couple of years. With recent advancements specifically around NLP/Conversational AI and Computer Vision applications powered by large scale models such as BERT, GPT-3 and Vision Transformers (ViT) having hundreds of millions to billions of parameters, deployment and management of such models is becoming challenging. In this talk, I will go over the state-of-the-art models, their Edge AI applications, deployment concerns and approaches on how to leverage them on Edge computing. I will share my experience of deploying such large models from Amazon AI, Uber AI, and Got It AI.

Machine vision workloads are complex, and their performance requirements often present challenges in areas like latency, security, energy use and reliability.  Hierarchal partitioning of those workloads often makes sense, where the machine vision software is split into multiple stages (for example, contrast enhancement, feature extraction, object recognition, threat detection), which are run at different layers of the [intelligent camera -> edge node -> MEC -> cloud] hierarchy. 

This talk will introduce the hierarchal cloud - edge architecture, and discuss the properties and capabilities of its many layers.   It will propose an example segmentation of machine vision algorithms, and investigate the tradeoffs of how we can map them onto the various layers of processing available in the hierarchy.  Finally, it will look at the dual flows of model training and inference for AI applications, and discuss which portions of those flows make sense in different edge layers, and how they can be secured, orchestrated and managed.

The adoption of edge computing is enormous in the automotive industry. So, what lessons have been learned that other industries can adopt and adapt for their own rapid innovation?