NVIDIA brings AI to the edge with EGX platform

NVIDIA wants to bring real-time artificial intelligence to global industries – and to do so, it has launched a new accelerated computing platform that allows companies to perform low-latency AI at the edge.

Companies will now be able to perceive, understand, and act in real-time on continuous streaming data between 5G base stations, warehouses, retail stores, factories and other locations.

According to IDC, the year 2025 will see 150 billion machine sensors and IoT devices will stream continuous data that needs to be processed, so edge servers must be distributed worldwide to process that data in real time.

"Enterprises demand more powerful computing at the edge to process their oceans of raw data — streaming in from countless interactions with customers and facilities — to make rapid, AI-enhanced decisions that can drive their business," comments NVIDIA vice president of enterprise and edge computing, Bob Pette.

NVIDIA launched the new accelerated computing platform, called NVIDIA EGX, to help solve those challenges. NVIDIA EGX was created to meet the growing demand to perform instantaneous, high-throughput AI at the edge — where data is created – with guaranteed response times, while reducing the amount of data that must be sent to the cloud.

EGX starts with the NVIDIA Jetson Nano, which in a few watts can provide one-half trillion operations per second (TOPS) of processing for tasks such as image recognition. It spans a full rack of NVIDIA T4 servers, delivering more than 10,000 TOPS for real-time speech recognition and other real-time AI tasks.

NVIDIA has also partnered with Red Hat to integrate NVIDIA Edge Stack with Red Hat's enterprise-grade Kubernetes container orchestration platform, OpenShift.

NVIDIA Edge Stack is optimized software that includes NVIDIA drivers, a CUDA Kubernetes plugin, a CUDA container runtime, CUDA-X libraries and containerized AI frameworks and applications, including TensorRT, TensorRT Inference Server and DeepStream. NVIDIA Edge Stack is optimized for certified servers and downloadable from the NVIDIA NGC registry.

"Red Hat is committed to providing a consistent experience for any workload, footprint and location, from the hybrid cloud to the edge," says Red Hat's chief technology officer Chris Wright.

"By combining Red Hat OpenShift and NVIDIA EGX-enabled platforms, customers can better optimize their distributed operations with a consistent, high-performance, container-centric environment.

NVIDIA EGX also includes the full range of NVIDIA AI computing technologies with Red Hat OpenShift and NVIDIA Edge Stack together with Mellanox and Cisco security, networking and storage technologies.

NVIDIA Edge Stack connects to major cloud IoT services, and customers can remotely manage their service from AWS IoT Greengrass and Microsoft Azure IoT Edge.

"Azure IoT Edge helps customers deploy cloud service to their IoT devices quickly and securely," says Azure IoT Edge director Sam George.

"We look forward to supporting NVIDIA's EGX edge platform on Azure IoT Edge devices so that customers can deploy AI workloads targeting EGX-compatible hardware.

Additionally, NVIDIA EGX supports an ecosystem of software solutions, including video analytics from companies such as DeepVision.

So far, early adopters include BMW Group Logistics, Harvard Medical School, and Seagate Technology.

EGX servers are available from global enterprise computing providers ATOS, Cisco, Dell EMC, Fujitsu, Hewlett Packard Enterprise, Inspur and Lenovo.

They are also available from major server and IoT system makers Abaco, Acer, ADLINK, Advantech, ASRock Rack, ASUS, AverMedia, Cloudian, Connect Tech, Curtiss-Wright, GIGABYTE, Leetop, MiiVii, Musashi Seimitsu, QCT, Sugon, Supermicro, Tyan, WiBase and Wiwynn.

NVIDIA EGX servers are tuned for NVIDIA Edge Stack and NGC-Ready validated for CUDA-accelerated containers.