DataCenterNews Asia logo
Specialist data center news for Asia
Story image

GTC18 - NVIDIA ready to go all out on inferencing

By Ben Moore
Wed 28 Mar 2018
FYI, this story is more than a year old

NVIDIA has announced a series of new technologies and partnerships that expand its potential inference market while lowering the cost of delivering deep learning-powered services.

An inference engine is the aspect of AI that allows a device to figure out new information based on a set of rules and what it already knows.

"GPU acceleration for production deep learning inference enables even the largest neural networks to be run in real-time and at the lowest cost," says NVIDIA accelerated computing vice president and general manager lan Buck.

"With rapidly expanding support for more intelligent applications and frameworks, we can now improve the quality of deep learning and help reduce the cost for 30 million hyperscale servers."

TensorRT 4, the latest iteration of the inference optimiser, offers highly accurate INT8 and FP16 network execution and can be used to optimise, validate and deploy trained neural networks in hyperscale data centers.

NVIDIA Tesla GPU-accelerated servers can replace several racks of CPU servers for deep learning inference applications and services, freeing up rack space and reducing energy and cooling requirements.

The company says that the new software delivers up to 190x faster deep learning inference compared with CPUs for common applications such as computer vision, neural machine translation, automatic speech recognition, speech synthesis and recommendation systems.

Google and NVIDIA engineers have also integrated TensorRT into TensorFlow 1.7, making it easier to run deep learning inference applications on GPUs.

“The TensorFlow team is collaborating very closely with NVIDIA to bring the best performance possible on NVIDIA GPUs to the deep learning community,” says Google engineering director Rajat Monga.

“TensorFlow's integration with TensorRT now delivers up to 8x higher inference throughput (compared to regular GPU execution within a low latency target) on NVIDIA deep learning platforms with Volta Tensor Core technology, enabling the highest performance for GPU inference within TensorFlow."

NVIDIA engineers have worked with Amazon, Facebook and Microsoft to ensure developers using ONNX frameworks such as Caffe 2, Chainer, CNTK, MXNet and Pytorch can now deploy to NVIDIA deep learning platforms.

NVIDIA partnered with Microsoft to build GPU-accelerated tools to help developers incorporate more intelligent features in Windows applications.

GPU acceleration for Kubernetes was also announced, which will facilitate enterprise inference deployment on multi-cloud GPU clusters.

NVIDIA is contributing GPU enhancements to the open-source community to support the Kubernetes ecosystem.

In addition, MathWorks announced TensorRT integration with MATLAB.

Engineers and scientists can now automatically generate high-performance inference engines from MATLAB for Jetson, NVIDIA Drive and Tesla platforms.

TensorRT can also be deployed on NVIDIA Drive autonomous vehicles and NVIDIA Jetson embedded platforms.

Deep neural networks on every framework can be trained on NVIDIA DGX systems in the data center and then deployed into all types of devices for real-time inferencing at the edge.

Related stories
Top stories
Story image
Sustainability
AirTrunk boosts Japan presence with West Tokyo data center
AirTrunk is planning to build TOK2, a new hyperscale data center in Japan which will strengthen the company’s presence in the country.
Story image
Sustainability
RDA and MVGX partner for sustainable data center development
Red Dot Analytics (RDA) and MetaVerse Green Exchange (MVGX) have entered a strategic partnership to make Singapore's data center development and operations more sustainable.
Story image
Sustainability
Siemens showcases new automated solutions for data centers
Siemens has implemented new automated solutions and AI in the Baltic region's largest data center, providing insight into the future of data center management.
Story image
Power / Energy
Keysight Technologies introduces new next-gen DPT solution
Keysight Technologies has announced its new next-generation Double-Pulse Tester (DPT) with the PD1550A Advanced Dynamic Power Device Analyser.
Story image
Data Center
Sime Darby Berhad to use Equinix APAC data centers
Equinix has expanded its digital infrastructure services, including its International Business Exchange data centers, to Sime Darby Berhad
Story image
Artificial Intelligence
International Space Station experiments with HPE’s edge and AI solutions
Hewlett Packard Enterprise drives innovation at the extreme edge on the International Space Station, with 24 completed experiments.
Story image
Infrastructure
Kyndryl and CDC Data Centres enter multi-year agreement
Kyndryl and CDC Data Centres have announced an agreement, under which Kyndryl will design and manage a customised environment within CDC's facilities.
Story image
Databricks
Databricks grows in APAC market, expands into Korea
Databricks officially launches a local office in Seoul, Korea, building on existing partnerships with Cloocus, Megazone and the Weverse Company
Story image
Sustainability
Grasping the opportunity to rethink the metrics of a sustainable data centre
A data centre traditionally has two distinct operations teams: the Facility Operations team, and the IT Operations team. Collaboration between them is the key to defining, measuring, and delivering long-term efficiency and sustainability improvements.
Story image
Sustainability
ABB unlocks sustainable energy upgrades for data centers
ABB says its new microgrid solutions will get data centers ready for the green revolution and use their own energy sources with a reduced carbon footprint.
Story image
Data Center
CBRE finds record levels of investment in APAC data centers
CBRE's new report finds direct investment in the sector more than doubled in 2021, surpassing investment volumes for the past four years combined
Story image
Akamai
Akamai announces new products across security, computing
Akamai has announced a series of new products and updates to existing products across its security and compute product lines, including its entry into the infrastructure as a service (IaaS) market.
Story image
Microsoft
Microsoft unveils adaptive accessories for disability access
Microsoft is introducing an expansive Inclusive Tech Lab to give people with disabilities greater access to technology through new software features and adaptive accessories.
Story image
Hyperscale
SpaceDC partners with Aofei for data center sales in Asia
SpaceDC has partnered with Aofei Data International to sell Aofei's data centers, CDN and SDN in China.
Story image
Cloud
Colt connectivity with AWS increases services in Asia
Colt Technology Services expands cloud connectivity to AWS Direct Connect Hosted services, with speeds of up to 10 Gbps in Asia.
Story image
SD-WAN
Orange moves Siemens AG’s entire operations to a SD-WAN
Orange Business Services has migrated Siemens AG's entire global operations, 1168 sites across 94 countries, to a SD-WAN
Story image
Cybersecurity
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Infrastructure
Report - Data investment the key to better business growth
New research from Digital Realty has revealed that almost half (47%) of IT leaders globally believe their business investment in data systems and infrastructure is a key obstacle or concern.
Story image
BitTitan
Why tenant consolidation is critical to cloud success
Consolidating tenants can improve cost management, security and engagement after a flurry of reactive activity following the widespread shift to remote operations.
Story image
Hyperscale
Seagate, Phison partnership to improve data center offering
Seagate and Phison have announced plans to expand their SSD portfolio to assist data center customers in reducing total cost ownership (TCO).
Story image
SaaS
Cisco reveals new tech, intends to prevent network issues
Cisco has revealed new technology intended to mitigate costly disruptions by aiding IT teams in learning, predicting and planning.
Story image
Cloud
Four major announcements from Google's Data Cloud Summit
Multiple data cloud innovations have been unveiled at the Google Cloud summit, allowing customers to work with unlimited data across all workloads.
Story image
Surveillance
Genetec launches new enclosure management system for data centers
Genetec has released a new enclosure management solution that will give data centers the ability to secure, monitor and manage access to racks and cabinets remotely.
Story image
Government
NEXTDC, NT govt unveil data centre development plans
NEXTDC and the Northern Territory government have shared detailed plans surrounding the development of a new data centre in Darwin.
Story image
Cable
New high-performance cable in the works for Asia
A new high-performance submarine cable is being built to enhance connectivity between Hong Kong, China and Southeast Asia.
Story image
Power / Energy
DigitalBridge makes $30 million equity investment in LEDC
Leading Edge Data Centres (LEDC) has announced it has secured an AUD$30 million equity investment in its regional edge network from an affiliate of DigitalBridge Group, DigitalBridge.
Story image
Digital Transformation
EdgeConneX enters Indonesia, plans for data center campus
EdgeConnex has announced it is expanding its presence in Asia with the acquisition of GTN Data Center in Indonesia.
Story image
Tech Data
Tech Data to use Pluribus Networks’ cloud solutions in APAC
Tech Data says using Pluribus Networks' Unified Cloud Fabric solution will be a "game-changer" for its data center infrastructure customers and partners.
Story image
Data analytics
Google Cloud forms Data Cloud Alliance for digital transformation
Google Cloud and other tech providers have formed an initiative to eliminate data lock-in and ensure access to data across storage, analytics, and database providers.
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Data Center
Fujitsu AU signs PPA to offset 40% of NSW data centre load
The agreement marks a key step for Fujitsu Australia in decarbonising its operations and providing lower-emissions services to its customers.
Story image
Research
New strategies for cloud-native attacks - Aqua Security
New research from Aqua Security reveals attackers are using more sophisticated techniques to target cloud-native environments.
Story image
Sustainability
NTT launches IoT Services for Sustainability offering
"We know what actions are needed to build a more sustainable future and have a robust suite of technologies available to help deliver this impact."
Story image
Telstra
Telstra expands business offerings in the Philippines
The expansion aims to offer more choice for customers and enhance connectivity into the Philippines, and within the country.
Story image
Sustainability
Video: 10 Minute IT Jams - SoftIron CMO on Data Center Sustainability
In a special Power/Energy feature week presentation, we are joined by SoftIron CMO Andrew Moloney.
Story image
Red Hat
Red Hat expands capabilities to provide streamlined application development in cloud
"Application development is undergoing significant change and developers need tools to support this transformation."
Story image
Data Center
Tier III Ready Datacenter solutions shortlisted for major awards
"These designs will accelerate data center clients' own Tier III certification, reduce the cost, and fast-track their time to market."
Story image
Data Center
Equinix enters Africa, closing US$320 million acquisition of MainOne
The completion of the acquisition augments Equinix's long-term strategy to become a leading African carrier-neutral digital infrastructure company.
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Sustainability
Power at the edge: the role of data centers in sustainability
The Singaporean moratorium on new data center projects was recently lifted, with one of the conditions being an increased focus on power efficiency and sustainability.
Story image
APAC
Odaseva expands in APAC and UK with more security features
Odaseva, a data platform for Salesforce, is establishing new headquarters in London as well as a new data center in India.
Story image
Sustainability
AyalaLand and FLOW partner for data center development
AyalaLand Logistics Holdings Corp (ALLHC) and FLOW Digital Infrastructure have entered into a framework agreement to bolster the development of carrier-neutral data centers in the Philippines.
Softiron
For every 10PB of storage run on HyperDrive vs. comparable alternatives, an estimated 6,656 tonnes of CO₂ are saved by reduced energy consumption alone over its lifespan. That’s the equivalent of taking nearly 1,500 cars off the road for a year.
Link image
Story image
Digital Transformation
Multiplex, NEXTDC making strong progress on S3 data centre
Multiplex has made a significant achievement on Stage 1 of NEXTDC’s S3 data centre, ‘topping out’ the structure in the Artarmon on Sydney’s lower North Shore.