DataCenterNews Asia logo
Specialist data center news for Asia
Story image

Opinion: Interconnectivity must scale in line with bandwidth demand

By Contributor
Thu 22 Nov 2018
FYI, this story is more than a year old

Article by Corning Optical Communications technical sales manager Clive Hogg

As network traffic in data centers grow, data center interconnectivity needs to seamlessly scale to keep pace with the massive increase in demand for bandwidth. The data center interconnect application has emerged as a critical and fast-growing segment in the network landscape.

This article will explore several of the reasons for this growth, including market changes, network architecture evolution, and technology changes. 

The growth of data has driven the construction of data center campuses, notably hyperscale data centers. To keep information flowing between the data centers in a single campus, each data center could be transmitting to other data centers at capacities of up to 200 Tbps today, with higher bandwidths necessary for the future (see Figure 1).

Figure 1. Conceptual campus layout. DCI requirements and distances are unique. Bandwidth demands can range as high as 100 Tbps and even 200 Tbps.

Traditionally, data center architecute is a three-tiered topology, which consists of core routers, aggregation routers, and access switches. The three-tiered architecture doesn’t address the increasing workload and latency demands of hyperscale data center campus environments. Today’s hyperscale data centers are migrating to spine-and-leaf architecture (see Figure 2) where the network is divided into two stages. The spine is used to aggregate and route packets towards the final destination, and the leaf is used to connect end-hosts and load balance connections across the spine.

The large spine switches are connected to a higher-level spine switch, often referred to as a campus or aggregate spine, to tie all the buildings in the campus together. By adopting flatter network architecture and high-radix switches, we expect the network to get bigger, more modular, and more scalable.

Figure 2. Spine-and-leaf architecture and high radix switch requires massive interconnects in the data center fabric.

Which mix of technology is required to deliver DCI connectivity?

Multiple approaches have been evaluated to deliver transmission rates at this level, but the prevalent model is to transmit at lower rates over many fibres. To reach 200 Tbps using this method requires more than 3000 fibres for each data center interconnection. When you consider the necessary fibres to connect each data center in a single campus, densities can easily surpass 10,000 fibres. 

A common question is when to use DWDM or other technologies to increase the throughput on every fibre as opposed to constantly upgrading the number of fibres. Data Center interconnect applications up to 10 km often use 1310-nm transceivers that don’t match the 1550-nm transmission wavelengths of DWDM systems. So the massive interconnects are supported by using high-fibre-count cable between data centers.

The next question becomes when to replace 1310-nm transceivers with pluggable DWDM transceivers in the edge switches by adding a mux/demux unit. The answer: when or if DWDM becomes cost-effective for these on campus data center interconnect links. Once this happens, the same bandwidth will be achieved by using DWDM transceivers associated with much lower fibre count cables.

The current prediction is that connections based on fibre-rich 1310-nm architectures will continue to be cheaper for the foreseeable future (see Figure 3).

Figure 3. Pluggable DWDM transceivers vs. 100G CWDM4.

Best practices to build extreme-density networks

It is important to understand the best ways to build out extreme-density networks as these networks present new challenges in both cabling and hardware. For example, using loose tube cables and single-fibre splicing is not scalable.

New cable and ribbon designs have doubled fibre capacity from 1728 fibres to 3456 fibres within the same cable diameter or cross-section. These generally fall into two approaches: one approach uses standard matrix ribbon with more closely packable subunits, and the other uses standard cable designs with a central or slotted core design with loosely bonded net design ribbons that can fold on each other (see Figure 4).

Figure 4. Different ribbon cable designs for extreme-density applications.

Leveraging these extreme-density cable designs enables higher fibre concentration in the same duct space. Figure 5 illustrates how using different combinations of new extreme density style cables enables network owners to achieve the fibre densities hyperscale-grade data center interconnections require.

Figure 5. Using extreme-density cable designs to double fibre capacity in the same duct space.

When leveraging new ribbon cable designs, network owners should consider the hardware and connectivity options that can adequately handle and scale with these very high fibre counts. It can be easy to overwhelm existing hardware, and there are several key areas to think through as you develop your complete network. 

If you are currently using 288-fibre ribbon cables in your inside plant environment, your hardware must be able to adequately accommodate 12 to 14 cables. Your hardware also must manage 288 separate ribbon splices. Using any single-fibre type cables and a single-fibre splicing method in this application is not feasible or advisable because of massive prep times and unwieldy fibre management. 

Another challenge is keeping track of fibres to ensure the correct splicing. Fibres need to be labelled and sorted immediately after the cable is opened because of the magnitude of fibres that must be tracked and routed. In most installations, redoing cable prep is manageable. In the case of extreme-density networks, a mistake could cost a week’s delay for just one location.

What does the future hold for extreme-density networks?

The most important factor right now is whether counts will stop at 3456 fibres, or go higher. Current trends suggest there will be requirements for fibre counts beyond 5000. With fibre packing density already approaching its physical limits, the options to reduce cable diameters in a meaningful way becomes more challenging. 

Development has focused on how to provide data center interconnect links to locations spaced much farther apart, and not co-located within the same campus. In most data center campus environment, data center interconnect lengths are 2 km or less. These relatively short distances enable one cable to provide connectivity without any splicing points. With edge data centers being deployed around metropolitan areas to reduce latency times, distances can approach up to 75 km. Extreme-density cable design makes less financial sense because of the cost to connect the high number of fibres over a long distance. In these cases, more traditional DWDM systems will continue to be the preferred choice, running over fewer fibres at 40G and higher.

As network owners prepare for 5G, demand for extreme-density cabling will migrate from data center environments to access markets. It will continue to be a challenge in the industry to develop products that can scale effectively to reach the required fibre counts while not overwhelming existing duct and inside plant environments.

The next gen is oriented around the support for up to 600G per wavelength and would only be used in shorter reach metro applications with flex grid line systems capability. Data Center network operators are adopting 100G inter data center networking technologies today, and will soon be moving to adopt 400G.

Related stories
Top stories
Story image
Artificial Intelligence
Vectra AI named as AWS security competency partner
Threat detection and response company Vectra AI has announced that it has become an Amazon Web Services Security Competency Partner.
Story image
Development
Intel Labs unveils integrated photonics research advancement
"This new research demonstrates that its possible to achieve well-matched output power with uniform and densely spaced wavelengths."
Story image
Macquarie Data Centres
Macquarie deal to pioneer CO2-cutting data centre tech in Australia
Macquarie Data Centres has signed a multi-year deal with ResetData, an Australian first provider using Submer data centre technology. 
Story image
Migration
SNP unveils next generation of CrystalBridge software platform
Data is a key pillar of every customer-centric organisation, as it relies on agile decisions to become increasingly sustainable and intelligent.
Story image
Sustainability
Intel unveils new investments for data center sustainability
Intel has announced two new investments, continuing its efforts to create more sustainable data center technology.
Story image
Cloud
Cisco Live showcases new offerings in its first hybrid event
Cisco Live 2022 has seen Cisco executives and customers take the stage to present a range of discussions in the company’s first-ever hybrid event.
Story image
Broadband
Singapore found to have the speediest internet rates in the world
New research from BanklessTimes has shown that Singapore has the highest recorded median internet speed in the world at 207.61 MBPS.
Story image
Cloud
SnapLogic improves Intelligent Integration Platform
SnapLogic has released new features and improvements to its Intelligent Integration Platform, which will allow IT, data and business teams to make select processes faster and more straightforward.
Story image
Artificial Intelligence
Databricks announces new offering for Unity Catalog
Databricks has significantly expanded data governance capabilities on the lakehouse by unveiling data lineage for Unity Catalog.
Story image
Partnerships
Thailand announces launch of the Thailand 5G Alliance
It will promote collaboration between the public and private sector, through companies such as Huawei, to commercially drive Thailand's 5G development.
Story image
Microsoft
SAS Viya on Microsoft Azure to deliver 204% return - study
The Forrester Total Economic Impact study finds SAS Viya on Microsoft Azure brings a 204% return on investment over three years.
Story image
Cloud
Cloudflare outage in 19 data centers worldwide due to own error
Cloudflare says its outage for 19 of its data centers yesterday was because of a change in a long-running project to increase resilience in its busiest locations.
Story image
Cybersecurity
Zscaler launches co-located data centres in Canberra and Auckland
The investment will offer public and private sector enterprises greater resilience in support of their zero trust cybersecurity posture.
Story image
Infosys
Preparing for the digital decade with the right workforce strategies
For a decade that started under the pall of the pandemic, the 2020s is poised to end with a bang with the digital economy swelling to a high across the world.
Story image
Cybersecurity
Secureworks researches new threat to Elasticsearch databases
Researchers from Secureworks' Counter Threat Unit have identified indexes of multiple internet-facing Elasticsearch databases replaced with a ransom note.
Story image
Infrastructure
Global investment in data centers more than doubled in 2021
DLA Piper's latest global survey finds the total investment in data center infrastructure worldwide rose from USD $24.4 billion in 2020 to USD $53.8 billion in 2021.
Story image
CASB
Juniper expands SASE offering with data loss prevention capabilities
Juniper has announced the expansion of its SASE offering with the addition of cloud access security broker (CASB) and data loss prevention (DLP) capabilities.
Story image
Sydney
Equinix and PGIM Real Estate open data centre in Sydney
Equinix and PGIM Real Estate, the real estate investment and financing arm of PGIM, have announced the first xScale data centre in Sydney, named SY9x.
Story image
Vietnam
Viettel IDC deploys Cloudian Hyperstore object storage for enhanced cloud solutions
Cloudian has announced that its Hyperstore object storage has been deployed by Vietnam telco Viettel IDC, citing the technology’s flexibility, multi-tenancy and ransomware protection as significant advantages.
Story image
Infrastructure
SolarWinds IT Trends Report highlights increased cloud complexity for businesses
SolarWinds' new IT Trends report has signalled a significant shift in the way businesses are dealing with hybrid cloud and infrastructure.
Story image
Sustainability
SoftIron named global leader for efficient DC infrastructure solutions
SoftIron has been named a global leader for supplying energy-efficient data infrastructure solutions for core-to-edge data centers after an assessment by Earth Capital Ltd.
Story image
Cloud
QuSecure partners with DataBridge Sites to showcase platform
QuSecure has partnered with DataBridge Sites to showcase its Quantum-as-a-Service (QaaS) orchestration platform, QuProtect.
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Infrastructure
Oracle Cloud Infrastructure expands distributed cloud services
“Distributed cloud is the next evolution of cloud computing, and provides customers with more flexibility and control in how they deploy cloud resources."
Story image
Cloud
Boomi surpasses 20,000 customers. Sets record for the iPaaS space
Boomi has announced it has surpassed the 20,000 customer mark, setting the record for the largest customer base among iPaaS vendors.
Story image
Infrastructure
New Uptime analysis highlights worsening downtime costs and consequences
New data from Uptime Institute has found that downtime costs and consequences are worsening as those involved in data infrastructure fail to find ways to curb outages.
Story image
Healthcare
SnapLogic launches Accelerator for Amazon HealthLake
SnapLogic has launched Accelerator to allow healthcare and life sciences organisations to turn raw data into healthcare-related insights and actions.
Story image
Robotic Process Automation / RPA
Micro Focus unveils Data Center Automation for SaaS delivery
MicroFocus has released Data Center Automation (DCA) for software-as-a-service (SaaS) delivery, offering more cost-effective vulnerability risk and IT compliance management.
Story image
Cybersecurity
Kaspersky opens three new centers to boost data management
Cybersecurity company Kaspersky has opened three new Transparency Centers, one in Japan, the second in Singapore and the third in the United States.
Story image
Hybrid Cloud
HPE GreenLake advances hybrid cloud experience with new services
"The innovations unveiled today further build on our vision to provide the market with an unmatched platform to spur innovation and drive transformation.”
Story image
Data Science
Neo4j announces service delivery alliance with Deloitte
Neo4j has announced a service delivery alliance with Deloitte Consulting Southeast Asia for a range of services to customers within the region.
Story image
Microsoft
Microsoft, Cloudian partnership offers data center flexibility
Cloudian’s HyperStore object storage platform is now integrated and validated to work with Microsoft SQ Server 2022, offering more flexible and scalable data centers.
Story image
Digital Transformation
The Huawei APAC conference kicks off with digital transformation
More than 1500 people from across APAC have gathered for the Huawei APAC Digital Innovation Congress to explore the future of digital innovation.
Story image
Sisense
Data and analytics could be key to higher selling prices in APAC
Sisense's latest report has found that almost half of data professionals in APAC think customised data and analytics can create better selling prices for their products.
Story image
Sustainability
Daikin and SP Group to build new energy efficient district cooling system
The project, set to be complete by 2025, will create a system with a cooling capacity of up to 36,000 refrigerant tonnes (RT). 
Story image
Public Cloud
Public cloud services revenues top $400 billion in 2021
"For the next several years, leading cloud providers will play a critical role in helping enterprises navigate the current storms of disruption."
Story image
Cloud
Vertiv introduces line of redundant power transfer switches
Vertiv has introduced Vertiv Geist Rack Transfer Switch (RTS), a new line of transfer switches that provides redundant power to single-corded devices.
Story image
Amazon Web Services / AWS
Qualtrics goes live on AWS Cloud Infrastructure in Japan
Organisations across Japan will now be able to access the Qualtrics XM/OS platform locally via data centre in the AWS Asia Pacific (Tokyo) region.
Story image
Sustainability
Evolution Data Centres reveals target of 20 tonnes of CO2 per GWh
Evolution Data Centres launches their new Sustainable Data Centre Charter, which includes targets like only 20 tonnes of carbon emissions per GWh by 2030.
Story image
Sustainability
Aligned Data Centers increases sustainability-linked loan
Aligned Data Centers has increased its sustainability-linked loan from $375 million to $1.75 billion to speed up the next phase of its strategic growth.
Story image
Sustainability
Legrand unveils Nexpand, a data center cabinet platform
Legrand has unveiled a new data center cabinet platform, Nexpand, to offer the necessary scalability and future-proof architecture for digital transformation.
Story image
Sustainability
Huawei unveils next-generation sustainable data centers
Huawei says its next-generation data centers will be powered by PowerPOD 3.0, which reduces the footprint by 40% and cuts the energy consumption by 70%.
Story image
Secure access service edge / SASE
Cisco unveils new cloud-managed networking offerings
Cisco has announced new cloud management capabilities that offer a unified experience across the Cisco Meraki, Cisco Catalyst and Cisco Nexus portfolios.
Story image
Colocation
Digital Edge chooses Nortek’s StatePoint for new data center
Digital Edge will use Nortek's StatePoint liquid cooling technology in its new data center, the first commercial colocation operator in Asia to do so.