Exclusive case study: Distributed Internet Exchanges go mainstream with the latest DCI platforms

Everyone – in business, at home, working, playing or studying – expects Internet access. And every company that relies on the Internet for selling services, content, communication or goods wants a ready link into that access. In between the two, we increasingly find an Internet Exchange (IX) – allowing direct, high bandwidth connections without the added latency and cost of directing traffic via upstream transit networks.

So it is hardly surprising that the IX market is thriving, and driving rapid expansion of IX facilities. Leading IX providers often operate multiple facilities within a single metro area to satisfy this demand, and they want all of their facilities to be interconnected seamlessly to operate as a single, large scale, "Distributed Internet Exchange".

According to Jonathan Hjembo, Senior Analyst at TeleGeography: "Distributed Internet exchange infrastructure is a key strategy to serve growing demand. DE-CIX in Frankfurt, LINX in London, and Equinix in major locations around the world all use dedicated network capacity interconnecting numerous sites throughout a given metro area to create virtual networking environments that nearly simulate co-location in a single building over a wider area.

As is the case with data center interconnection, a distributed IX requires very high capacity metro optical transport. To support the IX's promise of fast and easy connection, the links must be simple to deploy and scale. Another factor is the need to provide exchanges as close as possible to critical users, meaning that parts of the exchange will be located in central business districts where real estate costs are highest: so the equipment also needs to meet the space and power constraints of an urban data center environment.

The Evolving Internet Exchange

IX is the heart of today's Internet ubiquity, the place where the Internet actually gets connected. Since the early days of Network Access Points shared by a few dominant backbone providers, the idea of IX has grown and transformed into a robust Metro IX ecosystem serving a wide range of companies providing cloud-based content, applications and services.

Everyone wants to be in the right IX facilities to connect to their customers, partners and providers. Although traditional carriers and Internet Service Providers (ISPs) still have a major presence, there is a rise in new players. These include the major Internet Content Providers (ICPs) such as Google and Facebook and hyper-scale Cloud Platform Providers (CPPs) such as Amazon Web Services and Microsoft Azure. These large providers have the scale, resources and incentive to build their own networks into multiple major metro areas and peer directly with the fixed and mobile access providers who reach their end users. By doing so, they can better control service delivery and the quality of experience for their users by lowering latency and ensuring more predictable performance. At the same time they can reduce their own overheads and offer better value.

But beyond these giant players, there are thousands of others: virtually any provider offering cloud-based services also needs a presence in multiple IX facilities. What is more, there are a growing number of large enterprises locating in IX facilities to enable their hybrid cloud IT strategies – adding further diversity to the Metro IX ecosystem.

The IX providers are also becoming more diverse. Commercial operators such as Equinix, TelX and Interxion have built a business around carrier-neutral IX services, including highly-available co-location (rack space - power) and flexible high-capacity Ethernet interconnection. Cooperative, non-profit IX providers have also grown, and a few of them, such as DE-CIX (in Frankfurt), AMS-IX (Amsterdam) and LINX (London) are now among the largest IX sites in the world.

The need to link distributed exchanges

IX demands in major metro areas long ago outstripped the capacity of any single facility, and the number of facilities keeps growing. With multiple co-location and Internet Exchange providers spread over dozens of physical facilities, the IX customers and providers alike face a recurring problem: how do I connect to my customer, partner or provider in a different IX across town?

The largest IX providers, such as Equinix, want as much of the Metro IX ecosystem as possible to be contained within their own facilities so that they can solve that problem for their customers. Following the logic of "the network effect", they know that the value of their facilities and IX services increases exponentially as they increase the number of customers and providers co-located there and within easy reach of each other.

To meet that objective, they continue to expand: in some cases by building new facilities and in other cases they expand by acquiring existing IX providers and facilities. Of Equinix's 33 metro markets, 25 have more than one Equinix IX facility and are known as Internet Business Exchanges (IBXs). The Washington, DC, area alone has ten IBX sites, while Tokyo has five. Several other smaller IX providers also have multiple metro markets in which they operate more than one IX facility.

The requirements for distributed IX

Connecting multiple facilities into a single distributed IX presents a number of challenges. For a start it requires very high-capacity connection.

Within a single facility, customers typically connect to each other directly using fibre-optic cross connects at 1 to 100 Gb/s, or connect via virtual private connections through the IX provider's Ethernet switching platform. If a distributed IX is going to provide the same service as a single IX, the metro optical network connecting facilities must support dozens or even hundreds of high-speed "cross connects" between facilities, and aggregated virtual private connections that together can add up to multiple terabits per second (Tb/s) of total capacity.

A distributed IX also needs that level of capacity to scale easily and efficiently, without major re-engineering of the optical networks or service affecting maintenance outages. Traffic within the Metro IX ecosystem shows every sign of continuing to grow even faster than global Internet traffic. ACG research predicts: "For 2014–2019, Metro traffic is predicted to increase faster (13.0 per cent CAGR) than backbone traffic (8.6 per cent CAGR) as more regional data centers are located closer to the user community. As much as 70 per cent of the traffic is predicted to stay within the metro from which it originated and will drive the need for additional capacity by the traditional service provides, MSOs and data center operators.

As a consequence of this rapid growth, the equipment must allow for routine, straightforward capacity upgrades as distributed IX operators need their metro optical transport platforms to be extremely simple to install and manage. The IX operator's core competency is data center management and intra-facility interconnection – not network engineering – so they need a system that fits into the "rack and stack" operational approach and integrates well into data center operations systems. Traditional switching technology evolved to suit the many needs of telecommunications, so has many features that add complexity, cost and a need for specialist engineers.

Finally, most IX facilities are like other metro data center environments, where space and power are extremely scarce and constrained, so the metro optical infrastructure must have very high density and excellent power efficiency.

To address a growing demand for metro cloud and data center interconnection, leading manufacturers have been quick to develop new generation transport platforms, offering greatly increased capacity in the smallest possible footprint and power requirement. These exactly match the requirements for distributed IX in a metro area.

For example: a single 2-rack unit (RU) "Cloud Xpress" from Infinera, leverages photonic integrated circuit (PIC) technology to deliver a 500 Gb/s super-channel of dense wavelength division multiplexing (DWDM) bandwidth on a single fibre pair – supporting client interfaces including 10 GbE, 40 GbE and 100 GbE. Requiring less than 1 Watt per Gigabit per second, these compact units minimise power as well as space overheads. Racking and stacking compact units like these means that capacity can be readily scaled to exact customer needs via a simple interface  – up to 8 Tb/s on a single fibre pair – without interrupting services to customers.

Conclusion

As demand for IX services soars, the ideal technology to meet the challenges of metro IX consolidation has emerged from the data center interconnection market. Purpose-built equipment allows IX providers to deliver on their promise that locating anywhere in any of their IX facilities enables access to customers in other facilities as simply as if they were cross-connecting across the aisle in the same building. IX providers can increase available capacity between their facilities to meet current demands, while positioning themselves for easy network upgrades without affecting existing services. They also benefit from far greater operational simplicity, lower costs, minimal training outlay and absolutely no specialised optical expertise.

As demand for IX services continues to grow, providers can now meet their customers' needs by building high-performance distributed Internet Exchanges using the latest generation transport platforms that offer greatly increased capacity with the smallest possible footprint and power requirements.