Seagate: Data trends, opportunities, and challenges at the edge

The rise of big data and the development of edge technology have brought many opportunities for data infrastructure companies to the fore.

IDC figures show that by 2025, 175 zetabytes of data will be created.

The Asia-Pacific region, in particular, is set to be one of the biggest drivers of this growth, generating 20% of data in 2025.

Most importantly, data created at the edge will see the biggest growth, going up from 12% of data created to 22% from 2015 to 2025.

In order for organisations to truly take advantage of these trends, it's important for them to have a real understanding of both the opportunities and the challenges they bring.

Seagate recently hosted a panel discussing these topics featuring experts and executives including IDC datasphere senior vice president Dave Reinsel, Seagate executive vice president and head of operations Jeff Nygaard, and Seagate global sales and sales operation Ban Seng Teh.

Here are some of the highlights from the discussion:

Reinsel says the first challenge lies in the logistics in maintaining edge infrastructure in remote areas and ensuring that it's always working whereas the second is capturing new formats of data, managing it, and protecting it, and making sure that it's processed in a way that would allow organisations to extract value from it.

Nygaard adds there is a need for organisations to be consciously thinking about what data needs to be moved, either from the edge to the core or otherwise.

As for the opportunities, Reinsel sees having a connected edge as one of the opportunities to engage customers in unique and exciting ways.

"It's an opportunity to drive whole new levels of service and convenience to customers.

Teh says public cloud-wise, he's seen the proliferation of data centers spread across different geographies, as some regulators require data to be locally held and for lower latency.

He adds that public cloud providers are now providing edge services as they start to recognise the need for edge infrastructure.

In terms of the private cloud, Teh says a lot of companies are moving towards a hybrid cloud model.

"Surveys have shown that the majority of the companies thinking of utilising the public cloud will maintain their own data centers - so that's really interesting.

Nygaard says on top of latency and the ability to receive data in real time, there are other important drivers for the edge such as the size and the correlated cost of data.

"It's not free to move data from the edge to the cloud - so you should only be moving data if you have a very good reason to.

Reinsel agrees, and says he sees the role of the data center moving towards making sure data is managed in an appropriate way instead of in siloed environments to ensure data lakes don't become data swamps.

"That's going to be difficult because you're going to bring in all these new sources of data but the last thing you want to do is silo all these data sources, because if you don't intersect data, you can't apply analytics.

In this aspect, Teh says public cloud operators have an advantage because of the scale they have, so they're able to spend time and resources on opmisating how they manage and move data in the most efficient way.

In contrast, enterprise data centers don't have that scale, so they have to rely on available third-party services for that.

Nygaard stresses that it's important to have people with the skills needed to extract value from data for data centers to liberate data.

"You need people who know how to analyse data in big data sets as data is analysed at the database - we call that a 'citizen data scientist'.

"The second piece you need is people who are building the data pipes and thinking about the architecture of where that data is stored," says Nygaard.

"And finally, there needs to be an executive buy-in that the data is important for how a company is going to operate.

Reinsel adds that as organisations move towards automated decisionmaking, the validity of this data will differentiate their abilities to deliver positive and negative customer experiences.

"As you start to enable processes with AI, if you don't get the right data at the right time, you're going to create a poor customer experience and you're going to spoil the well for everyone else.

"Different countries are at different levels of maturity - but the direction every country is taking is the same," Teh says.

He adds that even countries that were lagging behind developing edge infrastructure originally are moving rapidly up the food chain, and these may require more handholding as because of skill or awareness of the technology as opposed to companies who already know what's out there, what needs to be done, and exactly what they want to buy.

Nygaard agrees, saying that there will be differences in readiness around skillsets, funding, and how it's set up, so it's important to understand what their real driver for why they want to expand their data scope.

"Of course, different regions around the world also have different privacy regulations, and different rules around access to data - so there are different elements involved in their decisionmaking.