Is my organization using the right data storage?

Are we ready to grow?

With the explosion of data from the Internet of Things (IoT), do I need to build something different?

And how do I even use all this data?

These are some of the biggest questions businesses around the world are asking Hitachi Vantara, the cloud and big data specialists.

In this exclusive interview, the company's VP of infrastructure solutions marketing, Bob Madaio, discusses how today's data-obsessed world has so much data that it doesn't know what to do with it.

"The big conversation businesses around the world are having today is focused on data and what we are all trying to do with the data.

“It's moved from a conversation on infrastructure to a data centric conversation. We've had a lot of interest from customers recently on storing data, and ensuring they are building the right foundations, the data infrastructure."

"The data topic, and how to treat data, has been a big conversation. And then, what are the implications here -am I using the right kind of storage, am I ready to grow? Everyone has IoT (Internet of Things) data coming in - now, do I need to build something different with that? How do I store and use that data."

The rise of the Internet of Things and machine learning has brought with it an explosion of data, and Madaio says we are all still learning what the true impact of it will be.

"What I mean by that is that customers today are still learning where to keep their data, asking ‘how much of the data do I need to keep? Do I need to send any of it back to the data center? Can I do it all on the edge?'"

"A lot of the IoT use cases [see] extreme amounts of data generated, and customers are learning now what percentage of data we actually need to keep."

"Another implication from this machine learning and IoT data is managing - and what I mean by that is if you look at the core storage market, one of the big focus areas our customers have always said is - how can scale the management if my data continues to grow at such a fast?"

"A lot of those technologies, like AI, are starting to push a lot of low-level decisions down to the software - so we're deploying a new system, we're adding capacity to database - we can run automation tools and use analytics where to put that data so it's most effective."

"That's allowing people to manage a lot more data per person than they used be able to manage."

Madaio dives deeper into the conversation of data by looking at a few of the top trends on storage, starting with the rise of object storage.

"We're seeing a move to simplify the operational side of data - which is, people can put data in an object store that they don't need in their traditional backup scheme. This means they can put it in a more effective place to scale what they do.

“But then, once they have it in place, the idea of leveraging metadata becomes really interesting and customers are trying to figure out how to leverage all the data they storing in their environment.

“We're seeing things like archives being built where customers can store data for years and years, we are talking hundreds of years, which is hard to even conceive."

"How do you come back in 20 years time and still be able to find certain information?"

"We need to add a layer of software above the physical hardware system that can change as the hardware comes out and still be able to find that information?"

"In our blob storage, our traditional storage business is virtualized and virtualization helps us break down some of the barriers in the application.

“Object storage does that now but on a whole new type of application market. Customers want to make sure the data can outlive their applications - so if you get that into an object store you can describe all that information, including when it came in, where it's relevant to.

“That metadata that's associated with the object store is giving people who want to analyze the data, who want to be able to find quickly, a whole new set of capabilities that they wouldn't have without it.

“So, instead of just putting files into a big bucket of other files, they can structure it a little bit so they can more quickly find any little bit of information about a customer, or every little bit of information about a particular machine, and where it's being used."

"There's definitely a lot of revisiting of how we're using that metadata, that explanatory information that gets stored in an object – and that's what's driving interest in object storage today."Flash and memory

"We're definitely seeing interest in the move to flash. That's a global trend, and we've had a lot of conversations here with customers regarding having tiered service levels and using different storage media."

"Firstly, we're seeing the trend towards all flash for production workloads, and tiering for secondary workloads. We also have a lot of interest in what comes next. Now, what people are deploying is a move to an all flash environment, we also are having customers begin to think about where this idea of NVMe - that's a hot topic on everybody's mind."

"That's in the planning stage right now, it's not really being deployed but everyone is trying to think about how they can use more flash and memory just to keep up with demands."