The Rise of the Data Platform for Hybrid Cloud

For many, the solution is an overarching software defined, virtualized data platform that delivers a common data plane and control plane across hybrid cloud environments. Ian Clatworthy, head of data platform product marketing at Hitachi Vantara, describes a data platform as “an integrated set of technologies that meets an organization’s data needs, enabling storage and delivery of data, the governance of data, and the security of data for a business.”

Gartner projects that these consolidated data storage platforms will constitute 70% of file and object storage by 2028, doubling from 35% in 2023. The research firm underscores that “Infrastructure and operations leaders must prioritize storage platforms to stay ahead of business demands.”

A transitional moment for enterprise data

Historically, organizations have stored their various types of data—file, block, object—in separate silos. Why change now? Because two main drivers are rendering traditional data storage schemes inadequate for today’s business needs: digital transformation and AI.

As digital transformation initiatives accelerate, organizations are discovering that having distinct storage solutions for each workload is inadequate for their escalating data volumes and changing business landscapes. The complexity of the modern data estate hinders many efforts toward change.

Clatworthy says that when organizations move to hybrid cloud environments, they may find, for example, that they have mainframe or data center data stored in one silo, block storage running on an appliance, apps running file storage, another silo for public cloud, and a separate VMware stack. The result is increased complexity and cost in their IT infrastructure, as well as reduced flexibility and efficiency.

Then, Clatworthy adds, “When we get to the world of generative AI that’s bubbling around the edges, and we’re going to have this mass explosion of data, we need to simplify how that data is managed so that applications can consume it. That’s where a platform comes in.”

Sinclair acknowledges that data silos have long been a challenging issue, but he says that today’s hype around AI has created an opportunity to finally and fully address them. “Initiatives that get a high level of focus from the C-suite, like AI,” he says, “tend to allow organizations to get the necessary budget to do the thing that they probably wanted to do for a long time.”

Security and regulatory compliance are also playing a role in the drive to get a handle on enterprise data. Says Sinclair, “When you have data stored across 12 locations, it’s more difficult to secure than if it’s in one location.” A data platform helps organizations secure data against escalating cyberattacks, particularly ransomware. And it enables organizations to gain visibility and control over far-flung data stores in order to comply with increasingly strict regulations around data protection and data privacy.

Another motivating factor is the scarcity of skilled data storage professionals, particularly when it comes to AI workloads. “When it’s tough to find expertise, it makes sense to invest in tools and technologies that make the task as intuitive and simple as possible,” says Sinclair.

Benefits of a data platform

A data platform can reduce costs, improve efficiency, automate processes, enable digital transformation, unlock the potential of AI and generative AI, and drive innovation across the organization by putting the right data at the fingertips of data scientists and business analysts.

These benefits not only deliver on internal IT objectives around performance and cost-savings, but also can be employed for business-wide innovation and improved bottom-line results.

Sustainability can also be a significant benefit of data infrastructure upgrades, and environmental objectives may increasingly motivate shifts to more modern storage solutions. With the climate costs of data becoming apparent—in North America alone, US data centers are responsible for as many CO2 emissions as the entire country of Mexico, says Clatworthy—customers are looking for changes to their data centers that use less power, work more efficiently, last longer, and make sustainable choices throughout the storage hardware lifecycle.

With increasing regulation around sustainability reporting, businesses now need to be able to track—and reduce— emissions stemming from their data. “For the first time,” Clatworthy says, “there is a true need for accountability and transparency around CO2 in the data center.”

Data platforms unlock the potential of generative AI

A recent Enterprise Strategy Group survey asked organizations to describe the primary benefits of using generative AI. The top answer was improving data processes and workflows, followed by supporting data analytics and business intelligence. After these came improving employee productivity, improving operational efficiency, improving the customer experience, and creating content.

Sinclair says, “If you put this together, the initial thought is, generative AI can make me a faster, leaner, stronger business. And the next thing is, ‘Wow we could really do some innovative things to improve the way we interact with our customers and the things we sell.’”

But for organizations to begin to reap the benefits of generative AI, they need to have the right data storage infrastructure in place. Sinclair says, “If I’m trying to leverage generative AI in multiple aspects of my company, then I’m going to need a data solution that’s scalable and that gives me flexibility.”

As Sinclair explains, a data platform can provide that: “A data platform gives you the flexibility to say, ‘I need to spin up certain resources, I need to move this data set over there. I need to provide an architecture that allows me to reduce the cost of complexity, so when the unknown happens, I’m in a better position to take advantage of it.”

Clatworthy adds that the simplicity of the data platform makes more resources available for innovation. “People are figuring out what they want to do with generative AI,” he says, “and while they’re doing that, they need to reduce the complexity of their data centers so they can free up people to go and innovate with AI.”

As the last few years have shown, we can’t predict when the next disruption might occur. Consequently, organizations need a flexible, resilient data solution that can help them respond to whatever the future brings.

And when it comes to generative AI, the next big use case might be something that we haven’t yet even imagined. “How do you make sure you’re properly prepared to take advantage of the next great innovation,” says Sinclair, “when you don’t know what it is yet?”