Gu Xie is the Head of Data Engineering at Group 1001. His innovative use of cutting-edge cloud technologies has accelerated the organization’s data analytics maturity, enabling rapid insights and the use of data to make smarter decisions across the enterprise.
Gu recently shared his thoughts with us about what he sees as the top data trends emerging in 2025, what’s coming down the road that he finds exciting, and some industry misconceptions he’d like to see put to rest.
What’s the most important trend in our industry you see gaining momentum in 2025, and why does it matter?
Gu Xie: I believe generative AI will continue to dominate the industry in 2025. In 2024, most organizations were in an experimental phase, where they were still figuring out what to do with this technology. I think 2025 is the year when we will see organizations deploy generative AI—and deploy it at scale. And I’m not just talking about big tech. I’m talking about mid-cap, large-cap, small-cap companies leveraging gen AI technologies to automate their existing workflow processes to enhance customer experience, customer service, and much more.
I firmly believe there’s a tremendous amount of value in generative AI technologies, and I foresee companies moving on past the hype to actually leverage them day to day to improve their own operational processes.
And that will also introduce a lot of challenges. How will large organizations deploy and handle AI models at scale in a way that is properly governed and managed? People in the industry talk about data governance, but I’m pretty sure soon we will be talking about AI governance.
Many of the concerns we’ve had with data analytics appear to be replicated forward into AI. With AI regulation on the horizon, how should organizations acquire data at scale, activate data with AI models, and govern those AI models in such a way to mitigate regulatory risk in the long term?
“I firmly believe there’s a tremendous amount of value in generative AI technologies, and I see companies moving on past the hype to actually leverage them day to day to improve their own operational processes.”
What’s something surprising you’ve noticed in 2024 that you predict will continue in the new year?
Gu: A focus on data governance and observability became hugely important in 2024, and I see that trend carrying forward into 2025. Many organizations, regardless of size, focused on this as a part of cost optimization and reduction initiatives as the zero-interest and easy funding eras came to an end. Now, organizations appear to be looking to tap into AI technologies—but to do so, trustworthy data is the ultimate prerequisite. In 2025, the focus won’t just be around data governance, but AI governance as well.
So how would a business prioritize their needs in this space? And how does that relate to any trends you’re seeing in organizing teams around data to create a data culture in the workspace?
Gu: I believe that, on a fundamental level, organizations do not need to work in functional silos. One of the crucial anti-patterns facing organizations is having separate and distinct teams of data analysts, data engineers, platform engineers, data scientists, and so on. Especially at large enterprises, it’s common to see the data analysts work on something, and then lob it over to a data engineer on a different team. In my opinion, this displaces a lot of the responsibilities and accountability, which impedes execution going forward.
In organizing this siloed umbrella by function, organizations are not likely to achieve optimal outcomes. In order for data culture to be unlocked, I believe organizations’ teams should be organized by departments, functions, key capabilities, and/or outcomes that are consistent with what those teams unlock.
Is that something you see more of in 2025, the need for more integrated teams?
Gu: I think so, and it would likely happen out of necessity. To achieve positive outcomes, organizations will likely need more integrated teams across the board that are cross-functional and not just made up of technical folks. Organizations should consider integrating folks from the business directly into these teams. Technical teams by themselves are less likely to succeed otherwise. Organizations will likely need both the technical and business sides to drive outcomes forward.
What do you see around the corner that’s most exciting to you—even if it won’t come to fruition in 2025?
Gu: I don’t know if I’m excited about this, but what I see right now is market consolidation happening across the board.
A lot of startups appear to be struggling to succeed in the marketplace. At the same time, large companies are faltering, unable to secure additional funding or drive a lot of the investor-desired outcomes. I believe there will be even more market consolidation. Where organizations used to have a mass proliferation of new tools, new products, and new services, they’re likely going to start seeing consolidation among major players—which I believe will ultimately drive a lot of the technologies and potentially more consolidated ecosystems in the next three to five years.
What do you see as the implications of that? Does that help or hurt the industry, and how?
Gu: I think that it does help. Looking at the MAD (ML/AI/Data) Landscape Diagram, companies are likely going to start seeing more prescriptive, smaller vendor sets that are tied to a particular ecosystem. Most likely the markets will consolidate around Snowflake, Databricks, and potentially BigQuery as well, and these companies are likely to end up purchasing and integrating many of these tools into their systems.
On the positive side, that reduces the amount of the decisions businesses must make about what data tooling to purchase. The negative side of that is reduced innovation and significantly higher costs associated with switching from one platform to another. This may likely drive more embedded adoption within the ecosystems that businesses are currently in. And because they’re purchasing a lot of the tools—such as when Google acquired Looker—it pushes cross-system integrations out of the equation and becomes a risk for a lot of the folks that have used existing services.
Another positive is that the industry is unlikely to have a single monopolistic ecosystem. Rather, the industry will likely have multiple different ecosystems, especially around Snowflake and Databricks, where they’re likely going to be competing with each other in the years to come. I believe this drives more innovation, more investments, and ultimately leads to better outcomes. It might be a duopoly, but let’s see how that goes.
What is the biggest misconception about the industry that you would like to see put to rest in 2025?
Gu: That AI is going to replace data teams.
Some people might think that once organizations have these intelligent systems that can figure out all the nuances and drive higher-level reasoning, those organizations will no longer need data analysts. They believe at some point, when AI automates a lot of the existing processes, organizations will get rid of the data engineers, too.
I don’t think that’s the future our industry is headed toward. Organizations do not appear to be replacing data teams; they appear to be upskilling them. They’re giving data teams more efficiency to do their work. If an organization wants to be able to drive the next outcome or to be truly data-centric, they’ll likely take the existing data teams and start spreading them across different departments. That way, they’re empowering the entire organization toward a decentralized approach, relating back to the idea of cross-functional teams organized around desired outcomes.
Do you see any emerging trends in data management that will impact the industry in 2025?
Gu: I’m starting to see the concept of a data platform team become a reality. Organizations realize that in order to achieve a decentralized vision where every single business unit is empowered through data analytics to drive decision-making, they will likely need to invest in a data platform team.
Functionally, a data platform team’s goal is to integrate, accelerate, and automate all the fundamental aspects of driving analytics for the organization, whether ingesting data at scale, modeling data, curating data, enabling dashboards, handling deployment, and so on.
This is foundational infrastructure that the data platform team would be managing and deploying at scale. The end goal is enablement of analytics velocity and allowing each team to drive from the moment they have an idea to having an insight from a dashboard or a report. The end goal is also being an AI/ML model to drive the analytics velocity to the point where the team is able to produce more models—not in months or weeks, but in days.
“To achieve positive outcomes, organizations will likely need more integrated teams across the board that are cross-functional and not just made up of technical folks. Organizations should consider having folks from the business side directly be part of these teams. Technical teams by themselves are less likely to succeed.”
For more insights and predictions for the data industry in 2025 and beyond, download our report featuring contributions from top industry executives and thought leaders.