Databricks Summit 2025 Recap: Key Themes Shaping the Next Phase of Data + AI

Now that the dust has settled and the post-conference buzz has quieted, it’s become very clear that the core themes throughout the event centered on unifying data and democratizing access to it.  The Databricks Data + AI Summit 2025, held in San Francisco from June 9th to 12th, attracted a staggering 22,000 attendees, indicating a sharp rise in interest in how Data and AI collide. And the trends that emerged are, it turns out, deeply intertwined with the very challenges Hydrolix is designed to solve.

Over a few days, I immersed myself in the ideas and conversations shaping the future of Data and AI. Staying abreast of industry trends and anticipating future directions is crucial as Hydrolix aims to lead the streaming data lake market, and in this post I touch on some of the major trends emerging for AI data. 

Unifying Data

A core message throughout the summit was the importance of consolidating all data into a single location. Doing this helps companies: 

• Avoid fragmentation and the high costs that come from maintaining and connecting disparate systems.
• Bring apps to where the data is rather than building multiple paths to multiple sources.
• Reduce the cost of so many vendors.
• Avoid unnecessary lock-in to a vendor that doesn’t solve the majority of the problem.

The benefits of consolidating data can really be seen in the discussions around democratizing data: when data becomes much more accessible, it is easier to extract and create value. 

One way to address some of these needs could be using open lakehouse formats like (Delta Lake or, Apache Iceberg)  which can be built upon to provide comprehensive governance across all data assets, AI models, and dashboards. If you don’t have the time or expertise to build out a data lake at the scale required by AI efforts, you can also use a prebuilt solution.

At Hydrolix, we empower customers to unify their data and query in a single location. Thanks to Hydrolix’s unique compression technology, retaining all your data to support data-intensive and AI use cases is now more affordable than ever.

Data Intelligence and Democratizing Data

Databricks anticipates that Data + AI will revolutionize all software by 2030, with a significant emphasis on “Data Intelligence.” Data Intelligence refers to the symbiotic relationship between AI and data. Both using AI to better understand and have greater access to data, and using data to make AI better. This concept centers on democratizing data access and leveraging AI to comprehend and utilize data across various applications. Solutions to facilitate this democratization included user-friendly tools like Notebook Assistant, which streamlines data and machine learning queries.

AI thrives on data, and the greater the data volume, the more effective the AI. Part of democratizing data includes expanding the availability of that data over longer periods of time. If everyone has access to data, but it only covers the last 30 days, are we unlocking insight—or limiting it? 

This is where Hydrolix comes in. We offer a more cost-effective and efficient solution for storing always-hot, petabyte-scale event data. This data is ideal for providing context to AI systems and training AI models. Our ability to provide “always hot” data that is queryable in seconds and minutes, rather than days, truly democratizes data. Our MCP server also facilitates natural language interaction with your data.

The Future of Data and AI

From the attendance and interest at the conference, it’s clear the industry has a deep interest in an AI + Agentic Future. The platforms discussed, including Mosaic AI and the new AgentBricks, empower customers to build and productionize AI agents, tackling key challenges like performance optimization and striking a balance between cost and quality. Enterprises are already racing towards this agentic future with workflows designed to increase internal productivity and enhance customer experiences.

Underlying this agentic future is data and as enterprises strive to mobilize and democratize their data, streaming data lakes such as Hydrolix are becoming foundational. These solutions are vital for managing the immense data volumes required without incurring excessive costs.

Apache Spark, Now with More Superpowers

There were a number of exciting announcements at the Databricks Summit, but something I was specifically tuned into was all of the Apache Spark announcements. Apache Spark 4.1 introduces several enhancements for pipeline orchestration, focusing on developer-friendliness and production readiness. These include unified pipeline definitions and execution, tightly integrated with Spark, and Python/SQL API support to improve accessibility and ease of use. 

All of these exciting updates mean you can now do more with your Hydrolix data, thanks to the Hydrolix Connector for Apache Spark!

Wrapping Up

Data has always been crucial for business success, and its importance is only amplified by the emergence of AI. The interplay between enhancing AI value with data and making data more accessible through AI will be an ongoing theme, particularly with the growth of agentic workflows from the data layer upwards. Hydrolix addresses these areas by facilitating large-scale data access via the Hydrolix Connector for Apache Spark and simplifying data access through our MCP server.If you attended the Databricks Data + AI Summit , I’d love to hear what resonated most with you —or what you’re already experimenting with back at work, connect with me on LinkedIn.

Next Steps

• Learn more about Hydrolix. 
• Learn more about TrafficPeak. 
• Learn more about Hydrolix for AWS.
• Learn about using the Hydrolix Connector for Apache Spark with Databricks.
• Contact Hydrolix about a proof of concept or demo.