A Splunk dashboard that refreshes every minute runs nearly 1,500 queries a day. At lower data volumes, that’s not a problem. But what happens when your infrastructure is generating terabytes of telemetry data on a daily basis?
At scale, each query can potentially scan billions of rows, leading to slow, inefficient dashboards. Combined with the high costs of a Splunk license, teams are typically forced to sample data or shorten retention windows, both to contain costs and to keep dashboards and queries performant.
Throwing away data isn’t the answer. By combining Splunk and Hydrolix, you can retain full-fidelity data at scale for 15 months, all while reducing the cost of your Splunk infrastructure 6x or more and keeping your Splunk workflows intact. The Hydrolix Search for Splunk app supports Hydrolix summary tables, one of the most powerful features in Hydrolix. Hydrolix summary tables provide real-time aggregations, giving you fast, efficient dashboards even at the massive scale where many legacy observability platforms struggle.
Learn more about reducing your Splunk costs with Hydrolix. Read the whitepaper: Reducing the TCO of Splunk with Tiered Observability: A Strategic Guide for Business Decision Makers.
Summary Tables Give You Pre-Computed Answers
A summary table (also called an aggregate table) stores pre-computed results such as sums, averages, unique counts, and percentiles. Instead of scanning billions of raw rows to answer “what’s my p95 request latency over the last 30 minutes by status code,” a summary table has already done that work. The query touches a fraction of the data.
Hydrolix summary tables have several properties that separate them from basic materialized views. They update at ingest time, not on a polling interval, so they stay current without periodically querying the raw data. They hold intermediate states which can quickly be computed into complex aggregations like unique IP counts and quantiles. You can recalculate aggregations across different time windows without ever querying the raw data. And they’re designed to handle late-arriving data by default: if a log record arrives hours after the fact, summary tables account for it automatically. The practical outcome is that teams can support real-time dashboards at scale without repeatedly scanning massive datasets.
The goal for a well-designed summary table is at least a 97–98% reduction in row count compared to the raw data. At that reduction rate, you’re running a query against a summary table that’s roughly 40GB, not a raw table holding 10 TB of data. Every refresh of your Splunk dashboard is hitting that 40 GB, not the 10 TB. That’s a 99.6% reduction in the amount of data being scanned.
The underlying raw data stays intact. You’re not aggregating while throwing the raw data away. Instead, you get an efficient, high-level view for the majority of your queries with the ability to drill down into individual log records for use cases like root cause analysis and forensics.
Summary Tables in Action for the Biggest Game of the Year
During the 2025 Super Bowl, FOX Sports used Hydrolix to ingest CDN log data across four providers in real time during the game. In roughly four hours, Hydrolix ingested close to 200 terabytes, at a peak rate of 17.4 GB per second.
There were 55,000 queries over the course of the event, with a median query time of under half a second. Each dashboard had a default 30-minute lookback window. Over any given 30-minute span, that meant working with an average of 19 billion raw rows. But the dashboards didn’t query those 19 billion rows directly. They queried summary tables that had reduced that row count by 98.4% to 99.9%, depending on the CDN source.
The result: a median query response time of 0.481 seconds across all dashboards, with data available for analysis 5–10 seconds after the event that generated it. That performance held across a four-hour window of extreme, sustained ingest. The summary tables absorbed the query load while the raw data continued arriving at scale. And the underlying raw data was always available for queries when needed.
There’s no sampling, no gaps in data, and no latency tradeoffs for your dashboards, even for the biggest events of the year. For FOX, this meant operations teams could monitor live streaming performance during one of the highest-traffic events of the year without sacrificing dashboard responsiveness or data fidelity.
How Summary Tables Work With Splunk
With the Hydrolix Search for Splunk app, querying a Hydrolix table, whether raw or summary, uses the | hdxsearch command and standard SPL. From Splunk’s perspective, a summary table query looks the same as a raw table query. The difference is what’s happening in Hydrolix: a much smaller dataset with the aggregations your dashboard needs. Teams using Splunk get all the benefits of faster, more cost-effective queries without worrying about configuration or complexity.
For a security operations team tracking WAF rule triggers, request anomalies, or denied IPs across a 15-minute window, a platform team monitoring API performance and infrastructure health, or a streaming team watching CDN delivery metrics during a live event, the efficiency adds up quickly. For a team with a dashboard that refreshes every minute, the daily query count goes from 1,500 raw-data scans to 1,500 summary-table lookups.
One of the core strengths of Hydrolix is always-hot, full-fidelity data, retained as long as you need it (with a default retention period of 15 months). You can query that data at any time for investigation, correlation, or other use cases where row-level detail matters. Summary tables handle the majority of the monitoring load while raw tables handle everything else. You don’t have to choose between your data and your bottom line.
Meanwhile, circuit breakers built into Hydrolix Search for Splunk protect against runaway queries, so your analysts don’t have to worry about accidentally breaking the bank during deep investigations.
Splunk and Hydrolix: Better Together
Splunk is a world-class observability tool, but it can be extremely costly. A 600 GB/day license for Splunk can cost enterprises more than one million dollars per year. As a result, teams often have to sacrifice data fidelity and retention to keep costs down, leading to gaps for important use cases like security investigations.
Splunk works best when you can keep all your data long-term. That’s where Hydrolix comes in. You can keep high-volume logs in Hydrolix instead of Splunk, reduce your Splunk license costs, and keep all your data.
Summary tables help teams keep months or years of telemetry data while maintaining fast, responsive dashboards. Instead of choosing between retention, performance, and cost, you can optimize for all three. Your operations teams get the resiliency and speed they need even when you’re ingesting millions of rows of telemetry data per second during major events and unexpected traffic spikes. And you can efficiently pull metrics from data that’s months or years old, all while retaining the ability to drill down into the raw logs for deeper investigation.
Next Steps
- If you’re already running the Hydrolix Search for Splunk app and haven’t set up summary tables, start with the summary tables documentation.
- If you’re not using Hydrolix yet, talk with an expert to learn more about how Hydrolix can help enhance your Splunk workflow while bringing costs down.
