Security teams have never had more sophisticated tools for detecting threats. Better analytics. Better detections. More automation. And now AI that can help analysts investigate and respond faster.
But none of that solves a more fundamental problem: you can’t investigate what you can’t see.
Consider what happens after an alert fires. The first questions are usually straightforward. Which user was involved? What endpoint generated the event? What process was running?
Then the investigation expands.
Where else did that account authenticate? What other systems were involved? When did the activity actually begin? Was the same behavior happening weeks earlier? What data was accessed or exfiltrated? Is the attacker still in the environment?
Answering those questions requires data. Often, a lot of it, stretching back weeks or months.
And increasingly, that’s where investigations run into trouble.
The telemetry analysts need has been archived somewhere difficult to access. It was filtered or sampled before it was stored. The retention window expired. Or the organization decided it simply couldn’t afford to collect the data in the first place.
The problem isn’t detection. It’s the economics of keeping the evidence.
The SIEM Was Never Designed to Store Everything
For years, security teams have asked the SIEM to do two very different jobs.
The first is what SIEMs are very good at: monitoring environments, correlating events, detecting threats, generating alerts, and helping analysts investigate incidents.
The second is serving as the long-term system of record for an enormous and constantly growing volume of security telemetry.
Those two jobs have very different infrastructure requirements.
As telemetry volumes grow into terabytes or petabytes per day, keeping every event in a traditional SIEM becomes prohibitively expensive. Security teams inevitably make tradeoffs. They shorten retention periods. Move older data into archives. Filter noisy sources. Sample telemetry. Decide that some data simply isn’t worth ingesting.
Those choices solve an immediate cost problem. But they create another problem that may not become apparent until months later, when an analyst needs the data that was discarded.
That’s the fundamental tension: organizations are forced to decide how much evidence they can afford to preserve before they know which evidence the next investigation will require.
We think there’s a better architecture.
Introducing the Hydrolix Security Data Layer
At Black Hat USA, we announced the Hydrolix Security Data Layer: a specialized data foundation designed to sit beneath the existing security stack.
The idea is straightforward. Instead of requiring the SIEM to be both a detection platform and the long-term repository for every piece of security telemetry, separate those responsibilities.
Let the SIEM do what it does best. Use a purpose-built data layer to ingest and retain complete, full-fidelity telemetry economically while keeping that data immediately searchable.
Hydrolix isn’t another SIEM. And our goal isn’t to give security teams another console where analysts have to spend their day.
The Security Data Layer complements the tools organizations already use. Security teams can continue working in platforms such as Splunk while Hydrolix provides the underlying data foundation for high-volume telemetry that needs to remain available across longer periods of time.
That distinction becomes increasingly important as security architectures evolve.
Detection tools need access to telemetry. Investigation tools need it. Automation platforms need it. AI agents need it. Each may perform a different function, but they all depend on the same thing: access to the underlying evidence.
Data that has been deleted can’t be analyzed by an AI agent. Data sitting in a cold archive can’t participate in a real-time workflow without first being retrieved. And telemetry that was never collected can’t help reconstruct an attack six months later.
A specialized data layer gives all of those systems a common foundation without forcing organizations to pay SIEM economics to retain every event.
Changing the Economics of Security Data
Making that architecture practical requires more than cheap object storage. Security data needs to remain searchable.
Hydrolix was built for high-volume event data. Its compression technology delivers 10-40x data compression, reducing storage requirements by as much as 97% while keeping data available for fast queries.
That changes the retention calculation.
Instead of asking, “How much of this telemetry can we afford to keep?” security teams can start asking the question that actually matters: “How much of this telemetry might we need?”
The difference becomes especially important during investigations that extend beyond the normal SIEM retention window.
An attack discovered today may have started weeks or months ago. An analyst may need to correlate activity across firewall logs, authentication events, endpoint telemetry, cloud logs, and other sources over that entire period.
Keeping more of that evidence shouldn’t mean accepting investigations that are too slow to be workable.
So we decided to test it.
Putting the Architecture to the Test
Over the past several months, Hydrolix has worked with RAD, one of Europe’s leading managed security service providers, to evaluate the Security Data Layer using production firewall telemetry and real-world detection logic.
The question wasn’t whether Hydrolix could replace Splunk. That’s not what we’re building.
We wanted to know whether organizations could retain substantially more security telemetry at a lower cost while still getting the query performance analysts need during an investigation.
The results showed the complementary strengths of the two architectures.
Splunk was faster for highly targeted point lookups, which makes sense for a platform optimized around indexed searches. But investigations don’t always involve looking for one known event. Analysts frequently need to correlate activity across large volumes of telemetry and much longer periods of time.
In RAD’s evaluation, Hydrolix completed broad correlation searches across production firewall telemetry in 1.6 seconds, compared with 14.8 seconds for standard Splunk search and 4.6 seconds using Splunk’s accelerated data model.
The point isn’t that one platform “won” a benchmark.
It’s that security teams don’t necessarily have to choose between retaining more data and being able to investigate it quickly.
Splunk can remain the monitoring, detection, and investigation environment analysts already know. Hydrolix can provide a high-performance, cost-efficient data layer underneath it, keeping substantially more telemetry available when an investigation needs to reach further back in time.
Keep the Evidence
Security has spent years getting smarter.
We’ve built better detections. Better analytics. Better automation. We’re now building AI systems capable of reasoning across security data and assisting analysts with increasingly complex investigations.
But every one of those technologies is ultimately constrained by the evidence available to it.
If an analyst discovers an intrusion today that began six months ago, the most sophisticated detection system in the world can’t reconstruct what happened if the underlying telemetry disappeared after 30 days.
That’s why we believe the next evolution in security architecture isn’t another monolithic platform.
It’s a more specialized stack in which different technologies are optimized for different jobs. SIEMs monitor and detect. Security tools investigate and respond. Automation platforms orchestrate. AI analyzes. And a specialized Security Data Layer makes sure the evidence they depend on is still there.
Because when the next investigation starts, “we didn’t keep the data” shouldn’t be the reason it ends.
Next Steps
- Learn more about the Hydrolix Security Data Layer.
- Talk with an expert about your use case.
