When your data lake turns into a swamp, you don’t lose visibility all at once. You lose it in the moments that matter most. The login surge that locks out employees. The app slowdown your SRE team cannot explain. The breach that only becomes obvious after the attacker is long gone.
Pramod Gosavi has seen all of this from the inside. An engineer and former operator at large enterprises like VMware who now invests in infrastructure and security at Blumberg Capital, he’s spent his career on both sides of the table.
“The first half, I built products. The second half, I was more on the business side. So I have both bottom up and top down perspective.”
Today, his message to data operators is simple: slow or hard to find data is not an analytics problem. It’s a business continuity problem. And your only real defense is hot data that’s ready when incidents happen, not hours later.
This is his playbook for using hot data as a defense strategy against business disruption.
Meet the Operator: Why Pramod Thinks in Incidents, Not Dashboards
Before you apply any of the following strategies, it helps to understand how Gosavi frames the job. He doesn’t start with tooling. He starts with failure modes.
From VMware’s virtual desktop infrastructure, where thousands of employees would log in at 8 or 9 in the morning, to modern security programs, the pattern is the same. Shared resources, small misconfigurations, and noisy neighbors create cascading issues that are impossible to debug if your telemetry is slow or buried.
As Gosavi recalls, “Eight or nine o’clock when all of the employees join, they all need to fire up their desktop and that desktop actually gets relayed from the data center. There is limited bandwidth on the network, so everything is extremely slow. One user is slow, starts hogging bandwidth, other users cannot log in, applications are crashing.”
Every action that follows is about avoiding that scenario by design.
1. Spot the Swamp Before It Sinks You
Objective: Stop treating your data lake as a dumping ground and start treating it as a production system that can fail.
For years, the default motion was simple: ship every log into a central data lake or log analytics cluster, then decide later what matters. In reality, “later” often meant “never.”
Gosavi has seen this outdated practice time and time again, “You have these log files coming in, they get thrown into a big data lake. I call it a swamp. It is very hard to pull data out and it ends up becoming less and less useful.”
Instead, teams need to treat their data lake like a production system, designed around the moments when something breaks, not as a place to dump everything and sort it out later.
Put this into practice by:
- Mapping your most critical incident flows. For each one, list the data sources you actually used the last time something broke.
- Classifying those sources as “must be hot” versus “nice to have later.”
- Reducing blind ingest from systems that have never helped you during an incident.
The goal is not to collect less data. It’s to make sure the data that matters most never sinks below the surface.
2. Work the Known Knowns First
Objective: Turn incident investigation from a wild goose chase into a repeatable triage sequence.
Gosavi uses a simple mental model for incident response: known knowns, known unknowns, and unknown unknowns:
“You do not want to do a wild goose chase. First you want to make sure of your known knowns. If an application is down, what changed between last time and this time? Were all the files uploaded? You start there.”
Put this into practice by:
- Standardizing your first 15 minutes.
- Check recent deploys, config changes, infrastructure updates.
- Confirm expected files and jobs actually completed.
- Instrumenting known unknowns.
- Make sure you can pull latency, error rates, and resource metrics in real time for your top services.
- Treat these metrics as hot data by default.
- Reserving AI for the unknown unknowns.
- Once the obvious questions are exhausted, use AI and large scale search on your logs to surface patterns no human could scan for quickly.
“AI makes finding hard-to-find data or insights very cheap. You can throw tons of terabytes of data and run queries. It is just not possible for a human to even read it at that speed, let alone understand what is going on.”
Gosavi’s advice? Start off simple, then gradually review the unknowns as needed. Don’t start with mystery hunting if you haven’t figured out the basics.
3. Design Your Data Plane for Attacker Speed
Objective: Assume attackers will move faster than your current investigation workflow and build your telemetry around that reality.
In security, Gosavi says time is weaponized: “The hackers, the way they operate is funny. They will come in and act as if nothing happened and you cannot figure it out. But when they act, they act very fast. They know they are going to get caught very quickly. When you look back, you see those signs, but nobody looked at them in time.”
Security teams already think in tiers: endpoint, network, identity, application. The weakest link is often the data plane that sits underneath those tiers.
Put this into practice by:
- Defining a small set of security critical signals that must always be hot: auth anomalies, privilege changes, new network routes, data exfil patterns.
- Segmenting those signals by geography, entity, or business unit so you can localize investigations instead of querying one global swamp.
- Monitoring basic health metrics like resource usage and latency at the edge, not only in your central lake.
Hot data here is not a luxury. It’s the difference between catching an attack in progress and reading about it in your own post mortem.
4. Redefine Hot vs Cold Data by Importance, Not Age
Objective: Move away from time-based definitions of hot and cold data and toward impact-based ones.
Most teams define hot data as “recent” and cold data as “older.” Gosavi uses a different filter: “Normally people talk about hot data from a duration point of view. I like to do it from importance. At any given time, you are going to suck in all the data and you cannot analyze all of it. You have to conceptually divide data into hot and cold.”
Put this into practice by:
- Marking any signal that reflects live system health as hot data, regardless of how long you keep it. This includes server metrics, network metrics, latency, error rates, queue depth, and key business events.
- Treating everything else as cold by default until it proves its value in an incident.
- Using pre-filtering and routing in your telemetry pipeline to:
- Normalize and compress hot streams for fast access.
- Divert low value verbose logs directly into cheaper storage.
Remember that “hot” is a function of context. A one-week old authentication log is cold until you are in the middle of a breach investigation. Then it becomes the hottest data you have.
5. Reframe the Executive Conversation Around the Cost of Late Insight
Objective: Help executives understand that the real risk is not storage cost, it’s time to answer.
Sooner or later, every data team gets asked the same question: why are we spending so much money on data infrastructure?
Gosavi’s prompt to operators is to shift the conversation. Don’t start with how much data you are storing. Start with what happens when you cannot analyze it in time.
“If I am not analyzing at the right time, I am wasting all my time and energy. I am spending so much time and money on a log analytics platform, but if it is taking a day or half a day to analyze the data, it is a waste of time because the damage has already happened. The applications are down, the hacker is already in.”
Put this into practice by:
- Quantifying the cost of your last major outage or incident across:
- Lost revenue and productivity.
- Breach or compliance exposure.
- Customer churn and reputational damage.
- Connecting each cost back to a missing or delayed data signal.
- Positioning spend on hot data infrastructure as insurance that pays out through shorter incidents, less blast radius, and fewer surprises.
When you frame it this way, “we are capturing all this data” stops sounding like safety and starts sounding like preventing risks.
Final Takeaway
Your job as a data operator is not simply to collect and store logs. It’s to make sure the right data is always hot, always ready, and always fast enough that the business feels the benefit before the incident does the damage.
Gosavi adds that maintaining this speed starts with real-time analytics. “I am a big fan of real-time analytics, analyzing the data quickly. Otherwise we just store a lot of data and eventually it goes nowhere. Everywhere, whether it is marketing, sales, cybersecurity, IT, we need to move towards more real-time,” he says.
Use this playbook to get there: drain the swamp, work the known knowns, design for attacker speed, redefine hot data by impact, reframe the cost conversation, and put an engine like Hydrolix behind it all so your hot data strategy is not just theory, it’s how your systems actually run.
Hydrolix is a data platform designed for real-time and historical analytics across petabyte-scale data. You can keep and analyze all your data at any time, whether it’s a minute old or a year old, with sub-second precision queries and high-density compression on object storage that keeps long-term storage cost-effective.
With Hydrolix, data teams can:
- Keep critical signals hot for incident response, streaming quality, and security.
- Retain far more history without exploding costs.
- Make sub-second precision queries on petabyte-scale data to help find root causes and drill down into specific incidents.
In every domain, from IT operations to cybersecurity to product analytics, the teams that win are the ones who can turn raw telemetry into fast, reliable decisions at scale.
