Dark data is one of the biggest challenges in cybersecurity. To prevent it, you need long-term retention, real-time analytics on all your data, and cost-effective storage that allows you to store massive volumes of data without breaking the budget.
That’s exactly what Hydrolix provides for The Honeynet Project.
In the latest episode of Hot Data, Krassimir Tzvetanov shares his insights into how Hydrolix is supporting cybersecurity research through its partnership with The Honeynet Project. This collaboration demonstrates how innovative storage technology can empower community-driven security initiatives.
A Commitment to Open Source and Community Support
Hydrolix has consistently demonstrated its dedication to supporting community projects and open source initiatives. With CEO Marty Kagan’s backing, the company has contributed to organizations like NANOG (North American Network Operators Group) and now The Honeynet Project. According to Tzvetanov, who serves on the NANOG program committee and has been involved with The Honeynet Project for 15 years, this support reflects a broader commitment to advancing the cybersecurity community.
Solving a Critical Challenge for Security Research
The Honeynet Project faced a significant obstacle: the lack of centralized data storage across its various chapters. Each chapter operated independently, making collaboration and data sharing difficult. Tzvetanov, who previously ran the Bay Area chapter and is now establishing a chapter at Purdue University, recognized that Hydrolix’s technology offered an ideal solution.
“We have the perfect solution for that,” Tzvetanov explained. “It’s not only the perfect solution, but it’s also fairly inexpensive.”
Hydrolix delivers what legacy platforms can’t do: getting insights in seconds from globally distributed infrastructure while enabling years of complete data retention. For an organization like The Honeynet Project, Hydrolix solves both scale barriers—bringing together data from chapters worldwide while maintaining the real-time performance needed for threat detection and security research.
Sponsorship and Storage Infrastructure
Hydrolix is sponsoring The Honeynet Project by providing both storage infrastructure and covering compute costs. Importantly, the company is not profiting from this arrangement or using the data for commercial threat intelligence. Instead, Hydrolix enables different Honeynet chapters to utilize the data for their research purposes.
According to Michael Cucchi, the CMO at Hydrolix, “A large part of what Hydrolix does is assist with the collection and analysis of critical real-time threat data for security teams. So we were proud to support The Honeynet Project, where our platform provides a real-time analytics layer for their researchers to monitor, identify, and investigate existing and emerging threats across the globe.”
This approach is particularly valuable for chapters operating on limited budgets. The Purdue chapter, for instance, will function as a student club with no dedicated funding for storage infrastructure.
Impressive Compression and Performance
The technical benefits of Hydrolix’s storage engine are substantial. The system has collected over 678 gigabytes of raw data from honeypots, compressed to just 30 gigabytes—a 22x compression ratio. For Suricata detection data, 250 gigabytes is compressed down to 10 gigabytes, achieving a 23x compression ratio.
Beyond compression, the Tzvetanov demonstrated querying data spanning nearly two years, with results returning in seconds. This access enables researchers to investigate incidents that may have occurred long ago—critical for identifying advanced persistent threats that can remain undetected for months or years.
Unified Data Access for Collaborative Research
By centralizing data storage, Hydrolix helps all Honeynet chapters to access a unified dataset rather than maintaining individual copies. Researchers can analyze patterns across multiple honeypots, examine traffic from different sources, and conduct investigations using SQL queries through a standard API.
The platform supports visualization tools like Grafana, allowing researchers to explore data ranging from SSH scanner activity to HTTP exploit attempts. The system stores detailed information, including IP addresses, autonomous system numbers, geographic origins, and specific attack signatures.
Empowering the Next Generation of Security Researchers
As Tzvetanov noted, The Honeynet Project provides invaluable data for security research with a unique time span and scope. The Purdue chapter and others can now conduct sophisticated analysis without worrying about storage costs or infrastructure management.
This partnership exemplifies how technology companies can support the broader security community. By providing efficient, cost-effective storage infrastructure, Hydrolix is enabling researchers, students, and security professionals to focus on their core mission: understanding and defending against cyber threats.
Next Steps
Interested in solving some of the biggest challenges in cybersecurity, including cost-effective long-term retention for real-time security analytics? Get a demo.
