Hydrolix Delivers Origin-to-Edge Observability for AWS

If you’re using Amazon services like CloudFront, analytics tools can give you detailed, real-time observability for your applications and digital assets.

There’s just one problem—origin-to-edge observability can be extremely challenging because edge services such as CDNs can produce huge volumes of logs, especially for major enterprises in verticals like broadcasting, gaming, and banking.

Enterprises dealing with terabytes of log data every day typically have to make major compromises, whether that’s short “hot” data retention windows, high costs, or discarding data altogether. Those compromises can lead to observability gaps, making it harder to monitor CDNs, detect DDoS attacks, and understand how applications are performing.

We’ve designed Hydrolix for AWS to serve a need when it comes to AWS observability: a cost-effective solution that gives you origin-to-edge observability for use cases like high data volumes, major events, and long-term data retention. All without the tradeoffs that typically come with large volumes of log data. You can keep all your data, eliminate gaps in your observability, and keep costs down. And Hydrolix for AWS is now available in public preview with a free 7-day trial in the AWS Marketplace.

Hydrolix will give you a deeper understanding of your Amazon estate, potential security risks, valuable information about end user behavior, multi-CDN monitoring, and more. And you’ll be able to monitor your estate at scale—it’s designed for users ingesting at least 1 TB/month, with no upper limit.

The next image shows some of the metrics you can monitor along with predefined filters so you can easily drill down into your data.

With Hydrolix for AWS, you get:

• Real-time analytics of all your CloudFront and WAF logs (with more AWS services coming soon)
• Multi-CDN monitoring so you can analyze data across CDNs
• 15 months of queryable log retention
• Full data ownership in your own S3 bucket for privacy, data sovereignty, and compliance
• Sub-second query latency whether your data is a minute or a year old
• A solution that’s more cost-effective for large volumes of data than competing solutions ($.45/GB of ingested data)
• 90% compression to minimize your storage footprint

And that’s just the start. Over the next several months, we’ll be adding support for more services, with the goal of providing complete origin-to-edge observability.

If you take part in the public preview, you’ll have the opportunity to help drive the product roadmap. Have a difficult observability problem, pain point, or an analytics issue that currently costs too much? You can talk directly with our team to learn how Hydrolix for AWS can help solve them.

Get Instant Observability With Prebuilt Dashboards

Hydrolix for AWS comes with prebuilt dashboards for monitoring CDNs, CloudFront logs, and more. The next image shows just a few of the metrics that are useful for CDN monitoring, including HTTP status code, cache hit rate, total bytes delivered, and error rate.

You also get a high-level view of CDN performance by geos, including total request count, total bytes delivered, the offload hit percentage, and the overall error rate for each geo.

You can customize dashboards for more detailed geo information that can support use cases beyond observability. For instance, an online retail store might want to quickly see which product pages and images have the highest count by geo for marketing, advertising, and data science use cases.

Hydrolix’s underlying architecture gives you exceptional needle-in-a-haystack query performance. That means you can quickly filter and find the raw logs you need.

The following short video demonstrates how easily you can drill down into your raw logs.

As the video shows, you can:

• Open a list of time-filtered raw logs, then drill down into individual logs.
• Hydrolix supports high-dimensionality logs—logs can have hundreds or even thousands of dimensions. The logs shown in this video include CloudFront dimensions.
• Select a specific dimension in a log line for easy comparisons against other lines from the filtered time period. This example drills down into total bytes delivered.

What Makes Hydrolix for AWS Unique: Powered by Hydrolix’s Streaming Data Lake

What makes Hydrolix for AWS different from other observability products? It all starts with Hydrolix’s streaming data lake, which is specifically designed for ingesting, transforming, storing, and querying large volumes of log data. Observability use cases like multi-CDN monitoring can generate a lot of logs, especially for major enterprises in verticals such as broadcasting, gaming, and banking. We work with customers that regularly generate 30TB/day of log data. They need a reliable, cost-effective solution that allows them to ingest their data at high scale (including during peak events) and then quickly act on performance issues, DDoS attacks, and more.

With Hydrolix’sstreaming data lake architecture, it typically takes about ten to twenty seconds for data to be logged, ingested, transformed, and ready for querying—even when ingesting millions of log lines per second. This is far faster than what typical observability solutions offer, and at a volume that major competitors are not able to match. Hydrolix’s streaming data lake architecture also gives you low-latency queries even on trillion-row datasets and high-density compression (90% or more) in S3-compatible storage so you can keep your data as long as you need for long-term analytics and compliance.

With log data volumes continuing to increase across the board for enterprises both large and small, it’s essential to have a future-proof monitoring solution that gives you the scale you need not just now but in coming years. This is in contrast to legacy observability architectures that already can’t provide the scale that many enterprises need.

That’s exactly what Hydrolix for AWS gives you: cost-effective insights into all your data at any scale for both now and in the future. You get full observability from the origin to the edge.

Get Started With Hydrolix for AWS

You can start ingesting data and getting insights in under twenty minutes. Follow along with the Getting started instructions in our docs and sign up for a 7-day free trial.