Evaluate in the Offseason to Improve Your Observability Game

For professional sports teams, the offseason is where the real work begins. That includes scouting, preparation, and getting your favorite teams up to speed for the biggest games of the year.

The process of monitoring peak events—sporting and otherwise—isn’t really that different. It involves a lot of scouting and preparation. For event monitoring to go well, much of the work has to happen in the offseason, and maintaining a high level of performance gets more challenging for bigger events. Just as importantly, enterprises are more likely to opt for a code freeze than add major new capabilities or switch platforms during peak season. If you’re not careful, you can be stuck with an ineffective platform at a time when you most need visibility into your systems.

So how do you scout out the right platform for monitoring peak events? A low-volume, self-serve trial won’t cut it. You need a proof of concept (PoC) to ensure that the platform you’re considering can meet the scale you need. Self-serve free tiers can help you evaluate the UI of a product and test whether you can ingest log data at low volumes. But to truly understand how well a product can handle major events, you typically need to go through a more involved process like a proof of concept.

Hydrolix is a streaming data lake designed for log-intensive use cases, including monitoring crucial data around peak events such as CDN and network logs. Hydrolix is ideally suited for monitoring large-scale events, and some of our biggest customers are major broadcasters and internet service providers. Our customers regularly monitor major events, including the biggest professional football game of the year (the one we can’t name because it’s trademarked).

In this post, you’ll learn about Hydrolix’s trial process (scouting) and how our customer success team works with enterprises to help them understand whether Hydrolix is a fit for their use cases.

Big Game Observability Requires Scouting

Observability platforms often have a free tier so you can do a basic evaluation of their products. However, this level of evaluation isn’t sufficient for determining whether a platform can handle events that generate billions of log lines in a matter of hours. For the biggest football game of the year, a major broadcaster used Hydrolix to ingest and more than 10 million log lines per second with the ability to query data immediately. Many other observability platforms go belly up and time out at that level of scale, something we’ve seen for ourselves when we’re working alongside customers that use both Hydrolix and other platforms together.

It doesn’t matter how nice your dashboards, alerts, and visualizations are if they time out during the major events when you need them most. At that point, you’re in the dark. It’s too late. And unfortunately, this happens to a lot of teams that are diligent about the evaluative process. It’s hard to know if your platform can handle the equivalent of a four-alarm fire until you’re in the middle of it, and too many teams have to deal with the low morale of a post-event retro where everything went wrong.

Scouting Hydrolix: Understanding the Trial Process

Enterprises need proof that the platform they choose can actually handle major events. Customer stories can provide helpful testimonials, but technical decision makers and their teams need hands-on opportunities to evaluate whether products fit their unique use cases. Hydrolix provides streaming ingest and data analysis at exceptional scale, and decision makers often come to us after failed trials elsewhere. We regularly hear variants on this refrain:

“We’ve gone to all of the other major observability providers and they can’t do the scale we need. What you’re promising is too good to be true.”

We understand the skepticism. Real-time monitoring at scale is hard. Technical decision makers know the challenges of working with large distributed systems. The bigger they get, the more potential there is for things to go wrong, and the harder it can be to pinpoint issues. And nobody wants to be troubleshooting issues with their observability platform when they need to be focused on monitoring their systems instead.

So how do we prove Hydrolix can handle the load? It starts with the proof of concept, which typically runs for thirty days and includes no license fees.

According to Greg Hallman, a senior sales executive at Hydrolix, the goal of a PoC is to “prove out scaling and high volume capabilities, run sample dashboards and ad hoc queries, see real world cloud costs, and compare performance and pricing versus other data platforms.”

According to Dan Sullivan, Lead Solutions Architect at Hydrolix: “Our customers have used PoCs to verify that Hydrolix can scale to meet the demanding scaling needs of online gaming companies and live sports event streamers.  Other customers have used PoCs to prove Hydrolix can implement their complex business logic in real time and at scale as data is ingested with no need to land and then transform data.”

While a 30-day trial is typical, some customers run longer PoCs, and these trials can even include peak events, so you get a chance to try the product in real-world conditions at scale. There is also the option to do a self-managed trial of Hydrolix through the AWS or GCP marketplace, which is limited to 30 days. You still get email support from Hydrolix’s customer success team, but it’s important to have familiarity with Kubernetes if you want to self-manage. Typically, we recommend a PoC to ensure success, fine-tune Hydrolix for your use case, and answer any questions you have. Why turn down additional help and support when it comes at no cost and Hydrolix’s success team can quickly get you up and running?

What to Evaluate During the PoC Process

During the PoC process, you should evaluate the following:

• Can the platform ingest data at the scale you need? A good PoC gives you the opportunity to model the solution at any scale. You can use a Hydrolix PoC to monitor and test actual real world peak events, even in conjunction with any solutions you’re already using. This is also the time to set up and test your ingest pipeline. In the case of Hydrolix, that’s typically Kafka, Kinesis, Hydrolix’s streaming API, or a combination.
• Can you transform and shape data to fit your use case? Many platforms don’t include data transformation, and you either need to add a separate tool for transformation or transform data after it’s already been stored, which is computationally expensive and can cause other issues such as lost context. In the case of Hydrolix, all data is transformed in real time before storage. These transforms are extremely flexible and can help you enrich, standardize, and obfuscate data as needed.
• Are storage, compression, compute usage, and query performance in line with expectations? If you’re evaluating a SaaS platform, you’ll need to look closely at your consumption (if the platform uses a consumption model) to determine if it’s in line with expected costs. If ingesting and querying data at scale comes with the high likelihood of eye-popping bills, the solution probably isn’t a good fit. With Hydrolix, users typically see 20x-50x compression and sub-second query performance even on very large data sets. With a PoC, you can directly experience Hydrolix’s performance for yourself, including performance on many different kinds of queries, compression rate on storage, actual compute usage, and more. You have much more visibility and control than you would with a SaaS platform, and you also have full ownership of your data in a virtual private cloud (VPC) of your choosing.
• Can you get support for your use case? Data at scale isn’t a simple commodity problem that can easily be solved with a one-size-fits-all commodity solution. Expertise and support is crucial to ensure that a solution fits your specific use case. Is the support for the solution responsive and available not just during a PoC but during major events? Hydrolix’s success and sales teams are available throughout the PoC process and beyond to answer questions and help you optimize your use case. 
• How can you optimize and customize the solution for your use case? Ingesting, storing, and querying data is just a starting point. There should be additional levers you can pull to further optimize performance and reduce costs. With Hydrolix, you can improve query performance and cut compute costs further with summary tables, ingest multiple data sources into one table, adjust for performance versus cost-effectiveness by fine-tuning Hydrolix’s Kubernetes infrastructure, and more. And because all data is stored in your virtual private cloud, you can potentially negotiate storage and compute discounts with your cloud provider. Hydrolix’s team can also help you optimize and find the ideal queries for your use case.

Next Steps

Learn how Hydrolix provides big game observability:

• Hydrolix Powers Observability at Scale for Akamai
• Hydrolix Delivers Big Game Observability for Major Broadcast and Streaming Content Provider

Get the proof you need firsthand and try Hydrolix for yourself. Contact Hydrolix about a proof of concept or demo.