Content delivery networks (CDNs) provide critical infrastructure for the majority of the world’s internet traffic, and large enterprises typically use a multi-CDN strategy to optimize content delivery for a wide range of use cases ranging from streaming events to major holiday sales.
With CDNs, you might think there’s one less thing to worry about—reliable content delivery. However, it’s actually extremely important to monitor CDN performance and analyze CDN log data for valuable insights when you’re using a multi-CDN strategy. CDNs can have varying performance in different geos as well as outages and other issues just like any other service. And the log data has value for use cases beyond observability and monitoring, including advertising and marketing.
This post will cover best practices for multi-CDN monitoring and how Hydrolix can help you achieve these best practices. Monitoring multiple CDNs can be very challenging for large enterprises because of the sheer volume of log data they emit. At Hydrolix, we see customers that ingest terabytes of CDN log data every day. Many other observability and monitoring platforms struggle to remain performant at this scale. If you’re dealing with large volumes of log data, you need to use a platform (like Hydrolix) that can handle the load.
Let’s take a closer look at some best practices for monitoring multiple CDNs.
Get started with CDN Insights, a pre-bundled solution that allows you to monitor and correlate data across all your CDNs—no custom configuration needed.
Implement a multi-CDN switching strategy.
Large enterprises typically use multiple CDNs to cache content at the edge for use cases like streaming video content. However, this is most effective if you also have a strategy for switching between CDNs to optimize performance. Enterprises typically use a DNS-based load balancing solution, but there are multiple architectures for multi-CDN switching—this resource talks about approaches for streaming video.
Traffic Management Approaches
For effective multi-CDN architecture, enterprises require a strategic traffic management approach to optimize performance, cost, and reliability. Enterprises that distribute traffic intelligently across multiple CDNs typically use a combination of methods, including the following.
Real-Time Performance-Based Routing
Real-time performance-based routing can help ensure optimal content delivery, especially for major events. Enterprises using this approach need to continuously monitor CDN latency, throughput, and availability to make data-driven routing decisions.
Setting up performance monitoring thresholds can reroute traffic that is dealing with issues such as high latency. Hydrolix ingests and analyzes CDN log data in real-time, allowing teams to quickly act on performance trends.
Geographic and Regional Optimization
CDN performance can vary across different regions. When enterprises analyze performance geographically, they can route traffic to the CDN that delivers the best experience for users in specific locations. Hydrolix simplifies this process by allowing you to store and query regional performance data efficiently. Using summary tables, you can track latency, availability, and other key metrics per region, ensuring performant content delivery. You can also analyze long-term data (15 months by default) to better understand how CDNs have performed historically by region.
Cost-Aware Traffic Distribution
Performance and cost balancing are critical in a multi-CDN strategy. By tracking cost metrics, enterprises can analyze usage patterns and optimize distribution without compromising user experience. You can use Hydrolix summary tables to make it easy to compare cost versus performance trade-offs, enabling smarter traffic allocation.
One log destination to rule them all.
In order to monitor CDNs effectively, you need to compare logs and metrics across multiple CDNs. Ideally, you shouldn’t have to perform complex, costly JOIN statements to compare data from different log sources.
In the case of Hydrolix, you can ingest multiple data sources into one table. This is especially valuable for grouping log sources that logically belong together like CDNs. By ingesting your CDN data into one table, you can compare CDN logs using simple queries—no JOINs needed.
Standardize log data across CDNs.
CDN log data often needs additional standardizing to maximize its usefulness. For example, each CDN has its own naming conventions:
| CDN | Akamai | Edgio | Fastly | Standard |
| Field Name | statusCode | sc-status | status | status_code |
If you don’t standardize naming conventions, it’s much harder to query and reason about your data.
Hydrolix transforms incoming log data in real time before storage. You can configure a transform field to fit your preferred naming convention. For example, this JSON configuration maps the field "status" from incoming log data to the column "status_code" in Hydrolix:
{
"name": "status_code",
"datatype": {
"type": "uint32",
"source": {
"from_input_field": "status"
}
}
}Enrich CDN data for added context.
To get the most out of your CDN log data, you’ll likely need to enrich it further. One very common example is the source of the logs. CDN logs typically don’t include the source name. As long as the logs remain in the origin system, their source is obvious. For example, you don’t need a source: "fastly" field if you’re looking at Fastly’s logs within their application. However, as soon as you send the logs to another system, you lose this valuable information if you don’t add context. What use is log data if you don’t know where it came from?
Another example is adding a latency field to compute the difference between log creation time and log ingest time. This is particularly helpful for diagnosing latency issues in your logging pipeline and understanding which CDN data is arriving late.
A third use case is extracting elements from a string and saving them in separate columns for easy query access. CDN log lines include valuable context in the form of URIs, which provide information about paths, query parameters, and more. You might want to extract parameters from a URI query string into separate fields, similar to this example on data enrichment from the Hydrolix docs.
One important caveat when it comes to log enrichment. You either need to use a platform that supports data enrichment at ingest time or add an ETL (extract—transform—load) pipeline that transforms your data before storage.
With Hydrolix, you can use built-in transforms to enrich logs and add custom metadata in real time before storage. These enrichments can be simple (such as new “virtual” fields that specify the source name and location of the CDN) or complex (SQL transforms that compute new fields).
Use summary tables to improve query performance and save money.
A real-time dashboard can make thousands of queries per day. To enhance performance and reduce the amount of query resources, you can store aggregates along with the results of regularly-used queries in a separate table. Depending on the use case and platform, this might be called a summary table, an aggregate table, or a materialized view. Since the costs and resources of these queries can add up over time, summary tables can dramatically reduce compute costs.
Hydrolix uses summary tables to provide accurate aggregates and query results even for data arriving in real time. Learn more about how summary tables work in Hydrolix.
Monitor top IP addresses to identify piracy and DoS attacks.
CDN log data provides valuable security information, including the ability to detect piracy (such as streaming piracy) and DoS attacks. By monitoring the top IPs as well as IP addresses by region, you can determine which users are taking up a disproportionate amount of bandwidth. Detailed IP address analytics can help you find which IPs are involved in stream piracy and which resources they’re streaming. While the obvious answer to stream piracy is to stop it, some enterprises find it even more effective to throttle it. End users will then be more likely to turn to your paid services to avoid buffering and outages. If you’re keeping all your log data, you can measure the results of both of these approaches to make data-driven decisions about what’s best for your organization.
Top IP addresses, top resources requested, and IP addresses by region are all examples of metrics you should be tracking, which means this data should be included in any summary tables you create.
Learn how Elkjøp, the largest consumer electronics retailer in the Nordic countries, used Akamai TrafficPeak, an observability solution powered by Hydrolix, to immediately detect and stop a DDoS attack.
Keep your data long-term for deeper insights.
When it comes to observability, enterprises typically focus on real-time monitoring while neglecting long-term insights. Performance and issue mitigation are indeed top priorities, but they shouldn’t be the only insights you get from your CDN logs. If you discard data quickly or keep it just for compliance purposes (without query access), you lose the potential for much deeper insights. CDN log data contains a wealth of information on user behavior, activity by geos, and much more.
For example, an e-commerce store can correlate the most popular product pages (URI path) and searches (query parameters) by geo to get a better understanding of how their products are performing in each region. In turn, this is valuable data for marketing and advertising, distribution, and determining which products should be continued or discontinued.
Unfortunately, long-term “hot” data retention is prohibitively expensive for many platforms. That’s not the case with Hydrolix, which includes 15 months of “hot” data by default, all while remaining much more affordable than typical observability solutions.
Analyze ROI to maximize the value of a multi-CDN approach.
A multi-CDN approach isn’t just about performance—it’s also about managing costs. Each CDN has its own pricing structure, which typically includes pricing by geographic region and added capabilities such as security and bot mitigation. Factors such as traffic and organizational priorities may vary from region to region. For example, an enterprise may have a particularly large audience or a major event in a specific region that requires coverage from multiple “premium” CDNs. Meanwhile, a region that has considerably less traffic may not require as much CDN coverage.
By monitoring CDN traffic by region, you can get a better understanding of which regions provide the best ROI for CDN coverage—and where you can potentially reduce coverage to increase savings. Summary tables can provide aggregate data by region over long periods of time, giving you all the data you need to fine-tune CDN coverage across regions.
Look beyond CDNs to better understand your services at the edge.
Servers at the edge can do more than cache content, providing compute, storage, and other services to end users. And the edge isn’t just about servers—it also includes devices such as sensors and potentially even mobile phones. These servers and devices can help transmit information, automate processes (IoT), and provide compute closer to end users.
Yet while edge computing offers many advantages, it also results in more logs, more services to monitor, and a greater attack space when it comes to potential security vulnerabilities. Like CDNs, other edge services can transmit huge volumes of logs, often from many different sources. The ability to ingest grouped data sources into a single table (for instance, IoT devices orchestrating a specific automation) makes it much easier to compare devices and ensure that your systems are operating as expected.
Use CDNs (and monitoring) as part of a comprehensive disaster recovery strategy.
CDNs play a key role in maintaining high availability and protecting uptime. The right observability approach enhances disaster recovery monitoring by unifying CDN access logs, origin infrastructure metrics, and user experience data to provide complete incident context. One common approach for enterprises is to have a primary CDN and then a backup CDN. You can combine a CDN switching strategy with monitoring to set up triggers and alerts to switch CDNs in the case of a major incident. And CDN monitoring can help you more quickly remediate issues, speeding up the recovery process.
Use CDNs to enhance your security posture—while monitoring and protecting an increased attack surface.
CDNs can provide important security benefits like DDoS protection, WAF and protection from bot traffic. However, they can also be a vector for attacks. Fortunately, CDN logs include data that can help protect against common OWASP vulnerabilities. Information like IP address, requests and responses (including headers) can provide rich context about malicious attempts, helping teams proactively mitigate issues. Because CDN logs contain comprehensive data about web activity, they are often one of the most important tools when it comes to understanding the source and cause of an attack vector.
However, root cause analysis is only possible if enterprises keep the underlying raw CDN logs. Since many breaches take months or even years to detect (according to IBM, the average time to detect a breach is 272 days), it’s important to keep that raw data long-term. Because Hydrolix uses high-density compression and object storage, it’s much more cost-effective to keep data at petabyte scale for use cases like forensic analysis—all while keeping data hot for queries regardless of its age.
Next Steps
- Learn how Hydrolix provides multi-CDN monitoring for a major streaming provider.
- Learn about TrafficPeak, Akamai’s edge services observability solution powered by Hydrolix’s data platform.
- Check out CDN Insights and sign up for a technical demo or a free trial.
