Blog List


Ponderings, insights and industry updates

  • Transform Your Data With Hydrolix
    Learn how to write transforms that give you maximum flexibility for processing, enriching, and standardizing your log data.
  • Latest N Rows Optimized: Crafting Efficiency with the Hydrolix Catalog
    Hydrolix can execute queries in a massively parallel fashion, thanks in part to its distributed nature and time-based partitioning. With time series data, time filters are often the preferred method … Read more
  • Analyze your Hydrolix Data with Apache Spark
    Big data can be challenging to scale and even more challenging to query. Without the right tools, you’ll likely run into performance issues like query timeouts, making it difficult or … Read more
  • Scaling Query Performance With Big Data
    Achieving consistent, low latency query performance in analytic systems can be challenging when working with a mix of workloads and varying numbers of concurrent queries. Hydrolix’s distributed architecture allows for … Read more
  • Summary Tables Simplified
    The latest version of Hydrolix Summary Tables significantly reduces the complexity of writing SQL aggregates while preserving the efficiency of querying summary tables. A common limitation of aggregate tables is … Read more
  • Hydrolix Supports Mode Analytics
    The Mode platform provides users with a hub for performing ad hoc analysis, self-service reporting and visualization, as well as sharing analysis with collaborators. In addition to providing SQL access … Read more
  • MaxMind Geo Lookup
    How to setup Hydrolix geo-lookup using MaxMind database using custom dictionary and function to simplify ingest and query.
  • Clean your data quickly
    Data are complicated, Hydrolix helps clean up your data before ingesting it.
  • Multi CDN monitoring
    How Paramount leverages Hydrolix to ingest, normalize their CDN logs from multiple providers
  • High performance dashboard with Grafana
    In this blog post we’ll see how to leverage Clickhouse plugin and Grafana to create really performant dashboard on Hydrolix data platform. Table of ContentsDashboard VariablesImprove your SQL statementAdd username … Read more
  • Monitoring query performance
    In the previous blog post eating our own logs we saw how Hydrolix cluster is indexing its own logs. Our logs contains lots of very useful information, in this blog … Read more
  • Eating our own logs
    Monitoring complex application deployed via Kubernetes can be a challenge especially when the application uses a large number of components. The Hydrolix platform is designed to make log processing and … Read more
  • Logs and metrics with Fluentbit
    Fluent Bit is a super fast, lightweight, and highly scalable logging and metrics processor and forwarder. In this article we’ll see how we can configure FluentBit to send metrics and … Read more
  • Handling Multiple Groupings in a Single Pass
    Handle multiple group in a single SQL statement without blowing up memory !
  • Video streaming analysis with CMCD
    In this blog post we are going to see how we can leverage a new feature to extract CMCD information from Akamai realtime logs delivery. The Common Media Client Data … Read more
  • Index and enrich Akamai logs in realtime
    Akamai new feature Datastream v2 allows streaming in real time of logs into HTTPS endpoint. In this blog post we’ll see how to setup Akamai and Hydrolix to receive, parse … Read more
  • Clickhouse Meetup
    Our very own Fed presenting Hydrolix to the clickhouse meetup in SF In his talk Fed talks about how Hydrolix forked Clickhouse to support high performance reading and writing to … Read more
  • When Append-Only Doesn’t Cut It
    Alter is a new mechanism at Hydrolix allowing user to modify and delete indexed data. When alter is running it has no impact on performance for users with our decouple approach. Alter is safer than any other approach as we can rollback or commit the change in an atomic way with our cloud storage approach creating new partition is free.
  • Indexing Journald logs with Vector
    How to index journald logs in Hydrolix with Vector.
  • Managing your noisy neighbour
    How Hydrolix can isolate users and teams to use dedicated server pools, avoiding noisy neighbours and improve reliability for everyone.
  • Big Data with small bill
    A month ago I started indexing Certificate transparency logs into Hydrolix.I’ve already created a blog post about this that you can view here. This one is more about the performance … Read more
  • Indexing Elastic Common Schema
    Elastic has been working for awhile now on standardisation for event data, this effort is called the Elastic Common Schema, which we’ll call ECS in this post. The principle is … Read more
  • Visualising Certificate Transparency Logs
    Everything is moving towards TLS encryption all the time and monitoring those certificates generated is very important for multiple reasons: Detection of malicious certificates Monitoring of mistakes An example of … Read more
  • Automatic Index Cloudfront Logs
    Hydrolix is able to automatically import batch data from AWS S3. In this blog post we’ll see how we use this feature to index Cloudfront logs into Hydrolix. Table of … Read more
  • Using Hydrolix with VSCode
    On top of being a great code editor, Visual Studio Code (VSCode) has a great ecosystem with various plugins allowing users to automate many different tasks. Hydrolix being a fully … Read more
  • Hydrolix from Zero to Ingest
    As I recently joined Hydrolix, literally three days ago, I wanted to share my experience from not knowing the product to starting to use it very quickly. Table of ContentsStep … Read more
  • Visualizing Fastly data with Hydrolix and Grafana
    In this post, we will show you how to setup Grafana to visualize your Fastly CDN data which you have streamed to Hydrolix. This post is part of a series … Read more
  • Analyzing Fastly CDN transaction logs with Hydrolix
    In this post, we will show you how to analyze the Fastly CDN logs you have streamed to Hydrolix. This post is part of a series showing how to use … Read more
  • Hydrolix Streaming Intake for Fastly CDN Logs
    In this post, we’ll show you how to configure Hydrolix to ingest your streaming Fastly CDN logs. This post is part of a series showing how to use Hydrolix and … Read more
  • Streaming Fastly CDN Logs to Hydrolix
    In this blog post, we’ll show you how to configure Fastly to stream logs to Hydrolix with just a few clicks and in less than 10 minutes. This post is … Read more
  • Hydrolix 18x Faster than Elasticsearch
    Mark Litwintschik, consultant, tech author and prolific benchmarker of databases, ran Hydrolix through its paces recently and published the results on his blog. Starting with a data set of 1.1 … Read more
  • Hydrolix: a Low Cost, High- Performance Data Lakehouse
    At our last company, my co-founder and I saw our data volumes grow to over 15 billion transaction logs a day. As our infrastructure costs went through the roof, we … Read more