A version of this article originally appeared in The New Stack: Rockset Users Stranded by OpenAI Acquisition: Now What?
Migrating to new data solutions can be extremely tough, especially when they catch you by surprise. That’s exactly what happened to users of Rockset when OpenAI acquired the company in June of this year and announced that they’d be offboarding Rockset customers.
The 7 tips included here on migrating to a new solution were originally included in a piece for The New Stack that was a response to the Rockset acquisition. These tips also apply to other types of migrations, such as when you’re forced to consider a new solution because your current one has high costs or no longer fits your use case.
This is exactly what is happening with many log data and observability solutions, which weren’t designed to cost-effectively handle the huge volumes of logs that many enterprises are dealing with today. As a result, many enterprises are forced to reconsider their existing platforms.
These 7 tips can help with surprise migrations. And if you’re looking for a solution that can handle large volumes of log data cost-effectively, then Hydrolix might be a good fit for your business.
1. Be Agile, Not Rushed
Even if you’re forced into an accelerated time frame for migrating to a new solution, it’s still important to evaluate several solutions and undergo a trial process. Business and engineering leaders need to set up preliminary calls with several solutions and see which ones are worth trying in more depth. In this case, being agile means being flexible and adapting based on the solutions available to your organization. It means giving yourself time to understand the transition process and create a strategy. This ties into the next point — knowing when to hold and when to fold.
Read about the importance of the evaluation process for big data use cases, including how Hydrolix conducts trials.
2. Know When To Hold and When To Fold
How well is your current solution working for you? Is there room for improvement — perhaps new opportunities with another solution? Do you need to keep and migrate all the data into a new solution? In this case, “holding” means trying to keep everything as close as possible to how it was — but that may not be possible. “Folding” doesn’t mean giving up. Instead, you can see if the next hand you draw is a better fit.
From a data perspective, if you already have a short retention window or aren’t regularly querying all of your data, you may only need to migrate some of it. Or you can move most of it to a storage bucket and worry about it later. If so, it’s okay to fold. Meanwhile, if you have mission-critical data, that’s a hold — and something to determine in the trial/POC process.
From a features perspective, which ofeatures are absolute must-haves for your integration? Which are nice-to-haves that you can live without in the short term? And which weren’t essential? For example, Rockset distinguished itself as a rare solution combining OLAP functionality with mutable data. However, many use cases involving real-time analytics and log data don’t require mutability. Immutability is typically preferable for log data.
Which features and data must you hold, and which can you live without?
3. It’s All About the People
This is where people are important — not just your people but those working for the vendors with solutions you’re considering. You need to understand the feasibility of a solution, get it up and running quickly, and then integrate it into your business. It’s okay to need help. You’ll likely need all the help you can get. The trial/proof of concept process is an opportunity to quickly get up and running, typically at no cost, with support from another organization that has a strong understanding of how their product works.
If you need to migrate your data, the success team of your next solution should be able to support that. The same applies to setting up different data sources, ensuring that ingested data has the shape you expect, optimizing queries for your use case, and more.
Great people and support are absolute musts for this kind of rapid transition. Of course, work with the people at your current solution as much as possible, too.
4. Don’t Go It Alone
Do you have a team of superstar engineers that can set up an MVP of a real-time analytics database in a few months? Even if you do, resist the temptation to DIY. There’s too much risk involved, at least in the short term. It’s not just about building an MVP — it’s about the optimizations and consistency you need to develop into a viable solution — a polished solution that works as expected. Trying to build a DIY solution can come with a lot of potential pitfalls—some which you might not even recognize until years down the road.
Most importantly, trying to DIY will likely divert resources from the products you need to build.
5. Open Source Isn’t the Only Answer
After an acquisition like OpenAI’s acquisition of Rockset, one typical response is that open source is the answer. The reasoning is that open source will always remain available, and you won’t have to deal with surprise acquisitions. Open source is often a great solution, and organizations like the Apache Software Foundation are building powerful tools like Flink, Pinot, and Druid that can support real-time analytics use cases.
However, in the short term, putting together an open source solution that fits your use case has some of the same challenges as DIY. Even a robust open source community that is happy to answer your questions differs from a customer success team that’s deeply motivated to integrate the solution for you and win your business. If it were easy to replicate what platforms like Rockset offered, it wouldn’t have existed in the first place. Customers would use RocksDB instead or some other open source solution. There is a cost with closed source, but the value is added. Specifically, the complex problems around scalability, reliability, and efficiency are already solved for you.
Beware of rushing into building something. Yes, open source is often a good answer, but be realistic about how much time you’ll need to set up a solution, how many resources it will take, and how much of a diversion it would be from your core business goals to go that route.
6. One Size Doesn’t Necessarily Fit All
Finding another solution that’s a close replacement to what you already have may be tempting, especially if it helps reduce friction for your teams. But this goes back to knowing when to hold and when to fold. You are going to lose some of the hands. Businesses that are forced to reconsider their existing solution already are facing a challenge and must spend time, energy, and resources finding a replacement.
You shouldn’t just evaluate one solution; you may even find that you want to use more than one solution. For example, in the short term, that could mean using a real-time analytics platform that provides powerful real-time analytics functionality and then sending some of the data from that platform to associated tables in another solution for machine learning applications. It could mean combining several closed source solutions, open source, and even some DIY in the long term.
7. Protect Your Business — and Your Talent
Migrations don’t just threaten your business — they put tremendous pressure on your teams. To navigate this terrain, you need all the talent you can get — from your teams, leaders, and other businesses that have built solutions designed to fit your use case. Fortunately, there are plenty of other solutions out there, and there’s a good chance that there’s one (or even multiple solutions) that can unlock new business use cases for you.
Next Steps
Considering a new solution for your log data? Hydrolix can help. Sign up for a trial or demo.
