RSS

Does Hydrolix support batch processing?

Yes, Hydrolix fully supports batch processing. This collection feature allows you to efficiently load data from a storage bucket into a target table, ensuring you can work with your data at scale. Hydrolix offers two mechanisms for batch ingestion: the Batch Job API, which handles one-off tasks for loading one or more files based on…

Jay Maloney

Published:

Oct 30, 2023

1 minute read

Yes, Hydrolix fully supports batch processing. This collection feature allows you to efficiently load data from a storage bucket into a target table, ensuring you can work with your data at scale.

Hydrolix offers two mechanisms for batch ingestion: the Batch Job API, which handles one-off tasks for loading one or more files based on job configurations, and Batch Auto-Ingest, which continuously ingests new files arriving in a storage bucket.

Supported data formats include CSV and JSON. Please note that Hydrolix requires read permissions to access external storage buckets for batch ingestion.

Ready to Start?

Cut data retention costs by 75%

Give Hydrolix a try or get in touch with us to learn more

Yes, Hydrolix fully supports batch processing. This collection feature allows you to efficiently load data from a storage bucket into a target table, ensuring you can work with your data at scale.

Hydrolix offers two mechanisms for batch ingestion: the Batch Job API, which handles one-off tasks for loading one or more files based on job configurations, and Batch Auto-Ingest, which continuously ingests new files arriving in a storage bucket.

Supported data formats include CSV and JSON. Please note that Hydrolix requires read permissions to access external storage buckets for batch ingestion.