On top of being a great code editor, Visual Studio Code (VSCode) has a great ecosystem with various plugins allowing users to automate many different tasks.
Hydrolix being a fully restful API, allows you to manage, configure ingest data.
Here we’ll see how we can use VSCode with Hydrolix to configure tables, transformation, and ingest jobs. After configuring our Hydrolix infrastructure from VSCode, we’ll also view data using SQL query using another plugin from VSCode.
Table of Contents
Installing the plugin
For this example we’ll use 2 different very popular plugins for VSCode:
- REST Client allows us to send HTTP requests and view the response in Visual Studio Code directly.
- ClickHouse driver for SQLTools allows us to do SQL query via the clickhouse connector.
There are several ways to install those plugins. The easiest way is to look into VSCode extensions:
After searching for each extension and installing those we can start working with Hydrolix.
Making your first API call
In this example I’ve setup several variables:
- @base_url: URL to use with Hydrolix API schema
- @username: email address you used when you created your cluster
- @password: password defined when you first login into your Hydrolix portal
The @variable creates a local variable within the file.
The ### and the # @name login allows us to delimit and name each API call within our file.
We’ll use this naming convention across the file.
Here we make a POST request into/login with the header:
“Content-Type: application/json”
And the POST body is a json object with the username and password.
At the bottom of the file we specify two variables which are getting populated automatically from the response:
- @acces_token: is the JWT contains in our response body
- @org_id: is the organization ID required for the next API call
Those two variables are critical as they are required for every other call to our REST API.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
### Global variable to replace with your own needs @base_url = https://david.hydrolix.io/config/v1 @username = "david@email.com" @password = "XXXXXXXXXX" ### Login authentication get access token and UUID Org variable # @name login POST {{base_url}}/login Content-Type: application/json { "username": {{username}}, "password": {{password}} } ### Store, parse the login response body to store the access token and organization id @access_token = {{login.response.body.auth_token.access_token}} @org_id = {{login.response.body.orgs[0].uuid}} |
Some API Examples
Creating a new project:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
### Create a new project called newproject # @name new_project @projectname = newproject POST {{base_url}}/orgs/{{org_id}}/projects/ Authorization: Bearer {{access_token}} Content-Type: application/json { "name": "{{projectname}}", "org": "{{org_id}}" } ### Store, parse project ID from response @projectid = {{new_project.response.body.uuid}} |
Create a new table:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
### Create a new table named tablename in the newproject # @name new_table @tablename = newtable POST {{base_url}}orgs/{{org_id}}/projects/{{projectid}}/tables/ Authorization: Bearer {{access_token}} Content-Type: application/json { "name": "{{tablename}}", "project": "{{projectid}}" } ### Store, parse table ID from response @tableid = {{new_table.response.body.uuid}} |
View table information:
|
1 2 3 4 5 |
### Get Table information # @name get_table GET {{base_url}}orgs/{{org_id}}/projects/{{projectid}}/tables/{{tableid}} Authorization: Bearer {{access_token}} Content-Type: application/json |
Hydrolix provides a full documentation of the different rest API available here:
https://docs.hydrolix.io/ref_home/config_api
We have a GitLab repository where we’ll host example usage of VSCode with our different API.
You can download a full example here which contains every steps from project creation, table creation, and transform to specify the format of the data and finally ingesting rows.
To play with this example save file as .http open it with VSCode, then you can either run each individual request via the Send Request button or run the whole file via ctrl + alt + R
Using SQL Client in VSCode
As Hydrolix is using Clickhouse SQL engine, you can configure your VSCode with the Clickhouse plugin to connect to Hydrolix and execute SQL queries.
To configure your SQL connection click on the database icon in VSCode and add a new connection:
Click on the Clickhouse Driver installed:
Then fill the next form with the information for your cluster:
You can now do SQL query directly from VSCode:
With a single Code Editor you can manage everything – from project creation to ingesting data and viewing it!