RBQL is a technology that provides SQL-like language for data-transformation and data-analysis queries for structured data (e.g. CSV files, log files, Python lists, JS arrays). RBQL evaluates input query using one of the available general-purpose "backend" languages (currently Python or JavaScript). RBQL allows to use non-SQL expressions inside SQL statements, e.g for JavaScript backend:
select a1, a2 * 10 where a1 == "Buy" && a4.indexOf('oil') != -1 order by parseInt(a2), a4 limit 100
Combination of SQL statements and expressions from a general-purpose language makes RBQL very easy to learn and use, you just need a basic understanding of SQL and one of the available backend languages. So maybe you already know RBQL!
The first word "Rainbow" in RBQL reflects its diverse multi-language nature.
RBQL has simple modular design and no third-party dependencies. It is available as standalone CLI apps, text editor plugins and 2 libraries (for Python and JS) which would allow you to easily enhance your own app with SQL capabilities.
RBQL is available for the following platforms:
RBQL – CLI App and library for Python. Also provides `%rbql` "magic" IPython/Jupyter command (usage example)
RBQL supports all main SQL keywords: SELECT, UPDATE, WHERE, ORDER BY, JOIN, DISTINCT, GROUP BY, TOP, LIMIT, ...
... And some cool features that traditional SQL engines do not provide!
RBQL implementation is based on a clever hack, see the explanation below if you are curious.
RBQL core idea is based on dynamic code generation and execution with exec and eval functions.
The main steps that RBQL engine performs when processing a query:
Shallow parsing: split the query into logical expressions such as "SELECT", "WHERE", "ORDER BY", etc.
Embed the expression segments into the main loop template code
Execute the hydrated loop code
Here you can find a very basic working script (only 15 lines of Python code) which implements this idea: mini_rbql.py
The diagram below gives an overview of the main RBQL components and data flow:
RBQL Web Demo
In this Web Demo only JavaScript backend is available.
SELECT top 10 * WHERE a7.indexOf('Sci-Fi') != -1 && a4 > 2010 order by parseInt(a4), parseInt(a8) desc — Simple query
SELECT a1, a2, a4 WHERE parseInt(a4) > 2014 order by a1 — Actually no need for "parseInt()" here, just a reminder that column vars are strings
select * order by Math.random() — Random shuffle
select COUNT(*), a2 group by a2 — Aggregate query
UPDATE SET a8 = a8 / 60 — UPDATE is just a special type of SELECT
select top 20 NR, * — Enumerate records
select JSON.stringify({name: a1, country: a2}) — Convert CSV to JSON records. How cool is that?
select `mv '${a1}.mp4' '${a4}_${a1.toLowerCase()}.mp4'` — Make a bash script to rename files, this is valid JS syntax
select a1, a100 — RBQL has warnings!
Use the default test table below or "upload" your own. Your data is safe: selected table doesn't actually get uploaded anywhere - all operations are performed locally in the browser. Every query result is a first-class table.
You can use the default demo join table or upload your own.
OR...
Select spreadsheet file to use. Your data is safe: the file won't be sent over network - all operations are performed locally in the browser. Performance of this demo is limited by your browser processing capabilities - do not submit very big files!
1.
2. Choose file separator character:
To refer to the table columns use index names: a1, a2, a3, ... and NR and/or values from the first row e.g. a.Title, a.Country and a["Film type"]