Logica: organizing your data queries, making them universally reusable and fun
source link: https://opensource.googleblog.com/2021/04/logica-organizing-your-data-queries.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Logica: organizing your data queries, making them universally reusable and fun
Monday, April 12, 2021
We present Logica, a novel open source Logic Programming language. A successor to Yedalog (a language developed at Google earlier) it is a Datalog-like logic programming language. Logica code compiles to SQL and runs on Google BigQuery (with experimental support for PostgreSQL and SQLite), but it is much more concise and supports the clean and reusable abstraction mechanisms that SQL lacks. It supports modules and imports, it can be used from an interactive Python notebook and it even makes testing your queries natural and easy. “Data is the new oil”, they say, and SQL is so far the lingua franca for working with data. When SQL (or “Structured English Query Language”, as it was first named) was invented in the 1970s, its authors might not have imagined the popularity that it would reach half a century later. Today, systems ranging from tiny smart watch applications to enterprise IT solutions, read and write their data using SQL. Even the browser that you are using to read this post now might have a working built-in SQL database in it.Despite the widespread adoption, SQL is not flawless. Constructing statements from long chains of English words (which are often capitalized to keep the old-fashioned COBOL spirit of the 70s alive!) can be very verbose—a single query spanning hundreds of lines is a routine occurrence. The main flaw of SQL, however, lies in its very limited support for abstraction.Good programming is about creating small, understandable, reusable pieces of logic that can be tested, given names, and organized into packages which can later be used to construct more useful pieces of logic. SQL resists this workflow. Although you can encapsulate certain repeated computations into views and functions, the syntax and support for these can vary among implementations, the notions of packages and imports are generally nonexistent, and higher-level constructions (e.g. passing a function to a function) are impossible.
This inherent resistance to decomposition of logic into bite-sized pieces is what leads into the contrived, lengthy queries, the copy-pasted chunks of code and, eventually, unmaintainable, unstructured (note the irony) SQL codebases. To make things worse, SQL code is rarely tested, because “testing SQL queries” sounds rather esoteric to most engineers, at best. Because of that, a number of alternative query languages and libraries have been developed. Of those, systems based on logic programming perhaps come the closest to addressing SQL’s limitations.Logic programming languages solve problems of SQL by using syntax of mathematical propositional logic rather than natural English language. The language of formal logic was designed by mathematicians specifically to make expression of complex statements easier and suits this purpose much better than natural language. Logica extends classical Logic programming syntax further, most notably with aggregation, hence the name, which stands for
MagicNumber(x: 2);
MagicNumber(x: 3);
MagicNumber(x: 5);
SELECT 2 AS x
UNION ALL
SELECT 3 AS x
UNION ALL
SELECT 5 AS x;
MagicNumber(x:) :-
x in [2, 3, 5];
MagicComment(comment_text:) :-
`comments`(user_id:, comment_text:),
user_id == 5;
SELECT comment_text FROM comments WHERE user_id = 5;
MagicComment(comment_text:) :-
`comments`(user_id:, comment_text:),
MagicNumber(x: user_id);
import my_project.magic.MagicNumber;
import my_project.magic.MagicComment;
MockComments(user_id: 1, comment_text: "Hello");
MockComments(user_id: 2, comment_text: "Logic");
MockComments(user_id: 3, comment_text: "Programming");
MagicCommentTest := MagicComment(`comments`: MockComments);
There is much more to Logica, so make sure you give it a try—chances are, you will love it! Start with this tutorial to learn Logica. Even if you do not end up using it in your next project, learning a new powerful language may open your mind to new ideas and perspectives on data processing and computing in general.
The simple examples above are only a small sample of how concise Logica code can be over SQL for complex queries. In particular, we did not even touch the topic of aggregations in this article. For all of this see examples section of the Logica open source repository.
We also hope that some of the readers consider contributing to Logica development. That’s what open source is all about!
By Konstantin Tretyakov and Evgeny Skvortsov – Logica Open Source Project
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK