HN DigestHacker Newsletter, digested

Issue #807#Data

Things I want in a modern relational query language

TL;DR

A developer who works mostly with MySQL and Db2 argues that SQL's relational ideas are sound. Its implementations are clumsy, and that pushes programmers toward NoSQL. A successor should keep the declarative model. It should add modern syntax, readable query plans, real user-defined types, sum types with pattern matching, and foreign keys that can point at several tables.

The article

  • The author blames much of NoSQL's rise on archaic SQL implementations. The ideas behind SQL are fine. Its PL/I-style syntax is a 1970s IBM choice. Parsers like MySQL's rarely say where an error is. Oracle shows that good error messages are possible.
  • SQL's declarative core sits close to functional programming. Yet most dialects ship thin standard libraries built for 1980s procedural code. Stored procedures and cursors push users toward mutable state. Defaults shape habits, so most real SQL ends up procedural.
  • Query planners stay cryptic unless you are already an optimization expert. That makes it easy to write an expensive query by accident. The author singles out MySQL's EXPLAIN as the worst.
  • Codd's relational model covers domains for user-defined types. In practice, domains are thin sugar over checks and ranges. Postgres supports them, probably thanks to its Ingres and QUEL heritage. Oracle added them only recently.
  • The author's example is IBM i's stack-frame function, a table full of mutually exclusive column groups, nullable fields and string enums. A sketch in F#-like syntax replaces them with a discriminated union of frame types. Queries and functions filter and destructure those types by pattern matching.
  • A foreign key that can reference several tables would let one ObjectPictures table link pictures to Software, Version or Download rows. Today that takes a separate many-to-many table per parent.

Why it matters

Worth opening for the code sketches, which the summary only describes. The IBM i stack-frame schema and the multi-target foreign key are concrete designs. Anyone working on a SQL successor or a typed query layer can argue with them line by line.

The discussion

A busy, opinionated thread that mostly ignored the article's specific proposals. The biggest fight was over whether SQL is actually hard or people just won't learn it. A second front argued whether LLMs lock SQL in or make new languages cheap to try. Many replies were links to alternatives, Datalog above all.

Is SQL hard, or just unfamiliar?

One camp, including a DBA, says SQL's syntax is simple and the complaints come from people who never learned it. Others say syntax is the smallest problem: queries don't compose beyond CTEs, few engineers understand isolation levels and locks, and plan switches cause outages. A side exchange added that developers can't be expected to know safe index builds, and that ORMs default to unsafe DDL. ORMs got credit for parameterizing queries against injection, though one commenter said hand-written parameterized queries are easy.

SQL isn't composable (you can't assign fragments to variables except for CTEs) so you can't easily test out subparts and build them up incrementally without just copy/pasting stuff around.

— @mike_hearn

The difficulties that even experienced programmers have with SQL are far more than just the syntax. And, given what most people need the database for, that difficulty is pretty disproportionate to the complexity of the task.

— @zbentley

LLMs: lock-in or a way out

The top comment predicts SQL becomes like assembly, mostly written by machines, so fewer people will bother designing a successor. Others agree that models trained on decades of SQL will do worse in a new language, citing a Dan Luu post on token counts. The opposing view is that LLMs pick up new syntax from a description and a few examples, which lowers the cost of switching. One commenter's non-technical staff now write sophisticated Grafana queries with an LLM and a schema document.

Now that it matters less how annoying SQL is to programmers, SQL will become more like assembly over time: something mostly computers write because it's complicated for humans to deal with directly.

— @scythmic_waves

LLMs are also very good at writing code for newly invented languages, especially if they can execute it and iterate. I strongly believe the barrier to switch languages is lowered in a post LLM world.

— @reitzensteinm

Why nothing has replaced SQL

The people who know a business's data best are SQL experts, so any replacement asks its natural users to give up a skill. Experts have attacked SQL since a 1983 typewritten critique, Stonebraker called it the COBOL of 2020, and better languages like Datalog have existed all along. Some said SQL won because it was battle-tested, others that it was a crappy partial implementation nobody managed to dislodge. Vendors can't ship SQL-free databases either, because customers expect every knob from their existing setup.

Introducing something else, then, means your most natural user base must migrate away from something they understand how to use well, and that's a hard sell.

— @tmoertel

if you don't support SQL you're at a big competitive disadvantage. because its a weird design with lots of sharp edges that's going to take a lot of your time - customers are going to be unhappy that you don't support the knobs and frills from their existing environment.

— @convolvatron

Change the interface, not the syntax

Several commenters want the database to accept a structured query format, such as a JSON AST or an IR, that any language can compile to. One built Rad around that idea, and others pointed to Substrait, Turso's VDBE bytecode, and Datalog projects such as Mangle, Datalevin and CodeQL. Short of that, people use pipe syntax, PRQL, LINQ or a dataframe library and treat SQL as an IR. One commenter wanted clients to compile their own query plans, and another replied that baked-in plans break when a view or index changes.

Right now we do this with ORMs, but it's a huge pain because the ORMs are like a transpiler with a really free-form and messy language as the target. All the ORMs are just pasting strings together.

— @BrenBarn

So, until the ultimate query language is developed, I'll take SQL with pipes. It's an easy sell and good enough to eliminate 90% of my gripes about SQL.

— @tmoertel

Live queries that stream deltas

Since 2011, one commenter has wanted queries that act as subscriptions, with the database streaming changes instead of the client re-running the query. Replies listed partial versions in Oracle, SQL Server, Snowflake, BigQuery, and Postgres triggers with NOTIFY, plus Materialize and Feldera for incremental queries. Oracle's version came out ahead, though it can't follow something like a COUNT(*). One reply argued that incremental computation is too hard to solve at the language level.

There were a ton of times in my time using SQL where the same query is run over and over, just to get/handle that delta.

— @mikewarot

It's really more like using SQL to select database cells to watch, than propagating changes through arbitrary query plans. For example, it can't handle a SELECT COUNT(*) FROM statement.

— @mike_hearn

Consensus

SQL is not going away soon. Expertise, vendor compatibility and LLM fluency all favor it. So the realistic gains come from layers on top, such as pipe syntax, PRQL or LINQ-style builders, or from a structured interface underneath.

Dissent

A vocal minority says SQL is fine and the complaints show a lack of skill, and that "just syntax" is no reason to switch. A few others say Datalog has been the better answer for decades and is worth adopting now.