Skip to content

Support from-first SQL syntax - #17295

Merged
Jefffrey merged 3 commits into
apache:mainfrom
simonvandel:support-from-first-syntax
Aug 24, 2025
Merged

Support from-first SQL syntax#17295
Jefffrey merged 3 commits into
apache:mainfrom
simonvandel:support-from-first-syntax

Conversation

@simonvandel

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

I noticed in #17278 that from without an explicit projection did not work.
This should fix it.

The from-first syntax is also available in duckdb. See https://duckdb.org/docs/stable/sql/query_syntax/from.html#examples

What changes are included in this PR?

Are these changes tested?

Yes, added SLT test

Are there any user-facing changes?

FROM-first syntax without projection is now supported.

I noticed in apache#17278 that `from`
without an explicit projection did not work.
This should fix it
@github-actionsgithub-actionsBot added sql SQL Planner sqllogictest SQL Logic Tests (.slt) labels Aug 22, 2025

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤯 -- thank you @simonvandel

I also verified duckdb does support this

D create table t(x int);
D insert into t values (1);
D from t;
┌───────┐
│ x │
│ int32 │
├───────┤
│ 1 │
└───────┘

@Jefffrey or @jonahgao -- any concern about supporting this syntax?

@alambalamb changed the title Support from-first syntaxSupport from-first SQL syntaxAug 23, 2025

@JefffreyJefffrey left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used this in DuckDB before and agree it's pretty slick & handy;

Maybe need to update the SQL reference docs with this new behaviour?

Comment threaddatafusion/sqllogictest/test_files/from-first.slt
@github-actionsgithub-actionsBot added the documentation Improvements or additions to documentation label Aug 23, 2025
@simonvandel

Copy link
Copy Markdown
ContributorAuthor

Maybe need to update the SQL reference docs with this new behaviour?

* https://datafusion.apache.org/user-guide/sql/select.html

I added some documentation in 9f1b436

@JefffreyJefffrey left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@Jefffrey
Jefffrey merged commit 85f879d into apache:mainAug 24, 2025
28 checks passed
adriangb added a commit to pydantic/datafusion that referenced this pull request Sep 11, 2025
When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR apache#17295 which added FROM-first syntax support.
The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection
Also updates the test expectation to correctly show the empty Projection
node in the query plan.
Fixesapache#17513
@alamb

Copy link
Copy Markdown
Contributor

@adriangb

Copy link
Copy Markdown
Contributor

@alamb we discussed a bit in #17520 (comment), I'm not sure that this PR introduced a bug. If I understand correctly select from t1 should be equivalent to select * from t1 -> then there is no bug. Now if select from t1 is meant to be select every row but no columns then there is a bug but it has nothing to do with the where clause or projection pushdown.

FWIW:

DataFusion CLI v50.0.0
> create table t1 (a int, b int);
0 row(s) fetched. Elapsed 0.004 seconds.
> insert into t1 values (1, 2);
+-------+
| count |
+-------+
| 1 |
+-------+
1 row(s) fetched. Elapsed 0.010 seconds.
> select * from t1;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
1 row(s) fetched. Elapsed 0.003 seconds.
> select from t1;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
1 row(s) fetched. Elapsed 0.004 seconds.
> from t1;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
1 row(s) fetched. Elapsed 0.001 seconds.

Postgres:

postgres=# create table t1 (a int, b int);
CREATE TABLE
postgres=# insert into t1 values (1), (2);
INSERT 0 2
postgres=# select * from t1;
a | b ---+---
1 | 2 | (2 rows)
postgres=# select from t1;
--
(2 rows)
postgres=# from t1;
ERROR: syntax error at or near "from"
LINE 1: from t1;

DuckDB:

DuckDB v1.3.2 (Ossivalis) 0b83e5d2f6
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D create table t1 AS (select1as a, 2as b);
D insert into t1 values (1, 2);
D selectfrom t1;
Parser Error:
SELECT clause without selection list
D from t1;
┌───────┬───────┐
│ a │ b │
│ int32 │ int32 │
├───────┼───────┤
│ 12 │
│ 12 │
└───────┴───────┘

@adriangb

adriangb commented Sep 11, 2025

Copy link
Copy Markdown
Contributor

So maybe the answer is that we should make select from t1; equivalent to "select no columns from t1" while keeping from t1; equivalent to select * from t1;?

@alamb

Copy link
Copy Markdown
Contributor

So maybe the answer is that we should make select from t1; equivalent to "select no columns from t1" while keeping from t1; equivalent to select * from t1;?

I think we should be consistent with other databases

This proposal seems reasonable to me

SHould we also make from t (without anything else) an error like it is in duckdb?

@adriangb

Copy link
Copy Markdown
Contributor

SHould we also make from t (without anything else) an error like it is in duckdb?

DuckDB does support FROM t;, it does not support SELECT FROM t;

alamb pushed a commit that referenced this pull request Sep 12, 2025
* Add failing test
* Fix regression in SELECT FROM syntax with WHERE clause
When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR #17295 which added FROM-first syntax support.
The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection
Also updates the test expectation to correctly show the empty Projection
node in the query plan.
Fixes#17513
* Revert
* Fix regression: SELECT FROM syntax should return empty projection
Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.
Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.
Fixes#17513
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* add a better regression test
* remove comment
* fmt
* Update datafusion/sqllogictest/test_files/projection.slt
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
* Update datafusion/core/tests/sql/select.rs
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
* revert docs
* fmt
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
alamb pushed a commit to alamb/datafusion that referenced this pull request Sep 12, 2025
* Add failing test
* Fix regression in SELECT FROM syntax with WHERE clause
When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR apache#17295 which added FROM-first syntax support.
The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection
Also updates the test expectation to correctly show the empty Projection
node in the query plan.
Fixesapache#17513
* Revert
* Fix regression: SELECT FROM syntax should return empty projection
Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.
Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.
Fixesapache#17513
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* add a better regression test
* remove comment
* fmt
* Update datafusion/sqllogictest/test_files/projection.slt
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
* Update datafusion/core/tests/sql/select.rs
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
* revert docs
* fmt
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
xudong963 pushed a commit that referenced this pull request Sep 13, 2025
* Add failing test
* Fix regression in SELECT FROM syntax with WHERE clause
When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR #17295 which added FROM-first syntax support.
The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection
Also updates the test expectation to correctly show the empty Projection
node in the query plan.
Fixes#17513
* Revert
* Fix regression: SELECT FROM syntax should return empty projection
Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.
Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.
Fixes#17513
🤖 Generated with [Claude Code](https://claude.ai/code)
* add a better regression test
* remove comment
* fmt
* Update datafusion/sqllogictest/test_files/projection.slt
* Update datafusion/core/tests/sql/select.rs
* revert docs
* fmt
---------
Co-authored-by: Adrian Garcia Badaracco <1755071+adriangb@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
samueleresca pushed a commit to samueleresca/datafusion that referenced this pull request Sep 15, 2025
* Add failing test
* Fix regression in SELECT FROM syntax with WHERE clause
When using 'SELECT FROM table WHERE condition', the query should create
an empty projection (no columns) while still filtering rows. This was
broken by PR apache#17295 which added FROM-first syntax support.
The issue was that both 'FROM table' and 'SELECT FROM table' resulted
in empty projection lists, making them indistinguishable. The fix checks
for the presence of a WHERE clause to differentiate:
- 'FROM table' (no WHERE) -> add wildcard projection (all columns)
- 'SELECT FROM table WHERE ...' -> keep empty projection
Also updates the test expectation to correctly show the empty Projection
node in the query plan.
Fixesapache#17513
* Revert
* Fix regression: SELECT FROM syntax should return empty projection
Removes automatic wildcard projection for empty projections, fixing
the regression where `SELECT FROM table` incorrectly returned all
columns instead of empty projection.
Note: This temporarily breaks FROM-first syntax. A proper fix would
require distinguishing between `FROM table` and `SELECT FROM table`
at the parser level.
Fixesapache#17513
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* add a better regression test
* remove comment
* fmt
* Update datafusion/sqllogictest/test_files/projection.slt
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
* Update datafusion/core/tests/sql/select.rs
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
* revert docs
* fmt
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Oleks V <comphead@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsqlSQL PlannersqllogictestSQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@simonvandel@alamb@adriangb@Jefffrey