Skip to content

feat(mssql): support core query constructs and sqlc macros - #4687

Open
EliSauder wants to merge 43 commits into
sqlc-dev:mainfrom
EliSauder:mssql/core-queries
Open

EliSauder wants to merge 43 commits into
sqlc-dev:mainfrom
EliSauder:mssql/core-queries

Conversation

@EliSauder

@EliSauder EliSauder commented Nov 13, 2026 •

Copy link
Copy Markdown

Summary

Expand SQL Server query support in the analysis core and preserve the parameters and result shapes needed by generated code.

  • Support CROSS/OUTER APPLY, recursive CTEs, VALUES-derived tables, OPENJSON/STRING_SPLIT/GENERATE_SERIES, PIVOT/UNPIVOT, and schema-defined scalar and table-valued functions.
  • Handle OUTPUT INSERTED/DELETED, WITH on INSERT, FOR JSON/XML results, correlated set-operation branches, and dialect-scoped recursive-CTE nullability.
  • Correct outer-join nullability, TOP/OFFSET parameter types, conversion and literal types, and parameters inside FOR SYSTEM_TIME, TABLESAMPLE, CONTAINS, and FREETEXT.
  • Enable sqlc.arg/narg/slice/embed for SQL Server, preserving native parameter spelling and case-insensitive identity. Handle bracketed identifiers, repeated slice occurrences, quoted star qualifiers, and safe embed grouping.

The shared core changes address: scope, parameter inference, AST traversal, and code generation. SQL Server-specific behavior is selected through its dialect settings and converter rather than imposed on other engines. Macro documentation and the preprocessor's agent guide are updated.

Scope and compatibility

Embeds are grouped into a table model only when the expanded columns and their nullability match that model; otherwise the result remains flat. Native SQL Server variables and bracketed identifiers are not rewritten as sqlc syntax.

This is not complete T-SQL coverage. In particular, broader unsupported-statement diagnostics, additional DDL, type-precedence rules, and Go driver-type changes are outside this PR.

Related independent drafts: #4688 (domain metadata) and #4689 (expression typing). Adjacent converter, dialect, and compiler edits may need conflict resolution when these are integrated; none requires another to land first.

Tests and validation

Regression coverage uses schema/query fixtures and committed CLI output under internal/endtoend/testdata, plus the existing preprocessor golden harness. New AST nodes are included in both walking and rewriting.

Validation completed during preparation on this branch's unchanged code tree:

  • Complete TestReplay/base/ corpus.
  • Touched-package tests and vet, including preprocessor and compiler coverage.
  • Generated-Go golden validation.
  • SQL Server 2025 checks during development using goldeneye and SQL Server's describe APIs.

The opt-in core context has pre-existing differences from the default analyzer; this PR does not claim the entire experimental-core corpus is green. This draft submission did not rerun the full live-database/examples suite. Upstream CI has not yet been evaluated.

Fork review and detailed history: EliSauder#3.

AI assistance

Prepared with GitHub Copilot assistance. Opened as a draft for author and maintainer review.

2026-11-13 - Draft reviewed and updated by @EliSauder

EliSauder and others added 30 commits November 12, 2026 18:47
The core analyzer bound every relation in a join with the nullability its
columns were declared with, so LEFT, RIGHT and FULL joins reported the
optional side as NOT NULL. A new dialect setting, outer_join_nullable,
makes those columns nullable. SQL Server turns it on; ClickHouse, which
fills unmatched rows with defaults, keeps the old behavior.

Co-authored-by: Copilot <[email protected]>
APPLY reached the analyzer as an unconverted node and failed with
"unsupported FROM item". Convert it to the join it is: CROSS APPLY to an
inner join and OUTER APPLY to a left join. The analyzer already binds
each FROM item against the ones before it, so the applied subquery's
references to the left side resolve.

Co-authored-by: Copilot <[email protected]>
A placeholder counting rows fell back to the core's default int. SQL
Server describes the parameter of TOP (@n), OFFSET @n ROWS and FETCH
NEXT @n ROWS as bigint, so the dialect now says so.

Co-authored-by: Copilot <[email protected]>
A CTE that read itself failed: the analyzer bound a CTE only once its
whole query was analyzed, so the self-reference named an unknown
relation. When a WITH clause is recursive, the anchor member is now bound
first and the full query analyzed against it. SQL Server describes every
column of a recursive CTE as nullable and read from no table, and so
does sqlc. T-SQL has no RECURSIVE keyword, so the converter marks every
WITH clause recursive; only a CTE that names itself is treated as one.

Co-authored-by: Copilot <[email protected]>
The table-valued built-ins reached the analyzer unconverted and failed
with "unsupported FROM item". The converter now reports each as a range
function with a column definition list: OPENJSON's key, value and type,
or the columns its WITH clause declares; STRING_SPLIT's value, plus its
ordinal when enable_ordinal is passed; GENERATE_SERIES's value.

The core analyzer reads a range function's column definition list,
where a definition with no type is the function's own result, and a
seeded "$n" result over an untyped placeholder takes the parameter's
declared type rather than leaking the pseudo-type.

Co-authored-by: Copilot <[email protected]>
SQL Server had no preprocessor dialect, so sqlc macros reached the T-SQL
parser as function calls and produced no parameter. Add one that rewrites
them to native @name placeholders, numbers bare @name alongside them and
folds name case as SQL Server does, leaving @@ system functions alone.

Named-arg queries cannot expand a slice into positional ?s, so expand it
into the @p<ordinal> names go-mssqldb gives unnamed arguments and pass the
other parameters through sql.Named.

Co-authored-by: Copilot <[email protected]>
The core analyzer path expanded a rewritten sqlc.embed(t) star into the
table's columns but never folded them back into one embedded column, so
SQL Server queries got flat rows instead of the table's struct. Fold each
embed star's columns into a column naming its source table, which also
resolves an embed of an alias to the table behind it.

Co-authored-by: Copilot <[email protected]>
A subquery's analyzer took the scope of the query it is nested in as its
outer scope. A set operation binds no scope of its own, so the branches
of a UNION nested in another query got none, and a correlated reference
in them, as in CROSS APPLY (SELECT ... WHERE x = a.id UNION ALL ...),
failed as an unknown column. They now see the scope the set operation
itself sees.

Co-authored-by: Copilot <[email protected]>
(VALUES ...) AS v(a, b) in FROM reached the analyzer unconverted. It is
now the subquery it stands for, each row a SELECT combined with UNION
ALL, and its columns are named by the alias list. A VALUES table can be
joined, applied, and refer to the relations before it.

Co-authored-by: Copilot <[email protected]>
The converter stripped the INSERTED and DELETED qualifiers from an
OUTPUT clause so the columns resolved against the target table. With a
FROM clause joining other tables, the bare names were ambiguous, so
DELETE b OUTPUT DELETED.id FROM books b JOIN authors a ... failed. And a
star lost its qualifier, so OUTPUT INSERTED.* was rewritten into broken
SQL: the expansion replaced only the star's first character and named
columns without the prefix SQL Server requires.

A dialect may now name, in returning_aliases, the pseudo-relations its
RETURNING list qualifies the target's columns with. The analyzer reads
them as the target while it types that list, and the converter keeps
the qualifiers, so INSERTED.* expands to inserted.id, inserted.name, ...

Co-authored-by: Copilot <[email protected]>
TRY_CAST and TRY_CONVERT yield NULL for a value they cannot convert, but
were converted as plain casts, which are NOT NULL over a NOT NULL
operand. Their target type is now marked nullable, as SQL Server
describes it. A placeholder cast this way is still typed NOT NULL by the
cast, as with CAST.

Co-authored-by: Copilot <[email protected]>
PARSE and TRY_PARSE were left unconverted, so their result was untyped
and a placeholder in them was lost. They are now the conversions they
are: PARSE as CAST, TRY_PARSE as the nullable TRY_CAST. What they parse
is a string, so a placeholder there is the nvarchar(4000) SQL Server
takes it as, and a culture placeholder is nvarchar.

Co-authored-by: Copilot <[email protected]>
@@rowcount, @@IDENTITY, @@TRANCOUNT and their relatives were left
unconverted, so a column reading one was untyped. Each is now the call
of a function seeded under its name, with the type and nullability SQL
Server describes it with.

Co-authored-by: Copilot <[email protected]>
Both reached the analyzer as an unconverted node. Their columns are the
source's columns less the rotated ones, plus the ones the rotation makes,
which only the analyzer knows, so the converter passes them on in a new
RangePivot node that the analyzer binds against its source.

A star over a PIVOT expands to columns named after its values, as 2023,
so the expansion quotes names that are no regular identifier.

Co-authored-by: Copilot <[email protected]>
A star expanded into a column named by a reserved word, or by something
that is not an identifier at all, such as PIVOT's [2023], was quoted
with double quotes, which SQL Server reads as an identifier only while
QUOTED_IDENTIFIER is on. SQL Server's own brackets quote it whatever the
setting, with a closing bracket doubled.

Co-authored-by: Copilot <[email protected]>
A query ending in FOR JSON or FOR XML returns one document column, but
was reported as returning its select list. It is now that column:
nvarchar(max) for JSON, ntext for XML, and xml with TYPE, nullable and
named as SQL Server names it; as a subquery, FOR XML without TYPE is
nvarchar(max). The query itself is kept as a subquery of the column, so
its placeholders and stars are still analyzed.

A query with no FROM, like that column's, binds no relations, so what
is nested in it now sees the enclosing query, as a set operation's
branches already did, and a correlated FOR JSON subquery resolves.

Co-authored-by: Copilot <[email protected]>
A query that uses one sqlc.slice() twice binds a single parameter, but
the generated code for a query bound by name replaced only the first
/*SLICE:x*/@x marker, leaving the others unbound. Every marker is now
replaced with the same placeholders, which the driver binds by position
however many times they appear.

Addresses review comment r4557024903 on #1.

Co-authored-by: Copilot <[email protected]>
Star expansion measures the reference it replaces by the quoting the
dialect writes, which for SQL Server became brackets. A qualifier the
user wrote in double quotes, which SQL Server also reads as an
identifier, no longer matched, so "o".* was replaced only in part and
left broken SQL. A double-quoted qualifier is now measured as written.

Addresses review comment r4557024959 on #1.

Co-authored-by: Copilot <[email protected]>
sqlc.embed() on the core path folded a star into the table its first
column came from, whatever else the star expanded to. The struct scans
every column of the table in order, so an embed of a subquery that
selects some of its columns, or reorders them, scanned the wrong number
of values or into the wrong fields. A star is now folded only when it is
exactly the table's columns in order, and is otherwise left flat.

Addresses review comment r4557025012 on #1.

Co-authored-by: Copilot <[email protected]>
The analyzers of nested queries share the map of CTEs they inherit, so
a WITH clause in a subquery added its names to the enclosing query's
map, where a CTE could hide a table of the same name: with an inner
WITH authors AS (...), a later SELECT name FROM authors in the outer
query read the CTE and failed. A WITH clause now binds its names in its
own copy of the map.

Addresses review comment r4557025055 on #1.

Co-authored-by: Copilot <[email protected]>
The preprocessor skips the quoted regions of a query, but did not know
SQL Server's bracketed identifiers, so [sqlc.arg(x)] was rewritten to
[@x], an @name inside brackets was taken for a parameter, and a
semicolon inside them split the statement. A dialect may now say that
[...] quotes an identifier, with ]] standing for a bracket in it, and
SQL Server does.

Addresses review comment r4557025105 on #1.

Co-authored-by: Copilot <[email protected]>
SQL Server binds a parameter by the name sqlc writes into the query, so
sqlc.arg('user-id') became @user-ID, which the server reads as @user
minus id, and a name with a space did not parse; the generated Go did
not compile either way. For a dialect that binds @name, a name passed to
sqlc.arg, sqlc.narg or sqlc.slice must now be one an @name can spell,
and is otherwise reported at the call.

Addresses review comment r4557025173 on #1.

Co-authored-by: Copilot <[email protected]>
A table read FOR SYSTEM_TIME AS OF @t, BETWEEN, FROM ... TO or
CONTAINED IN converted to the plain table, dropping the placeholders
the clause holds, so the generated code did not pass them. The table
reference has nowhere to hold an expression, so the times are now
analyzed with the query's WHERE clause, as the datetime2 SQL Server
takes them as.

Co-authored-by: Copilot <[email protected]>
The full-text predicates were left unconverted, which dropped a
placeholder for the search condition from the query's parameters, so
the generated code did not pass it. They are now calls of the functions
seeded under their names, whose search condition is the nvarchar(4000)
SQL Server takes.

Co-authored-by: Copilot <[email protected]>
The converter dropped the WITH clause of an INSERT, and the core
analyzer bound CTEs for SELECT, UPDATE and DELETE but not INSERT, so an
INSERT ... SELECT reading a CTE failed with an unknown relation. Both
now carry it.

Co-authored-by: Copilot <[email protected]>
CREATE FUNCTION was ignored, so a query calling the function failed. A
scalar function is now recorded with its parameters and result, which
SQL Server reports nullable whatever the arguments, so the converter
returns Nullable(T) and the schema records a T that may be NULL.

Co-authored-by: Copilot <[email protected]>
A table-valued function called in FROM was not converted, so the query
failed. A function in FROM is now called, and a table-valued function
returns rows of table(col type, ...): the columns a multi-statement
function declares, or those an inline function's query selects, which
the schema works out as it does a view's. A function returning such rows
binds as a relation of those columns.

Co-authored-by: Copilot <[email protected]>
A placeholder in VALUES took the type of the column it is inserted into,
but one selected by INSERT ... SELECT @A went untyped. A placeholder
selected as it is now takes its column's type too.

Co-authored-by: Copilot <[email protected]>
A Unicode string literal was typed as a varchar, as any string literal
is. N'...' is now an nvarchar as long as the string, as SQL Server
types it.

Co-authored-by: Copilot <[email protected]>
A type the catalog holds no row for, as decimal(5,1) cast to before any
column has it, is looked up by its family. Looking the family up by its
bare name applied the dialect's default arguments again, as SQL Server's
decimal is decimal(18,0), which held no row either, so the expression
went untyped and its operators resolved against nothing. The family is
now found by name when its bare name is no row.

Co-authored-by: Copilot <[email protected]>
EliSauder and others added 13 commits November 12, 2026 18:50
A literal such as 1.5 was typed as a float, and one too large for an
int, as 2147483648, as one too. SQL Server types each as the decimal
holding its digits, 1.5 as decimal(2,1) and 2147483648 as
decimal(10,0), and so now does sqlc. 1e5 is still a float.

Co-authored-by: Copilot <[email protected]>
TABLESAMPLE (@n PERCENT) and REPEATABLE (@seed) dropped their
placeholders. They are now analyzed with the query's WHERE clause, as
FOR SYSTEM_TIME's are: a percentage as a float, a count of rows and a
seed as a bigint.

Co-authored-by: Copilot <[email protected]>
…star

A placeholder selected by INSERT ... SELECT took the type of the column
at its position, but a star before it selects any number of columns, so
INSERT INTO t (id, name, at, n) SELECT t2.*, @n typed @n as name. Items
after a star are no longer paired.

Co-authored-by: Copilot <[email protected]>
sqlc.embed names a table, not a parameter, so sqlc.embed("order
details") was rejected as an invalid @name. Only sqlc.arg, narg and
slice are checked now.

Co-authored-by: Copilot <[email protected]>
SQL Server's names go on through #, $ and @ after the first character,
but the preprocessor stopped at them, so @A and @A#b were both read as
@A and given one number, and generated code bound only one of them.

Co-authored-by: Copilot <[email protected]>
A star qualified by a name that needs quoting was measured from the name
as sqlc quotes it for output, so "a""b".* read as [a"b] measured
short and left the star in the query, and "x]y".* measured long. Each
quoted form is now built from the name itself, doubling its quotes.

Co-authored-by: Copilot <[email protected]>
…e table's

sqlc.embed(b) on the optional side of an outer join folded b's columns
into the Book model, whose fields cannot hold NULL, so scanning a row
with no match failed. A table is now embedded only when each column is
as nullable as the table's, and otherwise its columns stay flat.

Co-authored-by: Copilot <[email protected]>
@EliSauder

Copy link
Copy Markdown
Author

I have been using sqlc with mssql as a trial and ran in to feature gaps so I figured I would spend the time and tokens to suggest some changes.

If this should be broken out into even more PRs I can go back and split it out. I think for this particular PR, potentially splitting the sqlc into its own PR could be argued for.

@EliSauder
EliSauder marked this pull request as ready for review November 13, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant