Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
features/sql: read result sets with plain JDBC instead of rxjava3-jdbc by cportele · Pull Request #625 · ldproxy/xtraplatform-spatial · GitHub
Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' features/sql: read result sets with plain JDBC instead of rxjava3-jdbc by cportele · Pull Request #625 · ldproxy/xtraplatform-spatial · GitHub
Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' features/sql: read result sets with plain JDBC instead of rxjava3-jdbc by cportele · Pull Request #625 · ldproxy/xtraplatform-spatial · GitHub
Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' features/sql: read result sets with plain JDBC instead of rxjava3-jdbc by cportele · Pull Request #625 · ldproxy/xtraplatform-spatial · GitHub
Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' features/sql: read result sets with plain JDBC instead of rxjava3-jdbc by cportele · Pull Request #625 · ldproxy/xtraplatform-spatial · GitHub
Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' features/sql: read result sets with plain JDBC instead of rxjava3-jdbc by cportele · Pull Request #625 · ldproxy/xtraplatform-spatial · GitHub
Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); features/sql: read result sets with plain JDBC instead of rxjava3-jdbc by cportele · Pull Request #625 · ldproxy/xtraplatform-spatial · GitHub
Skip to content

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc - #625

Open
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc
Open

features/sql: read result sets with plain JDBC instead of rxjava3-jdbc#625
cportele wants to merge 1 commit into
crud-writes-via-sessionfrom
reads-without-rxjava3-jdbc

Conversation

@cportele

Copy link
Copy Markdown
Contributor

Closes the last variant of the connection leak behind ldproxy/ldproxy#1761 — on the read path this
time — and, with no user left, removes the rxjava3-jdbc dependency. This goes beyond
ldproxy/ldproxy#1645, whose "remove usage of rxjava3-jdbc" concerns the SQL mutations and is
covered by #624.

Problem

Streamed reads — a positive fetch size, as used by single-shot queries — ran in a rxjava3-jdbc
transacted select, which commits and releases the connection only once the rows are exhausted.
An error while reading, or a cancellation (the read-stall timeout from #606, a client that went
away), left the connection leased for good.

Change

All reads lease a connection from the pool and stream the result set with Flowable.using +
Flowable.generate; statement, result set and connection are released on completion, error and
cancellation alike. A read with a fetch size still runs in a transaction so the driver uses a
server-side cursor (ended with a rollback, the cheapest way to close the cursor). Everything
reactive is unchanged: backpressure, execution on the subscribing thread, subscribeOn for
parallel single-shot sub-queries, the read-stall timeout, the SQL result logging.

Differences to the library's Select: errors raised while rows are streamed now reach the consumer
(the library swallowed them, which is what the transacted read was working around), resources are
released before completion is signalled downstream, and query strings are no longer scanned for
?/:name placeholders (ldproxy never binds parameters).

With no user left, the rxjava3-jdbc dependency, the Database in SqlConnectorRx and the unused
SqlDbmsAdapter.getRxType() are removed.

Why the library is not missed

rxjava3-jdbc is a general-purpose reactive JDBC wrapper; ldproxy used a narrow slice of it, and
that slice is exactly what the ~60 lines in SqlClientRx now do directly. Feature by feature:

rxjava3-jdbc capabilityuse in ldproxyafter this PR
Non-blocking connection pool (Pools.nonBlocking())never used — the Database was created with fromBlocking(hikariDataSource), so every read ran synchronously on the subscribing threadunchanged: lease, execute and read run on the subscribing thread; subscribeOn where parallelism is wanted
Parameter binding (?, :name, collection expansion, statement reuse across parameter groups)never used — every statement arrives as a fully rendered string; the library still parsed each one for placeholdersnot needed; no parsing of the SQL text (a latent hazard with a literal ?, e.g. the JSONB ? operator, is gone)
Row mapping (auto-mapping to interfaces/tuples, ResultSetMapper)not used beyond passing our own SqlRowVals.read(ResultSet, options)same mapper called directly
Transactions (transacted(), Tx, dependsOn, returnGeneratedKeys)the write path (removed in PR 1); for reads only to obtain a server-side cursorsetAutoCommit(false) + setFetchSize do that in two lines
Batching, stored procedures (Call), pool health-check queries (DatabaseType)never used (getRxType() had no caller)removed
Query timeoutpassed as 0 (none)none (the read-stall timeout covers the stall case)
Resource lifecycleFlowable.using(prepare, generate over rs.next(), close) — the same structure as the new code — but with the transacted variant's reference counting on top, which releases only on natural completionFlowable.using + Flowable.generate without the counter: release on completion, error and cancel

So the library provided no capability the read path used that plain JDBC under RxJava does not
provide equally; what it added on top was the reference-counted transaction handling that caused
the leak, an SQL placeholder parser we did not need, the swallowed streaming error (#1293), and a
fork (0.1.4-ii.1) to maintain. The reactive contract of the read path — a backpressured
Flowable<SqlRow>, one connection per subscription, cancellation from downstream — is provided by
RxJava itself and is unchanged.

Verification

  • SqlClientRxSpec: streamed and paged reads release statement, result set and connection on
    completion, on cancellation, on a failing next(), on a failing execute and on a failed lease;
    run() and session paths.
  • End to end: ordinary reads (items with counts, paging, bbox, single feature, HTML, extents)
    unchanged; six large reads aborted by a throttled client mid-stream returned their connection
    every time (pool at baseline, HikariCP total equal to the database's backend count); the
    search/result-set and stored-query smoke tests — which include a header-only read of a
    100k-feature stored query, i.e. a cancelled streamed read — and the transaction smoke tests pass.

Streamed reads (a positive fetch size, used by single-shot queries) ran in
a rxjava3-jdbc transacted select, which commits and releases the connection
only once the rows are exhausted. An error or a cancellation - the read
stall timeout, a client that went away - left the connection leased for
good, and enough of them starve the pool.
All reads now lease a connection from the pool and stream the result set
with Flowable.using and Flowable.generate: statement, result set and
connection are released on completion, error and cancellation alike. A
read with a fetch size still runs in a transaction so the driver uses a
server-side cursor; it is ended with a rollback. Errors raised while rows
are read reach the consumer.
rxjava3-jdbc is no longer used: the dependency, the Database in the
connector and the unused SqlDbmsAdapter.getRxType() are removed.
@cportele
cportele requested a review from azahnen as a code ownerAugust 29, 2026 13:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@cportele