Uh oh!
There was an error while loading. Please reload this page.
[#870] Stop replaying a rolled back read in the PersistIt backend - #871
Merged
vharseko merged 1 commit intoAug 19, 2026
Merged
Conversation
…ersistIt backend PDBStorage.read() replayed a rolled back read until it succeeded, which is what Storage.read asks for, on a premise the SPI never states. WriteOperation documents that an implementation must be idempotent because it might be retried; ReadOperation says nothing of the kind, and four read operations of this server are not idempotent. ExportJob has already written entries 1..N-1 to an LDIF stream whose writer is opened once, so a replay appends instead of truncating; VerifyJob accumulates its counters in instance fields that no attempt resets, so a replay trips ERR_VERIFY_WRONG_ENTRY_COUNT on a healthy backend; and the two reads of BackendStat print records and append to a map owned by their caller. The read now runs once and propagates a rollback to the caller. RollbackException extends RuntimeException, so it lands in the existing catch, which rolls back and rethrows: the transaction is still active there, end() runs in the finally, and a second rollback short-circuits on the pending flag. This changes no behaviour today. A read-only PersistIt transaction has no path to a RollbackException: the FETCH branch of Exchange$MvvVisitor.sawVersion contains none, checkPendingRollback is called from begin, commit, setStep and the write methods rather than from fetch or traverse, reads take their exchange with create == false so TimelyResource.addVersion is not entered, and begin() sits outside the try. Only commit() can raise one, and only when a rollback is already pending, which none but a write marks. The replay is removed because it is wrong whenever it does happen - JDBCStorage.read() had to remove it for real, where SQL Server deadlocks reach reads. Storage.read and ReadOperation are corrected to say that a read is not replayed and its implementation need not be idempotent, which is what the callers always assumed and what the JE and Cassandra backends already did.
maximthomas
approved these changes
Aug 19, 2026
Uh oh!
There was an error while loading. Please reload this page.
vharseko added a commit
to vharseko/OpenDJ
that referenced
this pull request
Aug 20, 2026
master added the table stamping of OpenIdentityPlatform#866, which closes the comment connection of a transaction in a finally block, while this branch wrapped the same transaction in a retry loop. The transaction object is now bound per attempt and its stamp session closed with it, so a replay stamps on a session of its own. Both sides also grew a helper returning the driver name behind a connection - driverNameOf() on master, getDriverName() here. Collapsed into master's name, keeping the body of this branch, which tolerates a connection that did not come from the pool. The read() javadoc no longer says that Storage#read asks for a replay: OpenIdentityPlatform#871 turned that contract around on master, and it now forbids one, which is what this implementation already did.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#870.
Problem
PDBStorage.read()replays a rolled back read for as long as it takes:It implements what
Storage.readasks for — "in case of a read operation rollback, implementations must ensure the read operation is retried until it succeeds" — on a premise the SPI never states.WriteOperationdocuments that an implementation "must be idempotent since operation might be retried";ReadOperationsays nothing of the kind, and four of them are not idempotent:ExportJob.java:114entry.toLDIF()(:227) has already written entries 1..N-1, andLDIFExportConfig.getWriter()(:184-206) opens the stream once, soOVERWRITEtruncates on the first attempt only: the replay appends. An LDIF with duplicates, reported as a successful export.VerifyJob.java:130keyCount(:75) andattrIndexList(:103) are instance fields no attempt resets, so the count doubles andERR_VERIFY_WRONG_ENTRY_COUNT(:394-397) fires on a healthy backend — whichverify-index --countErrorsreturns as its exit code.BackendStat.java:1278outinside the read (:1337), so the replay prints what it already printed.BackendStat.java:1103undefinedKeys, a map owned by the caller; its own counters are locals and would reset, the key list would not.Nothing keeps an export off a live backend either:
ExportTask.java:363takes no more thanacquireSharedLock, which the running server itself holds.Why it is latent rather than live
Checked against
org.openidentityplatform.commons.persistit:core:2.1.5, which this build resolves: a read-only transaction has no path to aRollbackException.Exchange$MvvVisitor.sawVersionis the only site on the MVCC path that creates one, and itslookupswitchonUsagesendsFETCHto a branch that contains none; theTransactionIndex.wwDependencycheck followed byTransaction.rollback()and the throw sits in theSTOREbranch. The index-to-constant mapping is the syntheticExchange$1.$SwitchMap.Transaction.checkPendingRollback()is called frombegin,commit,setStep,store,remove,removeTree,runandbeginCheckpoint— not fromfetchortraverse.TimelyResource.getVersionthrows only by way ofaddVersion, i.e. when a version has to be created; reads take their exchange withcreate == false(PDBStorage.java:371, 383).txn.begin()sits outside thetry, so even an inherited pending rollback would propagate rather than be replayed.That leaves
commit(), which throws only when a rollback is already pending, and onlyTransaction.rollback()marks one — reachable from write paths alone. So this PR changes no behaviour today. It removes a replay that is wrong whenever it does happen, and that the JDBC backend had to remove for real in #867, where SQL Server deadlocks reach reads and made it reachable.Fix
PDBStorage.read()runs the operation once and propagates a rollback to the caller.RollbackExceptionextendsRuntimeException, so it now lands in the existingcatch (final Exception e), which rolls back and rethrows; callingrollback()there after acommit()that threw is safe — the transaction is still active,end()runs in thefinally, and a second rollback short-circuits on the pending flag rather than aborting twice.This aligns the four backends:
JEStorage.read()(:855) andCASStorage.read()(:189) never replayed, andJDBCStorage.read()stopped in #867. The contract inStorage.readis corrected to say so, andReadOperationnow states that an implementation need not be idempotent — which is what its callers have always assumed.Deliberately not done: making the four callers replay-safe. It is the larger job for a trigger that does not exist, and the export cannot un-write the bytes it already handed to the LDIF stream.
Verification
No behavioural change to exercise — the removed branch is unreachable — so this is regression coverage only:
PDBTestCasePDBStorageTestTestImportAndExportTestRebuildTaskTestImportAndExportis the suite that drivesExportJobover the changedread().