HDDS-13415. Support DeleteRange operation as part of the rocksdb batch write by swamirishi · Pull Request #8774 · apache/ozone

swamirishi · 2025-07-10T02:24:42Z

What changes were proposed in this pull request?

Added support for DeleteRange operation in RocksDB batch writes
Key Changes:

New Operation Type:
- Added to the enum alongside existing and operations DELETE_RANGE``Op``DELETE``PUT
- Added new DeleteRangeOperation to perform deleteRange on a rocksdb column family.
Data Structure Improvements:
- Implemented interface for class to support ordering Comparable``Bytes
New API Methods:
- Added method to deleteRange``RDBBatchOperation
- Added to Table interface deleteRangeWithBatch
- Added to batchDeleteRange``RocksDatabase.ColumnFamily
Operation Handling:
- Introduced new interface for better operation abstraction Operation
- Added proper cleanup and resource management for operations
- Implemented ordered operation execution based on operation index
Interface Updates:
- Modified to include new method Table.java``deleteRangeWithBatch
- Updated and to implement the new range deletion functionality TypedTable``RDBTable

The changes improve the RocksDB integration by adding support for batch range deletions, which allows for more efficient deletion of multiple consecutive keys in a single operation. The implementation includes proper resource management and maintains operation ordering within batches.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13415

How was this patch tested?

Working on adding unit tests

…h write Change-Id: I65444c0155b471c31ea1283d563d9a948f5e4005

…impact Change-Id: I3b34e1bb6ca7587243f6eef176c396ca0132dbc3

swamirishi · 2025-07-10T09:13:44Z

I changed the initial patch to use a HashMap instead of using a deleteRange. We only have to skip the final entries at the when we have to commit the batch. This would have more memory overhead as the deleteRange wouldn't be deleting the keys from operation map. Using a TreeMap has a performance overhead when it has to maintain all the values in sorted order. This can have performance impact when batches don't have a deleteRange operation which could be infrequent.

Change-Id: Iab23d0b1316eac9c376fd06226d234b97e908b4f

Change-Id: I74fed5aebb5c0588cbf97f70a0f9acf94edfa87c

SaketaChalamchala · 2025-07-15T18:34:08Z

+ startKey = null;
+ endKey = null;
+ }
+ for (int i = startIndex; i < deleteRangeIndex; i++) {


@swamirishi, Shouldn't we check all operations prior to deleteRangeIndex i.e., [0, deleteRangeIndex) to discard operations that fall within the delete range Index
// If order of ops is, I think put<2, val1> is not discarded even though it falls in delRange<1, 3> put<2, val1> // Should be discarded delRange<1, 2> delRange<1, 3>

@SaketaChalamchala
The outerLoop will have deleteRangeIndices [1, 2, 3]
The inner Loop will perform put operation<2, val1> b/w indices [0, 1).
After the inner loop outerLoop delete range [1,2) at index 1 [line 430]
The inner Loop will perform put operation<2, val1> b/w indices [2, 2) which is nothing.
After the inner loop outerLoop delete range [1,3) at index 2 [line 430]
So it will perform all op. Yeah I agree what we could have possibly done is checked for continuous indexes of delRange[1, 2) , delRange [1, 3) and we could perform a union of overlapping ranges before performing a put operation to avoid it altogether.
Effectively [1,2) + [1,3) will be effectively [1,3)
I didn't want a lot of complexity in this logic. But if you feel we should do it I can make that change.

SaketaChalamchala · 2025-07-15T18:57:04Z

+ }
+
+ void deleteRange(byte[] startKey, byte[] endKey) {
+ delRangeCount++;


Would it be better to move the "discard keys that fall in deleteRange logic" here?
Iterate through the opsKeys map, remove keys that fall in range and then put the operation in batchOps.

It would make deleteRange operation not performant. The initial patch had TreeMap which will perform deleteRange efficiently with O(log(N)) in a BST. But because of that put, deletes would suffer O(1) vs O(log(n)) in hashMap and TreeMap. Instead of optimizing deleteRange which is going to be a lesser frequent operation than individual puts and delete. I made it HashMap and I am just taking a hit on commit where I sort Keys on the basis of sequence numbers which is going to be much faster with use of more memory.

jojochuang

a few cosmetic nitpicks otherwise it seems good.

jojochuang · 2025-07-30T23:33:13Z

+ }
+ for (int i = startIndex; i < deleteRangeIndex; i++) {
+ Operation op = ops.get(i);
+ Bytes key = op.getKey();


Suggest to add a check to verify key is not null pointer. And that is expected because the key is between startIndex and deleteRangeIndices.
Suggested change
Byteskey = op.getKey();
Byteskey = op.getKey();
Preconditions.checkArgument(key)

getKey() should never return a null value

jojochuang · 2025-07-30T23:42:05Z

- Preconditions.checkState(overwritten == null);
+ }

+ int overWriteOpIfExist(Bytes key, Operation operation) {


consider updating overWriteOpIfExist() so that DeleteRanger API usage is included.
Otherwise it means they don't contribute to batchSize tracking. This inconsistency could lead to inaccurate batch size reporting.

jojochuang · 2025-07-31T00:01:55Z

 */
- private final Map<Bytes, Object> ops = new HashMap<>();
+ private final Map<Bytes, Integer> opsKeys = new HashMap<>();
+ private final Map<Integer, Operation> batchOps = new HashMap<>();


add a description for batchOps similar to opsKeys?
Suggested change
privatefinalMap<Integer, Operation> batchOps = newHashMap<>();
/**
* Cacheofallbatchoperationstobeappliedforthisfamily, keyedbytheirinsertionorderindex.
*
* Eachentrymapsauniqueintegerindex (representingtheorderinwhichtheoperationwasadded)
* toan {@linkOperation} object (suchasput, delete, ordeleterange).
*
* Thisensuresthatalldatabaseoperationsareexecutedinthesameorderastheywereadded,
* whichiscrucialforcorrectlyhandlingoverlappingordependentoperations, suchasdeleteranges.
*/
privatefinalMap<Integer, Operation> batchOps = newHashMap<>();

jojochuang · 2025-07-31T00:06:54Z

- Preconditions.checkState(overwritten == null);
+ }

+ int overWriteOpIfExist(Bytes key, Operation operation) {


return value is never used, and can be made a private method

Change-Id: I54cdc1733e7f23e449b7336fbdff90c11b382bad

Change-Id: I0395802eed3ca7514d668948fee62ab21c3ac8d2

swamirishi · 2025-08-19T03:13:04Z

@jojochuang @SaketaChalamchala Can you review the code again I have addressed the review comments and also added the test cases.

jojochuang · 2025-08-20T23:20:55Z

H I Dm: Found reliance on default encoding in org.apache.hadoop.hdds.utils.db.TestRDBBatchOperation.testBatchOperationWithDeleteRange(): String.getBytes() At TestRDBBatchOperation.java:[line 105]

Change-Id: I93add67453d71ae576b457b7ec72ddd30982e22e

Change-Id: I6d50300c1af91db9afdd63c437f30889bd8f6fc9

SaketaChalamchala

Thanks for the patch @swamirishi. The updated patch includes overwriting operations covered by continuous delete ranges.
LGTM overall.

jojochuang · 2025-08-22T18:30:04Z

There's a JDK compilation error. Trigger a rebuild just in case.

Change-Id: I0ab3ec95ff18e84182f7412758d5da98fd979c13

swamirishi · 2025-08-24T14:37:23Z

@jojochuang Can this be merged?

jojochuang · 2025-08-24T18:55:42Z

+1 it passed in my local tree https://github.com/jojochuang/ozone/actions/runs/17170949740

…rocksdb batch write (apache#8774) (cherry picked from commit ae92a18) Conflicts: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/TypedTable.java hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/TestRDBTableStore.java Change-Id: I1f6eadab38ead1d3ba5a13dfdd00ec0c0913e722

szetszwo · 2025-12-22T20:23:30Z

+ // This method mimics the ByteWiseComparator in RocksDB.
+ @Override
+ public int compareTo(RDBBatchOperation.Bytes that) {


@swamirishi , Let's just use ByteWiseComparator. There is no reason to implemant a new compareTo method.

I realized that and was going to raise a patch for this

szetszwo

@swamirishi , this change in general was good. Thanks!

However, it would be much better if you didn't mixing the refactoring and the new DeleteRange feature. The change became unnecessarily big and difficult to review (Hope that we could respect reviewers' time and effort).

Also, the tests are way to simple compared with the changes. Could you add a better test?

szetszwo · 2025-12-23T21:14:07Z

+ final List<Pair<Pair<String, String>, Integer>> putKeys = new ArrayList<>();
+ final List<Pair<String, Integer>> deleteKeys = new ArrayList<>();
+ AtomicInteger cnt = new AtomicInteger(0);
+ try (MockedConstruction<ManagedWriteBatch> mockedConstruction = Mockito.mockConstruction(ManagedWriteBatch.class,


Why mocking but not testing with a real db? How do you test the mock to see if it could mock everything correctly?

szetszwo · 2025-12-23T21:15:04Z

+ final List<Pair<Pair<String, String>, Integer>> deleteKeyRangePairs = new ArrayList<>();
+ final List<Pair<Pair<String, String>, Integer>> putKeys = new ArrayList<>();


What is a List<Pair<Pair<String, String>, Integer>>? How could other people easily understand your code?

szetszwo · 2025-12-23T21:20:52Z

+ batchOperation.put(columnFamily, string2Bytes("key01"), string2Bytes("value01"));
+ batchOperation.put(columnFamily, string2Bytes("key02"), string2Bytes("value02"));
+ batchOperation.put(columnFamily, string2Bytes("key03"), string2Bytes("value03"));
+ batchOperation.put(columnFamily, string2Bytes("key03"), string2Bytes("value04"));
+ batchOperation.delete(columnFamily, string2Bytes("key05"));
+ batchOperation.deleteRange(columnFamily, string2Bytes("key01"), string2Bytes("key02"));
+ batchOperation.deleteRange(columnFamily, string2Bytes("key02"), string2Bytes("key03"));
+ batchOperation.put(columnFamily, string2Bytes("key04"), string2Bytes("value04"));
+ batchOperation.put(columnFamily, string2Bytes("key06"), string2Bytes("value05"));
+ batchOperation.deleteRange(columnFamily, string2Bytes("key06"), string2Bytes("key12"));
+ batchOperation.deleteRange(columnFamily, string2Bytes("key09"), string2Bytes("key10"));
+ RocksDatabase db = Mockito.mock(RocksDatabase.class);
+ doNothing().when(db).batchWrite(any());
+ batchOperation.commit(db);
+ assertEquals(deleteKeys, Collections.singletonList(Pair.of("key05", 1)));
+ assertEquals(deleteKeyRangePairs, asList(Pair.of(Pair.of("key01", "key02"), 2),
+ Pair.of(Pair.of("key02", "key03"), 3),
+ Pair.of(Pair.of("key06", "key12"), 5),
+ Pair.of(Pair.of("key09", "key10"), 6)));
+ assertEquals(putKeys, Arrays.asList(Pair.of(Pair.of("key03", "value04"), 0),
+ Pair.of(Pair.of("key04", "value04"), 4)));


This hard code test seem way too simple compared to the code added? All the keys just have the same length?
I suggest to have a random test to run at least 10,000 operations. Run them with two different DBs,
one with RDBBatchOperation, and
another one directly with RocksDB (without RDBBatchOperation).
Then, compare contents of two DBs .

oh nice! This is a great test to have. Let me add it

szetszwo · 2025-12-23T23:32:11Z

Filed HDDS-13415 for cleaning up the code.

HDDS-13415. Support DeleteRange operation as part of the rocksdb batc…
daa4685
…h write Change-Id: I65444c0155b471c31ea1283d563d9a948f5e4005

swamirishi force-pushed the HDDS-13415 branch from 0db2ad4 to daa4685Compare July 10, 2025 02:50

HDDS-13415. Using TreeMap when no deleteRange could have performance …
26059f4
…impact Change-Id: I3b34e1bb6ca7587243f6eef176c396ca0132dbc3

swamirishi added 2 commits July 10, 2025 05:15

Merge remote-tracking branch 'apache/master' into HEAD
878cf41
Change-Id: Iab23d0b1316eac9c376fd06226d234b97e908b4f

HDDS-13415. Fix build issues
b6ce285
Change-Id: I74fed5aebb5c0588cbf97f70a0f9acf94edfa87c

SaketaChalamchala reviewed Jul 15, 2025
View reviewed changes

SaketaChalamchala mentioned this pull request Jul 16, 2025
HDDS-13252. Use RocksDB deleteRange to cleanup AOS on snapshot creation #8690
Closed

smengcl added the snapshot https://issues.apache.org/jira/browse/HDDS-6517 label Jul 17, 2025

jojochuang reviewed Jul 31, 2025
View reviewed changes

swamirishi added 2 commits August 18, 2025 14:17

HDDS-13415. Handle continuous delete ranges
da27b38
Change-Id: I54cdc1733e7f23e449b7336fbdff90c11b382bad

HDDS-13415. Add test case
1bef216
Change-Id: I0395802eed3ca7514d668948fee62ab21c3ac8d2

swamirishi requested review from SaketaChalamchala and jojochuang August 19, 2025 03:12

jojochuang marked this pull request as ready for review August 20, 2025 22:37

swamirishi added 2 commits August 20, 2025 23:31

HDDS-13415. Fix findbugs
9c57362
Change-Id: I93add67453d71ae576b457b7ec72ddd30982e22e

HDDS-13415. Fix checkstyle
1a3b34f
Change-Id: I6d50300c1af91db9afdd63c437f30889bd8f6fc9

SaketaChalamchala approved these changes Aug 22, 2025
View reviewed changes

SaketaChalamchala mentioned this pull request Aug 22, 2025
HDDS-13252. Use deleteRangeWithBatch API to delete keys in snapshot scope from AOS deleted space. #8964
Merged

Merge remote-tracking branch 'apache/master' into HEAD
40f75eb
Change-Id: I0ab3ec95ff18e84182f7412758d5da98fd979c13

jojochuang merged commit ae92a18 into apache:masterAug 24, 2025
42 checks passed

szetszwo mentioned this pull request Dec 17, 2025
HDDS-14166. Support deleteWithBatch with ByteBuffer #9497
Closed

szetszwo reviewed Dec 22, 2025
View reviewed changes

szetszwo reviewed Dec 23, 2025
View reviewed changes

swamirishi mentioned this pull request Dec 24, 2025
HDDS-14239. Simplify Prepare Batch looping for DeleteRange Op processing in RDBBatchOperation #9553
Closed

szetszwo mentioned this pull request Dec 30, 2025
HDDS-14166. Get rid of byte array operations from RDBBatchOperation for PUT and DELETE #9552
Merged

	Byteskey = op.getKey();
	Byteskey = op.getKey();
	Preconditions.checkArgument(key)

-privatefinalMap<Integer, Operation> batchOps = newHashMap<>();
+/**
+ * Cacheofallbatchoperationstobeappliedforthisfamily, keyedbytheirinsertionorderindex.
+ *
+ * Eachentrymapsauniqueintegerindex (representingtheorderinwhichtheoperationwasadded)
+ * toan {@linkOperation} object (suchasput, delete, ordeleterange).
+ *
+ * Thisensuresthatalldatabaseoperationsareexecutedinthesameorderastheywereadded,
+ * whichiscrucialforcorrectlyhandlingoverlappingordependentoperations, suchasdeleteranges.
+ */
+privatefinalMap<Integer, Operation> batchOps = newHashMap<>();

		final List<Pair<Pair<String, String>, Integer>> deleteKeyRangePairs = new ArrayList<>();
		final List<Pair<Pair<String, String>, Integer>> putKeys = new ArrayList<>();

Uh oh!

Conversation

swamirishi commented Jul 10, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

swamirishi commented Jul 10, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swamirishiJul 17, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jojochuang left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jojochuangJul 31, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swamirishi commented Aug 19, 2025

Uh oh!

jojochuang commented Aug 20, 2025

Uh oh!

SaketaChalamchala left a comment

Choose a reason for hiding this comment

Uh oh!

jojochuang commented Aug 22, 2025

Uh oh!

swamirishi commented Aug 24, 2025

Uh oh!

jojochuang commented Aug 24, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swamirishiDec 24, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szetszwo commented Dec 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

swamirishi commented Jul 10, 2025•
edited
Loading

swamirishiJul 17, 2025•
edited
Loading

jojochuangJul 31, 2025•
edited
Loading

swamirishiDec 24, 2025•
edited
Loading