Skip to content

HDDS-11349. Add NullPointer handling when volume/bucket tables are not initialized - #7103

Merged
tanvipenumudy merged 3 commits into
apache:masterfrom
spacemonkd:HDDS-11349
Aug 21, 2024
Merged

HDDS-11349. Add NullPointer handling when volume/bucket tables are not initialized#7103
tanvipenumudy merged 3 commits into
apache:masterfrom
spacemonkd:HDDS-11349

Conversation

@spacemonkd

@spacemonkdspacemonkd commented Aug 21, 2024•
edited by tanvipenumudy
Loading

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

HDDS-11349. Add NullPointer handling when volume/bucket tables are not initialized

Please describe your PR in detail:

  • Currently if we call the /volumes API while the Volumes table is being initialized it returns a 500 error.
  • This PR will add the handling of this scenario
  • After this change, if we call /volumes or /buckets endpoint before the table is completely initialized, it will return
{
totalCount: 0,
volumes: []
}

{
buckets: []
totalCount: 0
}

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11349

How was this patch tested?

It was tested using unit tests and also manually verifying the response is properly returned.
Screenshot 2024-08-21 at 12 45 17

Screenshot 2024-08-21 at 12 45 26

@tanvipenumudytanvipenumudy changed the title HDDS-111349. Add NullPointer handling when Volumes table is not initialized.HDDS-11349. Add NullPointer handling when Volumes table is not initialized.Aug 21, 2024
@spacemonkdspacemonkd changed the title HDDS-11349. Add NullPointer handling when Volumes table is not initialized.HDDS-11349. Add NullPointer handling when Volumes/Buckets table is not initialized.Aug 21, 2024
devmadhuu
devmadhuu approved these changes Aug 21, 2024
View reviewed changes

@devmadhuudevmadhuu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @devabhishekpal . Changes LGTM +1

tanvipenumudy
tanvipenumudy approved these changes Aug 21, 2024
View reviewed changes

@tanvipenumudytanvipenumudy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @devabhishekpal for the patch, LGTM.

@tanvipenumudytanvipenumudy added the bug Something isn't working label Aug 21, 2024
@tanvipenumudy

tanvipenumudy commented Aug 21, 2024

Copy link
Copy Markdown
Contributor

Thank you @devabhishekpal for the patch, @devmadhuu for the review.

@tanvipenumudytanvipenumudy changed the title HDDS-11349. Add NullPointer handling when Volumes/Buckets table is not initialized.HDDS-11349. Add NullPointer handling when volume/bucket tables are not initializedAug 21, 2024
@tanvipenumudy
tanvipenumudy merged commit 3db5ad3 into apache:masterAug 21, 2024
@devmadhuu

devmadhuu commented Aug 21, 2024

Copy link
Copy Markdown
Contributor

@tanvipenumudy when you merge the PR, kindly also resolve the associated JIRA by mentioning the fixed version as 1.5.0 (current one)

@tanvipenumudy

tanvipenumudy commented Aug 21, 2024

Copy link
Copy Markdown
Contributor

Resolved the ticket: HDDS-11349, thank you @devmadhuu!

@spacemonkd
spacemonkd deleted the HDDS-11349 branch August 21, 2024 11:24
errose28 added a commit to errose28/ozone that referenced this pull request Aug 21, 2024
@errose28
Merge branch 'master' into HDDS-10239-container-reconciliation
7186a77
* master: (50 commits)
HDDS-11331. Fix Datanode unable to report for a long time (apache#7090)
HDDS-11346. FS CLI gives incorrect recursive volume deletion prompt (apache#7102)
HDDS-11349. Add NullPointer handling when volume/bucket tables are not initialized (apache#7103)
HDDS-11209. Avoid insufficient EC pipelines in the container pipeline cache (apache#6974)
HDDS-11284. refactor quota repair non-blocking while upgrade (apache#7035)
HDDS-9790. Add tests for Overview page (apache#6983)
HDDS-10904. [hsync] Enable PutBlock piggybacking and incremental chunk list by default (apache#7074)
HDDS-11322. [hsync] Block ECKeyOutputStream from calling hsync and hflush (apache#7098)
HDDS-11325. Intermittent failure in TestBlockOutputStreamWithFailures#testContainerClose (apache#7099)
HDDS-11340. Avoid extra PubBlock call when a full block is closed (apache#7094)
HDDS-11155. Improve Volumes page UI (apache#7048)
HDDS-11324. Negative value preOpLatencyMs in DN audit log (apache#7093)
HDDS-11246. [Recon] Use optional chaining instead of explicit undefined check for Objects in Container and Pipeline Module. (apache#7037)
HDDS-11323. Mark TestLeaseRecovery as flaky
HDDS-11338. Bump zstd-jni to 1.5.6-4 (apache#7085)
HDDS-11337. Bump Spring Framework to 5.3.39 (apache#7084)
HDDS-11327. [hsync] Revert config default ozone.fs.hsync.enabled to false (apache#7079)
HDDS-11325. Mark testWriteMoreThanMaxFlushSize as flaky
HDDS-11336. Bump slf4j to 2.0.16 (apache#7086)
HDDS-11335. Bump exec-maven-plugin to 3.4.1 (apache#7087)
...
Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 21, 2024
@errose28
Merge branch 'master' into HDDS-10239-container-reconciliation
3355701
* master: (50 commits)
HDDS-11331. Fix Datanode unable to report for a long time (apache#7090)
HDDS-11346. FS CLI gives incorrect recursive volume deletion prompt (apache#7102)
HDDS-11349. Add NullPointer handling when volume/bucket tables are not initialized (apache#7103)
HDDS-11209. Avoid insufficient EC pipelines in the container pipeline cache (apache#6974)
HDDS-11284. refactor quota repair non-blocking while upgrade (apache#7035)
HDDS-9790. Add tests for Overview page (apache#6983)
HDDS-10904. [hsync] Enable PutBlock piggybacking and incremental chunk list by default (apache#7074)
HDDS-11322. [hsync] Block ECKeyOutputStream from calling hsync and hflush (apache#7098)
HDDS-11325. Intermittent failure in TestBlockOutputStreamWithFailures#testContainerClose (apache#7099)
HDDS-11340. Avoid extra PubBlock call when a full block is closed (apache#7094)
HDDS-11155. Improve Volumes page UI (apache#7048)
HDDS-11324. Negative value preOpLatencyMs in DN audit log (apache#7093)
HDDS-11246. [Recon] Use optional chaining instead of explicit undefined check for Objects in Container and Pipeline Module. (apache#7037)
HDDS-11323. Mark TestLeaseRecovery as flaky
HDDS-11338. Bump zstd-jni to 1.5.6-4 (apache#7085)
HDDS-11337. Bump Spring Framework to 5.3.39 (apache#7084)
HDDS-11327. [hsync] Revert config default ozone.fs.hsync.enabled to false (apache#7079)
HDDS-11325. Mark testWriteMoreThanMaxFlushSize as flaky
HDDS-11336. Bump slf4j to 2.0.16 (apache#7086)
HDDS-11335. Bump exec-maven-plugin to 3.4.1 (apache#7087)
...
Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 26, 2024
@errose28
Merge branch 'HDDS-10239-container-reconciliation' into HDDS-11290-sc…
769a9aa
…an-on-error
* HDDS-10239-container-reconciliation: (428 commits)
HDDS-11081. Use thread-local instance of FileSystem in Freon tests (apache#7091)
HDDS-11333. Avoid hard-coded current version in upgrade/xcompat tests (apache#7089)
Mark TestPipelineManagerMXBean#testPipelineInfo as flaky
Mark TestAddRemoveOzoneManager#testForceBootstrap as flaky
HDDS-11352. HDDS-11353. Mark TestOzoneManagerHAWithStoppedNodes as flaky
HDDS-11354. Mark TestOzoneManagerSnapshotAcl#testLookupKeyWithNotAllowedUserForPrefixAcl as flaky
HDDS-11355. Mark TestMultiBlockWritesWithDnFailures#testMultiBlockWritesWithIntermittentDnFailures as flaky
HDDS-11227. Use server default key provider to encrypt/decrypt keys from multiple OMs. (apache#7081)
HDDS-11316. Improve Create Key and Chunk IO Dashboards (apache#7075)
HDDS-11239. Fix KeyOutputStream's exception handling when calling hsync concurrently (apache#7047)
HDDS-11254. Reconcile commands should be handled by datanode ReplicationSupervisor (apache#7076)
HDDS-11331. Fix Datanode unable to report for a long time (apache#7090)
HDDS-11346. FS CLI gives incorrect recursive volume deletion prompt (apache#7102)
HDDS-11349. Add NullPointer handling when volume/bucket tables are not initialized (apache#7103)
HDDS-11209. Avoid insufficient EC pipelines in the container pipeline cache (apache#6974)
HDDS-11284. refactor quota repair non-blocking while upgrade (apache#7035)
HDDS-9790. Add tests for Overview page (apache#6983)
HDDS-10904. [hsync] Enable PutBlock piggybacking and incremental chunk list by default (apache#7074)
HDDS-11322. [hsync] Block ECKeyOutputStream from calling hsync and hflush (apache#7098)
HDDS-11325. Intermittent failure in TestBlockOutputStreamWithFailures#testContainerClose (apache#7099)
...
Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/checksum/ContainerChecksumTreeManager.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 28, 2024
@errose28
Merge branch 'HDDS-10239-container-reconciliation' into HDDS-11253-co…
426beac
…rrupt-files
* HDDS-10239-container-reconciliation: (61 commits)
HDDS-11081. Use thread-local instance of FileSystem in Freon tests (apache#7091)
HDDS-11333. Avoid hard-coded current version in upgrade/xcompat tests (apache#7089)
Mark TestPipelineManagerMXBean#testPipelineInfo as flaky
Mark TestAddRemoveOzoneManager#testForceBootstrap as flaky
HDDS-11352. HDDS-11353. Mark TestOzoneManagerHAWithStoppedNodes as flaky
HDDS-11354. Mark TestOzoneManagerSnapshotAcl#testLookupKeyWithNotAllowedUserForPrefixAcl as flaky
HDDS-11355. Mark TestMultiBlockWritesWithDnFailures#testMultiBlockWritesWithIntermittentDnFailures as flaky
HDDS-11227. Use server default key provider to encrypt/decrypt keys from multiple OMs. (apache#7081)
HDDS-11316. Improve Create Key and Chunk IO Dashboards (apache#7075)
HDDS-11239. Fix KeyOutputStream's exception handling when calling hsync concurrently (apache#7047)
HDDS-11254. Reconcile commands should be handled by datanode ReplicationSupervisor (apache#7076)
HDDS-11331. Fix Datanode unable to report for a long time (apache#7090)
HDDS-11346. FS CLI gives incorrect recursive volume deletion prompt (apache#7102)
HDDS-11349. Add NullPointer handling when volume/bucket tables are not initialized (apache#7103)
HDDS-11209. Avoid insufficient EC pipelines in the container pipeline cache (apache#6974)
HDDS-11284. refactor quota repair non-blocking while upgrade (apache#7035)
HDDS-9790. Add tests for Overview page (apache#6983)
HDDS-10904. [hsync] Enable PutBlock piggybacking and incremental chunk list by default (apache#7074)
HDDS-11322. [hsync] Block ECKeyOutputStream from calling hsync and hflush (apache#7098)
HDDS-11325. Intermittent failure in TestBlockOutputStreamWithFailures#testContainerClose (apache#7099)
...
Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/checksum/ContainerChecksumTreeManager.java
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devmadhuudevmadhuudevmadhuu approved these changes
@tanvipenumudytanvipenumudytanvipenumudy approved these changes

Assignees

No one assigned

Labels

bugSomething isn't workingrecon

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@spacemonkd@tanvipenumudy@devmadhuu