Skip to content

HDDS-10699. Refactor ContainerBalancerTask and tests in TestContainerBalancerTask - #6537

Merged
adoroszlai merged 2 commits into
apache:masterfrom
unisonteam:container_balancer_tests_refactoring
May 17, 2024
Merged

HDDS-10699. Refactor ContainerBalancerTask and tests in TestContainerBalancerTask#6537
adoroszlai merged 2 commits into
apache:masterfrom
unisonteam:container_balancer_tests_refactoring

Conversation

@Montura

Copy link
Copy Markdown
Contributor

In PR for HDDS-9889 we discussed with Siddhant Sangwan that tests form org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerTask could be refactored using MockedSCM class (introduced in HDDS-9889)

What changes were proposed in this pull request?

  1. Refactor some tests from org.apache.hadoop.hdds.scm.container.balancer.TestContainerBalancerTask
  2. Made some cleanups in org.apache.hadoop.hdds.scm.container.balancer.ContainerBalancerTask related to changes in refactored test

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10699

How was this patch tested?

Use standalone tests

@Montura

This comment was marked as outdated.

@adoroszlai
adoroszlai marked this pull request as draft April 22, 2024 11:36
@Montura
Montura marked this pull request as ready for review May 1, 2024 21:11
@Montura
Montura marked this pull request as draft May 2, 2024 07:21
@Montura
Monturaforce-pushed the container_balancer_tests_refactoring branch from 398d112 to 14e70ddCompareMay 2, 2024 08:50
@Montura
Montura marked this pull request as ready for review May 2, 2024 08:50
@Montura
Monturaforce-pushed the container_balancer_tests_refactoring branch from 14e70dd to 8ededd4CompareMay 2, 2024 16:53
}
config.setIterations(1);
config.setMaxSizeToMovePerIteration(50 * STORAGE_UNIT);
mockedSCM.init(config);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relating to the above comment, we would not require to call init here specifically. Similarly for all the test cases in this class.
What is the reason to remove init call within startBalancerTask?

@MonturaMonturaMay 7, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Tejaskriya, in some tests we do additional mock-operations for MockedSCM like here.

We have to initialize MockedSCM to do initial mocks and then add some more test-specific mocks.

I'll think how to make it better

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UPD: fixed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM

@Montura
Monturaforce-pushed the container_balancer_tests_refactoring branch from a7d5691 to d133d78CompareMay 7, 2024 09:06
@Montura
Montura requested a review from TejaskriyaMay 7, 2024 09:10
@Montura
Monturaforce-pushed the container_balancer_tests_refactoring branch from d133d78 to 8b2f423CompareMay 7, 2024 09:29

@sumitagrawlsumitagrawl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Montura Thanks for working over this, LGTM

@Montura

Copy link
Copy Markdown
ContributorAuthor

@adoroszlai, merge the changes please

@adoroszlai
adoroszlai merged commit df5683f into apache:masterMay 17, 2024
@adoroszlai

Copy link
Copy Markdown
Contributor

Thanks @Montura for the patch, @sumitagrawl, @Tejaskriya for the review.

jojochuang pushed a commit to jojochuang/ozone that referenced this pull request May 23, 2024
errose28 added a commit to errose28/ozone that referenced this pull request May 28, 2024
…concile-cli
* HDDS-10239-container-reconciliation: (296 commits)
HDDS-10897. Refactor OzoneQuota (apache#6714)
HDDS-10422. Fix some warnings about exposing internal representation in hdds-common (apache#6351)
HDDS-10899. Refactor Lease callbacks (apache#6715)
HDDS-10890. Increase default value for hdds.container.ratis.log.appender.queue.num-elements (apache#6711)
HDDS-10832. Client should switch to streaming based on OpenKeySession replication (apache#6683)
HDDS-10435. Support S3 object tags for existing requests (apache#6607)
HDDS-10883. Improve logging in Recon for finalising DN logic. (apache#6704)
HDDS-8752. Enable TestOzoneRpcClientAbstract#testOverWriteKeyWithAndWithOutVersioning (apache#6702)
HDDS-10875. XceiverRatisServer#getRaftPeersInPipeline should be called before XceiverRatisServer#removeGroup (apache#6696)
HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. (apache#6376)
HDDS-10878. Bump zstd-jni to 1.5.6-3 (apache#6701)
HDDS-10877. Bump Dropwizard metrics to 3.2.6 (apache#6699)
HDDS-10876. Bump jackson to 2.16.2 (apache#6697)
HDDS-6116. Remove flaky tag from TestSCMInstallSnapshot (apache#6695)
HDDS-2643. TestOzoneDelegationTokenSecretManager#testRenewTokenFailureRenewalTime fails intermittently.
HDDS-10699. Refactor ContainerBalancerTask and TestContainerBalancerTask (apache#6537)
HDDS-10861. Ozone cli supports default ozone.om.service.id (apache#6680)
HDDS-10859. Improve error messages when decommission and maintenance fail-early (apache#6678)
HDDS-9031. Upgrade acceptance tests to Docker Compose v2 (apache#6667)
HDDS-10559. Add a warning or a check to run repair tool as System user (apache#6574)
...
Conflicts:
hadoop-ozone/dist/src/main/smoketest/admincli/container.robot
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@Montura@adoroszlai@sumitagrawl@Tejaskriya