Skip to content

HDFS-14361:SNN will always upload fsimage - #592

Closed
hunshenshi wants to merge 3 commits into
apache:trunkfrom
hunshenshi:HDFS-14361
Closed

HDFS-14361:SNN will always upload fsimage#592
hunshenshi wants to merge 3 commits into
apache:trunkfrom
hunshenshi:HDFS-14361

Conversation

@hunshenshi

@hunshenshihunshenshi commented Mar 12, 2019

Copy link
Copy Markdown
Contributor

Related to HDFS-12248.

booleansendRequest = isPrimaryCheckPointer
|| secsSinceLastUpload >= checkpointConf.getQuietPeriod();
doCheckpoint(sendRequest);

If sendRequest is true, SNN will upload fsimage. But isPrimaryCheckPointer always is true,

if (ie == null && ioe == null) {
//Update only when response from remote about success orlastUploadTime = monotonicNow();
// we are primary if we successfully updated the ANNthis.isPrimaryCheckPointer = success;
}

isPrimaryCheckPointer should be outside the if condition.

If the ANN update was not successful, then isPrimaryCheckPointer should be set to false.

@hunshenshihunshenshi reopened this Mar 12, 2019
@hadoop-yetus

hadoop-yetus commented Mar 12, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec1028Docker mode activated.
_ Prechecks _
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1206trunk passed
+1compile67trunk passed
+1checkstyle57trunk passed
+1mvnsite67trunk passed
+1shadedclient799branch has no errors when building and testing our client artifacts.
+1findbugs118trunk passed
+1javadoc48trunk passed
_ Patch Compile Tests _
+1mvninstall58the patch passed
+1compile54the patch passed
+1javac54the patch passed
+1checkstyle48the patch passed
+1mvnsite59the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient761patch has no errors when building and testing our client artifacts.
+1findbugs125the patch passed
+1javadoc46the patch passed
_ Other Tests _
-1unit5604hadoop-hdfs in the patch failed.
+1asflicense34The patch does not generate ASF License warnings.
10224
ReasonTests
Failed junit testshadoop.hdfs.server.datanode.TestBPOfferService
hadoop.hdfs.TestRollingUpgrade
hadoop.hdfs.tools.TestDFSZKFailoverController
hadoop.hdfs.qjournal.client.TestQJMWithFaults
hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized
SubsystemReport/Notes
DockerClient=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-592/1/artifact/out/Dockerfile
GITHUB PR#592
JIRA IssueHDFS-14361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux 3a2b70b36cb8 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / c730786
mavenversion: Apache Maven 3.3.9
Default Java1.8.0_191
findbugsv3.1.0-RC1
unithttps://builds.apache.org/job/hadoop-multibranch/job/PR-592/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-592/1/testReport/
Max. process+thread count2871 (vs. ulimit of 5500)
modulesC: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-592/1/console
Powered byApache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

hadoop-yetus commented Mar 12, 2019

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec42Docker mode activated.
_ Prechecks _
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1115trunk passed
+1compile63trunk passed
+1checkstyle53trunk passed
+1mvnsite68trunk passed
+1shadedclient807branch has no errors when building and testing our client artifacts.
+1findbugs121trunk passed
+1javadoc52trunk passed
_ Patch Compile Tests _
+1mvninstall61the patch passed
+1compile57the patch passed
+1javac57the patch passed
+1checkstyle49the patch passed
+1mvnsite61the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient749patch has no errors when building and testing our client artifacts.
+1findbugs128the patch passed
+1javadoc49the patch passed
_ Other Tests _
-1unit6290hadoop-hdfs in the patch failed.
+1asflicense36The patch does not generate ASF License warnings.
9838
ReasonTests
Failed junit testshadoop.hdfs.tools.TestDFSZKFailoverController
hadoop.hdfs.TestRollingUpgrade
hadoop.hdfs.server.datanode.TestBPOfferService
hadoop.hdfs.server.datanode.TestDirectoryScanner
SubsystemReport/Notes
DockerClient=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-592/2/artifact/out/Dockerfile
GITHUB PR#592
JIRA IssueHDFS-14361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux d201d7a9d7c6 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / c730786
mavenversion: Apache Maven 3.3.9
Default Java1.8.0_191
findbugsv3.1.0-RC1
unithttps://builds.apache.org/job/hadoop-multibranch/job/PR-592/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-592/2/testReport/
Max. process+thread count2720 (vs. ulimit of 5500)
modulesC: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-592/2/console
Powered byApache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
@lhaiesp@jagadish-v0
SAMZA-1794: setting application acl in launch context
1d2f054
Currently we don't set application acl for container launch context. See https://hadoop.apache.org/docs/r2.6.4/api/org/apache/hadoop/yarn/api/records/ContainerLaunchContext.html#setApplicationACLs(java.util.Map)
This could potentially cause problem if samza job is running on a secured YARN cluster. Say user A submits the job, then by default only user A can view the log and the status of the job. Even worse case is that user A submits the job through some proxy account, then even user A herself/himself couldn't access to logs/status of the application.
We need to make some changes for the YARN application submission to set application acls in launch context as configured.
Author: Hai Lu <halu@linkedin.com>
Reviewers: Jagadish<jagadish@apache.org>
Closesapache#592 from lhaiesp/master
@github-actions

github-actionsBot commented Dec 24, 2025

Copy link
Copy Markdown
Contributor

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Stale

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hunshenshi@hadoop-yetus