Skip to content

HADOOP-13617 : Retry requests with updated authentication - #361

Closed
jelmerk wants to merge 1 commit into
apache:trunkfrom
jelmerk:HADOOP_13617
Closed

HADOOP-13617 : Retry requests with updated authentication#361
jelmerk wants to merge 1 commit into
apache:trunkfrom
jelmerk:HADOOP_13617

Conversation

@jelmerk

@jelmerkjelmerk commented Apr 1, 2018

Copy link
Copy Markdown

We started using hadoop's openstack filesystem (Swift) as a storage backend for Apache Flink and ran into the problem that every few hours an error would be triggered that would cause our flink jobs to get restarted.

The problem happens when the token that is used to make api calls expires. At that point the server will return a 401 status code (unauthorized).

What should happen next is that the token should be refreshed and the request should be retried with the new token.

But what happens is that while a new token is requested , the request is retried with the old token and will always fail.

The fix is a simple one, set the correct authentication header on the request before retrying

It turns out that this issue had already been reported in 2016 and a patch was provided by the original reporter.

However for some reason the patch was never merged.

Because the code has changed a bit since the patch was created, i took the liberty of adapting it so it compiles cleanly on the current trunk. I also removed the test the original author created because it basically did a sleep until a token expired. (which usually takes hours)

@jelmerkjelmerk changed the title fix for HADOOP-13617HADOOP-13617 : retry requests with updated authenticationApr 2, 2018
@jelmerk
jelmerkforce-pushed the HADOOP_13617 branch 4 times, most recently from ceb94ad to 3fa373dCompareApril 3, 2018 06:56
@jelmerkjelmerk changed the title HADOOP-13617 : retry requests with updated authenticationHADOOP-13617 : Retry requests with updated authenticationApr 3, 2018
@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec90Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1200trunk passed
+1compile22trunk passed
+1checkstyle21trunk passed
+1mvnsite27trunk passed
+1shadedclient786branch has no errors when building and testing our client artifacts.
+1javadoc21trunk passed
0spotbugs39Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs35trunk passed
_ Patch Compile Tests _
+1mvninstall19the patch passed
+1compile18the patch passed
+1javac18the patch passed
+1checkstyle15the patch passed
+1mvnsite24the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient838patch has no errors when building and testing our client artifacts.
+1javadoc19the patch passed
+1findbugs42the patch passed
_ Other Tests _
+1unit21hadoop-openstack in the patch passed.
+1asflicense28The patch does not generate ASF License warnings.
3324
SubsystemReport/Notes
DockerClient=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/1/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux 30c63c1f3777 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / 7f1b76c
Default Java1.8.0_212
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/1/testReport/
Max. process+thread count306 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/1/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec44Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1119trunk passed
+1compile21trunk passed
+1checkstyle15trunk passed
+1mvnsite24trunk passed
+1shadedclient682branch has no errors when building and testing our client artifacts.
+1javadoc22trunk passed
0spotbugs41Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs39trunk passed
_ Patch Compile Tests _
+1mvninstall20the patch passed
+1compile19the patch passed
+1javac19the patch passed
+1checkstyle14the patch passed
+1mvnsite20the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient786patch has no errors when building and testing our client artifacts.
+1javadoc21the patch passed
+1findbugs39the patch passed
_ Other Tests _
+1unit18hadoop-openstack in the patch passed.
+1asflicense27The patch does not generate ASF License warnings.
3019
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/2/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux b43bb8d904e1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / c7c7a88
Default Java1.8.0_212
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/2/testReport/
Max. process+thread count446 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/2/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec75Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1299trunk passed
+1compile30trunk passed
+1checkstyle28trunk passed
+1mvnsite34trunk passed
+1shadedclient956branch has no errors when building and testing our client artifacts.
+1javadoc25trunk passed
0spotbugs46Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs44trunk passed
_ Patch Compile Tests _
+1mvninstall23the patch passed
+1compile21the patch passed
+1javac21the patch passed
+1checkstyle17the patch passed
+1mvnsite26the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient1009patch has no errors when building and testing our client artifacts.
+1javadoc21the patch passed
+1findbugs47the patch passed
_ Other Tests _
+1unit24hadoop-openstack in the patch passed.
+1asflicense31The patch does not generate ASF License warnings.
3800
SubsystemReport/Notes
DockerClient=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/3/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux 0af07408a0fe 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / e872ceb
Default Java1.8.0_212
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/3/testReport/
Max. process+thread count293 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/3/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec51Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1162trunk passed
+1compile17trunk passed
+1checkstyle17trunk passed
+1mvnsite20trunk passed
+1shadedclient683branch has no errors when building and testing our client artifacts.
+1javadoc19trunk passed
0spotbugs34Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs33trunk passed
_ Patch Compile Tests _
+1mvninstall17the patch passed
+1compile15the patch passed
+1javac15the patch passed
+1checkstyle14the patch passed
+1mvnsite17the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient725patch has no errors when building and testing our client artifacts.
+1javadoc17the patch passed
+1findbugs41the patch passed
_ Other Tests _
+1unit18hadoop-openstack in the patch passed.
+1asflicense25The patch does not generate ASF License warnings.
2983
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/4/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux 783475fff0ff 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / 00b5a27
Default Java1.8.0_212
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/4/testReport/
Max. process+thread count411 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/4/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec41Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1115trunk passed
+1compile19trunk passed
+1checkstyle19trunk passed
+1mvnsite20trunk passed
+1shadedclient664branch has no errors when building and testing our client artifacts.
+1javadoc16trunk passed
0spotbugs30Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs29trunk passed
_ Patch Compile Tests _
+1mvninstall15the patch passed
+1compile13the patch passed
+1javac13the patch passed
+1checkstyle12the patch passed
+1mvnsite16the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient725patch has no errors when building and testing our client artifacts.
+1javadoc17the patch passed
+1findbugs41the patch passed
_ Other Tests _
+1unit16hadoop-openstack in the patch passed.
+1asflicense25The patch does not generate ASF License warnings.
2881
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/5/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux 866f379d1389 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / e356e4f
Default Java1.8.0_222
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/5/testReport/
Max. process+thread count411 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/5/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec88Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1240trunk passed
+1compile17trunk passed
+1checkstyle25trunk passed
+1mvnsite29trunk passed
+1shadedclient835branch has no errors when building and testing our client artifacts.
+1javadoc25trunk passed
0spotbugs42Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs39trunk passed
_ Patch Compile Tests _
+1mvninstall21the patch passed
+1compile18the patch passed
+1javac18the patch passed
+1checkstyle15the patch passed
+1mvnsite19the patch passed
+1whitespace1The patch has no whitespace issues.
+1shadedclient857patch has no errors when building and testing our client artifacts.
+1javadoc23the patch passed
+1findbugs44the patch passed
_ Other Tests _
+1unit19hadoop-openstack in the patch passed.
+1asflicense28The patch does not generate ASF License warnings.
3437
SubsystemReport/Notes
DockerClient=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/6/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux c016ce8d452d 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / 094d736
Default Java1.8.0_222
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/6/testReport/
Max. process+thread count306 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/6/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec43Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1104trunk passed
+1compile18trunk passed
+1checkstyle16trunk passed
+1mvnsite20trunk passed
+1shadedclient707branch has no errors when building and testing our client artifacts.
+1javadoc20trunk passed
0spotbugs33Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs31trunk passed
_ Patch Compile Tests _
+1mvninstall18the patch passed
+1compile17the patch passed
+1javac17the patch passed
+1checkstyle13the patch passed
+1mvnsite16the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient748patch has no errors when building and testing our client artifacts.
+1javadoc16the patch passed
+1findbugs36the patch passed
_ Other Tests _
+1unit15hadoop-openstack in the patch passed.
+1asflicense26The patch does not generate ASF License warnings.
2950
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/7/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux 8ee2ee80d9a9 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / 69ddb36
Default Java1.8.0_222
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/7/testReport/
Max. process+thread count455 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/7/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec81Docker mode activated.
_ Prechecks _
+1dupname0No case conflicting files found.
+1@author0The patch does not contain any @author tags.
-1test4tests0The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1mvninstall1210trunk passed
+1compile17trunk passed
+1checkstyle17trunk passed
+1mvnsite21trunk passed
+1shadedclient784branch has no errors when building and testing our client artifacts.
+1javadoc19trunk passed
0spotbugs33Used deprecated FindBugs config; considering switching to SpotBugs.
+1findbugs30trunk passed
_ Patch Compile Tests _
+1mvninstall17the patch passed
+1compile14the patch passed
+1javac14the patch passed
+1checkstyle12the patch passed
+1mvnsite17the patch passed
+1whitespace0The patch has no whitespace issues.
+1shadedclient855patch has no errors when building and testing our client artifacts.
+1javadoc17the patch passed
+1findbugs46the patch passed
_ Other Tests _
+1unit19hadoop-openstack in the patch passed.
+1asflicense30The patch does not generate ASF License warnings.
3306
SubsystemReport/Notes
DockerClient=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-361/8/artifact/out/Dockerfile
GITHUB PR#361
Optional Testsdupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
unameLinux bde2c58d9502 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/hadoop.sh
git revisiontrunk / 3329257
Default Java1.8.0_222
Test Resultshttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/8/testReport/
Max. process+thread count306 (vs. ulimit of 5500)
modulesC: hadoop-tools/hadoop-openstack U: hadoop-tools/hadoop-openstack
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/8/console
versionsgit=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
0reexec0Docker mode activated.
-1patch11#361 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.
SubsystemReport/Notes
Console outputhttps://builds.apache.org/job/hadoop-multibranch/job/PR-361/9/console
versionsgit=2.17.1
Powered byApache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
This change allows user to register the metrics on the per-container basis.
Tested in beam runner and works as expected.
Author: xiliu <xiliu@xiliu-ld1.linkedin.biz>
Reviewers: Prateek Maheshwari <prateekm@apache.org>
Closesapache#361 from xinyuiscool/SAMZA-1504
@github-actions

Copy link
Copy Markdown
Contributor

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jelmerk@hadoop-yetus