This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
This repository was archived by the owner on Oct 13, 2023. It is now read-only.

[18.09 backport] Fix docker cp when container source path is / - #286

Merged
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix
Jun 20, 2019
Merged

[18.09 backport] Fix docker cp when container source path is /#286
andrewhsu merged 5 commits into
docker-archive:18.09from
thaJeztah:18.09_backport_cp_slash_fix

Conversation

@thaJeztah

@thaJeztahthaJeztah commented Jun 18, 2019

Copy link
Copy Markdown
Member

backport of moby#39357 for 18.09

Another attempt at fixing moby#39348
Fixesmoby#39348
Previous attempt at moby#39351

Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar dir/file it would tar from dir the file file
(as part of the IncludedFiles option).

However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar dir1/dir2 it would tar from dir1 the directory
dir2 (as part of IncludedFiles option).

Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a docker cp with container source set
to / (cf 3029e76 (moby#39292)).

The fix is to only do the path splitting logic if the source is a file.

Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.

This fix does not do that. Instead, it fixes the issue inline.

Signed-off-by: Tibor Vass tibor@docker.com

I added a couple of more tests than the actual issue needs, just to make sure there are no other regressions compared to before the cve fix (3029e76).

Huge thanks to @cpuguy83 ❤️who worked tirelessly with me to understand the code and make this PR.

cpuguy83and others added 4 commits June 18, 2019 14:43
CID=$(docker create alpine)
docker cp $CID:/ out
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 6db9f1c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 02f1eb8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before 7a7357d, archive.TarResourceRebase was being used to copy files
and folders from the container. That function splits the source path
into a dirname + basename pair to support copying a file:
if you wanted to tar `dir/file` it would tar from `dir` the file `file`
(as part of the IncludedFiles option).
However, that path splitting logic was kept for folders as well, which
resulted in weird inputs to archive.TarWithOptions:
if you wanted to tar `dir1/dir2` it would tar from `dir1` the directory
`dir2` (as part of IncludedFiles option).
Although it was weird, it worked fine until we started chrooting into
the container rootfs when doing a `docker cp` with container source set
to `/` (cf 3029e76).
The fix is to only do the path splitting logic if the source is a file.
Unfortunately, 7a7357d added support for LCOW by duplicating some of
this subtle logic. Ideally we would need to do more refactoring of the
archive codebase to properly encapsulate these behaviors behind well-
documented APIs.
This fix does not do that. Instead, it fixes the issue inline.
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 171538c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, getWalkRoot("/", "foo") would return "//foo"
Now it returns "/foo"
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7410f1a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztahthaJeztah added this to the 18.09.7 milestone Jun 18, 2019
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ping @tiborvass@kolyshkin@andrewhsu PTAL

@cpuguy83

Copy link
Copy Markdown

integration/container/copy_test.go:108:26:warning: cannot use ctx (variable of type context.Context) as *testing.T value in argument to container.Create (gosimple)

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

ah, booh

For reference on why this is needed:
docker-archive#280 (comment)
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 8f4b96f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
MemberAuthor

cherry-picked 8f4b96f

@thaJeztah

Copy link
Copy Markdown
MemberAuthor

only failures are DockerSuite.TestRunInteractiveWithRestartPolicy on experimental https://jenkins.dockerproject.org/job/Docker-PRs-experimental/45728/console

tracked through moby#39352

17:30:53 FAIL: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy
17:30:53 17:30:53 assertion failed: 17:30:53 Command: /usr/local/cli/docker run -i --name test-inter-restart --restart=always busybox sh
17:30:53 ExitCode: 0
17:30:53 Error: <nil>
17:30:53 Stdout: 17:30:53 Stderr: 17:30:53 17:30:53 Failures:
17:30:53 ExitCode was 0 expected 11

and on Janky https://jenkins.dockerproject.org/job/Docker-PRs/54598/console

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode
17:44:53 17:44:53 assertion failed: expected an error, got nil

@andrewhsu

Copy link
Copy Markdown

@kolyshkin

kolyshkin commented Jun 19, 2019

Copy link
Copy Markdown

kicked the janky ci as I haven't seen this earlier:

17:44:53 FAIL: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode

and it looks like another manifestation of that elusive flakiness we have it TestRunInteractiveWithRestartPolicy

@andrewhsu

Copy link
Copy Markdown

Looks like latest job run has success:

...
PASS: docker_cli_run_test.go:1792: DockerSuite.TestRunInteractiveWithRestartPolicy	11.104s
...
PASS: docker_cli_start_test.go:190: DockerSuite.TestStartReturnCorrectExitCode	2.289s
...

@andrewhsuandrewhsu left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewhsu
andrewhsu merged commit c513a4c into docker-archive:18.09Jun 20, 2019
@thaJeztah
thaJeztah deleted the 18.09_backport_cp_slash_fix branch June 20, 2019 06:51
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@thaJeztah@cpuguy83@andrewhsu@kolyshkin