Fast context switch: commands

[simonferquel]

Depends on #1500
This brings a context subcommands to manage contexts stored in the context store, as well as modifying docker info / docker version to show current context

[silvin-lubecki]

I wonder if we want the context here in the version command or in the info ? 🤔
@thaJeztah any thought?

[vdemeester]

• Really not fan of the dockerEndpointOptions and kubernetesEndpointOptions struct…
• Not sure for the command either : docker context set-docker-endpoint … vs docker context set docker-endpoint …, same with other set-* options… we may want to have more subcommands… This would allow docker context set defaultStackOrchestrator commands (targeting current context if no context is provided via a flag 👼 )

[silvin-lubecki]

Linter complains

cli/command/context/setoptions.go:44::warning: line is 213 characters (lll)

[silvin-lubecki]

One test is failing:

--- FAIL: TestVersionAlign (0.00s)
version_test.go:112: assertion failed: --- expected
+++ actual
@@ -1,4 +1,4 @@
Client:
-·Context:···········test-context
+·Context:·test-context
·Version:···········18.99.5-ce
·API·version:·······1.38

[codecov-io]

Codecov Report

Merging #1501 into master will decrease coverage by 0.15%.
The diff coverage is 51.51%.

@@ Coverage Diff @@## master #1501 +/- ##
==========================================
- Coverage 55.32% 55.16% -0.16% 
==========================================
Files 283 301 +18 Lines 19330 20359 +1029 ==========================================
+ Hits 10694 11232 +538 - Misses 7939 8323 +384 - Partials 697 804 +107

[andrewhsu]

@simonferquel when you get the chance, can you update the description with any user interaction changes? just a simple example of what is the new command or option or output from docker version or docker info or whatnot would help describe to the end user what this PR will bring it because there's a lot of code changes in this one.

[andrewhsu]

i talked with @simonferquel@silvin-lubecki@thaJeztah about this and opened up an issue to describe the bigger idea and reasoning and track outstanding issues #1586

[thaJeztah]

Just noticed this one: I don't think we should do the automatic fallback, but instead make it a hard fail. Falling back can be dangerous, for example:

docker --context=develop container rm -f somecontainer
WARNING: current context "develop" is not found on filesystem. Falling back to default Docker endpoint

And now you removed the container from "whatever happened to be in DOCKER_HOST

[simonferquel]

Erf, you are right, but this one will be tricky: docker context use implicitly requires an initialized dockerCli.
As this error can occur only if the user messes directly with the store dir, is it ok if I do a hard fail, indicating that he should fix his configfile manually?

[thaJeztah]

Trying to grasp your comment;

All the docker context commands are local operations, so (technically) should not need to care about the current context. i.e, if the "current" context uses unix:///no-such-thing), it should not be an error until I try to connect to dockerd or the k8s API, so docker context use <a valid context> should work (as it doesn't require a context to perform that action).

[simonferquel]

For the not-a-context context name, I suggest that for printing, we could re-introduce the <DOCKER_HOST> name (which is now invalid as a regular context name).

Also thinking that we need a way to switch back to the default/docker_host/unset situation (not a big fan of <DOCKER_HOST> as a name; it's very "shouty"); I'd be ok with a reserved name though, perhaps "default", "none" or "unset" (😅 I know) ? That way, I could switch between context X and back to the default situation.

I will go for default, as I don't like the idea of docker context ls printing a context named none.

For the import from kubeconfig, the problem is that you can't create a context without configuring a docker endpoint

Ah, right, gotcha

Maybe should I remove the --kubeconfig flag and make that feature into a separate subcommand?

So, if we can't import the exported --kubeconfig, what is the exact use-case? If the "regular" export contains the full configuration, is there a need to export just a subset of that information in a format that cannot be imported by the docker cli?

The idea is to be able to use it with other tools like kubectl etc.

For the ID, I really don't want to introduce an index of names to id, and this SHA based stuff seems good to me. However, I think that I should not call this an ID, because it is actually a SHA of the real identifier which is the name (and thus, it is confusing)

Right, so I'm wondering what we gained by introducing it (also see #1501 (comment)), as the original reason for it was to get a) consistency (have an ID as unique identifier and b) possibly allow name to be optional and/or allow renaming.

The only thing it brought is a "safe" name for the path (no chance on path-traversing and such), although we still use the name as default for export

What about naming it context_dir, and in the inspect function, I would report a metadata_dir and tls_dir with the path to the related storage ?

Perhaps xx_path (or CamelCase ..Path), as that's used elsewhere (see docker container inspect)

Are metadata and tls always relative to that path? If so, perhaps there's no need to include them separately

ok for _path suffixed fields

[silvin-lubecki]

nit: imports reordering

[thaJeztah]

Thanks! Getting close; left some comments inline 🤗

[thaJeztah]

Trying to grasp your comment;

All the docker context commands are local operations, so (technically) should not need to care about the current context. i.e, if the "current" context uses unix:///no-such-thing), it should not be an error until I try to connect to dockerd or the k8s API, so docker context use <a valid context> should work (as it doesn't require a context to perform that action).

[thaJeztah]

Why was this needed?

edit: ah, looks like you updated in a follow-up commit

[simonferquel]

yep it is not needed anymore (and removed)

[thaJeztah]

Looks like this one is in the wrong file (should probably be in inspect_test.go)

[thaJeztah]

nit (for a follow-up); for other commands we use "" as default, and in the code use something like;

	format := opts.format
if len(format) == 0 {
format = formatter.TableFormatKey
}

e.g.

cli/cli/command/system/df.go

Lines 45 to 48 in a900ba8

	format:=opts.format
	iflen(format) ==0 {
	format=formatter.TableFormatKey
	}

We can do a follow-up to decide which approach to take for all (either way makes some sense)

[simonferquel]

ok for a followup

[thaJeztah]

I noticed the default context still disappears after selecting a context. Can we make the default context always visible in docker context ls?

docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm
foobar unix:///var/run/docker.sock docker context use foobar
foobar
Current context is now "foobar"
docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
foobar * unix:///var/run/docker.sock docker context use default
default
Current context is now "default"
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm
foobar unix:///var/run/docker.sock 

[thaJeztah]

nit (ok for a follow-up): we should probably handle this with for other situations as well. Perhaps when using inspect; return the information that we obtained from the default (DOCKER_HOST). Currently;

docker context inspect default
[]
context "default" does not exist
docker context rm default
default: context "default" does not exist

[simonferquel]

Not sure about that: for now, when a context is set and I list the contexts, the default context appears with no docker/kubernetes endpoint. Not sure if Inspect makes sense.

Maybe indicate that the default context cannot be inspected ?

[thaJeztah]

Yes, I think it's ok to produce a better /helpful error for now (then we can revisit in future)

[simonferquel]

It is in fact part of the latest commit :)

[thaJeztah]

Also for a follow-up; use an interface for these (like the errdefs package (or use that package); https://github.com/moby/moby/blob/d48392a35b157612d28c48eb8e4fad1272fa1442/errdefs/defs.go#L3-L6

// ErrNotFound signals that the requested object doesn't existtypeErrNotFoundinterface {
NotFound()
}

https://github.com/moby/moby/blob/d48392a35b157612d28c48eb8e4fad1272fa1442/errdefs/is.go#L32-L36

// IsNotFound returns if the passed in error is an ErrNotFoundfuncIsNotFound(errerror) bool {
_, ok:=getImplementer(err).(ErrNotFound)
returnok
}

https://github.com/moby/moby/blob/d48392a35b157612d28c48eb8e4fad1272fa1442/errdefs/helpers.go#L13-L19

// NotFound is a helper to create an error of the class with the same name from any error typefuncNotFound(errerror) error {
iferr==nil {
returnnil
}
returnerrNotFound{err}
}

It would make these errors consistent with other handling, and also preserve the original error in some cases where it may be relevant.

[simonferquel]

All the docker context commands are local operations, so (technically) should not need to care about the current context. i.e, if the "current" context uses unix:///no-such-thing), it should not be an error until I try to connect to dockerd or the k8s API, so docker context use should work (as it doesn't require a context to perform that action).

We don't need to do any request, but the initialization of the CLI still occur. And thus it parses the Host to create the WithHost option (without issuing request). so unix:///some-not-existing-file work (it is valid syntactically), but unix///malformed (note the missing :) fails with a parsing error

[thaJeztah]

Arf, yes.

We'll have to think of a solution for that (e.g., if we can delay evaluating that option until it's actually needed 🤔 )

[simonferquel]

Actually, I now validate the Host on create/update so it should not be a problem anymore. Only case this error can happen, is if the user delete its store content manually but keep the currentContext set in its config file

[simonferquel]

ping @thaJeztah, can you PTAL the last commit ?

[thaJeztah]

The idea is to be able to use it with other tools like kubectl etc.

If I do a full docker context export, can I find the kubernetes configuration in the tar?
Still some questions about that, but ok to discuss after this one

Giving this another spin

docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm

docker context ls --format '{{json .}}'| jq .
{
"Current": true,
"Description": "Current DOCKER_HOST based configuration",
"DockerEndpoint": "unix:///var/run/docker.sock",
"KubernetesEndpoint": "",
"Name": "default",
"StackOrchestrator": "swarm"
}

Formatting placeholders work as expected 👍 :

docker context ls --format '{{json .Name}}'"default"

Validation works

docker context create --docker host=unix:/bla local
unable to create docker endpoint config: unable to apply docker endpoint options: unable to parse docker host `unix:/bla`
docker context create --docker host=unix:///var/run/docker.sock default
"default" is a reserved context name
docker context create --docker host=unix:///var/run/docker.sock locallocal
Successfully created context "local"
docker context create --docker host=unix:///var/run/docker.sock local
context "local" already exists

We could even consider showing the same error message when trying to create a context named default;

context "local" already exists

docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm
local unix:///var/run/docker.sock docker context use locallocal
Current context is now "local"
docker context ls
NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR
default Current DOCKER_HOST based configuration local* unix:///var/run/docker.sock docker context use default
default
Current context is now "default"

🎉 the default context is still there: and I can switch back- and forth awesome!

Inspecting;

docker context inspect local
[
{
"Name": "local",
"Metadata": {},
"Endpoints": {
"docker": {
"Host": "unix:///var/run/docker.sock",
"SkipTLSVerify": false
}
},
"TLSMaterial": {},
"Storage": {
"MetadataPath": "/root/.docker/contexts/meta/25bf8e1a2393f1108d37029b3df5593236c755742ec93465bbafa9b290bddcf6",
"TLSPath": "/root/.docker/contexts/tls/25bf8e1a2393f1108d37029b3df5593236c755742ec93465bbafa9b290bddcf6"
}
}
]

And custom format;

docker context inspect local --format '{{ .Storage.MetadataPath }}'
/root/.docker/contexts/meta/25bf8e1a2393f1108d37029b3df5593236c755742ec93465bbafa9b290bddcf6

"Pretty print" is still missing (but not a blocker for me; it's a "nice to have")

docker context inspect --format=pretty local
pretty

Exporting:

docker context export default
"default" is a reserved context name
docker context exportlocal -
cowardly refusing to export to a terminal, please specify a file path
docker context exportlocal - > ./bla.tar
docker context exportlocal
Written file "local.dockercontext"
docker context exportlocal
open local.dockercontext: file exists

Importing:

docker context import
"docker context import" requires exactly 2 arguments
docker context import bla ./bla.tar
bla
Successfully imported context "bla"
cat bla.tar | docker context import piped -
piped
Successfully imported context "piped"
docker context import imported-context ./local.dockercontext imported-context
Successfully imported context "imported-context"

(we could mention from file ... / from stdin as a future enhancement)

docker context import imported-context ./local.dockercontext context "imported-context" already exists
docker context import default ./local.dockercontext "default" is a reserved context name

Handling non existing / non-reachable contexts

docker context create nosuchhost --docker host=http://nosuchhost.nosuchdomain
nosuchhost
Successfully created context "nosuchhost"
docker context use nosuchhost
docker ps
error during connect: Get http://nosuchhost.nosuchdomain/v1.40/containers/json: dial tcp: lookup nosuchhost.nosuchdomain on 192.168.65.1:53: no such host
docker ps --help
Usage:	docker ps [OPTIONS]
...

Looks good!

[thaJeztah]

LGTM

Got some code nits / suggestions, but nothing blocking for now

can you cleanup the commits a bit?

[simonferquel]

The full export does not contain a kubeconfig file, no.
I'll squash the commits then, thanks a lot!

[thaJeztah]

The full export does not contain a kubeconfig file, no.

That's something we should think about doing, so that we can export / backup

[thaJeztah]

@silvin-lubecki@vdemeester still LGTY?

[vdemeester]

still LGTM 😉

[simonferquel]

Actually the full export contains everything we need to construct a kubeconfig file, but the layout is different (partly because we thought it would be nice to make it mandatory to put TLS material in a separate dir)