env_docker: support docker-in-docker

[GiedriusS]

If we are running e2e tests inside Docker then it's not so straightforward to access host's network. Let's use the special host.docker.internal for that.

[matej-g]

I was facing the same issue some time ago, but back then the special hostname host.docker.internal was only available for Mac (Docker Desktop).

I see the documentation says this workaround is available on all platforms (https://docs.docker.com/desktop/networking/#use-cases-and-workarounds-for-all-platforms), but I wonder if this applies only to Docker Desktop?

[douglascamata]

Can we refer to the constant from the same file that already holds this address and avoid repeating it?

[saswatamcode]

Thanks! LGTM mod the above comments!

I think for most platforms/setup localhost addrs work for Endpoint(), but likely needs an exception for docker-in-docker!

[GiedriusS]

Mhm, what do you think if we would also double check whether host.docker.internal is routable just before using it to be extra sure? 🤔

[matej-g]

Mhm, what do you think if we would also double check whether host.docker.internal is routable just before using it to be extra sure? 🤔

Mostly I just wonder if this will work on every platform / setup, as it's not clear to me from the current documentation. I only know in the past, the host.docker.internal worked only on MacOS. If that's still true, then I'd rather base this on detecting OS and using the appropriate hostname. However, if it is known for a fact that hostname works on all environments, then we can go with this approach. No need to strictly check the address before using it.

[douglascamata]

host.docker.internal does not work on Linux without some extra parameters being given to the containers being started. In a Linux setup, the address is host-gateway. It is possible to make host.docker.internal work by adding the following CLI args when starting containers: --add-host=host.docker.internal:host-gateway.

@matej-g OS check won't work as Docker-in-Docker will happily return "Linux", but as reported by @GiedriusS, still uses the host.docker.internal trick.

[matej-g]

I'm still missing a straightforward answer if this works on all environments out of the box or not (not sure if what @GiedriusS wrote should be taken as confirmation or assumption - I cannot find documented usage of that hostname anywhere outside of Docker Desktop). Either way, feel free to merge this, worst case scenario we can fix it afterwards and it should not brake existing uses outside of DinD.

[douglascamata]

@matej-g it doesn't work in all environments. It depends on the tools you are using/talking about:

• It works by default on Docker Desktop when the OS is macOS or WSL2.
• It works by default with Docker-in-Docker, as they use --add-host=host.docker.internal:host-gateway. Anyone running containers in this "docker-in-docker" instance (there are now 2 Docker daemons running) has access to host.docker.internal.

Many tools that want to be fully compatible with Docker are using host.docker.internal under the hood, like Rancher Desktop (which I use on a day to day basis).

Podman, for example, uses a different hostname: host.containers.internal. It supports the --add-host mapping too. So using Docker-in-Docker (truly Docker-in-Podman) should work in this case, even with host.docker.internal, as this is done in the inner Docker daemon.

[GiedriusS]

Sorry for the confusion. It seems like something on our Jenkins hosts either adds --add-host or it exists by default. By default on Linux it seems like it is not available:

# docker run -it --rm centos /bin/bash
[root@d5bfd4e71004 /]# ping host.docker.internal
ping: host.docker.internal: Name or service not known

At least on my machine. Hence, maybe let's try to resolve it with net.LookupIP in a best-effort way before using it?

[douglascamata]

I think trying to resolve it is a good idea. It'll be the most stable and failproof way of detecting whether the environment supports it, @GiedriusS.

[GiedriusS]

Added check + re-used the same constant. Thanks for the reviews 🙇

[douglascamata]

LGTM

[matej-g]

Nice addition, thanks you 🙇