How to use ssh-agent from devcontainer CLI ?

[hpe-ykoehler]

I have a devcontainer that I will be using in vscode and in CI.

In the CI we use devcontainer up/exec

in vscode, we see there is a dev container helper that sets up ssh-agent.

In the CI we setup ssh-agent and add the key, but I cannot find a way to have the same devcontainer works in both environment.

I run vscode on windows and remote container are on linux. I can mount my SSH_AUTH_SOCK to /tmp/ssh-agent.socket and then I can start ssh-agent -a /tmp/ssh-agent.socket, but ssh-add -L still report no identities and outside of the container ssh-add -L do report at least one key that I did add to the ssh-agent.

Would you have an example of how one can use devcontainer with docker-compose to access the local ssh-agent, in a compatible way that would also work within vscode itself?

(when I load the same devcontainer in vscode, the SSH_AUTH_SOCK gets transformed into a ".", yet my linux remove is using ssh-agent and has a proper SSH_AUTH_SOCK value). Somehow I think the Windows version is being used, for which no SSH_AUTH_SOCK exists, but I do have the ssh-agent running on Windows.

I would definitely prefer that the remote container uses the remote linux ssh-agent than my windows.

[metaskills]

Curious as well, there seems to be tons of old documentation on how to do this (https://stackoverflow.com/questions/43418188/ssh-agent-forwarding-during-docker-build) but it appears there is more modern (still old) tooling with docker build --ssh too. moby/moby#6396 (comment)

[chrmarti]

The ssh-agent forwarding is part of the Dev Containers extension and not part of the Dev Containers CLI. You could mount the ssh-agent's socket and then point SSH_AUTH_SOCK at it.

[hpe-ykoehler]

Strangely when I did mount the ssh-auth-socket to let's say /tmp/ssh-agent.socket and then set SSH_AUTH_SOCK to /tmp/ssh-agent.socket, I could confirm the mount was done and the env was set, but ssh-add was still unable to talk to the ssh-agent itself.

I then start a ssh-agent within the devcontainer itself telling it to use /tmp/ssh-agent.socket and ssh-add was happy but still not finding any of my key which I had in my how ssh-agent.

The other issue, is that when I did so, within vscode things stop working, because the SSH_AUTH_SOCK got converted to "." instead of the real SSH_AUTH_SOCK value, even thougth the dev conatiners extension is able to find that value propery.

[hpe-ykoehler]

 "runArgs": [
 "-e",
 "SSH_AUTH_SOCK=/tmp/ssh-agent.socket",
 "-v",
 "${env:SSH_AUTH_SOCK}:/tmp/ssh-agent.socket"
 ],

and docker-compose.yml:

 volumes:
 - ..:/workspaces:cached
 - ${SSH_AUTH_SOCK}:/tmp/ssh-agent.socket

If I do that in a docker container it works, but when I do it via the devcontainer cli it doesn't work.

Anyway it would be really nice for devcontainer teams to align and provide an example on how to use ssh-agent in devcontainer CLI in a way compatible with VSCode that is compatible with all OS/remote or not, etc. as it doesn't seem that easy to get working.

[chrmarti]

You only need to run an ssh-agent on your local machine (where the private keys are), not inside the container. Note that "runArgs" doesn't apply to Docker Compose. I can't get it to work on Mac right now, maybe this has changed.

Another option would be to mount ~/.ssh in the container to get access to the private keys.

[hpe-ykoehler]

Christof Marti (@chrmarti) I would prefer the option of ssh-agent because getting access to the key doesn't get you access to the passphrase to unlock the key which then raises another challenge.

It also simplifies cases where the key to use may not be in the .ssh, some users prefer to use a different folder to make it harder to attackers to find those, etc.

when I do not start the ssh-agent within the container, even thought the socket is mounted, ssh-add simply fail to connect to it and report an error.

I am using VSCode on Windows, then connect to Linux via Remote-SSH and use devcontainer.

Using VSCode, the dev containers helper connect my "Windows" ssh-agent to my devcontainer... which in itself is strange, as I would really like my remote linux to be the ssh-agent, as to not share my keys on my windows machine like I have to now. But that is another matter.

And as indicated then using SSH_AUTH_SOCK env variable it breaks VSCode because then SSH_AUTH_SOCK becomes . and the docker-compose complains that

• .:/tmp/ssh-agent.socket

is not a valid volume format.

It somehow look like I would need to start VSCode with SSH_AUTH_SOCK set to my linux SSH_AUTH_SOCK so that VSCode continue to work... which doesn't appear usable.

[chrmarti]

I agree, using the ssh-agent seems preferable.

Using VSCode, the dev containers helper connect my "Windows" ssh-agent to my devcontainer... which in itself is strange, as I would really like my remote linux to be the ssh-agent, as to not share my keys on my windows machine like I have to now. But that is another matter.

That should actually work. If SSH_AUTH_SOCK is properly set in your remote machine's shell startup scripts, the Dev Containers extension should prefer that over the local one. Try running with Dev Container's log level set to 'trace' in the user settings to get the env variables logged as seen by the extension.

And as indicated then using SSH_AUTH_SOCK env variable it breaks VSCode because then SSH_AUTH_SOCK becomes . and the docker-compose complains that

• .:/tmp/ssh-agent.socket

is not a valid volume format.

Docker Compose should do the variable replacement there. Maybe another indicator that SSH_AUTH_SOCK isn't properly picked up on the remote machine.

[hpe-ykoehler]

That should actually work. If SSH_AUTH_SOCK is properly set in your remote machine's shell startup scripts, the Dev Containers extension should prefer that over the local one. Try running with Dev Container's log level set to 'trace' in the user settings to get the env variables logged as seen by the extension.

SSH_AUTH_SOCK is set by my .bashrc but in my linux, it works nicely when I use it with ssh, but vscode still translate to .

One interesting thing is that my ssh-agent in order to use a single socket start ssh-agent, create a symlink to the socket and then set SSH_AUTH_SOCK to the symlink.

if [ ! -S /tmp/yk-ssh_auth_sock ]; then
 eval $(ssh-agent)
 ln -sf "$SSH_AUTH_SOCK" /tmp/yk-ssh_auth_sock
fi
export SSH_AUTH_SOCK=/tmp/yk-ssh_auth_sock

So maybe that could be a reason for this issue. Another could be that .bashrc is only used for some interactive login and I may need to put it in a .profile or .bash_profile, I will try that to see if things change.

But I can assure you that it currently prefer my Windows ssh-agent over my Linux one.

[11009 ms] Start: Launching Dev Containers helper.
[11010 ms] ssh-agent: SSH_AUTH_SOCK in container (/tmp/vscode-ssh-auth-f0cca852-df04-4f31-8740-5fdba9bd0967.sock) forwarded to local host (\\.\pipe\openssh-ssh-agent).
[11010 ms] X11 forwarding: DISPLAY not set on host.

I wish this part was a little more verbose and somehow provide clues as to why my Linux ssh-agent was not found, maybe some kind of warning that the SSH_AUTH_SOCK is not set, the extension fallback to the native host ssh-agent ? With some URL pointing to how to properly set SSH_AUTH_SOCK in a way for Remote-SSH to detect it.

Because the issue my SSH_AUTH_SOCK is set when I used ssh manually, it is also set when I used remote-ssh feature of VSCode, it is only apparently not set when I used the remote-devcontainer plugin feature.

[hpe-ykoehler]

I have removed the symlink by using ssh-agent -a and no change.

if [ ! -S /tmp/yk-ssh_auth_sock ]; then
 eval $(ssh-agent -a /tmp/yk-ssh_auth_sock)
fi

still attach to my windows ssh-agent:

[9765 ms] Start: Launching Dev Containers helper.
[9765 ms] ssh-agent: SSH_AUTH_SOCK in container (/tmp/vscode-ssh-auth-1c2a37c7-9284-4ad4-99a7-7013dc50c28a.sock) forwarded to local host (\\.\pipe\openssh-ssh-agent).
[9766 ms] X11 forwarding: DISPLAY not set on host.

I also added the ssh-agent setup in my .bash_profile and .profile, no impact.

[chrmarti]

Could you run with the Dev Container's log level set to 'trace' in the user settings to get the env variables logged as seen by the extension?

[chrmarti]

I will also improve logging for this. For now we can use the trace logging output.

[hpe-ykoehler]

[18871 ms] Start: Run: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[18955 ms] Stop (84 ms): Run: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[18956 ms] Stop (85 ms): Inspecting container
[18959 ms] Start: Run in container: /bin/sh
[18973 ms] Start: Run in container: uname -m
[19145 ms] x86_64
[19145 ms] 
[19145 ms] Stop (172 ms): Run in container: uname -m
[19145 ms] Start: Run in container: (cat /etc/os-release || cat /usr/lib/os-release) 2>/dev/null
[19153 ms] NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
[19153 ms] 
[19153 ms] Stop (8 ms): Run in container: (cat /etc/os-release || cat /usr/lib/os-release) 2>/dev/null
[19154 ms] Start: Run in container: cat /etc/passwd
[19157 ms] Stop (3 ms): Run in container: cat /etc/passwd
[19159 ms] Start: Run in container: test -f '/var/devcontainer/.patchEtcEnvironmentMarker'
[19161 ms] 
[19162 ms] 
[19162 ms] Exit code 1
[19162 ms] Stop (3 ms): Run in container: test -f '/var/devcontainer/.patchEtcEnvironmentMarker'
[19163 ms] Start: Run in container: /bin/sh
[19176 ms] Start: Run in container: test ! -f '/var/devcontainer/.patchEtcEnvironmentMarker' && set -o noclobber && mkdir -p '/var/devcontainer' && { > '/var/devcontainer/.patchEtcEnvironmentMarker' ; } 2> /dev/null
[19357 ms] 
[19357 ms] 
[19357 ms] Stop (181 ms): Run in container: test ! -f '/var/devcontainer/.patchEtcEnvironmentMarker' && set -o noclobber && mkdir -p '/var/devcontainer' && { > '/var/devcontainer/.patchEtcEnvironmentMarker' ; } 2> /dev/null
[19357 ms] Start: Run in container: cat >> /etc/environment <<'etcEnvrionmentEOF'
[19365 ms] 
[19365 ms] 
[19365 ms] Stop (8 ms): Run in container: cat >> /etc/environment <<'etcEnvrionmentEOF'
[19366 ms] Start: Run in container: test -f '/var/devcontainer/.patchEtcProfileMarker'
[19368 ms] 
[19368 ms] 
[19368 ms] Exit code 1
[19368 ms] Stop (2 ms): Run in container: test -f '/var/devcontainer/.patchEtcProfileMarker'
[19369 ms] Start: Run in container: test ! -f '/var/devcontainer/.patchEtcProfileMarker' && set -o noclobber && mkdir -p '/var/devcontainer' && { > '/var/devcontainer/.patchEtcProfileMarker' ; } 2> /dev/null
[19375 ms] 
[19375 ms] 
[19375 ms] Stop (6 ms): Run in container: test ! -f '/var/devcontainer/.patchEtcProfileMarker' && set -o noclobber && mkdir -p '/var/devcontainer' && { > '/var/devcontainer/.patchEtcProfileMarker' ; } 2> /dev/null
[19376 ms] Start: Run in container: sed -i -E 's/((^|\s)PATH=)([^\$]*)$/\1${PATH:-\3}/g' /etc/profile || true
[19387 ms] 
[19387 ms] 
[19387 ms] Stop (11 ms): Run in container: sed -i -E 's/((^|\s)PATH=)([^\$]*)$/\1${PATH:-\3}/g' /etc/profile || true
[19389 ms] Stop (14350 ms): Resolving Remote
[19422 ms] Stop (15424 ms): Run in Host: /net/spindles/ykoehler/.vscode-remote-containers/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884/node /net/spindles/ykoehler/.vscode-remote-containers/dist/dev-containers-cli-0.286.0/dist/spec-node/devContainersSpecCLI.js up --container-session-data-folder /tmp/devcontainers-933f80f4-6726-4784-aa66-128ff75f113e1679597210270 --workspace-folder /work/krypton/dal --workspace-mount-consistency cached --id-label devcontainer.local_folder=/work/krypton/dal --id-label devcontainer.config_file=/work/krypton/dal/.devcontainer/devcontainer.json --log-level trace --log-format json --config /work/krypton/dal/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true --terminal-columns 298 --terminal-rows 37
[19425 ms] Start: Run in Host: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[19630 ms] Stop (205 ms): Run in Host: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[19631 ms] Start: Run in Host: docker exec -i -u root 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 /bin/sh -c echo "New container started. Keep-alive process started." ; export VSCODE_REMOTE_CONTAINERS_SESSION=933f80f4-6726-4784-aa66-128ff75f113e1679597210270 ; /bin/sh
[19836 ms] Start: Run in Host: /net/spindles/ykoehler/.vscode-remote-containers/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884/node /net/spindles/ykoehler/.vscode-remote-containers/dist/dev-containers-cli-0.286.0/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /work/krypton/dal --id-label devcontainer.local_folder=/work/krypton/dal --id-label devcontainer.config_file=/work/krypton/dal/.devcontainer/devcontainer.json --container-id 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 --log-level trace --log-format json --config /work/krypton/dal/.devcontainer/devcontainer.json --mount-workspace-git-root true
[19899 ms] New container started. Keep-alive process started.
[20376 ms] @devcontainers/cli 0.35.0. Node.js v16.14.2. linux 5.15.0-53-generic x64.
[20375 ms] Start: Run: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[20463 ms] Stop (88 ms): Run: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[20496 ms] Stop (660 ms): Run in Host: /net/spindles/ykoehler/.vscode-remote-containers/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884/node /net/spindles/ykoehler/.vscode-remote-containers/dist/dev-containers-cli-0.286.0/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /work/krypton/dal --id-label devcontainer.local_folder=/work/krypton/dal --id-label devcontainer.config_file=/work/krypton/dal/.devcontainer/devcontainer.json --container-id 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 --log-level trace --log-format json --config /work/krypton/dal/.devcontainer/devcontainer.json --mount-workspace-git-root true
[20499 ms] Start: Run in Host: docker-compose version --short
[21049 ms] Stop (550 ms): Run in Host: docker-compose version --short
[21448 ms] Start: Run in Host: /net/spindles/ykoehler/.vscode-remote-containers/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884/node /net/spindles/ykoehler/.vscode-remote-containers/dist/dev-containers-cli-0.286.0/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /work/krypton/dal --id-label devcontainer.local_folder=/work/krypton/dal --id-label devcontainer.config_file=/work/krypton/dal/.devcontainer/devcontainer.json --container-id 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 --log-level trace --log-format json --config /work/krypton/dal/.devcontainer/devcontainer.json --include-merged-configuration --mount-workspace-git-root true
[21893 ms] @devcontainers/cli 0.35.0. Node.js v16.14.2. linux 5.15.0-53-generic x64.
[21893 ms] Start: Run: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[21979 ms] Stop (86 ms): Run: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[22013 ms] Stop (565 ms): Run in Host: /net/spindles/ykoehler/.vscode-remote-containers/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884/node /net/spindles/ykoehler/.vscode-remote-containers/dist/dev-containers-cli-0.286.0/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /work/krypton/dal --id-label devcontainer.local_folder=/work/krypton/dal --id-label devcontainer.config_file=/work/krypton/dal/.devcontainer/devcontainer.json --container-id 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 --log-level trace --log-format json --config /work/krypton/dal/.devcontainer/devcontainer.json --include-merged-configuration --mount-workspace-git-root true
[22015 ms] Start: Inspecting container
[22016 ms] Start: Run in Host: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[22213 ms] Stop (197 ms): Run in Host: docker inspect --type container 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7
[22214 ms] Stop (199 ms): Inspecting container
[22222 ms] Start: Run in Host: docker exec -i -u aos-dev -e VSCODE_REMOTE_CONTAINERS_SESSION=933f80f4-6726-4784-aa66-128ff75f113e1679597210270 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 /bin/sh
[22326 ms] Start: Run in container: uname -m
[22509 ms] x86_64
[22510 ms] 
[22510 ms] Stop (184 ms): Run in container: uname -m
[22510 ms] Start: Run in container: (cat /etc/os-release || cat /usr/lib/os-release) 2>/dev/null
[22609 ms] NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
[22610 ms] 
[22610 ms] Stop (100 ms): Run in container: (cat /etc/os-release || cat /usr/lib/os-release) 2>/dev/null
[22611 ms] Start: Run in container: cat /etc/passwd
[22708 ms] Stop (97 ms): Run in container: cat /etc/passwd
[22709 ms] Start: Updating configuration state
[22909 ms] Stop (200 ms): Updating configuration state
[22910 ms] Start: Setup shutdown monitor
[22912 ms] Forking shutdown monitor: c:\Users\koehlery\.vscode\extensions\ms-vscode-remote.remote-containers-0.286.0\dist\shutdown\shutdownMonitorProcess \\.\pipe\vscode-remote-containers-188be55c-b66a-4687-984e-451d68846e1d-sock dockerCompose Trace c:\Users\koehlery\AppData\Roaming\Code\logs\20230323T124001\window1\exthost\ms-vscode-remote.remote-containers 1679597211250
[22928 ms] Stop (18 ms): Setup shutdown monitor
[22930 ms] Start: Run in container: test -d /home/aos-dev/.vscode-server
[23026 ms] 
[23027 ms] 
[23027 ms] Exit code 1
[23027 ms] Stop (97 ms): Run in container: test -d /home/aos-dev/.vscode-server
[23028 ms] Start: Run in container: test -d /home/aos-dev/.vscode-remote
[23127 ms] 
[23127 ms] 
[23128 ms] Exit code 1
[23128 ms] Stop (100 ms): Run in container: test -d /home/aos-dev/.vscode-remote
[23129 ms] Start: Run in container: test ! -f '/home/aos-dev/.vscode-server/data/Machine/.writeMachineSettingsMarker' && set -o noclobber && mkdir -p '/home/aos-dev/.vscode-server/data/Machine' && { > '/home/aos-dev/.vscode-server/data/Machine/.writeMachineSettingsMarker' ; } 2> /dev/null
[23156 ms] Starting monitor process...
[23180 ms] Received message: [{"type":"configure","sequence":0,"options":{"cwd":"/work/krypton/dal","dockerCLI":"docker","dockerComposeCLI":{"version":"1.29.2","cmd":"docker-compose","args":[]},"env":{"USER":"ykoehler","SSH_CLIENT":"15.111.190.46 59187 22","XDG_SESSION_TYPE":"tty","SHLVL":"0","MOTD_SHOWN":"pam","HOME":"/net/spindles/ykoehler","DBUS_SESSION_BUS_ADDRESS":"unix:path=/run/user/11846/bus","LOGNAME":"ykoehler","REMOTE_CONTAINERS_IPC":"","_":"dc1b38b5-793b-473c-9beb-1443b3852f14","XDG_SESSION_CLASS":"user","XDG_SESSION_ID":"24863","PATH":"/net/spindles/ykoehler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin","XDG_RUNTIME_DIR":"/run/user/11846","REMOTE_CONTAINERS_SOCKETS":"[]","LANG":"en_US.UTF-8","SHELL":"/bin/bash","SSH_CONNECTION":"15.111.190.46 59187 10.82.21.252 22","XDG_DATA_DIRS":"/usr/share/gnome:/usr/local/share/:/usr/share/","DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"c:\\Users\\koehlery\\.vscode\\extensions\\ms-vscode-remote.remote-containers-0.286.0\\scripts\\ssh-askpass.bat","SSH_ASKPASS_REQUIRE":"force","VSCODE_SSH_ASKPASS_NODE":"C:\\Users\\koehlery\\AppData\\Local\\Programs\\Microsoft VS Code\\Code.exe","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"--ms-enable-electron-run-as-node","VSCODE_SSH_ASKPASS_MAIN":"c:\\Users\\koehlery\\.vscode\\extensions\\ms-vscode-remote.remote-containers-0.286.0\\dist\\common\\sshAskpass.js","VSCODE_SSH_ASKPASS_HANDLE":"\\\\.\\pipe\\ssh-askpass-7e8dfe39-ed35-48e1-be65-950df360332b-sock","DOCKER_CONTEXT":"default","VSCODE_DOCKER_HOST":"ssh://slrubdev-ykoehler22"},"containerId":"5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7","user":"aos-dev","sessionId":"933f80f4-6726-4784-aa66-128ff75f113e1679597210270","shutdownAction":"stopCompose","projectName":"dal_devcontainer","composeFiles":["/work/krypton/dal/.devcontainer/docker-compose.yml"],"service":"krypton-c-env","logLevel":1,"delay":10000}}]
[23236 ms] 
[23236 ms] 
[23236 ms] Stop (107 ms): Run in container: test ! -f '/home/aos-dev/.vscode-server/data/Machine/.writeMachineSettingsMarker' && set -o noclobber && mkdir -p '/home/aos-dev/.vscode-server/data/Machine' && { > '/home/aos-dev/.vscode-server/data/Machine/.writeMachineSettingsMarker' ; } 2> /dev/null
[23237 ms] Start: Run in container: cat /home/aos-dev/.vscode-server/data/Machine/settings.json
[23343 ms] 
[23344 ms] cat: /home/aos-dev/.vscode-server/data/Machine/settings.json: No such file or directory
[23344 ms] Exit code 1
[23344 ms] Stop (107 ms): Run in container: cat /home/aos-dev/.vscode-server/data/Machine/settings.json
[23345 ms] Start: Run in container: test -d /home/aos-dev/.vscode-server/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884
[23453 ms] 
[23453 ms] 
[23453 ms] Exit code 1
[23453 ms] Stop (108 ms): Run in container: test -d /home/aos-dev/.vscode-server/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884
[23453 ms] Start: Run in container: test -d /vscode/vscode-server/bin/linux-x64/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884
[23554 ms] 
[23554 ms] 
[23554 ms] Stop (101 ms): Run in container: test -d /vscode/vscode-server/bin/linux-x64/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884
[23554 ms] Start: Run in container: mkdir -p '/home/aos-dev/.vscode-server/bin' && ln -snf '/vscode/vscode-server/bin/linux-x64/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884' '/home/aos-dev/.vscode-server/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884'
[23661 ms] 
[23662 ms] 
[23662 ms] Stop (108 ms): Run in container: mkdir -p '/home/aos-dev/.vscode-server/bin' && ln -snf '/vscode/vscode-server/bin/linux-x64/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884' '/home/aos-dev/.vscode-server/bin/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884'
[23663 ms] Start: Run in Host: docker exec -i -u root 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 /bin/sh
[23664 ms] Start: Launching Dev Containers helper.
[23665 ms] ssh-agent: SSH_AUTH_SOCK in container (/tmp/vscode-ssh-auth-542f4882-bd97-4ec9-84e9-5890f4f5d07b.sock) forwarded to local host (\\.\pipe\openssh-ssh-agent).
[23666 ms] X11 forwarding: DISPLAY not set on host.
[23666 ms] Start: Run in Host: gpgconf --list-dir agent-extra-socket
[23776 ms] Start: Run in container: touch '/vscode/vscode-server/bin/linux-x64/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884'
[23879 ms] Stop (213 ms): Run in Host: gpgconf --list-dir agent-extra-socket
[23880 ms] /run/user/11846/gnupg/S.gpg-agent.extra
[23881 ms] 
[23881 ms] Start: Run in container: gpgconf --list-dir agent-socket
[23942 ms] 
[23942 ms] 
[23942 ms] Stop (166 ms): Run in container: touch '/vscode/vscode-server/bin/linux-x64/ee2b180d582a7f601fa6ecfdad8d9fd269ab1884'
[23993 ms] /home/aos-dev/.gnupg/S.gpg-agent
[23993 ms] 
[23993 ms] Stop (112 ms): Run in container: gpgconf --list-dir agent-socket
[23993 ms] Start: Run in container: gpgconf --list-dir homedir
[24101 ms] /home/aos-dev/.gnupg
[24102 ms] 
[24102 ms] Stop (109 ms): Run in container: gpgconf --list-dir homedir
[24102 ms] Start: Run in container: ls '/home/aos-dev/.gnupg/private-keys-v1.d' 2>/dev/null
[24204 ms] 
[24205 ms] 
[24205 ms] Exit code 2
[24205 ms] Stop (103 ms): Run in container: ls '/home/aos-dev/.gnupg/private-keys-v1.d' 2>/dev/null
[24205 ms] Start: Run in container: mkdir -p -m 700 '/home/aos-dev/.gnupg'
[24302 ms] 
[24302 ms] 
[24302 ms] Stop (97 ms): Run in container: mkdir -p -m 700 '/home/aos-dev/.gnupg'
[24302 ms] Start: Run in container: command -v docker >/dev/null 2>&1
[24302 ms] Start: Run in Host: gpgconf --list-dir homedir
[24403 ms] 
[24403 ms] 
[24403 ms] Stop (101 ms): Run in container: command -v docker >/dev/null 2>&1
[24404 ms] Start: Run in Host: docker exec -i -u aos-dev 5bbf06ee9b61785eb5ccd1e58e556086a53a497e9054160c6248d01a9906f1d7 /bin/sh
[24404 ms] Stop (740 ms): Launching Dev Containers helper.
[24404 ms] userEnvProbe: loginInteractiveShell (default)
[24405 ms] Start: Run in container: test -f '/tmp/devcontainers-933f80f4-6726-4784-aa66-128ff75f113e1679597210270/env-loginInteractiveShell.json'
[24512 ms] Stop (210 ms): Run in Host: gpgconf --list-dir homedir
[24514 ms] /net/spindles/ykoehler/.gnupg
[24514 ms] 
[24515 ms] 
[24516 ms] 
[24516 ms] Exit code 1
[24516 ms] Stop (111 ms): Run in container: test -f '/tmp/devcontainers-933f80f4-6726-4784-aa66-128ff75f113e1679597210270/env-loginInteractiveShell.json'
[24517 ms] Start: Run in container: echo ~
[24517 ms] Start: Run in container: gpgconf --list-dir homedir
[24517 ms] userEnvProbe: not found in cache
[24518 ms] userEnvProbe shell: /bin/bash
[24614 ms] /home/aos-dev/.gnupg
[24615 ms] 
[24615 ms] Stop (98 ms): Run in container: gpgconf --list-dir homedir
[24615 ms] Start: Run in container: # Test for /home/aos-dev/.ssh/known_hosts and ssh
[24702 ms] /home/aos-dev
[24703 ms] 
[24704 ms] Stop (187 ms): Run in container: echo ~
[24704 ms] Start: Run in container: cat <<'EOF-/tmp/vscode-remote-containers-542f4882-bd97-4ec9-84e9-5890f4f5d07b.js' >/tmp/vscode-remote-containers-542f4882-bd97-4ec9-84e9-5890f4f5d07b.js
[24726 ms] 
[24727 ms] 

In the area around ssh-agent, there apparently are no trace logs.