Debug-Container

This container can be thought of as the administrator's shell. Many of the debugging tools (such as ping, traceroute, and mtr) and man pages that an administrator might use to diagnose problems on the host are in this container.

Container Image Variants

Variant	Base Image	Tag	Support
CentOS Stream 9	quay.io/centos/centos:stream9	master / latest	Community
Azure Linux 3.0 (Microsoft)	mcr.microsoft.com/azurelinux/base/core:3.0	microsoft	Microsoft

Recommended: Use the microsoft variant (Dockerfile-microsoft) for production workloads requiring Microsoft support.

Included Tools

• Networking:
  • iproute
  • net-tools
  • mtr
  • dig (bind-utils)
  • ping (iputils)
  • ethtool
  • nmap / nmap-ncat
  • tcpdump
  • iperf3
  • curl / wget
• Monitoring:
  • htop
  • sysstat
  • numactl
  • procps-ng
• General:
  • vim
  • git
  • jq
  • python3 / pip3
  • speedtest-cli
• CentOS Stream 9 only:
  • hping3
  • dnsperf

Download

# CentOS Stream 9 (default)
docker pull ghcr.io/pichuang/debug-container:master
# Azure Linux 3.0 (Microsoft Supported)
docker pull ghcr.io/pichuang/debug-container:microsoft

How to use debug-container on specific hosts?

1. Bridge Mode (Container on OS):

docker run -it --rm --name debug-container ghcr.io/pichuang/debug-container:master

2. Host Mode (Container within OS):

docker run -it --rm --name debug --privileged \
--ipc=host --net=host --pid=host -e HOST=/host \
-e NAME=debug-container -e IMAGE=pichuang/debug-container \
-v /run:/run -v /var/log:/var/log \
-v /etc/localtime:/etc/localtime -v /:/host \
ghcr.io/pichuang/debug-container:master

3. Container Mode (Bridge another container)

docker run -it --rm --name debug-container --net container:<container_name> ghcr.io/pichuang/debug-container:master

How to use debug-container on Native Kubernetes/Tanzu Kubernetes Grid Cluster/Azure Kubernetes Service?

1. Namespace Level Debugging: Running one Pod in namespace and any node

kubectl run -n default debug-container --restart=Never --rm -i --tty --image ghcr.io/pichuang/debug-container:master -- /bin/bash

2. Namespace Level Debugging: Running one Pod in namespace and specific node

# Show all of nodes
kubectl get nodes
NAME STATUS ROLES AGE VERSION
aks-agentpool-40137516-vmss000000 Ready agent 82m v1.22.11
aks-agentpool-40137516-vmss000001 Ready agent 82m v1.22.11
aks-agentpool-40137516-vmss000002 Ready agent 82m v1.22.11
# Run the command
kubectl run -n default debug-container --restart=Never --rm -i --tty --overrides='{ "apiVersion": "v1", "spec": {"kubernetes.io/hostname":"aks-agentpool-40137516-vmss000002"}}' --image ghcr.io/pichuang/debug-container:master -- /bin/bash

3. Node Level Debugging: Running one Pod on specific node

kubectl run -n default debug-container --image ghcr.io/pichuang/debug-container:master \
--restart=Never -it --attach --rm \
--overrides='{ "apiVersion": "v1", "spec": { "nodeSelector":{"kubernetes.io/hostname":"aks-agentpool-40137516-vmss000002"}, "hostNetwork": true}}' -- /bin/bash
# or
$ kubectl debug node/aks-agentpool-40137516-vmss000002 -it --image=ghcr.io/pichuang/debug-container:master -- /bin/bash
Creating debugging pod node-debugger-aks-agentpool-40137516-vmss000002-psvms with container debugger on node aks-agentpool-40137516-vmss000002.
If you don't see a command prompt, try pressing enter.[root@aks-agentpool-14864487-vmss000000 /]# chroot /host /bin/bashroot [ / ]# cat /etc/os-release | head -n 2

4. Deploy as a Deployment

kubectl apply -f deployment-debug-container.yaml

How to use debug-container on Red Hat OpenShift?

1. Namespace Level Debugging: Running one Pod in project and any node

oc project <PROJECT NAME>
oc run ocp-debug-container --image ghcr.io/pichuang/debug-container:master \
--restart=Never --attach -i --tty --rm

2. Namespace Level Debugging: Running one Pod in project and specific node

oc project <PROJECT NAME>
oc run ocp-debug-container --image ghcr.io/pichuang/debug-container:master \
--restart=Never --attach -i --tty --rm \
--overrides='{ "apiVersion": "v1", "spec": { "kubernetes.io/hostname":"compute-1"}}}'

• Remind: Please replace kubernetes.io/hostname:<hostname>

3. Node Level Debugging: Running one Pod on specific node

oc project <PROJECT NAME>
oc run ocp-debug-container --image ghcr.io/pichuang/debug-container:master \
--restart=Never -it --attach --rm \
--overrides='{ "apiVersion": "v1", "spec": { "nodeSelector":{"kubernetes.io/hostname":"compute-1"}, "hostNetwork": true}}'

4. Running Container Level Debugging

oc project <PROJECT NAME>
oc rsh pod/<PDO NAME>

5. Running Pods Level Debugging

oc project <PROJECT NAME>
oc debug pods/<Pod NAME>

How to Import YAML?

---
apiVersion: v1kind: Podmetadata:
name: debug-containerspec:
containers:
- image: ghcr.io/pichuang/debug-container:mastername: debug-containercommand: [ "/bin/bash", "-c", "--" ]args: [ "while true; do sleep 30; done;" ]resources:
requests:
cpu: 100mmemory: 128Milimits:
cpu: 500mmemory: 512MisecurityContext:
runAsUser: 0runAsNonRoot: falseallowPrivilegeEscalation: falsecapabilities:
drop:
- ALLadd:
- NET_RAW
- NET_ADMIN

How to build the container images?

• CentOS Stream 9 (default):

make build-docker

• Azure Linux 3.0 (Microsoft Supported):

make build-docker-microsoft

• Azure Linux 3.0 with internal package mirror:

docker build -f Dockerfile-microsoft \
--build-arg PACKAGE_REPO_URL=https://internal.example.com/azurelinux \
-t debug-container:microsoft .

• If you choose buildah...

make build-buildah

Security

• Base images pinned with SHA256 digest
• Git dependencies pinned to specific commits
• pip packages pinned to exact versions
• All CI workflow actions pinned to commit SHA
• Container images signed with cosign
• SBOM generated with Anchore Syft
• Vulnerability scanning with Snyk
• Supply chain security assessed with OpenSSF Scorecard

Author

• Phil Huangphil.huang@microsoft.com