Bug: kafka testcontainer does not support adding a network

[champialex]

Describe the bug

KafkaContainer.tc_start only allows containers to have a single IP, so it is not possible to add a network

Changing
listeners = f"PLAINTEXT://{host}:{port},BROKER://$(hostname -i):9092" to
listeners = f"PLAINTEXT://{host}:{port},BROKER://$(hostname -i | cut -d' ' -f1):9092" fixes it

Exception in the container logs:

java.lang.IllegalArgumentException: Error creating broker listeners from 'PLAINTEXT://192.168.64.2:33647,BROKER://172.17.0.3 172.21.0.2:9092': Unable to parse BROKER://172.17.0.3 172.21.0.2:9092 to a broker endpoint
	at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:266)
	at kafka.server.KafkaConfig.effectiveAdvertisedListeners(KafkaConfig.scala:2154)
	at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:2275)
	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:2233)
	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1604)
	at kafka.Kafka$.buildServer(Kafka.scala:72)
	at kafka.Kafka$.main(Kafka.scala:91)
	at kafka.Kafka.main(Kafka.scala)
Caused by: org.apache.kafka.common.KafkaException: Unable to parse BROKER://172.17.0.3 172.21.0.2:9092 to a broker endpoint
	at kafka.cluster.EndPoint$.createEndPoint(EndPoint.scala:58)
	at kafka.utils.CoreUtils$.$anonfun$listenerListToEndPoints$10(CoreUtils.scala:263)
	at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
	at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
	at scala.collection.mutable.ArraySeq.map(ArraySeq.scala:37)
	at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:263)
	... 7 more

To Reproduce

 from kafka import KafkaAdminClient
 from testcontainers.kafka import KafkaContainer

 with Network() as network:
 kafka_ctr = KafkaContainer()
 kafka_ctr.with_network(network)
 kafka_ctr.with_network_aliases("kafka")

 with kafka_ctr:
 print("started") # Will not reach here and timeout
 admin_client = KafkaAdminClient(bootstrap_servers=[kafka_ctr.get_bootstrap_server()])
 print(admin_client.describe_cluster())

Runtime environment

[[package]]
name = "testcontainers"
version = "4.7.1"

# Get the operating system information (on a unix os).
$ uname -a
Darwin sydmacg366 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:31:10 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6031 arm64
# Get the python version.
$ python --version
Python 3.12.2
# Get the docker version and other docker information.
$ docker info

Client: Docker Engine - Community
 Version: 27.0.3
 Context: default
 Debug Mode: false
 Plugins:
 buildx: Docker Buildx (Docker Inc.)
 Version: v0.15.1
 Path: /Users/adecham/.docker/cli-plugins/docker-buildx
 compose: Docker Compose (Docker Inc.)
 Version: 2.28.1
 Path: /Users/adecham/.docker/cli-plugins/docker-compose

Server:
 Containers: 338
 Running: 1
 Paused: 0
 Stopped: 337
 Images: 24
 Server Version: 23.0.6
 Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Using metacopy: false
 Native Overlay Diff: true
 userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 0cae528dd6cb557f7201036e9f43420650207b58
 runc version: 860f061b76bb4fc671f0f9e900f7d80ff93d4eb7
 init version:
 Security Options:
 seccomp
 Profile: builtin
 Kernel Version: 6.1.55-0-virt
 Operating System: Alpine Linux v3.18
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 7.753GiB
 Name: colima
 ID: c74f716c-594e-4e7c-8ae1-f9f71efbd2ac
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
 127.0.0.0/8
 Live Restore Enabled: false

[alexanderankin]

seems related to #633

can you share more of the use case, i understand this is a complete reproduction but i am also kind of curious the use case, whether in python form or a general description - I do not use this feature myself so I want to understand a bit better. thanks.

[champialex]

Thanks for the quick turnaround!

The use case is when chaining several containers (in my case telegraf and kafka). Having a network alias makes it possible to do that without having to do complicated file generation based on actual IPs: I can just point telegraf to kafka:9092.