Hanging on setup

[vanosg]

Hi, on two different machines, docker-compose -f setup.yml up has hung at this point:

Creating elasticsearch ... done
Creating setup_logstash ... done
Creating setup_kibana ... done
Attaching to setup_logstash, setup_kibana
setup_1 | setup_kibana | -rw-rw-r-- 1 root root 1200 Sep 2 22:05 /usr/share/kibana/config/ca/ca.crt
setup_1 | setup_logstash | -rw-rw-r-- 1 root root 1200 Sep 2 22:05 /usr/share/logstash/config/ca/ca.crt

Is the process done here, and I have to manually tear down the containers? Or have they truly hung? Host box is ubuntu 18.01. Please let me know if I can provide anything else, thanks!

[fxdgear]

can you share the output of the elasticsearch container?

[vanosg]

<snip>
setup_1 | setup_elasticsearch | creating: /config/ssl/docker-cluster/elasticsearch/
setup_1 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.crt 
setup_1 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.key 
setup_1 | setup_elasticsearch | creating: /config/ssl/docker-cluster/kibana/
setup_1 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/kibana/kibana.crt 
setup_1 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/kibana/kibana.key 
setup_1 | setup_elasticsearch | creating: /config/ssl/docker-cluster/logstash/
setup_1 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/logstash/logstash.crt 
setup_1 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/logstash/logstash.key 
setup_1 | setup_elasticsearch | Move logstash certs to logstash config dir...
setup_1 | setup_elasticsearch | Move kibana certs to kibana config dir...
setup_1 | setup_elasticsearch | Move elasticsearch certs to elasticsearch config dir...
setup_1 | setup_elasticsearch exited with code 0

Then it goes into what I pasted before...

setup_1 | setup_elasticsearch exited with code 0
setup_1 | Found orphan containers (stack-docker_setup_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Starting elasticsearch ... done
Creating setup_kibana ... done
Creating setup_logstash ... done
Attaching to setup_kibana, setup_logstash
setup_1 | setup_kibana | -rw-rw-r-- 1 root root 1200 Sep 2 22:44 /usr/share/kibana/config/ca/ca.crt
setup_1 | setup_logstash | -rw-rw-r-- 1 root root 1200 Sep 2 22:44 /usr/share/logstash/config/ca/ca.crt

[vanosg]

If it helps,

Client:
 Version: 18.06.1-ce
 API version: 1.38
 Go version: go1.10.3
 Git commit: e68fc7a
 Built: Tue Aug 21 17:24:56 2018
 OS/Arch: linux/amd64
 Experimental: false

Server:
 Engine:
 Version: 18.06.1-ce
 API version: 1.38 (minimum version 1.12)
 Go version: go1.10.3
 Git commit: e68fc7a
 Built: Tue Aug 21 17:23:21 2018
 OS/Arch: linux/amd64
 Experimental: false

docker-compose version 1.22.0, build f46880fe

[vanosg]

I just tried this a fifth time, and it appears to have worked... I don't think I did anything different (but I did re-clone the dir, just to start fresh). Sorry to false-alarm on this

[fxdgear]

Might, have been cause it was downloading the docket images?

…

On Sun, Sep 2, 2018 at 18:46 Geo ***@***.***> wrote: I just tried this a *fifth* time, and it appears to have worked... I don't think I did anything different (but I did re-clone the dir, just to start fresh). Sorry to false-alarm on this — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#55 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAI8drBZgMrutym8xtHCTv9wIN_y_vDRks5uXHvmgaJpZM4WWyQr> .

[Crysalist]

I am getting the same issue.

• Downloaded the ZIP from the repo
• Extracted it.
• Installed newest docker-compose (v1.22.0)
• Did the -setup command
• hanging at :
  530ed487ab9_stack-docker-master_setup_1 | setup_kibana | -rw-rw-r-- 1 root root 1200 Sep 14 05:50 /usr/share/kibana/config/ca/ca.crt d530ed487ab9_stack-docker-master_setup_1 | setup_logstash | -rw-rw-r-- 1 root root 1200 Sep 14 05:50 /usr/share/logstash/config/ca/ca.crt

Then also

• deleted all docker images
• deleted all files
• rinse and repeat of the above but still no luck

[Crysalist]

Seems the command :
sysctl -w vm.max_map_count=262144
did not take the first time around. Redid it also and now it works.

[rajiff]

I run it on mac, i did below to move past, hope it helps others

docker-machine ssh
sudo sysctl -w vm.max_map_count=262144

[ohing504]

In my case, sysctl -w vm.max_map_count=262144 command not help.
I tried docker-compose -f setup.yml up multiple times, and it works rarely.

And I found out the codes on hanging in following codes:

stack-docker/scripts/setup-kibana.sh

Lines 18 to 20 in f9d13bd

	while [[ "$(curl -u "elastic:${ELASTIC_PASSWORD}" --cacert $cacert -s -o /dev/null -w '%{http_code}' $es_url)" != "200" ]]; do
	sleep 5
	done

stack-docker/scripts/setup-logstash.sh

Lines 14 to 18 in f9d13bd

	es_url=https://elasticsearch:9200
	# Wait for Elasticsearch to start up before doing anything.
	while [[ "$(curl -u "elastic:${ELASTIC_PASSWORD}" --cacert $cacert -s -o /dev/null -w '%{http_code}' $es_url)" != "200" ]]; do
	sleep 5
	done

My logs show Starting elasticsearch ... done but the response of above code is 000.
So, the setup scripts fall asleep...

$ ~/stack-docker$ docker-compose -f setup.yml up
Recreating stack-docker_setup_1_1f2213838db8 ... done
Attaching to stack-docker_setup_1_1f2213838db8
setup_1_1f2213838db8 | Found orphan containers (stack-docker_setup_1_1f2213838db8) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating setup_elasticsearch ... done
Attaching to setup_elasticsearch
setup_1_1f2213838db8 | setup_elasticsearch | Determining if x-pack is installed...
setup_1_1f2213838db8 | setup_elasticsearch | === CREATE Keystore ===
setup_1_1f2213838db8 | setup_elasticsearch | Elastic password is: EhS5NuYLmy3yre9ief4HRg==
setup_1_1f2213838db8 | setup_elasticsearch | Created elasticsearch keystore in /usr/share/elasticsearch/config
setup_1_1f2213838db8 | setup_elasticsearch | Setting bootstrap.password...
setup_1_1f2213838db8 | setup_elasticsearch | === CREATE SSL CERTS ===
setup_1_1f2213838db8 | setup_elasticsearch | Remove old ca zip...
setup_1_1f2213838db8 | setup_elasticsearch | Creating docker-cluster-ca.zip...
setup_1_1f2213838db8 | setup_elasticsearch | CA directory exists, removing...
setup_1_1f2213838db8 | setup_elasticsearch | Unzip ca files...
setup_1_1f2213838db8 | setup_elasticsearch | Archive: /config/ssl/docker-cluster-ca.zip
setup_1_1f2213838db8 | setup_elasticsearch | creating: /config/ssl/ca/
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/ca/ca.crt
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/ca/ca.key
setup_1_1f2213838db8 | setup_elasticsearch | Remove old docker-cluster.zip zip...
setup_1_1f2213838db8 | setup_elasticsearch | Create cluster certs zipfile...
setup_1_1f2213838db8 | setup_elasticsearch | Unzipping cluster certs zipfile...
setup_1_1f2213838db8 | setup_elasticsearch | Archive: /config/ssl/docker-cluster.zip
setup_1_1f2213838db8 | setup_elasticsearch | creating: /config/ssl/docker-cluster/elasticsearch/
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.crt
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.key
setup_1_1f2213838db8 | setup_elasticsearch | creating: /config/ssl/docker-cluster/kibana/
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/kibana/kibana.crt
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/kibana/kibana.key
setup_1_1f2213838db8 | setup_elasticsearch | creating: /config/ssl/docker-cluster/logstash/
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/logstash/logstash.crt
setup_1_1f2213838db8 | setup_elasticsearch | inflating: /config/ssl/docker-cluster/logstash/logstash.key
setup_1_1f2213838db8 | setup_elasticsearch | Move logstash certs to logstash config dir...
setup_1_1f2213838db8 | setup_elasticsearch | Move kibana certs to kibana config dir...
setup_1_1f2213838db8 | setup_elasticsearch | Move elasticsearch certs to elasticsearch config dir...
setup_1_1f2213838db8 | setup_elasticsearch exited with code 0
setup_1_1f2213838db8 | Found orphan containers (stack-docker_setup_1_1f2213838db8) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Starting elasticsearch ... done
Creating setup_logstash ... done
Creating setup_kibana ... done
Attaching to setup_kibana, setup_logstash
setup_1_1f2213838db8 | setup_kibana | -rw-rw-r-- 1 root root 1200 Nov 26 09:03 /usr/share/kibana/config/ca/ca.crt
setup_1_1f2213838db8 | setup_logstash | -rw-rw-r-- 1 root root 1200 Nov 26 09:03 /usr/share/logstash/config/ca/ca.crt

My docker list:

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ef0bee2964c4 docker.elastic.co/logstash/logstash:6.3.0 "/usr/local/bin/dock…" 5 minutes ago Up 4 minutes 5044/tcp, 9600/tcp setup_logstash
0f579245ad17 docker.elastic.co/kibana/kibana:6.3.0 "/bin/bash -c 'cat /…" 5 minutes ago Up 5 minutes 5601/tcp setup_kibana
7622d92acdc3 docker/compose:1.21.2 "/bin/ash -c 'cat ./…" 5 minutes ago Up 5 minutes stack-docker_setup_1_1f2213838db8

[fxdgear]

@ohing504

Can you please check the output of the elasticsearch container.

It looks like elasticsearch is failing to start for some reason.

[minhtd1981]

@ohing504

Can you please check the output of the elasticsearch container.

It looks like elasticsearch is failing to start for some reason.

I think I got the same issue. Here is the log:

[WARN ][o.e.h.n.Netty4HttpServerTransport] [STIiuSf] caught exception while handling client http traffic, closing connection [id: 0x80a4e5db, L:0.0.0.0/0.0.0.0:9200 ! R:/172.19.0.3:60480]
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca
 at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
 at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca
 at sun.security.ssl.Alert.createSSLException(Alert.java:128) ~[?:?]
 at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]
 at sun.security.ssl.TransportContext.fatal(TransportContext.java:308) ~[?:?]
 at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:279) ~[?:?]
 at sun.security.ssl.TransportContext.dispatch(TransportContext.java:181) ~[?:?]
 at sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) ~[?:?]
 at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:672) ~[?:?]
 at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:627) ~[?:?]
 at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:443) ~[?:?]
 at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:422) ~[?:?]
 at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) ~[?:?]
 at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
 at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final]
 ... 15 more

[jabaa]

Why is this issue closed? I have the same problem. Is there some solution for this? Can't finish the setup.

[BeunHaas34]

same issue here

[gnunesmoura]

I'm having the same issue.