Errors/warnings while installing/running Zato into Docker


#1

Hi everyone!

So I deleted my previous container (to give the problems described in this thread a fresh approach) and, while following the instructions to install Zato 3.0 into Docker (latest version) from scratch, I noticed these errors/warnings:

Step 8/37 : RUN git pull
 ---> Running in 9a16b6e6b3b9
From https://github.com/zatosource/zato
   f5dbf26..acf62f0  support/3.0 -> origin/support/3.0
   0901b2f..e4f4db5  dsuch-f-gh755-pub-sub-sql-dsuch2 -> origin/dsuch-f-gh755-pub-sub-sql-dsuch2
   9fe9b34..eadc46a  dsuch-f-gh847-pre-3.0-cleanup -> origin/dsuch-f-gh847-pre-3.0-cleanup
 * [new branch]      dsuch-f-gh870-wsx-outconns -> origin/dsuch-f-gh870-wsx-outconns
 * [new branch]      dsuch-singleton1 -> origin/dsuch-singleton1
 * [new branch]      dsuch-tmp1 -> origin/dsuch-tmp1
   82ecf46..cb84372  main       -> origin/main
 * [new branch]      pre-support/3.0 -> origin/pre-support/3.0
 * [new branch]      revert-865-rtrind-f-gh862-create-scheduler -> origin/revert-865-rtrind-f-gh862-create-scheduler
   28b4288..b157ae5  u/nus      -> origin/u/nus
Updating f5dbf26..acf62f0
Fast-forward
 code/_install-alpine.sh                            |  10 +-
 code/_install-rhel.sh                              |   6 +-
[..]
 .../src/zato/admin/web/views/stats.py              |   2 +-
 code/zato-web-admin/test/__init__.py               |   2 +-
 code/zato-web-admin/test/zato/__init__.py          |   2 +-
 code/zato-web-admin/test/zato/admin/__init__.py    |   2 +-
 .../zato-web-admin/test/zato/admin/web/__init__.py |   2 +-
 .../test/zato/admin/web/views/__init__.py          |   2 +-
 .../test/zato/admin/web/views/test_stats.py        |   2 +-
 233 files changed, 4220 insertions(+), 1068 deletions(-)
 create mode 100644 code/zato-common/src/zato/common/odb/query/generic.py
 create mode 100644 code/zato-common/src/zato/common/wsx_client.py
 create mode 100644 code/zato-server/src/zato/server/base/worker/generic.py
 delete mode 100644 code/zato-server/src/zato/server/connection/web_socket/outgoing.py
 create mode 100644 code/zato-server/src/zato/server/generic/__init__.py
 create mode 100644 code/zato-server/src/zato/server/generic/api/__init__.py
 create mode 100644 code/zato-server/src/zato/server/generic/api/outconn_wsx.py
 create mode 100644 code/zato-server/src/zato/server/generic/connection.py
 create mode 100644 code/zato-server/src/zato/server/service/internal/generic/__init__.py
 create mode 100644 code/zato-server/src/zato/server/service/internal/generic/connection.py
 create mode 100644 code/zato-web-admin/src/zato/admin/static/js/outgoing/wsx.js
 create mode 100644 code/zato-web-admin/src/zato/admin/templates/zato/outgoing/wsx.html
 create mode 100644 code/zato-web-admin/src/zato/admin/web/forms/outgoing/wsx.py
 create mode 100644 code/zato-web-admin/src/zato/admin/web/views/outgoing/wsx.py
Removing intermediate container 9a16b6e6b3b9
 ---> 4ff516bbc664
Step 9/37 : RUN ./bin/pip install -e ./zato-cy
 ---> Running in 5a3ccda4151c
Obtaining file:///opt/zato/3.0/code/zato-cy
watchdog 0.8.3 has requirement argh>=0.24.1, but you'll have argh 0.23.2 which is incompatible.
sphinx 1.7.5 has requirement Pygments>=2.0, but you'll have pygments 1.5 which is incompatible.
zato-apitest 1.11 has requirement base32-crockford==0.2.0, but you'll have base32-crockford 0.3.0 which is incompatible.
zato-apitest 1.11 has requirement cassandra-driver==2.1.1, but you'll have cassandra-driver 3.0.0 which is incompatible.
zato-apitest 1.11 has requirement click==4.0, but you'll have click 1.1 which is incompatible.
zato-apitest 1.11 has requirement configobj==5.0.5, but you'll have configobj 5.0.6 which is incompatible.
zato-apitest 1.11 has requirement lxml==3.3.5, but you'll have lxml 3.7.3 which is incompatible.
zato-apitest 1.11 has requirement parse==1.6.4, but you'll have parse 1.6.6 which is incompatible.
zato-apitest 1.11 has requirement psycopg2==2.5.3, but you'll have psycopg2 2.7.4 which is incompatible.
zato-apitest 1.11 has requirement python-dateutil==2.2, but you'll have python-dateutil 2.6.0 which is incompatible.
zato-apitest 1.11 has requirement requests==2.3.0, but you'll have requests 2.19.0 which is incompatible.
zato-apitest 1.11 has requirement six==1.6.1, but you'll have six 1.10.0 which is incompatible.
zato-apitest 1.11 has requirement sqlalchemy==0.9.7, but you'll have sqlalchemy 1.2.8 which is incompatible.
Installing collected packages: zato-cy
  Found existing installation: zato-cy 3.0.0+rev.f5dbf26b
    Can't uninstall 'zato-cy'. No files were found to uninstall.
  Running setup.py develop for zato-cy
Successfully installed zato-cy
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Some of them seem innocuous enough, like psycopg2 or sqlalchemy requested versions not being the latest, but some others… well, I am not sure. Should we be worried?

The installation did seem to finish okay, though. Then, when running the container, I noticed these initial error messages related to postgresql:

2018-10-09 10:03:29,127 CRIT Set uid to user 0
2018-10-09 10:03:29,132 INFO supervisord started with pid 8
2018-10-09 10:03:30,134 INFO spawned: 'postgresql' with pid 11
2018-10-09 10:03:30,136 INFO spawned: 'zato-web-admin' with pid 12
2018-10-09 10:03:30,138 INFO spawned: 'redis' with pid 13
2018-10-09 10:03:30,140 INFO spawned: 'zato-server1' with pid 14
2018-10-09 10:03:30,144 INFO spawned: 'zato-server2' with pid 16
2018-10-09 10:03:30,145 INFO spawned: 'ssh' with pid 17
2018-10-09 10:03:30,147 INFO spawned: 'zato-scheduler' with pid 18
2018-10-09 10:03:30,148 INFO spawned: 'zato-load-balancer' with pid 20
2018-10-09 10:03:30,150 DEBG fd 6 closed, stopped monitoring <POutputDispatcher at 139729000157912 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stdout)>
2018-10-09 10:03:30,151 DEBG 'postgresql' stderr output:
postgresql: unrecognized service

2018-10-09 10:03:30,151 DEBG fd 8 closed, stopped monitoring <POutputDispatcher at 139729000157552 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stderr)>
2018-10-09 10:03:30,151 INFO exited: postgresql (exit status 1; not expected)
2018-10-09 10:03:30,151 DEBG received SIGCLD indicating a child quit
2018-10-09 10:03:30,154 DEBG 'redis' stdout output:
13:C 09 Oct 10:03:30.152 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf

Grepping the installation inside the container I found these:

supervisord.conf:[program:postgresql]
supervisord.conf:command = service postgresql start
supervisord.log:2018-10-09 10:03:30,134 INFO spawned: 'postgresql' with pid 11
supervisord.log:2018-10-09 10:03:30,150 DEBG fd 6 closed, stopped monitoring <POutputDispatcher at 139729000157912 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stdout)>
supervisord.log:2018-10-09 10:03:30,151 DEBG 'postgresql' stderr output:
supervisord.log:postgresql: unrecognized service
supervisord.log:2018-10-09 10:03:30,151 DEBG fd 8 closed, stopped monitoring <POutputDispatcher at 139729000157552 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stderr)>
supervisord.log:2018-10-09 10:03:30,151 INFO exited: postgresql (exit status 1; not expected)
supervisord.log:2018-10-09 10:03:31,157 INFO spawned: 'postgresql' with pid 45
supervisord.log:2018-10-09 10:03:31,189 DEBG fd 6 closed, stopped monitoring <POutputDispatcher at 139729000157768 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stdout)>
supervisord.log:2018-10-09 10:03:31,190 DEBG 'postgresql' stderr output:
supervisord.log:postgresql: unrecognized service
supervisord.log:2018-10-09 10:03:31,190 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 139729000157552 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stderr)>
supervisord.log:2018-10-09 10:03:31,190 INFO exited: postgresql (exit status 1; not expected)
supervisord.log:2018-10-09 10:03:33,193 INFO spawned: 'postgresql' with pid 105
supervisord.log:2018-10-09 10:03:33,227 DEBG fd 6 closed, stopped monitoring <POutputDispatcher at 139729000158056 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stdout)>
supervisord.log:2018-10-09 10:03:33,227 DEBG 'postgresql' stderr output:
supervisord.log:postgresql: unrecognized service
supervisord.log:2018-10-09 10:03:33,227 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 139729000157552 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stderr)>
supervisord.log:2018-10-09 10:03:33,228 INFO exited: postgresql (exit status 1; not expected)
supervisord.log:2018-10-09 10:03:36,470 INFO spawned: 'postgresql' with pid 120
supervisord.log:2018-10-09 10:03:36,479 DEBG 'postgresql' stderr output:
supervisord.log:postgresql: unrecognized service
supervisord.log:2018-10-09 10:03:36,479 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 139729000157552 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stderr)>
supervisord.log:2018-10-09 10:03:36,479 DEBG fd 6 closed, stopped monitoring <POutputDispatcher at 139729000157768 for <Subprocess at 139729000524976 with name postgresql in state STARTING> (stdout)>
supervisord.log:2018-10-09 10:03:36,479 INFO exited: postgresql (exit status 1; not expected)
supervisord.log:2018-10-09 10:03:37,480 INFO gave up: postgresql entered FATAL state, too many start retries too quickly

The container started fine enough, it seems, and I am working with it right now, so far with no problems. Just thought these postgresql messages may require some touches on the configuration files :slight_smile:


#2

Also I noticed that, at the end of the installation, the following files have wrong owner and group (root, should be zato):

/opt/zato/3.0/.git/FETCH_HEAD
/opt/zato/3.0/.git/ORIG_HEAD
/opt/zato/3.0/.git/index

Web admin returns Server Error (500) while accessing channels
#3

zato-apitest uses older dependencies and in fact it would be better to decouple it from the main installation so as not to have such spurious warnings.

As for PostgreSQL, the automatically created Zato cluster from the Dockerfile here uses SQLite rather than PostgreSQL so these warnings are harmless in the sense that your cluster will work, yet, you are right that it warrants a closer investigation because eventually I would like for the default Docker-based cluster to use PostgreSQL. But for now, this will not interfere with your installation.