Set default number of servers in docker-compose file

Hi Everyone.

We are using Zato in Docker via a docker-compose file

version: "3"

services:
  zato:
    build:
      context: .
    container_name: zato-dev
    ports:
      - "xxxx:8183"
      - "yyyy:11223"
    environment:
      ZATO_WEB_ADMIN_PASSWORD: "my_password"
    volumes:
      - ./zato/:/opt/hot-deploy/services
      - ./tmp:/opt/zato/tmp
      - ./zato-enmasse.yml:/opt/zato/zato-enmasse.yml
      - ./zato-enmasse.yml:/opt/hot-deploy/enmasse/enmasse.yaml # Comment this part if you don't want to run enmasse at the start

I would like to know if there is a way to set the default number of server (which is one in zato dockerized) at the beginning (in environment variable for example). So that this way we could increase or decrease easily the number of servers as we want.

Thanks a lot for your help guyz

Hello Steve,

in the current version of the Docker Quickstart container, there is always one server and I agree that making it configurable makes sense.

The quickstart command accepts a “servers” parameter so you can create your own containers running this command inside with as many servers as needed, e.g. “zato quickstart --servers 3 …”.

Alternatively, you can create more containers with a load-balancer in front of them.

I cannot commit to a specific deadline but I can say that ultimately the “servers” parameter will be part of the Docker Quickstart too.

Regards.

1 Like

Hi @dsuch ,

Thank you for the fast reply.
We will try to check what suit the most in our case. But I think we certainly go for multiple containers.

I’ve come accros the following issue Zato cluster about creating a cluster. Could I use this docker compose or is there a new way to proceed to create the load balancer ?

Thanks

What you linked to is someone else’s Docker Compose file - we do not ship anything in Zato directly. Everyone creates their own because they are always specific to a given installation or environment.

Creating a load-balancer in front of your containers is something that you can do independently of Zato and this will again depend on how your environment looks like, where your datacenter is, if it is a public cloud or not.

If you run your own servers then you can simply use HAProxy but this is not tied to Zato as such - you can look up details on the web how to use HAProxy as a load-balancer and then make it run under Docker.

1 Like

Hello Steve,

I have had a look at the Docker image from HAProxy and it looks good - I suggest that you use it as your load-balancer.

https://hub.docker.com/_/haproxy

In particular, it is nice that one can simply prepare a haproxy.cfg file on host and then map it to the container. That is for sure useful.

Regards.

Hi dsush.

Big thanks for your help. That is the kind of thing I was looking for. I have never used HAproxy. I will try to set it and show you the final docker-compose file

Regards :+1:

Hi @dsuch

I have set HAproxy. But it seem I can not access the admin dashboard through it. I have access to REST CHANNEL via postman, because I have URL not found (CID:cdc39d3a76492fcba55f714f) when calling an inexistent URL so calling APIs is fine. My concern is about the dashboard.

Here is my HAproxy config file

global
    maxconn 100000
    log stdout local0

defaults
    mode http
    log global
    option httplog
    timeout connect 600s
    timeout client 600s
    timeout server 600s

frontend router_zato_8183
    bind :8183
    default_backend backend_zato_port_8183

frontend router_zato_8083
    bind :8083
    default_backend backend_zato_port_8083

backend backend_zato_port_8183
    server zato1 zato1:8183 check
    server zato2 zato2:8183 check

backend backend_zato_port_8083
    server zato1 zato1:11223 check
    server zato2 zato2:11223 check

My docker-compose file is the following:

version: "3"

services:
  zato1:
    build:
      context: .
    container_name: zato1
    ports:
      - "5183:8183"
      - "5083:11223"
    environment:
      ZATO_WEB_ADMIN_PASSWORD: "my_password"
    volumes:
      - ./zato/:/opt/hot-deploy/services
      - ./tmp:/opt/zato/tmp
      - ./zato-enmasse.yml:/opt/zato/zato-enmasse.yml
      - ./config/ansible/zato-quickstart-02.yaml:/zato-ansible/zato-quickstart-02.yaml
      - ./docs:/opt/zato/docs # Path where to generate documentation
      - ./logs:/opt/zato/logs # Path where to get logs
      - ./zato-enmasse.yml:/opt/hot-deploy/enmasse/enmasse.yaml # Comment this part if you don't want to run enmasse at the start

  zato2:
    build:
      context: .
    container_name: zato2
    ports:
      - "6183:8183"
      - "6083:11223"
    environment:
      ZATO_WEB_ADMIN_PASSWORD: "my_password"
    volumes:
      - ./zato/:/opt/hot-deploy/services
      - ./tmp:/opt/zato/tmp
      - ./zato-enmasse.yml:/opt/zato/zato-enmasse.yml
      - ./config/ansible/zato-quickstart-02.yaml:/zato-ansible/zato-quickstart-02.yaml
      - ./docs:/opt/zato/docs # Path where to generate documentation
      - ./logs:/opt/zato/logs # Path where to get logs
      - ./zato-enmasse.yml:/opt/hot-deploy/enmasse/enmasse.yaml # Comment this part if you don't want to run enmasse at the start

  haproxy:
    container_name: zato_HAproxy
    image: haproxy:2.6.2-alpine
    ports:
      - "8183:8183"
      - "8083:8083"
    volumes:
      - ./config/HAproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
    links:
      - zato1
      - zato2
    depends_on:
      - zato1
      - zato2

I am stuck in the connection page with no error. After login, it seems like I am not redirected.
Screen Shot 2022-08-18 at 13.17.42

Do you have any idea of I have omitted, or any suggestion please ?

Thank you for you help.

OK, but I am not sure what happens when you enter the credentials:

  • What is displayed in the browser?
  • What is stored in logs of both of the dashboards?
  • What happens if you take out the dashboards from the HAProxy config one by one?

To answer your questions

1. What is displayed in the browser, when you enter the credentials?
When I try to pass by HAproxy, Nothing! No error! It seems like a redirection to the login page, just as shown in the picture.
But I log in successfully when trying to log in each container Dashboard one by one

2. What is stored in logs of both of the dashboards, when you enter the credentials?
Here are the logs of HAproxy container

zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61610 [18/Aug/2022:15:45:51.010] router_zato_8183 backend_zato_port_8183/zato1 0/0/1/48/49 302 408 - - ---- 6/6/0/0/0 0/0 "POST /accounts/login/ HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61606 [18/Aug/2022:15:45:51.062] router_zato_8183 backend_zato_port_8183/zato2 0/0/0/18/19 302 257 - - ---- 5/5/0/0/0 0/0 "GET /zato/ HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61596 [18/Aug/2022:15:45:51.083] router_zato_8183 backend_zato_port_8183/zato1 0/0/0/27/27 200 5521 - - ---- 4/4/0/0/0 0/0 "GET /accounts/login/?next=/zato/ HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61600 [18/Aug/2022:15:45:51.125] router_zato_8183 backend_zato_port_8183/zato2 0/0/0/8/8 200 21585 - - ---- 5/5/4/2/0 0/0 "GET /static/css/jquery.ui.custom.css HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61602 [18/Aug/2022:15:45:51.126] router_zato_8183 backend_zato_port_8183/zato1 0/0/0/11/11 200 10212 - - ---- 5/5/3/1/0 0/0 "GET /static/css/formalize.css HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61612 [18/Aug/2022:15:45:51.126] router_zato_8183 backend_zato_port_8183/zato2 0/0/0/15/15 200 3483 - - ---- 5/5/2/1/0 0/0 "GET /static/superfish/css/superfish.css HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61746 [18/Aug/2022:15:45:51.127] router_zato_8183 backend_zato_port_8183/zato1 0/0/0/18/18 200 1301 - - ---- 5/5/1/0/0 0/0 "GET /static/css/bvalidator.css HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61744 [18/Aug/2022:15:45:51.127] router_zato_8183 backend_zato_port_8183/zato2 0/0/0/22/22 200 7383 - - ---- 5/5/0/0/0 0/0 "GET /static/css/style.css HTTP/1.1"
zato_HAproxy | <134>Aug 18 15:45:51 haproxy[9]: 192.168.16.1:61744 [18/Aug/2022:15:45:51.268] router_zato_8183 backend_zato_port_8183/zato1 0/0/0/7/7 200 2094 - - ---- 5/5/0/0/0 0/0 "GET /static/favicon/favicon-32x32.png HTTP/1.1"

But there is no log of zato container.

3. What happens if you take out the dashboards from the HAProxy config one by one, when you enter the credentials?
I can login successfully on each one, one by one

To confirm it:

  • When there is only one dashboard behind HAProxy, i.e. you remove the other one from the load-balancer’s configuration, then you can log in to the one that is left via HAProxy, no matter which dashboard it is that was left

  • When there are two dashboards behind HAProxy, you can never log in to either of them

Is that correct?

Yes that’s correct

Ok, but I still do not have the information what is in the logs of both dashboards when this happens.

You need to enter the containers where the dashboards are and check their logs in the file system. This is not the same as checking the container’s log which will not show anything to do with dashboards because the container’s logs show only server logs.

Dashboards will have their logs in “/path/to/env/web-admin/logs/server.log”. You need to observe both dashboards and then compare it to what goes to logs when there is only one dashboard.

The goal here is to confirm whether all requests go to the same dashboard or is the load-balancer distributes to both of them.

there is no error on logs.

first zato container

2022-08-18 14:06:06,450 - INFO - 2634:MainThread - zato.admin.web.views.main:63 - Login request received
2022-08-18 14:06:06,450 - INFO - 2634:MainThread - zato.admin.web.views.main:71 - Login request -> GET

Second zato container

2022-08-18 14:06:16,575 - INFO - 2632:MainThread - zato.admin.web.views.main:63 - Login request received
2022-08-18 14:06:16,575 - INFO - 2632:MainThread - zato.admin.web.views.main:77 - Login request -> POST
2022-08-18 14:06:16,575 - INFO - 2632:MainThread - zato.admin.web.views.main:84 - Login username -> `admin`
2022-08-18 14:06:16,611 - INFO - 2632:MainThread - zato.admin.web.views.main:95 - User password confirmed `admin`
2022-08-18 14:06:16,612 - INFO - 2632:MainThread - zato.admin.web.views.main:132 - User credentials are valid, redirecting `admin` to `/zato/`
2022-08-18 14:06:16,644 - INFO - 2632:MainThread - zato.admin.web.views.main:63 - Login request received
2022-08-18 14:06:16,644 - INFO - 2632:MainThread - zato.admin.web.views.main:71 - Login request -> GET

I think I have found the problem about the setting (2 zato containers and one HAproxy)

By using HAproxy, let’s suppose we come first to zato1, when log in. We enter our username and our password successfully. But as it is a load balancer, we are redirect to the zato2. As we are not log in the second one, we need to re-enter username and password. We successfully log in but HAproxy redirect us again to zato1.
As we are disconnected from zato1, we need to enter the username and the password. So we go back in the same cycle again and again.

Hi @dsuch .

I have found the solution. I have set cookies on HAproxy to manage connexion. You could check the final setting if you want

docker-compose file

version: "3"

services:
  zato1:
    build:
      context: .
    container_name: zato1
    ports:
      - "8801:8183"
      - "11223"
    environment:
      ZATO_WEB_ADMIN_PASSWORD: "my_pasword"
    volumes:
      - ./zato/:/opt/hot-deploy/services
      - ./tmp:/opt/zato/tmp
      - ./zato-enmasse.yml:/opt/zato/zato-enmasse.yml
      - ./config/ansible/zato-quickstart-02.yaml:/zato-ansible/zato-quickstart-02.yaml
      - ./docs:/opt/zato/docs # Path where to generate documentation
      - ./logs:/opt/zato/logs # Path where to get logs
      - ./zato-enmasse.yml:/opt/hot-deploy/enmasse/enmasse.yaml # Comment this part if you don't want to run enmasse at the start

  zato2:
    build:
      context: .
    container_name: zato2
    ports:
      - "8001:8183"
      - "11223"
    environment:
      ZATO_WEB_ADMIN_PASSWORD: "my_password"
    volumes:
      - ./zato/:/opt/hot-deploy/services
      - ./tmp:/opt/zato/tmp
      - ./zato-enmasse.yml:/opt/zato/zato-enmasse.yml
      - ./config/ansible/zato-quickstart-02.yaml:/zato-ansible/zato-quickstart-02.yaml
      - ./docs:/opt/zato/docs # Path where to generate documentation
      - ./logs:/opt/zato/logs # Path where to get logs
      - ./zato-enmasse.yml:/opt/hot-deploy/enmasse/enmasse.yaml # Comment this part if you don't want to run enmasse at the start

  haproxy:
    container_name: zato_HAproxy
    image: haproxy:2.6.2-alpine
    ports:
      - "8183:8183"
      - "8083:8083"
    volumes:
      - ./config/HAproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
    links:
      - zato1
      - zato2
    depends_on:
      - zato1
      - zato2

haproxy.cfg file


global
    maxconn 100000
    log stdout local0

defaults
    mode http
    log global
    option httplog
    timeout connect 600s
    timeout client 600s
    timeout server 600s

frontend router_zato_8183
    bind :8183
    default_backend backend_zato_port_8183

frontend router_zato_8083
    bind :8083
    default_backend backend_zato_port_8083

backend backend_zato_port_8183
    cookie WEBSVR insert
    server zato1 zato1:8183 cookie 1 check
    server zato2 zato2:8183 cookie 2 check

backend backend_zato_port_8083
    server zato1 zato1:11223 check
    server zato2 zato2:11223 check

Now I have access to the dashboard. And i can call REST channel via postman.
The only trouble is, when there is a need to create REST CHANNEL by dashboard, we would have to log into each zato container to create it. To avoid that, I thing we will have to manage enmasse to make more dynamic.

Thanks again for your help @dsuch :+1:. We really appreciate it. We think we will go with this setting for the moment.

Best regards

Thanks and yes, enmasse is what should be used for automation of the containers with servers.

How to mount an enmasse is explained in the Importing enmasse definitions section below:

https://zato.io/en/docs/admin/guide/install/docker.html