Production enviroment

apinar · April 17, 2019, 6:51am

Hi,

I need create the production enviroment and I’ve made an installation from sources with python3 support, but I don’t know if the version is stable, because I’ve got some errors. Other point that I don’t know what to do if the certificates, how do I have to create and configurate them?

I put here the steps and errors that I’ve got:

Zato installation
Install from sources:
# sudo bash
# mkdir /opt/zato
# cd /opt/zato
# git clone https://github.com/zatosource/zato
# cd zato/code
# ./install.sh -p python3

Problems founded during installation:
hvac 0.7.2 has requirement requests>=2.21.0, but you’ll have requests 2.19.0 which is incompatible.
jsonschema 3.0.1 has requirement six>=1.11.0, but you’ll have six 1.10.0 which is incompatible.
warlock 1.1.0 has requirement jsonschema<3,>=0.7, but you’ll have jsonschema 3.0.1 which is incompatible.
pyasn1-modules 0.2.4 has requirement pyasn1<0.5.0,>=0.4.1, but you’ll have pyasn1 0.1.8 which is incompatible.
python-ldap 3.2.0 has requirement pyasn1>=0.3.7, but you’ll have pyasn1 0.1.8 which is incompatible.

I solved them doing the installation of these packages doing sudo with -H option, I think that doing Zato installation being sudo with option -H solved them too:
# sudo -H bash
# pip3 install hvac
# pip3 install jsonschema
# pip3 install warlock
# pip3 install pyasn1-modules
# pip3 install python-ldap

Creating the cluster
I assume that the installation use a different hosts for DB and the type is MySQL, and for Redis.

Create directory for the cluster
# mkdir /opt/zato/clusters
# mkdir /opt/zato/clusters/[CLUSTER_NAME]

Create ODB Zato structure
# zato create odb --odb_host [DB_HOST] --odb_user [DB_USER] --odb_db_name [DB_NAME] mysql --odb_port [DB_PORT]
# zato create cluster --odb_host [DB_HOST] --odb_port 3306 --odb_user [DB_USER] --odb_db_name [DB_NAME] mysql localhost 11223 20151 [REDIS_HOST] 6379 [CLUSTER_NAME] admin

Certificates. I really need help here, because I don’t know how create (buy) the certificates for a production enviroment, can someone help me/us at this point? I continue with
the steps that are indicated in the documentation:
# mkdir /opt/zato/clusters/[CLUSTER_NAME]/ca
# zato ca create ca /opt/zato/clusters/[CLUSTER_NAME]/ca
# zato ca create web_admin /opt/zato/clusters/[CLUSTER_NAME]/ca
# zato ca create lb_agent /opt/zato/clusters/[CLUSTER_NAME]/ca lb_agent_[CLUSTER_NAME]
# zato ca create server /opt/zato/clusters/[CLUSTER_NAME]/ca [CLUSTER_NAME] server_node1
# zato ca create server /opt/zato/clusters/[CLUSTER_NAME]/ca [CLUSTER_NAME] server_node2
# zato ca create scheduler /opt/zato/clusters/[CLUSTER_NAME]/ca [CLUSTER_NAME] scheduler

Create web admin panel:
# mkdir /opt/zato/clusters/[CLUSTER_NAME]/web_admin
# zato create web_admin --verbose --odb_host [DB_HOST] --odb_port [DB_PORT] --odb_user [DB_USER] --odb_db_name [DB_NAME]
–pub_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-pub/web-admin-pub.pem --priv_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-priv/web-admin-priv.pem
–cert_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-cert/web-admin-cert.pem --ca_certs_path /opt/zato/clusters/[CLUSTER_NAME]/ca/ca-material/ca-cert.pem
–tech_account_password [TECH_PASS] /opt/zato/clusters/[CLUSTER_NAME]/web_admin mysql admin
  	At this point I get this error message when I try to create the web_admin:
  		File "/usr/local/bin/zato", line 11, in <module>
  			load_entry_point('zato-cli', 'console_scripts', 'zato')()
  		File "/opt/zato/zato/code/zato-cli/src/zato/cli/zato_command.py", line 392, in main
  			return run_command(get_parser().parse_args())
  		File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 374, in run_command
  			command_class[args.command](args).run(args)
  		Traceback (most recent call last):
  		File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 650, in run
  			return_code = self.execute(args)
  		File "/opt/zato/zato/code/zato-cli/src/zato/cli/create_web_admin.py", line 129, in execute
  			lb_agent_use_tls = bool(args.get('ca_certs_path'))
  		AttributeError: 'Namespace' object has no attribute 'get'

  	The line 129 of file "/opt/zato/zato/code/zato-cli/src/zato/cli/create_web_admin.py":
  		lb_agent_use_tls = bool(args.get('ca_certs_path'))
Create server node 1:
# mkdir /opt/zato/clusters/[CLUSTER_NAME]/server_node1
# zato create server --odb_host [DB_HOST] --odb_port [DB_PORT] --odb_user [DB_USER] --odb_db_name [DB_NAME]
–pub_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-pub/[CLUSTER_NAME]-server_node1-pub.pem
–priv_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-priv/[CLUSTER_NAME]-server_node1-priv.pem
–cert_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-cert/[CLUSTER_NAME]-server_node1-cert.pem
–ca_certs_path /opt/zato/clusters/[CLUSTER_NAME]/ca/ca-material/ca-cert.pem
–secret_key [SECRET_KEY] --jwt_secret [JWT_SECRET] --http_port 17010
/opt/zato/clusters/[CLUSTER_NAME]/server_node1/ mysql [REDIS_HOST] 6379 [CLUSTER_NAME] server_node1
  	At this point I get this error message when I try to create the server:
  		Could not create the server, e:`Traceback (most recent call last):
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/create_server.py", line 731, in execute
  		    key1 = key1.decode('utf8')
  		AttributeError: 'str' object has no attribute 'decode'
  		`
  		OK

  	The line 731 of file "/opt/zato/zato/code/zato-cli/src/zato/cli/create_server.py":
  		key1 = key1.decode('utf8') 


  	Another error in the same file but in other line:
  		Could not create the server, e:`Traceback (most recent call last):
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/create_server.py", line 742, in execute
  		    zato_misc_jwt_secret = fernet1.encrypt(getattr(args, 'jwt_secret', Fernet.generate_key()))
  		  File "/opt/zato/zato/code/lib/python3.6/site-packages/cryptography/fernet.py", line 51, in encrypt
  		    return self._encrypt_from_parts(data, current_time, iv)
  		  File "/opt/zato/zato/code/lib/python3.6/site-packages/cryptography/fernet.py", line 55, in _encrypt_from_parts
  		    raise TypeError("data must be bytes.")
  		TypeError: data must be bytes.
  		`
  		OK

  	Line 742:
  		zato_misc_jwt_secret = fernet1.encrypt(getattr(args, 'jwt_secret', Fernet.generate_key()))
Create load balancer:
# mkdir /opt/zato/clusters/[CLUSTER_NAME]/lb_agent
# zato create load_balancer --pub_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-pub/lb-agent-pub.pem
–priv_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-priv/lb-agent-priv.pem
–cert_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-cert/lb-agent-cert.pem
–ca_certs_path /opt/zato/clusters/[CLUSTER_NAME]/ca/ca-material/ca-cert.pem /opt/zato/clusters/[CLUSTER_NAME]/lb_agent/
  	At this point I get this error message when I try to create the load balancer:
  		File "/usr/local/bin/zato", line 11, in <module>
  		    load_entry_point('zato-cli', 'console_scripts', 'zato')()
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/zato_command.py", line 392, in main
  		    return run_command(get_parser().parse_args())
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 374, in run_command
  		    command_class[args.command](args).run(args)
  		  Traceback (most recent call last):
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 650, in run
  		    return_code = self.execute(args)
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/create_lb.py", line 131, in execute
  		    is_tls_enabled = bool(args.get('priv_key_path'))
  		AttributeError: 'Namespace' object has no attribute 'get'

  	Line 131 of file "/opt/zato/zato/code/zato-cli/src/zato/cli/create_lb.py":
  		is_tls_enabled = bool(args.get('priv_key_path'))
Create scheduler:
# mkdir /opt/zato/clusters/[CLUSTER_NAME]/scheduler/
# zato create scheduler --odb_host [DB_HOST] --odb_port [DB_PORT] --odb_user [DB_USER] --odb_db_name [DB_NAME]
–pub_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-pub/[CLUSTER_NAME]-scheduler-pub.pem
–priv_key_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-priv/[CLUSTER_NAME]-scheduler-priv.pem
–cert_path /opt/zato/clusters/[CLUSTER_NAME]/ca/out-cert/[CLUSTER_NAME]-scheduler-cert.pem
–ca_certs_path /opt/zato/clusters/[CLUSTER_NAME]/ca/ca-material/ca-cert.pem --secret_key [SECRET_KEY]
/opt/zato/clusters/[CLUSTER_NAME]/scheduler/ mysql [REDIS_HOST] [REDIS_PORT] [CLUSTER_NAME]
  	At this point I get this error message when I try to create the scheduler:
  		File "/usr/local/bin/zato", line 11, in <module>
  		    load_entry_point('zato-cli', 'console_scripts', 'zato')()
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/zato_command.py", line 392, in main
  		    return run_command(get_parser().parse_args())
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 374, in run_command
  		    command_class[args.command](args).run(args)
  		  Traceback (most recent call last):
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 650, in run
  		    return_code = self.execute(args)
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/create_scheduler.py", line 230, in execute
  		    secret_key = secret_key.decode('utf8')
  		AttributeError: 'str' object has no attribute 'decode'

  	Line 230 in the file "/opt/zato/zato/code/zato-cli/src/zato/cli/create_scheduler.py":
  		secret_key = secret_key.decode('utf8')

  	Another error at line 239:
  		File "/usr/local/bin/zato", line 11, in <module>
  		    load_entry_point('zato-cli', 'console_scripts', 'zato')()
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/zato_command.py", line 392, in main
  		    return run_command(get_parser().parse_args())
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 374, in run_command
  		    command_class[args.command](args).run(args)
  		  Traceback (most recent call last):
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/__init__.py", line 650, in run
  		    return_code = self.execute(args)
  		  File "/opt/zato/zato/code/zato-cli/src/zato/cli/create_scheduler.py", line 239, in execute
  		    use_tls = bool(args.get('priv_key_path'))

  	Line 239:
  		use_tls = bool(args.get('priv_key_path'))

Can someone explain how to create the production enviroment and which version do I have to use?

Best regards.

dsuch · April 17, 2019, 7:17am

Hello @apinar,

Zato 3.1 is not a stable version yet. It will be considered stable once it has been released in June this year. Current stable version is Zato 3.0.

The idea with install.sh is that it is self-contained - you can run it and it will do everything needed to install Zato. If it does not then you can report it, either here or on GitHub. Certainly no manual steps are needed in addition to install.sh.

In 3.1, CA certificates are no longer needed on input to zato create commands, they are optional now. That is a very recent change, implemented literally a few days ago.

Regards.

jsabater · April 17, 2019, 8:29am

Plus one to use sudo --set-home instead of just sudo?

Not as a sollution to the conflicts in package versions, though. That’s another matter.

apinar · April 17, 2019, 12:15pm

Well, in my case, I couldn’t update the packages until I not --set-home. I did it because when I tried to update them from terminal, fails too and in the pip3 error message suggest relaunch installation sudoing with -H option.

apinar · April 17, 2019, 12:15pm

Hello @dsuch ,

well the installation have worked well, I got the problems creating the cluster. If I understand well, then, in June, with the 3.1 the certificates are not needed for a production enviroment?

And for create the production enviroment today, how can I install the 3.0? is it possible do it from sources or only from package? how can I configure the certificates to ensure the enviroment?

Thanks in advance.