Zato 3.0 on RHEL6

rtrind · 2018-07-03 19:37:00 UTC

Hello,

Following up on the things discussed at Zato 2.0.8 released and ODB HA reconnection issue (affecting scheduled services), the issues seems to be the same on Zato 3.0.

I created a VM from scratch with RHEL6.7 and after dealing with quite some problems with git and curl to allow TLS 1.2 to be used, I managed to install zato using the instructions from the docs without any problems.

When I try to run the zato quickstart create command, we again see the issues related to sqlite/pysqlite (and also some warnings about dependencies on some packages on the zato-cy deployment):

[zato@zato30play code]$ ./bin/pip install -e ./zato-cy
Obtaining file:///opt/zato/3.0/code/zato-cy
watchdog 0.8.3 has requirement argh>=0.24.1, but you’ll have argh 0.23.2 which is incompatible.
sphinx 1.7.5 has requirement Pygments>=2.0, but you’ll have pygments 1.5 which is incompatible.
zato-apitest 1.11 has requirement argparse==1.2.1, but you’ll have argparse 1.4.0 which is incompatible.
zato-apitest 1.11 has requirement base32-crockford==0.2.0, but you’ll have base32-crockford 0.3.0 which is incompatible.
zato-apitest 1.11 has requirement cassandra-driver==2.1.1, but you’ll have cassandra-driver 3.0.0 which is incompatible.
zato-apitest 1.11 has requirement click==4.0, but you’ll have click 1.1 which is incompatible.
zato-apitest 1.11 has requirement configobj==5.0.5, but you’ll have configobj 5.0.6 which is incompatible.
zato-apitest 1.11 has requirement lxml==3.3.5, but you’ll have lxml 3.7.3 which is incompatible.
zato-apitest 1.11 has requirement parse==1.6.4, but you’ll have parse 1.6.6 which is incompatible.
zato-apitest 1.11 has requirement psycopg2==2.5.3, but you’ll have psycopg2 2.7.4 which is incompatible.
zato-apitest 1.11 has requirement python-dateutil==2.2, but you’ll have python-dateutil 2.6.0 which is incompatible.
zato-apitest 1.11 has requirement requests==2.3.0, but you’ll have requests 2.19.0 which is incompatible.
zato-apitest 1.11 has requirement six==1.6.1, but you’ll have six 1.10.0 which is incompatible.
zato-apitest 1.11 has requirement sqlalchemy==0.9.7, but you’ll have sqlalchemy 1.2.8 which is incompatible.
Installing collected packages: zato-cy
Found existing installation: zato-cy 3.0.0+rev.f5dbf26b
Uninstalling zato-cy-3.0.0+rev.f5dbf26b:
Successfully uninstalled zato-cy-3.0.0+rev.f5dbf26b
Running setup.py develop for zato-cy
Successfully installed zato-cy
[zato@zato30play code]$ zato --version
Zato 3.0.0+rev.f5dbf26b
[zato@zato30play code]$ mkdir -p ~/env/LAB1
[zato@zato30play code]$ zato quickstart create ~/env/LAB1 sqlite localhost 6379 --kvdb_password ‘’ --verbose
[1/9] Certificate authority created
Traceback (most recent call last):
File “/opt/zato/current/bin/zato”, line 11, in
load_entry_point(‘zato-cli’, ‘console_scripts’, ‘zato’)()
File “/opt/zato/3.0/code/zato-cli/src/zato/cli/zato_command.py”, line 362, in main
return run_command(get_parser().parse_args())
File “/opt/zato/3.0/code/zato-cli/src/zato/cli/init.py”, line 354, in run_command
command_classargs.command.run(args)
File “/opt/zato/3.0/code/zato-cli/src/zato/cli/init.py”, line 630, in run
return_code = self.execute(args)
File “/opt/zato/3.0/code/zato-cli/src/zato/cli/quickstart.py”, line 329, in execute
if create_odb.Create(args).execute(args, False) == self.SYS_ERROR.ODB_EXISTS:
File “/opt/zato/3.0/code/zato-cli/src/zato/cli/create_odb.py”, line 29, in execute
engine = self._get_engine(args)
File “/opt/zato/3.0/code/zato-cli/src/zato/cli/init.py”, line 546, in _get_engine
return sqlalchemy.create_engine(get_engine_url(args), connect_args=connect_args)
File “/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/init.py”, line 424, in create_engine
return strategy.create(*args, **kwargs)
File “/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py”, line 81, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File “/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py”, line 339, in dbapi
raise e
ImportError: No module named _sqlite3

Restored the VM to a previous state and rebuild it from source, same issue.

I’m not sure how installing python27 on RHEL6 would change anything, since this seems to be related to the python embedded on zato (on 2.0.7 this error did not exist).

Any ideas on what could be done to make zato work on RHEL6 without too much manual (and prone to errors later) procedures?

Any help is appreciated. Thanks!

dsuch · 2018-07-04 03:38:03 UTC

Hi @rtrind,

support for SQLite under RHEL 6 was dropped in 3.0 - you are right that installing 2.7 in the OS would not bring much benefit.

I meant to drop it for good but actually, a colleague of mine will soon work on new package building scripts and this perhaps will be a good opportunity to bring it back. I will let you know.

For now (and this will continue to work), I suggest to use MySQL or PostgreSQL.

rtrind · 2018-07-13 18:11:54 UTC

Ok, tried using MySQL community edition 8.0 and for some reason it does not work on the quickstart command (it seems to be trying to connect without password for some reason):

[zato@zato30play ~]$ zato quickstart create ~/LAB1 mysql localhost 6739 --kvdb_password '' --odb_password <mypwhere> --odb_user zatouser --odb_db_name zatodb --odb_host localhost --odb_port 3306 --verbose
Options saved in file /opt/zato/zato.2018_07_13_16_22_45_055140.config
[1/9] Certificate authority created
Traceback (most recent call last):
  File "/opt/zato/current/bin/zato", line 11, in <module>
    load_entry_point('zato-cli', 'console_scripts', 'zato')()
  File "/opt/zato/3.0/code/zato-cli/src/zato/cli/zato_command.py", line 362, in main
    return run_command(get_parser().parse_args())
  File "/opt/zato/3.0/code/zato-cli/src/zato/cli/__init__.py", line 354, in run_command
    command_class[args.command](args).run(args)
  File "/opt/zato/3.0/code/zato-cli/src/zato/cli/__init__.py", line 630, in run
    return_code = self.execute(args)
  File "/opt/zato/3.0/code/zato-cli/src/zato/cli/quickstart.py", line 329, in execute
    if create_odb.Create(args).execute(args, False) == self.SYS_ERROR.ODB_EXISTS:
  File "/opt/zato/3.0/code/zato-cli/src/zato/cli/create_odb.py", line 32, in execute
    if engine.dialect.has_table(engine.connect(), 'install_state'):
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2102, in connect
    return self._connection_cls(self, **kwargs)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 90, in __init__
    if connection is not None else engine.raw_connection()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2188, in raw_connection
    self.pool.unique_connection, _connection)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2162, in _wrap_pool_connect
    e, dialect, self)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception_noconnection
    exc_info
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2158, in _wrap_pool_connect
    return fn()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 345, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 791, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 532, in checkout
    rec = pool._do_get()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1196, in _do_get
    self._dec_overflow()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1193, in _do_get
    return self._create_connection()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 350, in _create_connection
    return _ConnectionRecord(self)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 477, in __init__
    self.__connect(first_connect_check=True)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/pool.py", line 674, in __connect
    connection = pool._invoke_creator(self)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 106, in connect
    return dialect.connect(*cargs, **cparams)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 411, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect
    return Connection(*args, **kwargs)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/pymysql/connections.py", line 704, in __init__
    self.connect()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/pymysql/connections.py", line 974, in connect
    self._request_authentication()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/pymysql/connections.py", line 1203, in _request_authentication
    auth_packet = self._read_packet()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/pymysql/connections.py", line 1059, in _read_packet
    packet.check_error()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/pymysql/connections.py", line 384, in check_error
    err.raise_mysql_exception(self._data)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)
OperationalError: (pymysql.err.OperationalError) (1045, u"Access denied for user 'zato1'@'localhost' (using password: NO)") (Background on this error at: http://sqlalche.me/e/e3q8)

Reverted my image to a pre-state, installed mysql community 5.7 and now the quickstart fails on the web-admin step, silently:

[zato@zato30play ~]$ zato quickstart create ~/LAB1 mysql localhost 6739 --kvdb_password '' --odb_password <mypwhere> --odb_user zatouser --odb_db_name zatodb --odb_host localhost --odb_port 3306 --verbose
[1/9] Certificate authority created
[2/9] ODB schema created
[3/9] ODB initial data created
[4/9] server1 created
[5/9] server2 created
[6/9] Load-balancer created
[zato@zato30play ~]$

Rollback the image again and redid the step to create the environment manually. When trying to create the web-admin, same silent behavior, so I don’t know exactly why the web-admin is not being created:

> [zato@zato30play LAB1]$ zato create web_admin --odb_host localhost --odb_port 3663 --odb_user zatouser --odb_db_name zatodb --odb_password <mypwhere> --tech_account_password techpw ~/env/LAB1/web-admin mysql ~/env/LAB1/ca/out-pub/web-admin-pub*.pem ~/env/LAB1/ca/out-priv/web-admin-priv*.pem ~/env/LAB1/ca/out-cert/web-admin-cert*.pem ~/env/LAB1/ca/ca-material/ca-cert.pem techacc1 --verbose
[zato@zato30play LAB1]$

Ideas? Is it still related to the ODB? Should I try postgres at least for the test environment or is this unrelated? Any ways for zato create web-admin to output more data to the console, so I can understand the underlying issue?

Extra question: what would be the recommended upgrade procedure for production with minimal downtime and rollback possibilities? Can I install zato 3 alongside zato 2 and rebind the “current” symbolic link to go back to previous version? Is zato 3 ODB/Redis data backwards compatible with zato 2?

dsuch · 2018-07-13 18:24:36 UTC

Coincidentally, I happen to work on this very situation right now, quickstart create stopping at step [6/9] but only on RHEL 6. I will get back to you once this is resolved.

dsuch · 2018-07-16 14:55:27 UTC

Hello,

please try out package zato-3.0-stable2.el6.x86_64.rpm:

https://zato.io/docs/3.0/admin/guide/install/rhel.html

It resolves this issue and brings SQLite back for RHEL 6.

Regards.

rtrind · 2018-07-16 15:27:15 UTC

Thanks for the quick action, I will see if I can fit the new test rounds today and report back asap.

[]'s

rtrind · 2018-07-16 18:22:05 UTC

Quickstart server working properly now using sqlite. Thanks for the quick solution!

Will continue my tests and report back any new findings. If nothing blocker is find, I will recreate my preproduction environment and test my services against it, mainly trying to check if the scheduler issues stop happening.

Also there was a question which was not answered about the migration from Zato 2 to 3 on my production environment. Assuming the tests are OK and everything is working OK, how would you see the smoothest path to have the new version working with possibilities to rollback to the previous version if something breaks? My machines are baremetal, no VMs and no possibilities of easy snapshot rollbacks like my test environment.

Any suggestions are appreciated.

Thanks again for the patch!

EDIT: We celebrated too soon… the servers are giving errors when starting:

2018-07-16 15:30:14,954 - INFO - 6749:MainThread - gunicorn.main:176 - Starting gunicorn 18.0
2018-07-16 15:30:14,958 - INFO - 6749:MainThread - gunicorn.main:176 - Listening at: http://0.0.0.0:17010 (6749)
2018-07-16 15:30:14,961 - INFO - 6749:MainThread - gunicorn.main:176 - Using worker: gevent
2018-07-16 15:30:14,976 - INFO - 6763:MainThread - gunicorn.main:176 - Booting worker with pid: 6763
2018-07-16 15:30:16,245 - INFO - 6763:MainThread - zato.server.base.parallel:422 - Preferred address of `server1@quickstart-349993` (pid: 6763) is `http://10.0.2.15:17010`
2018-07-16 15:30:17,610 - ERROR - 6763:MainThread - gunicorn.main:182 - Exception in worker process:
Traceback (most recent call last):
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker
    self.cfg.post_fork(self, worker)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 781, in post_fork
    ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 428, in start_server
    self.set_up_config(server)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/config.py", line 191, in set_up_config
    query = self.odb.get_channel_web_socket_list(server.cluster.id, True)
  File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/api.py", line 1134, in get_channel_web_socket_list
    return query.channel_web_socket_list(session, cluster_id, needs_columns)
  File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 92, in inner
    tool = _SearchWrapper(func(*args), **kwargs)
  File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 67, in __init__
    self.total = q.session.execute(total_q).scalar()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1176, in execute
    bind, close_with_result=True).execute(clause, params or {})
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 508, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) no such column: sec_vault_conn_1.id [SQL: u'SELECT count(*) AS count_1 \nFROM cluster, channel_web_socket LEFT OUTER JOIN service ON service.id = channel_web_socket.service_id LEFT OUTER JOIN sec_base ON sec_base.id = channel_web_socket.security_id LEFT OUTER JOIN (sec_base AS sec_base_1 JOIN sec_vault_conn AS sec_vault_conn_1 ON sec_base_1.id = sec_vault_conn_1.id) ON sec_base.id = sec_vault_conn_1.id \nWHERE cluster.id = channel_web_socket.cluster_id AND cluster.id = ?'] [parameters: (1,)] (Background on this error at: http://sqlalche.me/e/e3q8)
Traceback (most recent call last):
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker
    self.cfg.post_fork(self, worker)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 781, in post_fork
    ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 428, in start_server
    self.set_up_config(server)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/config.py", line 191, in set_up_config
    query = self.odb.get_channel_web_socket_list(server.cluster.id, True)
  File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/api.py", line 1134, in get_channel_web_socket_list
    return query.channel_web_socket_list(session, cluster_id, needs_columns)
  File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 92, in inner
    tool = _SearchWrapper(func(*args), **kwargs)
  File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 67, in __init__
    self.total = q.session.execute(total_q).scalar()
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1176, in execute
    bind, close_with_result=True).execute(clause, params or {})
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 508, in do_execute
    cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) no such column: sec_vault_conn_1.id [SQL: u'SELECT count(*) AS count_1 \nFROM cluster, channel_web_socket LEFT OUTER JOIN service ON service.id = channel_web_socket.service_id LEFT OUTER JOIN sec_base ON sec_base.id = channel_web_socket.security_id LEFT OUTER JOIN (sec_base AS sec_base_1 JOIN sec_vault_conn AS sec_vault_conn_1 ON sec_base_1.id = sec_vault_conn_1.id) ON sec_base.id = sec_vault_conn_1.id \nWHERE cluster.id = channel_web_socket.cluster_id AND cluster.id = ?'] [parameters: (1,)] (Background on this error at: http://sqlalche.me/e/e3q8)
2018-07-16 15:30:17,615 - INFO - 6763:MainThread - gunicorn.main:176 - Worker exiting (pid: 6763)
2018-07-16 15:30:17,623 - ERROR - 6763:DummyThread-2 - springpython.context.ApplicationContext:109 - Could not destroy object 'server', exception 'Traceback (most recent call last):
  File "/opt/zato/3.0/code/lib/python2.7/site-packages/springpython/context/__init__.py", line 106, in shutdown_hook
    destroy_method()
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 829, in destroy
    self.invoke('zato.channel.web-socket.client.delete-by-server')
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 696, in invoke
    *args, **kwargs)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/__init__.py", line 1281, in invoke
    }, channel, None, needs_response=True, serialize=serialize)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/__init__.py", line 1342, in on_message_invoke_service
    service, is_active = self.server.service_store.new_instance_by_name(msg['service'])
  File "/opt/zato/3.0/code/zato-server/src/zato/server/service/store.py", line 204, in new_instance_by_name
    impl_name = self.name_to_impl_name[name]
KeyError: u'zato.channel.web-socket.client.delete-by-server'
'
2018-07-16 15:30:18,326 - INFO - 6749:MainThread - gunicorn.main:176 - Shutting down: Master
2018-07-16 15:30:18,327 - INFO - 6749:MainThread - gunicorn.main:176 - Reason: Worker failed to boot.

Related to sqlite patch still?

rtrind · 2018-07-17 15:04:04 UTC

Hi @dsuch, not trying to rush you or anything, but since I edited the post with an error and didn’t receive a notification, maybe you didn’t as well, just to let you know there are still errors on RHEL6.x build. See post above.

Thanks!

EDIT: Using the new version, the quickstart with mysql seems to be working, the servers started properly now, will ignore the sqlite error (since my production environment does not use it anyway). If you want me to open an issue on the github let me know.

dsuch · 2018-07-17 15:37:49 UTC

I didn’t get a notification either, yes, please open a ticket on GH about SQLite. Thanks.

dsuch · 2018-07-17 16:02:26 UTC

As to your question about the migration from 2.0 - the only stable part is that an enmasse export from 2.0 is expected to import cleanly into a 3.0 environment.

Other than that, the ODB has changed significantly. Redis less so. There were several new config files added. Scheduler is now started explicitly from command line.

Without VMs what I would likely do is setting up a parallel environment with the same functionality and switch at a convenient time but I have not tried to install 2.0 and 3.0 in the same system.

rtrind · 2018-07-17 19:30:51 UTC

I had never heard of enmasse before, I will study it and come back with extra comments.

My main concern is exactly the lack of machines to install a secondary Zato environment completely isolated from the original.

I do not mind in installing an isolated MySQL ODB, other Redis instance and using enmasse (or even reconfiguring all the services manually), and installing Zato 3 to use those new instances, as long as this allows the previous Zato version to eventually be back in production in case of problems in the new environment.

Do you see this working? If yes, I can try to reproduce this in my lab environment.

Continuing with the tests, my first service is not working, seemingly because of changes on the distributed lock mechanism:

2018-07-17 16:07:54,302 - INFO - 4019:DummyThread-96 - service_name:124 - lab - Starting new ffw:lab periodic check...
2018-07-17 16:07:54,302 - ERROR - 4019:DummyThread-96 - service_name:121 - lab - Service didn't finished properly. Marking error
2018-07-17 16:07:54,303 - ERROR - 4019:DummyThread-96 - service_name:107 - lock() got an unexpected keyword argument 'expires'
Traceback (most recent call last):
  File "/opt/zato/env/LAB1/server1/work/hot-deploy/current/ffw_service.py", line 83, in handle
    with self.lock(db.naming(p_comp_lock_name), expires=self.timeout_seconds, timeout=2):
TypeError: lock() got an unexpected keyword argument 'expires'

According to the docs, it did not seem to be changed from the previous version, but at the source code the signature was changed to:
def lock(self, name=None, ttl=20, block=10):

This leads me to question how many other minor structures changed from previous versions to Zato3, hopefully they are not too many. I will remove the naming from the parameter, to make it backwards compatible and please update the docs to reflect the new naming scheme on https://zato.io/docs/progguide/dist-locks.html.

Thanks!

PS.: Do you prefer for me to open reports for documentation issues on github as well?

dsuch · 2018-07-17 19:50:44 UTC

There should not be this exception - the code should handle new parameters gracefully so this is something that should be fixed and it will be.

Everything that is new or changed is listed in the changelog and if something does not conform to it, feel free to post it here or to GH, as you prefer.

I don’t see why your approach should not work - if MySQL and Redis are separate then this is what matters.

When you install Zato 3.0 over 2.0, you will end up with /opt/zato/2.0 and /opt/zato/3.0 with the /opt/zato/current symlink pointing to the latter and this should also be fine.

The only thing that is shared is the cache that Suds uses for SOAP definitions, it is kept in /tmp/suds, that is where Suds stores it for the whole system, regardless of what Zato environment it is, but if you don’t have Suds-based SOAP connections then it matters not.