Quickstart server working properly now using sqlite. Thanks for the quick solution!
Will continue my tests and report back any new findings. If nothing blocker is find, I will recreate my preproduction environment and test my services against it, mainly trying to check if the scheduler issues stop happening.
Also there was a question which was not answered about the migration from Zato 2 to 3 on my production environment. Assuming the tests are OK and everything is working OK, how would you see the smoothest path to have the new version working with possibilities to rollback to the previous version if something breaks? My machines are baremetal, no VMs and no possibilities of easy snapshot rollbacks like my test environment.
Any suggestions are appreciated.
Thanks again for the patch!
EDIT: We celebrated too soon… the servers are giving errors when starting:
2018-07-16 15:30:14,954 - INFO - 6749:MainThread - gunicorn.main:176 - Starting gunicorn 18.0
2018-07-16 15:30:14,958 - INFO - 6749:MainThread - gunicorn.main:176 - Listening at: http://0.0.0.0:17010 (6749)
2018-07-16 15:30:14,961 - INFO - 6749:MainThread - gunicorn.main:176 - Using worker: gevent
2018-07-16 15:30:14,976 - INFO - 6763:MainThread - gunicorn.main:176 - Booting worker with pid: 6763
2018-07-16 15:30:16,245 - INFO - 6763:MainThread - zato.server.base.parallel:422 - Preferred address of `server1@quickstart-349993` (pid: 6763) is `http://10.0.2.15:17010`
2018-07-16 15:30:17,610 - ERROR - 6763:MainThread - gunicorn.main:182 - Exception in worker process:
Traceback (most recent call last):
File "/opt/zato/3.0/code/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker
self.cfg.post_fork(self, worker)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 781, in post_fork
ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 428, in start_server
self.set_up_config(server)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/config.py", line 191, in set_up_config
query = self.odb.get_channel_web_socket_list(server.cluster.id, True)
File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/api.py", line 1134, in get_channel_web_socket_list
return query.channel_web_socket_list(session, cluster_id, needs_columns)
File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 92, in inner
tool = _SearchWrapper(func(*args), **kwargs)
File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 67, in __init__
self.total = q.session.execute(total_q).scalar()
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1176, in execute
bind, close_with_result=True).execute(clause, params or {})
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 508, in do_execute
cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) no such column: sec_vault_conn_1.id [SQL: u'SELECT count(*) AS count_1 \nFROM cluster, channel_web_socket LEFT OUTER JOIN service ON service.id = channel_web_socket.service_id LEFT OUTER JOIN sec_base ON sec_base.id = channel_web_socket.security_id LEFT OUTER JOIN (sec_base AS sec_base_1 JOIN sec_vault_conn AS sec_vault_conn_1 ON sec_base_1.id = sec_vault_conn_1.id) ON sec_base.id = sec_vault_conn_1.id \nWHERE cluster.id = channel_web_socket.cluster_id AND cluster.id = ?'] [parameters: (1,)] (Background on this error at: http://sqlalche.me/e/e3q8)
Traceback (most recent call last):
File "/opt/zato/3.0/code/lib/python2.7/site-packages/gunicorn/arbiter.py", line 495, in spawn_worker
self.cfg.post_fork(self, worker)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 781, in post_fork
ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 428, in start_server
self.set_up_config(server)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/config.py", line 191, in set_up_config
query = self.odb.get_channel_web_socket_list(server.cluster.id, True)
File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/api.py", line 1134, in get_channel_web_socket_list
return query.channel_web_socket_list(session, cluster_id, needs_columns)
File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 92, in inner
tool = _SearchWrapper(func(*args), **kwargs)
File "/opt/zato/3.0/code/zato-common/src/zato/common/odb/query/__init__.py", line 67, in __init__
self.total = q.session.execute(total_q).scalar()
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1176, in execute
bind, close_with_result=True).execute(clause, params or {})
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/opt/zato/3.0/code/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 508, in do_execute
cursor.execute(statement, parameters)
OperationalError: (sqlite3.OperationalError) no such column: sec_vault_conn_1.id [SQL: u'SELECT count(*) AS count_1 \nFROM cluster, channel_web_socket LEFT OUTER JOIN service ON service.id = channel_web_socket.service_id LEFT OUTER JOIN sec_base ON sec_base.id = channel_web_socket.security_id LEFT OUTER JOIN (sec_base AS sec_base_1 JOIN sec_vault_conn AS sec_vault_conn_1 ON sec_base_1.id = sec_vault_conn_1.id) ON sec_base.id = sec_vault_conn_1.id \nWHERE cluster.id = channel_web_socket.cluster_id AND cluster.id = ?'] [parameters: (1,)] (Background on this error at: http://sqlalche.me/e/e3q8)
2018-07-16 15:30:17,615 - INFO - 6763:MainThread - gunicorn.main:176 - Worker exiting (pid: 6763)
2018-07-16 15:30:17,623 - ERROR - 6763:DummyThread-2 - springpython.context.ApplicationContext:109 - Could not destroy object 'server', exception 'Traceback (most recent call last):
File "/opt/zato/3.0/code/lib/python2.7/site-packages/springpython/context/__init__.py", line 106, in shutdown_hook
destroy_method()
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 829, in destroy
self.invoke('zato.channel.web-socket.client.delete-by-server')
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/__init__.py", line 696, in invoke
*args, **kwargs)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/__init__.py", line 1281, in invoke
}, channel, None, needs_response=True, serialize=serialize)
File "/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/__init__.py", line 1342, in on_message_invoke_service
service, is_active = self.server.service_store.new_instance_by_name(msg['service'])
File "/opt/zato/3.0/code/zato-server/src/zato/server/service/store.py", line 204, in new_instance_by_name
impl_name = self.name_to_impl_name[name]
KeyError: u'zato.channel.web-socket.client.delete-by-server'
'
2018-07-16 15:30:18,326 - INFO - 6749:MainThread - gunicorn.main:176 - Shutting down: Master
2018-07-16 15:30:18,327 - INFO - 6749:MainThread - gunicorn.main:176 - Reason: Worker failed to boot.
Related to sqlite patch still?