Zato server is not starting with the latest git pull


#1

Hi,

We are receiving the following error on git pull and doing sync internal

2018-10-26 13:10:19,194 - INFO - 4448:MainThread - gunicorn.main:176 - Starting gunicorn 18.0
2018-10-26 13:10:19,197 - INFO - 4448:MainThread - gunicorn.main:176 - Listening at: http://0.0.0.0:17010 (4448)
2018-10-26 13:10:19,197 - INFO - 4448:MainThread - gunicorn.main:176 - Using worker: gevent
2018-10-26 13:10:19,201 - INFO - 4458:MainThread - gunicorn.main:176 - Booting worker with pid: 4458
2018-10-26 13:10:19,608 - INFO - 4458:MainThread - zato.server.base.parallel:421 - Preferred address of server1@perch (pid: 4458) is http://10.0.2.15:17010
2018-10-26 13:10:20,405 - ERROR - 4458:MainThread - gunicorn.main:182 - Exception in worker process:
Traceback (most recent call last):
** File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/gunicorn/arbiter.py”, line 495, in spawn_worker**
** self.cfg.post_fork(self, worker)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 781, in post_fork**
** ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 427, in start_server**
** self.set_up_config(server)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/config.py”, line 325, in set_up_config**
** hs_item[‘match_target_compiled’] = Matcher(hs_item[‘match_target’], hs_item.get(‘match_slash’, ‘’))**
** File “src/zato/cy/url_dispatcher.pyx”, line 54, in src.zato.cy.url_dispatcher.Matcher.init**
TypeError: init() takes exactly 1 positional argument (2 given)
Traceback (most recent call last):
** File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/gunicorn/arbiter.py”, line 495, in spawn_worker**
** self.cfg.post_fork(self, worker)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 781, in post_fork**
** ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 427, in start_server**
** self.set_up_config(server)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/config.py”, line 325, in set_up_config**
** hs_item[‘match_target_compiled’] = Matcher(hs_item[‘match_target’], hs_item.get(‘match_slash’, ‘’))**
** File “src/zato/cy/url_dispatcher.pyx”, line 54, in src.zato.cy.url_dispatcher.Matcher.init**
TypeError: init() takes exactly 1 positional argument (2 given)
2018-10-26 13:10:20,406 - INFO - 4458:MainThread - gunicorn.main:176 - Worker exiting (pid: 4458)
2018-10-26 13:10:20,408 - ERROR - 4458:DummyThread-2 - springpython.context.ApplicationContext:109 - Could not destroy object ‘server’, exception 'Traceback (most recent call last):
** File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/springpython/context/init.py”, line 106, in shutdown_hook**
** destroy_method()**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 833, in destroy**
** self.invoke(‘zato.channel.web-socket.client.delete-by-server’)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 696, in invoke**
** args, kwargs)
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/init.py”, line 1311, in invoke
*
** }, channel, None, needs_response=True, serialize=serialize)**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/init.py”, line 1372, in on_message_invoke_service**
** service, is_active = self.server.service_store.new_instance_by_name(msg[‘service’])**
** File “/opt/zato/3.0/code/zato-server/src/zato/server/service/store.py”, line 217, in new_instance_by_name**
** impl_name = self.name_to_impl_name[name]**
KeyError: u’zato.channel.web-socket.client.delete-by-server’

2018-10-26 13:10:20,663 - INFO - 4448:MainThread - gunicorn.main:176 - Shutting down: Master
2018-10-26 13:10:20,663 - INFO - 4448:MainThread - gunicorn.main:176 - Reason: Worker failed to boot.

The zato server is not getting started because of the above error. Not sure of the root cause.

What could be the possible issue here?

Regards,
Sai Bhargav


#2

You did not run all the steps, one of them is compilation of the code that you pull from GitHub:

$ cd ./code
$ ./bin/pip install -e ./zato-cy

#3

Hi dsuch,

I have followed the steps in the document and restarted zato
steps followed

  1. git pull
  2. ./bin/pip install -e ./zato-cy
  3. ran sync internal
  4. stopped zato
  5. started zato

and now we are getting the following error

2018-11-05 09:51:43,770 - ERROR - 9669:MainThread - gunicorn.main:182 - Exception in worker process:
Traceback (most recent call last):
File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/gunicorn/arbiter.py”, line 495, in spawn_worker
self.cfg.post_fork(self, worker)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 781, in post_fork
ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 430, in start_server
is_first, locally_deployed = self._after_init_common(server)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 344, in _after_init_common
is_first, locally_deployed = self.maybe_on_first_worker(server, self.kvdb.conn)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 276, in maybe_on_first_worker
locally_deployed = import_initial_services_jobs(is_first)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 242, in import_initial_services_jobs
self.internal_service_modules, self.base_dir, self.sync_internal, is_first))
File “/opt/zato/3.0/code/zato-server/src/zato/server/service/store.py”, line 276, in import_internal_services
items = bunchify(dill_load(f))
File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/dill/dill.py”, line 250, in load
obj = pik.load()
File “/usr/lib/python2.7/pickle.py”, line 864, in load
dispatchkey
File “/usr/lib/python2.7/pickle.py”, line 1096, in load_global
klass = self.find_class(module, name)
File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/dill/dill.py”, line 406, in find_class
return StockUnpickler.find_class(self, module, name)
File “/usr/lib/python2.7/pickle.py”, line 1132, in find_class
klass = getattr(mod, name)
AttributeError: ‘module’ object has no attribute ‘GetToken’
Traceback (most recent call last):
File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/gunicorn/arbiter.py”, line 495, in spawn_worker
self.cfg.post_fork(self, worker)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 781, in post_fork
ParallelServer.start_server(worker.app.zato_wsgi_app, arbiter.zato_deployment_key)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 430, in start_server
is_first, locally_deployed = self._after_init_common(server)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 344, in _after_init_common
is_first, locally_deployed = self.maybe_on_first_worker(server, self.kvdb.conn)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 276, in maybe_on_first_worker
locally_deployed = import_initial_services_jobs(is_first)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 242, in import_initial_services_jobs
self.internal_service_modules, self.base_dir, self.sync_internal, is_first))
File “/opt/zato/3.0/code/zato-server/src/zato/server/service/store.py”, line 276, in import_internal_services
items = bunchify(dill_load(f))
File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/dill/dill.py”, line 250, in load
obj = pik.load()
File “/usr/lib/python2.7/pickle.py”, line 864, in load
dispatchkey
File “/usr/lib/python2.7/pickle.py”, line 1096, in load_global
klass = self.find_class(module, name)
File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/dill/dill.py”, line 406, in find_class
return StockUnpickler.find_class(self, module, name)
File “/usr/lib/python2.7/pickle.py”, line 1132, in find_class
klass = getattr(mod, name)
AttributeError: ‘module’ object has no attribute ‘GetToken’
2018-11-05 09:51:43,770 - INFO - 9669:MainThread - gunicorn.main:176 - Worker exiting (pid: 9669)
2018-11-05 09:51:43,775 - ERROR - 9669:DummyThread-1 - springpython.context.ApplicationContext:109 - Could not destroy object ‘server’, exception ‘Traceback (most recent call last):
File “/opt/zato/3.0/code/local/lib/python2.7/site-packages/springpython/context/init.py”, line 106, in shutdown_hook
destroy_method()
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 833, in destroy
self.invoke(‘zato.channel.web-socket.client.delete-by-server’)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/parallel/init.py”, line 696, in invoke
*args, **kwargs)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/init.py”, line 1311, in invoke
}, channel, None, needs_response=True, serialize=serialize)
File “/opt/zato/3.0/code/zato-server/src/zato/server/base/worker/init.py”, line 1372, in on_message_invoke_service
service, is_active = self.server.service_store.new_instance_by_name(msg[‘service’])
File “/opt/zato/3.0/code/zato-server/src/zato/server/service/store.py”, line 217, in new_instance_by_name
impl_name = self.name_to_impl_name[name]
KeyError: u’zato.channel.web-socket.client.delete-by-server’

2018-11-05 09:51:44,055 - INFO - 9656:MainThread - gunicorn.main:176 - Shutting down: Master
2018-11-05 09:51:44,055 - INFO - 9656:MainThread - gunicorn.main:176 - Reason: Worker failed to boot.

Regards,
Sai Bhargav


#4

If anything does not work as expected, you need to send in the exact commands used and the exact output they produced.


#5

Hi dsuch,

I have sorted it out. There was a problem with the packages.

I will make sure from next time will send the detailed output of the commands executed. Thanks for your help

Thanks,
Sai Bhargav