(Migrated) problem with zato and fabric

Community support
#1

(This message has been automatically imported from the retired mailing list)

Hello.

I try to install the service in this gist:
https://gist.github.com/njordr/9481215 , but zato gives me the following
error:

2014-03-11 08:33:56,743 - ERROR - 21697:Dummy-29 -
zato.server.service.store:22 - Could not load source
mod_name:[zimbra_queues]
file_name:[/var/zato/qs-1/server1/work/hot-deploy/current/zimbra_queues.py],
e:[Traceback (most recent call last):
File “/usr/local/zato-1.1/zato-server/src/zato/server/service/store.py”,
line 159, in import_services_from_file
mod = imp.load_source(mod_name, file_name)
File “/var/zato/qs-1/server1/work/hot-deploy/current/zimbra_queues.py”,
line 8, in
from fabric.api import run,hide,env
File “/usr/local/zato-1.1/lib/python2.7/site-packages/fabric/api.py”,
line 9, in
from fabric.context_managers import (cd, hide, settings, show, path,
prefix,
File
"/usr/local/zato-1.1/lib/python2.7/site-packages/fabric/context_managers.py",
line 42, in
from fabric.state import output, win32, connections, env
File “/usr/local/zato-1.1/lib/python2.7/site-packages/fabric/state.py”,
line 10, in
from fabric.version import get_version
File “/usr/local/zato-1.1/lib/python2.7/site-packages/fabric/version.py”,
line 105, in
version = get_version(‘short’)
File “/usr/local/zato-1.1/lib/python2.7/site-packages/fabric/version.py”,
line 56, in get_version
sha = git_sha()
File “/usr/local/zato-1.1/lib/python2.7/site-packages/fabric/version.py”,
line 24, in git_sha
return p.communicate()[0]
File “/usr/lib/python2.7/subprocess.py”, line 798, in communicate
return self._communicate(input)
File “/usr/lib/python2.7/subprocess.py”, line 1400, in _communicate
stdout, stderr = self._communicate_with_poll(input)
File “/usr/lib/python2.7/subprocess.py”, line 1430, in
_communicate_with_poll
poller = select.poll()
AttributeError: ‘module’ object has no attribute ‘poll’

I’ve installed fabric system wide and with zato virtualenv pip

Can you help me?

Thank you

#2

On 03/11/2014 08:39 AM, Giovanni Colapinto wrote:

I try to install the service in this gist:
https://gist.github.com/njordr/9481215 , but zato gives me the following
error:

File “/usr/lib/python2.7/subprocess.py”, line 1430, in
_communicate_with_poll
poller = select.poll()
AttributeError: ‘module’ object has no attribute ‘poll’

Hi Giovanni,

I’ve never used fabric but it looks fabric starts subprocesses and that
interferes with gevent’s monkey-patching the standard library’s modules
in order to make them work in an async way.

https://groups.google.com/forum/?escaped_fragment=topic/gevent/IzWhGQHq7n0

Zato doesn’t monkey-patch anything itself, it’s left up to gunicorn to
do it so there’s no way right I could offer for you to selectively turn
off certain parts of the monkey-patching process. It would have to be
developed in git master.

That said, is there any way to tell fabric not to use subprocesses?
Given that Zato uses gevent, starting subprocesses may be not the most
efficient way anyway?

#3

On 03/11/2014 08:39 AM, Giovanni Colapinto wrote:

I try to install the service in this gist:
https://gist.github.com/njordr/9481215 , but zato gives me the following
error:

File “/usr/lib/python2.7/subprocess.py”, line 1430, in
_communicate_with_poll
poller = select.poll()
AttributeError: ‘module’ object has no attribute ‘poll’

Hi Giovanni,

I’ve never used fabric but it looks fabric starts subprocesses and that
interferes with gevent’s monkey-patching the standard library’s modules
in order to make them work in an async way.

https://groups.google.com/forum/?escaped_fragment=topic/gevent/IzWhGQHq7n0

https://github.com/kennethreitz/requests/issues/487

Zato doesn’t monkey-patch anything itself, it’s left up to gunicorn to
do it so there’s no way right I could offer for you to selectively turn
off certain parts of the monkey-patching process. It would have to be
developed in git master.

That said, is there any way to tell fabric not to use subprocesses?
Given that Zato uses gevent, starting subprocesses may be not the most
efficient way anyway?

#4

Thank you for the answer.

I fall in the same problem when I try to use subprocess in a zato service,
so I believe they belong to the same issue.
Is there a “zato way” to launch a process?

Cheers

On Tue, Mar 11, 2014 at 11:25 AM, Dariusz Suchojad dsuch@zato.io wrote:

On 03/11/2014 08:39 AM, Giovanni Colapinto wrote:

I try to install the service in this gist:
https://gist.github.com/njordr/9481215 , but zato gives me the following
error:

File “/usr/lib/python2.7/subprocess.py”, line 1430, in
_communicate_with_poll
poller = select.poll()
AttributeError: ‘module’ object has no attribute ‘poll’

Hi Giovanni,

I’ve never used fabric but it looks fabric starts subprocesses and that
interferes with gevent’s monkey-patching the standard library’s modules
in order to make them work in an async way.

https://groups.google.com/forum/?escaped_fragment=topic/gevent/IzWhGQHq7n0

https://github.com/kennethreitz/requests/issues/487

https://stackoverflow.com/questions/19498835/celery-tasks-not-works-with-gevent

Zato doesn’t monkey-patch anything itself, it’s left up to gunicorn to
do it so there’s no way right I could offer for you to selectively turn
off certain parts of the monkey-patching process. It would have to be
developed in git master.

That said, is there any way to tell fabric not to use subprocesses?
Given that Zato uses gevent, starting subprocesses may be not the most
efficient way anyway?

#5

On 03/11/2014 01:11 PM, Giovanni Colapinto wrote:

I fall in the same problem when I try to use subprocess in a zato service,
so I believe they belong to the same issue.
Is there a “zato way” to launch a process?

I’ve never tried it myself but it looks gevent does support it, as
documented here

http://www.gevent.org/gevent.subprocess.html

Please let me know if this is it.

#6

I can confirm that with gevent it works!!! :smiley:

But I had to upgrade gevent version in zato virtualenv, otherwise it cannot
find gevent subprocess module

Is it ok that I need to launch

ln -s /usr/local/zato-1.1/lib/python2.7/site-packages/easy-install.pth
/usr/local/zato-1.1/eggs/

otherwise zato’s pip exits with errors because it cannot uninstall gevent
and greenlet?

Cheers

On Tue, Mar 11, 2014 at 1:17 PM, Dariusz Suchojad dsuch@zato.io wrote:

On 03/11/2014 01:11 PM, Giovanni Colapinto wrote:

I fall in the same problem when I try to use subprocess in a zato
service,
so I believe they belong to the same issue.
Is there a “zato way” to launch a process?

I’ve never tried it myself but it looks gevent does support it, as
documented here

http://www.gevent.org/gevent.subprocess.html

Please let me know if this is it.

#7

On 03/11/2014 04:39 PM, Giovanni Colapinto wrote:

I can confirm that with gevent it works!!! :smiley:

But I had to upgrade gevent version in zato virtualenv, otherwise it cannot
find gevent subprocess module

Great to hear it - note that the git version already uses gevent 1.0.

Is it ok that I need to launch

ln -s /usr/local/zato-1.1/lib/python2.7/site-packages/easy-install.pth
/usr/local/zato-1.1/eggs/

otherwise zato’s pip exits with errors because it cannot uninstall gevent
and greenlet?

Sure, it is okay for you to customize buildout.cfg and versions.cfg to
your liking. Some people add more dependencies, some only update
selected ones. This is an isolated environment you can tweak so it best
fits your environment.