What is the canonical way to add extra python packages to my Zato server

Hi,

I installed Zato on ubuntu using the debian installer. My project needs the pymssql package. Installing it with
/opt/zato/current/bin/pip install pymssql works like a charm.

Is this the canonical way to add extra packages to your Zato server? Maybe from a provisioning viewpoint there are better alternatives?

Regards,

Jan

Hi @jjmurre,

using pip sounds fine.

There is another way, that of updating versions.cfg and buildout.cfg files but if you are more familiar with pip than buildout then pip install package-name is perfectly fine.

However, if a dependency is not on PyPI nor any other package index (such as your own internal one) then you can place individual Python modules in zato_extra_paths too.

https://zato.io/docs/admin/guide/enabling-extra-libs.html

Thanx, I am a long-time buildout user, I’ll go for buildout.

Is there a particular way to do this in docker? The pip method doesn’t seem to work for me. I am able to install the package but cannot access it from my service module. I am looking into using zato_extra_paths by mounting it as a volume. Would this work?

Thanks

Hello @myke,

yes, the idea with zato_extra_paths is that it is part of PYTHONPATH so anything you place there becomes importable in runtime.

You can also alter the default Dockerfile and customise it as needed - people do it regularly to make small adjustments or additions per their needs.

Also, I am not clear what you mean by the pip-installed module’s not being accessible to your services?

Regards.

Thanks for the response @dsuch

With-in docker I am installing the package as pip3 install package_name which runs fine but from my service I am unable to import any of the modules.

Please always post full exceptions, otherwise it will be impossible to diagnose or discuss anything. Thanks.

My bad!
2020-08-17 18:13:36,364 - WARNING - 2128:DummyThread-689 - zato.hot-deploy.create:0 - No services were deployed from module TestService.py

2020-08-17 18:13:36,379 - WARNING - 2128:DummyThread-689 - zato.server.service:0 - Traceback (most recent call last):
File “/opt/zato/3.1.0/code/zato-server/src/zato/server/service/init.py”, line 660, in update_handle
self._invoke(service, channel)
File “/opt/zato/3.1.0/code/zato-server/src/zato/server/service/init.py”, line 551, in _invoke
service.handle()
File “/opt/zato/3.1.0/code/zato-server/src/zato/server/service/internal/hot_deploy/init.py”, line 270, in handle
for service_id in services_deployed:
TypeError: ‘NoneType’ object is not iterable

When I remove the import statements for those installed modules, I am able to deploy the service

Thanks

OK but what are the imports? Also, please wrap any code in code tags to make them display in a way that is readable, otherwise it is just plain unformatted text.

Below are the imports;

from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials

Thanks

What happens if you log in to the container via SSH and run the “py” command (note that it is not “python”), which will start a Python shell, and then just run the imports in that shell?

From now on, please post everything, all the commands, imports, tracebacks, source code, everything.

docker exec -it 1f9eed6740c1 /bin/bash root@1f9eed6740c1:/opt/zato/3.1.0/code py bash: py: command not found

I used the following though:

root@1f9eed6740c1:/opt/zato/3.1.0/code# python3
Python 3.6.8 (default, Oct 7 2019, 12:59:55) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from apiclient.discovery import build
>>> from oauth2client.service_account import ServiceAccountCredentials
>>> >>>

It appears that you installed the packages using the system-wide Python version, the one that is invoked via python or python3.

Zato uses its own Python installation under virtualenv to be independent of the system packages and to ensure that its own packages do not interfere with the system ones either.

What happens if you run /opt/zato/code/3.1.0/code/bin/python and then try the imports?

If that fails with an ImportError then make sure to pip install your packages using the pip version from the same directory, e.g. /opt/zato/code/3.1.0/code/bin/pip install <package-name>.

1 Like

Thanks @dsuch this has certainly helped. Are there best practices on where to place a plain text file that I would like to access from a service? My service needs to load a file but I am not sure where to place this file.

Thanks

Please open a new forum thread, explaining your use-case - what kind of a file it is, its format and intended usage patterns, e.g. will change often, will it be read-only, served to API clients etc.