(Migrated) Need a pointer on how to make zato aware of pyodbc

(This message has been automatically imported from the retired mailing list)

Hi,
I am experimenting with zato, trying to create a SIO service that will
expose some records from a sql server I have running.

Must have missed something as I cannot get pyodbc to work. When
hot-deploying the server logs on my quickstart cluster complains about
pyodbc missing.

2014-07-08 09:44:17,511 - ERROR - 2238:Dummy-104 -
zato.server.service.store:22 - Could not load source
mod_name:[getLedgersByRunDate]
file_name:[/opt/zato/tmp/q/server1/work/hot-deploy/current/getLedgersByRunDate.py],
e:[Traceback (most recent call last):
File “/opt/zato/1.1/zato-server/src/zato/server/service/store.py”, line
159, in import_services_from_file
mod = imp.load_source(mod_name, file_name)
File
"/opt/zato/tmp/q/server1/work/hot-deploy/current/getLedgersByRunDate.py",
line 2, in
import pyodbc
ImportError: No module named pyodbc
]

Now, pyodbc works great in any other non-zato python script I write.

Anyone got a tip for me? Thanx

-Bad

On 07/08/2014 09:51 AM, Baad Sequel wrote:

ImportError: No module named pyodbc
]

Now, pyodbc works great in any other non-zato python script I write.

Hi Baad,

Zato’s dependencies are in 99% isolated from the underlying OS. The only
ones that it shares with your OS are scipy and numpy.

Hence pyodbc may be available in your default Python’s installation but,
given that it’s not bundled with Zato, you can’t make use of it unless
you install it additionally.

The steps are below, assuming Zato binaries are in /opt/zato/1.1/

  • Open /opt/zato/1.1/versions.cfg

  • Add the version of pyodbc you’d like to make use of

  • Close the file

  • Open /opt/zato/1.1/buildout.cfg

  • Add pyodbc to zato.eggs stanza

  • Close the file

  • cd /opt/zato/1.1

  • ./bin/buildout

  • Stop and start the environment again

This will download and install pyodbc for you under /opt/zato/1.1/eggs
and make it available on PYTHONPATH for your services to import it.

I’m not sure if pyodbc has any dependencies - if there are any they will
have to be added to versions.cfg and buildout.cfg as well.

Please post a message if this helps or perhaps you need assistance.

thanks a lot,

On 07/08/2014 09:51 AM, Baad Sequel wrote:

ImportError: No module named pyodbc
]

Now, pyodbc works great in any other non-zato python script I write.

Hi Baad,

Zato’s dependencies are in 99% isolated from the underlying OS. The only
ones that it shares with your OS are scipy and numpy.

Hence pyodbc may be available in your default Python’s installation but,
given that it’s not bundled with Zato, you can’t make use of it unless
you install it additionally.

The steps are below, assuming Zato binaries are in /opt/zato/1.1/

  • Open /opt/zato/1.1/versions.cfg

  • Add the version of pyodbc you’d like to make use of

  • Close the file

  • Open /opt/zato/1.1/buildout.cfg

  • Add pyodbc to zato.eggs stanza

  • Close the file

  • cd /opt/zato/1.1

  • ./bin/buildout

  • Stop and start the environment again

This will download and install pyodbc for you under /opt/zato/1.1/eggs
and make it available on PYTHONPATH for your services to import it.

I’m not sure if pyodbc has any dependencies - if there are any they will
have to be added to versions.cfg and buildout.cfg as well.

Please post a message if this helps or perhaps you need assistance.

thanks a lot,

Hi,

How would I go about doing this for Zato 3.1 ?
Seems to me that requirements.txt has replaced versions.cfg and there is no longer any /bin/buildout

I do see the script update.sh doing pip install -r on requirements.txt so perhaps this is the new route to making Zato aware of pyodbc ?

Just need to ask before I potentially wreck the test environment I have up and running now… :smiley:

Hi, @badsequel!

Yes, that would be the case. But take into consideration Python 2.7 and Python 3.x, as some packages are only compatible with the former or the latter, or both. Or they have (slightly) different names. For packages specific to 2.7 or 3, you have to use _req_py27.txt and _req_py3.txt, respectively.

Also keep in mind that these files are part of the repository, so maybe you want to add a new, custom file for your installation and keep it up separately. As long as you are installing packages as the zato user you’ll be inside the virtualenv it uses.

Let us know how it goes :slight_smile:

Thanks for the comments @jsabater , I really appreciate it!

My Zato 3.1 test environment is now aware of pyodbc and happily fetching rows from a ms sql server.

What I did with the quickstart cluster I am toying with is this:

  • stop the cluster
  • copy the original requirements.txt to a backup file
  • add pyodbc==4.0.26 to requirements.txt
  • cp zato-qs-start.sh to zato-qs-start_post_update.sh
  • nano zato-qs-start_post_update.sh : add the --sync_internal flag to the server startup lines
  • run update.sh
  • run zato-qs-start_post_update.sh
  • test with a service that fetches a couple rows from a ms sql server and dump them to server log.
  • celebrate with a glass of wine.

Edit: forgot a step, fixed a typo

Just incidentally, I tend to wait for the first server to finish its start-up process when using –sync-internal before starting the second server (around 30 seconds on my Zato 3.0 set-up). As per the release notes, now with Zato 3.1 that start-up time has been reduced, so maybe starting server2 right after server1 can be scripted now the way you’ve done it. :+1: