(Migrated) 2.0.1 libraries / django

(This message has been automatically imported from the retired mailing list)

Are there any best practices or recommended approaches for adding additiona=
l packages into the zato virtual environment? We rely on a handful of addi=
tional packages to help out with calling some third party webservices. My =
approach for handling these right now is to maintain a requirements.txt fil=
e and install into the zato venv with the -no-deps switch (as I’ve found th=
at omitting this pulls in “official” versions of some libraries, most notab=
ly requests, which seems to break zato quite badly). This is…a bit unwie=
ldly, and it’s quickly turning into an exercise in “hunt down the dependenc=
y”, plus there is the omnipresent risk of a developer basically wrecking ou=
r environment by forgetting the -no-deps switch.

Additionally, I’m noticing that django appears to be available in the zato =
environment now; this is triggering some odd behavior in a package we use. =
Specifically, if it is able to import django (which it now is), it checks =
a django setting to disable itself, then attempts to use django’s persisten=
ce framework to store some data. As this is unconfigured (rightly so, in f=
act) in the zato environment, this is basically causing every call through =
this library to fail. Is there any way of setting django settings via zato=
so we can force-disable this behavior from the package?

Thanks!
Andrew Todd
Senior Developer

[G&G Outfitters]

4901 Forbes Blvd. | Lanham, MD 20706
P. 240.487.2999 | F. 301.731.5199 | E. atodd@ggoutfitters.com

Click on http://www.ggoutfitters.com for all of your promotional needs!!!

Follow us on [https://secure.sureshiponline.com/catalogs/ggo/GGO-Facebook.p=
ng] https://www.facebook.com/pages/GG-Outfitters/100759079981924 [https:=
//secure.sureshiponline.com/catalogs/ggo/GGO-Linkedin.png] <http://www.link=
edin.com/company/g&g-outfitters> [https://secure.sureshiponline.com/catalo=
gs/ggo/GGO-Twitter.png] http://twitter.com/ggoutfitters

[https://secure.sureshiponline.com/catalogs/ggo/GGO-StayGreen.jpg]Please co=
nsider the environment before printing this e-mail.

This message contains confidential information and is intended only for the=
individual(s) addressed in the message. If you are not the named addressee=
, you should not disseminate, distribute, or copy this e-mail. If you are n=
ot the intended recipient, you are notified that disclosing, distributing, =
or copying this e-mail is strictly prohibited.

Hi Andrew,

all the dependencies are managed through buildout’s files:

  • version.cfg
  • buildout.cfg

When Zato itself or a user needs an additional library it is added to
both files.

For instance, let’s say we develop payments-related services and need to
depend on the ‘securepay’ package.

Hence:

  • Its current version is added to versions.cfg
  • Its name is added to buildout.cfg

So (surrounded with some context) we now have:

versions.cfg:

[versions]

sec-wall = 1.2
secure-pay = 0.3
setproctitle = 1.1.6

buildout.cfg

eggs =

sec-wall
securepay
setproctitle

Now ./bin/buildout is run which installs the dependency at an exact version.

At this point it may very well turn out that this package depends on
other packages or versions other than what we have. Each such dependency
needs to be added/updated in both files and ./bin/buildout needs to
execute again.

At times ./bin/buildout may disappear - this is a buildout’s expected,
though difficult to understand, behaviour. In that situation, one needs
to run bootstrap.py again which will bring buildout about:

$ ./bin/python bootstrap.py -v 1.7.0

There is really no other way - the process takes initially 30 minutes if
there are several or more dependencies but once they are fixed at exact
versions, the build becomes repeatable.

The only exception appears to be setuptools - apparently one is supposed
to get used to its disappearing from time to time and needs to fix the
build occasionally:

https://mail.python.org/pipermail/distutils-sig/2013-June/021098.html

This is, incidentally, one of the reasons we have self-contained DEBs or
RPMs and don’t install anything using ‘pip install zato’.

As for Django - it was always available in Zato’s PYTHONPATH, we
depended on it since the very beginning.

What did change though is that, when a binary DEB/RPM is being created,
django-settings is now installed from a GH checkout rather than a PyPI
release because that was one of the packages that was not compatible
with PEP 440 and suddenly one could not install it.

But the only change the fork did was to update the version - 1.3.11
instead of 1.3-11.

Unfortunately, the situation with packaging for Python is less than
ideal and binary DEB/RPM packages shield you from it to a great extent -
it’s simply my colleagues who need to deal with it instead of Zato users

  • but from time to time one needs to face it.

And ultimately, having literally spent weeks on packaging issues, which
is way, way too much time - the least time consuming way turned out to
be depend on exact versions in buildout.