(Migrated) Upgrade to zato 2.0.6

(This message has been automatically imported from the retired mailing list)

Hi I’ve some modules and functions that must be accesible for all services.
Reading the doc and mailing list give the solution to put those modules in zato_extra_paths.
After the upgrade with apt-get upgrade , those modules disappear, I made again the symbolic link and appear again.

Is it possible to make the package that don’t delete remove that folder

Regards
Tolo

What about have the update in the same folder without create a new
virtualenv for the new zato version?. That would be the case needed when
you install more packages through pip in the previous zato installation
that will be lost when the upgrade happens and will require you to install
all of those again.

On Thu, Oct 22, 2015 at 11:52 AM, Dariusz Suchojad dsuch@zato.io wrote:

On 22/10/15 17:40, Brian Candler wrote:

I would vote for either
/opt/zato/zato_extra_paths

Ok, so let’s keep it here then. The parent directory of servers doesn’t
have to be the same one for all servers, I’ve seen things like a common
zato group with zato1 and zato2 users and servers in:

/home/zato1/env/server
/home/zato2/env/server

thanks,


Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python

Indeed

On Thu, Oct 22, 2015 at 12:29 PM, Dariusz Suchojad dsuch@zato.io wrote:

On 22/10/15 19:28, Axel Mendoza Pupo wrote:

What about have the update in the same folder without create a new
virtualenv for the new zato version?. That would be the case needed when
you install more packages through pip in the previous zato installation
that will be lost when the upgrade happens and will require you to
install all of those again.

You mean like keeping everything in /opt/zato/2.0 and updating in place?


Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python

And what about packages installed through pip on the zato virtualenv? need
to get installed again, just like zato_extra_path

On Thu, Oct 22, 2015 at 1:09 PM, Dariusz Suchojad dsuch@zato.io wrote:

On 22/10/15 19:32, Axel Mendoza Pupo wrote:

Indeed

But then we’d need to think about edge cases like stale .pyc files left
over from previous installs or other things we don’t even have to
consider now.

Let’s keep the binaries as they are, there is no need touch anything
regarding their location, it’s fine as it is.

The only thing is zato_extra_path and simply moving it one level up to
/opt/zato should do the trick nicely.

thanks,


Dariusz Suchojad

https://zato.io
ESB, SOA, REST, APIs and Cloud Integrations in Python

On 22/10/15 15:55, tolo.trm wrote:

Hi I’ve some modules and functions that must be accesible for all services.
Reading the doc and mailing list give the solution to put those modules in zato_extra_paths.
After the upgrade with apt-get upgrade , those modules disappear, I made again the symbolic link and appear again.

Is it possible to make the package that don’t delete remove that folder

Hi Tolo,

your code is still safe in /opt/zato/backup/2.0.5/zato_extra_paths so
it’s not lost for good.

There was also an idea of making /opt/zato/2.0-current that would
symlink to 2.0.6, 2.0.7 or another version as needed - in that manner if
you had deployment scripts placing the common code in zato_extra_paths
the scripts would not need to understand what the latest version is.
We’ll get to that idea as well - thanks for raising the point!

On 22/10/2015 14:55, tolo.trm wrote:

Hi I’ve some modules and functions that must be accesible for all services.
Reading the doc and mailing list give the solution to put those modules in zato_extra_paths.
After the upgrade with apt-get upgrade , those modules disappear, I made again the symbolic link and appear again.

Is it possible to make the package that don’t delete remove that folder
https://github.com/zatosource/zato/issues/447

In the mean time, you need to pin the zato package so it doesn’t
auto-upgrade. Under Debian/Ubuntu:

 sudo apt-mark hold zato

Then when you are ready to upgrade: unhold, do the upgrade, recreate
zato_extra_paths sylinks, hold.

Regards,

Brian.

On 22/10/2015 16:17, Dariusz Suchojad wrote:

If everyone is happy with it, we can try adding zato_extra_paths on the
level of servers in addition to that of binaries.
I’m not quite sure what’s meant by that. Do you mean current directory
where you run ./zato-qs-start.sh etc?

Otherwise, I think /opt/zato/zato_extra_paths would be fine.

The issue is that zato seems to use this directory for its own purposes
(symlinks to numpy and scipy) as well as user packages. Does zato
actually need numpy and scipy? If not, it could just leave this
directory alone, and anyone who wants it can install their own symlinks.

On 22/10/2015 16:30, Dariusz Suchojad wrote:

For instance, if you have servers in …

/home/zato/env/server1
/home/zato/env/server2

… then there would be two new directories like …

/home/zato/env/server1/zato_extra_paths
/home/zato/env/server2/zato_extra_paths

… that would not be touched when binaries in /opt/zato/2.0.x change.

The drawback is that there will be multiple places to keep in sync,
multiple zato_extra_paths directories, one for each server.
I see. That’s less convenient (and easy to get wrong if expanding the
number of servers)

I would vote for either
/opt/zato/zato_extra_paths

or the parent directory which contains the servers, e.g. in the above
example
/home/zato/env/zato_extra_paths

On 22/10/15 17:12, Brian Candler wrote:

In the mean time, you need to pin the zato package so it doesn’t
auto-upgrade. Under Debian/Ubuntu:

sudo apt-mark hold zato

Then when you are ready to upgrade: unhold, do the upgrade, recreate
zato_extra_paths sylinks, hold.

If everyone is happy with it, we can try adding zato_extra_paths on the
level of servers in addition to that of binaries.

The advantage of keeping in along with binaries is that there is one
place the extra libraries are stored in. But I can see how it needs some
attention during upgrades.

On 22/10/15 17:24, Brian Candler wrote:

On 22/10/2015 16:17, Dariusz Suchojad wrote:

If everyone is happy with it, we can try adding zato_extra_paths on the
level of servers in addition to that of binaries.
I’m not quite sure what’s meant by that. Do you mean current directory
where you run ./zato-qs-start.sh etc?

For instance, if you have servers in …

/home/zato/env/server1
/home/zato/env/server2

… then there would be two new directories like …

/home/zato/env/server1/zato_extra_paths
/home/zato/env/server2/zato_extra_paths

… that would not be touched when binaries in /opt/zato/2.0.x change.

The drawback is that there will be multiple places to keep in sync,
multiple zato_extra_paths directories, one for each server.

Otherwise, I think /opt/zato/zato_extra_paths would be fine.

Yes, that’s a good idea - it would be used by user code then without
Zato’s interfering with it during upgrades.

The issue is that zato seems to use this directory for its own purposes
(symlinks to numpy and scipy) as well as user packages. Does zato
actually need numpy and scipy? If not, it could just leave this
directory alone, and anyone who wants it can install their own symlinks.

Yes, they are indirect dependencies required to output statistics. They
are inconvenient to compile from source/PyPI and it’s a major gain to
simply symlink to them.

On 22/10/15 17:40, Brian Candler wrote:

I would vote for either
/opt/zato/zato_extra_paths

Ok, so let’s keep it here then. The parent directory of servers doesn’t
have to be the same one for all servers, I’ve seen things like a common
zato group with zato1 and zato2 users and servers in:

/home/zato1/env/server
/home/zato2/env/server

thanks,

On 22/10/15 19:28, Axel Mendoza Pupo wrote:

What about have the update in the same folder without create a new
virtualenv for the new zato version?. That would be the case needed when
you install more packages through pip in the previous zato installation
that will be lost when the upgrade happens and will require you to
install all of those again.

You mean like keeping everything in /opt/zato/2.0 and updating in place?

On 22/10/15 19:32, Axel Mendoza Pupo wrote:

Indeed

But then we’d need to think about edge cases like stale .pyc files left
over from previous installs or other things we don’t even have to
consider now.

Let’s keep the binaries as they are, there is no need touch anything
regarding their location, it’s fine as it is.

The only thing is zato_extra_path and simply moving it one level up to
/opt/zato should do the trick nicely.

thanks,

On 22/10/15 21:25, Axel Mendoza Pupo wrote:

And what about packages installed through pip on the zato virtualenv?
need to get installed again, just like zato_extra_path

Yes, I can see where you’re coming from - something would have to be
done about this part too.

Sorry, I don’t have anything in particular in mind right now, I’m just
confirming I recognize your need.

thanks again,