SQLAlchemy connections when invoking other services


#1

Hey there!

So I now have a bunch of services in my prototype application and I am using self.invoke() on a number of tier 2 services. All those invoked, tier 1 services access the PostgreSQL database like this (very much simplified version):

with closing(self.outgoing.sql.get(conn).session()) as session:
    result = session.Query()  # Whatever query
    if result:
        # Some code
        self.response.status_code = OK
        self.response.payload = result

So now my tier 2 services both access some information on their own and invoke other tier 1 services. In some cases they just invoke other services and do not access the database on their own. Anyway, the thing is that the block above which (if I am not mistaken):

  • Reuses a connection from the pool.
  • Starts a transaction (in general, there exists services for CRUD operations).
  • Commits or roll backs the transaction, if applicable.
  • Closes the connection (i.e. returns the connection to the pool).

So I was wondering which is the internal behaviour in Zato when you invoke another services which also uses a connection to the database. Is it reused? Is the session (transaction) reused? If not, does it start another transaction on its own?

I am asking all these questions because I am now creating the service that creates a new reservation, which invokes services to:

  • Get a list of extras.
  • Insert or update a guest.
  • Insert a booking.
  • Get information of a room.

And each of these tier 1 services have a connection block such as the one described at the beginning of this post.

P.S. Just incidentally, PostgreSQL features savepoints, which are kind of partial commits of transactions.


#2

Hi @jsabater,

when you run self.invoke(service_name, request), it is guaranteed that you execute it in the very same Python process that your service runs.

This is a stable feature designed in this manner specifically to be able to invoke other services with live Python objects and to be able to have access to full Python stack objects, e.g. you can open an SQL transaction (session) in one service and then send it on input to other services that will make use of it.

In this way, you can treat self.invoke as a simple Python method invocation.

If you want to invoke a service on another server, you need to state it explicitly, e.g. self.invoke(‘service-name@server2’, request) - this will run it on server2, which may be the current one or not, but you cannot provide complex Python objects to it, it must be simple types, such as dicts or JSON.


#3

Okay, so when I want to create a transaction for a number of inserts made by different services, I’ll add a session to the input_data dictionary.

I’ll try this later on today if I find the time, but I was just wondering if I should/would have to add it to the list of optional parameters… :thinking:


#4

So I’ve been trying to make this happen without success so far. The session is being received by the invoked SimpleIO service as a unicode type. There is no point in trying to cast it to Session(). I guess it is normal since input_data is a dictionary.

Could you please elaborate on how to pass the session to the invoked service when using SimpleIO? Perhaps it has to do with the data format (i.e. from zato.common import DATA_FORMAT)? Perhaps a specific, separate parameter of the invoke() method?


#5

AsIs is the datatype that will never alter input or output elements in any way, it is effectively a pass-through one.

For instance, this will ensure that the ‘session’ parameter will not be serialized to string:

from zato.server.service import AsIs

...

class SimpleIO:
  input_required = ...
  input_optional = AsIs('session'),

#6

I tried that, but with no success. I will give it another try and get back to you. Thanks for the help!


#7

So I guess I was not doing it the way it was supposed to because it is working fine now. Cheers! :slight_smile: