(Migrated) error and availability management with zato

(This message has been automatically imported from the retired mailing list)

Hello,

I am currently studying zato as a potential candidate for an ESB connecting
mostly REST and Soap services. I have no real experience yet but I wonder
about a few things. If i missed the points in the documentation, feel free
to send me simple links !

What is the preferred way of dealing with exceptions in a service ? For a
service behind an amqp channel for example we might want any error to be
caught and the original message redirected to a dead letter queue. Should
it be implemented by a large try/catch inside the service or is there
another mechanism to use ?

What is the preferred way of dealing with unavailability of a backend
server ? For example I might want to run a is-alive scheduled task that
will ping regularly a server and de-activate the corresponding service(s)
in zato. That way the service would not consume its amqp channel and the
retry policy of the broker would be used. Is this sort of thing easy with
zato ? Or should it be done differently ?

Thanks

Alban Mouton

On 07/07/2014 08:41 PM, Alban Mouton wrote:

Hello Alban,

What is the preferred way of dealing with exceptions in a service ? For a
service behind an amqp channel for example we might want any error to be
caught and the original message redirected to a dead letter queue. Should
it be implemented by a large try/catch inside the service or is there
another mechanism to use ?

Yes, a try/catch inside the body or hooks would be the way to go. With
the latter, you could store a flag in the service execution’s
environment, called say ‘processed_ok’. An after_handle hook would check
if the flag is present and roll back the message to a DLQ if it’s not.

https://zato.io/docs/progguide/service-dev.html#environ
https://zato.io/docs/progguide/service-hooks.html

Ideally this should be wrapped inside a transaction - i.e. an AMQP
message is not really confirmed to have been taken off a queue until a
service completes successfully.

This is not how it works right now, however. At the moment, there is a
separate AMQP connector process which listens on queue and notifies
services - running in different processes or indeed on different hosts
that form a cluster of Zato servers.

The reason for that behavior is that is that only quite recently the
underlying AMQP libraries for Python became really thread-safe. Zato is
not a threaded server, it’s using gevent’s greenlets but for these
purposes thread-safe = greenlet-safe so put simply, this is a todo on
Zato’s end.

What is the preferred way of dealing with unavailability of a backend
server ? For example I might want to run a is-alive scheduled task that
will ping regularly a server and de-activate the corresponding service(s)
in zato. That way the service would not consume its amqp channel and the
retry policy of the broker would be used. Is this sort of thing easy with
zato ? Or should it be done differently ?

Can you please clarify if by a backend server you mean a Zato server?

thanks a lot,

On 07/07/2014 08:41 PM, Alban Mouton wrote:

Hello Alban,

What is the preferred way of dealing with exceptions in a service ? For a
service behind an amqp channel for example we might want any error to be
caught and the original message redirected to a dead letter queue. Should
it be implemented by a large try/catch inside the service or is there
another mechanism to use ?

Yes, a try/catch inside the body or hooks would be the way to go. With
the latter, you could store a flag in the service execution’s
environment, called say ‘processed_ok’. An after_handle hook would check
if the flag is present and roll back the message to a DLQ if it’s not.

https://zato.io/docs/progguide/service-dev.html#environ
https://zato.io/docs/progguide/service-hooks.html

Ideally this should be wrapped inside a transaction - i.e. an AMQP
message is not really confirmed to have been taken off a queue until a
service completes successfully.

This is not how it works right now, however. At the moment, there is a
separate AMQP connector process which listens on queue and notifies
services - running in different processes or indeed on different hosts
that form a cluster of Zato servers.

The reason for that behavior is that is that only quite recently the
underlying AMQP libraries for Python became really thread-safe. Zato is
not a threaded server, it’s using gevent’s greenlets but for these
purposes thread-safe = greenlet-safe so put simply, this is a todo on
Zato’s end.

What is the preferred way of dealing with unavailability of a backend
server ? For example I might want to run a is-alive scheduled task that
will ping regularly a server and de-activate the corresponding service(s)
in zato. That way the service would not consume its amqp channel and the
retry policy of the broker would be used. Is this sort of thing easy with
zato ? Or should it be done differently ?

Can you please clarify if by a backend server you mean a Zato server?

thanks a lot,