Hi everyone,
I would like to integrate opentelemetry to zato. Is there any tutorial or useful links about this??
Thank you guyz for your help
Hi Steve,
I can see that there is a Python API for Open Telemetry but I do not have experience with using it.
One you learn more yourself, be sure to leave a message here in the forum with details of how you did it.
Regards.
One idea is to create a zato service that uses the python OT sdk and api to collect and route metrics.
Then, in other services, you can call it like Zato | Invoking other services async_invoke
In addition, for all the internal data and metrics for zato itself, logs, loadbalancer, etc, you can create a service that also feeds those into the OT zato service. This is a bit of work, but it’s one way to do it. And, these OT services could be re-used across zato instances.
1 Like
Thanks for your suggestion.
I’m trying to implement it. I’ll let you know when finished.
I also have a question about starting of zato. Is there any kind of python file that Zato run first before staring or before running all services??
Thanks for your help
Hi Steve,
a server can be configured to run one or more startup service after the server has already started.
To configure it, open file “server.conf” and find an entry called “startup_services_any_worker”, as below:
Each line in this stanza is a key=value pair. Each key is the name of a service that you would like to execute when the server starts. Each value is the service’s input. The value can be omitted in which case a given service will not have any input.
Note that, by default, there will be other built-in services configured to run at startup. You need to keep them where they are, do not delete them from “startup_services_any_worker”.
[startup_services_any_worker]
my.service.1=my.input.1
my.service.2=my.input.2
my.service.3=
Regards.
Hi guys.
Finaly, I’ve created a service that is run once when zato starts. I’ve set the name in the server.conf file at “startup_services_any_worker”. The service create a tracer and connect to the opentelemetry collector at the begenning. It works. I just need to stabilize some things and it will be good 
I am also trying to export some metrics to prometheus. But it seems like this kind of solution doesn’t work with it.
So I would like to know if you have any hint to give about how to add prometheus metrics exportation on zato
Thanks you for your help!
Someone else might have other ideas. But I’d be tempted to add a dependency for GitHub - prometheus/client_python: Prometheus instrumentation library for Python applications and try to use that from a service
I’ve seen this example. But I’m not sure how to integrate it. Let’s suppose we have the following service
from zato.server.service import Service
from prometheus_client import start_http_server, Summary
import random
import time
# Create a metric to track time spent and requests made.
REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request')
class PrometheusService(Service):
@REQUEST_TIME.time()
def process_request(self, t):
"""A dummy function that takes some time."""
time.sleep(t)
def handle(self):
# Decorate function with metric.
start_http_server(8000)
# Generate some requests.
while True:
self.process_request(random.random())
How would you call this service? Because start_http_server need to be up even after the end of service call?
If I understand it correctly, you need to call start_http_server in a method that will act as an infinite loop. You cannot do it in def handle because that method must return, i.e. it cannot run ad infinitum.
Perhaps a way to go forward with start_http_server is to call it through a method in a startup service? That is, you would have another startup service and that service would call start_http_server before the actual services run?
Regards.
Hello everybody
Sorry for the late reply. I was trying to figure this things out.
Finaly, I have found another solution which requires to install your own package. I don’t know why but it seems like zato does not run startup_services_any_worker services.
The second reason to create a package is because it is much easier to create every metric instance in it so that the start_http_server would detect them systematically and add them all along. If you create them outside the package, you’ll have a problem to register them in the prometheus registry. So when zato will call a service, it will not declare the metric
Another reason is because
So I post my solution for anyone who would lik to. integrate prometheus to zato
Step 1: create a prometheus package
folder structure:
/- zatoprom
/— init.py
# package zatoprom
import os
from typing import Literal
import logging
import prometheus_client
from prometheus_client import Counter, Gauge, Summary, Histogram, Enum, Info, start_wsgi_server, write_to_textfile
log = logging.getLogger(__name__)
class CreateMonitoringClient:
registry = None
metrics_dict = {} # JSON that will contain all metrics created
@classmethod
def mwz_initiate(cls) :
if cls.registry is None:
# This part needs to be set one time at the start of the server
log.debug("PROMOTHEUS-INITIALIZATION")
cls.registry = prometheus_client.REGISTRY
# initialize a metric systematically at starting of zato
cls.set_metric(
"Counter",
metric_name="starting_metric",
documentation="Number of request on zato",
labels=('type',)
)
start_wsgi_server(8000, registry=cls.registry)
os.environ['IS_PROMETHEUS_SET'] = "1"
return cls.registry
@classmethod
def set_metric(cls, metric_type: Literal["Counter", "Gauge", "Summary", "Histogram", "Info", "Enum"], metric_name: str,documentation: str, labels: tuple= None, states: list = None) -> prometheus_client.metrics:
cls.mwz_initiate()
if cls.metrics_dict.get(metric_name, None) is not None:
log.debug("PROMOTHEUS-METRIC_FOUND")
return cls.metrics_dict[custom_metric_name]
log.debug("PROMOTHEUS-SET_METRIC")
if metric_type == "Counter":
cls.metrics_dict[metric_name] = Counter(
name=metric_name,
documentation=documentation,
labelnames=labels if labels is not None else (),
registry=cls.registry,
)
elif metric_type == "Gauge":
cls.metrics_dict[metric_name] = Gauge(
name=metric_name,
documentation=documentation,
labelnames=labels if labels is not None else (),
registry=cls.registry,
)
elif metric_type == "Summary":
cls.metrics_dict[metric_name] = Summary(
name=metric_name,
documentation=documentation,
labelnames=labels if labels is not None else (),
registry=cls.registry,
)
elif metric_type == "Histogram":
cls.metrics_dict[metric_name] = Histogram(
name=metric_name,
documentation=documentation,
registry=cls.registry,
)
elif metric_type == "Info":
cls.metrics_dict[metric_name] = Info(
name=metric_name,
documentation=documentation,
labelnames=labels if labels is not None else (),
registry=cls.registry,
)
elif metric_type == "Enum":
cls.metrics_dict[metric_name] = Enum(
name=metric_name,
documentation=documentation,
states=states if states is not None else (),
registry=cls.registry,
)
return cls.metrics_dict[metric_name]
Step 2: Create a metric in your service and call your package
from __future__ import absolute_import, division, print_function, unicode_literals
import json
import os
from zato.server.service import Service
from zatoprom import CreateMonitoringClient
class TestPrometheus(Service):
name = 'test-prometheus'
def handle(self):
self.logger.info('TEST_PROMETHEUS_START | STATUS= %s ' % os.getenv('IS_PROMETHEUS_SET', None))
self.response.payload = json.dumps(
{
'prometheus_env_status': os.getenv('IS_PROMETHEUS_SET', None),
'message': 'check if prometheus is set'
}
)
counter = CreateMonitoringClient.set_metric("Counter", self.name.replace('-', '_'), "%s test counter" % self.name)
counter.inc()
self.logger.info('TEST_PROMETHEUS_FINISH')
For the moment, this is the solution that works for us. I’ll let you if there is an update. If you have an easier solution, please let us know, we will be happy to test