(Migrated) Issue with unicode on a quick start installation

(This message has been automatically imported from the retired mailing list)

Hi community,
I am just discovering zato. So I installed zato 1.1 and set up the =
quickstart.
Here are the details:
nacer@host:~$ zato quickstart create zato/qs/ postgresql localhost 5432 =
zato zato localhost 6379

ODB database password (will not be echoed):=20
Enter the odb_password again (will not be echoed):=20

Key/value database password (will not be echoed):=20
Enter the kvdb_password again (will not be echoed):=20
[1/8] Certificate authority created
[2/8] ODB schema already exists
[3/8] ODB initial data created
[4/8] server1 created
[5/8] server2 created
[6/8] Load-balancer created
Superuser created successfully.
[7/8] Web admin created
[8/8] Management scripts created
Quickstart cluster quickstart-90018 created
Web admin user:[admin], password:[xxxxxxxxxxx]
Start the cluster by issuing the /home/nacer/zato/qs/zato-qs-start.sh =
command
Visit https://zato.io/support for more information and support options

nacer@host:~$ ls zato/qs/
ca load-balancer server1 server2 web-admin zato-qs-restart.sh =
zato-qs-start.sh zato-qs-stop.sh

nacer@host:~$ bash zato/qs/zato-qs-start.sh=20
Starting the Zato quickstart environment
Running sanity checks
[1/6] Redis connection OK
[2/6] SQL ODB connection OK
[3/6] Load-balancer started
[4/6] server1 started
[5/6] server2 started
[6/6] Web admin started
Zato quickstart environment started
Visit https://zato.io/support for more information and support options

After that, only the load balancer, web admin and one of the two server =
are listening on their respective ports.

nacer@host:~/zato/qs/server1$ curl localhost:11223/zato/ping -d ‘{}’

503 Service Unavailable

No server is available to handle this request.

So I looked into the running server logs to guest what was wrong and I =
discover that none of the services were added because of a unicode =
decode issue.
Please find attached the logs directories of the two servers.

I am using a debian wheezy box.

Please could you give me some support?
Best regards

On 01/06/2014 09:50 AM, Nacer Adamou Saidou wrote:

So I looked into the running server logs to guest what was wrong and I discover that none of the services were added because of a unicode decode issue.
Please find attached the logs directories of the two servers.

Hi Nacer,

thanks for sending it in.

This is what I believe is happening:

  • An integrity error is caught when a server is starting. This is OK and
    expected at that point.

  • The error is being logged on TRACE1 level, however, because the code
    assumes the error message will be in ASCII, it fails with a
    UnicodeDecodeError.

Question now is - does your Postgres installation use messages outside
of ASCII? Can you please check Postgres logs around the time these
UnicodeDecodeErrors happen and send it across?

On a related note, what is the encoding your Postgres uses? Is it UTF-8?

many thanks,

On 01/06/2014 09:50 AM, Nacer Adamou Saidou wrote:

So I looked into the running server logs to guest what was wrong and I discover that none of the services were added because of a unicode decode issue.
Please find attached the logs directories of the two servers.

Hi Nacer,

thanks for sending it in.

This is what I believe is happening:

  • An integrity error is caught when a server is starting. This is OK and
    expected at that point.

  • The error is being logged on TRACE1 level, however, because the code
    assumes the error message will be in ASCII, it fails with a
    UnicodeDecodeError.

Question now is - does your Postgres installation use messages outside
of ASCII? Can you please check Postgres logs around the time these
UnicodeDecodeErrors happen and send it across?

On a related note, what is the encoding your Postgres uses? Is it UTF-8?

many thanks,

On 01/07/2014 06:33 AM, Nacer Adamou Saidou wrote:

Hello Nacer,

My postgres installation use by default UTF-8 as you can see below:
postgres@xxxxxx:~$ psql zato -c 'SHOW SERVER_ENCODING’
server_encoding

UTF8
(1 ligne)

Must I configure it to use ascii?

UTF-8 is fine - no need to change anything.

In the postgresql, I have a lot of integrity errors like the one below (sorry in french):
2014-01-06 08:11:03 EAT ERREUR: la valeur d’une clé dupliquée rompt la contrainte unique « service_name_cluster_id_key »
2014-01-06 08:11:03 EAT DÉTAIL: La clé « (name, cluster_id)=(zato.stats.base-aggregating-service, 1) » existe déjà.

French is fine and this looks like an issue with how Postgres messages -
not data in tables, just messages like ‘existe déjà’ - are handled in
Zato when an error is returned by Postgres.

I’ll set up Postgres with log messages in French locally and give you
more information in upcoming days.

many thanks again,

Thank you very much for your support.
Best regards

On 7 January 2014 17:00, Dariusz Suchojad dsuch@zato.io wrote:

On 01/07/2014 06:33 AM, Nacer Adamou Saidou wrote:

Hello Nacer,

My postgres installation use by default UTF-8 as you can see below:
postgres@xxxxxx:~$ psql zato -c 'SHOW SERVER_ENCODING’
server_encoding

UTF8
(1 ligne)

Must I configure it to use ascii?

UTF-8 is fine - no need to change anything.

In the postgresql, I have a lot of integrity errors like the one below
(sorry in french):
2014-01-06 08:11:03 EAT ERREUR: la valeur d’une cl=C3=A9 dupliqu=C3=A9=
e rompt la
contrainte unique =C2=AB service_name_cluster_id_key =C2=BB

2014-01-06 08:11:03 EAT D=C3=89TAIL: La cl=C3=A9 =C2=AB (name,
cluster_id)=3D(zato.stats.base-aggregating-service, 1) =C2=BB existe d=C3=
=A9j=C3=A0.

French is fine and this looks like an issue with how Postgres messages -
not data in tables, just messages like ‘existe d=C3=A9j=C3=A0’ - are hand=
led in
Zato when an error is returned by Postgres.

I’ll set up Postgres with log messages in French locally and give you
more information in upcoming days.

many thanks again,


Dariusz Suchojad

https://zato.io
ESB, SOA and cloud integrations in Python

–=20

Nacer Adamou Sa=C3=AFdou
Blog: http://nacerix.blogspot.com
Twitter: nacerix
Identi.ca: nacerix
http://paper.li/nacerix
http://netvibes.net/nacerix
https://launchpad.net/~adamou-nacer