Hello,
I’m happy to let you know that Zato 2.0.8 has just been released and packages for Ubuntu, RHEL/CentOS and Debian can be installed from the usual locations.
This is mostly a patch release that closes tickets opened since 2.0.7 was released.
On the other hand, a few performance improvements have been backported from the upcoming 3.0 release - HTTP servers will be now up to 30% faster than with 2.0.7.
Many thanks to everyone who helped in making this release!
Hello,
I tried upgrade my test cluster which is running on 2.0.7 CentOS zato rpm package. Most of the components were able to run on 2.0.8, except web admin. It fail to start. I tried running on foreground mode and I got the following error:
I thought sqlites modules are part of python 2.7 package?
zato start webadmin --fg
Traceback (most recent call last):
File “/opt/zato/2.0.8/code/bin/py”, line 193, in
_val, {}, “main”, alter_sys=True)
File “/opt/zato/2.0.8/code/lib/python2.7/runpy.py”, line 188, in run_module
fname, loader, pkg_name)
File “/opt/zato/2.0.8/code/lib/python2.7/runpy.py”, line 82, in _run_module_code
mod_name, mod_fname, mod_loader, pkg_name)
File “/opt/zato/2.0.8/code/lib/python2.7/runpy.py”, line 72, in _run_code
exec code in run_globals
File “/srv/zato/2.0.8/code/zato-web-admin/src/zato/admin/main.py”, line 59, in
main()
File “/srv/zato/2.0.8/code/zato-web-admin/src/zato/admin/main.py”, line 52, in main
django.setup()
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/init.py”, line 17, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/conf/init.py”, line 55, in getattr
self._setup(name)
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/conf/init.py”, line 43, in _setup
self._wrapped = Settings(settings_module)
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/conf/init.py”, line 99, in init
mod = importlib.import_module(self.SETTINGS_MODULE)
File “/opt/zato/2.0.8/code/lib/python2.7/importlib/init.py”, line 37, in import_module
import(name)
File “/opt/zato/2.0.8/code/zato-web-admin/src/zato/admin/settings.py”, line 155, in
_settings_db_engine = create_engine(‘sqlite:///{}’.format(_settings_db_path))
File “/opt/zato/2.0.8/code/eggs/SQLAlchemy-0.9.9-py2.7-linux-x86_64.egg/sqlalchemy/engine/init.py”, line 386, in create_engine
return strategy.create(*args, **kwargs)
File “/opt/zato/2.0.8/code/eggs/SQLAlchemy-0.9.9-py2.7-linux-x86_64.egg/sqlalchemy/engine/strategies.py”, line 74, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File “/opt/zato/2.0.8/code/eggs/SQLAlchemy-0.9.9-py2.7-linux-x86_64.egg/sqlalchemy/dialects/sqlite/pysqlite.py”, line 339, in dbapi
raise e
ImportError: No module named pysqlite2
Regards,
Keith
Hi @keith,
pysqlite2 is not a module that Zato comes with. Is it something that you installed yourself using /opt/zato/current/bin/pip? If so, you need to install it in the upgraded environment again.
I am quite sure i did not install pysqlite2 using pip. According to SQLAlchemy docs, pysqlite2 will only be loaded if sqlite3 is not found. I check 2.0.8 and sqlite3 seems to be already installed in /opt/zato/2.0.8/code/lib/python2.7/sqlite3. But not sure why it failed to load.
Changing back to 2.0.7 (by pointing soft link “current” to 2.0.7) is ok but with 2.0.8, my web admin can’t start. Am I starting the web admin in foreground mode correctly by using “–fg” ? When I start without --fg, the process just died without throwing any error in the logs.
When using Python 2.5 and above, the built in sqlite3 driver is
already installed and no additional installation is needed. Otherwise,
the pysqlite2 driver needs to be present. This is the same driver as
sqlite3, just with a different name.
The pysqlite2 driver will be loaded first, and if not found, sqlite3
is loaded. This allows an explicitly installed pysqlite driver to take
precedence over the built in one. As with all dialects, a specific
DBAPI module may be provided to :func:~sqlalchemy.create_engine() to control
this explicitly::
from sqlite3 import dbapi2 as sqlite
e = create_engine('sqlite+pysqlite:///file.db', module=sqlite)
@classmethod
def dbapi(cls):
try:
from pysqlite2 import dbapi2 as sqlite
except ImportError as e:
try:
from sqlite3 import dbapi2 as sqlite # try 2.5+ stdlib name.
except ImportError:
raise e
return sqlite
This looks something specific to CentOS 6.x - it doesn’t ship with Python 2.7 and there are always extra checks required, I will see to it but it won’t be immediately. Please revert to 2.0.7 for now.
Sorry, did not get what you mean by “it doesn’t ship with Python 2.7”. Are you refering to Zato CentOS 6.x rpm package that we installed from your repository doesn’t ship with Python2.7 or are you refering to CentOS site-wide Python 2.7 package? We are not using site-wide CentOS 6.x as be default, it’s Python 2.6 package. We have been using Zato CentOS 6.x rpm package that ship with Python 2.7. If I am not wrong, zato 2.0.7 is package with py version 2.7.6 and zato 2.0.8 is package with 2.7.13 py version.
We are hopeful that with this upgrade, it will resolve our pymysql ssl issues and also singleton reconnection issues. 
Hi @keith - what I meant was that, in general, CentOS 6 always needs extra, special attention.
You are right that RPMs with Zato have their own Python 2.7 embedded and this is the version that needs to be double-checked with regards to pysqlite2/SQLite.
Hi,
Was wondering is there any updates on 2.0.8 centos rpm packages? We hit the road block which prevent us from upgrading to 2.0.8 the last time.
cheers.
Keith
Hey, @keith,
Based on my topic (ODB HA reconnection issue (affecting scheduled services)) which hit a similar roadblock (for RHEL6), I believe @dsuch is going to take a look at it this weekend.
[]'s
Hi @dsuch
I spinned up a centos-6.9 VM, Did a fresh yum install on zato-2.0. repo.
[zato@testzato ~]$ zato --version
Traceback (most recent call last):
File “/opt/zato/current/code/bin/zato”, line 178, in
import zato.cli.zato_command
File “/opt/zato/2.0.8/code/zato-cli/src/zato/cli/zato_command.py”, line 20, in
from zato.cli import ca_create_ca as ca_create_ca_mod, ca_create_lb_agent as ca_create_lb_agent_mod,
File “/opt/zato/2.0.8/code/zato-cli/src/zato/cli/enmasse.py”, line 56, in
from zato.server.service.internal.notif.cloud.openstack import swift as notif_cloud_openstack_swift_mod
File “/opt/zato/2.0.8/code/zato-server/src/zato/server/service/internal/notif/cloud/openstack/swift.py”, line 19, in
from globre import match as globre_match
ImportError: No module named globre
Did a pip install on globre and resolve this issues.
Create a server and started , no issue.
Create load_balancer and started, no issue.
Create web_admin and start. Same issues as reported previously in this thread and (ODB HA reconnection issue (affecting scheduled services)1) :
[zato@testzato ~]$ zato start WA --fg
Traceback (most recent call last):
File “/opt/zato/2.0.8/code/bin/py”, line 193, in
_val, {}, “main”, alter_sys=True)
File “/opt/zato/2.0.8/code/lib/python2.7/runpy.py”, line 188, in run_module
fname, loader, pkg_name)
File “/opt/zato/2.0.8/code/lib/python2.7/runpy.py”, line 82, in _run_module_code
mod_name, mod_fname, mod_loader, pkg_name)
File “/opt/zato/2.0.8/code/lib/python2.7/runpy.py”, line 72, in _run_code
exec code in run_globals
File “/opt/zato/2.0.8/code/zato-web-admin/src/zato/admin/main.py”, line 59, in
main()
File “/opt/zato/2.0.8/code/zato-web-admin/src/zato/admin/main.py”, line 52, in main
django.setup()
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/init.py”, line 17, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/conf/init.py”, line 55, in getattr
self._setup(name)
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/conf/init.py”, line 43, in _setup
self._wrapped = Settings(settings_module)
File “/opt/zato/2.0.8/code/eggs/Django-1.9.3-py2.7.egg/django/conf/init.py”, line 99, in init
mod = importlib.import_module(self.SETTINGS_MODULE)
File “/opt/zato/2.0.8/code/lib/python2.7/importlib/init.py”, line 37, in import_module
import(name)
File “/opt/zato/2.0.8/code/zato-web-admin/src/zato/admin/settings.py”, line 155, in
_settings_db_engine = create_engine(‘sqlite:///{}’.format(_settings_db_path))
File “/opt/zato/2.0.8/code/eggs/SQLAlchemy-0.9.9-py2.7-linux-x86_64.egg/sqlalchemy/engine/init.py”, line 386, in create_engine
return strategy.create(*args, **kwargs)
File “/opt/zato/2.0.8/code/eggs/SQLAlchemy-0.9.9-py2.7-linux-x86_64.egg/sqlalchemy/engine/strategies.py”, line 74, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File “/opt/zato/2.0.8/code/eggs/SQLAlchemy-0.9.9-py2.7-linux-x86_64.egg/sqlalchemy/dialects/sqlite/pysqlite.py”, line 339, in dbapi
raise e
ImportError: No module named pysqlite2
Regards,
I did a test on compiling python-2.7.13 from source on the centos-6.9 VM. Notice that at the end of the compilation, some modules (which includes sqlite3, etc) were not able to build. I then install the necessary devel packages as fellow:
sqlite-devel
openssl-devel
bzip2-devel
zlib-devel
readline-devel
gdbm-devel
Recompile python-2.7.13 again and did a test to import sqlite3. No more import error.
I then did a rebuild on zato following what you have written on ODB HA reconnection issue (affecting scheduled services), to rebuild so that the necessary sqlite3 packages will properly build.
The rebuild Zato packages still have “no globre modules” and “no django_openid_auth module”. This can be overcome easily with pip install.
Using the rebuild zato package, I create a server and load balancer without any issues. However upon creating the web-admin, I encounter the following:
[zato@testzato ~]$ zato create web_admin --odb_host localhost --odb_port 3306 --odb_user zatotest --odb_db_name ZATOTEST WA mysql /opt/zato/CA/out-pub/web-admin-pub-2018-01-22_02-49-36.pem /opt/zato/CA/out-priv/web-admin-priv-2018-01-22_02-49-36.pem /opt/zato/CA/out-cert/web-admin-cert-2018-01-22_02-49-36.pem /opt/zato/CA/ca-material/ca-cert.pem testacc1
ODB database password (will not be echoed):
Enter the odb_password again (will not be echoed):
Technical account password (will not be echoed):
Enter the tech_account_password again (will not be echoed):
Traceback (most recent call last):
File “/opt/zato/current/code/bin/zato”, line 181, in
sys.exit(zato.cli.zato_command.main())
File “/opt/zato/2.0.8rev1/code/zato-cli/src/zato/cli/zato_command.py”, line 262, in main
return run_command(get_parser().parse_args())
File “/opt/zato/2.0.8rev1/code/zato-cli/src/zato/cli/init.py”, line 266, in run_command
command_classargs.command.run(args)
File “/opt/zato/2.0.8rev1/code/zato-cli/src/zato/cli/init.py”, line 483, in run
return_code = self.execute(args)
File “/opt/zato/2.0.8rev1/code/zato-cli/src/zato/cli/create_web_admin.py”, line 148, in execute
call_command(‘migrate’, run_syncdb=True, interactive=False, verbosity=0)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/core/management/init.py”, line 119, in call_command
return command.execute(*args, **defaults)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/core/management/base.py”, line 399, in execute
output = self.handle(*args, **options)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/core/management/commands/migrate.py”, line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/migrations/executor.py”, line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/migrations/executor.py”, line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/migrations/executor.py”, line 198, in apply_migration
state = migration.apply(state, schema_editor)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/migrations/migration.py”, line 123, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/migrations/operations/models.py”, line 59, in database_forwards
schema_editor.create_model(model)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/backends/base/schema.py”, line 284, in create_model
self.execute(sql, params or None)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/backends/base/schema.py”, line 110, in execute
cursor.execute(sql, params)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/backends/utils.py”, line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/utils.py”, line 95, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/backends/utils.py”, line 62, in execute
return self.cursor.execute(sql)
File “/opt/zato/2.0.8rev1/code/eggs/Django-1.9.3-py2.7.egg/django/db/backends/mysql/base.py”, line 112, in execute
return self.cursor.execute(query, args)
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/cursors.py”, line 132, in execute
result = self._query(query)
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/cursors.py”, line 271, in _query
conn.query(q)
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/connections.py”, line 726, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/connections.py”, line 861, in _read_query_result
result.read()
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/connections.py”, line 1064, in read
first_packet = self.connection._read_packet()
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/connections.py”, line 826, in _read_packet
packet.check_error()
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/connections.py”, line 370, in check_error
raise_mysql_exception(self._data)
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/err.py”, line 116, in raise_mysql_exception
_check_mysql_exception(errinfo)
File “/opt/zato/2.0.8rev1/code/eggs/PyMySQL-0.6.2-py2.7.egg/pymysql/err.py”, line 112, in _check_mysql_exception
raise InternalError(errno, errorvalue)
InternalError: (1170, u"BLOB/TEXT column ‘claimed_id’ used in key specification without a key length")
Regards,