Python 3 support

Python 3 support discussion.

Can I help make a roadmap of tasks that need to be done toward this goal? Then, as I have time in the next six months I can donate time toward helping with python 3 zato.

Hi Sam,

yes, that would be very nice, it’s great to hear it!

Llikely this is going to be smaller incompatible bits scattered across some dependencies rather than huge blocks of development. Our code as such does not use any Python 2-specific internal APIs and even if there are some pieces like str vs. unicode, there shouldn’t be much of it.

  • Let’s do it for Zato 3.x but not for 2.x

  • I’m not sure what Python 3.x version in particular to settle on - we can agree for now that it’s Python 3.6

  • What is needed first is confirmation that Zato installation from source will work under Python 3 - solely that, just confirming that installer can go ahead, if not, it needs to be extended to let Zato run with Python 3

  • Let’s start with Ubuntu 14.04 64-bit - this is what most people use

  • Once the installer can produce a working binary, we’d need to run unit-tests and API tests - it’s all built in and available under ./run-tests.py - I can guide you on how to use it

  • Tests will uncover what needs to be ported - both in the core and in dependencies. Many dependencies for sure work with both Python lines, if some dependency doesn’t, we’ll have to resolve it on a case-by-case basis

  • We need Zato source code that will work with both Python 2.7 and 3.x - definitely no two parallel code bases

  • After the tests confirm that everything works fine, we can think how to make the ‘zato’ command work with either Python version - I would see Python 2.7 the default and something like ‘zato -py3 start /path/to/server’ for Python 3. Or we can have an environment variable ZATO_PYTHON_VERSION=3 to make it work with Python 3 by default - it’s really not that important at this point

What do you think?

Sam, it really sounds great and exciting…

Darius why don’t you think to organize a webinar for community on which you will tell how to develop Zato in detail? Here is proposal of subjects:

  • Zato components and code organization
  • Creating development environment (vagrantbox, python paths, env vars, system services, etc…)
  • How to build and test
  • How to write tests
  • Managing dependencies, how is it decided to fix version of packages? e.g why Babel is in version ==1.3?
  • Tips and tricks

Regards,

I think it all sounds good @dsuch

Where do we find the zato 3 branch?

The env variable sounds like a good idea to bridge the gap.

I will check out install from source soon.

Where is your preferred place to report back progress etc?

Thanks!

Is it true thar zato 3 is “main” git branch, while 2 is “support/2”?

@ali - yes, that is a good idea, I will think about arranging it!

Hi Sam,

here is how the code is structured in the GitHub repository.

  • main - latest stable development branch, this is the branch that feature branches are merged to. This is the branch that Zato 3.x will be built from. Afterwards, the subsequent version will be built from main and so on.

  • support/x.y branches, they are for fixes pertaining to a given base release only. For instance, support/2.0 is for fixes to 2.0.x. This is what patch releases are built from. For instance, right now the latest patch release for 2.0.x is 2.0.7. Patches accumulate in support/2.0 and 2.0.8 will be built from this branch. Similarly, after 3.0 is released, there will be a new branch called support/3.0.

  • Feature branches - this is where development happens. Each non-absolutely trivial feature or change gets its own ticket in GitHub. Next a feature branch is created for this ticket. Once a feature is complete - it is merged to main.

There are naming conventions regarding features branches. Each branch has a name such as:

  • dsuch-f-gh723-add-exe-agent

Where:

  • dsuch - branch owner, the person responsible for this work
  • f - feature (can be also b - bug)
  • gh723 - indicates that it is based on GitHub ticket #723
  • add-exe-agent - a short description of the branch so as not to have to remember dozens of ticket numbers

There is also one convention regarding commit messages. Each commit related to any ticket needs to be in the format of “GH #NNN - …” where NNN is the ticket number. For instance:

  • GH #725 - Adding API tests for FixedWidth.
  • GH #681 - Added core layout of a Cython-based LRU/TTL cache.
  • GH #547 - Close the initial connection used only to confirm overall connectivity.

In this manner GitHub’s UI is able to nicely link commits to tickets and the other way around.

To get you quickly started, I have just created a feature branch for you called samrose-f-gh706-add-python3-support, where:

I also added you to the list of committers to Zato.

Please make sure that you frequently merge changes from main to samrose-f-gh706-add-python3-support, there are batches of commits added to main as development branches get progressively merged into it.

@rafal will send you quick information on how to set up a development box under Vagrant - this is purely for your convenience, if you’d like to use it.

We can use either GitHub or the forum - perhaps it would be more convenient to discuss ongoing matters in GitHub ticket #706 and periodically send summaries to the forum for everyone else’s benefit?

Hi @oldbay,

Ali Rıza Keleş (@ali) will now work on adding Python 3 support to Zato too. We also have Sam Rose who volunteered to assist us.

There is only a handful of dependencies that are not Python 3-compatible and some of them can be removed altogether (springpython, distutils2, sec-wall, threadpool for sure). There is no worry that the next Zato version will work with both Python 2.7 and Python 3.x.

About the only thing that may be still considered is whether to have Python 3.x support for RHEL 6.x, depends on what sort of effort it takes - we’ll see about it. Other than that, it’s all OK.

I will try to catch up to this effort soon, I am still interested in helping. I had not heard from @rafal yet on setting up under vagrant (i’d love to use that if possible) and I will catch up on https://github.com/zatosource/zato/issues/706

Thanks!