First, thanks for working on integrating SFTP on Zato. Being a native integration can save people some of the headaches I had when first dealing with the framework.
When I first wanted to replace the FTP system with an SFTP one, based on a customer request, I found out how many parts of the code I would need to replace. I later introduced a class interface with the common methods needed and integrated them with the options at my disposal, so I could more easily replace them wherever needed and maybe this is a good policy to adopt on Zato as well. The more transparent an outgoing file based connection is, the easier it would be to change them in a service without changing code directly. Not sure how hard it would be to satisfy the same methods in different file systems, but so far for FTP and SFTP it was a good approach. I can share the interface signatures tomorrow, when I have the code in front of me.
Right now, I have an interface for the regular FTP supported by Zato (which is based on the ancient fs 0.4.0) and the ssh2-python 0.17.0, which uses an embedded libssh2 library, which allowed my to use a non-blocking mode for SFTP calls.
I’m not sure if the next section will be useful for you or not, but since most of my pains inside Zato were (and some still are) related to SFTP issues, here are some insight in all my discoveries during this exploration:
- fs 0.4.0 is ancient and I never understood why the library was never updated with new Zato versions. First I though it was a coroutine problem on later versions but even today I have several examples of code using fs calls which blocks the workers (my service never ends properly in such cases);
- paramiko is unusable, since it depends on libraries which conflict with some embedded libraries on Zato (as of versions 2 and 3), it also does not support non-blocking mode;
- fs 0.4.0 supports SFTP using paramiko, so it has all the same problems for SFTP support;
- Newer versions of fs dropped the SFTP support altogether;
- asyncssh is Python 3 only;
- Fabric is more command line focused and also powered by paramiko (Ansible has some of the same problems);
- ssh2-python depends on libssh2 (which implements SFTP calls using C), which has some small problems, like when opening a remote file, some methods have different signatures compared to the standard library. One example is the seek method, which lacks the second parameter to specify where to anchor the reference point, which breaks some integrations (for me it was opening a ZipFile remotely, which did not work and required me to download the file locally for interaction). Even so, it was the closest to perfect I could get at Python 2, with non-blocking mode (it required using gevent sockets and select functions directly to avoid blocking when waiting for network calls);
- parallel-ssh can use either paramiko or libssh2 underneath, but it has an explicit dependency on paramiko, even if you want to use libssh2 only. In future versions the author will make the paramiko dependency optional (no ETA on this, though).
When Zato becomes a Python 3 framework, some of this problems may be alleviated, but I just wanted to share part of my knowledge, in case you need to pass through the same decision tree on your side.
Will be back tomorrow to share my common interface, which may bring some light into my usage of SFTP.