Remote Jupyter notebook attached to zato kernel gives an Invalid Signature error

I’m hoping maybe somebody has got this working and/or can give some insight.

I understand that this is more of a Jupyter forum question but I think it would be helpful if this subject were addressed here too.

Ipython alone is ok but the notebook is so much better for me. I’ve tried a couple of times to get this to work but there is always a wrench in the system. This time it is the error in the jupyter stout that says:

[IPKernelApp] Invalid Message
Traceback (most recent call last):
File “/opt/zato/2.0.5/eggs/ipython-0.13.2-py2.7.egg/IPython/zmq/ipkernel.py”, line 201, in dispatch_shell
msg = self.session.unserialize(msg, content=True, copy=False)
File “/opt/zato/2.0.5/eggs/ipython-0.13.2-py2.7.egg/IPython/zmq/session.py”, line 725, in unserialize
raise ValueError(“Invalid Signature: %r”%signature)
ValueError: Invalid Signature: ‘813318blah’

I am not an expert at the Jupyter notebook but I can manage to attach to my system-wide ipython kernel when I start jupyter as root user with remote connections enabled and all is semms well but I cannot import zato because it is not pointing then to the zato ipython kernel. My intent is to import zato using Jupyter Notebook and author services while attached remotely because I don’t have a local browser installed.

But when I install Jupyter using /opt/zato/2.0.5/bin/pip install jupyter and then I initialize a configuration directory in /opt/zato/.jupyter then I get the configuration looking like this:

c.NotebookApp.certfile = u’/opt/zato/.jupyter/mycert.pem’
c.NotebookApp.keyfile = u’/opt/zato/.jupyter/mykey.key’
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.password = u’sha1:mykeyblah’
c.NotebookApp.port = 8888

Then I can attach remotely to that ip:port but when I launch the kernel which is pointing to the ipython at /opt/zato/2.0.5/bin/ipython I see the invalid signature error above and can’t run any commands on the notebook.

One thing I’ve tried is setting the key to empty string in the kernel json file: /opt/zato/.local/share/jupyter/runtime/kernel-blah.json

The ipython doc said it would bypass signature checking but then I get a jupyter error that is looking for the signature so it seems you may be able to bypass with ipython but not the jupyter notebook.

I love Jupyter Notebook. I really don’t even use an IDE much anymore. I do a-lot with REST and XML/JSON and these notebooks are just such a great way to visualize your data and construct the code in parts with the live kernel. I personally think that it would be very helpful to have write all of my zato services this way.

Thanks for looking and please let me know if anybody has gotten this to work or can slap me around with the obvious that I’m missing.

Hello @bigale,

not being a Jupyter user I cannot really offer any assistance but your post is intriguing - would you be able to offer a bit more details about how you are using it with Zato? This is really interesting!

Thanks.

Thanks for your reply. I suppose the primary reason I like using Jupyter is that you get all the benefits of IPython and you get to scroll around and jump up and down and all around quickly seeing all of your code output cells at a glance. You never have to stop and start the debugger with IPython and that is great for productivity.

As a side note, I have also use the built-in Jupyter html widgets to make quick little UI’s for stuff I do often. I even did a UI that way using the zato wsdl and suds together to make it easier to quickly make a copy of a channel for instance which you cannot do from the zato ui. You don’t need the kernel attached to do that stuff though because you just use pycurl to the zato public api and off you go. The point is that it only takes a few lines of code using Suds and Jupyter Notebook Widget to make your own UI for Zato. This is nice for those of us who know next to nothing about front end development with django and so forth.

That being said, I’ve not done anything significant with IPython and zato so I’m not sure how Jupyter will help because of the service oriented nature of zato. I also haven’t spent much time with it.

For instance, say I pasted in a basic service to an IPython cell. I can see the zato object model using the .tab quick keys and so forth. I can instantiate that service by type theService=MyService() but I don’t even know how to call invoke the service using IPython because I usually do that from the scheduler or service invoker. I don’t even know if it can be accomplished using IPython but in my mind when you import zato in IPython, then you should be able to use it if you knew how.

Let’s just suppose that I knew how to instantiate the service and in debugging, I set the return value of the service to a variable when I invoke it. Now I can be productive with IPython and super-productive with Jupyter. I can put a return statement in my code instead of logging or pickling the variable/object to the kvdb and reading it back in order to have it in scope to write more code with it.

Then I write something like theService=MyService.invoke() then execute that in one cell, in another cell execute for item in theService: self.logger.info(item). Then change the code in MyService and do it again.

Thanks!

So here is the process one can use to debug things like giant data structures remotely using jupyter if anybody is interested.

As stated above, if one was able to attach remotely to the kernel then the process would likely be more to the point. Then you would not need to paste code back to the service when done, instead you would debug and just save right to the pickup dir. This should work fine for me now to get going.

###
# how to use Jupyter (or any remote debugger) and zato kvdb service
# to remotely debug a service with lots of big fat data structures
###
# in the ZATO SERVICE - instead of writing self.logger.info(a_nested_dictionary)
# convert dict to json and write to kvdb
###
a_nested_dictionary_json = json.dumps(a_nested_dictionary)
self.kvdb.conn.set("a_nested_dictionary_json",a_nested_dictionary_json)

# Jupyter cells on remote system (not zato host)
from collections import OrderedDict
import pycurl
from StringIO import StringIO
import json
def get_kvdb_json_value():
    """
    Make the actual call to the public api on zato and return result.
    Setup call to get kvdb value.  Setup pubapi password in zato using
    Security menu and setting password via HTTP Basic Auth
    """
    # I use this structure to utilize lists of commands to pass to zato
    optDict = OrderedDict([(u'command', u'GET a_nested_dictionary_json')])
    jd=json.dumps(optDict)
    username = 'pubapi'
    password = 'mypassword'
    buffer = StringIO()
    #CURL
    c = pycurl.Curl()
    theCmd = "http://myhost:11223/zato/json/zato.kvdb.remote-command.execute"
    print(theCmd, jd)
    c.setopt(c.URL, theCmd)
    c.setopt(pycurl.HTTP_VERSION, pycurl.CURL_HTTP_VERSION_1_0)
    c.setopt(c.USERPWD, "{}:{}".format(username, password))
    c.setopt(c.POSTFIELDS, jd)
    c.setopt(c.WRITEDATA, buffer)
    c.perform()
    c.close()

    body_json = buffer.getvalue()
        return body_json
# Debug in REMOTE JUPYTER CELL.  Convert json back to python object if needed.
debug_json_value = get_kvdb_json_value()
body_dict = json.loads(debug_json_value)
python_object_ready_to_debug = body_dict['zato_kvdb_remote_command_execute_response']['result']

for item in python_object_ready_to_debug:
    print(item)

# Once this code is debugged then paste to service and start again