Webhook pattern in zato?

#1

Has anyone done a “webhook” pattern with zato? I can see ways to handle most of this with Zato with a few integrated services

Django for instance has https://thorn.readthedocs.io/en/stable/getting-started/django.html

#2

Hi @samuel.rose,

since Zato 3.0, we have multi-protocol publish/subscribe topics and message queues. REST is one of the supported protocols.

Essentially, this turns Zato into a full-featured message broker in addition to all the other features.

https://zato.io/docs/pubsub/index.html

#3

Zato 3’s native REST pub/sub endpoint support is pretty nice. Though depending on the sending sides security/auth options it may not work for you. It was not an option for Hubspot webhooks due to hubspots lack of authorization/credentials config on their end. Not real big on white listing IP’s either (which really is not an option with hubspot anyways.)

Here’s a simple alternative pattern:

  1. A single rest channel, with a url path like this: /zato/webhooks/{source}
  2. Pointing to a simple service that inspects the payload and performs some form of authentication.
  3. The service pops the payload onto a pub/sub queue based on the {source}.

Inside the service:

Based on the {source} input parameter, we can implement our own security mechanisms as needed. In our case none of our current webhook publishers support sending headers or even basic auth. Specifically, Hubspot sends a payload signature that we rehash on our end to confirm authenticity. Other systems may just send a shared key as a url parameter (over ssl).

# generate hubspot signature
app_secret = 'asdffasdf-asdf-asdf-asdf-asdf'
request_sig = hashlib.sha256( app_secret + self.request.raw_request ).hexdigest()
# check if signatures match
if 'HTTP_X_HUBSPOT_SIGNATURE' in self.wsgi_environ and request_sig == self.wsgi_environ['HTTP_X_HUBSPOT_SIGNATURE']:
  legit = True

Once the request is authenticated, we can pop the payload onto a specific pub/sub topic:

if legit:
  # self.logger.info( 'HUBSPOT_LEGIT' )
  # pop this legit request(s) onto the hubspot topic
  for event in json.load( StringIO( self.request.raw_request ) ):
     msg_id = self.pubsub.publish( '/inbound/hubspot', data=str(event) )

Here’s the whole service:

** Note: The if/then approach below may get messy with a lot of publishers, but it will get you off the ground.

# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function, unicode_literals
from zato.common import ZatoException
from zato.server.service import Service

import sentry_sdk

import json
import hashlib
import httplib

from StringIO import StringIO

class Webhooks( Service ):
    name = 'Webhooks'
    class SimpleIO:
        input_required = ( 'source' )

    def handle( self ):
        try:
            self.response.status_code = httplib.OK
            legit = False

            sentry_sdk.init("https://asdfasdfasdfasdfasdf@sentry.io/999999")

            if self.request.input.source == 'someapp':
                if 'shared_secret' in self.request.input and self.request.input.shared_secret == 'awesome':
                    legit = True

                if legit:
                    # pop this legit request onto the someapp topic
                    msg_id = self.pubsub.publish( '/inbound/someapp', data=self.request.raw_request )
                    self.logger.info(msg_id)
                else:
                    self.response.status_code = httplib.FORBIDDEN

            if self.request.input.source == 'hubspot':

                if 'curl_test' in self.request.input and self.request.input.curl_test == 'true':
                    legit = True
                else:
                    # generate hubspot signature
                    app_secret = 'asdfasdfasdf0-asdfasdfasdf-asdfasdfsdf'
                    request_sig = hashlib.sha256( app_secret + self.request.raw_request ).hexdigest()
                    # check if signatures match
                    if 'HTTP_X_HUBSPOT_SIGNATURE' in self.wsgi_environ and request_sig == self.wsgi_environ['HTTP_X_HUBSPOT_SIGNATURE']:
                        legit = True

                if legit:
                    # self.logger.info( 'HUBSPOT_LEGIT' )
                    # pop this legit request(s) onto the hubspot topic
                    for event in json.load( StringIO( self.request.raw_request ) ):
                        msg_id = self.pubsub.publish( '/inbound/hubspot', data=str(event) )
                else:
                    self.response.status_code = httplib.FORBIDDEN

        except Exception as e:
            self.logger.error('Caught an exception %s', e.message)
            sentry_sdk.capture_exception()
            self.response.status_code = httplib.INTERNAL_SERVER_ERROR

        self.response.payload = ""

Afterthoughts

It would be sweet if zato had support for security definitions that called a service to authenticate…

#4

That sounds intriguing - do you mean a new security type, along with Basic Auth, JWT etc. that would actually be a Zato service itself, receiving an incoming request on input and deciding if it to be let in or not?

#5

@dsuch Exactly. That’s basically what this little service is doing. I think it would fill the REST authentication gaps for us. It would also allow us to use the native REST pub/sub as well.

#6

No doubt this could be a useful addition - would you open a GitHub ticket for it to ensure this will be implemented? Thanks.

#7

Will do. On my todo list.