One endpoint, different parameter sets


#1

Hi,

I have an endpoint that needs to support GET, DELETE and PUT. For GET and DELETE I only needs an id parameter.
For PUT I need more params. So, I define:

SimpleIO:
input_required = (‘id’,)
input_optional = (‘param1’, ‘param2’)

It is a pity that I cannot have the protection that SimpleIO provides for my PUT. I can write my own code in the service of course, but I was wandering if I am missing something in Zato that could be used in this case.

Regards, Jan


#2

Hi @jjmurre,

even though I like this idea, I have been sitting on your question because I just cannot find a good syntax for this feature.

So far I have come up with four options, as below.

class SimpleIO:

  class GET:
    input_optional = ('foo', 'bar')

  class POST:
    input_optional = ('baz, qwe')
class GET_SimpleIO:
    input_optional = ('foo', 'bar')

class POST_SimpleIO:
    input_optional = ('baz, qwe')
class SimpleIO_GET:
    input_optional = ('foo', 'bar')

class SimpleIO_POST:
    input_optional = ('baz, qwe')
@input_optional('foo', 'bar')
@input_required('baz', 'hi')
def handle_GET(self):
    ...

@input_required('foo', 'bar')
def handle_POST(self):
    ...

I quite like the idea with decorators but it may become unreadable when you add defaults or non-string parameters. The first one with nested classes reads fine as well. What do you think?


#3

Hi @dsuch,

I like decorators too, but agree with you that it may become too cluttered. And, it is a break with the current approach of the SimpleIO innner class.

For the other versions, I think the first one is very readable. However, according to the Zen of python “flat is better than nested” :slight_smile: And, for the “handle_XX” methods the approach with handle_GET/handle_PUT etc. is already in place.

So, in that case the third version (SimpleIO_GET etc.) is maybe the best one.

Just my 2 cents,

Regards, appreciate it that you take time to think about this issue,

Jan


#4

Hi, @dusch and @jjmurre.

I am facing a similar, if not the same, situation. I’m at the first stages of developing a REST services API with Zato 3.0 which, RESTfulness discussions aside, would ideally have the following behaviour (example) for the User model (incidentally, I am using SQLAlchemy):

  • GET /app/users to get a list/collection of users.
  • GET /app/users/{id} to get a user.
  • DELETE /app/users/{id} to delete a user.
  • PUT /app/users/{id} to update/modify a user.
  • POST /app/users to create a new user.
  • GET /app/users/{id}/roles to get the list of roles of a user.
  • DELETE /app/users/{id}/roles/{id} to delete a role of a user.
  • Etcetera.

Also I plan on user query string parameters to filter, sort and paginate the list, to request an expanded entity, and so on, such as:

  • GET /app/users?id[gte]=10&limit=10&offset=10&sort_by=email&order_by=desc

Addmitedly, filtering by id > 10 does not make much sense, but it’s just an example. :slight_smile:

For the sake of clarity, a simplified version of the SQLAlchemy model would look something like this:

# Base is the object returned by declarative_base()
class User(Base):
    __tablename__ = 'user'

    id = Column(Integer, primary_key=True)
    username = Column(String(20))
    password = Column(String(255))
    name = Column(String(50))
    surname = Column(String(50))
    email = Column(String(255))
    is_admin = Column(Boolean, default=False)

Now, how to do this with Zato 3? I’ve been playing with it for a couple of weeks now and, as per your previous reply, this would be one (imperfect, incomplete) approach using SimpleIO:

# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, print_function, unicode_literals
from contextlib import closing
from httplib import OK, NO_CONTENT, CREATED
from zato.server.service import Service
from app.schema import user
# from urlparse import parse_qs
class User(Service):

    # GET /app/users/{id}
    class SimpleIO_GET:
        input_required = ('id')
        output_required = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

    def handle_GET(self):
        conn = self.kvdb.conn.get('app:database:conn')
        id_ = self.request.input.id
        with closing(self.outgoing.sql.get(conn).session()) as session:
            result = session.query(user.User).filter(user.User.id == id_).one_or_none()
            if result:
                self.response.status_code = OK
                self.response.payload = result
            else:
                self.response.status_code = NO_CONTENT

    # POST /app/users
    class SimpleIO_POST:
        input_required = ('username', 'password', 'name', 'surname', 'email', 'is_admin')
        output_required = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

    def handle_POST(self):
        conn = self.kvdb.conn.get('app:database:conn')
        p = self.request.input
        u = user.User(username=p.username, password=p.password, name=p.name, surname=p.surname, email=p.email, is_admin=p.is_admin)
        with closing(self.outgoing.sql.get(conn).session()) as session:
            session.add(u)
            session.commit()
            self.response.status_code = CREATED
            self.response.payload = u
            self.response.headers['Location'] = 'http://127.0.0.1:11223/app/users/%s' % (u.id)

No sanitization of parameters, exception handling and others, I know. But it’s a first a version. :slight_smile:

Now, some issues and questions.

Getting a list/collection

How would the code to handle a call to get a collection (GET /app/users) fit in? Something like this pseudo-code:

class SimpleIO_GET:
    input_required = ()
    output_optional = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

def handle_GET(self):
    conn = self.kvdb.conn.get('app:database:conn')
    with closing(self.outgoing.sql.get(conn).session()) as session:
        result = session.query(user.User).order_by(user.User.id)
        output = []
        for row in result:
            output.append(row)
        self.response.payload[:] = output

Would both handles have to be merged and have an initial “if” depending on whether we have an id or not?

Cannot have different SimpleIO classes inside the service

As far as I know, the proposed version in the first reply (any of the 4) have been implemented yet. At least I have not been able to find any documentation. Is that correct? Any plans?

Would it be implemented, how would we code so that both calls using the verb GET are separated so that one can have input_required = (‘id’) and the other one have input_required = ()? Or what would be the way to do it? What about decorators with the channel configuration? Such as:

class User(Service):

    @channel('/app/users/{id}')
    class SimpleIO_GET:
        input_required = ('id')
        output_optional = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

        def handle(self):
            pass

    @channel('/app/users')
    class SimpleIO_GET:
        input_required = ('id')
        output_optional = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

        def handle(self):
            pass

Maybe the verb could be part of the decorator, too:

@channel('/app/users', verb=GET)

Channels

The abovementioned approach would require the following channels:

  • /app/users through all methods calls user.user
  • /app/users/{id} through all methods calls user.user

How would we configure the more complex channels?

  • /app/users/{id}/roles
  • /app/users/{id}/roles/{id}

What methods would they call?

Summary

If SimpleIO_VERB (or similar) were to be implemented, and maybe something else, this could become a way to implement REST APIs in Zato. Query string would be handled through parse_qs. Maybe more issues would arise aside from telling get_by_id from get_list, but this is as far as I have been able to go so far.

P.S. On the following post I will include a completely different approach that does not use the handle_VERB feature and has different pros and cons than this one


#5

Following my previous reply on the topic, this is another approach I have been working on:

from __future__ import absolute_import, division, print_function, unicode_literals
from contextlib import closing
from httplib import OK, NO_CONTENT, CREATED
from zato.server.service import Service
from app.schema import user
# from urlparse import parse_qs

class Get(Service):
    """Service class to get a userby id through channel /app/users/get/{id}."""

    class SimpleIO:
        input_required = ('id')
        output_optional = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

    def handle(self):
        """Returns a login given its id."""
        conn = self.kvdb.conn.get('app:database:conn')
        id_ = self.request.input.id
        with closing(self.outgoing.sql.get(conn).session()) as session:
            result = session.query(user.User).filter(user.User.id == id_).one_or_none()
            if result:
                self.response.status_code = OK
                self.response.payload = result
            else:
                self.response.status_code = NO_CONTENT

class List(Service):
    """Service class to get a list of all users in the system through channel /app/users/list."""

    class SimpleIO:
        input_required = ()
        output_optional = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

    def handle(self):
        """Returns a list of all users in the system."""

        conn = self.kvdb.conn.get('app:database:conn')
        with closing(self.outgoing.sql.get(conn).session()) as session:
            result = session.query(user.User).order_by(user.User.id)
            output = []
            for row in result:
                output.append(row)
            self.response.payload[:] = output

class Create(Service):
    """Service class to create a new user through channel /app/users/create."""

    class SimpleIO:
        input_required = ('username', 'password', 'name', 'surname', 'email', 'is_admin')
        output_required = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')

    def handle(self):
        """Creates a new user. All attributes are mandatory. Returns location header."""
        conn = self.kvdb.conn.get('genesisng:database:conn')
        p = self.request.input
        u = user.User(username=p.username, password=p.password, name=p.name, surname=p.surname, email=p.email, is_admin=p.is_admin)
        with closing(self.outgoing.sql.get(conn).session()) as session:
            session.add(user.User)
            session.commit()
            self.response.status_code = CREATED
            self.response.payload = u
            self.response.headers['Location'] = 'http://127.0.0.1:11223/app/users/get/%s' % (u.id)

So, IMHO, this is not a RESTful approach but it’s valid, working approach to create an API which would be using just GET and POST but could/would work.

Exception handling, paramter sanitization, query string handling to filter, paginate and sort, etc. would have to be added, of course.

For services such as “get a user and all its roles” we would have a channel such as /app/users/{id}/roles that would call the service GetRoles(Service) or GetRolesByUser(Service) inside the user.py file, And so on.

All the code from my experiements is available at https://bitbucket.org/jsabater/genesisng/

Overall, from my two replies, I would really appreciate two things:

  • A proposed way to code a RESTful API with the existing mechanisms in Zato 3.0.
  • A future way to code a RESTful API with Zato 4 and an estimation.

Thanks in advance and keep up the good work.


#6

We don’t have SimpleIO per HTTP verb so the way to go at this point is to have a separate service for each REST channel - this is what works and the end result is as REST-ful as the other approach.

As for how it would like in the future - I still don’t know, I have not been analyzing it closer yet. I’m quite certain that there won’t be decorators to specify which channes a particular method/verb applies to - the overall design is that services need to know as little as possible about the channels they are exposed through, which is why it is then possible to mount the same service through REST, AMQP, SOAP or WebSockets. I.e. REST is just one of protocols.

As for returning a list with multiple results from SimpleIO - please have a look at output_repeated = True, I believe this is what you are looking for. I will add a usage example to documentation soon.


class MyService(Service):
  class SimpleIO:
    output_required = ('id', 'name')
    output_repeated = True

  def handle(self):
    # Assume it is a list of dictionaries with keys 'id' and 'name'
    data = get_data()
    self.response.payload[:] = data

#7

Thanks for your reply, @dsuch.

So, from your reply, I understand that the approach would be the one in my second reply, with a different class for each service. Now, how would the channel configuration need to be? Would we be able to configure channels for each method (GET, PUT, DELETE, POST) of the same URL (e.g. /app/users) so that they would point to the list, update, delete and create service classes? As per my tests, I have not been able to do so.

Thanks.


#8

Regarding returning multiple results, I have been using:

self.response.payload[:] = sqlalchemy_result

with no trouble (a list of dictionaries, as you mention in your reply). I had not noticed about the output_repeated = True parameter. Aside from thanking you for pointing it out, what is the difference? Have I been doing it (half) wrong? What am I missing?

Thanks :slight_smile:


#9

Right now incoming HTTP requests are matched against URL paths only and the HTTP verb is ignored. This is enforced when a channel is created or updated.

This means that you cannot have multiple channels with the same URL path but different verbs, that is, paths need to be unique, such as /app/users/get, /app/users/create (I realize this is redundant if you want to use separate verbs).

If you open a separate ticket in GH for it, I can see about having it take the verb into account though I’m not clear yet how to design it - but I would just like to keep track of it in GH.


#10

Yes, this will also work - there is no difference between a list of dictionaries and a list of SQLAlchemy objects as far as output_repeated goes; support for the latter is explicitly available, I just did not know you were using it already.


#11

Hi, @dsuch.

I have added the ticket in Github:

Thanks.


#12

Following up on the original discussion on this thread, in order to be able to use the def handle_VERB(self) feature when implementing REST APIs and make use of all of the advantages of SimpleIO, I agree with @jjmurre on the possible approach:

class MyService(Service):
    class SimpleIO_GET:
        input_required = ('id')
    def handle_GET(self):
        pass

    class SimpleIO_POST:
        input_required = ('name', 'surname')
        input_optional = ('email')
    def handle_POST(self):
        pass

This approach would follow the current handle_VERB(self): modus operandi.

Also, I think that this discussion would be related to issue #877 on github as, if implementing #877 and implementing this issue, we could have all methods required for the classic two-endpoints-multiple-verbs REST approach in one single service.

GET /app/users calls user.user and executes handle_GET
POST /app/users calls user.user and executes handle_POST
PUT /app/users/{id} calls user.user and executes handle_PUT
DELETE /app/users/{id} calls user.user and executes handle_DELETE
GET /app/users/{id} calls user.user and executes handle_GET

There would still be the issue of telling the difference from the first call (list of users) from the last call (details of a single user), but that could be sorted out by having two different services, such as:

GET /app/users calls user1.user and executes handle_GET
POST /app/users calls user1.user and executes handle_POST
PUT /app/users/{id} calls user2.user and executes handle_PUT
DELETE /app/users/{id} calls user2.user and executes handle_DELETE
GET /app/users/{id} calls user2.user and executes handle_GET

What are your thoughts on this matter?