Usage of output_required and output_optional when a SimpleIO service that returns a list

#1

When using SimpleIO in a service, if we are returning a list of dictionaries, we shall use:

  • output_repeated = True attribute in the SimpleIO class.
  • self.response.payload[:] = output in the service.

In this type of service, we know the attributes present in the dictionary (id, name, surname, etc.) but we don’t know how many rows we’ll be returning (from zero to a lot). So, how should we configure output_required and output_optional in the SimpleIO class?

For example:

# Channel GET /app/customers calls customer.get-customer
class GetCustomers(Service):

    class SimpleIO:
        input_required = ()
        output_optional = ('id', 'name', 'surname', 'email', ...)
        output_repeated = True

    def handle(self):
        conn = self.kvdb.conn.get('app:database:conn')
        with closing(self.outgoing.sql.get(conn).session()) as session:
            self.response.payload[:]  = session.query(Customer).order_by(Customer.id)

Would we be using output_optional or output_required in this example?

Thanks.

#2

You can use output_required for keys that you know for 100% that will be available on output, that is, you guarantee that they will exist in each dictionary. In case they are missing, an exception will be raised - this is validation of output in other words.

On the other hand, you can use output_optional if you are not sure if a given key will be available in each dictionary.

A practical example will be:

output_required = ('id', 'username')
output_optional = ('email', 'display_name')

… assuming that in your data model users must always have an ID and their username but email and display name are optional.

You can also play around with skip_empty_keys = True/False and allow_empty_required = True/False.

They are new in 3.0, I will add docs for these options under ticket #876 that you opened, for which I am grateful.

#3

So, in summary, output_required and output_optional:

  • When returning a single dictionary, they validate the keys of such dictionary, as you’ve described in your reply.
  • When returning a list of dictionaries, they validate the keys of each dictionary in the list (the same way you’ve described before but for each tuple).

And using output_required or output_optional depends on what you consider the minimum and maximum number of fields in the dictionary (useful when support field projections on resources, I guess).

Correct? :slight_smile:

#4

Yes, this is correct and I would only add that the emphasis is on which keys are returned. E.g. your SQLAlchemy models returned by a particular query may have thirty attributes but with SIO you may want to return only six specific ones. Same goes for dictionary keys.

#5

I have been implementing fields projection on my prototype service (yup, still testing Zato, heh :slight_smile:). So, using SimpleIO I process calls such as:

/app/users/list
/app/users/list?fields=name&fields=surname 
/app/users/list?fields=name&fields=surname&fields=email

And so on. So, if my output_required looks like:

output_required = ('id', 'username', 'name', surname', 'email', 'is_admin')

When the client uses fields projection to get a subset of fields (e.g. just the name and the surname), the resulting JSON has a lot of empty keys. When no fields projection is used, the returned JSON is just fine.

So, is there a way to dinamically change the value of output_required when the code detects a valid use of the fields parameter?

Thanks in advance.

#6

SimpleIO definitions are of static nature, in fact, they are resolved when a service is deployed.

I wonder if skip_empty_keys = True flag of a SIO definition is not what you are looking for?

#7

By the way, if the fields all seem to be optional then would you not rather use output_optional instead of output_required?

#8

My fault, it was already output_optional, therefore my question. I tried with skip_empty_keys = True and it worked the way we expected :slight_smile:

Currently, my SimpleIO class looks like this:

class SimpleIO:
    input_required = ()
    input_optional = ('page', 'size', 'sort_by', 'order_by', 'filters', 'search', 'fields')
    output_optional = ('id', 'username', 'password', 'name', 'surname', 'email', 'is_admin')
    output_repeat = True
    skip_empty_keys = True

Thanks for the help.

#9

Looks good though instead of output_repeat it would be output_repeated and you do not need to declare anything if you do not use it, i.e. input_required can be omitted unless you meant for this class to be subclassed and for children classes to append to it, e.g.:

class BaseSIO:
  input_required = ()
  output_required = ('abcd', 'efgh')

class ChildSIO(BaseSIO):
  input_required = BaseSIO.input_required + ('elem1', 'elem2')