Apitest path params and inexistence of keys in response


#1

Hi everyone!

A question regarding zato-apitest. When you have a channel such as /app/users/{id}/get and you want to pass 1 as the value of {id}, how do you do that? It’s not a JSON pointer, it’s not a query string param, it’s not a header, so I’ve run out of options.

Also, regarding JSON responses, I see there is no option to check whether a key does not exist. You can check it’s empty, and you can check a header does not exist, but you cannot check a key in the response dictionary does not exist. For instance, a user’s password (which should never be returned to the client side in my application).

Thanks.


#2

You need to refer to a variable using the syntax as below. Normally, you would just use #user_id or @user_id but if the variables are to be embedded in strings, then this special syntax is needed.

Naturally, the variable may come from a previous invocation, below it is constant just for illustration purposes.

    Given I store "1" under "user_id"
    Given URL path "/app/users/#{user_id}/get"

As to your other question, there is no step to confirm that a key does not exist. The closest is “The JSON Pointer {path} is null” but it would still require a key, even if with a null value.

Creating new steps is very easy - they are all just one to five lines of code, so you can easily add one and send it for inclusion. Just let me know if you need assistance with it.


#3

I have set the following variables in the features/config.ini file:

[user]
address=http://127.0.0.1:11223
app=genesisng
path_login_get=logins/#{id}/get
default_login_id=1

Should I be able to do this?

Feature: Login details

Scenario: REST login details

    Given address "@address"
    Given I store "@default_login" under "id"
    Given URL path "/@app/@path_login_get"
    Given HTTP method "GET"
    Given format "JSON"

    When the URL is invoked

    Then response is equal to that from "login-get.json"
    And status is "200"

And json/response/login-get.json would be:

{"response": {"username": "jsabater", "surname": "Sabater", "name": "Jaume", "id": 1, "is_admin": true, "email": "jsabater@gmail.com"}}

This is the result I am getting:

$ apitest run ~/Projects/genesisng/apitest/
Feature: Login details

  Scenario: REST login details 
    Given address "@address"
    Given I store "@default_login_id" under "id"
    Given URL path "/@app/@path_login_get"
    Given HTTP method "GET"
    Given format "JSON"
    When the URL is invoked
      Traceback (most recent call last):
        File "/usr/local/lib/python2.7/dist-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/usr/local/lib/python2.7/dist-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "/usr/local/lib/python2.7/dist-packages/zato/apitest/steps/common.py", line 67, in when_the_url_is_invoked
          ctx.zato.response.data_impl = json.loads(ctx.zato.response.data_text)
        File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
          return _default_decoder.decode(s)
        File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
          obj, end = self.raw_decode(s, idx=_w(s, 0).end())
        File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
          raise ValueError("No JSON object could be decoded")
      ValueError: No JSON object could be decoded

    Then response is equal to that from "login-get.json"
    And status is "200"

Yes, I’ve seen you can extend it. I’ll check it out and see if I can do this extension.


#4

You are likely getting a JSON parsing error because the server replies with a plain 404 Not Found message.

I have never considered the case of configuration from config.ini referring to variables that exist only in runtime, so your exact syntax will not work - the # sign is treated as a comment indicator in config.ini.

What will work is below, i.e. you can refer to variables and mix them with URL paths but only in feature files, not in config.ini.

[user]
app=genesisng
default_login_id=1
Given I store "@default_login_id" under "id"
Given address "http://localhost:17010"
Given URL path "/@{app}/logins/#{id}/get"

#5

Hi, @dsuch. Thanks for the reply. Sounds fine by me.

Regarding the JSON parsing error, I believe it, indeed, has to do with the payload of the service response being empty:

self.response.status = NOT_FOUND
self.response.payload = ''

The second scenario in the following test fails:

Feature: Login validation
  
Scenario: REST login successful validation

    Given address "@address"
    Given URL path "/genesisng/logins/validate"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-validate-jsabater.json"

    When the URL is invoked

    Then response is equal to that from "login-validate-jsabater.json"
    And status is "200"

    And context is cleaned up

Scenario: REST login unsuccessful validation

    Given address "@address"
    Given URL path "/genesisng/logins/validate"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-validate-yoda.json"

    When the URL is invoked

    Then response is equal to ""
    And status is "404"

This is the output:

Feature: Login validation

  Scenario: REST login successful validation 
    Given address "@address"
    Given URL path "/genesisng/logins/validate"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-validate-jsabater.json"
    When the URL is invoked
    Then response is equal to that from "login-validate-jsabater.json"
    And status is "200"
    And context is cleaned up

  Scenario: REST login unsuccessful validation 
    Given address "@address"
    Given URL path "/genesisng/logins/validate"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-validate-yoda.json"
    When the URL is invoked
      Traceback (most recent call last):
        File "/usr/local/lib/python2.7/dist-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/usr/local/lib/python2.7/dist-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "/usr/local/lib/python2.7/dist-packages/zato/apitest/steps/common.py", line 67, in when_the_url_is_invoked
          ctx.zato.response.data_impl = json.loads(ctx.zato.response.data_text)
        File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
          return _default_decoder.decode(s)
        File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
          obj, end = self.raw_decode(s, idx=_w(s, 0).end())
        File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
          raise ValueError("No JSON object could be decoded")
      ValueError: No JSON object could be decoded

    Then response is equal to ""
    And status is "404"

If I remove the Then response is equal to “” I get the same result.

Should I be returning an empty dictionary as payload to sort this out? I really thought an empty payload was the way to go.

Thanks.


#6

To return an empty string, you would need to build it like below, otherwise there is no JSON object to deserialize on zato-apitest’s end.

self.response.payload = '""'

E.g.

>>> import json
>>> json.loads('')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Yet:

>>> json.loads('""')
u''
>>> 

If you use SimpleIO then you can just not assign anything explicitly in which case an empty dictionary will be returned - otherwise, since you are returning a string, Zato does not inspect it and gives you full control over what the output is, even if there is none at all.

Afterwards, in your API test, I think you could just use the “Then JSON response doesn’t exist” step unless you need to check an exact datatype.

All of this looks to be a corner-case of not returning anything vs. returning an empty string and how it is interpreted in Python or elsewhere.

I have just tried it in JavaScript and JSON.parse works the same:

JSON.parse('');  // SyntaxError: Unexpected end of JSON input
JSON.parse('""'); // Fine

#7

Thanks for the feedback, @dsuch. I believe I started assigning ‘’ to payloads when I saw it somewhere on the docs/forum but when I jumped to SimpleIO I kept using it without taking into consideration the feedback you just provided me with, which makes a lot of sense. I am now not assigning anything to self.response.payload when I don’t have anything to return and the tests are working fine.

One thing, though, is that I am not testing for an empty or non-existent response when using apitest because I cannot find the test you mentioned. The closest thing I could find was Then JSON Pointer “/” is empty but the test failed anyway. Not a problem, though, unless not testing for an empty response could, in some case I haven’t run into yet, be a problem.

Thanks!


#8

So I removed all the self.response.payload = ‘’ lines I had and now I am a better situation but not yet there. I have one test with two scenarios, creation and deletion of the just created record:

Feature: Login creation and deletion
  
Scenario: REST login creation with required fields only

    Given address "@address"
    Given URL path "/genesisng/logins/create"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-create-lskywalker.json"

    When the URL is invoked

    Then JSON Pointer "/response/id" isn't empty
    And JSON Pointer "/response/username" is "lskywalker"
    And JSON Pointer "/response/name" is "Luke"
    And JSON Pointer "/response/email" is "lskywalker@gmail.com"
    And JSON Pointer "/response/is_admin" is false
    And status is "201"
    And header "Location" isn't empty
    And I store "/response/id" from response under "id"

Scenario: REST login deletion of previously created login

    Given address "@address"
    Given URL path "/genesisng/logins/#{id}/delete"
    Given HTTP method "GET"
    Given format "JSON"

    When the URL is invoked

    Then status is "204"

login-create-lskywalker.json contains this:

{"username": "lskywalker", "password": "123456", "name": "Luke", "email": "lskywalker@gmail.com"}

Login creation works fine, and I can get the user id and store it into a variable to be used when deleting the login, but login deletion stills ends with the cannot decode JSON error:

$ apitest run ~/Projects/genesisng/apitest/
Feature: Login creation and deletion

  Scenario: REST login creation with required fields only 
    Given address "@address"
    Given URL path "/genesisng/logins/create"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-create-lskywalker.json"
    When the URL is invoked
    Then JSON Pointer "/response/id" isn't empty
    And JSON Pointer "/response/username" is "lskywalker"
    And JSON Pointer "/response/name" is "Luke"
    And JSON Pointer "/response/email" is "lskywalker@gmail.com"
    And JSON Pointer "/response/is_admin" is false
    And status is "201"
    And header "Location" isn't empty
    And I store "/response/id" from response under "id"

  Scenario: REST login deletion of previously created login 
    Given address "@address"
    Given URL path "/genesisng/logins/#{id}/delete"
    Given HTTP method "GET"
    Given format "JSON"
    When the URL is invoked
      Traceback (most recent call last):
        File "/usr/local/lib/python2.7/dist-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/usr/local/lib/python2.7/dist-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "/usr/local/lib/python2.7/dist-packages/zato/apitest/steps/common.py", line 67, in when_the_url_is_invoked
          ctx.zato.response.data_impl = json.loads(ctx.zato.response.data_text)
        File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
          return _default_decoder.decode(s)
        File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
          obj, end = self.raw_decode(s, idx=_w(s, 0).end())
        File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
          raise ValueError("No JSON object could be decoded")
      ValueError: No JSON object could be decoded

    Then status is "204"

The login Delete service now looks as follows:

class Delete(Service):
    """Service class to delete an existing login."""
    """Channel /genesisng/logins/{id}/delete."""

    class SimpleIO:
        input_required = (Integer('id'))

    def handle(self):
        conn = self.user_config.genesisng.database.connection
        id_ = self.request.input.id

        with closing(self.outgoing.sql.get(conn).session()) as session:
            deleted = session.query(Login).filter(Login.id == id_).delete()
            session.commit()

            if deleted:
                self.response.status_code = NO_CONTENT
            else:
                self.response.status_code = NOT_FOUND

I tried adding a line such as self.response.payload = {} but I am still getting the JSON decode error.

Do you have any idea what may be the issue, @dsuch?

Thanks.


#9

What is the response from the server when invoked from command line, e.g. curl -v … ?


#10

I’ve modified the test to just create the login, then I’ve used curl to delete it:

$ apitest run ~/Projects/genesisng/apitest/
Feature: Login creation and deletion

  Scenario: REST login creation with required fields only 
    Given address "@address"
    Given URL path "/genesisng/logins/create"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-create-lskywalker.json"
    When the URL is invoked
    Then JSON Pointer "/response/id" isn't empty
    And JSON Pointer "/response/username" is "lskywalker"
    And JSON Pointer "/response/name" is "Luke"
    And JSON Pointer "/response/email" is "lskywalker@gmail.com"
    And JSON Pointer "/response/is_admin" is false
    And status is "201"
    And header "Location" isn't empty
    And I store "/response/id" from response under "id"

1 feature passed, 0 failed, 0 skipped
1 scenario passed, 0 failed, 0 skipped
14 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.023s
(genesisng) jsabater@ranma:~/Projects/genesisng/apitest/features$ curl -v -g "http://127.0.0.1:11223/genesisng/logins/1026/delete"; echo ""
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 11223 (#0)
> GET /genesisng/logins/1026/delete HTTP/1.1
> Host: 127.0.0.1:11223
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 204 No Content
< Server: Zato
< Date: Mon, 05 Nov 2018 14:22:17 GMT
< Connection: close
< Content-Type: application/json
< X-Zato-CID: 473e2b02a83e26ec68aee84b
< 
* Closing connection 0

FYI, in my previous post I forgot to mention that the newly created login was being deleted despite of the JSON decoding error.


#11

I think I know what is happening - in your test you specify the JSON format but you do not exchange JSON messages.

There is no JSON in the request because user ID is in the path and the response is empty so there is no JSON either.

What if you delete this line, Given format "JSON", from the delete scenario?


#12

Nope, still the same:

$ apitest run ~/Projects/genesisng/apitest/
Feature: Login creation and deletion

  Scenario: REST login creation with required fields only 
    Given address "@address"
    Given URL path "/genesisng/logins/create"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-create-lskywalker.json"
    When the URL is invoked
    Then JSON Pointer "/response/id" isn't empty
    And JSON Pointer "/response/username" is "lskywalker"
    And JSON Pointer "/response/name" is "Luke"
    And JSON Pointer "/response/email" is "lskywalker@gmail.com"
    And JSON Pointer "/response/is_admin" is false
    And status is "201"
    And header "Location" isn't empty
    And I store "/response/id" from response under "id"

  Scenario: REST login deletion of previously created login 
    Given address "@address"
    Given URL path "/genesisng/logins/#{id}/delete"
    Given HTTP method "GET"
    When the URL is invoked
      Traceback (most recent call last):
        File "/usr/local/lib/python2.7/dist-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/usr/local/lib/python2.7/dist-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "/usr/local/lib/python2.7/dist-packages/zato/apitest/steps/common.py", line 67, in when_the_url_is_invoked
          ctx.zato.response.data_impl = json.loads(ctx.zato.response.data_text)
        File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
          return _default_decoder.decode(s)
        File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
          obj, end = self.raw_decode(s, idx=_w(s, 0).end())
        File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
          raise ValueError("No JSON object could be decoded")
      ValueError: No JSON object could be decoded

    Then status is "204"


Failing scenarios:
  login-create.feature:22  REST login deletion of previously created login

0 features passed, 1 failed, 0 skipped
1 scenario passed, 1 failed, 0 skipped
17 steps passed, 1 failed, 1 skipped, 0 undefined
Took 0m0.033s

I also tried adding these two lines to the deletion scenario:

    Given format "JSON"
    Given request is "{}"

But the result was still the same.


#13

Ah, I see it now.

When you specify Given format "JSON" in the first scenario it is inherited in the following one. This is usually a nice thing because it lets you create setup scenarios first with common configuration, that do not even invoke anything, but that other scenarios make use of.

In this case, though, you do not have JSON when you delete users so the implicitly inherited format is an obstacle.

I can see two choices:

  • In the first scenario, you end it with And context is cleaned up which will clean up anything that would have been inherited in further scenarios

  • In the second scenario, you explicitly provide Given format "RAW"


#14

Tried both with no luck:

  • I cannot use And context is cleaned up because I need the id from the created record to delete it in the second scenario.
  • I tried the Given format "RAW" but it still fails.

This is the test right now:

Feature: Login creation and deletion

Scenario: REST login creation with required fields only

    Given address "@address"
    Given URL path "/genesisng/logins/create"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-create-lskywalker.json"

    When the URL is invoked

    Then JSON Pointer "/response/id" isn't empty
    And JSON Pointer "/response/username" is "lskywalker"
    And JSON Pointer "/response/name" is "Luke"
    And JSON Pointer "/response/email" is "lskywalker@gmail.com"
    And JSON Pointer "/response/is_admin" is false
    And status is "201"
    And header "Location" isn't empty
    And I store "/response/id" from response under "id"

Scenario: REST login deletion of previously created login

    Given address "@address"
    Given URL path "/genesisng/logins/#{id}/delete"
    Given HTTP method "GET"
    Given format "RAW"

    When the URL is invoked

    Then status is "204"

This is the output of the test:

$ apitest run ~/Projects/genesisng/apitest/
Feature: Login creation and deletion

  Scenario: REST login creation with required fields only 
    Given address "@address"
    Given URL path "/genesisng/logins/create"
    Given HTTP method "POST"
    Given format "JSON"
    Given request "login-create-lskywalker.json"
    When the URL is invoked
    Then JSON Pointer "/response/id" isn't empty
    And JSON Pointer "/response/username" is "lskywalker"
    And JSON Pointer "/response/name" is "Luke"
    And JSON Pointer "/response/email" is "lskywalker@gmail.com"
    And JSON Pointer "/response/is_admin" is false
    And status is "201"
    And header "Location" isn't empty
    And I store "/response/id" from response under "id"

  Scenario: REST login deletion of previously created login 
    Given address "@address"
    Given URL path "/genesisng/logins/#{id}/delete"
    Given HTTP method "GET"
    Given format "RAW"
    When the URL is invoked
    Then status is "204"
      Assertion Failed: Status expected `204`, received `500`

And this is the trace I found on the logs:

2018-11-05 15:37:40,176 DEBG 'zato-server1' stdout output:
2018-11-05 15:37:40,176 - WARNING - 137:DummyThread-122 - zato.common.util:488 - Could not parse request as JSON:`username=lskywalker&password=123456&name=Luke&email=lskywalker%40gmail.com`, e:`Traceback (most recent call last):
  File "/opt/zato/3.0/code/zato-common/src/zato/common/util/__init__.py", line 486, in payload_from_request
    payload = loads(request)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/anyjson/__init__.py", line 135, in loads
    return implementation.loads(value, *args, **kwargs)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/anyjson/__init__.py", line 99, in loads
    return self._decode(s, *args, **kwargs)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/simplejson/__init__.py", line 488, in loads
    return _default_decoder.decode(s)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/simplejson/decoder.py", line 389, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
ValueError: Expecting value: line 1 column 1 (char 0)
`

2018-11-05 15:37:40,177 DEBG 'zato-server1' stdout output:
2018-11-05 15:37:40,176 - ERROR - 137:DummyThread-122 - zato.server.connection.http_soap.channel:324 - Caught an exception, cid:`8c824acf60e4c72ed17eaf45`, status_code:`500`, _format_exc:`Traceback (most recent call last):
  File "/opt/zato/3.0/code/zato-server/src/zato/server/connection/http_soap/channel.py", line 268, in dispatch
    payload, worker_store, self.simple_io_config, post_data, path_info, soap_action)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/connection/http_soap/channel.py", line 502, in handle
    params_priority=channel_item.params_pri)
  File "/opt/zato/3.0/code/zato-server/src/zato/server/service/__init__.py", line 431, in update_handle
    payload = payload_from_request(cid, raw_request, data_format, transport)
  File "/opt/zato/3.0/code/zato-common/src/zato/common/util/__init__.py", line 486, in payload_from_request
    payload = loads(request)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/anyjson/__init__.py", line 135, in loads
    return implementation.loads(value, *args, **kwargs)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/anyjson/__init__.py", line 99, in loads
    return self._decode(s, *args, **kwargs)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/simplejson/__init__.py", line 488, in loads
    return _default_decoder.decode(s)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/opt/zato/3.0/code/local/lib/python2.7/site-packages/simplejson/decoder.py", line 389, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
ValueError: Expecting value: line 1 column 1 (char 0)

Maybe my approach is not an expected one? How would you have me test the Delete service? Load the test data, then execute tests, which will try to delete a known id? Then, if I have to test again, do it after droping and creating the schema, and loading the test data again? (so that the scenario is exactly as before)


#15

Your approach is fine, there is no need to recycle test data before each scenario - you can just set it up by invoking your APIs, this is good.

What happens here is similar to the previous situation:

  • In your first scenario you have Given request "login-create-lskywalker.json"
  • This is inherited by the second scenario
  • The second scenario does not use JSON (data format=RAW), which is as expected
  • But the second scenario still sends the inherited JSON request
  • That JSON request appears to be actually a query string / POST string
  • A Zato channel of yours has JSON as its data format configured
  • It is this Zato channel that receives the request from the JSON file and it is the channel that raises this exception

That is, I assume that the contents of “login-create-lskywalker.json” is not JSON?

This all means that cleaning up the context after the first step is the best approach, seeing as you do not want to inherit anything really.

I have never seen such a situation before because I always use JSON to convey everything, without relaying anything in headers, query string and other HTTP meta-constructs (most of my projects use REST as just one of their protocols), so it did not occur to me initially what was going on.


#16

Okay, now I understand that scenarios do inherit everything from the previous ones. I didn’t think it went that far.

The JSON file used to create the login, login-create-lskywalker.json, is, indeed, a JSON, such as:

{"username": "lskywalker", "password": "123456", "name": "Luke", "email": "lskywalker@gmail.com"}

The whole problem here is the id that the database generates for this new record, which I need in order to delete if afterwards.

So cleaning up the context would be the way to go, as per your comments. Except that I will lose the id I stored, won’t I?

And I store "/response/id" from response under "id"

Is there a way to clean up everything but keep the variable?


#17

Right, I missed your note that the ID was needed - in that case, instead of cleaning up the context, can you send a blank request in the delete scenario, i.e. keep the the RAW format but with a step such as Given request is "{}".

That is in the way of a workaround because it is not possible to clean up the context while retaining parts of it. It is a good idea to make it possible, though.


#18

Yup, that worked! Finally! I’ve been stuck in this for a week! :stuck_out_tongue: