Cannot import name 'optional_' from 'zato.common.typing_'

louistwiice · May 13, 2022, 8:23pm

Hi Everyone,

I have followed the tutorial about Data Model. I have created a Basic service for test

from dataclasses import dataclass

# Zato
from zato.common.typing_ import list_, optional_

from zato.server.service import Model, Service
from zato.common import DATA_FORMAT


@dataclass(init=False)
class Phone(Model):
    imei:       str
    owner_id:   int
    owner_name: str
    identifier: optional_[str]


@dataclass(init=False)
class GetPhoneListRequest(Model):
    client_id: int
    name: str


@dataclass(init=False)
class GetPhoneListResponse(Model):
    phone_list:    list_[Phone]
    response_type: str
    client_id: str


class BaseService(Service):

    def before_handle(self):
        self.logger.info('In BASE handle')
        self.response.content_type = 'application/json'


class GetPhoneDetails(BaseService):

    name = 'api.user.get-phone'

    class SimpleIO:
        input  = GetPhoneListRequest
        output = GetPhoneListResponse

    def handle(self):

        # Enable type checking and type completion
        self.logger.info('START')

        request = self.request.input # typeGetPhoneListRequest

        # Log details of our request
        self.logger.info('Processing client `%s`', request.client_id, extra={'cid': self.cid})

        # Build our response now - in a full service this information
        # would be read from an exteran system or database.

        # Our list of phones to return
        phone_list = []

        # Build the fist phone ..
        phone1 = Phone()
        phone1.imei = '123'
        phone1.owner_id = 456
        phone1.owner_name = 'John Doe'

        # .. the second one ..
        phone2 = Phone()
        phone2.imei = '789'
        phone2.owner_id = 999
        phone2.owner_name = 'Jane Doe'
        phone2.identifier = 'JDaef4400bfsH'

        # .. populate the container for phones tha we return ..
        phone_list.append(phone1)
        phone_list.append(phone2)

        # .. build the top-level response element ..
        response = GetPhoneListResponse()
        response.response_type = 'RZH'
        response.client_id = request.client_id
        response.phone_list = phone_list

        # .. and return the response to our caller
        self.response.payload = response

I have created a Phone model and I have set an optional field identifier . After editing my file I’ve had the following error

zato38   | 2022-05-13 20:20:10,989 - INFO - 3065:Dummy-581 - zato.hot-deploy.create:0 - Creating tar archive
zato38   | 2022-05-13 20:20:10,990 - INFO - 3065:Dummy-581 - zato.hot-deploy.create:0 - Creating tar archive
zato38   | 2022-05-13 20:20:11,003 - ERROR - 3065:Dummy-581 - zato.server.service.store:0 - Could not load source, file_name:`/opt/hot-deploy/services/TestPhoneNumber.py`, e:`Traceback (most recent call last):
zato38   |   File "/opt/zato/3.2.0/code/zato-server/src/zato/server/service/store.py", line 1184, in import_objects_from_file
zato38   |     mod_info = import_module_from_path(file_name, base_dir)
zato38   |   File "/opt/zato/3.2.0/code/zato-common/src/zato/common/util/api.py", line 600, in import_module_from_path
zato38   |     spec.loader.exec_module(mod)
zato38   |   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
zato38   |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
zato38   |   File "/opt/hot-deploy/services/TestPhoneNumber.py", line 4, in <module>
zato38   |     from zato.common.typing_ import list_, optional_
zato38   | ImportError: cannot import name 'optional_' from 'zato.common.typing_' (/opt/zato/3.2.0/code/zato-common/src/zato/common/typing_.py)
zato38   | `
zato38   | 2022-05-13 20:20:11,082 - ERROR - 3065:Dummy-581 - zato.server.service.store:0 - Could not load source, file_name:`/opt/hot-deploy/services/TestPhoneNumber.py`, e:`Traceback (most recent call last):
zato38   |   File "/opt/zato/3.2.0/code/zato-server/src/zato/server/service/store.py", line 1184, in import_objects_from_file
zato38   |     mod_info = import_module_from_path(file_name, base_dir)
zato38   |   File "/opt/zato/3.2.0/code/zato-common/src/zato/common/util/api.py", line 600, in import_module_from_path
zato38   |     spec.loader.exec_module(mod)
zato38   |   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
zato38   |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
zato38   |   File "/opt/hot-deploy/services/TestPhoneNumber.py", line 4, in <module>
zato38   |     from zato.common.typing_ import list_, optional_
zato38   | ImportError: cannot import name 'optional_' from 'zato.common.typing_' (/opt/zato/3.2.0/code/zato-common/src/zato/common/typing_.py)
zato38   | `
zato38   | 2022-05-13 20:20:11,133 - WARNING - 3065:Dummy-581 - zato.hot-deploy.create:0 - No services nor models were deployed from module `TestPhoneNumber.py`
zato38   | 2022-05-13 20:20:12,126 - INFO - 3065:Dummy-583 - zato.hot-deploy.create:0 - Creating tar archive
zato38   | 2022-05-13 20:20:12,126 - INFO - 3065:Dummy-583 - zato.hot-deploy.create:0 - Creating tar archive

Do i need to install a package before using zato common?

Thank you very much for your help

dsuch · May 14, 2022, 10:11am

Thanks, the correct import is optional rather than optional_, i.e. the full import will be:

from zato.common.typing_ import optional

The documentation has been corrected now.

louistwiice · May 16, 2022, 9:26am

Thank you!

I have updated and use the correction with the same service. I receive the following error, when I make a POST request on Postman:

zato38   | 2022-05-16 09:21:28,109 - ERROR - 3086:Dummy-147 - zato.server.connection.http_soap.channel:0 - Caught an exception, cid:`77bc3bb88a469906ecbe5a25`, status_code:`HTTPStatus.INTERNAL_SERVER_ERROR`, `Traceback (most recent call last):
zato38   |   File "/opt/zato/3.2.0/code/zato-server/src/zato/server/connection/http_soap/channel.py", line 344, in dispatch
zato38   |     response = self.request_handler.handle(cid, url_match, channel_item, wsgi_environ,
zato38   |   File "/opt/zato/3.2.0/code/zato-server/src/zato/server/connection/http_soap/channel.py", line 690, in handle
zato38   |     response = service.update_handle(self._set_response_data, service, raw_request,
zato38   |   File "/opt/zato/3.2.0/code/zato-server/src/zato/server/service/__init__.py", line 918, in update_handle
zato38   |     raise e
zato38   |   File "/opt/zato/3.2.0/code/zato-server/src/zato/server/service/__init__.py", line 858, in update_handle
zato38   |     self._invoke(service, channel)
zato38   |   File "/opt/zato/3.2.0/code/zato-server/src/zato/server/service/__init__.py", line 743, in _invoke
zato38   |     service.handle()
zato38   |   File "/opt/hot-deploy/services/TestPhoneNumber.py", line 88, in handle
zato38   |     self.response.payload = response
zato38   |   File "src/zato/cy/reqresp/response.py", line 163, in zato.cy.reqresp.response.Response._set_payload
zato38   |   File "/opt/zato/3.2.0/code/zato-common/src/zato/common/marshal_/api.py", line 109, in to_dict
zato38   |     return asdict(self)
zato38   |   File "/usr/lib/python3.8/dataclasses.py", line 1073, in asdict
zato38   |     return _asdict_inner(obj, dict_factory)
zato38   |   File "/usr/lib/python3.8/dataclasses.py", line 1080, in _asdict_inner
zato38   |     value = _asdict_inner(getattr(obj, f.name), dict_factory)
zato38   |   File "/usr/lib/python3.8/dataclasses.py", line 1108, in _asdict_inner
zato38   |     return type(obj)(_asdict_inner(v, dict_factory) for v in obj)
zato38   |   File "/usr/lib/python3.8/dataclasses.py", line 1108, in <genexpr>
zato38   |     return type(obj)(_asdict_inner(v, dict_factory) for v in obj)
zato38   |   File "/usr/lib/python3.8/dataclasses.py", line 1080, in _asdict_inner
zato38   |     value = _asdict_inner(getattr(obj, f.name), dict_factory)
zato38   | AttributeError: 'Phone' object has no attribute 'identifier'
zato38   | `

Postman response is:

{
    "result": "Error",
    "cid": "77bc3bb88a469906ecbe5a25",
    "details": [
        "'Phone' object has no attribute 'identifier'"
    ]
}

I think Zato didn’t take identifer as optional or Maybe I have missed a part during my implementation.

I think optional means I don’t need to initiate a value and when I do not, the default value is None/null systematically. Or do I need to set it to None by myself ?

Thank you for your help !!!

dsuch · May 16, 2022, 9:52am

OK, but what is the request that you are sending from Postman?

louistwiice · May 16, 2022, 10:02am

Here is the full code

from dataclasses import dataclass

# Zato
from zato.common.typing_ import list_, optional

from zato.server.service import Model, Service
from zato.common import DATA_FORMAT


@dataclass(init=False)
class Phone(Model):
    imei:       str
    owner_id:   int
    owner_name: str
    identifier: optional[str]


@dataclass(init=False)
class GetPhoneListRequest(Model):
    client_id: int
    name: str


@dataclass(init=False)
class GetPhoneListResponse(Model):
    phone_list:    list_[Phone]
    response_type: str
    client_id: str


class BaseService(Service):

    def before_handle(self):
        self.logger.info('In BASE handle')
        self.response.content_type = 'application/json'


class GetPhoneDetails(BaseService):

    name = 'api.user.get-phone'

    class SimpleIO:
        input  = GetPhoneListRequest
        output = GetPhoneListResponse

    def handle(self):

        # Enable type checking and type completion
        self.logger.info('START')

        request = self.request.input # typeGetPhoneListRequest

        # Log details of our request
        self.logger.info('Processing client `%s`', request.client_id, extra={'cid': self.cid})

        # Build our response now - in a full service this information
        # would be read from an exteran system or database.

        # Our list of phones to return
        phone_list = []

        # Build the fist phone ..
        phone1 = Phone()
        phone1.imei = '123'
        phone1.owner_id = 456
        phone1.owner_name = 'John Doe' # <------- I don' want to add optional field "identifier"

        # .. the second one ..
        phone2 = Phone()
        phone2.imei = '789'
        phone2.owner_id = 999
        phone2.owner_name = 'Jane Doe'
        phone2.identifier = "AZseed666oybqQwzQd" # <------- I have add optional field "identifier"

        # .. populate the container for phones tha we return ..
        phone_list.append(phone1)
        phone_list.append(phone2)

        # .. build the top-level response element ..
        response = GetPhoneListResponse()
        response.response_type = 'RZH'
        response.client_id = request.client_id
        response.phone_list = phone_list

        # .. and return the response to our caller
        self.response.payload = response
        self.logger.info('response %s', self.wsgi_environ['HTTP_X_FORWARDED_FOR'])

The request

{
    "client_id": "1111",
    "name": "william Norris"
}

The response

{
    "result": "Error",
    "cid": "87386d2e6fe0ac31c4b0322f",
    "details": [
        "'Phone' object has no attribute 'identifier'"
    ]
}

I have put optional field on my response fields, not on my Request fields

dsuch · May 16, 2022, 10:20am

Thanks, let’s just agree - any time you post anything, please send all the code, requests, responses and tracebacks along with the output from “zato --version” without my having to ask for it.

As to this particular case, yes, you need to provide a default value yourself.

On the one hand, this is because that is how dataclasses in Python work. This is a feature built into Python itself and Zato simply uses it.

But, you are right that it would be possible for Zato to apply defaults anyway, it is just there is no clear way to know what it should be in general.

We would think that it should be simple but even in this situation here, we have at least two choices, either make the “identifier” field return None / null or have it return an empty string. In more specific cases, some people would even prefer to return a string such as “novalue” because some other systems would expect it. Or, because this field represents a phone number, the value should really be on a business level, e.g. “+1 111222333”. Or maybe being optional means that this field should not be returned at all.

Even if we skip the more advanced scenarios, we have a few basic choices and all of them are good for some people but not for everyone. This is why you just need to provide the default value yourself.

For instance:

identifier: optional[str] = None

Or:

identifier: optional[str] = ''

Or:

identifier: optional[str] = 'my.default.value'

louistwiice · May 16, 2022, 10:28am

Thank you for the fast reply!

I take a note for the next time.

The way to provide the default value is great. We have others cases where we need other return than None. So that is really helpful. That’s help me for the others cases.