(Migrated) https proxy

(This message has been automatically imported from the retired mailing list)

Hi,

I want to use a http proxy with my zato servers. As described in docs,
i set misc.http_proxy to my proxy server which is sqiud allowing
transparent https.

It only works outgoing http connections, but https connections fails.
For example, in my plain_http channel definition if I point
http://api.ipify.org/, it returns my proxy ip address as expected. If
I point https://api.ipify.org/, it returns my zato server ip address,
not the one of proxy.

So I added HTTP_PROXY and HTTPS_PROXY parameters into os_environ
section, now everything is OK. This method is described here:
http://docs.python-requests.org/en/latest/user/advanced/#proxies

I have three question:

  • Why doesn’t misc.http_proxy parameter handle https?
  • Is it OK to configure proxies in os_environ as I did?
  • And what if I need to use proxy for only some specific connections,
    not all outgoings?

Thanks,

On 30/10/15 16:19, Ali Rıza KELEŞ wrote:

Hi Ali,

[…]

So I added HTTP_PROXY and HTTPS_PROXY parameters into os_environ
section, now everything is OK. This method is described here:
http://docs.python-requests.org/en/latest/user/advanced/#proxies

I have three question:

  • Why doesn’t misc.http_proxy parameter handle https?

That’s an omission that can be rectified in a patch release. I wasn’t
aware one could set HTTPS_PROXY in addition to the other one.

  • Is it OK to configure proxies in os_environ as I did?

Yes, this is fine - misc.http_proxy sets the HTTP_PROXY in os.environ
exactly just like you did manually. It’s just a convenience shortcut.

  • And what if I need to use proxy for only some specific connections,
    not all outgoings?

There is no direct support for doing it, as in GUI and config entries
specific to that feature, but you can use [os_environ] to set NO_PROXY
to a comma separated list of addresses that should not use proxies
defined in HTTP_PROXY and HTTPS_PROXY.

This is the other direction around, I understand it. Please open a
ticket in GH with a feature suggestion that proxies be per-outgoing
connection in addition to os.environ.

thanks,

On 31/10/15 12:22, Ali Rıza KELEŞ wrote:

https://github.com/zatosource/zato/issues/495

This is important for my case. I can write some lines of code for zato.
I will fork and try to implement this feature. You may guide me from
where I should start to examine zato source code…

Thanks, the thing is first to find out how to tell the requests package
to use a proxy or not in a given call.

The invoke_http method here …

https://github.com/zatosource/zato/blob/master/code/zato-server/src/zato/server/connection/http_soap/outgoing.py#L70

As you can see, the method accepts both *args and **kwargs.

These are the same *args and **kwargs you use in an outgoing connection
call, for instance:

conn = self.outgoing.plain_http[‘My Conn’].conn
conn.post(self.cid, ‘data’, some_arg=‘value’)

Now this some_arg is passed to requests 1:1 through **kwargs above.

So if there is any way to signal through an argument that requests use
or not an HTTP proxy, you can just provide it in your call in the
outgoing connection.

I’m not sure if there is one, you can check its docs + source code.

Other than that, the *_PROXY variables are read from os.environ so if
the above doesn’t work we may need to set a required proxy in os.environ
right before a call and unset it right after it somehow taking into
account the fact that it’s a per-process variable, i.e. all HTTP calls
within a Zato server share the same os.environ dictionary.

On Oct 31, 2015 1:19 AM, “Dariusz Suchojad” dsuch@zato.io wrote:

This is the other direction around, I understand it. Please open a
ticket in GH with a feature suggestion that proxies be per-outgoing
connection in addition to os.environ.

https://github.com/zatosource/zato

https://github.com/zatosource/zato/issues/495

This is important for my case. I can write some lines of code for zato. I
will fork and try to implement this feature. You may guide me from where I
should start to examine zato source code…

Thanks…


Ali Riza

Hi Darius,

I have another issue with proxies.

For proxy usage, suds needs a proxy option which must be set like this:

from suds.client import Client
from suds.cache import NoCache
wsdl=3D'http://wdsl.example.com/?wdsl'
*proxies =3D {'http': 'proxy.example.com:3128'}*
cli =3D Client(wsdl, cache=3DNoCache(), *proxy=3Dproxies*)

Suds cares neither misc.http_proxy settings nor os_environ.http_proxy
settings. I have to tell suds proxy as above and it works perfectly.

Is there any way to make Zato’s suds proxy usage as above?

Thanks.

On 31 October 2015 at 13:39, Dariusz Suchojad dsuch@zato.io wrote:

On 31/10/15 12:22, Ali R=C4=B1za KELE=C5=9E wrote:

https://github.com/zatosource/zato/issues/495

This is important for my case. I can write some lines of code for zato.
I will fork and try to implement this feature. You may guide me from
where I should start to examine zato source code…

Thanks, the thing is first to find out how to tell the requests package
to use a proxy or not in a given call.

The invoke_http method here …

https://github.com/zatosource/zato/blob/master/code/zato-server/src/zato/=
server/connection/http_soap/outgoing.py#L70

As you can see, the method accepts both *args and **kwargs.

These are the same *args and **kwargs you use in an outgoing connection
call, for instance:

conn =3D self.outgoing.plain_http[‘My Conn’].conn
conn.post(self.cid, ‘data’, some_arg=3D’value’)

Now this some_arg is passed to requests 1:1 through **kwargs above.

So if there is any way to signal through an argument that requests use
or not an HTTP proxy, you can just provide it in your call in the
outgoing connection.

I’m not sure if there is one, you can check its docs + source code.

Other than that, the *_PROXY variables are read from os.environ so if
the above doesn’t work we may need to set a required proxy in os.environ
right before a call and unset it right after it somehow taking into
account the fact that it’s a per-process variable, i.e. all HTTP calls
within a Zato server share the same os.environ dictionary.

On 19/11/15 10:19, Ali Rıza KELEŞ wrote:

Suds cares neither misc.http_proxy settings nor os_environ.http_proxy
settings. I have to tell suds proxy as above and it works perfectly.

Is there any way to make Zato’s suds proxy usage as above?

Hi Ali,

I’m afraid there isn’t any short of your patching suds after the
installation of Zato in order to support the use case.

We already have a few of patches for various libraries, so please have a
look at the locations below in your /opt/zato/current directory:

  • buildout.cfg (e.g. patch_requests1 key)
  • patches (e.g. requests)

You can add your patches in same way and then run ./bin/buildout which
will apply them.