(Migrated) Adding more XML security

(This message has been automatically imported from the retired mailing list)

Hello,

this is something that was preliminary discussed off-list but we’ve
moved it here so others can perhaps get involved.

The idea is to add more in the area of XML security. In particular, what
I would personally like to see is a simple means to encrypt, decrypt,
verify and sign incoming/outgoing messages.

Unfortunately, it is my sad experience that working with xmlsec bindings
in Python is a frustrating process. Last time I tried it I ended up
invking xmlsec1 in a subprocess

http://mail.python.org/pipermail/soap/2012-November/001004.html

which was OK in that given situation because that particular SAML2
Identity Provider needed so much time to complete all the operations
that a fraction of second worth of overhead I introduced was something
negligible.

In Zato though I’d rather not take this approach and would like to add
full support with GUI, enmasse, docs, examples and a clean API.

Ultimately, as with other security-related stuff a service would need to
be concerned with whatever way it was secured on a given channel. So if
someone creates a SOAP/XML channel, pick a ‘needs decryption, key name =
foobar’ flag, the service will simply receive the payload and everything
will be taken care of by Zato itself.

For people who need to access raw requests, API along the lines of what
is here would be provided

Can you please suggest what libraries to use?

These are the ones I’m aware of

  1. https://pypi.python.org/pypi/dm.xmlsec.binding/1.0b4

  2. https://github.com/dnet/SudsSigner

  3. https://pypi.python.org/pypi/pyxmlsec-next/0.3.1

  4. Tried it Sep 2012, could not get it work but I can see there were new
    releases since then

  5. Never used it but I know the author is a sharp fellow

  6. Never used it but I know the author is subscribed to this list :slight_smile:

If someone was willing to contribute these 4 standalone functions

decrypt(key_as_string, msg)
encrypt(key_as_string, msg)
verify(key_as_string, msg)
sign(key_as_string, msg)

(if it’s better not use strings but some other objects, like BIO, feel
free to suggest it)

I’d be very happy to plug them into Zato - I’d add key aliases so you
can update keys from browser/CLI/API, GUI to assign security to
channels/outconns and generally, would make Zato take care of how to use
them.

I just wouldn’t discourage any potential contributors by saying the full
stack must be updated, I can happily do it myself.

A thing to keep in mind is that everything must support
Ubuntu/Debian/Mind/Fedora, OS X. Also RHEL/SLES. And Windows with time.
In other words, this should need as little SWIG magic as possible,
preferably should be as self-contained as possible.

About RHEL/SLES/Windows -> these systems will have fat installers
anyway, all pre-compiled dependencies will be bundled along but whatever
library/libraries will be used, they must work on Linux, OS X and Windows.

What do you think?

On 07/08/2013 07:48 PM, Dariusz Suchojad wrote:

Ultimately, as with other security-related stuff a service would need to
be concerned with whatever way it was secured on a given channel. So if
someone creates a SOAP/XML channel, pick a ‘needs decryption, key name =
foobar’ flag, the service will simply receive the payload and everything
will be taken care of by Zato itself.

Naturally, I meant a service would /not/ be concerned with security :slight_smile:

If someone was willing to contribute these 4 standalone functions

decrypt(key_as_string, msg)
encrypt(key_as_string, msg)
verify(key_as_string, msg)
sign(key_as_string, msg)

(if it’s better not use strings but some other objects, like BIO, feel
free to suggest it)

I’d be very happy to plug them into Zato - I’d add key aliases so you
can update keys from browser/CLI/API, GUI to assign security to
channels/outconns and generally, would make Zato take care of how to use
them.

Another point is, the feature discussed here could land in zato-labs
for now

If we had these high-level standalone functions, convenience wrappers
could be add in labs and the full support would be added in one of
future releases.

In other words, this is something that could be used straight-away,
without waiting for a proper release.

On 07/08/2013 07:48 PM, Dariusz Suchojad wrote:

Ultimately, as with other security-related stuff a service would need to
be concerned with whatever way it was secured on a given channel. So if
someone creates a SOAP/XML channel, pick a ‘needs decryption, key name =
foobar’ flag, the service will simply receive the payload and everything
will be taken care of by Zato itself.

Naturally, I meant a service would /not/ be concerned with security :slight_smile:

If someone was willing to contribute these 4 standalone functions

decrypt(key_as_string, msg)
encrypt(key_as_string, msg)
verify(key_as_string, msg)
sign(key_as_string, msg)

(if it’s better not use strings but some other objects, like BIO, feel
free to suggest it)

I’d be very happy to plug them into Zato - I’d add key aliases so you
can update keys from browser/CLI/API, GUI to assign security to
channels/outconns and generally, would make Zato take care of how to use
them.

Another point is, the feature discussed here could land in zato-labs
for now

If we had these high-level standalone functions, convenience wrappers
could be add in labs and the full support would be added in one of
future releases.

In other words, this is something that could be used straight-away,
without waiting for a proper release.

On 07/08/2013 07:48 PM, Dariusz Suchojad wrote:

  1. https://pypi.python.org/pypi/pyxmlsec-next/0.3.1

Hm, this one is GPL so it’s a showstopper.

http://pyxmlsec.labs.libre-entreprise.org/

I’ll contact the authors, there must be more people who’d like to use
this library in non-GPL projects. Maybe the license can be changed to LGPL?

On 07/08/2013 07:48 PM, Dariusz Suchojad wrote:

  1. https://pypi.python.org/pypi/pyxmlsec-next/0.3.1

Hm, this one is GPL so it’s a showstopper.

http://pyxmlsec.labs.libre-entreprise.org/

I’ll contact the authors, there must be more people who’d like to use
this library in non-GPL projects. Maybe the license can be changed to LGPL?

Hi

El lunes, 8 de julio de 2013, Dariusz Suchojad escribi=F3:

Hello,

this is something that was preliminary discussed off-list but we’ve moved
it here so others can perhaps get involved.

The idea is to add more in the area of XML security. In particular, what
I would personally like to see is a simple means to encrypt, decrypt,
verify and sign incoming/outgoing messages.

Unfortunately, it is my sad experience that working with xmlsec bindings
in Python is a frustrating process. Last time I tried it I ended up invking
xmlsec1 in a subprocess

http://mail.python.org/pipermail/soap/2012-November/001004.html

which was OK in that given situation because that particular SAML2
Identity Provider needed so much time to complete all the operations that a
fraction of second worth of overhead I introduced was something negligible.

That’s what Roland Hedberg of the pysaml2 did. But having this in
production with the django-saml2 lib (lgs) presented some problems in
high traffic sites / hours because unix forking is suboptimal for this use
case.

In Zato though I’d rather not take this approach and would like to add
full support with GUI, enmasse, docs, examples and a clean API.

Ultimately, as with other security-related stuff a service would need to
be concerned with whatever way it was secured on a given channel. So if
someone creates a SOAP/XML channel, pick a ‘needs decryption, key name =3D
foobar’ flag, the service will simply receive the payload and everything
will be taken care of by Zato itself.

For people who need to access raw requests, API along the lines of what
is here would be provided

https://gist.github.com/dsuch/5950797

Can you please suggest what libraries to use?

These are the ones I’m aware of

  1. https://pypi.python.org/pypi/dm.xmlsec.binding/1.0b4

  2. https://github.com/dnet/SudsSigner

  3. https://pypi.python.org/pypi/pyxmlsec-next/0.3.1

  4. Tried it Sep 2012, could not get it work but I can see there were new
    releases since then

  5. Never used it but I know the author is a sharp fellow

I tried it but some Axis 1.4 WS I worked with didn’t understand it because
it needed the BinaryToken element in XML and the complete client
certificate. So I cloned it in github (erny)

  1. Never used it but I know the author is subscribed to this list :slight_smile:

Well this is just a copy with the patch referenced by dnet so I could
install it with pip (setuptools / distribute /buildout)
:wink:

But SudsSigner plugin uses pyxmlsec. So these are not exactly alternatives.
I think the pysaml2 list enumerated some alternatives. May be lxml has
already some support? We just need the C14n stuff which I think is part of
the libxslt part of libxml2.

If someone was willing to contribute these 4 standalone functions

decrypt(key_as_string, msg)
encrypt(key_as_string, msg)
verify(key_as_string, msg)
sign(key_as_string, msg)

This would be cool, but I don’t know if this would be enough for the axis
service to work. As dnet stated it seems to work with Apache Cxf.

(if it’s better not use strings but some other objects, like BIO, feel
free to suggest it)

I’d be very happy to plug them into Zato - I’d add key aliases so you can
update keys from browser/CLI/API, GUI to assign security to
channels/outconns and generally, would make Zato take care of how to use
them.

I just wouldn’t discourage any potential contributors by saying the full
stack must be updated, I can happily do it myself.

A thing to keep in mind is that everything must support
Ubuntu/Debian/Mind/Fedora, OS X. Also RHEL/SLES. And Windows with time. In
other words, this should need as little SWIG magic as possible, preferably
should be as self-contained as possible.

I haven’t had any problems using SudsSigner in CentOS 6 / 64.

About RHEL/SLES/Windows -> these systems will have fat installers anyway,
all pre-compiled dependencies will be bundled along but whatever
library/libraries will be used, they must work on Linux, OS X and Windows.

What do you think?

I think that having first class support for WS-* in python, just like SAML2
and other OASIS / IETF standards would be great but I understand that some
of the specs are rather complex and very expensive to implement.

For now, standard SOAP WS-Security BinaryToken would be nice. On the other
hand I would make it optional as it may require to install special
software. So its support could be detected on start time.

Regards.
Erny

On 07/09/2013 08:23 PM, Ernesto Revilla Derksen wrote:

We just need the C14n stuff which I
think is part of the libxslt part of libxml2.

Sorry, do you mean ‘we’ as in Zato’s feature we’re discussing or you and
your co-workers?

If someone was willing to contribute these 4 standalone functions

decrypt(key_as_string, msg)
encrypt(key_as_string, msg)
verify(key_as_string, msg)
sign(key_as_string, msg)

This would be cool, but I don’t know if this would be enough for
the axis service to work. As dnet stated it seems to work with Apache Cxf.

Exactly, testing it out with popular Java/.NET frameworks is surely needed.

But hm, what do you mean by its not being enough? Do you think that
these 4 basic functions won’t suffice?

Also, we cannot use anything that is based, directly or not, on pysecxml
because it’s GPL. I asked the author to re-license the library under
LGPL and we’ll see how it goes. But for now this cannot be used in Zato.

I think that having first class support for WS-* in python, just like
SAML2 and other OASIS / IETF standards would be great but I understand
that some of the specs are rather complex and very expensive to implement.

I’d be very happy if Zato was a SAML2 client only for now. Wouldn’t
really want to turn it into a full-featured Identity Provider as there
are other projects specializing in it though naturally if someone wishes
to add such a thing for their own needs I won’t stop anyone.

Of OASIS standards, I’m particularly eager to implement all the things
that are to do with distributed transactions - this is something that
Python doesn’t have and it would be great to have it. Both in SOAP and
JSON and indeed, in any data format.

For now, standard SOAP WS-Security BinaryToken would be nice. On the
other hand I would make it optional as it may require to install special
software. So its support could be detected on start time.

Sure, it’s like Oracle or WebSphere MQ. Not everyone uses it so they
need to be enabled manually. The same thing could be done here.

https://zato.io/docs/admin/guide/enabling-extra-libs.html

Hi

El mi=E9rcoles, 10 de julio de 2013, Dariusz Suchojad escribi=F3:

On 07/09/2013 08:23 PM, Ernesto Revilla Derksen wrote:

We just need the C14n stuff which I
think is part of the libxslt part of libxml2.

Sorry, do you mean ‘we’ as in Zato’s feature we’re discussing or you and
your co-workers?

“We” as the zato interested people (no co-workers)

If someone was willing to contribute these 4 standalone functions

decrypt(key_as_string, msg)
encrypt(key_as_string, msg)
verify(key_as_string, msg)
sign(key_as_string, msg)

This would be cool, but I don’t know if this would be enough for
the axis service to work. As dnet stated it seems to work with Apache
Cxf.

Exactly, testing it out with popular Java/.NET frameworks is surely neede=
d.

The test directory could have some minimal Apache Axis and Cxf Ws
projects.

But hm, what do you mean by its not being enough? Do you think that these
4 basic functions won’t suffice?

May be I’m wrong or may be there is a parameter missing (target format /
hash / signature algorithm).

Also, we cannot use anything that is based, directly or not, on pysecxml
because it’s GPL. I asked the author to re-license the library under LGPL
and we’ll see how it goes. But for now this cannot be used in Zato.

Validating SAML2 assertions would be nice. I could imagine some use cases
but currently I don’t have the need (although it could arise soon. This
comes very handy when dealing with SAML2 SSO and credentials or AuthN
scheme have to be converted. )

Thanks to this message:
http://lists.labs.libre-entreprise.org/pipermail/pyxmlsec-devel/2012-July/0=
00093.html
I just found dm.xmlsec.binding:

https://pypi.python.org/pypi/dm.xmlsec.binding/1.1b1
It seems to be BSD, maintained and documented.

I’ll try to create a version of SudsSigner which uses this binding.

I think that having first class support for WS-* in python, just like

SAML2 and other OASIS / IETF standards would be great but I understand
that some of the specs are rather complex and very expensive to implemen=
t.

I’d be very happy if Zato was a SAML2 client only for now. Wouldn’t reall=
y
want to turn it into a full-featured Identity Provider as there are other
projects specializing in it though naturally if someone wishes to add suc=
h
a thing for their own needs I won’t stop anyone.

Of OASIS standards, I’m particularly eager to implement all the things
that are to do with distributed transactions - this is something that
Python doesn’t have and it would be great to have it. Both in SOAP and JS=
ON
and indeed, in any data format.

Sorry, but here I can’t be of much help.

For now, standard SOAP WS-Security BinaryToken would be nice. On the

other hand I would make it optional as it may require to install special
software. So its support could be detected on start time.

Sure, it’s like Oracle or WebSphere MQ. Not everyone uses it so they need
to be enabled manually. The same thing could be done here.

https://zato.io/docs/admin/**guide/enabling-extra-libs.html<https://zato.=
io/docs/admin/guide/enabling-extra-libs.html>

Great. I’ll have a look.

Regards.
Erny

On 07/10/2013 11:14 PM, Ernesto Revilla Derksen wrote:

      > decrypt(key_as_string, msg)
      > encrypt(key_as_string, msg)
      > verify(key_as_string, msg)
      > sign(key_as_string, msg)

The test directory could have some minimal Apache Axis and Cxf Ws
projects.

May be I’m wrong or may be there is a parameter missing (target format /
hash / signature algorithm)./dm.xmlsec.binding/1.1b1
It seems to be BSD, maintained and documented.

I’ll try to create a version of SudsSigner which uses this binding.

That would be very cool! I understand that of the four methods above,
this would let one verify XML Security signatures?