sympasoap oddity with utf-8 input about sympa HOT 8 CLOSED

Okay, that seems to have worked on the live system as well. Thanks for your help here!

from sympa.

Hi @dpc22 ,

This is correctly encoded using the 2 byte UTF-8 sequence: "0xc2 0xa3" in my SOAP client.

Please provide a sample of the input data, the client script you used and how you made sure the client encoded it correctly.

from sympa.

I attach my example Python script which fails (.txt extension required by github)

sync.py.txt

The equivalent Perl script seems to work:

sync.pl.txt

The only obvious difference is:

$soap->default_ns('urn:sympasoap');

I can't find a direct equivalence to "$soap->default_ns()" in the Zeep library that I am using in Python.

There is: "zeep.set_ns_prefix()", but that takes two arguments.

     |  set_ns_prefix(self, prefix, namespace)
     |      Set a shortcut for the given namespace.

The following didn't help:

zeep.set_ns_prefix(None, 'urn:sympasoap');

Afraid that I don't know what SOAP namespaces do, so I'm blundering around in the dark rather.

from sympa.

I'm pretty sure that my Python code was originally derived from: https://pypi.org/project/sympasoap/.

That doesn't seem to do anything with namespaces either.

(Edit to add)

It also has a normalize method which just discards any non-ASCII characters on the GeCOS field before invoking the SOAP add method. Presumably the author ran into the same issue, but didn't come up with a more sensible fix.

from sympa.

https://docs.python-zeep.org/en/master/transport.html#debugging

tells me how to dump the raw XML which is sent to the sympasoap server.

The raw HTTP POST request was:

zeep.transports: HTTP Post to https://test.lists.cam.ac.uk/sympasoap:
<?xml version='1.0' encoding='utf-8'?>
<soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/"><soap-env:Body><ns0:add xmlns:ns0="urn:sympasoap"><list>test-dpc22</list><email>[email protected]</email><gecos>Test £</gecos><quiet>true</quiet></ns0:add></soap-env:Body></soap-env:Envelope>

We have <xml ... encoding='utf-8'>

The <gecos> field appears to be correctly encoded as UTF-8: if I send the output to a file and use "od -c", I see the two byte sequence: "0xc2 0xa3" sent by the SOAP client.

0000440   l   >   <   g   e   c   o   s   >   T   e   s   t     302 243
0000460   <   /   g   e   c   o   s   >   <   q   u   i   e   t   >   t

>>> hex(0o302)
'0xc2'
>>> hex(0o243)
'0xa3'

I have a dedicated test server if I can add useful debugging at the server end. The normal Sympa verbose logging didn't tell me anything.

from sympa.

@dpc22, could you please apply #1592 and check if the problem will be solved?

from sympa.

Thank you.

That seems to have fixed the problem on my test server.

I did need to add a patch for src/lib/Makefile.in in order to backport your fix from the GIT repository to the 6.2.72 release tarball given:

rename from src/lib/Sympa/WWW/SOAP/Transport.pm
rename to src/lib/Sympa/WWW/SOAP/FastCGI.pm

I will apply the fix to the live system either tomorrow morning or Monday morning.

from sympa.

Duplicate of #1541.

from sympa.