Encoding problem about urlwatch HOT 12 CLOSED

For which URL does this happen?

from urlwatch.

http://ecrime-project.eu/

the text that causes the error is in the footer of the webpage

from urlwatch.

Ok, I've tested with this page and it properly specifies UTF-8 both in the HTTP header and in the meta http-equiv tag. So I guess you have any filters enabled for this URL? If so, which?

from urlwatch.

yes, html2text

from urlwatch.

What is your system locale set to? You can check with the following snippet:

$ python3
[...]
>>> import sys
>>> sys.getdefaultencoding()
'utf-8'

from urlwatch.

also utf8

>>> import sys
>>> sys.getdefaultencoding()
'utf-8'

from urlwatch.

do you also have the problem only when running as a cronjob?
then look for the solution here:
#48

from urlwatch.

It looks like Python's stdout is set to latin-1 for some reason. Can you please run the following?

$ python3
[...]
>>> import sys, os
>>> sys.stdout.encoding
'UTF-8'
>>> os.environ.get('PYTHONIOENCODING', '')
''

from urlwatch.

python3
>>> import sys, os
>>> sys.stdout.encoding
'ISO-8859-1'
>>> os.environ.get('PYTHONIOENCODING', '')
''

from urlwatch.

Try running urlwatch with PYTHONIOENCODING="UTF-8" urlwatch. That should solve your problem. I just wonder why your Python uses latin-1 as stdout encoding...
What distribution are you using?

from urlwatch.

Arch Linux

from urlwatch.

PYTHONIOENCODING="UTF-8" urlwatch does the trick for me! Thanks

from urlwatch.