'utf-8' codec can't decode byte 0xa3 in position 0: invalid start byte about chinesener HOT 10 OPEN

I found this question can be solved as below:

in utils.py change as follows:

def test_ner(results, path):
"""
Run perl script to evaluate model
"""
output_file = os.path.join(path, "ner_predict.utf8")
with open(output_file, "w", encoding='utf8') as f:
to_write = []
for block in results:
for line in block:
to_write.append(line + "\n")
to_write.append("\n")
f.writelines(to_write)
eval_lines = return_report(output_file)
return eval_lines

The reason is that only when you write the file use "utf8" can you open the file use "utf8", and it have nothing to do with the tensorflow version.

from chinesener.

请问你解决了吗？

from chinesener.

还没有，悲剧。

from chinesener.

What's wrong with it？My tensorflow is 1.4.

from chinesener.

@yyHaker ,good job, it help me solved this problem,thanks

from chinesener.

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa3 in position 0: invalid start byte

2018-02-02 12:08:27,160 - log\train.log - INFO - iteration:1 step:1000/1044, NER loss: 5.380470
2018-02-02 12:08:36,132 - log\train.log - INFO - evaluate:dev
Traceback (most recent call last):
运行到这还是那个编码问题，你们遇到了吗？

from chinesener.

This is still the encoding problem, you can debug to find the encoding problem

from chinesener.

@yyHaker Thanks!

from chinesener.

This is a encoding problem. If you coding in Linux ,please trans the coding by Notepad++.But ,if you coding in Windows ,Please use this :
import codecs
with codecs.open(filename, 'r', 'utf-8') as f:
#this is your process

from chinesener.

it is very easy. You just need to change the 'utf-8' to 'gbk' in the 'return_report' of 'utils.py'.

from chinesener.