Error loading files with paths with non-ANSI characters about blingfire HOT 2 CLOSED

Hi David

The solution might be to add one more Load function that takes wchar_t name as input. Feel free to prepare a PR for this.

Also as described here https://stackoverflow.com/questions/23285759/fopen-file-name-with-utf8-string-in-windows you can try getting a short file name for your file name and feed that.

I am not sure if setting locale to UTF-8 (setlocale(LC_ALL, ".65001");) would make it work as well with UTF-8 string passed as a file name.

An alternative solution can be to take a path as UTF-8 string (as a super set of ASCII) and if there are any non ascii bytes then convert it to wchat_t (utf16le ?) string and use the _wfopen_s under the hood.

Thank you!

from blingfire.

The approach described in https://stackoverflow.com/questions/23285759/fopen-file-name-with-utf8-string-in-windows worked well for us. We're now only sending short path names to BlingFire, which works well for all our supported scenarios.

If I find any shortcomings that require us to add a wide character Load function I'll prepare a PR. For now I'll close this issue.

Thank you!

from blingfire.