Ng Wai Foong
1 min readJul 12, 2019

--

Hi,

What did you put at the header? I was not referring to the data but rather the encoder.py file. In the following line 19, some of the characters are non-ASCII characters (example is ¬). By default, python is parsed as ASCII and this is why it is throwing an error. You need to specify the encoding at the top or saved the file as utf-8 via text editor such as Notepad++.

bs = list(range(ord("!"), ord("~")+1))+list(range(ord("¡"), ord("¬")+1))+list(range(ord("®"), ord("ÿ")+1))

If your data contains characters that required different encoding, I will recommend you to modify the code at line 109 as well (the following example used utf8):

with open(os.path.join('models', model_name, 'encoder.json'), 'r', encoding='utf-8') as f:

Thanks

--

--

Ng Wai Foong
Ng Wai Foong

Written by Ng Wai Foong

Senior AI Engineer@Yoozoo | Content Writer #NLP #datascience #programming #machinelearning | Linkedin: https://www.linkedin.com/in/wai-foong-ng-694619185/

No responses yet