Comments (9)
Looking into it.
from blingfire.
I also find this problem of not supporting macOs platform. Really hope for the open source of it.
from blingfire.
from blingfire.
Same problem:
$ python3 -m virtualenv env
Using base prefix '/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7'
/Users/bosma/Library/Python/3.7/lib/python/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
New python executable in /Users/bosma/projects/env/bin/python3.7
Also creating executable in /Users/bosma/projects/env/bin/python
Installing setuptools, pip, wheel...done.
$ source env/bin/activate
$ pip install blingfire
Collecting blingfire
Downloading https://files.pythonhosted.org/packages/6b/aa/f999e3ca3e826cbdefe7dea7f90baf06db26448ef4cfd80f4800d84d660f/blingfire-0.0.9-py3-none-any.whl (248kB)
|████████████████████████████████| 256kB 1.5MB/s
Installing collected packages: blingfire
Successfully installed blingfire-0.0.9
$ python
Python 3.7.0 (default, Aug 17 2018, 21:14:48)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.from blingfire import *
Traceback (most recent call last):
File "", line 1, in
File "/Users/bosma/projects/env/lib/python3.7/site-packages/blingfire/init.py", line 5, in
import numpy as np
ModuleNotFoundError: No module named 'numpy'
text = 'This is the Bling-Fire tokenizer'
output = text_to_words(text)
KeyboardInterrupt$ pip install numpy
Collecting numpy
Downloading https://files.pythonhosted.org/packages/43/6e/71a3af8680a159a141fab5b4d19988111a09c02ffbfdeb42175cca0fa341/numpy-1.16.3-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (13.9MB)
|████████████████████████████████| 13.9MB 2.1MB/s
Installing collected packages: numpy
Successfully installed numpy-1.16.3
$ python
Python 3.7.0 (default, Aug 17 2018, 21:14:48)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.from blingfire import *
text = 'This is the Bling-Fire tokenizer'
output = text_to_words(text)
Traceback (most recent call last):
File "", line 1, in
File "/Users/bosma/projects/env/lib/python3.7/site-packages/blingfire/init.py", line 58, in text_to_words
o_len = blingfire.TextToWords(c_char_p(s_bytes), c_int(len(s_bytes)), byref(o_bytes), c_int(o_bytes_count))
AttributeError: 'NoneType' object has no attribute 'TextToWords'
from blingfire.
Looking into it.
Any suggested fixes to make it work no macOS?
from blingfire.
there is no mac OS support:
BlingFire/scripts/blingfiretok.py
Lines 14 to 22 in 52a5d30
from blingfire.
fwiw, I was able to get it working on Mac with the following changes: master...ankane:mac. There's probably a cleaner way to do it.
from blingfire.
Thank you, Andrew, for doing the change!
Starting from version 0.0.16 blingfire package supports Mac.
from blingfire.
Awesome, thanks @SergeiAlonichau!
from blingfire.
Related Issues (20)
- Byte offsets for original input bytes to allow non-destructive tokenization
- Trouble installing for custom model creation HOT 2
- M2M100 Marianmt tokenizers
- what is the last char of the last word from GetWords?
- How to create i2w model HOT 1
- Missing numpy dependency on setup.py
- Missing vcruntime140.dll and vcruntime140_1.dll dependencies HOT 4
- Could java call the tokenizer of bin file
- Add xlm-roberta-large tokenization support
- BlingFire fails with all-lowercase text
- "terminate called after throwing an instance of 'std::runtime_error'" HOT 1
- Issues building on Mac OSX M2 HOT 1
- Unable to Modify Tokenization Logic
- /O2 in CMakeLists.txt is incompatible with vcpkg using Ninja
- Import issue on MacOS M1 HOT 5
- Support for CLIP tokenizers from Hugging Face
- Build_Dll_For_Linux_ARM64 job fails
- c# example negative offset for Starts
- Loading the bert_base_tok.bin model sometimes throws an exception
- Unable to load DLL 'blingfiretokdll' or one of its dependencies: 找不到指定的模块。 (0x8007007E) System.DllNotFoundException: Unable to load DLL 'blingfiretokdll' or one of its dependencies: 找不到指定的模块。 (0x8007007E) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from blingfire.