Comments (1)
Salam,
Thank you for your message,
The tokenization process spearate words from texts only, It doesn't make any analysis on words.
If you want to get lemma or stems from words, I suggest to use Qalsadi Morphological analyzer .
Or you can use only stemmer like Tashaphyne to extract stems.
from pyarabic.
Related Issues (20)
- module 'pyarabic.araby' has no attribute 'sentence_tokenize'
- Import Error - ModuleNotFoundError: No module named 'six' HOT 1
- Issue checking for a valid Arabic word HOT 1
- initial and middle dotless noon is not working HOT 13
- normalize_searchtext import errors + typo's HOT 1
- اضافة خاصية tokenize مع حفظ مواقع الكلمات HOT 1
- function araby.is_arabicword return false for some arabic word HOT 4
- New features: normalizing digits HOT 2
- Clean Arabic Text (quranic marks, esthetic symbols)
- New Feature: Arabic Text Standardize
- Soundex for Arabic text HOT 3
- Correct swaping keyboard error
- Convert Arabic glyphs into standard letters
- Are all of kinds of Arabic text normalization work in PyArabic?
- Package documentation? HOT 1
- Update the strip_tashkeel and strip_diactricts to remove the alef after tanween al fateh HOT 2
- Hi
- normalize_ligature not having the rigth format HOT 10
- prefix and suffix HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyarabic.