Comments (2)
@jenniferajiang - I second this as well. A cogstack-nifi 'slim' version was suggested in our prev call for the SLaM folks also. Ticket created on our dev board. CU-8692wktvq
from medcat.
@jenniferajiang - Due to how PyPI & pip are configured, along with how pytorch is deployed we cannot have a cpu only wheel of MedCAT.
PyPI is set up so it only downloads wheels from "https://pypi.org/simple", unless changed explicitly from the user (such as --index-url
like you've done here). This is for security purposes so you can always be sure that the wheel is coming from a trusted source, or the user consciously changed the source yourself.
Pytorch also do not offer CPU-only wheels on PyPI, they're available at https://download.pytorch.org/whl/cpu
. In setup.py you're very limited in changing the source of the wheel you want. You provide a specific wheel via:
setup
(
...
install_requires = ["torch@https://download.pytorch.org/whl/cpu/
torch-2.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"]
...
)
which isn't ideal as there are multiple versions depending on versions, python, os, architecture, etc...
I've looked at multiple work arounds for this, and all of them offering this functionality have been purposely closed off (requirements.txt, setup.cfg, pyproject.toml). I think it's better for us to accept the constraints rather than find another work around which would likely be removed.
My first suggestion is we document how to install a CPU only version - using the options provided to us by pip:
pip install medcat --extra-index-url https://download.pytorch.org/whl/cpu/
this will install the most relevant up to date version of pytorch for the cpu given our constraints provided in setup.py (currently torch>=1.13.0
). This solution seems the most common among people I've seen trying to do the same as us here.
The second suggestion would be a wrapper bash script that would install MedCAT along with whatever extra parameters are required. This could be called when starting up a docker so the preferred version is installed saving network / disk usage.
I've added the extra documentation to the README.md and added a bash wrapper setup.sh
for the pip install command (https://github.com/CogStack/MedCAT/tree/cpu-only).
Added conversations regarding PyPI and Pytorch:
Pytorch - pytorch/pytorch#26340
PyPI - https://discuss.python.org/t/specifying-extra-index-url-in-setup-cfg-option-dependencies/19377
from medcat.
Related Issues (20)
- Upgrade spacy dependency HOT 3
- Loosen dependency constraints HOT 1
- How to use CUI filtering
- Version 1.6.1 is not on pypi HOT 1
- new install attempt with SNOMED model throws spacy/thinc config validation error HOT 4
- remove elasticsearch dependency HOT 5
- Show nested ents results in error HOT 2
- Modifing the weights of words in the models HOT 3
- i am running NER-L tutorial but while running this cell, model and dataset are not downloading. HOT 1
- Model resources no longer available. HOT 4
- TypeError: Snomed._refset_df2dict() takes 1 positional argument but 2 were given HOT 1
- Concept not found if token order is slightly changed contrary to mentioned note in paper HOT 3
- Error: [E050] when trying to load sample annotate entity project
- MedCAT model used in validation HOT 3
- Stopwords do not load properly HOT 4
- How do I download the models? HOT 3
- Service unavailable HOT 4
- How to use SNOMED trained model HOT 4
- Writing SNOMED dataframe - cannot access local variable in preprocess_snomed file HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from medcat.