Comments (3)
I run the code on a A100 GPU but get error:
module 'fused_dense_lib' has no attribute 'linear_act_forward'
my environment:
`Package Version
absl-py 2.0.0
accelerate 0.21.0
adabench 1.2.64
aii-pypai 0.1.40.45
aiofiles 23.2.1
aiohttp 3.9.1
aiosignal 1.3.1
aistudio-analyzer 0.0.4.102
aistudio-common 0.0.28.48
aistudio-notebook 2.0.125
aistudio-serving 0.0.0.62
alipay-pcache 0.1.6
aliyun-python-sdk-core 2.14.0
aliyun-python-sdk-kms 2.16.2
altair 5.2.0
annotated-types 0.6.0
ant-couler 0.0.1rc17
antlr4-python3-runtime 4.9.3
anyio 4.2.0
apex 0.1
archspec 0.2.1
argo-workflows 3.5.1
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
astroid 3.0.2
asttokens 2.4.1
async-timeout 4.0.3
atorch 1.1.0rc8
attrs 23.1.0
autopep8 2.0.4
av 11.0.0
backcall 0.2.0
beautifulsoup4 4.12.2
bigmodelvis 0.0.1
bitarray 2.8.5
bitsandbytes 0.39.0
bleach 6.1.0
blinker 1.7.0
boltons 23.0.0
boto3 1.34.2
botocore 1.34.2
Brotli 1.0.9
cachetools 3.1.1
cattrs 23.2.3
certifi 2023.11.17
cffi 1.16.0
charset-normalizer 2.0.4
cheroot 10.0.0
click 6.7
click-config-file 0.6.0
cloudpickle 3.0.0
colorama 0.4.6
comm 0.2.1
conda 23.11.0
conda-content-trust 0.2.0
conda-libmamba-solver 23.12.0
conda-package-handling 2.2.0
conda_package_streaming 0.9.0
configobj 5.0.8
configparser 6.0.0
contourpy 1.1.1
couler-core 0.1.1rc11
crcmod 1.7
cryptography 41.0.7
cycler 0.12.1
Cython 3.0.6
datasets 2.15.0
debugpy 1.8.0
decorator 5.1.1
deepspeed 0.10.3
defusedxml 0.7.1
delta-center-client 0.0.4
Deprecated 1.2.14
deprecation 2.1.0
diffusers 0.24.0
dill 0.3.7
distlib 0.3.8
distro 1.8.0
dlrover 0.3.6
docker 4.1.0
docstring-to-markdown 0.13
easydl-sdk 0.0.6
einops 0.7.0
entrypoints 0.4
evaluate 0.4.0
exceptiongroup 1.2.0
executing 2.0.1
fairscale 0.4.1
fastapi 0.108.0
fastjsonschema 2.19.1
fastmoe 1.0.0
fasttext 0.9.2
fe 0.3.33
ffmpeg-python 0.2.0
ffmpy 0.3.1
filelock 3.13.1
flake8 6.1.0
flash-attn 2.0.4
flash-attn-1 0.2.6.post2
Flask 3.0.0
flatbuffers 24.3.25
fonttools 4.46.0
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2023.10.0
ftfy 6.1.3
future 1.0.0
gitdb 4.0.11
GitPython 3.1.40
google-auth 2.25.2
google-auth-oauthlib 0.4.6
gradio 4.13.0
gradio_client 0.8.0
grpcio 1.34.1
grpcio-tools 1.34.1
h11 0.14.0
hjson 3.1.0
httpcore 1.0.2
httpx 0.26.0
huggingface-hub 0.19.4
icetk 0.0.7
idna 3.4
importlib-metadata 7.0.0
importlib-resources 6.1.1
iniconfig 2.0.0
ipykernel 6.28.0
ipython 8.12.3
ipython-genutils 0.2.0
isodate 0.6.1
isoduration 20.11.0
isort 5.13.2
itsdangerous 2.1.2
jaraco.functools 4.0.0
jax 0.4.13
jaxlib 0.4.13
jedi 0.19.1
jedi-language-server 0.41.2
Jinja2 2.11.3
jinjasql 0.1.8
jmespath 0.10.0
joblib 1.3.2
jsonpatch 1.32
jsonpath-ng 1.6.0
jsonpointer 2.1
jsonschema 4.20.0
jsonschema-specifications 2023.11.2
jupyter_client 8.6.0
jupyter_core 5.7.1
jupyter-events 0.9.0
jupyter-lsp 2.2.1
jupyter_server 2.10.1
jupyter_server_terminals 0.5.1
jupyterlab_pygments 0.3.0
kiwisolver 1.4.5
kmitool 0.0.9
kubemaker 0.2.17
kubernetes 9.0.0
langdetect 1.0.9
libmambapy 1.5.3
loralib 0.1.1
lsh 0.1.2
lsprotocol 2023.0.0
lxml 4.9.3
M2Crypto 0.38.0
Markdown 3.5.1
markdown-it-py 3.0.0
MarkupSafe 2.0.1
marshmallow 3.20.1
matplotlib 3.7.4
matplotlib-inline 0.1.6
mccabe 0.7.0
mdurl 0.1.2
mediapipe 0.10.11
megatron.core 0.1
menuinst 2.0.1
mistune 0.8.4
ml-dtypes 0.2.0
mock 5.1.0
more-itertools 10.1.0
mpi4py 3.1.5
mpmath 1.3.0
msgpack 1.0.7
multidict 6.0.4
multiprocess 0.70.15
nbclient 0.5.13
nbconvert 6.4.4
nbformat 5.9.2
nest-asyncio 1.5.8
networkx 3.0
ninja 1.11.1.1
nltk 3.8.1
notebook 6.4.6
numpy 1.23.5
oauthlib 3.2.2
odps 3.5.1
omegaconf 2.2.3
opencv-contrib-python 4.9.0.80
opencv-python 4.8.1.78
opendelta 0.3.2
opt-einsum 3.3.0
orjson 3.9.10
oss2 2.6.0
osscmd 0.4.5
overrides 3.1.0
packaging 23.1
pandas 1.0.0
pandocfilters 1.5.0
parameterized 0.9.0
parso 0.8.3
pathos 0.3.0
peft 0.3.0
peppercorn 0.6
pexpect 4.9.0
pickleshare 0.7.5
Pillow 9.3.0
pip 23.3.1
pkgutil_resolve_name 1.3.10
platformdirs 3.10.0
pluggy 1.0.0
ply 3.11
pox 0.3.3
ppft 1.7.6.7
prettytable 3.9.0
prometheus-client 0.19.0
prompt-toolkit 3.0.43
protobuf 3.20.0
psutil 5.9.6
PTable 0.9.2
ptyprocess 0.7.0
pure-eval 0.2.2
py 1.11.0
py-cpuinfo 9.0.0
py-spy 0.3.14
pyaml 21.10.1
pyarrow 12.0.0
pyarrow-hotfix 0.6
pyasn1 0.5.1
pyasn1-modules 0.3.0
pybind11 2.11.1
pycodestyle 2.11.1
pycosat 0.6.6
pycparser 2.21
pycryptodome 3.19.0
pydantic 1.10.8
pydantic_core 2.14.6
pyDes 2.0.1
pydocstyle 6.3.0
pydub 0.25.1
pyflakes 3.1.0
pygls 1.2.1
Pygments 2.17.2
pyhocon 0.3.60
pyinotify 0.9.6
pylint 3.0.3
pynvml 11.4.1
pyodps 0.11.4.1
Pyomo 6.7.0
pyOpenSSL 23.2.0
pyparsing 3.1.1
PySocks 1.7.1
pytest 7.4.3
python-dateutil 2.8.2
python-json-logger 2.0.7
python-lsp-jsonrpc 1.1.2
python-lsp-server 1.9.0
python-multipart 0.0.6
pytoolconfig 1.2.6
pytz 2023.3.post1
PyWavelets 1.4.1
PyYAML 6.0.1
pyzmq 25.1.2
ray 2.9.0
referencing 0.32.0
regex 2023.10.3
requests 2.31.0
requests-file 1.5.1
requests-oauthlib 1.3.1
requests-toolbelt 1.0.0
responses 0.18.0
retry 0.9.2
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.7.0
rope 1.11.0
rouge-chinese 1.0.3
rouge-score 0.1.2
rpds-py 0.14.1
rsa 4.9
ruamel.yaml 0.16.10
ruamel.yaml.clib 0.2.6
ruff 0.1.11
ruff-lsp 0.0.49
s3transfer 0.9.0
safetensors 0.4.1
scikit-learn 1.3.2
scipy 1.10.1
semantic-version 2.10.0
Send2Trash 1.8.2
sentencepiece 0.1.97
setuptools 68.2.2
shellingham 1.5.4
six 1.16.0
smmap 5.0.1
sniffio 1.3.0
snowballstemmer 2.2.0
sounddevice 0.4.6
soupsieve 2.5
sqlparse 0.4.4
stack-data 0.6.3
starlette 0.32.0.post1
stringcase 1.2.0
StringGenerator 0.4.4
sympy 1.12
tabulate 0.8.2
tensorboard 2.11.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorboardX 2.6
termcolor 2.4.0
terminado 0.18.0
testpath 0.6.0
threadpoolctl 3.2.0
tinycss2 1.2.1
titans 0.0.7
tldextract 5.1.1
tokenizers 0.13.3
tomli 2.0.1
tomlkit 0.12.0
toolz 0.12.0
torch 2.1.0+cu121
torchaudio 2.1.0+cu121
torchpippy 0.1.1+cecc4fc
torchvision 0.16.0+cu121
tornado 6.4
tqdm 4.65.0
traitlets 5.14.1
transformers 4.31.0
triton 2.1.0
typer 0.9.0
types-python-dateutil 2.8.19.20240106
typing_extensions 4.9.0
tzdata 2023.3
ujson 5.9.0
uncertainty-calibration 0.1.4
Unidecode 1.3.7
unifile-sdk 0.1.14
uri-template 1.3.0
urllib3 1.26.18
uvicorn 0.25.0
virtualenv 20.25.0
watchdog 2.3.1
wcwidth 0.2.12
web.py 0.62
webcolors 1.13
webencodings 0.5.1
websocket-client 1.7.0
websockets 11.0.3
Werkzeug 3.0.1
wfbuilder 1.0.56.43
wget 3.2
whatthepatch 1.0.5
wheel 0.41.2
wrapt 1.16.0
xattr 1.0.0
xxhash 3.4.1
yacs 0.1.8
yapf 0.40.2
yarl 1.9.4
zdfs-dfs 2.3.2
zeep 4.2.1
zipp 3.17.0
zstandard 0.19.0`
from var.
Try from flash_attn.ops.fused_dense import fused_mlp_func
and check if your fused_mlp_func
works.
Maybe you've installed flash attention but haven't compiled that fused mlp operator. You may run:
cd /path/to/flash-attention
cd csrc/fused_dense_lib && pip3 install .
to compile and install it.
from var.
Try
from flash_attn.ops.fused_dense import fused_mlp_func
and check if yourfused_mlp_func
works. Maybe you've installed flash attention but haven't compiled that fused mlp operator. You may run:cd /path/to/flash-attention cd csrc/fused_dense_lib && pip3 install .to compile and install it.
Thanks for your quick reply, and I get the results~
from var.
Related Issues (20)
- Abnormal sample results with `demo_sample.ipynb` HOT 2
- 请问VQVAE(stage1)阶段是怎样使用多级VectorQuantizer的? HOT 3
- AdaLNSelfAttn.forward 请问下这句目的是什么 HOT 2
- How many GPU did you use for training? HOT 2
- FID misalignment HOT 4
- class VQVAE forward function error HOT 1
- training with multi-gpu but stuck HOT 2
- Can the training log be released?
- training code for VQVAE HOT 2
- 请问下两个阶段ablation的细节 HOT 3
- Scalability to multimodal large language models?
- 大佬 关于demo生成的图像的问题 HOT 1
- Question about the cross-antropy loss average? HOT 3
- 请问下训练512x512分辨率的图像也使用16x16的codebook size吗 HOT 2
- Image reconstruction via Transformer.
- Question on autoregressive_infer_cfg HOT 2
- The 512 ckp?
- Inference after training on own dataset HOT 2
- There was no increase in speed after installing flash-attn and xformer.
- Training code for VQVAE
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from var.