problem stll in environment about dnabert_2 HOT 6 CLOSED

What's your current error message?

from dnabert_2.

no obvious error, just the following, so I asked GPT.

AssertionError Traceback (most recent call last)
Cell In[4], line 3
1 dna = "ACGTAGCATCGGATCTATCTATCGACACTTGGTTATCGATCTACGAGCATCTCGTTAGC"
2 inputs = tokenizer(dna, return_tensors = 'pt')["input_ids"]
----> 3 hidden_states = model(inputs)[0] # [1, sequence_length, 768]
5 # embedding with mean pooling
6 embedding_mean = torch.mean(hidden_states[0], dim=0)

File ~/.conda/envs/dnat/lib/python3.8/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
1509 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1510 else:
-> 1511 return self._call_impl(*args, **kwargs)
.......
File ~/.cache/huggingface/modules/transformers_modules/DNABERT-2-117M/flash_attn_triton.py:1021, in _FlashAttnQKVPackedFunc.forward(ctx, qkv, bias, causal, softmax_scale)
1019 if qkv.stride(-1) != 1:
1020 qkv = qkv.contiguous()
-> 1021 o, lse, ctx.softmax_scale = _flash_attn_forward(
1022 qkv[:, :, 0],
1023 qkv[:, :, 1],
1024 qkv[:, :, 2],
1025 bias=bias,
1026 causal=causal,
1027 softmax_scale=softmax_scale)
1028 ctx.save_for_backward(qkv, o, lse, bias)
1029 ctx.causal = causal

File ~/.cache/huggingface/modules/transformers_modules/DNABERT-2-117M/flash_attn_triton.py:781, in _flash_attn_forward(q, k, v, bias, causal, softmax_scale)
778 assert q.dtype == k.dtype == v.dtype, 'All tensors must have the same type'
779 assert q.dtype in [torch.float16,
780 torch.bfloat16], 'Only support fp16 and bf16'
--> 781 assert q.is_cuda and k.is_cuda and v.is_cuda
782 softmax_scale = softmax_scale or 1.0 / math.sqrt(d)
784 has_bias = bias is not None

And GPT tell me :
It looks like the error is occurring within a custom implementation of attention mechanism (_FlashAttnQKVPackedFunc.forward). The assertion error is raised because the tensors q, k, and v are expected to be on a CUDA device (GPU), but the check assert q.is_cuda and k.is_cuda and v.is_cuda fails.

To resolve this issue, make sure that the tensors involved in the attention mechanism are on the same device. You can achieve this by explicitly moving the tensors to the GPU using the .to(device) method.
so I print the device:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
output： false
So I think I should uninstall torch to cuda these:
pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch (my Cuda is 11.4)

from dnabert_2.

Please try "pip uninstall triton".

from dnabert_2.

Yes, I create a new environment,and I do not pip triton.

from dnabert_2.

It automatically install triton so you need to manually remove it.

from dnabert_2.

YES, you are right!!!! Thank you so much ~~~~~

from dnabert_2.