Memory climbs to >12GB using: ./verbaflow -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct convert before being killed by OS (out of memory).
Is this normal?

Thanks for your contributions :)

`
Jays-iMac-Pro:verbaflow jay.binks$ go build ./cmd/verbaflow

github.com/nlpodyssey/rwkv

../../go/pkg/mod/github.com/nlpodyssey/[email protected]/layer.go:16:14: undefined: ag.Scalar
../../go/pkg/mod/github.com/nlpodyssey/[email protected]/rwkv.go:51:28: undefined: ag.Scalar
../../go/pkg/mod/github.com/nlpodyssey/[email protected]/rwkv.go:76:35: undefined: ag.Scalar
../../go/pkg/mod/github.com/nlpodyssey/[email protected]/state.go:28:14: undefined: ag.Var
../../go/pkg/mod/github.com/nlpodyssey/[email protected]/state.go:29:14: undefined: ag.Var
../../go/pkg/mod/github.com/nlpodyssey/[email protected]/state.go:30:14: undefined: ag.Var
../../go/pkg/mod/github.com/nlpodyssey/[email protected]/state.go:31:14: undefined: ag.Var
../../go/pkg/mod/github.com/nlpodyssey/[email protected]/state.go:32:14: undefined: ag.Var
`

Hi

I cloned the repo and ran ./verbaflow convert rwkv-4-pile-1b5-instruct/ after downloading the model, but it fails:

1:08PM DBG Converting model in dir: rwkv-4-pile-1b5-instruct/
1:08PM FTL error="class not found: torch BFloat16Storage"

Any ideas?

First of all, thank you for providing such a great open source project! I tried the default model in the README, everything is OK, except when I want the model to output Chinese. The output is garbled.

macos
go 1.20

4:12PM TRC Input: "\nQ: 中文会不会？\n\nA:"
4:12PM TRC Building prompt from template: "{{.Text}}"
4:12PM TRC Input fields: {Text:
Q: 中文会不会？

A: Question: TargetLanguage:}
4:12PM TRC Final prompt: "\nQ: 中文会不会？\n\nA:"
ä¸įä¼ļ4:13PM DBG Done.
^^^^^^^

After following the instructions in the README to download, convert, and format the model, running a prompt would take an incredibly long time, often waiting a few minutes at a time per word. I'm using a 2021 M1 Pro for context. Should it be this slow?

1.5 billion model works as intended. But 3 billion model refuses to convert on my system.

(mps) ➜  verbaflow git:(main) ✗ ./verbaflow -model-dir models/nlpodyssey/rwkv-4-pile-3b-instruct download
(mps) ➜  verbaflow git:(main) ✗ ./verbaflow -model-dir models/nlpodyssey/rwkv-4-pile-3b-instruct convert
5:28PM FTL error="model conversion failed: failed to convert embeddings: only BFloat16Storage is supported, actual *pytorch.FloatStorage"

Similarly, 1.5 billion model works well with original BlinkDL/RWKV-LM code, but 3 billion model throws dimension mismatch error. Is current 3B model working correctly at your end ?