[Question] Mac run but get importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes about llava HOT 3 OPEN

Plz anybody help me with this, i really want to run a local llava on my mac 🙏

from llava.

I got different errors.

My command is:

python -m llava.serve.model_worker --device mps --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path liuhaotian/llava-v1.5-13b

The error message is:

raise ImportError(PROTOBUF_IMPORT_ERROR.format(error_message))
2024-06-30 16:02:02 | ERROR | stderr | ImportError:
2024-06-30 16:02:02 | ERROR | stderr | The new behaviour of LlamaTokenizer (with self.legacy = False) requires the protobuf library but it was not found in your environment. Checkout the instructions on the
2024-06-30 16:02:02 | ERROR | stderr | installation page of its repo: https://github.com/protocolbuffers/protobuf/tree/master/python#installation and follow the ones
2024-06-30 16:02:02 | ERROR | stderr | that match your environment. Please note that you may need to restart your runtime after installation.

from llava.

Question

I want to run this model on my mac (with 16G RAM and M1).but since i followed the instructions int the macos.readme

when i am trying to start a model.(4-bit) using the following command:

python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path liuhaotian/llava-v1.5-13b --load-4bit --device mps

The full output it returns:

#### I tried: I tried to pip install bitsandbytes to fix it, but this time when i run i get :

RuntimeError: No GPU found. A GPU is needed for quantization.

No response

I think it asked to uninstall bitsandbytes in instruction for Mac.

from llava.