Like grep but for natural language questions. Based on Mixtral 8x7B. ~15 tokens/s on Nvidia RTX 3070 with 8GB memory.
If nvidia driver that supports cuda 12.1 exists, it installs cuda version, else cpu version. It's ~48GB.
curl https://raw.githubusercontent.com/moritztng/fltr/main/install.sh -o install.sh && bash install.sh && source ~/.bashrc
fltr --file emails.txt --prompt "Is the following email spam? Email:" --batch-size 32
It will output all lines in the file where the answer is yes.