lmstudio-ai / configs Goto Github PK

some tools can modify the download URL by setting HF_ENDPOINT like:

https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/constants.py#L57

which is previous motioned in:

https://huggingface.co/docs/huggingface_hub/v0.13.3/package_reference/environment_variables

I think adapting this can solve following #1, #18 issues in some degree

Cant figure out how to change max token length from 1500.

Gemma models that have been quantized using Llamacpp are not working. Please look into the issue

error

"llama.cpp error: 'create_tensor: tensor 'output.weight' not found'"

I will open a issue on the llamacpp github aswell addressing this

ggerganov/llama.cpp#5706

System:
Ryzen 5600x
rtx 3080 gpu
b550 motherboard
64gb ddr4 ram
windows 10 OS

Hi, i'm getting this error on Ubuntu 22:04
09:19:35.587 › GPU info: '00:02.0 VGA compatible controller: Intel Corporation Device a7a0 (rev 04)' 09:19:35.604 › Got GPU Type: intel 09:19:35.605 › LM Studio: gpu type = Intel A JavaScript error occurred in the main process Uncaught Exception: Error: /usr/local/lib/libclblast.so.1: impossibile aprire il file oggetto condiviso: File o directory non esistente at process.func [as dlopen] (node:electron/js2c/asar_bundle:2:1822) at Module._extensions..node (node:internal/modules/cjs/loader:1326:18) at Object.func [as .node] (node:electron/js2c/asar_bundle:2:1822) at Module.load (node:internal/modules/cjs/loader:1096:32) at Module._load (node:internal/modules/cjs/loader:937:12) at f._load (node:electron/js2c/asar_bundle:2:13330) at Module.require (node:internal/modules/cjs/loader:1120:19) at require (node:internal/modules/cjs/helpers:103:18) at 6829 (/tmp/.mount_LM_StuGLW2O1/resources/app/.webpack/main/index.js:2:2077) at r (/tmp/.mount_LM_StuGLW2O1/resources/app/.webpack/main/index.js:8:420935) libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null) ^Csimone@simone-ThinkBook-14-G6-IRL:~/Applicazioni$

I have downloaded mistral instruct, and then tried to chat with it. It returns "(Exit code: 1). Please check settings and try loading the model again. " when i select it. Terminal shows this:
Error: libcuda.so.1: cannot open shared object file: No such file or directory
at process.func [as dlopen] (node:electron/js2c/asar_bundle:2:1822)
at Module._extensions..node (node:internal/modules/cjs/loader:1326:18)
at Object.func [as .node] (node:electron/js2c/asar_bundle:2:1822)
at Module.load (node:internal/modules/cjs/loader:1096:32)
at Module._load (node:internal/modules/cjs/loader:937:12)
at f._load (node:electron/js2c/asar_bundle:2:13330)
at Module.require (node:internal/modules/cjs/loader:1120:19)
at require (node:internal/modules/cjs/helpers:103:18)
at 4001 (/tmp/.mount_LM_StuIiaazs/resources/app/.webpack/main/utility.js:2:4008)
at n (/tmp/.mount_LM_StuIiaazs/resources/app/.webpack/main/utility.js:2:594240) {
code: 'ERR_DLOPEN_FAILED'
}

Node.js v18.14.0
[appserver] Error loading model: {
"cause": "(Exit code: 1). Please check settings and try loading the model again. ",
"suggestion": "",
"title": "Model error"
}

          The same issue, today, in Windows 10 with LM-Studio-0.2.10-Setup.exe & LM-Studio-0.2.10-Setup-avx-beta-4.exe, in the middle of the installing progress and also and the same Error at the ending..

Laptop Asus ASUSTeK COMPUTER INC.
product X75A1

Microsoft Windows 10 (10.0) Professional 64-bit (Build 19045)
DirectX Version 12.0
Number of cores 2 (max 2)
Number of threads 2 (max 2)
Manufacturer GenuineIntel
Name Intel Celeron 1000M
Codename Ivy Bridge
Specification Intel(R) Celeron(R) CPU 1000M @ 1.80GHz
Package (platform ID) Socket 988B rPGA (0x4)
CPUID 6.A.9
Extended CPUID 6.3A
Core Stepping E1/L1
Technology 22 nm
TDP Limit 35.0 Watts
Tjmax 105.0 �C
Core Speed 1769.3 MHz
Multiplier x Bus Speed 18.0 x 98.3 MHz
Base frequency (cores) 98.3 MHz
Base frequency (mem.) 98.3 MHz
Stock frequency 1800 MHz
Instructions sets MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, EM64T
Microcode Revision 0x21
L1 Data cache 2 x 32 KB (8-way, 64-byte line)
L1 Instruction cache 2 x 32 KB (8-way, 64-byte line)
L2 cache 2 x 256 KB (8-way, 64-byte line)
L3 cache 2 MB (8-way, 64-byte line)
FID/VID Control yes

Chipset

Northbridge Intel Ivy Bridge rev. 09
Southbridge Intel HM70 rev. 04
Bus Specification PCI-Express 2.0 (5.0 GT/s)
Memory Type DDR3
Memory Size 12 GBytes
Channels Dual
Memory Frequency 786.4 MHz (1:6)
CAS# latency (CL) 11.0
RAS# to CAS# delay (tRCD) 11
RAS# Precharge (tRP) 11
Cycle Time (tRAS) 28
Row Refresh Cycle Time (tRFC) 208
Command Rate (CR) 1T
Host Bridge 0x0154

Memory SPD
DIMM # 1
SMBus address 0x52
Memory type DDR3L
Module format SO-DIMM
Module Manufacturer(ID) SK Hynix (AD00000000000000000000000000)
SDRAM Manufacturer (ID) SK Hynix (AD00000000000000000000000000)
Size 8192 MBytes
Max bandwidth PC3-12800 (800 MHz)
Part number HMT41GS6MFR8C-PB
Serial number 35CE8862
Manufacturing date Week 33/Year 23
Number of banks 8
Nominal Voltage 1.35 Volts
EPP no
XMP no
AMP no
EXPO no
JEDEC timings table CL-tRCD-tRP-tRAS-tRC @ frequency
JEDEC #1 5.0-5-5-14-19 @ 380 MHz
JEDEC #2 6.0-6-6-16-22 @ 457 MHz
JEDEC #3 7.0-7-7-19-26 @ 533 MHz
JEDEC #4 8.0-8-8-22-30 @ 609 MHz
JEDEC #5 9.0-9-9-24-33 @ 685 MHz
JEDEC #6 10.0-10-10-27-37 @ 761 MHz
JEDEC #7 11.0-11-11-28-39 @ 800 MHz

Originally posted by @Galapagospeed in #14 (comment)

The reason why this happens is that there is a % character at the end of the following json file:
I got: Error reading file ~/.cache/lm-studio/config-presets/config.map.json: SyntaxError: Unexpected end of JSON input

Because of the config, the user cannot download any models.

TEMP FIX: the user can remove the % character at the end of json file, after running the AppImage.

Please fix this, so that new users are not hindered from downloading models :)

Hello,

I just upgraded to 0.2.18 and now my GPU Offload settings are not working. My Apple Silicon MacOS system does not offload to the GPUs, instead my CPUs are maxed out?

thanks,
James

Hi,

Firstly, I want to express my gratitude for creating such an exceptional product like LM Studio. It has been a few weeks since I started using it, and I’m thoroughly impressed with its capabilities.

I would like to suggest some enhancements for the chat section that could significantly improve the user experience. Here are my thoughts:

1. AI Text Generation Drafts: Similar to the features available in ChatGPT and Gemini, it would be highly beneficial to have the ability to generate multiple drafts and select the preferred one. This would eliminate the need to copy text to an external editor like MS Word, which is quite a cumbersome process.

2. Grammar and Spelling Checker: While the software currently highlights incorrect spelling, it lacks the feature to provide correction suggestions. Incorporating this would streamline the writing process and reduce the need for external grammar checking tools.

3. Color Coding for User and AI Prompts: The ability to color-code prompts would greatly enhance text visibility, especially after long chat sessions. It would also prevent accidental deletion of important text and make it easier to identify unwanted text.
4. Undo for Deleted Generations: An ‘undo’ feature for accidental deletions of AI-generated text would be a lifesaver. It’s frustrating to lose good content with no way to recover it.

5. Integrated Word Editing Feature: Incorporating a word editing feature directly within LM Studio would be a game-changer. It would allow users to draft and edit text without relying on external text editors. As someone who frequently uses LLM for writing stories and reports, I believe a UI tailored for this purpose would set LM Studio apart from other local LLM backends.
The inclusion of a comprehensive text formatting toolkit within LM Studio would greatly enhance the writing and editing experience. Features such as indentation, bullet points, bold fonts, italics, and other basic text editing capabilities would allow for more sophisticated document creation. This would facilitate users in crafting well-structured and visually appealing content directly within the platform.

Additionally, the ability to utilize multiple AI models for specialized tasks would be revolutionary. For instance, having a dedicated Editor Model that can automatically review and refine content generated by another AI in the Writer Role would streamline the content creation process. Ideally, this could be implemented in a separate window, enabling real-time collaboration between the models to produce polished and ready-to-publish text. Or, a model capable of summarization can be used to summarize the chat between the user and another model- the writer, then feed in the summarization directly to the writer as an appended prompt.

This dual-model functionality could potentially transform LM Studio into a powerhouse for content creators, providing an all-in-one solution for generating, editing, and finalizing written material.

I’m confident that these enhancements would not only improve my workflow but also benefit the wider LM Studio user community.

Thank you for your dedication to improving LM Studio.

Best regards,

By default, LMStudio is processing 1 request at a time, queuing incoming request .
In the console, I was not able to locate any configuration that would enable parallel requests processing.
So my question is:
Is there any way to enable parallel request processing in LMStudion when running on CPU?

If such a feature is not available, is there any plan to add it ?

Thanks

Hi,

Thank you for your efforts. I like the app a lot. I switched from Oobabooga webui since I tried LM Studio.

However, It's missing a feature that I think is very important for someone who likes to write stories emails.

There are moments where by mistake I delete prompts, and I have no way to restore them.
Ctrl+Z just undoes the last text I changed, which is also annoying because If make a change to the system prompt earlier,
and I want to undo some changes into the text of the prompt, the system prompts gets undone.

It's truly frustrating, and I urge you to add this feature ASAP.

Also, could you please add the option for spelling correction?

Thank you very much.

airllm, it said open source, and can inference 70B LLM with 4GB single GPU.

llama.cpp has LLaVA support for a while now

With a simple prompt like 'give me an image of a sunrise' it shows:

Here is an image of a sunrise:

However the image link is broken. Does LM Studio really support image generation with codellama or any other models?
There is no document of how to use/configure it.

Can I provide my custom GBNF to LM Studio and receive the response in the desired format? For example, in llama.cpp, this is achieved through the parameter grammar-file grammars/answer.gbnf

Hermes-2-Pro-Llama-3-8B model uses a new role called "tool" similar to the OpenAI API standard to pass tool results back to the model but the current preset config is too restrictive and doesn't allow adding new roles or non-alternating turns such as multiple system prompts.

Currently when i send function results or errors back to the model with "tool" role i get the BadRequestError:

[
   {
      "role": "user",
      "content": "Your task is to Extract key financial data and ratios from NVDA's SEC filings."
    },
    {
      "role": "assistant",
      "content": "<scratchpad>\nWe need to first find the latest 10-Q and 10-K forms for NVDA and extract relevant information from them.\n</scatchpad>\n\n<tool_call>\n{\"arguments\": {\"data\": \"NVDA|what was last quarter's revenue\"}, \"name\": \"search_10q\"}\n</tool_call>\n"
    },
    {
      "role": "tool",
      "content": "<tool_error>\nXML Parse Error: mismatched tag: line 3, column 2\n</tool_error>\n"
    }
  ]

openai.BadRequestError: Error code: 400 - {'error': "'messages' array must only contain objects with a 'role' field that is either 'user', 'assistant', or 'system'. Got 'tool'."}

There's probably a better way to do this like the Jinja prompt templates but here's what I'm suggesting for the time being allowing multiple input prefixes as a list:

{
  "name": "ChatML",
  "inference_params": {
    "input_prefixes": [
        "<|im_end|>\n<|im_start|>user\n",
        "<|im_end|>\n<|im_start|>tool\n",
        "<|im_end|>\n<|im_start|>agent\n"
    ],
    "input_suffix": "<|im_end|>\n<|im_start|>assistant\n",
    "antiprompt": [
      "<|im_start|>",
      "<|im_end|>"
    ],
    "pre_prompt_prefix": "<|im_start|>system\n",
    "pre_prompt_suffix": "",
    "pre_prompt": "Perform the task to the best of your ability."
  }
}

I change oin the json file the thread to 8, but after finish process my prompt, inform 4threads.
The CPU usage of my 3950x on the LLM studio CPU metter never go over 49%

Why cant push more the CPU in order to speed up the process?

Here is faraday did:

and Jan

However I cannot select GPU in LM Studio, and it always use my CPU and memory on board.

It will we great if we can interact with other services that are not running locally, using the LM Studio only as GUI by default (of course, we will loose the capability to load and run locally in a dynamic way all models, but it could be interesting in some cases)

I am looking for the configuration of zephyr-7b-beta.Q5_K_M.gguf

I downloaded the gguf file manually; how can I add it to LM studio? I added it to folder, but i have You have 1 uncategorized model files

is there any possible to support MPI ?

This program creates json files that I don't need, how can I fix it, that is, I created several of my presets, but the default ones are loaded anyway. And I only need my presets, I forbade the program to access the Internet, access to the "config-presets" folder, why does it need these presets, I don't need them, they are superfluous, I already have my personal ones, why can't I use the previous one, and not "default_lm_studio_windows.preset.json" ?
And why would you ask me about changes to the preset in a new chat, it's obvious that I don't want to change it.

I try this https://twitter.com/LMStudioAI/status/1734640355318944190 and nothing happen, can't add file like in video?

Debian Linux 12, LM Studio appimage 15.

File has permissions to run but upon attempting to run:

./LM_Studio-0.2.14-beta-1.AppImage
13:05:36.565 › GPU info: '0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Device 164e (rev c9)'
13:05:36.571 › Got GPU Type: amd
13:05:36.571 › LM Studio: gpu type = AMD
A JavaScript error occurred in the main process
Uncaught Exception:
Error: /usr/local/lib/libclblast.so: cannot open shared object file: No such file or directory
at process.func [as dlopen] (node:electron/js2c/asar_bundle:2:1822)
at Module._extensions..node (node:internal/modules/cjs/loader:1326:18)
at Object.func [as .node] (node:electron/js2c/asar_bundle:2:1822)
at Module.load (node:internal/modules/cjs/loader:1096:32)
at Module._load (node:internal/modules/cjs/loader:937:12)
at f._load (node:electron/js2c/asar_bundle:2:13330)
at Module.require (node:internal/modules/cjs/loader:1120:19)
at require (node:internal/modules/cjs/helpers:103:18)
at 6829 (/tmp/.mount_LM_StuXw6OJ0/resources/app/.webpack/main/index.js:2:1930)
at r (/tmp/.mount_LM_StuXw6OJ0/resources/app/.webpack/main/index.js:8:401969)
libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)

OS: Ubuntu 22.04 jammy
Kernel: x86_64 Linux 6.5.0-21-generic
Uptime: 10m
Packages: 1762
Shell: bash 5.1.16
Resolution: 2560x1080
DE: GNOME 42.0
WM: Mutter
WM Theme: Adwaita
GTK Theme: Yaru [GTK2/3]
Icon Theme: Yaru
Font: Ubuntu 11

node -v
npm -v
v18.19.1
10.2.4

Hello,

How to stop translate in english the answer look :

I attempted to run LM Studio on my Ubuntu 20.04 server. I accessed my server remotely via Windows Remote Desktop and xrdp. After upgrading some runtime libraries, I successfully ran LM Studio on my remote server and connected to it locally through Python (specifically, Jupyter). However, I noticed that once I closed the remote desktop session, LM Studio was also terminated. Is there a way to provide a virtual background running solution?

Are there any plans to do:

• switching interface language
• change the font size in the chat window
• changing the names of chat participants

Hi,
I hope this message finds you well.

I recently faced this persistent problem where each time I try to run a Command-R based model, I get this error message and the models just doesn't load. I try loading the model on the Oobabooga webui, and it loads just fine,

The complete error message is as follows:

"llama.cpp error: 'error loading model vocabulary: unknown pre-tokenizer type: 'command-r''"
Diagnostics info
{
"memory": {
"ram_capacity": "31.75 GB",
"ram_unused": "22.74 GB"
},
"gpu": {
"type": "NvidiaCuda",
"vram_recommended_capacity": "24.00 GB",
"vram_unused": "22.76 GB"
},
"os": {
"platform": "win32",
"version": "10.0.22631",
"supports_avx2": true
},
"app": {
"version": "0.2.22",
"downloadsDir": "D:\LM Studio\models"
},
"model": {}
}

Model I tried: https://huggingface.co/bartowski/35b-beta-long-GGUF/blob/main/35b-beta-long-Q4_K_M.gguf
https://huggingface.co/MarsupialAI/Coomand-R-35B-v1_iMatrix_GGUF/blob/main/Coomand-R-35B-v1_iQ3m.gguf
https://huggingface.co/TheDrummer/Coomand-R-35B-v1-GGUF/blob/main/Coomand-R-35B-v1-Q3_K_M.gguf

Each time I try to load these models, I get the same error.

Could you please shed some light on the issue and provide a fix?

Thank you in advance :)

I am running the latest LM Studio from the website and I realized that GPU Offload is Greyed Out.

nvidia-smi showing the following:

Sun Jan 21 17:21:12 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 546.33 Driver Version: 546.33 CUDA Version: 12.3 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+===========+|
| 0 NVIDIA GeForce RTX 4090 WDDM | 00000000:01:00.0 On | Off |
| 0% 42C P8 29W / 450W | 2613MiB / 24564MiB | 10% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

Am I missing anything here?

I'm behind a vpn and I can't download any LLama model via LM and the network times out.
But if I paste the downloaded URL into the browser, it can be downloaded.
The LM is apparently requesting the network directly, without using the system network proxy setup
I use windows 11.