saddam213 / llamastack Goto Github PK

I took a screen capture and it shows how the instruct never finishes - it just keeps going on and on.

To recreate:

1. clone the repo
2. Build the project
3. Set the WPF project as startup project
4. Run the application

I tried to run the example but encountered this error in LlamaStackContext.cs. I've installed everything and I'm using the latest version of LlamaSharp.

'IInferenceParams' does not contain a definition for 'MinP' and no accessible extension method 'MinP' accepting a first argument of type 'IInferenceParams' could be found (are you missing a using directive or an assembly reference?) LLamaStack.Core C:\Users\Me\source\repos\LLamaStack-master\LLamaStack.Core\LLamaStackContext.cs 143 Active

        public int Sample(LLamaTokenDataArray tokenDataArray, IInferenceParams inferenceParams, ref float? mirostatMu)
        {
            return _context.Sample
            (
                tokenDataArray,
                ref mirostatMu,
                inferenceParams.Temperature,
                inferenceParams.Mirostat,
                inferenceParams.MirostatTau,
                inferenceParams.MirostatEta,
                inferenceParams.TopK,
                inferenceParams.TopP,
                inferenceParams.TfsZ,
                inferenceParams.TypicalP,
                inferenceParams.Grammar,
                inferenceParams.MinP
            );
        }

I also get the following errors when trying to start the WPF example.

The type reference cannot find a public type named 'SessionHistoryModel'. Line 116 Position 90. LLamaStack.WPF C:\Users\Me\source\repos\LLamaStack-master\LLamaStack.WPF\Views\InferenceView.xaml 116

The type reference cannot find a public type named 'ModelSessionState`1'. Line 34 Position 82. LLamaStack.WPF C:\Users\Me\source\repos\LLamaStack-master\LLamaStack.WPF\Views\SessionLoadView.xaml 34

When more than one Inferences queue'd and you call ModelSessionService.SaveAsync() it causes an exception in the Session Guard.

Example:

var result = await _modelSessionService.QueueInferTextAsync(_modelSession.SessionId, message); await _modelSessionService.SaveAsync(_modelSession.SessionId);

implement openai style webapi,so can be used by various openai client
openai doc:
https://platform.openai.com/docs/api-reference/chat

Hello,
I am getting the below issue. Could you please help me fix it?

Here are the details:
Log:

[12/16/2023 3:51:18 PM] [Information] [LLamaStack] - Application started. Press Ctrl+C to shut down.
[12/16/2023 3:51:18 PM] [Information] [LLamaStack] - Hosting environment: Production
[12/16/2023 3:51:18 PM] [Information] [LLamaStack] - Content root path: C:\Users\imran\source\repos\LLamaStack\LLamaStack.WPF\bin\Debug\net8.0-windows
[12/16/2023 3:52:42 PM] [Error] [LLamaStack] - [CreateSession] - Method not found: 'Void LLama.Common.ModelParams.set_ContextSize(UInt32)'.

To recreate:

1. Clone the repo
2. Delete all projects except the WPF
3. Upgrade to .net8
4. Install LLamaSharp, LLamaStack and LLamaSharp.Backend.Cpu
5. Download the llama-2-7b-chat.Q2_K.gguf
6. Run the app and change the model path
7. Click on Begin Session

when upgrade llamasharp 0.7.0,error occurs