Comments (6)
Hi,
We need to update our SDK and examples to explain how to make it work with Next.js.
In the meantine, here is an example that works well with Next.js App router and Lunary:
lunary.init({ appId: "..." });
const openai = new OpenAI({
apiKey: "sk-...",
});
monitorOpenAI(openai);
export const runtime = "edge";
export async function GET() {
const result = await openai.chat.completions.create({
model: "gpt-3.5-turbo",
temperature: 0.9,
stream: true,
messages: [
{ role: "system", content: "You are an helpful assistant" },
{ role: "user", content: "Print a random string" },
],
});
const stream = iteratorToStream(result);
return new Response(stream);
}
function iteratorToStream(iterator: any) {
const encoder = new TextEncoder();
return new ReadableStream({
async pull(controller) {
try {
const { value, done } = await iterator.next();
if (done) return controller.close();
const bytes = encoder.encode(JSON.stringify(value) + "\n");
controller.enqueue(bytes);
} catch (error) {
controller.error(error);
}
},
});
}
from lunary.
This is still an issue:
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY
})
const res = await openai.chat.completions.create({
model: 'gpt-4-0125-preview',
user: userId,
messages,
temperature: 0.7,
stream: true
})
res
is of type Stream<OpenAI.Chat.Completions.ChatCompletionChunk>
const openai = monitorOpenAI(
new OpenAI({
apiKey: process.env.OPENAI_API_KEY
})
)
const res = await openai.chat.completions.create({
model: 'gpt-4-0125-preview',
user: userId,
messages,
temperature: 0.7,
stream: true
})
res
is of type OpenAI.Chat.Completions.ChatCompletion
from lunary.
As a workaround:
It seems to work in spite of the type error. Adding @ts-ignore
allows the build stage to pass and then the code operates as expected. e.g.
// @ts-ignore
const stream = OpenAIStream(res, {
async onCompletion(completion) {
It looks like there are some pretty gnarly type-gymnastics being done inside the openai
npm module.
from lunary.
LLM-769 Server-side error with vercel 'ai' npm package
from lunary.
Can I contribute to this?
from lunary.
Hi @BigStar-2024, yes you can
from lunary.
Related Issues (20)
- LunaryModule not found - Langchain HOT 6
- Streaming Mode with openai package cause LLMonitor to throw error HOT 1
- Sorry to be that guy... *types*? HOT 3
- Should't this be DATABASE_URL? HOT 3
- Unable to create local deployment HOT 2
- relation "tag_cache" does not exist - relation "model_name_cache" does not exist HOT 3
- getting started to run locally - errors with npm run dev HOT 12
- 404 on Ops Submodule HOT 1
- instance of BaseCallbackHandler expected HOT 1
- Confused about the Self-Hosted Community tier HOT 1
- Self hosting errors HOT 1
- Errors while self hosting HOT 3
- Source of benchmark page HOT 1
- Feature: Adding tags via langchain calls HOT 5
- Self hosted: High db usage while idle HOT 4
- Best way to trace cached LLM and tool calls HOT 21
- Tags not displaying in tools traces HOT 4
- frontend env variable HOT 2
- Deleted
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lunary.