How to use Netra Runtime with the Vercel AI SDK

What you need

• The packages: npm install ai @ai-sdk/openai-compatible (see the OpenAI-compatible provider docs).
• A Netra Runtime API key from your dashboard at app.netraruntime.com. The API base URL is https://api.netraruntime.com/v1 (self-hosted deployments use their own endpoint URL instead).

Step 1: create the provider

Create a provider once and reuse it everywhere in your app:

import { createOpenAICompatible } from "@ai-sdk/openai-compatible"

export const netra = createOpenAICompatible({
    name: "netra",
    baseURL: "https://api.netraruntime.com/v1",
    apiKey: process.env.NETRA_API_KEY,
})

Step 2: generate and stream

Pass a model instance to the AI SDK functions as usual:

import { generateText, streamText } from "ai"
import { netra } from "./netra"

const { text } = await generateText({
    model: netra("qwen3.6-35b"),
    prompt: "Say hello from Netra Runtime.",
})

const result = streamText({
    model: netra("qwen3.6-35b"),
    prompt: "Explain continuous batching in two sentences.",
})
for await (const chunk of result.textStream) {
    process.stdout.write(chunk)
}

In a Next.js route handler, return result.toUIMessageStreamResponse() and useChat on the client renders the stream token by token. Token streaming is where fast inference shows: time-to-first-token is what the user feels on every message.

Notes

• Tool calling and structured output. tools and generateObject work if your deployed model supports tool calling and JSON output; the SDK sends both in standard OpenAI format.
• Keep the key server-side. Create the provider only in server code (route handlers, server actions) so NETRA_API_KEY never reaches the browser.
• The @ai-sdk/openai package also works. createOpenAI({ baseURL, apiKey }) targets the same endpoints, but the openai-compatible package makes no OpenAI-specific assumptions, so prefer it for custom backends.

Frequently asked questions

Related

• Use Netra Runtime with any OpenAI-compatible tool
• How to use Netra Runtime with LangChain
• What is token streaming?
• Inference latency & throughput