Netra Logo
Pricing About

Learn / How to use Netra Runtime with LangChain

How to use Netra Runtime with LangChain

LangChain is the most widely used framework for building LLM applications: chains, agents, and RAG pipelines in Python or JavaScript. Its OpenAI integration accepts a custom base URL, so a model served by Netra Runtime drops into any existing LangChain app with two constructor arguments. No new integration package, no code changes elsewhere.

What you need

  • The OpenAI integration package: pip install langchain-openai (Python docs) or npm install @langchain/openai (JavaScript docs).
  • A Netra Runtime API key from your dashboard at app.netraruntime.com. The API base URL is https://api.netraruntime.com/v1 (self-hosted deployments use their own endpoint URL instead).

Python

Pass base_url and api_key to ChatOpenAI and use the model like any other:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="qwen3.6-35b",
    api_key="YOUR_NETRA_API_KEY",
    base_url="https://api.netraruntime.com/v1",
)

print(llm.invoke("Say hello from Netra Runtime.").content)

Streaming works the same way as with any OpenAI backend, and matters for chat UIs because users see tokens as they are generated:

for chunk in llm.stream("Explain continuous batching in two sentences."):
    print(chunk.content, end="", flush=True)

JavaScript / TypeScript

In @langchain/openai, the base URL goes in the configuration object:

import { ChatOpenAI } from "@langchain/openai"

const llm = new ChatOpenAI({
    model: "qwen3.6-35b",
    apiKey: process.env.NETRA_API_KEY,
    configuration: { baseURL: "https://api.netraruntime.com/v1" },
})

const response = await llm.invoke("Say hello from Netra Runtime.")
console.log(response.content)

Agents, RAG, and tools

Everything downstream of the model object works unchanged. The same llm instance plugs into LCEL chains, LangGraph agents built with create_react_agent, and retrieval pipelines. If your deployed model supports tool calling, llm.bind_tools(...) sends tool schemas in the standard OpenAI format and Netra passes them through to the model. For embeddings in a RAG pipeline, configure the embeddings class with the same base URL and key, provided your deployment serves an embeddings endpoint.

Configuration by environment variable

To keep credentials out of code, set the standard variables and construct ChatOpenAI with only the model name:

export OPENAI_API_KEY="your-netra-api-key"
export OPENAI_BASE_URL="https://api.netraruntime.com/v1"

This is also the cleanest way to switch an existing LangChain codebase to Netra: change the two variables in your deployment environment and every ChatOpenAI in the app follows.

Frequently asked questions

Do I need a special LangChain integration package for Netra Runtime?

No. Netra Runtime exposes an OpenAI-compatible API, so the standard langchain-openai package (or @langchain/openai in JavaScript) works by setting base_url and api_key on ChatOpenAI.

Does tool calling work through LangChain on Netra Runtime?

Yes, if the model you deployed supports tool calling. bind_tools and LangGraph agents send tools in the standard OpenAI format, which Netra's API passes to the model.

Can I set the Netra connection through environment variables?

Yes. ChatOpenAI reads OPENAI_API_KEY and OPENAI_BASE_URL from the environment, so you can configure the connection without touching code.

Related