Netra Logo
Pricing About

Learn / How to use Netra Runtime with CrewAI

How to use Netra Runtime with CrewAI

CrewAI is a Python framework for multi-agent systems: you define agents with roles and tasks, and they collaborate to produce a result. Under the hood every agent call goes through LiteLLM, which speaks to any OpenAI-compatible endpoint. That makes Netra Runtime a drop-in backend for a crew, and since multi-agent runs multiply the number of LLM calls per task, serving speed shows up directly in how long a crew takes to finish.

What you need

  • CrewAI installed: pip install crewai (see CrewAI's LLM configuration docs).
  • A Netra Runtime API key from your dashboard at app.netraruntime.com. The API base URL is https://api.netraruntime.com/v1 (self-hosted deployments use their own endpoint URL instead).

Step 1: define the LLM

Create an LLM with the openai/ prefix on the model id. The prefix tells LiteLLM to use the OpenAI chat-completions protocol against your base URL:

from crewai import LLM

netra_llm = LLM(
    model="openai/qwen3.6-35b",
    base_url="https://api.netraruntime.com/v1",
    api_key="YOUR_NETRA_API_KEY",
)

Step 2: assign it to agents

Pass the LLM to each agent, or to as many as should run on Netra:

from crewai import Agent, Crew, Task

researcher = Agent(
    role="Researcher",
    goal="Gather the facts needed to answer the question",
    backstory="A careful analyst who cites sources.",
    llm=netra_llm,
)

writer = Agent(
    role="Writer",
    goal="Turn research into a clear answer",
    backstory="A concise technical writer.",
    llm=netra_llm,
)

task = Task(description="Explain what continuous batching is.", expected_output="Three clear paragraphs.", agent=writer)
crew = Crew(agents=[researcher, writer], tasks=[task])
print(crew.kickoff())

Agents can also use different models: point a heavyweight deployment at the research agent and a smaller, cheaper one at formatting work. That split is one of the practical levers in reducing inference cost.

Alternative: environment variables

To configure the connection without touching code, set the standard OpenAI variables before starting your crew:

export OPENAI_API_BASE="https://api.netraruntime.com/v1"
export OPENAI_API_KEY="your-netra-api-key"
export OPENAI_MODEL_NAME="openai/qwen3.6-35b"

Frequently asked questions

Why does the CrewAI model name need the openai/ prefix?

CrewAI routes requests through LiteLLM, which uses the prefix to pick the wire protocol. openai/ tells it to speak the OpenAI chat-completions format to your base URL, which is what Netra Runtime exposes.

Can different CrewAI agents use different models?

Yes. Each Agent takes its own llm argument, so a heavyweight Netra-served model can do research while a smaller, cheaper deployment handles formatting or summarization.

Do CrewAI tools work with a Netra-served model?

Yes, if the deployed model supports tool calling. CrewAI sends tool schemas in the standard OpenAI format, which Netra's API passes through to the model.

Related