Learn

Netra Runtime is an AI infrastructure platform for high-performance inference — run, train, and deploy frontier AI models you own, through an OpenAI-compatible API. These guides explain how LLM inference works, the metrics that matter, and how Netra performs in practice.

Start here

What is Netra Runtime?

Category, capabilities, pricing, and benchmarked performance in one place.

What is LLM inference?

How language models generate output, and why the serving engine matters.

Core concepts

Inference latency & throughput

TTFT, time-per-output-token, end-to-end latency, throughput, and concurrency explained.

OpenAI-compatible inference API

What it is, why it matters, and which endpoints are supported.

Continuous batching

The serving technique that keeps GPUs busy and raises throughput.

KV cache

What it stores, the memory trade-off, and why it matters for long context.

Model quantization

Lower-precision weights for smaller, faster, cheaper inference.

Token streaming

Sending output token by token so responses feel instant.

Knowledge distillation

Transferring a large model's knowledge into a smaller, faster one.

Paged attention

Virtual memory for the KV cache, and why every modern engine uses it.

Prompt caching

Reuse a repeated prompt prefix to cut cost and time-to-first-token.

Speculative decoding

Draft with a small model, verify with the large one, same output.

Sovereign AI

Control over data, model, compute, and governance, at company scale.

GPU spot instances

Spot vs on-demand, preemption, and when cheap interruptible GPUs pay off.

Guides

Deploy a fine-tuned LLM

Steps and trade-offs for serving your customized model in production.

Reduce LLM inference cost

Practical levers to produce more tokens per GPU-second.

Serve small models in production

When small fine-tuned models win, and how to deploy them.

LLM inference for AI agents

Why agentic, long-context workloads stress inference, and how Netra serves them.

Integrations

Any OpenAI-compatible tool

The three-values pattern behind every integration, plus a curl test.

OpenClaw

Make a Netra-served model the brain of your personal AI assistant.

n8n

Power AI Agent and chat model nodes with one credential change.

Dify

Hermes Agent

Point Nous Research's agent at your endpoint with one wizard run.

LangChain

Two ChatOpenAI arguments in Python or JavaScript, nothing else.

Open WebUI

A private ChatGPT-style UI on top of a model you own.

LlamaIndex

RAG over your documents with OpenAILike and your endpoint.

Vercel AI SDK

generateText, streamText, and useChat on your own endpoint.

CrewAI

Run multi-agent crews on a model served by Netra.

Comparisons

Netra Runtime vs vLLM

An honest comparison anchored on a published long-context benchmark.

Hosted vs self-hosted inference

Control, cost, latency, and ops — and when each makes sense.

Tools & calculators

LLM memory requirements

How much VRAM an LLM needs: weights, KV cache, and quantization.

Fine-tuning memory

GPU memory to fine-tune: weights, gradients, optimizer, and activations.

LoRA & QLoRA

Fine-tune with small adapters, and a 4-bit base, to save memory.

Tokenization

What a token is, and why counts differ across GPT, Claude, and Llama.

OCR

How optical character recognition and AI OCR turn images into text.

Image segmentation

Pixel-level masks, Segment Anything, and running it locally.

Looking for the tools themselves? Browse all free tools.