sam3.c

Highlights

• Pure C11 with zero Python/PyTorch dependencies.
• Three backbones: Hiera (full accuracy), EfficientViT-B1 (fastest), TinyViT-21M (balanced).
• Apple Metal GPU acceleration plus multithreaded, SIMD-optimized CPU kernels.
• FP32, FP16, BF16, and quantized int8 precisions.
• Video object tracking with memory-based frame propagation.
• A CLI (sam3_cli) for segmentation, conversion, and inspection, plus Python and Rust bindings.

Performance

On an Apple M4 with the Metal backend, EfficientViT segments an image end-to-end in roughly 250 ms (~4 FPS), and the Metal backend reaches about 90.8% of theoretical F32 peak. Hiera-Large trades speed for maximum accuracy at about 3.6 s per image. Inspired by ggml and llama.cpp.

How it works

SAM3 is a promptable segmentation model: you give it an image plus a prompt (a point, a box, or a text phrase like "zebra") and it returns a precise mask for the object you meant. The model has three parts, and sam3.c implements all three from scratch in C. The image encoder (the backbone) turns the image into a dense feature map; this is the expensive step, so its output is cached and reused across prompts on the same image. The prompt encoder turns your point, box, or text into embeddings. The mask decoder combines the two and predicts the mask. Because the heavy image features are computed once, follow-up prompts on the same picture are near-instant.

sam3.c ships three interchangeable backbones so you can trade accuracy for speed: Hiera for full accuracy, TinyViT-21M for a balance, and EfficientViT-B1 for maximum speed. Weights are converted ahead of time into a compact .sam3 file and loaded by memory-mapping, so start-up is instant and the process only pages in what it touches. Inference runs on Apple's Metal GPU where available and falls back to multithreaded, SIMD-optimized CPU kernels everywhere else, with FP32, FP16, BF16, and int8 precisions to fit different hardware.

Why pure C

Most model code depends on Python and PyTorch, which are heavy to install, slow to start, and awkward to embed in another application. Writing the whole pipeline in portable C11 removes that entirely: a single small binary with no runtime dependencies that starts instantly, embeds anywhere, and can be called from Python or Rust through thin bindings. It is the same philosophy as llama.cpp and ggml, applied to segmentation: the model becomes something you can drop into a desktop app or a build pipeline without shipping a Python environment alongside it.

Learn more

New to segmentation? Read what image segmentation is and how Segment Anything works.