Learn / What is image segmentation?
What is image segmentation? Segment Anything (SAM) explained
Image segmentation is the computer-vision task of labeling which pixels belong to each object in an image — a pixel-level mask, not just a bounding box. Promptable models like Meta's Segment Anything (SAM) made this fast and general: you click a point, drag a box, or name an object and get a mask. Netra builds the open-source sam3.c and Limbus to run SAM3 segmentation efficiently and locally.
Segmentation vs classification vs detection
These three vision tasks answer different questions about an image. Classification asks "what is this?", detection asks "where are the objects?", and segmentation asks "which exact pixels are each object?".
| Task | Output | Answers |
|---|---|---|
| Classification | A label for the whole image | What is this? |
| Object detection | Bounding boxes | Where are the objects? |
| Segmentation | Per-pixel masks | Which pixels are each object? |
Segment Anything (SAM)
Segment Anything (SAM), introduced by Kirillov et al. (2023), is a promptable segmentation model trained on SA-1B, a dataset of over 1 billion masks across 11 million images. Because it is promptable and trained at that scale, it generalizes zero-shot: give it a point, a box, or a description and it returns a mask, even for objects and images it never saw in training. SAM3 is the latest generation of this line of models.
Running segmentation locally
Segmentation models are often run in the cloud, but they can also run on-device — which keeps images private and avoids per-image cost. Netra's sam3.c implements the full SAM3 pipeline in pure C with no Python dependencies, using the Metal GPU on Apple Silicon and multithreaded CPU elsewhere. Limbus is a local-first desktop app built on sam3.c for annotating segmentation datasets, where your images and labels never leave your machine.
Try it yourself
Frequently asked questions
What is the difference between image segmentation and object detection?
Object detection draws a bounding box around each object, while image segmentation labels the exact pixels that belong to it as a mask. Segmentation is more precise and is used when object shape and outline matter.
What is Segment Anything (SAM)?
Segment Anything (SAM) is a promptable segmentation model from Meta, introduced by Kirillov et al. in 2023 and trained on the SA-1B dataset of over 1 billion masks. You prompt it with a point, a box, or text and it returns a mask, generalizing zero-shot to new images and objects.
Can I run image segmentation locally?
Yes. Netra's open-source sam3.c runs SAM3 segmentation in pure C on Apple Silicon and x86 CPUs, and Limbus is a local-first desktop app built on it, so images and annotations never leave your machine.
References
- Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollar, P., & Girshick, R. (2023). Segment Anything. arXiv:2304.02643.