EdgeTAM: segment what you click, on less hardware

EdgeTAM segments the thing you point at, as SAM 2 does, from a model built for phones. It takes the same prompts (clicks, a box, or both) and returns the same result. Its image path has 9.1M parameters against SAM 2 tiny's 31.4M, and its masks agree with SAM 2 tiny's at 0.94 IoU on box prompts. Use it when the encode time matters more than the last bit of mask quality.

Quick start

A click on the laptop in the example photograph:

Python
import mozo
from mozo.image import load_image

image = load_image("example.jpg")
model = mozo.get_model("edgetam/edgetam")

found = model.predict(image, points=[[1020, 960]], labels=[1])

for candidate in found:
    print(round(candidate.confidence, 3), candidate.bbox)
Text only
0.977 [746.0, 781.0, 1349.0, 1130.0]
0.437 [512.0, 781.0, 1649.0, 1281.0]
0.113 [756.0, 868.0, 1078.0, 1115.0]
Python
found = model.predict(image, boxes=[685, 783, 1355, 1136], multimask_output=False, name="laptop")
found[0].class_name, round(found[0].confidence, 3)     # ('laptop', 0.985)

Over HTTP, with mozo start running:

Bash
curl -X POST "http://localhost:8000/predict/edgetam/edgetam?point=1020,960&label=1" -F "file=@example.jpg"

Variants

ID Weights (torch) Runtimes Licence
edgetam/edgetam 56 MB torch Apache-2.0

EdgeTAM is published as one model, so there is one variant.

Inputs and outputs

Python
model.predict(image, points=None, labels=None, boxes=None, multimask_output=True, name=None)
model.prepare(image)

The parameters, the result and the HTTP query parameters (point, label, box, multimask, name, sha256) are the same as SAM 2's: see SAM 2's inputs and outputs. In short:

How it behaves

EdgeTAM behaves as SAM 2 does, and SAM 2's page explains each point in full:

Cost

On CPU at 2 megapixels, EdgeTAM encodes an image in 272 ms and decodes a prompt in 33 ms, against 439 ms for SAM 2 tiny's encode. On the example photograph on an Apple M1 Max (mps), the first click took 0.26 s and the box after it 0.05 s.

Weights and licence

Code and weights are Apache-2.0. Mozo's code is an extraction of facebookresearch/EdgeTAM, with its image trunk from huggingface/pytorch-image-models; it is a separate extraction from SAM 2's, and mozo/vendors/edgetam_deploy/PROVENANCE.md records both upstream commits. The variant ships with its licence and a NOTICE naming the exact upstream release.