Moebius: remove an object from a photo
Moebius removes things from photographs. Give it an image and a mask, and it repaints the masked
region so the thing was never there. It answers with a picture: there is no Detections, no class
and no confidence number. Pair it with a model that finds what to remove, such as
SAM 3 for a phrase or EdgeTAM for a click.
Quick start
Remove every cup from the example photograph:
import mozo
from mozo.image import load_image
image = load_image("example.jpg")
cups = mozo.get_model("sam3/sam3").predict(image, "a cup")
clean = mozo.get_model("moebius/general").predict(image, cups, seed=0, dilate=8)
clean.shape == image.shape # True: same size, and outside the mask, the same bytesOver HTTP the selection is a rectangle, and the answer is a PNG:
curl -X POST "http://localhost:8000/predict/moebius/general?box=632,589,706,694&seed=0" \
-F "file=@example.jpg" -o clean.pngRepeat box to remove several things. For a mask that follows an object's outline rather than a
rectangle, run a segmenter and use the Python API.
Variants
| ID | Weights (torch) | Runtimes | Licence |
|---|---|---|---|
moebius/general |
1,073 MB | torch | Apache-2.0 |
moebius/places2 |
1,073 MB | torch | Apache-2.0 |
general is for arbitrary photographs, places2 for scenes and backgrounds. Upstream also
publishes two face-specific checkpoints; mozo does not carry them. The weights are a UNet and an
autoencoder, downloaded together. general also publishes ONNX graphs, which mozo does not run.
Inputs and outputs
model.predict(image, mask, *, seed=0, steps=20, guidance=2.0, dilate=0, feather=3)| Parameter | Type | Default | Description |
|---|---|---|---|
image |
str or numpy.ndarray |
A file path or an HxWx3 RGB array. |
|
mask |
numpy.ndarray or Detections |
A boolean HxW array, or detections: each one's mask, else its polygon, else its box. |
|
seed |
int |
0 |
Which sample to draw. The same seed gives the same picture. |
steps |
int |
20 |
Denoising steps requested. Nineteen of twenty run, as upstream does. |
guidance |
float |
2.0 |
Classifier-free guidance scale, upstream's own value. |
dilate |
int |
0 |
Grow the mask by this many pixels first. |
feather |
int |
3 |
Blur radius on the mask edge before compositing. 0 for a hard edge. |
All the regions are removed in one pass over their union. A detection with only a box removes the whole rectangle, so prefer a segmenter's masks.
Returns an HxWx3 uint8 RGB array, the same size as image. An empty mask returns the image
unchanged without running the model.
Over HTTP: box (repeatable, x1,y1,x2,y2) is required, and seed is optional. The response is an
8-bit PNG, lossless so two seeds can be compared honestly.
How it behaves
It draws a sample, not an estimate
Every other model in mozo describes an image. This one rewrites one, and the model does not
estimate anything: it draws a sample. Run it again with seed=1 and you get a different, equally
valid removal; neither is more correct. That is why seed is an argument and not a hidden
constant.
Everything the seam does not reach comes back byte-identical
Upstream returns the decoder's reconstruction of the whole frame, which changes every pixel, including the ones nobody selected. Mozo composites instead: the hole is the model's, and everything else is your own array, unchanged. That is what makes it safe to point at a photograph you care about.
At the border, the seam is feathered with a 3-pixel blur so it reads as a gradient rather than a
cut-out, and the blend spreads a few pixels past your selection. On the example photograph, with
a cup's box as the mask, no pixel more than 7 px outside the box changed. Pass feather=0 and
nothing outside the mask changes at all, at the cost of a visible cut-out.
If the thing is still faintly there, raise dilate
An object's shadow and its antialiased edge sit outside the mask that found it, and removing the object without them leaves an outline. This is the most common surprise, and 8 to 16 pixels usually settles it.
The same setting fixes the other cause. On a mask only a few pixels across, the feather pulls the blend below full strength even at the centre, so a little of the original survives everywhere. Growing the mask fixes that; shrinking the feather only makes the seam obvious.
It runs at 512×512 and nothing else
This is a property of the weights, not a setting: the cross-attention's position table has a row
per latent cell, so there is nowhere to put the positions of a larger image. predict resizes for
you and composites back at your resolution, which means detail inside the hole is capped at 512.
On a large photograph that is visible if you look for it.
Cost
On the example photograph (1920×1281) on an Apple M1 Max (mps), removing six cups took 18.3 s
with the default 20 steps.
Weights and licence
Code and weights are Apache-2.0, from hustvl/Moebius. The authors state the weights' licence as
Apache-2.0 on GitHub and MIT on the Hugging Face model card; the NOTICE published with each variant
records both. The autoencoder is hustvl/PixelHacker's, MIT, a separate work published beside it
with its own NOTICE. mozo/vendors/moebius_deploy/PROVENANCE.md records the upstream source and
why the face checkpoints are left out.