The task: a mask for every pixel

Removing a background means producing a mask: a greyscale map the same size as the photo, where white means "keep", black means "remove" and grey means "partly transparent". The mask is then applied as an alpha channel, and the result is saved in a format that supports transparency, usually PNG.

Doing this by hand means tracing edges with a pen tool. AI tools generate the mask automatically.

How the model decides

Background removal uses image segmentation models, a kind of neural network trained on many thousands of images in which people have carefully marked the foreground. During training, the network learns patterns that tend to belong to a main subject, such as people, animals and products, and patterns that usually belong to backgrounds.

When you run it on a new photo, the model:

  1. Scales the image to the size it was trained on.
  2. Predicts, for each position, how likely it is to be foreground.
  3. Produces a soft mask with smooth transitions at edges.
  4. Scales the mask back to the original size and applies it.

The model never "understands" the photo the way a person does; it recognises visual patterns. That is why results are excellent for typical portraits and products and less reliable for unusual scenes.

Why some edges are difficult

  • Hair and fur: fine strands mix subject and background colours in the same pixels. A perfect result needs partial transparency, which is hard to predict.
  • Glass, smoke and veils: semi-transparent objects should let the new background show through, but the model only sees the old background behind them.
  • Low contrast: a white product on a white table, or dark clothing against a dark wall, gives few clues about where the edge is.
  • Several subjects: the model may keep or drop a second person, a pet or a held object unpredictably.
  • Shadows: a natural shadow may be removed, making the subject look as if it is floating.

Our tutorial on clean cutouts with background removal covers practical ways to get better results.

Running the model in your browser

Many background removers upload your photo to a server. The Convertios Remove Background tool takes a different approach: it uses the open-source IMG.LY background-removal library (AGPL-3.0) inside your browser. On first use, the browser downloads and caches about 40 MB of model and runtime files. After that, your image is processed locally and never uploaded; you download a transparent PNG.

The trade-offs of running locally:

  • Privacy: the photo stays on your device.
  • First-use download: the model must be fetched once, which takes time on slow connections.
  • Device performance: older phones take longer than laptops.

Our explainer on how in-browser tools work describes the technologies, such as WebAssembly and GPU acceleration, that make this possible.

After removing the background

  • Keep the original photo; you may want to retry with a cleaner shot.
  • The PNG keeps transparency. If a website needs a smaller file, resize it with the Image Resizer, choosing PNG or WebP to keep transparency, or compress it with the image compressor as PNG or WebP. Converting to JPG fills transparent areas with a solid colour.