SAM2 is remarkably good at understanding what you clicked on. What it hands back is a 256×256 grid of confidence values — and if you turn that into a mask carelessly, every boundary in the picture comes out as a staircase, which is painfully visible the moment the object starts moving.

Most of what follows is about the few pixels either side of that boundary: how they get upscaled, how they get softened, and what to do when SAM2 draws the line somewhere you did not want it.

New to SAM2 in WobblePic? See AI Segmentation with SAM2 for the basics.

The Problem: Staircase Boundaries

SAM2’s decoder outputs a low-resolution logits map — a 256×256 grid of confidence values, regardless of the original image size. To create a usable mask, this small grid needs to be upscaled to match the full image resolution.

The naive approach is nearest-neighbor upscaling: each pixel in the 256×256 grid is simply repeated to fill its corresponding block of pixels in the full-resolution image. This is fast, but it produces visible staircase artifacts along the mask boundary. When you wobble an object segmented this way, the blocky edges are obvious and distracting.

For an image viewer where visual quality matters, this isn’t good enough.

The Solution: Bilinear Logits Interpolation

WobblePic takes a different approach. Instead of upscaling the final binary mask, it upscales the raw logits (continuous confidence values) using bilinear interpolation before applying the threshold.

Here’s why this matters:

  1. Raw logits are continuous values (e.g., -2.3, 0.1, 4.7) that represent how confident SAM2 is that each pixel belongs to the object
  2. Bilinear interpolation smoothly blends between neighboring logit values, creating gradual transitions instead of hard steps
  3. After interpolation, the threshold is applied (typically logit > 0 = object), producing a mask with smooth, sub-pixel-accurate boundaries

The result is a mask boundary that follows the actual contour of the object rather than the 256×256 grid lines. Curved edges look curved. Fine details like hair or fur are preserved more faithfully.

Before (nearest-neighbor):     After (bilinear logits):
┌──┬──┬──┐                     ╭──────╮
│██│██│  │                     │██████│
│██│██│██│  →                  │████████╮
│  │██│██│                     ╰──██████│
└──┴──┴──┘                        ╰────╯
Blocky staircase               Smooth contour

This technique adds minimal computational cost — bilinear interpolation is lightweight compared to the SAM2 inference itself — but the visual improvement is significant.

The mask never becomes binary

Smoothing the contour is only half of it. A mask with a perfectly smooth outline is still a hard yes-or-no along that outline, and a hard edge is exactly what makes a segmented wobble look like a sticker sliding over a photograph.

So after the mask is restored to full resolution, it is blurred with a Gaussian — radius 12 by default — and kept as floating-point values rather than being reduced back to a binary map. The boundary becomes a ramp roughly a dozen pixels wide, and every pixel in that ramp carries a weight between 0 and 1.

That weight is not just for display. It is the same number the deformation uses: each vertex’s mask weight multiplies the pull it receives from a drag. A vertex at 1.0 follows your cursor fully, a vertex at 0.0 never moves, and the ones in the ramp move partially. The wobble does not stop at the object’s edge — it fades out across it, which is what the eye reads as the object being attached to the picture rather than cut out of it.

The same blurred mask drives the visual cue too: the selection is drawn 10% brighter and the background 10% darker, and because the mask is soft, that highlight has no hard rim either.

A useful consequence: a mask that is slightly wrong matters less than you would expect. If SAM2 puts the boundary a few pixels inside the true edge, the feathering spreads the transition back over it, and the result still looks intentional. Precision buys less here than softness does.

Multiple Prompt Types for Better Selection

Accuracy isn’t just about mask boundaries. It’s also about selecting the right object in the first place. WobblePic supports multiple prompt types to give you control:

Point Prompt (Click)

The simplest approach — click on the object you want. SAM2 infers the object boundary from a single point. This works well for clearly defined objects but can be ambiguous when objects overlap or have complex shapes.

Box Prompt (Drag)

For more precise selection, drag a rectangle around the object. The bounding box gives SAM2 much more context about what you intend to select, especially helpful for:

  • Objects close to other objects
  • Elongated or irregularly shaped subjects
  • Scenes where a point click might select a parent object (e.g., clicking a wheel selects the whole car)

Box prompts generally produce more accurate masks on the first try because they constrain the search space.

Interactive Refinement

Even the best AI model doesn’t always get it right on the first attempt. WobblePic lets you iteratively improve the selection:

  • Shift + Click/Drag — Add to the current mask (include missed regions)
  • Alt + Click/Drag — Remove from the current mask (exclude unwanted areas)

It is worth being precise about what happens here, because there are two ways to build this and WobblePic takes the cheaper one.

SAM2 itself supports multi-point prompting: hand the decoder several points at once, labelled positive and negative, and it re-infers a single mask that accounts for all of them. WobblePic does not do that. Each refinement click is decoded on its own as a fresh single prompt, and the resulting mask is combined with the previous one in image space:

if mode == "expand":     mask = np.maximum(prev, new)               # union
elif mode == "subtract": mask = clip(prev - new, 0, 255)            # difference

The trade-off is real and worth knowing when you are refining. Because the model never sees your correction, it cannot redraw a boundary it got wrong — subtracting removes an area you painted out, but it will not persuade SAM2 that the shadow under the cake was never part of the cake. What you get instead is predictability: every refinement costs exactly one ~50 ms decode, the previous mask is never re-litigated by the model, and nothing you have already accepted can shift underneath you.

In practice that is the right trade for adding a missed ear or trimming a stray corner, which is most refinement. When the very first mask is badly wrong, a box prompt is the better move than a sequence of corrections.

flowchart LR
    P["Point<br/>(1 click)"] --> S1[SAM2]
    B["Box<br/>(drag)"] --> S2[SAM2]
    R["Refinement<br/>(shift / alt)"] --> S3[SAM2]
    S1 --> M1["Single mask<br/>fastest, may be ambiguous"]
    S2 --> M2["Constrained mask<br/>more accurate first try"]
    S3 --> M3["Iterative mask<br/>most controllable"]

Settings Panel: Selection Range

WobblePic’s Settings Panel includes a Selection Range slider that controls the mask threshold. This directly affects segmentation accuracy:

  • Lower values — More conservative selection, only including high-confidence pixels. Good for hard-edged objects like electronics or geometric shapes.
  • Higher values — More aggressive selection, capturing more of the object including uncertain boundary pixels. Good for soft-edged objects like fluffy animals or food with irregular surfaces.

By adjusting this slider, you can fine-tune the mask to match the specific characteristics of each image.

What actually carries the quality

Four things do the work, and they are not equally weighted:

  1. Upscaling the logits before thresholding, rather than thresholding first — this is what removes the staircase, and it costs almost nothing
  2. Feathering the mask and keeping it continuous, which softens the transition and quietly forgives a boundary that sits a few pixels off
  3. Box prompts when a point prompt keeps picking the wrong scope
  4. The Selection Range threshold, for the images where the default sits in the wrong place

The first two are the interesting ones, and they point the same way: on a moving image, how the boundary transitions matters more than exactly where it falls. A pixel-perfect contour with a hard edge looks worse than an approximate contour with a soft one — which is a good thing, because the second is much cheaper to get.