Stable Diffusion Prompt Inpainting

Prompt /

Stable Diffusion is an AI image generation model that can create images from text prompts. One powerful feature is inpainting, which allows filling in missing parts of an image by providing a text description.

Inpainting works by taking an existing image with holes or missing sections, then generating the missing pixels conditioned on the available context. This allows restoring damaged photographs, removing unwanted objects, or even imagining whole new scenes.

Crafting effective inpainting prompts requires understanding Stable Diffusion’s capabilities, creatively describing the desired changes, and tweaking prompts to get better results. This article provides an overview of best practices with examples.

Describing What to Add

The key to inpainting is using text to precisely depict the missing regions you want filled in. For example:

A photo of a bride in a wedding dress. Inpaint the groom standing beside her, wearing a black tuxedo and red bowtie.

Focus on visual details – clothing, poses, facial expressions. Use vivid language to set the scene. Refer to existing parts of the image for consistency.

Inpaint a Person

To add a person, describe their:

  • Gender, age range
  • Ethnicity, skin tone
  • Height, build
  • Hair style and color
  • Facial features
  • Clothing, accessories
  • Pose and expression

Inpaint an Object

When adding objects, identify the:

  • Category – vehicle, furniture, electronics etc.
  • Size, orientation perspective
  • Colors, materials
  • Distinguishing characteristics
  • Positioning relative to other elements

Describing What to Remove

Inpainting can also erase unwanted parts of an image. For example:

A photo of Times Square. Inpaint remove all cars, buses and people from the street.

This works best for removing whole objects. Referencing the category tends to be more reliable than describing specific unwanted items.

Prompt Structure Tips

  • Start with a brief factual description of the existing image
  • Use “Inpaint” to signal the beginning of changes
  • Keep prompts under 300 characters for best performance
  • List multiple changes as comma separated items
  • End with a period after the last change

Advanced Prompt Design

As you gain experience with inpainting, try these additional techniques:

Use Multiple Inpaint Steps

Break down large changes into separate layers:

A photo of a groom alone. 
Inpaint add a bride beside him, in a white wedding dress looking happy.
Inpaint they are standing under a floral wedding arch.
Inpaint add guests sitting in white folding chairs facing them.

This allows more control compared to describing everything at once.

Guide Attention

Call attention to critical regions you want to change:

A photo of a beach. Inpaint focus on the sky and add colorful hot air balloons flying through clouds.

Prioritizing the most relevant areas tends to produce better quality.

Blend Edges Naturally

Help seamlessly integrate additions using spatial references:

A photo of an empty field. Inpaint add a large oak tree on the left side casting a shadow across the grass.

Indicating surroundings assists blending inpainted elements.

Refine With Multiple Tries

Iteratively improve results by tweaking and rerunning prompts:

Try 1: Inpaint add a pet cat sitting on the couch. 
Try 2: Inpaint add an orange pet cat sitting on the right side of the couch.

Making small targeted adjustments often yields enhancements.

Useful Stable Diffusion Inpainting Websites

  • Lexica – Web UI for inpainting, editing and creating images
  • Replicate – Notebook for tweaking model parameters
  • Automatic1111 Webui – Local UI for advanced users
  • Nightcafe – Online creation with image editing options

Related Posts