Tricks for Optimizing Stable Diffusion Prompts

Stable Diffusion is an incredible AI image generation tool, but getting good results requires carefully crafting prompts. As someone who has spent countless hours experimenting, let me share what I’ve learned about optimizing prompts.

Understand How Stable Diffusion Works

To write effective prompts, you first need to understand what goes on under the hood. Stable Diffusion is trained on image and text pairs to generate new images based on text descriptions. The key things to know:

  • It focuses heavily on the last part of prompts
  • It struggles with ambiguity – be as specific as possible
  • It works best with short, clear prompt structures

Keep these core principles in mind as you engineer better prompts.

Structure Your Prompts

With Stable Diffusion, structure matters. Well-formatted prompts lead to better results.

I recommend this basic structure:

  1. Short description of image content
  2. List of styles/qualities
  3. List of modifiers to tweak the image

For example:

A still life painting of fruit in a bowl, textured oil painting, soft lighting, depth of field

Breaking down your prompt makes it easier for Stable Diffusion to generate what you want.

Use the Right Descriptors

Choosing the right words is critical. Rely on these prompt writing tips:

  • Avoid ambiguity – be extremely specific
  • Use adjectives like “extremely” and “very”
  • Quantify details when possible – e.g. size, number of objects
  • List multiple related descriptors

Here’s an example with more details:

An extremely detailed oil painting of 5 brightly colored pieces of fruit realistically rendered in a small white ceramic bowl, soft natural lighting, depth of field blur

Apply Styles and Modifiers

Leverage styles and modifiers to refine your image:

  • Styles – oil painting, watercolor, pixel art
  • Modifiers – lighting, depth of field, blur, crop, angle

Styles guide the overall look and feel while modifiers fine-tune elements like lighting and perspective.

Use Negative Prompts

Negative prompts specify what you don’t want to avoid unwanted elements. For example:

Futuristic cityscape, HD detailed, cinematic lighting, volumetric rays, by Asher Brown Durand, matte painting, highly detailed, octane render, Unreal engine, --logo, --watermark, --signature  

The dashes exclude watermarks and signatures.

Optimize Prompt Length

I’ve found prompt lengths between 70-150 words work best. Anything shorter lacks needed details. Anything longer tends to confuse Stable Diffusion.

70 words is fine for simple prompts. More complex, detailed images need longer prompts around 100-150 words.

Test and Iterate

Getting good prompts requires testing and refining.

  • Generate 5-10 images per prompt
  • Tweak prompts based on results
  • Repeat until satisfied

Save your best prompts to reuse and share!

Over time, you’ll develop an intuition for prompts that produce what you envision.

Prompt Engineering Resources

Here are some handy sites to aid prompt engineering:

PromptHero

Features a searchable database of popular prompts and lets you optimize prompts.

Lexica

Discover and share prompts. Browse by category and popularity.

PromptBase

Another prompt sharing and discovery platform with handy tools.

Stable Diffusion Discord

Connect with the SD community and learn from experienced users.

So in summary, carefully structuring prompts, choosing descriptive words, and iterating based on results is key to mastering Stable Diffusion. With practice, you’ll be creating incredible AI art in no time! Let me know if you have any other prompt optimization tricks.