Stable Diffusion Prompt Modifiers

I’ve been experimenting with Stable Diffusion for a few months now, and one of the most important things I’ve learned is how to properly use prompt modifiers to guide the AI image generation process. In this article, I want to share my personal experiences and advice when it comes to crafting effective prompts.

What Are Prompt Modifiers?

In simple terms, prompt modifiers are additional descriptive terms and phrases you add to the end of your textual prompt to influence certain qualities of the generated image. For example:

A cute puppy playing in the grass during golden hour lighting

Here, “golden hour lighting” is acting as a prompt modifier to specify the desired lighting and mood for the image.

Some common types of prompt modifiers include:

  • Style Modifiers: Terms that indicate an artistic style (e.g. “impressionist”, “art deco”)
  • Quality Modifiers: Words to enhance image quality (e.g. “intricate details”, “cinematic lighting”)
  • Subject Modifiers: Words that describe the subject matter (e.g. “majestic”, “menacing”)

By chaining several modifiers together, you can exert fine-grained control over the output:

An astronaut floating through a nebula, trending on artstation, intricate details, sharp focus, illustration by greg rutkowski and alphonse mucha 

Helpful Resources for Finding Modifiers

When first starting out, it can be tricky to know which modifiers work well. Here are some great resources I’ve used:

Browsing examples from these sites gives you a good feel for effective phrasing. I also save prompts that produce results I like to reuse elements later.

Prompt Structure Tips

When constructing prompts, I try to follow this general structure:

[Subject] [Style Modifiers] [Quality Modifiers] [Subject Modifiers]

Keeping related modifiers grouped together helps the AI understand correlations. I also find placing the core subject first ensures it remains the focus.

Some other prompt writing tips I follow:

  • Use comma separation for modifier lists
  • Avoid ambiguous descriptive terms
  • Prioritize the most important modifiers first
  • Try not to exceed ~40-50 tokens

Test, Iterate, Improve

Like anything in AI, writing good prompts requires experimentation and patience. I always start broad and gradually add modifiers until I hone the output.

Testing prompts with different random seeds is also important. It provides variation between generations to better evaluate modifier effects.

I hope these personal insights on prompt engineering help you on your own AI art journey! Let me know if you have any other questions.

Useful Prompt Engineering Resources