Steps for crafting prompts in Stable Diffusion

Stable Diffusion is an image generation model trained on vast datasets of images and their captions. To generate a new image, you provide a text prompt describing the desired output. The key is crafting effective prompts that communicate what you want while working within the model’s capabilities.

Start with Precise Descriptions

The most basic prompts are simple descriptions of the image you want to generate. For example:

A scenic photograph of a lake surrounded by tall pine trees, with a mountain range in the background

Focus on details like subjects, styles, composition, and mood. The more precise your description, the better chance the AI understands your intent.

Guide the Style and Quality

Expand on descriptions by specifying artistic styles, qualities, and medium. For example:

An oil painting by Claude Monet of a lake surrounded by pine trees with reflection in the water, in very high resolution

Terms like “oil painting” and referring to Claude Monet help define the style. Asking for “very high resolution” targets quality.

Set the Tone and Composition

Suggest emotionally evocative words and descriptive phrases to convey tone and composition:

A majestic snow-capped mountain towering over a tranquil alpine lake at sunrise, dramatic lighting

Phrases like “majestic,” “tranquil,” “towering over,” and “dramatic lighting” set the tone and guide positioning of elements.

Define Relationships Between Subjects

Show relationships between subjects using positional terms and descriptive verbs. For example:

Two cats sitting together on a couch, one grooming the other affectionately

Words like “together,” “sitting,” and “grooming…affectionately” define the subjects’ pose and relationship.

Use Comparisons Mindfully

You can guide styling and composition through comparisons. For example:

A powerful wizard casting a spell, digital art like The Lord of the Rings

But comparisons can also lead the AI astray if overused. Lean on the model’s default knowledge before resorting to comparisons.

Iterate on Details

Start broad then add details iteratively in subsequent generations:

A tall oak tree on a hilltop overlooking a pond
A mature, robust oak tree standing alone on a grassy hilltop overlooking a small, round pond, digital art

Building up details through multiple attempts lets you home in on your vision while allowing the model freedom.

Balance Guidance Against Surprise

Too much guidance can limit the AI’s creativity. After setting the core prompt, leave room for the model to surprise you:

An alien creature with blue fur, extravagant clothing, posing dramatically

Core elements like “alien creature,” “blue fur,” and “posing dramatically” set the stage while letting the AI get creative with the clothing and other details.

Avoid Problematic Content

Stable Diffusion can generate harmful, biased, or illegal content if prompted. Craft prompts responsibly and redirect the model if responses concern you.

Helpful Prompt Engineering Resources