# Welcome to dreamlook.ai docs Finetune Stable Diffusion and Generate Images in a Blink! ## Frequently Asked Questions * [How do I train a model from the website?](/train-models/using-the-website) * [How do I train a model from the API?](/train-models/using-the-api) * [How do I use a trained model in AUTOMATIC1111?](/generate-images/automatic1111) * [Can I train models on objects? or on styles?](#i-want-to-train-a-model-on-pictures-of-myself) * [Can I train or generate images involving nudity?](#can-i-train-or-generate-images-involving-nudity) * [What happens to the images I upload?](#what-happens-to-the-images-i-upload) * [How are the models trained?](#how-are-the-models-trained) ## Can I train a model on objects or styles? **Yes, so you can train on anything you want! You can use the instance prompt for this.** You can for example use β€œukj style” for a style or β€œphoto of ukj object” for an object. See for example [this article from Nitrosocke](https://github.com/nitrosocke/dreambooth-training-guide#basic-dreambooth-guide) for more information about training on styles. ## What happens to the images I upload? **They're deleted after 48h, along with the trained models. Upgrade to increase storage duration.** For privacy and security reasons, we quickly delete your data from our servers. This includes both uploaded images as well as the trained models. This also means that when creating models, you only have 48h to download them by default, after which we can't help you recover them. You can extend storage duration by subscribing to one of our plans. [Check out the Pricing page for full details.](https://dreamlook.ai/pricing) ## How are the models trained? **We are finetuning both text encoder and UNet.** Our method gives equivalent results to the [diffusers implementation](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth.py). At this time, we do not use regularization images (also known as prior preservation). ## Can I train or generate images involving nudity? No, as per our [Terms of Use](https://dreamlook.ai/terms), you are not allowed to train from or generate images involving nudity. Training images showing people must show adults who have given their explicit permission. ## Why use dreamlook.ai? 1. **Privacy**: We keep your data for a maximum of 48h by default. After that, we delete both your provided training images and the resulting model. 2. **No lock-in**: After finetuning your own custom models, you can download them and use them for whatever you want. The models belongs to you. 3. **Speed**: dreamlook.ai is the fastest finetuning service. The result is equivalent to other finetuning platforms: there's no cutting corners from our training process. We use state of the art infrastructure to provide the best performance. 4. **Ease-of-use:** You don't have to be a developer to use dreamlook.ai! But if you are, you will still love this tool. We are striving to maximize developer happiness. You get all of this at competitive pricing. Train your first model for free, then pay as you go. ## How do I reach out? You are always welcome to [join our Discord](https://discord.gg/yX9D9KxHMS). Otherwise, feel free to reach out to . # Train models You can train models either from the dreamlook.ai website, or using our API # Using the website Train a custom model from the dreamlook.ai web interface ## Starting a finetuning job Head over to , and sign up if you haven't already. You will be greeted by the simplified model creation interface. From this interface, you can set the training parameters used to finetune a Stable Diffusion model. Finetuning models is an art! You can find many resources online. Read our guide if you are new to this: {% content-ref url="/pages/FRHzCdH1jblXZDvCao64" %} [Good SD1.5 default settings](/guides/good-sd1.5-default-settings) {% endcontent-ref %} A simple example would be training the model on someone's face, using 12 images, with the instance prompt `photo of ukj person` and using the standard Stable Diffusion v1.5 model:
It will cost you tokens to start the job. Buy some if you don't have any left. Your first run is on us. When everything is set up, the "Start job" button will turn blue. Click it to launch the process! πŸš€ ## Downloading the resulting checkpoint As soon as you start the job, you will see it appear in the "Latest jobs" list underneath:
It will take a few minutes for the model to be trained. It usually takes around 3 minutes to train 1'200 steps for SD1.5 models. As soon as your model is ready, you will get an email, and the download link will appear on the list. You need to download it quickly: after 48 hours, it is by default deleted from our servers. ## Generating images using the trained model You can now use your checkpoint to generate images, just follow our guides [Generate images](/generate-images). # Using the API Train custom models and generate images using the dreamlook.ai API {% hint style="info" %} View the full API documentation here: {% endhint %} ## Start a mode training job {% hint style="warning" %} **By default, trained checkpoints are automatically deleted after 48h.**\ You can subscribe to a Plan to increase this duration: {% endhint %} We will explain step by step how to train a Stable Diffusion model using the API. 1. Sign up or log in on [dreamlook.ai](https://dreamlook.ai/) 2. Generate an API key on [dreamlook.ai/api-keys](https://dreamlook.ai/api-keys). Treat this key like a password: keep it safe, and don't expose it publicly! 3. Export your API key as the environment variable `DREAMLOOK_API_KEY` : ```sh export DREAMLOOK_API_KEY="dl-YOURKEYHERE" ``` 4. Go to and switch "Expert mode" on:
You can now configure your run:
4. Copy the content of "API request" and paste it into a terminal. Running the command will trigger a training job:
You will get a unique job ID as a response: ``` { "job_id":"db_63d83f95" } ``` Jobs created from the API are also visible from the website, so you can check to see how your job is progressing. ## Dry runs During development, you may want to perform tests without actually being charged tokens. You can use the dry run mode for this purpose. In your request payload to `/dreambooth`, add the field `"dry_run": true` to start a job in dry run mode. Here's an example: ```bash curl 'https://api.dreamlook.ai/dreambooth' \ -H "content-type: application/json" \ -H "authorization: Bearer $DREAMLOOK_API_KEY" \ --data-raw '{ "steps": 1200, "learning_rate": 0.000001, "instance_prompt": "photo of ukj person", "base_model": "stable-diffusion-v1-5", "saved_model_format":"original", "saved_model_weights_format": "safetensors", "dry_run": true, "image_urls": [ "https://images.unsplash.com/photo-1593439960697-140200051c2c", "https://images.unsplash.com/photo-1593440012688-b6ede71180c3", "https://images.unsplash.com/photo-1593440497401-b87d3bb3fb8b", "https://images.unsplash.com/photo-1593202764227-040c0c69f3d9", "https://images.unsplash.com/photo-1593202620896-f7cb529ed581", "https://images.unsplash.com/photo-1593202873672-79aeb34143d8", "https://images.unsplash.com/photo-1593202695230-db2abb34a742" ] }' ``` Response: ```json { "job_id": "TEST_50d2285d" } ``` As you can see, a dummy job ID is returned, prefixed with `"TEST_"`. This is not a real job: the job ID is not tracked, and you won't see it in your job list either from the API or from the website. If you have specified a callback (see below), the callback will be sent along with some dummy data (including sample checkpoints, generated images, thumbnails, etc.). Dry runs also work on our `image_gen` and `dreambooth_image_gen` endpoints. ## Callbacks: Receive notifications when your jobs are done You may want to add a callback to your request. Typically, this is a publicly accessible URL that points to your own backend and triggers further processing on your end after your job finishes. When the run has finished, the job payload (including results) will be sent via a POST request to this URL. The payload corresponds to the job object, as returned by the `/jobs/{job_type}/{job_id}` endpoint. ### How to test callbacks 1. Head over to [dreamlook.ai/dreambooth](https://dreamlook.ai/dreambooth-api) 2. Set up the run parameters and upload your images 3. Enter your publicly accessible URL into the field "Callback URL" 4. Click the button "Test it". This will send a POST request with a test payload to your endpoint. The test payload contains a signed URL that points to a untrained Stable Diffusion 1.5 model (regardless of your parameters). You won't be charged for this action. The resulting job ID is a test ID (`"TEST_"`) that is not tracked. ## Keeping track of your jobs ### Reading the state of a single job During or after training, you can retrieve the status and results for an individual job using the `/jobs/{job_type}/{job_id}` endpoint. `job_type` can be one of `dreambooth`, `image_gen`, or `dreambooth_image_gen`. Here's an example with a Dreambooth job with ID `db_1ccb410f`: ```bash curl \ -H "content-type: application/json" \ -H "authorization: Bearer $DREAMLOOK_API_KEY" \ https://api.dreamlook.ai/jobs/dreambooth/db_1ccb410f ``` Response: ```json { "job_type": "dreambooth", "job_id": "db_1ccb410f", "state": "success", "state_updated": "2023-06-05T14:01:16.279400", "state_running_start": "2023-06-05T13:59:46.960858", "state_running_end": "2023-06-05T14:01:16.279400", "state_queued_start": "2023-06-05T13:59:38.121577", "state_queued_end": "2023-06-05T13:59:46.960858", "queued_duration": 8.839281, "running_duration": 89.318542, "queueing_state": "default", "config": { "image_urls": [ "gs://nyxai-galactus-prd.appspot.com/dreambooth/4DNzz1vuLIa7gdaNwPDkJL1Onv73/2023-6-5-9-59_11zmplck7u/images/21a498e6_kxq8tsip_photo_2023-05-26-22-03-50.jpeg", "gs://nyxai-galactus-prd.appspot.com/dreambooth/4DNzz1vuLIa7gdaNwPDkJL1Onv73/2023-6-5-9-59_11zmplck7u/images/4723de21_115rofqb_photo_2023-05-26-22-03-49.jpeg", "gs://nyxai-galactus-prd.appspot.com/dreambooth/4DNzz1vuLIa7gdaNwPDkJL1Onv73/2023-6-5-9-59_11zmplck7u/images/c5529bdc_z8lm9jlk_photo_2023-05-26-22-03-46.jpeg", "gs://nyxai-galactus-prd.appspot.com/dreambooth/4DNzz1vuLIa7gdaNwPDkJL1Onv73/2023-6-5-9-59_11zmplck7u/images/d8cc970b_zxwng9gd_photo_2023-05-26-22-03-42.jpeg", "gs://nyxai-galactus-prd.appspot.com/dreambooth/4DNzz1vuLIa7gdaNwPDkJL1Onv73/2023-6-5-9-59_11zmplck7u/images/e60013d5_d0g5zh0d_photo_2023-05-26-22-03-44.jpeg", "gs://nyxai-galactus-prd.appspot.com/dreambooth/4DNzz1vuLIa7gdaNwPDkJL1Onv73/2023-6-5-9-59_11zmplck7u/images/f3277b89_xwgmpaxv_photo_2023-05-26-22-03-47.jpeg" ], "instance_prompt": "photo of ukj person", "steps": 100, "saved_model_format": "original", "saved_model_weights_format": "safetensors", "base_model": "stable-diffusion-v1-5", "checkpoint_id": null, "crop_method": "center", "learning_rate": 1e-06, "learning_rate_te": 1e-06, "learning_rate_te1": null, "learning_rate_te2": null, "enable_offset_noise": false, "callback": null, "extract_lora": "original", "captions_file_url": null, "save_checkpoints_at": null, "text_encoder_training": null, "checkpoint_availability_mode": "standard", "priority": 100 }, "dreambooth_result": { "checkpoints": [ { "checkpoint_id": "ckp_12ed2b03", "url": "https://storage.googleapis.com/nyxai-standard-usc2/models/db_1ccb410f/dreamlookai_stable-diffusion-v1-5_step_100_db_1ccb410f_ckp_12ed2b03.safetensors?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=174139095815-compute%40developer.gserviceaccount.com%2F20230605%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20230605T140116Z&X-Goog-Expires=172800&X-Goog-SignedHeaders=host&X-Goog-Signature=916d6d912be0e369e979d0866d093ce7d7a907002984984b6276eb4202a10d82b59360682749ca31718e2686b70802693a5bc50fd2271d54008aa252d2fd796b0a007944cec24523f474218aadcec12d1917083929b871dce930c6d37e5caf57dd04989af2cb017fc78962f41711adf1a3ab24546ed82d001bfef3341c5440179470d8a8268c4d79784419a6063ec47ddf482bf187e4467cace6533bfc5fc96ee67de91698b7963f28dce38cb38ac9b551030ed49ba10727fbadfec33e9d17421ba555472029e305069fb4def91f0523e8bedd6d44d824a5eb87b8d74da459f10fb9292f5ee936687e8546e2f0a883388c2ab65ef72f2ef6ac7d13aaedccefea", "is_extracted_lora": false, "model_type": "sd-v1", "checkpoint_type": "checkpoint", "training_step": 100, "size_bytes": 2132625438, "file_format": "safetensors", "availability_mode": "standard", "is_available": true }, { "checkpoint_id": "ckp_dc155e79", "url": "https://storage.googleapis.com/nyxai-standard-usc2/models/db_1ccb410f/dreamlookai_stable-diffusion-v1-5_lora_step_100_db_1ccb410f_ckp_dc155e79.safetensors?X-Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=174139095815-compute%40developer.gserviceaccount.com%2F20230605%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20230605T140116Z&X-Goog-Expires=172800&X-Goog-SignedHeaders=host&X-Goog-Signature=2fc29caa6cb82fb26803f869221eebd6508a525292efebd1d1fbeac9417af6ce39dcec696e54fe4dddcaad5eadc4d50ed2944c8119991056b8b31fe139812b69a40951ed3470d0e0c729a83e6314a97e522c068e0905901b4ccda5eefd3d8a0389ed51b6a9186e99d18987345be95f51382f3d9ddda17f462221cc048c23f45280213c51438abc85ed7a4fa1945d25fc7f69ce31ea4c3709c0a04786f3ce0b295ee2fd0a7b4dca35a4b07a9289393b5b64fab073977bd8a0f3da21936e47d712477cf2e07f63e4e7ee7ee41f4ef1f195bafa6d91fd5016fff6dc2d8cf302621c2bba312a6f9bb52386807755e0d4c2d8ce4de2c198b347d68058cb882ce47a73", "is_extracted_lora": true, "checkpoint_type": "lora", "model_type": "sd-v1", "training_step": 100, "size_bytes": 75611048, "file_format": "safetensors", "availability_mode": "standard", "is_available": true } ], "sanitized_images": [] }, "image_gen_results": null, "percentage_done": 100, "failure_reason": null, "failure_reason_data": null, "thumbnails": null, "tags": [] } ``` The job object looks similar for all job types: * `state`: Can be one of `queued`, `running`, `success`, `failure` * `config`: Contains the configuration the job was started with * `dreambooth_result:` Any results from model training * `checkpoints:` List of checkpoints associated with this run. Checkpoints can be full checkpoints or LoRA checkpoints (`checkpoint_type="lora"`) . Also note the `training_step` for each checkpoint. * `image_gen_results:` Any results from image generation (only for `image-gen` or `dreambooth-image-gen` jobs) ### List all your jobs In order to list all existing jobs, use the endpoint `/jobs/{job_type}` (e.g. `/jobs/dreambooth`). By default this lists the most recent 50 jobs. ## Error codes
HTTP response codeMeaning
307Temporary redirect. You may be using an outdated endpoint that is now deprecated.
404The endpoint or resource does not exist.
401Unauthorized. You need to provide a valid API key.
403Forbidden. This may happen if you don't have enough quota (tokens) to fulfill the request.
422Your job configuration/payload is invalid or not allowed.
5xxError codes of 500 or higher indicate that there is something wrong on our side. Please try again or reach out to us.
# Generate images Let's put these custom models to use! We give step by step guides for three popular options: * [Using the website](/generate-images/using-the-website), easiest & fastest * [Using the API](/generate-images/using-the-api), to automate image creation * [DiffusionBee](/generate-images/diffusionbee), for Mac users * [AUTOMATIC1111](/generate-images/automatic1111), the most flexible solution # Using the website The fastest and easiest way to generate images Go to in order to start generating images. ## Generate images from base models 1. Select a model ("Stable Diffusion 1.5", "Realistic Vision", etc...) 2. Pick a prompt sample (e.g. "Woman portrait") or fill in your own prompts 3. Click "Generate" ## Generate images from your trained models (i.e. checkpoints) 1. First train a model on {% hint style="danger" %} Models are only available for 48h after they are trained by default. Afterwards they are gone and no images can be generated from them anymore. {% endhint %} 2. The trained model then appear in the "Model" dropdown:

Dropdown for "Model" on https://dreamlook.ai/image-gen

2. Write a prompt containing your instance prompt (in our example this would be "photo of ukj person"). The instance prompt acts as a "trigger" to generate images of your subject/object. {% hint style="info" %} If you don't use your instance prompt token (by default, `ukj`) in your prompt it's unlikely you'll get any good results. {% endhint %} 2. Click "Generate" πŸš€ ## FAQ ### I can't see my model in the drop down. Where did it go? We delete models after 48h, so if your training run is older than 48h you won't see it in the dropdown. Currently, we don't offer generating images on models trained with Avatar service (), so you won't find those models in the drop down. ### What is "High Res Fix"? High Res Fix (HRF) is a method of generating images at higher resolution than 512x512 while maintaining image quality and reducing artefacts (such as multiple heads). The process includes txt2img at lower resolution, resize to target resolution, and img2img at target resolution, so it can be seen as an upscaling method. Our implementation of HRF is slightly different from the one in AUTOMATIC1111. Note that enabling HRF has an impact on performance. If you want to optimize for speed, choose 512x512 without HRF. ### Can I use prompt weighting (emphasis) like in AUTOMATIC1111? Yes. We use the same syntax as AUTOMATIC1111, meaning that words in brackets, such as`(word)`, increase attention/emphasis to that word by a factor of 1.1, whereas `[word]` decreases attention by a factor of 1.1. You can specify an arbitrary weight with a colon (e.g., `woman in a (red:1.4) dress`). If you want to use literal brackets, you should escape them with backlashes (`\(example\)`). You can find a good summary of the rules on the [AUTOMATIC1111 wiki page](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#attentionemphasis). ### Do you support img2img? What about different samplers? We currently only offer txt2img and not img2img. We focus on keeping the platform easy to use and build highly optimized features that give workable results out of the box. Please share with us if you are missing a feature - we are happy to consider adding it. # Using the API Integrate image creation into your workflows using the dreamlook.ai API ## Start an image generation job 1. Generate an API key on [dreamlook.ai/api-keys](https://dreamlook.ai/api-keys). Treat this key like a password: keep it safe, and don't expose it publicly! 2. Export your API key as the environment variable `DREAMLOOK_API_KEY` : ```sh export DREAMLOOK_API_KEY="dl-YOURKEYHERE" ``` 3. Go to and configure your generation parameters as described in [Using the website](/generate-images/using-the-website) 4. Copy the content of "API request" and paste it into a terminal. Running the command will trigger a new generation job:
5. You will get a unique job ID as a response: ``` { "job_id":"ig_63d83f95" } ``` Jobs created from the API are also visible from the website, so you can check to see how your job is progressing. ## Keeping track of your jobs Check the documentation in [Using the API](/train-models/using-the-api) to see how to list your jobs and check their state. Image generation jobs have the job type `image-gen`. Here's an example of a image generation job response: ```json { "job_type": "image-gen", "job_id": "ig_90c7ebd5", "state": "success", "state_updated": "2023-06-29T16:28:31.317874", "state_running_start": "2023-06-29T16:27:39.284893", "state_running_end": "2023-06-29T16:28:31.317874", "state_queued_start": "2023-06-29T16:25:28.018608", "state_queued_end": "2023-06-29T16:27:39.284893", "queued_duration": 131.266285, "running_duration": 52.032981, "queueing_state": "default", "config": { "prompt": "portrait photo of ukj man, color, Martin Schoeller, serious eyes, perfect eyes, cinematic, 80mm portrait photography, dramatic lighting photography, national geographic, portrait, photo, photography, Stoic, cinematic, 4k, epic, detailed photograph, shot on kodak detailed, bokeh, cinematic, hbo, dark moody, volumetric fog", "negative_prompt": "nudity, amputee, bad anatomy, blurry, fuzzy, disfigured, misshaped, mutant, deformed, bad art, out of frame, off center", "width": 768, "height": 1024, "num_samples": 16, "checkpoint_id": "ckp_15f9112b", "model_type": "sd-v1", "base_model": null, "num_inference_steps": 20, "scheduler_type": "dpm++", "enable_hrf": true, "denoising_strength": 0.6, "seed": -1 }, "dreambooth_result": null, "image_gen_results": [ { "checkpoint_id": "ckp_15f9112b", "config_idx": 0, "generated_images": [ { "generated_image_id": "c7102f3f", "url": "https://storage.googleapis.com/nyxai-standard-usc2/images/generated_images/ig_90c7ebc5/ig_90c7ebc5_c7102f3f.png", "seed": 2811570194, "generation_idx": 0 }, { "generated_image_id": "577ab3c6", "url": "https://storage.googleapis.com/nyxai-standard-usc2/images/generated_images/ig_90c7ebc5/ig_90c7ebc5_577ab3c6.png", "seed": 1587936907, "generation_idx": 2 }, ... ] } ], "percentage_done": 100, "failure_reason": null, "failure_reason_data": null, "thumbnails": [ "https://storage.googleapis.com/nyxai-public-usc2/images/thumbnails/ig_90c7ebc5/ig_90c7ebc5_c7102f3f_thumbnail.png", "https://storage.googleapis.com/nyxai-public-usc2/images/thumbnails/ig_90c7ebc5/ig_90c7ebc5_3ef6b3fc_thumbnail.png", "https://storage.googleapis.com/nyxai-public-usc2/images/thumbnails/ig_90c7ebc5/ig_90c7ebc5_732c4f70_thumbnail.png", "https://storage.googleapis.com/nyxai-public-usc2/images/thumbnails/ig_90c7ebc5/ig_90c7ebc5_93c5e4d0_thumbnail.png" ], "tags": [] } ``` # DiffusionBee Super simple image generation on Mac using your custom models {% hint style="info" %} [**dreamlook.ai**](https://dreamlook.ai/) **lets you finetune Stable Diffusion models in minutes (first run is free!)**\ Follow this guide to train models on [dreamlook.ai](/) that you can use in DiffusionBee. {% endhint %} DiffusionBee is a free and open-source Mac application that allows you to generate images on your computer: **πŸ”—** [**https://diffusionbee.com**](https://diffusionbee.com/) The custom models you train on [dreamlook.ai](https://dreamlook.ai/) can easily be used in DiffusionBee, allowing you to create any kind of images of yourself, your pets or anything else you want! ## Step 1: Train your custom model 1. Head to and enable "Expert mode" 2. Select the following parameters: * "Checkpoint format" must be "**CompVis (AUTOMATIC1111-compatible)"**
You can choose the other options freely such as the number of images, the number of training steps, the base model etc. ## Step 2: Add your custom model to DiffusionBee After your model is trained, download it to your Mac:
Then add it as a custom model in DiffusionBee: * Open DiffusionBee settings:
* In the Settings, under Custom Models, click on "Add New Model" and select the checkpoint you downloaded:
* It will now be available as an option in the "Text To Image" tab under "Custom Model":
## Step 3: Generate images! You can now generate images using your custom model. This will work with both "Text To Image" and "Image To Image" πŸŽ¨πŸ§‘β€πŸŽ¨
Check out [DiffusionBee's help pages](https://github.com/divamgupta/diffusionbee-stable-diffusion-ui/blob/master/DOCUMENTATION.md#text-to-image) if you need more need guidance about how to use it. # AUTOMATIC1111 State-of-the-art image generation using your custom models {% hint style="info" %} [**dreamlook.ai**](https://dreamlook.ai/) **lets you finetune Stable Diffusion models in minutes (first run is free!)**\ Follow this guide to use custom models trained on [dreamlook.ai](/) in AUTOMATIC1111. {% endhint %} AUTOMATIC1111 is one of the most advanced tool to create images with Stable Diffusion: **πŸ”—** [**https://github.com/AUTOMATIC1111/stable-diffusion-webui**](https://github.com/AUTOMATIC1111/stable-diffusion-webui) ## Step 1: Train your custom model on [dreamlook.ai](https://dreamlook.ai/) Head to , and enable "Expert mode" to expose all the training options. Select the checkpoint format "CompVis (AUTOMATIC1111-compatible)":
You can choose other options freely such as the number of images, the number of training steps, the base model etc. Click on "Start job" when you're ready to train. After the training finishes, find the CLI command to download the model on the model page. \ You will need it later to download the model to your AUTOMATIC1111 instance:
## Step 2: Start an AUTOMATIC1111 instance There are [multiple ways](https://github.com/AUTOMATIC1111/stable-diffusion-webui#installation-and-running) to run AUTOMATIC1111, and the precise steps will depend on what you use. In this guide, we will run it on Google Colab, using the excellent notebook from TheLastBen: **πŸ”—** [**https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast\_stable\_diffusion\_AUTOMATIC1111.ipynb**](https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb) * Simply open this Colab notebook, and run all the cells:
* After a while, you will see a public URL displayed:
* Open that URL to reveal the AUTOMATIC1111 web ui!
## Step 3: Add your custom model to AUTOMATIC1111 * Now that everything is setup, let's interrupt the web ui so we can download the checkpoint:
* Use the URL to your model from [dreamlook.ai](https://dreamlook.ai/) (see [Step 1](#step-1-train-your-custom-model)), then create a new cell in the notebook and download the checkpoint to the AUTOMATIC1111 model folder: ```notebook-python !wget --output "/content/gdrive/MyDrive/sd/stable-diffusion-webui/models/Stable-diffusion/custom_model.safetensors" "" ```
* You can now restart the web ui by running the previous cell. You will now be able to select your custom model in the "Stable Diffusion checkpoint" dropdown:
*It should be straightforward to adapt these steps if you use another AUTOMATIC1111 notebook or run it on another platform. If you have any issue, just ask on the* [*dreamlook.ai Discord server*](https://discord.gg/yX9D9KxHMS)*!* ## Step 4: Generate images! You can now use the full power of the AUTOMATIC1111 web ui to create images πŸ€©πŸ–ΌοΈ
AUTOMATIC1111 is a powerful tool with a large array of options. There are many great tutorials to get started on YouTube - we particularly recommend content from [Aitrepreneur](https://www.youtube.com/@Aitrepreneur). # Advanced features # Extract LoRA Create lightweight LoRA files extracted from full Stable Diffusion models {% hint style="info" %} [**dreamlook.ai**](https://dreamlook.ai/) **lets you finetune Stable Diffusion models in minutes (first run is free!)**\ Follow this guide to create LoRA files from your own images on [dreamlook.ai](/). {% endhint %} ## LoRA files: a smaller alternative to full checkpoints When finetuning Stable Diffusion models, the trained models are huge files of at least 2 GB. \ It can be a challenge to download, store and load such files if you train hundreds of models! LoRA files allow you to train only some of the parameters of the model, which result in a 75 MB file. The result is not as good as when training the full model, but the result is much more manageable. This can be a good trade-off in some situations. Another advantage of LoRA files is that you can use multiple LoRAs at the same time when doing image generation, and you can even specify different weights for each of them (which can be positive or negative). ## What are extracted LoRAs? There are two ways to get LoRA files: 1. **Train the full model, then extract a LoRA file from it.** In this method you allow full finetuning of the base model and then extract a LoRA model from the finetuned model. Technically speaking, this can be achieved by running SVD on the the finetuned model checkpoint weights. This is what we offer on [dreamlook.ai](https://dreamlook.ai/). 2. **Fine-tune LoRA models natively**. You can train LoRA models directly with relatively minimal resources, using libraries such as Kohya's GUI (). We currently don't offer native LoRA finetuning on [dreamlook.ai](https://dreamlook.ai/). **Extracted LoRAs give better results than native LoRAs.** They require more resources, as you need to train the full model, but this is something we do very efficiently on [dreamlook.ai](https://dreamlook.ai/) (models can be trained in minutes). LoRA files extracted on [dreamlook.ai](https://dreamlook.ai/) have rank 64, which results in a 75 MB file. ## How do I use extracted LoRA on [dreamlook.ai](https://dreamlook.ai/)? There is no extra cost to extract a LoRA file when training a model on [dreamlook.ai](https://dreamlook.ai/) and it is enabled by default. At the end of your run you should see both full checkpoint and LoRA appear. For API users, you can set the option `extract_lora` to `original`. This option is set to `disabled` by default, which means LoRA extraction does not happen. # Offset Noise Train models able to create very dark or very bright images {% hint style="info" %} [**dreamlook.ai**](https://dreamlook.ai/) **lets you finetune Stable Diffusion models in minutes (first SD1.5 run is free!).** Follow the guide below to train models with Offset Noise enabled. {% endhint %} ## What is Offset Noise? Using Offset Noise during training improves the contrast in images created by the model. The improvement is especially visible in images with very dark or very bright lighting conditions. **Output of models trained without Offset Noise (left) vs with Offset Noise (right):**
Models trained with offset noise are able to create very dark or very bright images

Left: without Offset Noise. Right: with Offset Noise

Without Offset Noise, Stable Diffusion models struggle to create images that are very dark of very bright. The results always include bright areas even if the prompt describes dimly lit scenes. This is due to limitations in the training procedure that is normally used, which Offset Noise fixes. The images showcased above come from models that were trained from the same base model (`stable-diffusion-v1-5`), using the same training parameters and images (1200 steps, LR 1e-6, 16 images) except for the Offset Noise scale. The images were generated using the exact same parameters including seed: ``` Prompt: dramatic photographic portrait of a techwear handsome ukj person, on the rooftop of a futuristic city at night, sigma 85mm f/1.4, 4k, depth of field, high resolution, 4k, 8k, hd, full color Negative prompt: lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, text, signature, watermark, username, blurry Steps: 30, Sampler: Euler a, CFG scale: 10, Seed: 2650451456, Size: 600x1000, Model hash: 9714774f, Denoising strength: 0.7, First pass size: 0x0 ``` A word of caution: while Offset Noise allows the trained model to produce better images in dark and bright conditions, it will also have a noticeable impact on model outputs in other situations. You may need to adjust your prompts to address this. This is why we don't enable it by default. ## How to use Offset Noise on [dreamlook.ai](https://dreamlook.ai/)? #### Using the [dreamlook.ai](https://dreamlook.ai/) website Head to the Training page: The parameter can be enabled by checking the "Offset Noise" checkbox under "Advanced settings"
**πŸ”—** [**How to train models using the website**](/train-models/using-the-website) #### Using the API When using the API, Offset Noise can be enabled using the boolean parameter `enable_offset_noise`. This value is `false` by default, meaning that Offset Noise is not used if you don't specify it explicitly. **πŸ”—** [**How to train models using the API**](/train-models/using-the-api) ## How does Offset Noise work? The idea of Offset Noise was first introduced in a post by Nicholas Guttenberg from Cross Labs: The YouTube channel **koiboi** features a good explainer video: {% embed url="" %} # Image captions Improve your results by providing captions When training Stable Diffusion models, captions can help the model better understand the concepts and styles in your training images. ## Using captions Take the following steps to provide captions. 1. Open a text editor and create a new file `captions.json` (you can name it whatever, but it has to be a JSON file) 2. For each image file in your training dataset, write a caption in the JSON file. Each individual caption can be up to 77 tokens long (around 300 characters). Use the filename **including file ending** but without the folder or path: {% code title="captions.json" overflow="wrap" %} ```json { "myimage0001.jpeg": "ukj soda bottle on a wooden table in warm afternoon lighting, pine trees in the background", "myimage0002.jpeg": "two ukj soda bottles next to a tropical waterfall", ... } ``` {% endcode %} You can also provide multiple captions per image by providing a list. In that case we will randomly sample them during training: ```json { ... "myimage0002.jpeg": [ "two ukj soda bottles next to a tropical waterfall", "two ukj soda bottles by a waterfall in the jungle" ], ... } ``` 4. Go to and enable "Expert mode" at the top of the page. 5. Upload the JSON file by clicking on "Image captions" under "Advanced settings". 6. Configure the other parameters for your job and start it. That's it! {% hint style="info" %} You do not have to provide captions for *all* images. When not providing captions for some of the training images we will fall back to `instance_prompt`. {% endhint %} {% hint style="info" %} When using the API, the filename is the last part of the image URL, excluding any URL parameter. For example: * URL: `https://myserver/images/image001.jpeg?token=x91dj1kjh41bxlj1` * Filename to use in JSON file: `image001.jpeg` {% endhint %} ## Using individual `.txt` files Some captioning tools use individual `.txt` files for captions:
We provide a Colab notebook to convert captions in this format into our JSON format: **πŸ”—** [**https://colab.research.google.com/drive/13s9cMduESF4Wzv8tVcajQPLjrdoH5hH3**](https://colab.research.google.com/drive/13s9cMduESF4Wzv8tVcajQPLjrdoH5hH3#scrollTo=jmHafIiwa9F6) Simply follow the instructions in the notebook to create the JSON file, then upload it to [dreamlook.ai](https://dreamlook.ai/) as described above when configuring your job. ## Failure cases The job may fail under the following conditions: * The file could not be parsed as JSON. * The file contains captions in an invalid format - see above. * Not a single provided caption could be matched to a filename. If this happens, the tokens used for this job are immediately returned. ## How to best write captions? This is an active field of research and this likely evolve over time. Since writing captions manually can be quite tedious, a common practice is to use AI models such as GPT4V or BLIP2 to write captions automatically. **Don't hesitate to ask on** [**our Discord server**](https://discord.gg/yX9D9KxHMS) **if you are looking for more guidance!** # Guides Training Stable Diffusion model is an art! We provide some guidelines to help you get started. # Good SD1.5 default settings Guidelines to get you started with the art of finetuning! ## What are good default settings to train a model? ### Training images The first step is to assemble a good training set. These recommendations are valid no matter if you want to train a model on a person or an object. * **Use at least 10 to 15 images** for best results. * **The images should be good**: in focus, well lit, clearly showing the subject, without other major elements in the image. * **The images should be varied**: include multiple camera angles, lightning angles and backgrounds. When training on a person, it's important that they show various face expressions. Avoid accessories that cover the visage or excessive makeup, except if they're an integral part of the person's identity. * Adding more images will help, **but only if the images are good**. Adding mediocre images will decrease quality rather than improving it. ### Training parameters * **Finetuning steps:** a good starting point is to 100 training steps per training image. So for example for 13 images, you would train for 1'300 steps. * **Learning rate:** don't touch it unless you really know what you're doing. The default is 1e-6 (`0.000001`) * **Instance prompt:** `photo of ukj person` is a good default when training on a person. Use `photo of ukj object` for an object, `ukj style` for a style. * **Base model:** `stable-diffusion-v1-5` is the standard base model, powerful and versatile. `realistic-vision-v1-3` is great for realism, and a good choice when training on people. * **Image cropping:** use "Face Crop" if you're training on a single person's face, otherwise leave it to "Center Crop". * **Checkpoint format:** leave it on "CompVis", the format compatible with the AUTOMATIC1111 web UI. * **Use safetensors:** leave it on, it saves the model in a more recent file format that ensures safety. * **Callback URL:** leave it blank. * **Offset Noise:** leave it off if you don't want to create images that are specifically very bright or very dark. ### What about LoRa files? The recommendations above are also good to train LoRa files. LoRa files are 20x smaller than full models (100MB vs 2GB), but may not reach the same level of quality. ## Going further This article from HuggingFace provides a good analysis in more details on how to train models: Ultimately, you will need to learn what works best for your specific subject and your specific images through trial and error: * Try out various number of training steps, training between 90 and 110 steps per images * Try out various base models * Try out various selections of training images # Improving SD1.5 results quality A case by case guide to making your results better **First things first: it's important to understand that training models is not (yet) a precise science.** There are no magic numbers, only rules of thumb! You will need to iterate to find what works best for your specific use case. ## My images don't look like the subject! There can be multiple reasons for this. Start by improving your training set quality, before trying to tweak other parameters. ### Improve the training set quality * **Improve your training images.** Only include high quality images, that are well lit, in focus, and show the subject clearly. More images is generally better, but only if all the images are good. Adding mediocre images will **not** help. * **Crop your images to square format.** The model is trained on 512x512 squares. Depending where your object is located in your training images, it may end up being cropped! By cropping the images yourself to a square format, you'll make sure the model is trained precisely on what you want. When training on a person, you can also use "Face Crop" which will help optimize cropping, but nothing beats careful manual cropping. * **Make sure your images are large enough.** The model is trained on 512x512 squares. If your images are smaller than that, they will be stretched which will degrade results. ### Tweak the training parameters * **Increase the number of training steps.** Instead of 100 steps per image, increase to 110-120 steps. * **Try out other base models.** The different base models react differently to each subject, so it can be worth trying them all. ## My images all look the same! This is a clear sign of overfitting, which can be caused by: * Training for too many steps * The training images are too similar * Training with a too high learning rate. We don't recommend changing the default (`0.000001`) unless you have a good reason to # Stable Diffusion XL Here is how you get the most out of SDXL finetuning! ## Training SDXL models If you are new to SDXL training, here are recommended settings to get your started! ### 🎭 Training on portraits * **5-15 high resolution images:** SDXL does not require many images to get good results. Make sure your images are at least 1024x1024, or they will be scaled up which can introduce artifacts. * **Train for 1500 steps.** You don't need to adjust this number based on the number of images, 1500 steps will give you a good starting point with anything between 5 and 15 images. * **Use the base Stable Diffusion XL model (**`stable-diffusion-xl-v1-0`**).** Other models are harder to train and should only be used when you have more experience with SDXL training. * **Set the learning rate at 1e-5 (**`learning_rate`**).** Leave the learning rate as default. * **Set the Text Encoder 1 (`learning_rate_te1`) learning rate to 3e-6.** Currently we don't support training of Text Encoder 2 (hence its learning rate is set to zero). * **Disable Offset Noise.** Don't use Offset Noise except if you specifically want to generate very bright or very dark images. Check out the [Good SD1.5 default settings](/guides/good-sd1.5-default-settings) guide for general advice about image selection. ### 🎨 Training on style * **Use 50 to 500 high-quality images.** Again, make sure they're at least 1024x1024 to avoid image resizing. * **Train for 100 steps per image.** The optimal number will vary depending on your specific style, so this is the first thing you want to experiment with. * **Use the base Stable Diffusion XL model (**`stable-diffusion-xl-v1-0`**).** * **Set learning rate at 5e-6 (**`learning_rate`**).** This lower learning rate will help to model train on such a large dataset. * **Set the Text Encoder 1 (`learning_rate_te1`) learning rate to 3e-6.** Currently we don't support training of Text Encoder 2 (hence its learning rate is set to zero). * **Disable Offset Noise.** * You can use **image captions** if you have them available, but this is not strictly necessary. If the captions are good, it will help the training process. [See how to format your captions.](/advanced-features/image-captions) ## Generating images with SDXL Unlike SD1.5, SDXL works very well with short prompts. **Using your familiar Stable Diffusion 1.5 prompts with SDXL will generally result in poor images.** If you have trained a model with instance prompt `"photo of ukj person"`, prompts such as `"portrait photo of ukj person as santa claus"`, or `"professional photo of ukj person as cowboy"` will give decent results out of the box. We recommend to use the `dpmpp-2m-karras` scheduler for best results. We generate 1024x1024 images with SDXL at around 3.5s/img (30 inference steps). ## How are you training the model? By default we are training the UNet and Text Encoder 1 at a resolution of 1024x1024. Training of text encoder 2 is not supported. In order to disable training of a text encoder, set its learning rate to zero. In order to increase training efficiency, we are using SNR-Ξ³ at 5.0. ## Is LoRA supported for SDXL? Yes! A LoRA file is extracted at the end of your run by default. The rank for our SDXL LoRA is 64. The resulting file is roughly 440MB in size (14x smaller than the full checkpoint). ## How do I improve on my results? We have a very enthusiastic and diverse community of model builders [on our Discord](https://discord.gg/t9Yj4CEwnA). Come say hi, we will try to help you as much as possible! # Payment Buy tokens and subscriptions on dreamlook.ai ## Pricing [Visit the Pricing page to check full pricing details](https://dreamlook.ai/pricing) [Visit the Billing page to purchase token packages and subscriptions](https://dreamlook.ai/billing) ## Which payment methods do you accept? We accept payments from customers worldwide. Supported methods: * Visa, Mastercard and American Express cards * Apple Pay * Google Pay * PayPal * WeChat Pay * Alipay ## How are payments processed? All our financial processing is handled by Stripe. We do not store any information about your card. Stripe has bank-level security standards. ## Can I get an invoice or a receipt? Yes, you can get an invoice or receipt for your purchases. Head to the [Billing page](https://dreamlook.ai/billing) after your purchase and click on "View" for the corresponding transaction. # API reference For our most up-to-date API spec please visit either or