📲Diffusion Model Training - Dreambooth

Futruism's factory (artificial intelligence model) and factory training are supported by Dreambooth, initially released by Google Research for training diffusion models.

Note: Futruism was not involved in this research nor contributed to its development. Futruism utilizes this framework and provides linkage for reference to enhance transparency.

Dreambooth: Fine-tuning Text-to-Image Diffusion Models for Theme-Driven Generation

Abstract: Large-scale text-to-image models represent a significant leap in the evolution of artificial intelligence, capable of synthesizing high-quality and diverse images based on given text prompts. However, these models lack the ability to imitate the appearance of subjects from a given reference set and synthesize their novel representations in different contexts. In this work, we propose a new approach to personalized text-to-image diffusion models (specialized according to user needs). By simply inputting a few theme images, we can fine-tune pre-trained text-to-image models (Imagen, although our method is not limited to a specific model), enabling them to associate a unique identifier with a specific theme. By leveraging semantic priors embedded in the model and new self-supervised class-specific priors for preserving loss, our technique can synthesize subjects in different scenes, poses, perspectives, and lighting conditions not present in the reference images. We apply our technique to several tasks previously deemed impossible, including subject reconstruction, text-guided view synthesis, appearance modification, and artistic rendering (while retaining key features of the subject). Project page: https://dreambooth.github.io/1

Introduction: Can you imagine your beloved dog traveling the world, or your favorite handbag showcased in the most upscale Parisian display room? Is your parrot the protagonist of a picture book? Rendering these imagined scenarios is a highly challenging task, requiring the synthesis of specific themes (objects, animals, etc.) in new environments to seamlessly integrate into scenes. Recently developed large-scale text-to-image models represent a significant leap in artificial intelligence evolution, capable of synthesizing high-quality and diverse images based on text prompts written in natural language. One of the primary advantages of such models is the powerful semantic priors learned from large collections of image-caption pairs. For example, through this prior learning, the word "dog" can be associated with various instances of dogs appearing in different poses and contexts within images. Although the synthesis capabilities of these models are unprecedented, they lack the ability to mimic the appearance of subjects from a given reference set and synthesize new representations of the same subject in different contexts.