Apps & Guides

Fooocus: Rethinking of SD and MJ

The advent of Stable Diffusion and MidJourney has revolutionized our understanding of the potential of generative neural networks. These tools have unveiled a fresh perspective on the process of image creation and the extent to which we can manipulate it. The primary approach involves providing the system with prompts about the desired outcome. Essentially, we highlight three important aspects: object, style, and environment.

Additional prompts that provide more specific instructions, such as the desired composition, type of camera/lens, and colorization, are also important, but not indispensable. The more comprehensive the instructions, the easier it is for the neural network to process. The role of a prompt engineer has even emerged in the professional space. However, this role can be easily replaced by the same generative neural networks. By combining image creation with text creation skills, we can generate extra prompts to achieve an optimal outcome.

This is the fundamental concept of Fooocus. It integrates the XL Stable Diffusion model and a GPT2-based prompt generator, which enriches and details your simple prompt. Moreover, Fooocus is equipped with various enhancements and extensions. These features facilitate the generation of spectacular images through a straightforward interface, devoid of complex tools. Let’s delve into its functionality and install Fooocus on a LeaderGPU dedicated server.

Prerequisites

Begin with the installation prerequisites and reboot afterward:

sudo apt update && sudo apt -y upgrade && sudo ubuntu-drivers autoinstall && sudo shutdown -r now

Download the shell script that installs Anaconda for managing virtual environments:

wget https://repo.anaconda.com/archive/Anaconda3-2023.09-0-Linux-x86_64.sh

Set the execution flag and provide data access:

chmod a+x Anaconda3-2023.09-0-Linux-x86_64.sh

Run the installation script:

./Anaconda3-2023.09-0-Linux-x86_64.sh

After the process is finished, we recommend disconnecting the SSH-session and preparing for port forwarding. You need to forward port 7865 from the remote server to a local loopback address, 127.0.0.1:7865. For more information, please refer to one of our previous guides: Stable Video Diffusion. Then, reconnect and proceed with cloning the project’s repository on a GitHub.

Fooocus install

git clone https://github.com/lllyasviel/Fooocus.git

Change directory to Fooocus:

cd Fooocus

Create a virtual environment using Anaconda and the YAML-config prepared by the project’s author:

conda env create -f environment.yaml

Let’s change our base environment to a newly created one:

conda activate fooocus

The following step is to install Python libraries:

pip install -r requirements_versions.txt

Now, everything is ready to start:

Fooocus start

python entry_with_update.py

The initial startup may take some time as the application verifies and downloads all the necessary files for operation. You might want to grab a cup of coffee in the meantime. Once the process is complete, open your browser and type the following URL into the address bar:

http://127.0.0.1:7865

Enter your simple prompt and click the Generate button. If you want more control, tick Advanced and select the necessary options:

The real magic unfolds behind the scenes. The moment you hit the Generate button, your input prompt is transferred to the GPT2-based language model. This model transforms your brief prompt into a mix of elaborative positive and negative prompts. This mix is subsequently input into the Stable Diffusion XL model, fine-tuned to emulate MidJourney style. As a result, even a brief prompt can generate impressive results.

Certainly, there’s no restriction on writing your own prompts. However, after multiple iterations, it becomes evident that even in the absence of this, the generated content remains intriguing and diverse.