Low-Rank Adaptation, LoRA, allows for fine-tuning without the need to retrain the entire model. You can teach FLUX a specific style or a specific aesthetic, even a face.
A while back, there was a big trend of uploading a dozen photos to some shady app, maybe paying a few euros, and getting a set of images back. We prefer to keep our photos private, train a FLUX add-on, and write as many prompts as we want.
- With your GitHub account, sign in to Replicate and setup payment. You will spent 5 dollars for training and some cents per image.
- We need a dataset. Pictures of your face, 1024×1024, at least 20. Put them in a ZIP archive.
- Open the LoRA trainer and train a new model. Set the model to “private”. Upload your ZIP archive.
- Choose a trigger word to use in your prompts later. There’s no easy way to recover it if you lose it, so it’s best to name your model after your trigger word.
- Captioning. You have to tell the model what it is seeing in each picture. You can leave it to Replicate to caption your photos, then you must include your trigger word in the autocaption_prefix, e.g. “A photo of TRIGGERWORD”.
- Optional: Create your own captions. Include a TXT file for each image in the ZIP archive. You control the variables you want to change later in your prompt. It’s counterintuitive, but you don’t describe to the model what you want it to learn. I wouldn’t describe the shape and color of my glasses, my eye color, or my freckles, but my expression and the overall setting. This guide on Reddit is super helpful.
- Create the training. This can take some time, maybe 20 minutes? Meanwhile, make yourself familiar with FLUX prompting which is different from Midjourney or Stable Diffusion. Check out prompt archives with examples and ask yourself, what kind of community is this that prompts only beautiful women in suggestive poses?
- Run trained model. Include your trigger word in the prompt. (Definitely check out what happens when you use adjectives like beautiful or stunning.) I played around with the settings a bit. Most examples here are from a training set of 27 images, where I wrote my own captions with the help of ChatGPT.
My partner’s comment, after I sent her lots of AI portraits, was: “Ole, stop it! That’s creepy!!”
