Paint Along With Frida #5: Sequential Art | Artificial Intelligence | Articles | Ant Smith

TL;DR: Creating a graphic novel with Stable Diffusion generative AI.

Mine! a tale of two ferret brothers abandoned on the mean streets of Loftus:

[click the cover page to read the whole comic in fullscreen]

Creating Sequential Art

In the image below you can see 9 different renditions of 'white ferret' that Frida drew for me:

You can see not only are all these ferrets pretty poorly drawn, they each look like very different individuals.

To get round this I created a Textual Inversion trained on 6 photographs of our own ferrets. I saved this training in a file called myferret.pt and used 'myferret' in the prompt instead of 'white ferret' which caused Frida to draw from this new training.

Read more about creating and using Textual Inversions at https://stable-diffusion-art.com/embedding/.

There are also other ways of training the AI, but this is the simplest and quickest - I used only 6 photographs of real ferrets to re-train Frida.

It can be difficult to get the AI to pay sufficient attention to the embedding, so I often had to up the weight of the myferret token in the prompt - e.g. (myferret:1.6).

Other Processing

My intention in creating the comic was that the imagery (each frame) would be wholly AI generated, with the frames assembled into pages and texts added thereafter (in Photoshop^®™ as it happens). I didn't do any in-frame compositing, what you see (frame by frame) is exactly what Frida drew for me.

However, I did allow myself to do tonal adjustments on the frames, either the Black and White conversions or else just boosting contrast and vibrancy. I could have probably avoided the need for that if I was more skilled with the AI but I feel the result still stands as an AI-generated comic.

I did have to generate each of the frames at the right aspect ratio for how they fit on the page. Once I started assembling the comic I found some generated images were the wrong shape for where they would sit, so I went back and generated new images at the right aspect ratio. Each of the frames are the full-size images Frida drew (with her own upscaling where needed).

I think it worked, and Frida drew me an enchanting little tale with recognisable characterisations.