Paint Along With Frida #2: Creative Freedom | Artificial Intelligence | Articles | Ant Smith

The most disconcerting thing about AI generative art is that it can only draw concepts it has seen before - try asking for a centaur and you will have a hard time getting what you want...

So when I asked for a 'hybrid male torso, octopus head' I was suprised to find it drew a very competent Cthulhu creature. So competent I couldn't help but wonder, whose art was it ripping off now?

I don't want to be creating works that one day turn out to be virtual rip-offs of some artist I had never heard of (but the AI had).

It seems like common practice in the 'AI community' to cite the styles of specific artists - in fact you can find lists of artists that the AIs know, both long dead and certainly still living. It isn't at all clear that these artists consent to their work being used in this way - in fact the latest Stable Diffusion allows for artists to opt-out of being included in the training data, so there's definitely an open question on the 'cultural acceptability' of what these things generate. I guess like the early days of audio sampling...

It's also very common for people to use a negative watermark prompt, to avoide the AI drawing something like the watermarks it has seen in its training images; because it has been trained on images that are expressely asserting their ownership! I saw a watermark kind of thing when I asked for a fairytale castle, my guess is the AI was drawing heavily on Disney at that point!

Ownership is a very tricky concept, and although I believe all ideas are public property, the effort to turn them into something concrete is not! The problem with AI generation is that they 'ape' rather than 'copy' artists, so it is difficult for it to say what work has 'influenced' the output, or how strong that 'influence' was.

With an unclear copyright proposition, and a very limited 'consent' model for artists, there is a mire of concerns between what can, and what should, be done with such software platforms.

For now, I avoid suppressing 'watermarks' so at least I am aware that the AI is drawing upon images that had an ownership asserted; and I rarely cite specific artists to define the style of the result. This makes it more difficult to create the kinds of 'stunning' images one sees from others - but those are 'stunning' in a very ordinary way... I mean, a lot of AI art ends up looking very same-y.

I think the true creative power of these systems are released when you start training them with your own images. But more on that later.

However, a well constructed prompt can deliver something that at least feels like a creative (not copying) endeavour. And there's a lot of freedom in the 'parameters' that are set to produce images. So I want to look at (most) of these parameters first, before getting into the nitty gritty of so called 'prompt engineering'.

It can take hours to get an agreeable result with hundreds of images generated.

The seed value has a major impact on the composition you end up with, as it defines what the starting 'random noise' looks like. If you get something you like, but isn't quite right (and doesn't get fixed by tweaking the number of steps) you can tweak the seed number. Adding or subtracting 1 or 2 will typically generate the same kind of composition. Be aware though, this only works for small seed changes, the composition can jump to something very different at any moment.

Effect of seed variation, left side shows the early effect of the seed (1 step)

Clearly in the above video the main subject changes dramatically from image to image - but there is a strong commonality in the composition throughout with very many of the cats set against greenery. The prompt used to generate these images was simply: a cat jumping. Typically a prompt would have more to say than that and would influence the composition as well - in which case the impact of tweaking the seed may be subtler and sometimes things like 'extra limbs' disappear with a slight tweak.

The AI tries to create a coherent image from the given prompt and the random noise the seed generates. It either introduces things into the composition (lower CFG values) or it mangles the things it has been asked to draw to achieve this. In the following video you can see a seated figure, becomes a squatting figure, becomes a figure with 3 legs, becomes a standing figure as the CFG is increased (freedom is reduced):

Effect of CFG variation (1-30 in 0.5 increments)

The number of steps used has a subtle impact (well, sometimes less subtle at low step counts or randomly with ancestral samplers). Working through (very) many generations at differing step counts feels like real creative effort - so even if the image is largely informed by some portion of the original training data, you are at least bringing something new to the party. You can feel as though you are creating a collaboration even if it isn't a wholly new work.

Evolution of an image from 1 to 120 steps

As few steps as possible are used to generate the image, but the step count is increased to try and improve finer details.

Because so many images need to be created with minor tweaks, it is possible to set a batch of generations off. A simple batch will just step through consequtive seed numbers. Here's the results of a 4x batch for the prompt "man, masculine, filthy rags, shirtless, rotting flesh, laying on back in mud, wounds, ((blood)). 16mm lens ultra wide-angle panorama, ground view. Gloomy, cobwebs, dirt, dust.". I asked it to start with a random seed and it generated the four images using seed values: 84222796, 84222797, 84222798, 84222799:

[The 'x/y/z plot' script can be used to create more complex batches varying any of the parameters]

You can also play games with the requested image size. Stable Diffusion 1.5 was trained on 512px images - so that's all it really knows what to draw. If you use SD1.5 and ask for a 1024px image it will repeat itself in order to fill the canvas. Which can create interetsing results, like the Cthulhu image I'm using to illustrate this article. With careful use you can encourage, say, the generation of a pair of twins. The scope of this may seem limited, but it is another degree of freedom.

Finally, without (yet) getting into inpainting or training, you can also get fascinating results by interrupting the AI. It's pretty non-deterministic (that is to say hit-n-miss), but still a thing to play with...

Here I asked for 120 steps and then randomly interrupted at different points:

There is no doubt that there are enormous questions of 'validity' and 'ownership' when working with AI image generation, and as things stand it's totally down to the 'operator' of the AI (you, me) to decide if they are truly 'present' in the results. I love how these tools democratise art, make it possible for folk like me (who can't draw for toffee) to create visual works, but perhaps more than in any other creative endeavour, the artist's (operator's) integrity is paramount.

Actually, the same is true in photography, but photographers rarely bother to stop and think aout such things. Perhaps now they will have to!