Prototypes
Before making The TrancePorter I spent a long time experimenting with sound and visuals. The concept was straightforward enough, frequency is frequency whether it expresses in the visible spectrum or in an audio range. So translating from vision (colour) to audio (pitch) wasn't going to be much of a problem.
Of course, the images I wanted to use (from my photography) were stills and music needs a time component. So I devised a system with both video and audio synthesis. By animating the images I could release the pixel data over time and create an audio flow.
At first I wanted to create the video just however it panned out using games, rules or models to cause the pixels to transform - expecting I could then analyse for movement/change and use the analysed events to synch the music.
I used The Game of LIfe, Gray-Scott diffusion modelling and Mandlebrots to mix and merge images. Many of the results were quite beautiful - but I foud it impossible to extract a coherent time sequence from the videos. The audio and the visuals justnever seemed to belong together. Plus, my underlying drive was to create a systme where the composer collaborated with tthe pixel data to create the music - and these visual synthesis methods seemed too heavy handed, with the algorithm playing a greater role than the data in the music that got created.
I tried working with sprites that would allow individual pixels to blossom into shapes and colours that would take over areas of the image. It was easier to instill some rhythmic order, but the image itself was still not playing a strong enough role in directing the music. Changing images needs to have a significant effect on the output!
At the same time, I think I was too niave initially, in my understanding of 'music'. I thought that constraining major visual shifts to the beat of the tempo would be sufficient, but it isn't. Some of the tracks are (I think) really interesting soundscapes, and they all meet that brief of being 'an immersive trance experience' - but I needed to go much further to be ablel to say that I am synthesising music from images.
I really like the timbral quality of these early tracks, which has been created by directly generating waveforms for the notes - in fact writing the synthesiser component was extremely rewarding. But it really felt like re-inventing the wheel, and I doubt one guy can create a software sythesiser ina realistic timeframe, especially with all the other components needed. And I want as wide a create expression as possible, so one synth will never be enough. So utlimately I switched to generating midi.