Ways of Converting Video into Audio

[Salvatorin]

I'm doing a lot of experimental audio/visual stuff lately and I find myself wanting to hear what the images 'sound' like. I'm not a very educated person with software, but I suppose you could assign frequencies to the different cymk channels or generate audio from the codec info.

 

ANyone know anything about this?

[KovalainenFanBoy]

Audacity > File > Import > Raw data > pick a video/image

 

What you're refering to is usually refered to as "sonification" and it's usually done in order to manipulate the images themselves, (here's a gif I made using only Audacity), as in, importing a track into Audacity, applying reverb/echo/EQ/pretty much anything to the resulting waveform, and re-exporting it as an image again, as the results are much more interesting than the sounds (mostly random noise). There's interesting rythmic patterns to be found sometimes in the data, so if that's your thing go for it. But you may as well just feed random files to the audio card, the results are gonna be pretty much the same whether you use jpeg, pdf, exe or whatever

 

There may be more interesting ways of translating images to sound, but if you take it literally this is pretty much all there is to it

[awepittance]

I'm doing a lot of experimental audio/visual stuff lately and I find myself wanting to hear what the images 'sound' like. I'm not a very educated person with software, but I suppose you could assign frequencies to the different cymk channels or generate audio from the codec info.

 

ANyone know anything about this?

I did a mod on the Boxus patch in jitter that plays a quicktime file's images as an audio spectrum. I can send it if you want. IT was originally meant to convert audio to a jitter spectrum and then let you manipulate the audio using video effects. Unfortunately it still relies on a 15+ year old FFT algorithm that sounds pretty 'samey' unless you dial it in pretty specifically, just takes some practice to find the sweatspots

[TubularCorporation]

EDIT: Posted to quick, Ithis isn't really relevant to what Salvatorin is looking for but I'm leaving it in case anyone else interested in processing video with audio effects in hardware wants to run with it. Also, the stuff at the end about Pixivisor is something you might want to check out, Salvatorin! It's not as precise as the stuff other people already described, but it's a different way to convert picture to audio and back that you might like. Virtua ANS (on the same site) would probably be of interest, too, and both are free (VANS is a few dllars for the iOs/Android versions).



Last fall I was messing a bit with running the color channels of RGBHV analog video (i.e. VGA, the only difference is the cable. a VGA to RGBHV breakout cable is pretty cheap and gives you the sync and color channels on separate BNC connections to play with) and it worked well when you passed the results straight to a VGA monitor, but I haven't yet found a VGA to composite video converter that will accept the signal - every one I've tried will blank the picture as soon as things get too glitchy. What you probably need is a timebase corrector that will accept RGBHV video input, but the one I have only takes component and composite. The TBC should take your mangled signals and reconstruct them into something that the capture device will recognize - it works great for capturing VHS tapes that are damaged - without it the capture device will drop a lot of frames or reject the signal altogether, but with the TBC pached in even tapes that hardly have any picture left on them will capture properly. A professional, broadcast quality timebase converter costs next to nothing these days since everyone has moved to digital for video production work. I paid $50usd for mine a year ago on eBay (the MSRP in the mid 2000's was over $5000 for that model) but that was before I knew I'd need to deal with RGBHV.

 

The beauty of using VGA/RGBHV is that the sync signals are on separate wires from the color signals, so you don't need to worry about them at all, and can just treat the color channels as audio, more or less. The gain is a bit different but it's close enough. I've found digital delays, clean bandpass filters and analog chorus the msot interesting; subtle settings that you would hardly notice in a mix usually work best so far.

 

PixiVisor is pretty fun, too. Complex images don't handle processing that well, they mostly turn into an indecipherable mess, but with simpe things like geometric shapes and text you can get away with more. I've had luck converting stills and short loops of simple shapes into GIFs, converting them to audio clips with PixiVisor, and then loading them into the MPC and playing them back into PixiVisor to convert them back to video.

[Danny O Flannagin]

Audacity > File > Import > Raw data > pick a video/image

 

What you're refering to is usually refered to as "sonification" and it's usually done in order to manipulate the images themselves, (here's a gif I made using only Audacity), as in, importing a track into Audacity, applying reverb/echo/EQ/pretty much anything to the resulting waveform, and re-exporting it as an image again, as the results are much more interesting than the sounds (mostly random noise). There's interesting rythmic patterns to be found sometimes in the data, so if that's your thing go for it. But you may as well just feed random files to the audio card, the results are gonna be pretty much the same whether you use jpeg, pdf, exe or whatever

 

There may be more interesting ways of translating images to sound, but if you take it literally this is pretty much all there is to it

That gif looks like an autechre beat

[logakght]

^dats tight

[granty]

This looks good: http://www.creativeapplications.net/android/phonopaper-for-iosandroid-graphical-sound-generator-and-writer/

 

 

[awepittance]

This looks good: http://www.creativeapplications.net/android/phonopaper-for-iosandroid-graphical-sound-generator-and-writer/

 

 

wow thats pretty fucking awesome, very similar concept to the one Boxus and I put together. The difference is this guy figured out how to print out the spectrums and play them back using an ipad(?). really impressive. Gallery exhibit idea: a room with walls covered in spectrum printouts, fill room with 50 people who have smartphones who can run the app, have them go at it, record in surround sound.

We tried using a webcam to playback printed up peices of paper, but the camera quality on these tablets actually makes it sound good. damn.

if you had a robot controlled arm you could actually have a tablet like this playback a whole premade song (and be able to adjust the tempo) line by line scrolling down a giant piece of paper