GPU / Shader based sound effects and (re-)synthesis

There is massive computing power available these days in modern computers: the GPU provided by graphics cards is super powerful. It is multitudinous faster than the CPU – yet we still mostly rely on the CPU for processing audio effects.

What if we could have a bunch of nodes that utilize this massive amount of unused computing power for audio processing?

Now that we already have shaders in MNodes – it’s not far-fetched to think about the GPU being used as an audio processor! You could stream audio buffers into textures and use shaders to compute extremely precise FFT, and then use more shaders to manipulate a sound in ways previously impossible because of the sheer amount of processing power you have in a GPU.

You could bend and twist sound way more fluidly than you ever could on a CPU! When cleverly used, a user could distort sounds with graphical tools and never even write a line of code. MNodes would transform the involved parameters into shader parameters, and under the hood, the GPU would compute insane effects not possible otherwise. Going further, the user could also enrich this experience with their own shaders that manipulate sound in ways previously unthinkable, just because a GPU is so much more powerful than a CPU.

I don’t know about involved latency regarding streaming audio to a texture, computing stuff, and then sending the audio back again to RAM to process it with other nodes. But I think it’s a domain we should think about, and if turnaround times are acceptable, this would open a whole new galaxy of possibilities!

Especially spectral effects would benefit a lot from the additional precision that you could reach with that much computing power at hand. Image processing techniques could transform images to sound and animate them to produce unheard of effects.

I believe it is a domain we should tap into to see where this leads us! :slightly_smiling_face:

not everybody has a top notch gpu tho and with todays prices yeah

Don’t worry! Even the weakest GPU - which exists inside almost any system these days - is much more powerful than the CPU in that system. So no need for an expensive card!

MNodes already uses shaders for its Scene nodes. So we are already headed in that direction.

And it’s an option after all. I do not request MNodes to be based on that principle! :slight_smile:

The future is bright. :sun:

(And yes, the current GPU market is a bad joke…)