Quantcast

Getting data from IFFT or PV_ Outside of Synthdef

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Getting data from IFFT or PV_ Outside of Synthdef

adamsamuel_
Hello all,

I've been trying to figure out if its possible to use the PV_ FFT processing Ugens on individual Patterns/arrays, but they're only intended to work on continuous signals inside of Synths, as far as I can tell. Is it possible to create a couple of buffers, and to morph between them as an offline process, like I'm attempting to do below for instance? I feel like I'm pretty much there, if only I could get a single resulting buffer or array out of the final PV_Morph or IFFT Ugens. I'm guessing Ugens need to run continuously, rather than one frame at a time as I'm trying to do, so maybe that's part of the problem?

I'd appreciate any and all help with this.
Thanks,
Adam

(
r = Buffer.alloc(s,64,1);
u = Buffer.alloc(s,64,1);
)

(
a = Pfin(64, Pseq([1, 2, 3], inf)).asStream.nextN(64);
t = FloatArray.newFrom(a);
c = Buffer.loadCollection(s, t, 1);
c.normalize;
c;
)

(
e = Pfin(64, Pseq([3, 2, 1, 0], inf)).asStream.nextN(64);
w = FloatArray.newFrom(e);
x = Buffer.loadCollection(s, w, 1);
x.normalize;
x;
)

(
d = FFT(r, c);
f = FFT(u, x);
m = PV_Morph(d, f, 0.5);
n = IFFT(m, 1, 0);
)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Getting data from IFFT or PV_ Outside of Synthdef

ddw_music
adamsamuel_ wrote
I've been trying to figure out if its possible to use the PV_ FFT processing Ugens on individual Patterns/arrays, but they're only intended to work on continuous signals inside of Synths, as far as I can tell. Is it possible to create a couple of buffers, and to morph between them as an offline process, like I'm attempting to do below for instance? I feel like I'm pretty much there, if only I could get a single resulting buffer or array out of the final PV_Morph or IFFT Ugens. I'm guessing Ugens need to run continuously, rather than one frame at a time as I'm trying to do, so maybe that's part of the problem?
This question has come up a few times lately, so, some notes.

A new FFT frame is calculated every frameSize * hop samples. This is likely to be too fast to get data back into the language reliably.

The largest FFT frame you can get back is 1024 samples. With the normal hop size of 0.5, that's one frame every 512 samples, or 11 ms at 44.1kHz. That will keep the language quite busy. Larger frames require a bit more complexity to retrieve the data (but you have more time between frames).

So real-time access to FFT data outside of a SynthDef is probably not realistic.

Non-realtime: If you're FFTing a known signal, like from a soundfile, you can read the time domain samples into the language and use Signal:fft. You're responsible for windowing each frame.

Otherwise, you could try PV_RecordBuf (from sc3-plugins) to save FFT frames, write that buffer to disk, and read the frames in the language with SoundFile. Caveat: PV_RecordBuf and PV_PlayBuf have been broken in Linux for some time, and we don't know if it's the record or the play unit causing the glitches. I can't guarantee success here, but it might work.

TL;DR what you're asking about isn't simple. There's no established procedure for it.

hjh
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Getting data from IFFT or PV_ Outside of Synthdef

ddw_music
ddw_music wrote
The largest FFT frame you can get back is 1024 samples.
Oh, I ended that sentence too early. Corrected:

The largest FFT frame you can get back *using one b_getn message* is 1024 samples.

Bigger frames require multiple requests.

hjh
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Getting data from IFFT or PV_ Outside of Synthdef

Daniel Mayer
In reply to this post by adamsamuel_

Am 19.03.2017 um 05:03 schrieb adamsamuel_ <[hidden email]>:

> Hello all,
>
> I've been trying to figure out if its possible to use the PV_ FFT processing
> Ugens on individual Patterns/arrays, but they're only intended to work on
> continuous signals inside of Synths, as far as I can tell. Is it possible to
> create a couple of buffers, and to morph between them as an offline process,
> like I'm attempting to do below for instance? I feel like I'm pretty much
> there, if only I could get a single resulting buffer or array out of the
> final PV_Morph or IFFT Ugens. I'm guessing Ugens need to run continuously,
> rather than one frame at a time as I'm trying to do, so maybe that's part of
> the problem?
>

Hello,

I'm not quite clear about what you exactly want to achieve.

1.) Is it you want to synthesize spectra from Arrays/Patterns and morph
between them ? Wouldn't additive synthesis be a more straight option then ?

2.) Influence / control the FFT process by patterns. This works straight,
many variants possible, e.g. also with sending arrays

(
p = Platform.resourceDir +/+ "sounds/a11wlk01.wav";
b = Buffer.read(s, p);
)

(
SynthDef(\bin_scramble, { |wipe = 0.1, width = 0.2, amp = 1,
        shift = 0, stretch = 1|
        var in, chainL, chainR, chain;
    in = PlayBuf.ar(1, b, BufRateScale.kr(b), doneAction: 2);
        chain = FFT({ LocalBuf(2048) } ! 2, in);
        chain = PV_MagShift(chain, stretch, shift);
        Out.ar(0, IFFT(chain) * amp);
}).add;
)


(
Pfindur(b.duration,
        Pmono(
                \bin_scramble_1,
                \dur, 0.02,
                \trig, 0,
                \shift, Pseg(Pwhite(-5, -10), Pwhite(0.1, 0.3))
        )
).play
)

3.) Work on FFT data language-side, possibly in realtime. As James already mentioned, this is hard,
though there are ways. I'd only consider that if you have a strong reason to do so.


Regards

Daniel

-----------------------------
www.daniel-mayer.at
-----------------------------









_______________________________________________
sc-users mailing list

info (subscription, etc.): http://www.birmingham.ac.uk/facilities/ea-studios/research/supercollider/mailinglist.aspx
archive: http://www.listarc.bham.ac.uk/marchives/sc-users/
search: http://www.listarc.bham.ac.uk/lists/sc-users/search/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Getting data from IFFT or PV_ Outside of Synthdef

adamsamuel_
Hi Daniel and James,

Thanks for both of your responses, apologies about my question being unclear, I'm only just really getting to grips with programming in Supercollider.

It'd be easier to explain if I put the problem into context:

I've imagined having a small array of floats, something like [1, 0, 0.5, 1, 0.25, 1, 0, 0], and a rhythm generator getting and using this array's values for its loudnesses - durations being equal the first hit gets array[0], the second gets Array[1], etc. Obviously this is what Supercollider's pattern framework is for, but instead of using that, I thought it might be interesting and possibly musically fruitful to treat an array as if it's a very small buffer that might contain a tiny bit of audio, still using its values for the rhythm, but applying FFT/audio types of effects to the Array-buffer to hear how the result of doing so affects the rhythm using it to get its loudness values from. A simple example would be starting with two arrays like above, and using the PV_XFade ugen to interpolate between the two arrays as audio buffers, so that the rhythm using the result interpolates between two rhythmic configurations too, but other FFT/audio ugens would be interesting to experiment with in this way too I think - unless there's something fundamental I'm misunderstanding, I think this is possible.

If I can process an array, as a buffer, through an FFT/audio effect, offline, and get it back from the server, I'd pretty much be there. This doesn't need to happen real time, because the array-buffers wont ever actually be heard themselves as we're only using their data for the rhythm to get its loudnesses from. They only need to be processed on demand. I might process the array-buffer every time I change some variable, for instance.

I hope that was a bit clearer. From your 3 options, it's number 3, but without the real-time so hopefully that makes things easier. I only need to process one Array-buffer on demand and get it back into the language for something else to make use of.

Thanks again,
Adam

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Getting data from IFFT or PV_ Outside of Synthdef

Daniel Mayer

Am 19.03.2017 um 21:18 schrieb adamsamuel_ <[hidden email]>:

I hope that was a bit clearer. From your 3 options, it's number 3, but
without the real-time so hopefully that makes things easier. I only need to
process one Array-buffer on demand and get it back into the language for
something else to make use of.

Thanks again,
Adam


I see, it might be interesting to explore crossing categories, why not.
All possible in principle and not too difficult. One open question would be 
the kind of remapping from (in general bipolar) audio to rhythm.

If blending rhythms is the main purpose there might be other easier options



But if you want to try out audio processing and reprocessing as you described,
have a look into NRT synthesis.
You could write a Function that processes the rhythm/audio and writes to and reads from a buffer/disk.
the resulting "rhythm" could then immediately be fed into a running proxy pattern (like Pbindef)

As the audio files are short this operation can happen within fractions of a second, thus
would feel almost *like realtime*.

Regards

Daniel

-----------------------------
www.daniel-mayer.at
-----------------------------
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Getting data from IFFT or PV_ Outside of Synthdef

ddw_music
In reply to this post by adamsamuel_
adamsamuel_ wrote
I've imagined having a small array of floats, something like [1, 0, 0.5, 1, 0.25, 1, 0, 0], and a rhythm generator getting and using this array's values for its loudnesses - durations being equal the first hit gets array[0], the second gets Array[1], etc. Obviously this is what Supercollider's pattern framework is for, but instead of using that, I thought it might be interesting and possibly musically fruitful to treat an array as if it's a very small buffer that might contain a tiny bit of audio, still using its values for the rhythm, but applying FFT/audio types of effects to the Array-buffer to hear how the result of doing so affects the rhythm using it to get its loudness values from. A simple example would be starting with two arrays like above, and using the PV_XFade ugen to interpolate between the two arrays as audio buffers, so that the rhythm using the result interpolates between two rhythmic configurations too, but other FFT/audio ugens would be interesting to experiment with in this way too I think - unless there's something fundamental I'm misunderstanding, I think this is possible.
Here's an example of client-side FFT for this purpose, avoiding the server altogether:

// client-side FFT requires a Signal
a = Signal[1, 0, 0.5, 1, 0.25, 1, 0, 0];

// returns an FFT in Cartesian coordinates
f = a.fft(Signal.newClear(a.size), Signal.fftCosTable(a.size));

// Polar coordinates are generally more useful for manipulation
f = f.asPolar;

// A full FFT reflects bins around Nyquist:
// DC, bin1, bin2 ... bin(size/2) ... bin1
// Dropping the last half makes it easier to mess around
f = Polar(f.rho[0 .. a.size div: 2], f.theta[0 .. a.size div: 2]);

// Simulate PV_BrickWall: zero out the top 2 bins
f = Polar(f.rho.copy[3..4] = 0, f.theta.copy[3..4] = 0);

// Reassemble full FFT (with reflection)
f = Polar(
        (f.rho ++ f.rho[a.size div: 2 - 1 .. 1]).as(Signal),
        (f.theta ++ f.theta[a.size div: 2 - 1 .. 1]).as(Signal)
);

// Convert back to Cartesian coordinates
f = f.asComplex;

// IFFT
b = f.real.ifft(f.imag, Signal.fftCosTable(a.size));

// Real component is your modified signal
b.real

[a.as(Array), b.real.as(Array)].lace(a.size * 2).plot(numChannels: 2);

hjh
Loading...