How to draw waveform of current input via AudioIn ?

[dhowe]

How would one access the last k frames of input (the raw sample data) from an AudioIn object ?

[kevinstadler]

This would indeed be a cool feature -- it's not that straightforward to implement as the AudioIn's data is currently directly passed on to the filters and outputs without an explicit buffer that one could access. The solution for this would be to augment the AudioIn class with something like JSyn's FixedRateMonoReader to write into a (configurable) fixed-length buffer that we can then read from. The data read can then be used for visualisation purposes, or also passed on to sound output again, presumably by writing the read sound input frames to a previously allocated AudioSample.

[kevinstadler]

Online documentation will be added when the updated version of the library goes live later today, in the meantime you can already have a look at the example sketch here: https://github.com/processing/processing-sound/blob/master/examples/Analysis/AudioWaveform/AudioWaveform.pde
Note that you won't be able to execute this code until you've updated the library (or built it yourself from the latest repository version).

[dhowe]

Excellent, thanks.

And I assume you can get full resolution of the sample data, by specifying the full number of sample frames in the constructor for the waveform object, something like new Waveform(this, 44100 * sampleLengthInSeconds) ?

And will a new Waveform be required for every Sample object then? Or are they reusable?

[kevinstadler]

The data you read is always at the full resolution, i.e. at whatever sampling rate the library is running on. So even new Waveform(this, 22050); would work, the object would simply always return the last half second worth of samples.

The usage of Waveform will be completely parallel to that of the FFT class, so it can only analyse one input source at a time, but you can re-use the same Waveform object by swapping out the sound object that it's analysing.

[kevinstadler]

Note that the Waveform object will only return meaningful data for objects which are actually playing back. If you're simply interested in the content of an AudioSample/SoundFile object loaded previously, all of this information can already be retrieved using those classes' read() method: https://wiki.processing.org/reference/libraries/sound/AudioSample_read_.html

[dhowe]

So I think that the following code would give the raw data for a created sample (correct me if I'm wrong) :

sample = new AudioSample(this, 100000, 22050);
float[] sampleContent = new float[sample.frames()];
sample.read(sampleContent);

And the following for a loaded Sample:

 file = new SoundFile(this, "sample.mp3");
 float[] fileContent = new float[file.frames()];
 sample.read(fileContent);

While I can guess at the Waveform API from your comments above, I still don't see it in the docs or the javadocs. @kevinstadler, has there been some change to the roadmap ? thanks!

ps. I'm guessing the waveform/audioin would work like this?

  ain = new AudioIn(this, 0);
  ain.start();
  waveform = new Waveform(this, 44100);
  waveform.input(ain);
  float[] inContent = new float[44100];
  waveform.read(inContent);

[kevinstadler]

Almost, just the method name is slightly different, but try this:

int windowSize = 12345;
ain = new AudioIn(this, 0);
ain.start();
waveform = new Waveform(this, windowSize);
waveform.input(ain);
float[] inContent = waveform.analyze();

You are right that the returned float[] array has as many elements as the number of samples that the Waveform is initialized with (12345 in my code). The class has its own internal waveform.data array though which has exactly the right size, so you don't necessarily have to create your own one.

Hope that helps!

[dhowe]

great, thanks much!

[dhowe]

actually, what works for me (in case anyone else refers to this ticket) is:

int windowSize = 12345;
ain = new AudioIn(this, 0);
ain.start();
waveform = new Waveform(this, windowSize);
waveform.input(ain);
waveform.analyze();
float[] inContent = waveform.data;