Group Abstract Group Abstract

Message Boards Message Boards

Using Pseudo-Hilbert Curves to Assist People Perceive Images via Sound

Posted 5 years ago
10 Replies
Posted 5 years ago

I was wondering if there is a way I could do this process in reverse using machine learning to simulate a human learning the association between sound and pixel color to create images from audio.

POSTED BY: Srinath Rangan

enter image description here - Congratulations! This post is now featured in our Staff Pick column as distinguished by a badge on your profile of a Featured Contributor! Thank you, keep it coming, and consider contributing your work to the The Notebook Archive!

POSTED BY: EDITORIAL BOARD
Posted 5 years ago

Does anyone know if there is a way to find the relative SoundVolume of a sound clip at a specific point? This would help me create a backwards algorithm that can convert the sounds back to images.

POSTED BY: Srinath Rangan
Posted 5 years ago

Does anyone know how to do this? Without it, I have resorted to creating a massive association of HSB to audio files which is inefficient.

POSTED BY: Srinath Rangan

Nice post, Srinath. You can use AudioLoudness or AudioLocalMeasurements ("RMSAmplitude" property) to get a time series of loudness/amplitudes for an audio object.

POSTED BY: Shadi Ashnai
Posted 5 years ago

Hi Ms. Ashnai,

Thank you for your reply! I am trying to use AudioLoudness, but I am having a bit of trouble deciphering the scale of the y-axis. Is there a way to rescale the loudness values back to {0,1} to match SoundVolume?

POSTED BY: Srinath Rangan

AudioLoudness can compute different loudness definitions. Units are mentioned in the reference page.

"RMSAmplitude" gives root mean square of the values and for a typical audio data being in -1 to 1 range, will return values in the 0 to 1 range.

See this:

a = ExampleData[{"Audio", "DogBark"}];
Show[AudioPlot[a, PlotLayout -> "Averaged"], 
 ListLinePlot[AudioLocalMeasurements[a, "RMSAmplitude"], 
  PlotStyle -> Red]]

enter image description here

POSTED BY: Shadi Ashnai
Posted 5 years ago

Hello Ms. Ashnai,

In the below image, the SValues list represents the saturation values of pixels of the image (all ranging from 0 to 1). These values are what I used directly for SoundVolume values of the audio. As you can see from the AudioPlot, because as you said, RMSAmplitude is the root mean square, I am not able to get the peak values. Is there a way to read the loudness levels at the top of each bar (every 0.1 seconds) in the AudioPlot?

Thank you,

Srinath

RMSAmplitude vs. True Values

POSTED BY: Srinath Rangan

Try computing max instead of RMS amplitudes:

AudioLocalMeasurements[a, "Max"]

You can also control the granularity of the partitions using PartitionGranularity. e.g.:

AudioLocalMeasurements[a, "Max", PartitionGranularity -> .01]

POSTED BY: Shadi Ashnai
Posted 5 years ago

It's working now. Thank you!

POSTED BY: Updating Name
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard