Message Boards Message Boards

Hiding secret messages in music

The new computational audio features of Mathematica are really impressive. In no time you can cook up things as soon as you understand the basic algorithm. Even the simplest tricks can give quite surprising results. Before you read on, please do play this (file also uploaded to this post):

audio file.

Have you noticed anything? I think that it sounds pretty much like the well known example

ExampleData[{"Sound", "Apollo11SmallStep"}]

As a matter of fact, it contains something a little special. If you import the file into Mathematica

secsound = Import["/Users/thiel/Desktop/secretsound.wav"];

and calculate a spectrogram of it

Spectrogram[secsound, ColorFunction -> "Rainbow", Frame -> None, ImageSize -> Full]

you get this:

enter image description here

How to inject images into spectrograms

I will show now, how you can get this in a couple of lines of code. First we need to get a binary matrix (later we can do better!) of the message. This is quite straight forward:

imgdata = Reverse@ImageData[ColorNegate@Image[Binarize[Rasterize[Text["  Wolfram  "], RasterSize -> 100, ImageSize -> 100]]]];

There is nothing really tricky in here. The text is "Wolfram", I rasterise the image and resize it. Then I binarise it. The ColorNegate is needed to exchange ones and zeros in the matrix. Because I play with images and matrices, and they have a different coordinate system (the origin is at different corners) I need to Reverse the whole thing. I can plot the result like so:


enter image description here

I now need to generate a sound that produces high amplitudes at the right places in the Spectrogram. At each and every place in the imagedata matrix that contains a one I need to proceed a little sine wave with the right frequency. The rows correspond to frequencies and the rows to time. So lets collect the different frequencies at each time:

list = Flatten[Position[#, 1]] & /@ Transpose[imgdata];

Ok. Next we generate the corresponding sums of Sin-functions and generate a list of data from them.

listcompete = 0.1*Flatten[Table[Table[N@Total[Sin[2 Pi 300  # t] &@(# & /@ list)[[k]]], {t, 0, 8.71/200., 1/16000}], {k, 1, 100}], 1];

We can generate the corresponding sound:

Audio[Sound[SampledSoundList[listcompete, 8000]]]

This gives you a little window like this

enter image description here

which allows you to "listen to Wolfram in dolphin language". The file is attached to this post and sounds really cool.

How to hide the images in sound files/music

In order to hide the string in the "A small step ..." recording, we first shift the frequencies a little bit and decrease the amplitude:

listcompete2 = 0.02*Flatten[Table[Table[N@Total[Sin[2 Pi 1200  # t] &@(# & /@ list)[[k]]], {t, 0, 8.71/100., 1/64000}], {k, 1, 100}], 1];

It looks now like this:

Spectrogram[Sound[SampledSoundList[listcompete2, 64000]], ColorFunction -> "Rainbow"]

enter image description here

If you want to listen to it use:

Audio[Sound[SampledSoundList[listcompete2, 64000]]]

You should hear a sort of hissing sound. The new function AudioChannelCombine will now help us to merge the two sound objects:

AudioChannelCombine[{Audio[ExampleData[{"Sound", "Apollo11SmallStep"}]], Audio[Sound[SampledSoundList[listcompete2, 64000]]]}]

The resulting object is attached to this post. The spectrogram

Spectrogram[AudioChannelCombine[{Audio[ExampleData[{"Sound", "Apollo11SmallStep"}]], 
Audio[Sound[SampledSoundList[listcompete2, 64000]]]}], ColorFunction -> "Rainbow", Frame -> None]

enter image description here

clearly shows the secret message blended into the Apollo message.

Like this you can export the sound object:

secsound = Import["/Users/thiel/Desktop/secretsound.wav"];

Look for hidden messages

This type of procedure is quite well known. I also saw it on BBC's Click, but I cannot remember the episode. Now you can use this to look for hidden messages in the internet or in music. Here are two examples: file1 and file2, both from the website above.

Let's look at file1

Spectrogram[snd, 150, Frame -> None]

enter image description here

and file2

Spectrogram[Import[""], 150, Frame -> None, AspectRatio -> 1]

enter image description here

You can clearly see that the second image is actually "grayscale". It is quite possible to achieve this by not Binarize-ing the image and using the grayscale values as amplitudes.

Where to go from here?

We could certainly try to increase the resolution and get everything into a nice little function to do all the steps for us. Another interesting thing would be to add multiple images/slices of a 3 D image into sound, and basically hide an entire 3D object. I'd love to see this being 3D printed.

Can you detect the message in this sound file?



POSTED BY: Marco Thiel
9 Replies

Of course, this approach is rather out-dated now. I posted it 4 years ago. The Wolfram Language advances at an incredible speed. Here is a newer version of this:

img = ExampleData[{"TestImage", "Mandrill"}]

enter image description here

Then you can run:

audio = InverseSpectrogram[img] // Audio

which gives you the audio signal. Then you simply calculate the Spectorgram:

Spectrogram[audio, Frame -> False, AspectRatio -> 1, PlotRange -> All, ColorFunction -> "Rainbow"]

enter image description here



POSTED BY: Marco Thiel

Is it safe to listen to, though ? For the ears, I mean.

I ask that because I once messed around with sound generation and I ended up creating very high pitched sines that hurt my ears. Like seriously hurt. Ever since, I'm kind of scared of computer-generated sounds.

POSTED BY: Lucien Grondin

Is there any way to get the Spectrogram function to display the Y axis (frequency) on a logarithmic scale? Some examples of images embedded in audio files, e.g. from Aphex Twin's Windowlicker single, display properly such a spectrogram.



POSTED BY: Christopher Fox

Great post. I can use this technique to turn a graphic into music. This idea has been used before, but it was mainly an update to a player piano roll. With an appropriate settings for rasterize and a little better tone generator, I think this can be made to render graphics into electronic music. It will be nice to be free of the confines of equal temperament and 'fake' natural instruments.

I remember seeing drawings by Edgard Varèse that he made as studies for his poeme electronique. It will be interesting to turn the picture into sound. It won't sound like the piece, of course.

Also reminds me of:

where there was also codes hidden in the music...

POSTED BY: Sander Huisman

Very cool! It has been used by various musicians:

Thanks for sharing!

POSTED BY: Sander Huisman

enter image description here - another post of yours has been selected for the Staff Picks group, congratulations !

We are happy to see you at the top of the "Featured Contributor" board. Thank you for your wonderful contributions, and please keep them coming!

POSTED BY: Moderation Team

Neat idea, Marco! By the way, there is a nice sharing service for for audio snippets with no account registration. I have uploaded your attachments there.

POSTED BY: Vitaliy Kaurov

Dear Vitaliy,

thanks a lot for that hint! That website will be very, very useful for posts about the new computational audio features. That will facilitate future posts greatly. Thanks for sharing.



POSTED BY: Marco Thiel
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
or Discard

Group Abstract Group Abstract