Message Boards Message Boards

noise removal question (when ground truth distribution is known)

Posted 9 years ago

hello, i'm trying to denoise EEG data acquired inside an MRI scanner, which is quite the hostile environment for the EEG signal. the details are not important, but i have the following question:

assume i have 2 signals, signal A and signal B. signal A was acquired in a noise free environment (ie, outside the scanner) while signal B was acquired in a noisy environment (ie, inside the scanner) but both signals are coming from the same source (ie, my brain). my question is: is there a way to use the information in the clean signal (signal A) to remove the noise from signal B? thanks, Russell

POSTED BY: Russell Butler
11 Replies

I think this is where a Wiener Filter might work. In essence you want to take advantage of the fact that the cross-correlation between your EEG signal and the MRI noise is very small. You also know the auto-correlation of of an EEG signal without the MRI which could be helpful. From that info you should be able to construct a powerful filter to get rid of the MRI noise. Here are some references: http://mathworld.wolfram.com/WienerFilter.html
http://www.owlnet.rice.edu/~elec539/Projects99/BACH/proj2/wiener.html

POSTED BY: Kay Herbert

ListPlot of the absolute value of Fourier on the two signals shows some regularity for the "good" signal that is not so obvious in the bad one. So a possibility is as follows. Important: This relies on them having the same length. Else the idea can be made to work but will require some adjustments.

Take the Fourier transform of the good signal. Construct a mask for frequencies that have large amplitudes. Apply this mask on the FT of the noisy signal. Use the IFT of that masked signal.

Here is code I used. I first downloaded the files to my /tmp directory.

iin = Import["/tmp/inside.txt", "CSV"];
sigin = iin[[1]];

fin = Fourier[sigin];
fin2 = Map[If[Abs[#] > 40, #, 0] &, fin];
mask = Sign[Abs[fin2]];

iout = Import["/tmp/outside.txt", "CSV"];
sigout = iout[[1]];

fout = Fourier[sigout];
fout2 = mask*sigout;
out2 = Re[InverseFourier[fout2]];

I don't know if this will do what you want but it might give some ideas.

POSTED BY: Daniel Lichtblau
Posted 9 years ago

Yes -- I can see the problem. ;-)

Perhaps some algorithm could be used to describe an artifact in terms of a few detectable parameters, like the positions and amplitudes of the positive and negative peaks which are clustered closely enough to be assigned to an individual artifact, then construct the assumed artifact, and subtract it out. But assuring that what's left is really the signal seems a bit of a stretch.

I'm still hoping for a few really brave souls to join this discussion!

POSTED BY: David Keith
Posted 9 years ago

yeah, pretty much. thats probably the reason there's over 50 articles published on it and no adequate solution in the past 15 years, haha

POSTED BY: Russell Butler
Posted 9 years ago

You mean that the red line is representative of the signal only, and that the blue line shows the signal with added noise? And the noise is much larger than the signal? And the noise occupies the same frequency band? And is not reproducible from run to run?

enter image description here

POSTED BY: David Keith
Posted 9 years ago

i'm attaching 3 files,

outside.txt => signal only (time series) inside.txt => signal + noise (time series, same electrode as outside.txt) artifact.pdf => qualitative description

keep in mind that inside.txt and outside.txt are not time-aligned, ie, they were acquired at different times.

by distribution i just mean the histogram

thanks!

Attachments:
POSTED BY: Russell Butler
Posted 9 years ago

Do you have any example data you can share? Useful would be signal+noise, signal only, and noise only.

What do you mean by the "distribution" of a signal?

POSTED BY: David Keith
Posted 9 years ago

a notch filter won't work. the noise band completely overlaps my signal's band. we do use a dedicated EKG channel to capture the heart beats, but that is just to pinpoint the onset of each beat so we can align them to compute the average for artifact average subtraction...this is the standard method in the field, but gives quite poor results.
subtraction won't work because they are two different time series'.

i was hoping some type of blind source separation or machine learning could help here...based on the distribution of the "clean" signal, it should be possible to somehow modify the noisy signal so that its distribution resembles that of the clean one.

oh and i tried the wiener filter, didn't work :(

POSTED BY: Russell Butler
Posted 9 years ago

hey Dave, thanks for the reply but unfortunately the noise is in the same band as my signal...

POSTED BY: Russell Butler
Posted 9 years ago

We need to be able to either distinguish the noise from the signal, for example by frequency band, or somehow construct the noise signal so it can be subtracted from the total.

If the noise occupies narrow bands, even within the signal band, it may be possible to notch it out while still leaving a useful signal.

If that isn't the case, can the noise waveform itself be determined? For example, in the instrumentation is it possible in a second channel to capture the noise without the signal. If this is possible, then one might subtract it from the total, with an amplitude such as to minimize the resulting signal energy.

If that is not possible, but the noise signal is a reproducible waveform, perhaps it can be determined, up to some amplitude and phase, by subtracting your available signal from the signal+noise you have, and then use the same method.

But if the noise is not reproducible, and is spread over the frequency band of the signal, I do not see how they can be distinguished.

Maybe some of the signal processing experts here could chime in?

Best regards, David

POSTED BY: David Keith
Posted 9 years ago

If a significant fraction of the noise is in a frequency band that can be neglected, you might use the two signals to determine a useful filter. Do a Fourier or Peridodogram on each, and use the information to design a filter which improves the signal to noise ratio of the noisy signal.

POSTED BY: David Keith
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract