ListPlot of the absolute value of Fourier on the two signals shows some regularity for the "good" signal that is not so obvious in the bad one. So a possibility is as follows. Important: This relies on them having the same length. Else the idea can be made to work but will require some adjustments.

Take the Fourier transform of the good signal. Construct a mask for frequencies that have large amplitudes. Apply this mask on the FT of the noisy signal. Use the IFT of that masked signal.

Here is code I used. I first downloaded the files to my /tmp directory.

iin = Import["/tmp/inside.txt", "CSV"];
sigin = iin[[1]];

fin = Fourier[sigin];
fin2 = Map[If[Abs[#] > 40, #, 0] &, fin];
mask = Sign[Abs[fin2]];

iout = Import["/tmp/outside.txt", "CSV"];
sigout = iout[[1]];

fout = Fourier[sigout];
fout2 = mask*sigout;
out2 = Re[InverseFourier[fout2]];

I don't know if this will do what you want but it might give some ideas.

yeah, pretty much. thats probably the reason there's over 50 articles published on it and no adequate solution in the past 15 years, haha

i'm attaching 3 files,

outside.txt => signal only (time series) inside.txt => signal + noise (time series, same electrode as outside.txt) artifact.pdf => qualitative description

keep in mind that inside.txt and outside.txt are not time-aligned, ie, they were acquired at different times.

by distribution i just mean the histogram

thanks!

Attachments:

a notch filter won't work. the noise band completely overlaps my signal's band. we do use a dedicated EKG channel to capture the heart beats, but that is just to pinpoint the onset of each beat so we can align them to compute the average for artifact average subtraction...this is the standard method in the field, but gives quite poor results.
subtraction won't work because they are two different time series'.

i was hoping some type of blind source separation or machine learning could help here...based on the distribution of the "clean" signal, it should be possible to somehow modify the noisy signal so that its distribution resembles that of the clean one.

oh and i tried the wiener filter, didn't work :(

We need to be able to either distinguish the noise from the signal, for example by frequency band, or somehow construct the noise signal so it can be subtracted from the total.

If the noise occupies narrow bands, even within the signal band, it may be possible to notch it out while still leaving a useful signal.

If that isn't the case, can the noise waveform itself be determined? For example, in the instrumentation is it possible in a second channel to capture the noise without the signal. If this is possible, then one might subtract it from the total, with an amplitude such as to minimize the resulting signal energy.

If that is not possible, but the noise signal is a reproducible waveform, perhaps it can be determined, up to some amplitude and phase, by subtracting your available signal from the signal+noise you have, and then use the same method.

But if the noise is not reproducible, and is spread over the frequency band of the signal, I do not see how they can be distinguished.

Maybe some of the signal processing experts here could chime in?

Best regards, David