Spam postings in non-English - and how to filter them out.

Meta
1

Hi everyone,

quite regularly, one finds postings in non-English Languages in this community. Often there are dozens of them that clog up the first pages of this community. It loos like this:

ScreenShot2015-10-05at13.30.53.png
ScreenShot2015-10-05at13.30.53.png1061×744 134 KB

It would be kind of cool if these postings were filtered out, by the system. This is quite straight forward with the Wolfram language. This is a very simple approach (when you run this, the moderation team might have removed the Korean post, so this might not work anymore):

Classify["Language", #] & @ StringSplit[StringSplit[Import[" http://community.wolfram.com/groups/-/m/t/575914 ", "Plaintext"], "GROUPS"][[2]], "POSTED BY: "][[1]]

which gives:

ScreenShot2015-10-05at13.35.53.png

and

Classify["Language", #] & @StringSplit[StringSplit[Import[" https://wolfram.discourse.group/t/6841/1 ", "Plaintext"], "GROUPS"][[2]], "POSTED BY: "][[1]]

which gives

ScreenShot2015-10-05at13.35.59.png

Of course, it is quite straight forward to train a neuronal network just for this. Interestingly,

Classify["Spam", #]

does not appear to work; probably it is trained on English spam. The community also has an RSS feed, so it is quite easy to use Mathematica to check every couple of minutes where one of these spam attacks is taking place.

Cheers,

Marco

4 Likes
2

This is a great idea to automate spam deletion via WL tools. We are already considering various mechanisms and will add this points to the discussion. Thanks Marco!