Message Boards Message Boards

0
|
10309 Views
|
2 Replies
|
4 Total Likes
View groups...
Share
Share this post:

How does one extract "Word Frequency History"?

Posted 10 years ago

Hi - Again a warning: Beginner.

I would like to accomplish two goals:

  1. Get a clear example of how to get the Word Frequency History for a particular list of words (the words come from a list) for a range of dates ? The output would be the data. I do not want the pod from Wolfram|Alpha. Just the data for analysis. I have gone though the "People and History Page" where one is directed to the WordData function.
  2. Is it possible create a function that generates x words with a positive slope (according to frequency) and another with a negative slope... Basically a measure of relevancy?

Lastly, it was not particularly clear to me (after some research) where the word frequency data comes from. The definitions I know come from wordnet.

Thank you in advance.

POSTED BY: Itay Livni
2 Replies
Posted 10 years ago

Thanks Kyle - This is a great answer although I was naively hoping to keep everything in Mathematica :)

Word Frequencies in Written and Spoken English: Based on the British National Corpus. Pearson ESL, 2001

This is interesting because the Wolfram|Alpha Data goes to 2007'sh and can be downloaded.

Alternatively, you can also download Google’s ngrams datasets to gain information about word frequency...

The ngram datasets are something I looked at earlier, however:

  1. I could not verify the data against Wolfram|Alpha's
  2. Some quick sanity tests did not pass (not for this forum)

I was very much led astray by a crumb in the People & History Reference Guide

Very helpful!

POSTED BY: Itay Livni
POSTED BY: Kyle Keane
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract