Group Abstract Group Abstract

Message Boards Message Boards

Get an updated value for WordFrequencyData?

POSTED BY: Claudio Chaib
3 Replies
Sort By:

I cannot help you further except noting that

  1. The original dataset is not newer than 2008 (the paper is from 2010)

  2. Mathematica's function WordFrequencyData considers only this data set. I suggest you read the documentation on WordFrequency WordData and related functions...

best

yehuda

POSTED BY: Yehuda Ben-Shimol

Hi Yehuda,

Thank you for taking the time to answer me. I ran your code and tried a few things here, but I couldn't make any progress. Unfortunately, I have no idea how to use it for WordFrequenceData or whether to use it in another command. Sorry if I still don't understand this type of command or what to do with it... I humbly ask, could you help me a little more to show how I use the result of this code in WordFrequenceData or similar resource?

I will be extremely grateful. Thank you very much.
POSTED BY: Claudio Chaib

There is additional functionality that allows you to analyze the raw text data As it is divided to 100 fragments you may use the following to download it as a first step and continue from there

 Table[URLDownloadSubmit[
  "http://commondatastorage.googleapis.com/books/syntactic-ngrams/eng/\
nodes." <> IntegerString[n, 10, 2] <> "-of-99.gz", 
  "~/Downloads/" <> IntegerString[n, 10, 2] <> ".gz", 
  HandlerFunctions -> <|"TaskFinished" -> Print|>], {n, 0, 98}]

best
POSTED BY: Yehuda Ben-Shimol
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard