Message Boards Message Boards

Is a list of 40000 english words too big for Wolfram Dev. Plat. Free Plan?

Posted 7 years ago

I just started learn Wolfram Language on Wolfram Development Platform Free Plan. I tried to count number of syllables in English words. Fortunately WordList[] gives 40127 common english words.

What I want to do in one line is

Counts[Map[ WordData[#, "Hyphenation" ]&, WordList[] ] //Flatten ]//WordCloud

It didn't work. Same task written in step by step is

WordData["syllabicate", "Hyphenation"]
words =  WordList[]
syllables = Map[ WordData[#, "Hyphenation" ]&, words ]//Flatten
counts = Counts[syllables] //Sort//Reverse
WordCloud[counts]

Map function (3rd line) didn't return any result. I don't think 40000 words is too big for Wolfram Language. Are there something wrong on my code? Or free plan isn't enough to do this?

POSTED BY: Roman SUZUKI
5 Replies
Posted 7 years ago

Thank you. It looks have long tail.

POSTED BY: Roman SUZUKI

Here is a log-log plot thereof:

ListLogLogPlot[(Reverse@SortBy[Normal[wordcounts] /. Rule -> List, Last])[[2 ;;, 2]], FrameLabel -> {"# rank", "# occurences"}, 
LabelStyle -> Directive[Bold, 18], PlotTheme -> "Marketing"]

enter image description here

Cheers,

Marco

POSTED BY: Marco Thiel
Posted 7 years ago

Thank you for trying. I don't have Mathematica and running macOS 10.12.6. A reason I called Counts[] is that I want not only draw WordCloud but also to find what syllables are major in many ways. By drawing BarChart or something else.

I will try split a result of WordList[].

Thanks.

POSTED BY: Roman SUZUKI

Dear Roman,

all this is really simple in Mathematica.

wordcounts = Flatten[WordData[#, "Hyphenation"] & /@ WordList[]] // Counts

and then

BarChart[#[[All, 2]], ChartLabels -> (Rotate[Style[#[[1]], 15], Pi/2] & /@ (Reverse@SortBy[Normal[wordcounts] /. Rule -> List, Last])[[2 ;; 52]]), ImageSize -> 650] &@(Reverse@SortBy[Normal[wordcounts] /. Rule -> List, Last])[[2 ;; 52]]

enter image description here

Cheers,

Marco

POSTED BY: Marco Thiel

Hi,

could you elaborate a bit on this? I am on OSX 11.12.6 and MMA 11.2 and it appears to work just fine:

Flatten[WordData[#, "Hyphenation"] & /@ WordList[]] // Counts

enter image description here

and

WordCloud[Flatten[WordData[#, "Hyphenation"] & /@ WordList[]]]

enter image description here

It also runs quite fast - about 10 seconds. I am running it in the Cloud right now. I don't have the free account, but a standard one. I'll let you know.

UPDATE: This is still running. I can see that on the free version it might not work. I think that in particular the WordData for the hyphenation is slow. You can still split the words into smaller groups (downloading all words is quite fast) and so split the computations into shorter bits.

UPDATE 2: It actually did not finish in the cloud even on paid-for accounts (standard Mathematica Online and Producer Development Cloud).

Cheers,

Marco

POSTED BY: Marco Thiel
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract