Hi Kacie,

Your question seems to be unrelated to this thread which is more about sentiment analysis. You should probably ask a new question.

Anyway, the structure and format of the text of Hamlet that you are using will dictate how to proceed with the analysis. Take a look at my answer to this question. It should help you get started. If you get stuck, ask a new question and include the code you have tried, and explain the problem you have.

Again, thank you to all for the information.

For the letters, I copy and paste the transcription into MS Word so I can edit them. This means deleting the annotation references, poetry, and long quotations. The poetry and quotations could affect the sentiment analysis. Then I copy and paste the edited transcript into a Mathematica notebook.

I have a notebook for each letter, so I just change the file name and the name of the text. The notebook then calculates a work cloud, sentiment analysis for the whole letter, a pie chart for the whole letter, identifies the sentences, plots the sentence positive sentiments (and moving averages), etc.

When I’ve finished all the letters, I’ll use TimeLine to show both sets of letters in order. I’ve not decided on what other information to present, so I enjoy the suggestions and references.

Russell’s 1918 letters from Brixton Prison are available at https://russell-letters.mcmaster.ca/ For each letter there is an image of the original and an annotated transcription. I’m using the transcriptions of the letters in English (some are in French) after removing the references to the annotations. In some cases, Russell quotes passages of text and poems, which I also delete, since they might influence the overall sentiment analysis.

In addition, there is a database of Russell related information at https://bracers.mcmaster.ca/ It doesn’t have transcriptions because of ownership and copyright issues. In the larger project I’m looking at social network analysis related to Russell for 1918. The database, today, returned 1167 entries for 1918.

I’m not sure what to use as training data for a classier. The general consensus is that the writing style in letters is not the same as in published articles and books.

I considered classifying each sentence and letter by hand, but I’m concerned it would be too subjective. Also, since it would take some time, I’m concerned about being consistent.

I like the objectivity from the Mathematica classifier, but given some of the points raised, I may need to adjust some of the classifications.

Daniel,

This works reasonably well on the few that I tried. Will likely need adjusting if the format is different in some of the transcripts. Also does not remove footnote numbers from the text. Will have to parse the HTML to do that.

brixtonLetter[number_] := Module[{plaintext, start, end},
  plaintext = 
   Import["https://russell-letters.mcmaster.ca/brixton-letter-" <> ToString@number, "Plaintext"];
  start = StringPosition[plaintext, "Original"][[1, 2]] + 2;
  end = StringPosition[plaintext, "Notes"][[1, 1]] - 1;
  StringTake[plaintext, {start, end}]
  ]

To get the transcript for letter 20

brixtonLetter[20]

Daniel,

I was using "12.3.1 for Mac OS X ARM (64-bit) (July 8, 2021)". Tried it on 13.0 PR3 "13.0.0 for Mac OS X ARM (64-bit) (November 12, 2021)" and got the same failure.

Looks like a bug in 13.0.

I also use sentiment analysis quite regularly. But there are some quirks you need to be aware of. E.g.

Classify["Sentiment", "The weather is horrible."]

gives "Negative" as expected.

Classify["Sentiment", "The weather is horrible!"]

gives "Positive". The exclamation mark has an interesting effect on nearly any statement:

Classify["Sentiment", "This day is the worst of my life and I am so sad!"]

is classified as "Positive". In many situations these issues can be mended though.

I think that there might be larger issues to consider than "only" a gradual change of language over time. Also, when analysing for example, social media posts in real time during political debates, one would use rather different Classifiers for say the US and the UK.

I would be quite interested in the analysis of Bertrand Russell's letters and would love to see the results.

Cheers from Scotland, Marco

PS: you can easily build your own classifier of course, using datasets like:

https://www.baeldung.com/cs/sentiment-analysis-training-data https://analyticsindiamag.com/10-popular-datasets-for-sentiment-analysis/ https://blog.cambridgespark.com/50-free-machine-learning-datasets-sentiment-analysis-b9388f79c124 https://www.kaggle.com/c/sentiment-analysis-evaluation/data https://www.kaggle.com/kazanova/sentiment140

This might be interesting, too:

https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-017-0121-9

It is also possible to use data from GDELT, which has newspaper articles and has data on how positive/negative the articles are. Not ideal for sentence by sentence analysis.

For a lecture I have "redone" the analysis here: https://www.jstor.org/stable/118328 based on the automatic classification in the Wolfram Language.

Is the data for this publicly available somewhere? Also I wonder what would be good training data for a classifier. Another question (or two): Were there so many letters that it is difficult to classify by hand? Or are there subtleties that might make classification by readers today difficult?