Message Boards Message Boards

Zen Buddhism & Existentialism Visualized w/ Networks

To celebrate Nietzsche birthday let's compute something relevant a bit to to the philosophy he founded, existentialism, and it's Eastern counterpart (as some people think) - zen.

enter image description here

Wolfram Function Repository provides a unique opportunity for the Wolfram Language users to publish their own algorithms and immediately use them from within the Wolfram Language. I have recently published a function KeywordsGraph:

https://resources.wolframcloud.com/FunctionRepository/resources/KeywordsGraph

that can be used in the field of Computational Humanities. It generates a weighted graph connecting frequently used keywords of a text that are sequential neighbors and thus visualizing the flow and clustering of ideas in the text. It is well automated, so non-programmer humanities professional can easily use it in her research. Moreover some more sophisticated options are available for fine tuning of the algorithm. Please see the supporting docs and the source notebook for the function usage.

enter image description here

For the visualization above Stanford Encyclopedia of Philosophy articles were the original text source. We can outline some main key-points:

  • VertexWeight is set for every vertex to the number of times the corresponding keyword is encountered in text. This is reflected in the VertexSize.
  • EdgeWeight is set for every edge to the number of times an edge connection is made. Among other applications, this also help to build more meaningful CommunityGraphPlot as some of its methods take EdgeWeight in account.
  • Modularity-based clustering is used to find graph communities with CommunityGraphPlot.

The Wolfram Language code works as follows. First we access the text of articles on Japanese Zen Buddhist Philosophy and Existentialism

zen = Import["https://plato.stanford.edu/entries/japanese-zen"];
existentialism = Import["https://plato.stanford.edu/entries/existentialism"];

The we define the styles for graphs:

styles={
VertexLabelStyle->Directive[GrayLevel[.8],14],
EdgeStyle->Opacity[.3],
VertexSize->"VertexWeight",
GraphStyle->"Prototype",
VertexLabels -> Placed[Automatic, Above],
Background->Black};

The graphs are built as follows below. Note some text-filtering done with StringTake, because we do not need formal intro and bibliography. Also note for zen string "\[CloseCurlyQuote]s" is blacklisted. 

zen=CommunityGraphPlot[
    ResourceFunction["KeywordsGraph"][StringTake[zen,{610,87832}],35,{"\[CloseCurlyQuote]s"},styles],
PlotLabel->Style["Zen Buddhism",50,Lighter@Red,FontFamily->"Phosphate"],
CommunityBoundaryStyle->Directive[Yellow,Dashed,Opacity[.5]],
CommunityRegionStyle->Directive[Gray,Opacity[.3]],ImageSize->700{1,1}]

exi=CommunityGraphPlot[
    ResourceFunction["KeywordsGraph"][StringTake[existentialism,{592,86395}],35,{},styles],
PlotLabel->Style["existentialism",50,Lighter@Red,FontFamily->"Phosphate"],
CommunityBoundaryStyle->Directive[Yellow,Dashed,Opacity[.5]],
CommunityRegionStyle->Directive[Gray,Opacity[.3]],ImageSize->700{1,1}]

Final visual is made with:

Grid[{{zen}, {exi}}, Spacings -> {0, 0}]

Essential info for citation:

DETAILs: https://wolfr.am/zen-existentialism

TOOLs: all processing, algorithms, & visualizations are in Wolfram Language: http://wolfr.am/language

DATA: Stanford Encyclopedia of Philosophy:

ALGORITHM (published at Wolfram Function Repository): https://resources.wolframcloud.com/FunctionRepository/resources/KeywordsGraph

KeywordsGraph is a type of visualization I developed designated to produce a weighted graph connecting frequently used keywords of a text that are sequential neighbors and thus visualizing the flow and clustering of ideas in the text. Here modularity-based clustering is used in on the resulting KeywordsGraph to visualize which ideas are more related. The vertex size reflects on frequency corresponding term in the text. The flow of ideas can be also visualized (see the docs for algorithm).
POSTED BY: Vitaliy Kaurov
9 Replies
Sort By:
Rohit Namjoshi
Rohit Namjoshi
Posted 5 years ago

Hi Vitaliy,

Thank you and @Marco Thiel for the stopwords fix. I wrapped it in a helper.

deleteStopwords[text_] := 
 With[{words = TextWords[text], stopWords = ToLowerCase@WordList["Stopwords"]},
  words // ToLowerCase // Select[! MemberQ[stopWords, #] &] // StringRiffle]

fixedText = deleteStopwords[text];

To add tooltips to the edges, add EdgeLabels -> Placed["Name", Tooltip] to styles.

@Jeremy Sykes I published a version with tooltips to the Wolfram Cloud. It is very sensitive to mouse position. I don't know how to make it less sensitive, will experiment.
POSTED BY: Rohit Namjoshi

Nicely done, @Rohit Namjoshi ! But now, to fix all the issues, we must embark on some journey... Let's do this one step at a time. Get the text and words:

text=Import["http://dhspriory.org/kenny/PhilTexts/Sartre/BeingAndNothingness.pdf","Plaintext"];
words=TextWords[text];

Use @Marco Thiel's function from the post you linked:

https://community.wolfram.com/groups/-/m/t/1572809

stopWords=ToLowerCase[WordList["Stopwords"]];
myDeleteStopwords[list_]:=Select[list,!MemberQ[stopWords,#]&]

to process hyphenated words properly

newTEXT = words // ToLowerCase // myDeleteStopwords // StringRiffle;

Note, StringRiffle makes text from words agin as KeywordsGraph takes text, not list of words - maybe something to fix in future... Now you see those hyphenated words were very important, like for-itself and in-itself are special terms:

newTEXT // TextWords // Counts // TakeLargest[#, 12] &

<|for-itself->1509,world->1437,consciousness->1385,fact->947,object->916,relation->859,freedom->829,nothingness->811,body->722,in-itself->700,order->660,past->652|>

Redefine styles with "StopWords" and "LowerCase" both False as we do not need them anymore:

styles={
VertexLabelStyle->Directive[GrayLevel[.8],15],
EdgeStyle->Opacity[.3],
VertexSize->"VertexWeight",
GraphStyle->"Prototype",
VertexLabels -> Placed[Automatic, Above],
Background->Black,
"StopWords"->False,
"LowerCase"->False};

Do some research with CMD-SHIFT-E to understand it is not just an empty string, but some strange symbol that got in from the PDF file:

Cell[BoxData["\"\<\.01\>\""], "Input"]

Now go fancy, add FindGraphCommunities to be able to specify method of clustering:

CommunityGraphPlot[#,FindGraphCommunities[#,Method->"Modularity"],
PlotLabel->Style["Being and Nothingness",50,Lighter@Red,FontFamily->"Phosphate"],
CommunityBoundaryStyle->Directive[Yellow,Dashed,Opacity[.5]],
CommunityRegionStyle->Directive[Gray,Opacity[.3]],
ImageSize->1000{1,1}]&@
ResourceFunction["KeywordsGraph"][newTEXT,45,{"","other's"},styles]

Click on image to magnify and zoom:

enter image description here
POSTED BY: Vitaliy Kaurov

Thanks Rohit, that is great! And I hope we can find the Zen text! Now this I understand! Now the visual is certainly appealing, but those lines do get lost in the weeds, so it's difficult to relate the different terms from just eye-balling the graphic. Just fantasizing here, but a fascinating visual would be to be able to mouse over each term and have the routes to the different terms highlight. Does it do that in the notebook version?

And yes, those hyphenated definitions were very important in the text as I recall with very discrete meanings.

Nice work, Vitaliy and Rohit!
POSTED BY: Jeremy Sykes

Hi all, I might be able to fill in the Buddhism part as well as other Chinese philosophical texts which I have been studying for over two decades +. I would also take the analysis further by grouping words of similar meanings. Also sometimes things are lost in translation so be sure to consult experts on exact meaning.

There is a huge Buddhism treatise of Tibetan origin but is considered canon by both Chinese and Tibetan. The book is call The Great Treatise on the Path to Englightenment. It has been completely translated into English by the translation team of HH Dalai Lama. There was some PDF version available publicly for awhile but it’s now no where to be found. It would be very revealing if this function is used on it.

POSTED BY: Jack I Houng
Rohit Namjoshi
Rohit Namjoshi
Posted 5 years ago

Unfortunately the problem with DeleteStopwords causes important hyphenated words to be excluded which distorts the relationships. Top 10 hyphenated words. The first two have a high enough frequency to appear in the top 35.

words // ToLowerCase // Select[StringContainsQ[#, "-"] &] // Counts // TakeLargest[#, 10] &

(* <|"for-itself" -> 1509, "in-itself" -> 700, "non-being" -> 122, 
 "reflected-on" -> 113, "being-in-itself" -> 112, 
 "being-for-others" -> 96, "self-consciousness" -> 72, 
 "other-as-object" -> 65, "non-thetic" -> 60, "being-there" -> 46|> *)
POSTED BY: Rohit Namjoshi
Rohit Namjoshi
Rohit Namjoshi
Posted 5 years ago

Hi Vitaliy,

Thank you for publishing the KeywordsGraph function. I tried it on the "Sartre: Being and Nothingness" link provided by @Jeremy Sykes. There is an issue with DeleteStopwords and hyphenated words that has been raised before.

text = Import["http://dhspriory.org/kenny/PhilTexts/Sartre/BeingAndNothingness.pdf", "Plaintext"];
words = TextWords[text];
words // DeleteStopwords // ToLowerCase // Counts // TakeLargest[#, 12] &

(* <|"for-" -> 1571, "world" -> 1437, "consciousness" -> 1385, 
 "fact" -> 947, "object" -> 916, "relation" -> 859, "freedom" -> 829, 
 "nothingness" -> 811, "­" -> 742, "body" -> 722, "in-" -> 702, 
 "order" -> 660|> *)

Notice the "for-", "in-" and the empty string. I worked around it by adding them to the blacklist. However there is still a vertex with an empty label. Not sure why.

CommunityGraphPlot[
 ResourceFunction["KeywordsGraph"][text, 35, {"for", "in", "", " "}, styles],
 PlotLabel -> 
  Style["Being and Nothingness", 50, Lighter@Red, 
   FontFamily -> "Phosphate"], 
 CommunityBoundaryStyle -> Directive[Yellow, Dashed, Opacity[.5]], 
 CommunityRegionStyle -> Directive[Gray, Opacity[.3]], 
 ImageSize -> 700 {1, 1}]

enter image description here
POSTED BY: Rohit Namjoshi

@Jeremy Sykes, thanks for the links! If I find some time I will run the visualization on those texts.

I've been trying to figure out what this says

The visualization simply shows what are the most frequent terms in a text and which terms a used closely together. For instance, for me it is quite interesting to see that Space and Time are some key concepts for zen but not for the existentialism.

I'd rather see this with done with the texts of several works

Yes, dedicated texts and studies are very interesting (I will try to get to them). But so are the summaries if they are well written, and Stanford Encyclopedia of Philosophy, I think, is a quality source. An advantage of a summary is it focuses on the whole field encompassing many works, authors, and distilled terminology. Disadvantage of course - it is not an original key work :-)
POSTED BY: Vitaliy Kaurov

As an ex-philosophy student, I've been trying to figure out what this says. I think for one, I'd rather see this with done with the texts of several works. Let's say,

  1. Sartre: Being and Nothingness http://dhspriory.org/kenny/PhilTexts/Sartre/BeingAndNothingness.pdf

Why Write http://teaching.thenoiseofthestreet.net/dms463/sartre.pdf

  1. Heidegger: Being and Time http://pdf-objects.com/files/Heidegger-Martin-Being-and-Time-trans.-Macquarrie-Robinson-Blackwell-1962.pdf

3: Hegel: Phenomenology of Spirit https://www.marxists.org/reference/archive/hegel/works/ph/pinkard-translation-of-phenomenology.pdf

4: Kierkegaard: Fear and Trembling https://www.solargeneral.org/wp-content/uploads/library/fear-and-trembling-johannes-de-silentio.pdf

Unfortunately, I don't know much about zen. So the second chart is out of my league.
POSTED BY: Jeremy Sykes

Thanks for that demonstration Vitaliy!
POSTED BY: Eleazar Johannian
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract