Format a text for a proper analysis?

Questions
1

Hi, I am currently working on a project that takes song lyrics and grades their sentiment. I am formatting the text for a proper analysis and am coming across an issue. The first image I am posting is what the output should look like when I enter the function

Map[Select[#, StringLength[#] > 1 &] &, tokensPop]

The second image uses the same function except with a different text source. Both are formatted in the same manner, however the second image does not yield an output that I can work with. Any ideas as to how I can fix this?

Thank you,

Michael

CorrectImage.PNG
CorrectImage.PNG1720×747 54.4 KB

incorrectimage.PNG
incorrectimage.PNG1795×913 69.3 KB

2

Really difficult to guess without an actual minimal working example that elicits the problem. If I am reading the error message correctly, it looks like some of the items Select sees are atomic, whereas Select needs expressions with parts.

3

It looks to me like your first example is a list of a list, your second example is a flat list. As Daniel states, it’s hard to help more without an example.

Regards