Group Abstract Group Abstract

Message Boards Message Boards

0
|
75.2K Views
|
3 Replies
|
2 Total Likes
View groups...
Share
Share this post:

StringSplit to split at word boundaries but excluding single quote

To StringSplit a string at word boundaries, I use the code

ParseText[ text_] :=
 StringSplit[ToLowerCase[text], RegularExpression["\\W+"]]

Applying this function results in a problem result. Thus,

   sample1 = "These are the times that try men's souls."; 
  ParseText[sample1]

returns

{"these", "are", "the", "times", "that", "try", "men", "s", "souls"}

However, this results in splitting words like "men's", "can't", "don't", etc into two words. What regular expression can I use to keep words like these intact?
POSTED BY: Lawrence Winkler
3 Replies
Sort By:
POSTED BY: Hans Michel
POSTED BY: Lawrence Winkler

I am not that good with Regular Expressions so here is a possibility using a StringExpression:

In[1]:= ParseText[text_] :=  StringSplit[ToLowerCase[text], Except[LetterCharacter | "'"]]

In[2]:= sample1 = "These are the times that try men's souls.";

In[3]:= ParseText[sample1]

Out[3]= {"these", "are", "the", "times", "that", "try", "men's", "souls"}
POSTED BY: David Reiss
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard