A data adventure in Boston, 1929: historical census corpus analysis

1929namesoccupations.rtf (717 KB)
DOWNLOAD-DESKTOP-NOTEBOOK.nb (17.1 MB)
fts-2.txt (1.61 MB)
topboysnames1929.csv (803 Bytes)
topgirlnames1929.csv (824 Bytes)

7 Likes

news-congrads-kkluyshnik-02-04-19.jpg – you have earned Featured Contributor Badge 2015-11-02_10-07-11.png
Your exceptional post has been selected for our editorial column Staff Picks http://wolfr.am/StaffPicks and Your Profile is now distinguished by a Featured Contributor Badge and is displayed on the Featured Contributor Board. Thank you!

This is interesting! It was amusing to see the name Rose being the most frequent on this page of the residents list, then seeing this reflects in the word cloud as one of the most common names in Boston back then.
Thank you a lot for sharing!

2 Likes

Hi, Rory. This was fun to read. Two things I especially liked about your post:

  • It shows a more conversational, stream-of-consciousness style for a computational essay. Presenting an idea and supporting it with code and graphics doesn’t have to be a dry, serious “essay”. It can have personality and vitality.
  • You show that data cleaning, part of many computational analysis projects, takes persistence and clever use of the Wolfram Language’s string and pattern matching functions. You started with pre-computer primary documents and were able to extract usable data. Data cleaning can be discouraging for those doing this kind of project for the first time.

Thanks for the post!

3 Likes

Thanks, Mark! I had a lot of fun! Glad you enjoyed. People often overlook the data cleaning, and it’s nice to sometimes actually demonstrate what goes into getting a nice dataset.

1 Like

Thanks Ahmed! Yes, always nice when first assumptions are validated. And the number of couples named Mary and Joseph was my biggest haha moment, though it didn’t make it into the essay.

2 Likes

Hi, Rory.

Great post!

Hear, hear, to Mark’s comments!

I’ve done a lot of sleuthing in US and NYS census data, manually transcribing data that Ancestry.com doesn’t automatically handle. I’ve also relied on the many editions of the Buffalo City Directory for the late 19th and early 20th centuries to piece together movements of my ancestors. All this work entailed handling snippets of data, while you on the other hand have dealt with the problem whole cloth!

In the past I thought about taking one residential neighborhood and compiling a history of it from census data. Your tools could make this dream a reality.

Ah, a thought just occurred to me to help you along with your novel. Look at some news papers from the period and study the advertising.

2 Likes

Brilliant!