Here's another way to get rhyming words:

WolframAlpha["word", {{"Rhyme:WordData", 1}, "ComputableData"}]

You can get lexically similar words with:

WolframAlpha["word", {{"LexicallyCloseWords:WordData", 1}, 
  "ComputableData"}]

And, for puns, may want a supply of homophones:

WolframAlpha["where", {{"Homophones:WordData", 1}, "ComputableData"}]

With some words, to get this to work, you need to specify that the input is a word:

WolframAlpha["see", {{"Homophones:WordData", 1}, "ComputableData"}, 
 InputAssumptions -> {"*C.see-_*Word-"}]

Q. What do you call a gator bowdlerizer? A. An alligator expurgator.

(Joke Type 2 works now, with the correction.)

@Jesse Friedman we were playing with friends with your code and I think in this piece:

words = # -> {WordData[#, "Hyphenation"], 
      Flatten[WordData[#, "Synonyms"][[All, 2]]]} & /@  WordData[All, part];

the symbol part is undefined. Could you add the missing part - forgive the pun ;) ?

A poem derived from the first. (Arranging and sequencing in a certain way changes the emotional valence from funny to something else.)

My Conviction

I like my birth like I like my medication: giving.

I like my breakfast like I like my divide: continental.

I like my coffee like I like my tea: iced.

I like my cat like I like my twin: Siamese.

I like my map like I like my pitcher: relief.

I like my machine like I like my region: infernal.

I like my life like I like my world: phantasy.

I like my correlation like I like my distortion: nonlinear.

I like my fever like I like my stroke: haemorrhagic.

I like my disorder like I like my psychosis: schizophrenic.

I like my intelligence like I like my schizophrenia: borderline.

I like my pronoun like I like my verb: reflexive.

I like my balance like I like my cubism: analytical.

I like my error like I like my language: programming.

I like my file like I like my trauma: blunt.

I like my charge like I like my conviction: murder.

Very nicely done!