Message Boards Message Boards

Let's play with ImageIdentify

Posted 8 years ago

Hello,

for my first post on these groups (hi all!), here is a small recreational function combining ImageIdentify and image search on the web. The idea is to start from a given word (or sentence, or...), search for the internet for images corresponding to this word, and then perform ImageIdentify on the result, which gives a new word. And to repeat the process until we get with ImageIdentify what we are searching for. It can lead to nice/surprising/funny combination of text and images.

Technically, the function just combines ImageIdentify and image search, and uses FixedPoint to repeat the process until we find the result. The final function is called ImgPlayFixP, which takes a string (the initial word) as argument:

ImgPlay[x_?StringQ] := 
 Module[{a, a2, b, xx}, xx = StringReplace[x, " " :> "+"];
  Sow[xx, "name"]; 
  a = Import["http://images.google.com/search?q=" <> xx, "Images"]; 
  If[Length[a] > 0, a2 = RandomChoice[a], 
   a2 = First@
     Import["http://images.google.com/search?q=mickey+mouse", 
      "Images"]]; Sow[a2, "Image"]; 
  b = StringReplace[#, " " :> "+"] &@
    CommonName@ImageIdentify[a2, SpecificityGoal -> "High"]; b]

ImgPlayFixP[x_?StringQ] := 
 Module[{aa}, 
  aa = #[[2]] &@Reap[FixedPoint[ImgPlay, x, 10], {"Image", "name"}]; 
  Grid[{StringReplace[#, "+" -> " "] & /@ aa[[2, 1]], aa[[1, 1]]}, 
   Frame -> All]]

A bit of randomness is present (random choice of the picture among the results), and in the case where the search gives no result, we use a picture of Mickey Mouse (because why not...).

Here is another version of the function, which does not need any input, and uses a random word instead (RandomWord function):

ImgPlayFixPRandom[] := 
 Module[{aa, w}, Off[$CharacterEncoding::utf8]; 
  w = RandomWord[{"KnownWords", "Noun"}]; Print[w]; 
  aa = #[[2]] &@Reap[FixedPoint[ImgPlay, w, 10], {"Image", "name"}]; 
  Grid[{StringReplace[#, "+" -> " "] & /@ aa[[2, 1]], aa[[1, 1]]}, 
   Frame -> All]]

Here are a few example of results (note that because of the randomess, you can get different results when starting with the same word). The last two were obtained with the "random word" version. Reading of the pictures: the initial word is the one on the top left; this word gives the image on the bottom left; Imageidentify on this image then gives the word on the second column, etc. The last image corresponds to a fixed point.

Initial word : biscuit

initial word : lune

initial word: general relativity

random initial word : trucking

random initial word : genesis

The code can certainly be improved. It could also be fun to run the function on a very large number of initial words, and to plot a graph showing the "trajectories" of the function among the words...

Thibaut

9 Replies

enter image description here - you earned "Featured Contributor" badge, congratulations !

This is a great post and it has been selected for the curated Staff Picks group. Your profile is now distinguished by a "Featured Contributor" badge and displayed on the "Featured Contributor" board.

POSTED BY: Moderation Team

Thanks to everyone for the comments.


@Carlo : Yes, you are right. My way of dealing with the URL was rather basic and certainly not bugproof. I have now improved the code with URLBuild:

ImgPlay2[x_?StringQ] := Module[{a, a2, b},
  Sow[x, "name"]; 
  a = Import[
    URLBuild[{"http://images.google.com/search"}, {"q" -> x}], 
    "Images"]; 
  If[Length[a] > 0, a2 = RandomChoice[a], 
   a2 = First@
     Import["http://images.google.com/search?q=mickey+mouse", 
      "Images"]]; Sow[a2, "Image"]; 
  b = CommonName@ImageIdentify[a2, SpecificityGoal -> "High"]; b]

@ Arnoud : From the list of entities you obtain, applying the ImgPlay function, and using ImageCollage, I get this picture of your office (I'm sure you can recognize it :-) ):
Arnoud's office :)

HI,

I've noticed you're using StringReplace to remove spaces from your URL. You might want to consider using something like URLEncode to escape the other non-safe characters.

In[7]:= URLEncode["èü/:%"]

Out[7]= "%C3%A8%C3%BC%2F%3A%25"

Or even URLBuild to directly construct the whole URL.

In[5]:= URLBuild[{"https://images.google.com/search"}, { 
  "q" -> "foo bar"}]

Out[5]= "https://images.google.com/search?q=foo+bar"
POSTED BY: Carlo Barbieri

Nice!

Another fun one to play around with (requires a webcam):

cc = {};
Dynamic[{c = CurrentImage[], cc = Union[cc, {ImageIdentify[c]}]}]

Randomly pointing my webcam in different directions gives an array of identified entities (some quite reasonable and others less so):

enter image description here

POSTED BY: Arnoud Buzing

Wow amazing! nice succinct code. I'm seriously wondering where you are, the combination of oil-refinery, laser printer, cooling tower, and plane seat makes for an interesting scene!

POSTED BY: Sander Huisman

My office looks out on a McDonalds restaurant. Which would explain "oil refinery" and "industrial plant" ...

POSTED BY: Arnoud Buzing

Exercise: figure out what fraction of buildings in the US don't overlook a McDonalds "restaurant".

Bonus points for using height of buildings and GeoVisibleRegion.

POSTED BY: Carlo Barbieri

Hahaha! Funny but kinda sad.

POSTED BY: Sander Huisman

I never thought about using Sow inside a function and the Reap 'outside' it.... nice...

POSTED BY: Sander Huisman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract