Over the past two weeks, I have been a student at the Wolfram Summer High School, where I have worked on a project called “Using Machine Learning to Identify Road Signs.” The result of the project is a microsite that uses Wolfram's ImageIdentify neural network to identify road signs in user-taken pictures.

Creating the Training Data

In order to simulate an image of a road sign taken by a user, I took a set of background images from Flickr and images of each of the 6 road signs I was classifying, and overlaid the signs on different background images at various angles and

ServiceConnect["Flickr"]
Normal[ServiceExecute["Flickr", "ImportImage", {"Keys" -> #, "ImageSize" -> "Large"}] & /@ IDs]

Because the image of the road sign will not always be directly facing the camera and in the center of the image, I then used ImagePerspectiveTransform to rotate and tilt the image. This was done in order to compensate for the angles that the road signs would be at in the images take by users. The resulting signs were then overlaid on the backgrounds at randomly-generated positions on the image. In total, there are 9000 examples in the training set.

overlay[sign_Image, background_Image, iterations_Integer] := 
 Flatten[Table[
   Join[{ImageCompose[ImageCrop[background, {224, 224}], 
      ImagePerspectiveTransformation[
       ImageResize[sign, RandomReal[{25, 100}]], 
       IdentityMatrix[2] + RandomReal[{-.2, .2}, {2, 2}], 
       DataRange -> Full], {RandomReal[{40, 100}], 
       RandomReal[{70, 100}]}], 
     ColorConvert[
      ImageCompose[ImageCrop[background, {224, 224}], 
       ImagePerspectiveTransformation[
        ImageResize[sign, RandomReal[{25, 100}]], 
        IdentityMatrix[2] + RandomReal[{-.2, .2}, {2, 2}], 
        DataRange -> Full], {RandomReal[{40, 100}], 
        RandomReal[{70, 100}]}], "Grayscale"]}], iterations]]

training = Flatten[Table[createTraining[Flatten[overlay[Keys[signs][[x]], #, 15] & /@ bgs], Values[signs][[x]]], {x, 6}]];

Training the Classifier

The microsite leverages the Wolfram Language’s Neural Network functionalities to classify images into one of the 6 road sign classes.

The classifier is based on Wolfram’s ImageIdentify Net, which can be found with the NetModel function by getting the "UninitializedEvaluationNet" property of the model. The neural network was modified to classify the six road signs used.

I then trained the network on the training data created earlier.

Creating the Microsite

The final product of this project is the microsite, which allows the user to take a picture of a road sign to find out what it means.

CloudDeploy[
 FormPage["image" -> "Image", 
  Column[{image, "This is a " <> ToString[net[image]] <> " sign"}], 
  AppearanceRules -> <|"Title" -> "What sign is that?", 
    "Description" -> 
     "Take a picture of a sign and we'll tell you what it means"|>], 
 Permissions -> "Public"]

The final microsite can be found here: https://www.wolframcloud.com/objects/3ea6ba27-23f2-4d47-b61f-f38788b79393

I would like to thank my mentor John and all the instructors at the Wolfram Summer Camp. They have been amazingly helpful and I have learned so much over the past two weeks. I couldn't have hoped for a better summer camp experience here.