Message Boards Message Boards

[WSC18] Classifying & Converting the Major Currencies By Using Neural Nets

GROUPS:

enter image description here

Introduction

Throughout the years, I've traveled around the world experiencing new places, eating great food, and learning about other cultures. In order to do some of these things, I needed to use the national currency of the country I was in. Whether it was using Euros in France to buy some macarons, or using the Pound in England to buy tea, I noticed that each currency is unique and its design is a part of the country's identity. My project allows a user to input as many images of different coins, and select a currency to convert them all to. The Microsite identifies what country the coin is from, the name of the currency, the value in native currency, and the converted value.

Gathering Data

Since the Wolfram database didn't have a large dataset of foreign coin images, we had to gather some data manually. We started out by using WebImageSearch and keywords to pull images off of Bing. Since coin designs change every couple years we used en.ucoin.com as a reference as to what the current design was. Then, we edited the set of images that the WebImageSearch gave me. We then exported the relevant images to a folder named after the coin's country of origin, and named the file after its name and value. The files were saved as .wxf. We did this process for all 42 coins and additionally pulled more data from Google Images. enter image description here

Different Approaches to Feeding the Neural Network

Creating the right data set and matching it with a network took 4 attempts

Attempt 1

For the first try, we created training data by joining all of the data saved in the folders (at the time this was just Euros and Canadian Dollars) and randomized it. We threaded each folder and gave the images labels of the country. We tried to use the Classify function but found that it was not complex enough.

fullEUData = 
  Thread[Flatten[
     Map[ImageResize[#, {50, 50}] &, Import[#]] & /@ 
      FileNames["*", "coin_data/euro"]] -> "EU"];

fullCanadianData = 
  Thread[Flatten[
     Map[ImageResize[#, {50, 50}] &, Import[#]] & /@ 
      FileNames["*", "coin_data/canadian"]] -> "Canadian"];

trainingData = RandomSample@Join[fullEUData, fullCanadianData];

cf = Classify[trainingData]

enter image description here

Attempt 2

Since I didn't have the skills to build my own neural network, my mentor suggested we use the ImageIdentify net. We were able to modify the net to fit the goal of the project and had great results with the net. However, during this attempt we found that the net was depending on the transparent or white backgrounds of the pure images. enter image description here

Attempt 3

After analyzing the results of the 2nd attempt, we decided to process and format the images. We removed the backgrounds from each image, randomized the brightness, contrast, and angles, and layered the coins onto a background of blended color and noise.

preprocess[image_ (*Removes background*)
  ] :=

 ImagePad[RemoveBackground@image, 5, Padding -> None]

background // Clear;
background := (*Creates a random background*)
  Blend[{ConstantImage[RandomColor[], {224, 224}], 
    RandomImage[1, {224, 224}, ColorSpace -> "RGB"]}, RandomReal[]];

randomLighting[
   image_] := (*randomizes brightness and contrast of image*)

  ImageAdjust[image, RandomReal[{-.25, .25}, 2]];

overlay[coin_Image, background_Image] := (*Layers everything together*)

  ImageCompose[
  ImageCrop[background, {224, 224}]
  ,
  ImagePerspectiveTransformation[
   randomLighting@ImageResize[coin, RandomReal[{60, 260}]], 
   IdentityMatrix[2] + RandomReal[{-.75, .75}, {2, 2}],
   DataRange -> Full,
   Background -> White
   ]
  ,
  {RandomReal[{60, 190}], RandomReal[{70, 180}]}
  ]

For this attempt we also created data using a generator rather than having a pre made dataset. This helped the net train much quicker and was great at identifying 3 currencies. However, when we increased the number of currencies to 7, it struggled. enter image description here

Attempt 4 / Final Attempt

In the last 3 attempts we had assigned the images from each country the same label, ex: "USA" or "UK". We realized that the net could get confused due to thinking up to 7 different coins were all the same thing. So, we changed the labels from country, to country & value. Ex: "USA_0.25". Then, we joined all of the countries data and that was the fullOriginalData set. This was the final attempt and worked the best because each image of a specific value and country had its own label, therefore avoiding confusion for the neural net.

fullOriginalData = 
  RandomSample[
   Join[fullCanadianData, fullEUData, fullUKData, fullChinaData, 
    fullJapanData, fullSwissData, fullUSData, fullUSData]];

We used the same process to format the images with backgrounds, but instead of using a generator to create data, we used a pre-made set. In order to create test data we split the fullData 80%, 20%.

createRandomData // ClearAll;

createRandomData[coin_ -> label_, background_] :=

  Thread[overlay[preprocess@coin, background] -> label];

data1 = Table[createRandomData[RandomChoice[fullOriginalData], background], 
   Length[fullOriginalData]];
data2 = Table[createRandomData[RandomChoice[fullOriginalData], background], 
   Length[fullOriginalData]];

(*Random group of data*)

fullData = RandomSample@Flatten@Join[data1, data2];

Later on, we increased the data set from 900 to 5,000.

trained = 
 NetTrain[new, trainingData, ValidationSet -> testdata, 
  BatchSize -> 20, MaxTrainingRounds -> 30]

Accuracy

The results of the neural net were extremely accurate. With a 99.58% accuracy rate on training data, and a 99.52% accuracy rate on test data, the program is able to make an accurate guess every time. enter image description here

Assigning the Output with a List of Characteristics

The net outputs the label which gives the country and value, ex: "USA_0.25". In order to have the microsite's output look better, we created a list of lists that included written out characteristics.

assignments = 
  Association[ {"CAN_1" -> {"Canada", "CanadianDollars", 1.00}, 
    "CAN_2" -> {"Canada", "CanadianDollars", 2.00}, 
    "CAN_0.50" -> {"Canada", "CanadianDollars", 0.50}, 
    "CAN_0.25" -> {"Canada", "CanadianDollars", 0.25}, 
    "CAN_0.10" -> {"Canada", "CanadianDollars", 0.10}, 
    "CAN_0.05" -> {"Canada", "CanadianDollars", 0.05}, 
    "CAN_0.01" -> {"Canada", "CanadianDollars", 0.01},

    "EU_1" -> {"European Union" , "Euros", 1.00}, 
    "EU_0.50" -> {"European Union", "Euros", 0.50}, 
    "EU_0.20" -> {"European Union", "Euros", 0.20}, 
    "EU_2" -> {"European Union", "Euros", 2.00}, 
    "EU_0.10" -> {"European Union", "Euros", 0.10}, 
    "EU_0.05" -> {"European Union", "Euros", 0.05}, 
    "EU_0.02" -> {"European Union", "Euros", 0.02}, 
    "EU_0.01" -> {"European Union", "Euros", 0.01},

    "UK_1" -> {"United Kingdom", "BritishPounds", 1.00}, 
    "UK_2" -> {"United Kingdom", "BritishPounds",  2.00}, 
    "UK_0.50" -> {"United Kingdom", "BritishPounds", 0.50}, 
    "UK_0.20" -> {"United Kingdom", "BritishPounds", 0.20}, 
    "UK_0.10" -> {"United Kingdom", "BritishPounds", 0.10}, 
    "UK_0.05" -> {"United Kingdom", "BritishPounds", 0.05}, 
    "UK_0.02" -> {"United Kingdom", "BritishPounds", 0.02}, 
    "UK_0.01" -> {"United Kingdom", "BritishPounds", 0.01},

    "CHN_1" -> {"China", "ChineseYuan", 1.00}, 
    "CHN_0.50" -> {"China", "ChineseYuan", 0.50}, 
    "CHN_0.10" -> {"China", "ChineseYuan", 0.10},

    "JPN_1" -> {"Japan", "Yen", 1}, "JPN_5" -> {"Japan", "Yen", 5}, 
    "JPN_10" -> {"Japan", "Yen", 10}, 
    "JPN_50" -> {"Japan", "Yen", 50}, 
    "JPN_100" -> {"Japan", "Yen", 100}, 
    "JPN_500" -> {"Japan", "Yen", 500},

    "SUI_5" -> {"Switzerland", "SwissFrancs", 5.00}, 
    "SUI_2" -> {"Switzerland", "SwissFrancs", 2.00}, 
    "SUI_1" -> {"Switzerland", "SwissFrancs", 1.00}, 
    "SUI_0.50" -> {"Switzerland", "SwissFrancs", 0.50}, 
    "SUI_0.20" -> {"Switzerland", "SwissFrancs", 0.20}, 
    "SUI_0.05" -> {"Switzerland", "SwissFrancs", 0.05},

    "USA_0.25" -> {"United States", "USDollars", 0.25}, 
    "USA_0.10" -> {"United States", "USDollars", 0.10}, 
    "USA_0.05" -> {"United States", "USDollars", 0.05}, 
    "USA_0.01" -> {"United States", "USDollars", 0.01}
    }];

The second element in the sublist is the name of the currency saved in Mathematica. This string is an input for the function CurrencyConvert. Not only does the Microsite list this value as the name of the currency, but it also fetches it to use in the conversion feature.

Deploying the Microsite

Using CloudDeploy and FormFunction, we made a microsite that allows the user to input as many images as they'd like from the 7 currencies into the drop box. The site also allows the user to select one of the 7 currencies in the dropdown menu to covert all of the coins to. The output gives the image, country, name of currency, value in native currency, and value in converted currency all in table form. At the bottom of the page the output also neatly displays the total in original currencies, and the total in the converted currency. enter image description here enter image description here

Future Work

In the future, this project can be expanded by: Adding all of the currencies in the world , Since the design changes often, date the currencies by years , Use this to create a mobile app that allows the user to easily take a photo of the coins they have.

Acknowledgements

This project could not have been completed without the help and insight from my mentor Rick Hennigan

Click Here to view the Microsite

Computational Essay Is Down Below

Attachments:
POSTED BY: Morgan Lee
Answer
9 days ago

enter image description here - Congratulations! This post is now a Staff Pick as distinguished by a badge on your profile! Thank you, keep it coming!

POSTED BY: Moderation Team
Answer
3 days ago

Group Abstract Group Abstract