Introduction:

Have you ever picked up a banana at a supermarket and asked yourself: how ripe is this? Well using supervised machine learning methods, this project seeks to identify the ripeness of a given banana using not one but two, unique ripeness scales – a standard one (ripe, not ripe) and a numerical one. Upload any image, banana or not. The microsite will first tell you whether or not the image is of a banana. If so, the site will tell you where the banana lies on the respective scales. Using the Spoonacular API, the site also displays select recipes that use the corresponding type of banana.

The Data Set

The first phase of the project, as in any machine learning project, is finding a data set. Unfortunately, since a comprehensive, virtual data set of bananas does not exist, one must create one. In this project, I used a Bing API to import banana images into Mathematica and create a data set (see the code below).

bing = ServiceConnect["BingSearch"];

Unfortunately, uploading large amounts of data often causes technical errors. To minimize losses during the crashes, I exported each individual image to the hard drive (see the code below).

Table[Export["/Users/kevinxu/Desktop/Banana Test/Images/" ~~ ToString@Unique["pineapple"] ~~ ".png", #] & /@  
Select[bing["Search", {"Query" -> "Pineapple", "SearchType" -> "Image", MaxItems -> 20, "Elements" -> "Images", "StartIndex" -> 1}], ImageQ],{i,1,61,20}];

Supervised Classification by Type

After finding a set of banana images, the next step is to label them by their type – overripe, underripe, and ripe – in preparation for the supervised machine learning. Images usually correspond to their search queries. Taking advantage of this shortens the classification process. Below is a line of code that allows for one to do this.

underripetraininglabeled =  Thread /@ Thread[underripetraining -> "underripe"];

I split the data set into three parts.

Labeled Training Set

A labeled training set of underripe, overripe, and ripe images.

trainingSet = Join[underripetraininglabeled, overripetraininglabeled, ripetraininglabeled];

Testing Set

An unlabeled testing set of underripe, overripe, and ripe banana images

underripetesting = underFilesSmaller[[111 ;;]];

Accuracy Reference Set

A labeled reference set of underripe, overripe, and ripe banana images

testingSetReference = Join[underripingtestingReference, overripetestingReference, ripetestingReference];

Classify

Once the data set is complete, one can create a classifier. I used the mathematica classify command to create a classifier function that distinguishes between ripe, underripe, and overripe bananas.

c = Classify[trainingSet];

Doing the Same with a Numerical Scale

A three term classifier is highly subjective and dependent on the user. A numerical scale provides a much needed objective scale.

Explorations

I created a linear ripeness indicator, based off correlations between ripeness images and RGB values in color spectrum graphs.

{72.8105, 20.8593, 4.45034, -48.3582, -49.7619};

In my case, a value of one was assigned to bananas that were on the brink of spoilage. A value of zero was assigned to a green banana fresh off a tree. A value of 0.5 was assigned to the "perfect" banana.

Indicator

After developing the scale, the next step is to tag a value onto each banana image.

newNumericalTest = Thread[trainingSet[[All, 1]][[1 ;; 620]] -> {0.05, 0.4, 0.1, 0.2, 
0.1, 0.25, 0.5, 0.08, 0.14, 0.18, 0.1, 0.22, 0.13, 0.10, 0.21, 
0.10, 0.20, 0.99, 0.14, 0.20, 0.06, 0.17, 0.20, 0.08, 0.14, 0.03, 
0.07, 0.10, 0.10, 0.13, 0.16, 0.18, 0.13, 0.13, 0.13, 0.1, 0.9, 
0.12, 0.15, 0.05, 0.13, 0.15, 0.05, 0.07, .06, 0.02, 0.10, 0.13, 
0.08, 0.10, 0.08, 0.10, 0.11, 0.13, 0.13, 0.15, 0.16, 0.9, 0.06, 
0.10, 0.03, 0.10, 0.17, 0.110, 0.10, 0.04, 0.08, 0.13, 0.05, 0.07,.....

Numerical Classifier Function

numClassify = Classify[newNumericalTest];

Deploying the MicroSite

After developing the two classifiers, I created a microsite that uses them to analyze user-inputed images.

The output interface is shown below.

A Ripe Banana

The output shows both the numerical classification and the type classification of the select banana image. A number line plot identifies where the banana lies on the numerical scale of ripeness. Recipes based off the ripeness of the banana are shown below, with hyperlinks if one wants to see more.

Not a Banana

Since the object inputted was not a banana, the numerical value returned is -1. No recipes are shown. "

Access the Site

Link: https://www.wolframcloud.com/objects/user-885cc732-98f5-4749-888f-6a84b77e85d5/numClassify

Further Research

The same methods can be applied to other fruits like apples or avocados. Hopefully, I can expand the site in the future without sacrificing too much of the evaluation speed.