There does seem to be a bug in the "News Aggregator" example code. Try the following:

articles = Partition[TextSentences[#], 10] & /@ pages;

instead of

articles = Partition[TextSentences[#], 10] & /@ articles;

I was able to retrieve 162 articles.

I found it amusing that one of the poll answers proposed ML as an alternative to traditional programming, which was not the expected answer. In that sense, I do agree the correct answer was the most agreeable one.

That said, some of the next poll questions I thought demonstrated the point of the first, which is to say there are many, many ways to use either to solve a problem. In some of those cases, such as writing a procedure, there's a clearly better choice and a clearly more cumbersome one.

I'd put it more like this. Fundamentally, traditional programming languages "decide" what to do based on some collection of rules. They express a primitive aspect of a traditional computer, the logic gate: something is true or false. Complexity in decision-making builds up from there.

Machine learning is intended as a replacement to traditional programming, I argue, but in a particular case. To wit: when the amount of data available is Big, and the values of its features show some propensity for changing (or evolving) over time, machine learning is the more likely choice.

Traditional programming makes decisions through a pathway of rules. Machine learning operates on assigning meaning to data (e.g., labelling, feature reduction), or associating data (e.g., classification), provided there is enough data to determine statistical significance, and then accuracy of its inferences.

I argue you can replace traditional programming with machine learning for the same reasons, in hindsight, we've been using traditional programming in lieu of machine learning. The only reason we didn't, any sooner than we did, was for lack of resources to do it. Now that we have enough to do both, we can freely our waste our time committing to the wrong approach, as we often do, or selecting the more plausible approach.

If you have any experience watching customers choose ML for a project because ML was the hot new thing, I imagine you can see my point of view.

Firstly thanks for a very interesting series of lectures and discussions. I have a specific neural network example code that I would like to discuss and just need some assistance in finding the best way to approach this example. If not appropriate to discuss in depth here please advise if we can discuss off the forum. I am a looking at a supervised learning classification example of a rotated qubit. Basically an interval [0, pi] of rotation is discretized and measurements of spin-up or spin-down are associated with each each chose rotation angle in this interval. The rotation angle labels are represented as one-hot vectors (that is of the form {1,0,0,0...}). Hence this is a classification problem. I have a working code (attached) but I think it needs refinement (or a different approach). I am new to using Mathematica for machine learning applications.
My training and validation data sets are of the form:

{{975, 25} -> {1, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {919, 81} -> {0, 1, 0, 
   0, 0, 0, 0, 0, 0, 0}, {801, 199} -> {0, 0, 1, 0, 0, 0, 0, 0, 0, 
   0}, {652, 348} -> {0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, {505, 495} -> {0,
    0, 0, 0, 1, 0, 0, 0, 0, 0}, {357, 643} -> {0, 0, 0, 0, 0, 1, 0, 0,
    0, 0}, {190, 810} -> {0, 0, 0, 0, 0, 0, 1, 0, 0, 0}, {108, 
   892} -> {0, 0, 0, 0, 0, 0, 0, 1, 0, 0}, {15, 985} -> {0, 0, 0, 0, 
   0, 0, 0, 0, 1, 0}, {0, 1000} -> {0, 0, 0, 0, 0, 0, 0, 0, 0, 1}}

The tuples on the left are basically the data of how many spin-up and spin-down outcomes are recorded for each angle. Basically I simulate 1000 measurements for each rotation angle (10 rotation angles in the above training set but 40 in the attached code), and associate the outcomes of spin-up or spin-down with the rotation angle label (one-hot vectors). I then test the code for {1,0} and {0,1}, that is one spin-up and one-spin down and plot the associated angles probabilities. If you have a chance please have a look at my code and how I trained the data in the attached code. The output I am interested in is an accurate probability distribution for each bin (theta angle) given {1,0} and {0,1} after training. Any insights and tips for improving the accuracy of the neural network for this classification example would be appreciated.

Attachments:

Classification_e...nb

When I try to run the code in the downloaded version of the book, there is a dependency on an external tool, see screen shot. How do I resolve this?

Thanks for a quick answer. I followed the link, downloaded the .zip file. It has the same code in it.

Carl,

I like your reply! That's a fantastic point you make. The math or model can be wrong but it also can be useful and accurate.

Sorry for mistakenly referring to "Clustering" as Chapter 5 from the book. It is actually Chapter 6. Tomorrow, Thursday June 13th, we will be covering Chapter 6: Clustering.

I see that now. Here is the cleaned up code that should work:

Show[Plot[
  QuantityMagnitude@PDF[dist, Quantity[x, "Feet"]], {x, 58, 88}, 
  Filling -> Bottom, PlotRange -> {{0, 140}, {0, Automatic}}, 
  PlotStyle -> Gray, Frame -> {True, True, False, False}, 
  PlotLabel -> "Predictive distribution for 23 mi/h", 
  PlotLegends -> SwatchLegend[{"68% probability interval"}], 
  FrameLabel -> {"distance (ft)", "probablity density"}, 
  ImageSize -> 360], 
 Plot[QuantityMagnitude@PDF[dist, Quantity[x, "Feet"]], {x, 0, 140}, 
  PlotLegends -> {"probability density"}, Filling -> Bottom, 
  PlotRange -> {0, Automatic}]]

Hi Abrita, I have a quick question: what model could I use to identify : How much does a variable contribute to the classification?. My predictor variable is binary and I have 11 explanatory numerical variables. Thanks

You can use SHAPValues to figure out the contribution of a feature. It is available as a n option for both PredictorFunction and ClassifierFunction.

 c = 
 Classify[{<|"Age" -> 32, "Height" -> 160|> -> 
    "Female", <|"Height" -> 183, "Age" -> 41|> -> 
    "Male", <|"Height" -> 123|> -> 
    "Female", <|"Height" -> 175, "Age" -> 21|> -> 
    "Male", <|"Age" -> 11|> -> 
    "Male", <|"Age" -> 52, "Height" -> 164|> -> "Female"}]

In[14]:= c[<|"Age" -> 12, "Height" -> 120|>, "SHAPValues"]
Out[14]= <|"Female" -> <|"Age" -> 0.915746, "Height" -> 26.9166|>, 
 "Male" -> <|"Age" -> 1.09201, "Height" -> 0.0371519|>|>

The way SHAP values are computed is basically to see how much results change if different features in the data are dropped.

Nice!, what if a want some global values. For example, when I used "LogitModelFit" I got a "ParameterTable" with the Estimates for each feature, is possible to have something like that for a Classifier?

"All models are wrong, but some are useful". - George Box.

You can use Information with "MethodParameters" option to get the parameter values for the model.

Wolfram Language also has built-in support fro Time Series Analysis that may be of interest for this dataset https://reference.wolfram.com/language/guide/TimeSeries.html

Thanks for your response. Is each sample not associated with a one-hot vector class? Would it then not be the same amount of samples and classes? Yes there are only two inputs in this example. Could you please advise on how Predict could be used in this case, would it output a list of probabilities associated with the labels (one-hot vectors)?