Message Boards

GROUPS:

Farai Fredric Mlambo

[WSS20] A Computational Introduction to Statistical Modelling

Farai Fredric Mlambo, University of the Witwatersrand

Posted 6 months ago

1280 Views

1 Reply

5 Total Likes

Follow this post

enter image description here

Introduction

The Wolfram Language presents an unparalleled functional, symbolic and numerical architecture for the fitting and analysis of statistical models. Statistical modelling functions in the Wolfram Language create symbolic fitted model objects from which various properties can be queried for reporting and/or further analysis. Coupled with the Wolfram Language’s top-level superfunctions and system coherence, these symbolic fitted model objects are accessible from a wide host of print-ready Wolfram Language’s visualization tools.

With a computational language, such the Wolfram Language - Statistical Modelling and Machine Learning has never been easier. What is a computational language?

According to Stephen Wolfram -

“It’s a language for expressing things in a computational way—and for capturing computational ways of thinking about things. It’s not just a language for telling computers what to do. It’s a language that both computers and humans can use to represent computational ways of thinking about things. It’s a language that puts into concrete form a computational view of everything. It’s a language that lets one use the computational paradigm as a framework for formulating and organizing one’s thoughts” - Stephen Wolfram’s Blog post: What We've Built Is a Computational Language (and That's Very Important!)

Although the Wolfram Language is such a powerful computational and programming language, also being ubiquitous (usually with a campus-wide license) at most of top universities abroad and here (South Africa), students and researchers don not seem to know that they have such a tool. Moreover, those that know about the Wolfram Language seem to think that “Mathematica”, as it is known is only for mathematicians and not for students and researchers in the Natural Sciences, Engineering, Commerce, Arts and Humanities.

Being an educator (both high School and University) in the Mathematical and Statistical Sciences for about 15 years, having used Mathematica for about 10 years, I felt the need for compiling a text showcasing how one can perform statistical modelling and machine learning in the Wolfram Language. Compiling sample chapters for this text became my project for the Educational Innovation Track of the 2020 Wolfram Summer School. The proposed title of this text is: A Computational Introduction to Statistical Modelling.

This text is intended to exhibit the art of computational and symbolic and statistical model fitting and analysis in the Wolfram Language. The text also introduces the reader to the principles of machine learning in the Wolfram Language, as an extension from the statistical model fitting and analysis. Based on the coherence of the Wolfram Language, the principles of symbolic model fitting and analysis remain consistent in the machine learning paradigm. This is made clear in the text, with examples from classical machine learning problems with predictions of quantitative outcomes and classification problems involving categorical responses.

Considering that this book is intended for students and researchers from non-statistical fields, there will be a preface, which will serves as an introduction and/or primer to the Wolfram Language. Although the reader can get a quick and high level tour of the principles of the Wolfram Language in this section, it recommended that the reader should also use the following book by Stephen Wolfram: An Elementary Introduction to the Wolfram Language.

This text will have a preface then the body. The preface is intended to be an introduction to the Wolfram Language. We will highlight the key principles of the language, also introducing the reader to the various resources available, as will be used in the text.

Preface: A High Level Introduction to the Wolfram Language

In this section, the reader will be introduced to the following concepts:

Licencing and Product Installation

◼

Desktop Mathematica

◼

Cloud Mathematica

◼

Students and Academic Staff

◼

Industry

◼

What is the Wolfram Language?

◼

A bit of history

◼

Mathematica

◼

A Computational Language

◼

Working with Notebooks

◼

What are Notebooks?

◼

Why Notebooks?

◼

Computational Essays

◼

Working with Wolfram Documentation

◼

What is the Wolfram Documentation?

◼

Guides, Tutorials, TechNotes etc

◼

Desktop and Web Documentation

◼

Symbols, Objects and Expressions

◼

Why Symbolic?

◼

Everything is an Expression

◼

Computational Language

◼

Built-In Knowledge in the Wolfram Language

◼

Why so much Knowledge?

◼

How to use the Built-In Knowledge

◼

Data Types in the Wolfram Language

◼

Lists

◼

Associations

◼

Sparse Arrays

◼

Data Sources in the Wolfram Language

◼

Example Data

◼

The Wolfram Data Repository

◼

Functional Programming in the Wolfram Language

◼

Programming Paradigms

◼

Why Functional Programming?

◼

Good Examples

◼

WolframAlpha

◼

What is WolframAlpha?

◼

More than a Search Engine

◼

Wolfram Alpha for Students and Educators

◼

Computable Results from WolframAlpha

◼

The Wolfram Function Repository

◼

Why a Function Repository?

◼

How to use the Function Repository

◼

How to Contribute to the Function Repository

◼

The Wolfram Neural Network Repository

◼

What is the Wolfram Neural Network Repository?

◼

Advantages of using the Wolfram Neural Net Repository

◼

Computational Language

◼

The Wolfram Demonstrations Site: Bringing Life to Science, Education and Research

◼

What is the Wolfram Demonstrations Site?

◼

How students and academics can use the Wolfram Demonstrations Site

◼

How to Contribute to the Wolfram Demonstrations Site

◼

The body of this book will be divided into four main parts. These parts are further divided into various chapters of common interest.

PART 1: DESCRIPTIVE STATISTICS, STATISTICAL DISTRIBUTIONS AND HYPOTHESIS TESTING IN THE WOLFRAM LANGUAGE

This part serves as a primer and/or introduction to the principles of Statistical Modelling in the Wolfram Language. Considering that both the classical (frequentist) statistical and Bayesian statistical models to be discussed have some distributional assumptions about the residuals, it is essential that we establish a foundation for how the principles of mathematical statistics are presented in the Wolfram Language. Starting from the very basics of descriptive statistical tabulation and visualization, this part introduces the reader to principles of exploratory analysis in the Wolfram Language.

We maintain the term Wolfram Language, considering that the reader will also be introduced to Wolfram Alpha (web and notebook version), as a source of curated, computable data and fast exploratory data analysis. The reader will also be introduced to the Wolfram Data Repository with the examples used in this part and the rest of the text.

Chapter 1. Exploratory and Descriptive Statistics in the Wolfram Language

◼

Importing and Exporting Data

◼

Symbolic Univariate Graphing and Data Summaries

◼

Symbolic Multivariate Graphing and Data Summaries

◼

Finding Clusters in Data

◼

Chapter 2. Statistical Distributions in the Wolfram Language

◼

Symbolic graphing and data summaries

◼

Probabilities and Expectations

◼

Symbolic Simulation of random variates

◼

Parameter Estimation

◼

Derived and Formula Distributions

◼

Mixture Distributions

◼

Parametric Mixtures

◼

Multivariate Derived Distributions

◼

Copulas

◼

Nonparametric Distributions

◼

Survival Distributions

◼

Chapter 3. Hypothesis Testing in the Wolfram Language

◼

Introduction to Symbolic Hypothesis Testing

◼

Tests of Location

◼

Distributional Goodness-of-Fit

◼

Dependency Tests

◼

Variance Tests

◼

Testing with Estimated Parameters

◼

Time Series Tests

◼

PART 2: INTRODUCTION TO STATISTICAL MODELS IN THE WOLFRAM LANGUAGE

This part presents the core parts of statistical and machine learning, model building and assessment as introduced in this book. Starting with the classical linear regression model, all the way to resampling and cross-validation methods, this chapter introduces the reader to the art of statistical model fitting and assessment in the Wolfram Language. This part also illustrates how the Generalized linear models (Gaussian Families, Binomial Families, Poisson Families, Inverse Gaussian Families and Gamma Families) are fitted in the Wolfram Language. Non-linear models are also presented, with examples from science and economics where non-linearities are ubiquitous.

Resampling and cross validation techniques serve as a bridge from statistical models to reality. Considering potential limitations of drawing more samples from a population, computational intensive algorithms such as the bootstrap and cross-validation have proved practical tool in modern statistical analysis. This part introduces the reader to resampling techniques, as implemented in the Wolfram Language. These will be applied to the various standard linear models, generalized linear models and non-linear models presented. In this part, the reader will also be introduced to the Wolfram Function Repository, where “consumption ready” Wolfram Language functions for statistical and machine learning models are contributed by a host of Wolfram users.

Chapter 4. Linear Regression Models in the Wolfram Language

◼

A Mathematical Introduction to Statistical Modelling

◼

Fitting Linear Regression Models

◼

Assessment of Linear Regression Models

◼

Bayesian Linear Regression Models

◼

General Discussion on the Limitations of Linear Regression Models

◼

Chapter 5. Generalized Linear Regression models in the Wolfram Language

◼

The Mathematical Statistics of Generalized Linear models

◼

Binomial Families

◼

Gaussian Families

◼

Poisson Families

◼

Inverse Gaussian Families

◼

Gamma Families

◼

Chapter 6. Non-Linear Regression models in the Wolfram Language

◼

The Mathematics and Statistics of Non-Linear Models

◼

Introduction to the Non-Linear Model Fitting Super-Function

◼

Wavelet Smoothing

◼

Cubic Splines

◼

Multi-Nonlinear Models

◼

Practical Examples from Classical Science, Economics, Engineering and Finance

◼

Chapter 7. Resampling Techniques and Cross-Validation in the Wolfram Language

◼

The Mathematics and Statistics of Resampling and Cross-Validation

◼

Validation

◼

Leave-One-Out Cross-Validation

◼

k-fold Cross-Validation

◼

Introduction to the Bootstrap

◼

Model Selection Case Studies: Examples Using n Degree Polynomials on Real Data

◼

PART 3: INTRODUCTION TO MACHINE LEARNING IN THE WOLFRAM LANGAUGE

The symbolic architecture of the Wolfram Language, coupled with its robust optimization packages developed for over three decades form a solid substratum for the training of the latest machine learning algorithms. This part introduces the reader to the principles of machine learning in the Wolfram Language, starting with the high level machine learning learning super-functions for prediction (quantitative response) and classification (categorical outcomes). The reader explores a wide array of innovative machine learning techniques for non-linear regression and classification using the Wolfram Language.

In this part, the reader is introduced to the art of training Neural Networks “from first principles”. This chapter also showcases the various layers on neural networks, with their function equivalents in the core Wolfram Language. Finally, the reader is also introduced to the Wolfram Neural Network Repository where pre-trained, computable neural networks are available for modification and use in the Wolfram Language.

Chapter 8. Introduction to Machine Learning in the Wolfram Language

◼

Supervised vs. Unsupervised Learning, a High Level Introduction

◼

Two Super-Functions for Machine Learning: Classify and Predict

◼

Symbolic Machine Learning Model Assessment for Wolfram Language Super-Functions: Classifier Measurements and Predictor Measurements

◼

Cross-Validation and Super-Functions: Classify and Predict

◼

Chapter 9. Neural Networks in the Wolfram Language

◼

What are Artificial Neural Networks? An In-Depth Tour from Biology to Computational Mathematics, and Finally to Computational Statistics.

◼

Building Blocks of Artificial Neural Networks in the Wolfram Language:

◼

Layers

◼

Chains

◼

Graphs

◼

Initialization

◼

Training of Neural Networks in the Wolfram Language (CPUs and GPUs)

◼

Symbolic Neural Network Model Assessment

◼

Cross-validation and Neural Networks

◼

Using the Neural Network Repository

◼

PART 4: RANDOM PROCESSES IN TIME

A random process models the progression of a system over time, where the evolution is random rather than deterministic. The key point is that observations that are close in time are dependent, and this can be used to model, simulate, and predict the behavior of the process. Random processes are used in a variety of fields including economics, finance, engineering, physics, and biology.

Building on its strong capabilities for distributions, the Wolfram Language provides cohesive and comprehensive random process support. Using a symbolic representation of a process makes it easy to simulate its behavior, estimate parameters from data, and compute state probabilities at different times. There is additional functionality for special classes of random processes such as Markov chains, queues, time series, and stochastic differential equations.

Chapter 10. Time Series Models in the Wolfram Language

◼

Time series refers to a sequence of observations following each other in time, where adjacent observations are correlated. This can be used to model, simulate, and forecast behavior for a system. Time series models are frequently used in fields such as economics, finance, biology, and engineering.

The Wolfram Language provides a full suite of time series functionality, including standard models such as MA, AR, and ARMA, as well as several extensions. Time series models can be simulated, estimated from data, and used to produce forecasts of future behavior.

The following models will be implemented:

Moving Average (MA) Processes

◼

Autoregressive (AR) Processes

◼

Autoregressive Moving-Average (ARMA) Processes

◼

Seasonal Autoregressive Moving-Average (SARMA) Processes

◼

Autoregressive Integrated Moving-Average (ARIMA) Processes

◼

Seasonal Autoregressive Integrated Moving-Average (SARIMA) Processes

◼

Fractional Autoregressive Integrated Moving-Average (FARIMA) Processes

◼

Autoregressive Conditionally Heteroscedastic (ARCH) Processes

◼

Generalized Autoregressive Conditionally Heteroscedastic (GARCH) Processes

◼

Chapter 11. Parametric Stochastic Processes in the Wolfram Language

◼

Parametric random processes are processes specified using a few parameters. They provide standard models for a variety of areas, including finance (interest rates, …), insurance (claims process, …), and physics (radioactive decay, …).

The Wolfram Language provides complete support for working with parametric random processes. The symbolic representation of a process makes it easy to simulate its behavior, estimate parameters from data, and compute state probabilities at different times. There is a full suite of standard properties, including state distributions at different times as well as mean and covariance functions.

Random Walk Process

◼

Bernoulli Process

◼

White Noise Process

◼

Binomial Process

◼

Renewal Process

◼

Compound Renewal Process

◼

Chapter 12. Stochastic Differential Equation Processes in the Wolfram Language

◼

Stochastic differential equations (SDEs) occur where a system described by differential equations is influenced by random noise. Stochastic differential equations are used in finance (interest rate, stock prices, …), biology (population, epidemics, …), physics (particles in fluids, thermal noise, …), and control and signal processing (controller, filtering, …).

The Wolfram Language provides common special SDEs specified by a few parameters as well as general Ito and Stratonovich SDEs and systems specified by their differential equations. The symbolic representation of SDE processes allows a uniform way to compute a variety of properties, from simulation and mean and covariance functions to full state distributions at different times.

The following SDE processes will be implemented:

Wiener Process

◼

Ornstein Uhlenbeck Process

◼

Brownian Bridge Process

◼

Geometric Brownian Motion Process

◼

Cox Ingersoll Ross Process

◼

Ito Process

◼

Stratonovich Process

◼

EXAMPLES:

We now turn to some examples from the text. Three examples will be presented - two from Chapter 1 (Exploratory and Descriptive Statistics in the Wolfram Language) and another one from Chapter 8 (Introduction to Machine Learning in the Wolfram Language).

Example 1: Bivariate Exploratory Data Analysis (from Chapter 1)

Consider the Wolfram Language built-in Old Faithful geyser data includes durations and waiting times for geyser eruptions.

To obtain the data, we use the function ExampleData and specify the class of the data to be Statistics and and name of the data to be OldFaithful.

We use the function Short to suppress some of the data and allow some of the observations to be seen.

Short[ofdata=ExampleData[{"Statistics","OldFaithful"}]]

In[]:=

{{3.6,79},{1.8,54},{3.333,74},266,{4.417,90},{1.817,46},{4.467,74}}

Out[]//Short=

Let us visualise the data:

ListPlot[ofdata]

In[]:=

Out[]=

What are the properties of this data?

ExampleData[{"Statistics","OldFaithful"},"Properties"]

In[]:=

{ApplicationAreas,ColumnDescriptions,ColumnHeadings,ColumnTypes,DataElements,DataType,Description,Dimensions,EventData,EventSeries,LongDescription,Name,ObservationCount,Source,TimeSeries}

Out[]=

Let us use the property "LongDescription" to get a detailed description of the data:

ExampleData[{"Statistics","OldFaithful"},"LongDescription"]

In[]:=

Data on eruptions of Old Faithful geyser, October 1980. Variables are the duration in seconds of the current eruption, and the time in minutes to the next eruption. Collected by volunteers, and supplied by the Yellowstone National Park Geologist. Data was not collected between approximately midnight and 6 AM.

Out[]=

Clearly the data is bivariate. We can use the property "ColumnDescriptions" to know the description of the properties:

ExampleData[{"Statistics","OldFaithful"},"ColumnDescriptions"]

In[]:=

{Eruption time in minutes,Waiting time to next eruption in minutes}

Out[]=

There are numerous ways to visualize bivariate data, including pointwise plots, density projections, and histograms based on binning or kernel density estimation. The function DensityHistogram plots the density histogram for bivariate data, while the function Histogram3D generates a 3D histogram for bivariate data and the function SmoothHistogram3D plots the 3D kernel histogram for bivariate data.

Here is a plot of the bivariate density using a density histogram:

DensityHistogram[ofdata,20,"PDF"]

In[]:=

Out[]=

We can also view the smooth density histogram, using the function SmoothDensityHistogram.

SmoothDensityHistogram[ofdata]

In[]:=

Out[]=

One can see that the data is clustered into two. We will get back to this point.

Let us visualize the densities in a 3D histogram:

Histogram3D[ofdata,20,"PDF",AxesLabel{"Duration","WaitingTime","Probability"}]

In[]:=

Out[]=

Let us draw a smooth histogram:

SmoothHistogram3D[ofdata,AxesLabel{"Duration","WaitingTime","Probability"}]

In[]:=

Out[]=

We now plot the four figures and table them:

GraphicsGrid[{{Histogram3D[ofdata,20,"PDF"],SmoothHistogram3D[ofdata]},{DensityHistogram[ofdata,20,"PDF"],SmoothDensityHistogram[ofdata]}},Frame->All,ImageSizeLarge]

In[]:=

Out[]=

Let us calculate the mean, variance, standard deviation, skew and kurtosis of waiting durations and waiting time.

Table[f[N[ofdata]],{f,{Mean,Median,Variance,StandardDeviation,Skewness,Kurtosis}}]

In[]:=

{{3.48778,70.8971},{4.,76.},{1.30273,184.823},{1.14137,13.595},{-0.415841,-0.416319},{1.4994,1.85737}}

Out[]=

We can tabulate the

,…,

percentiles of the Old and Faithful data, together with the smallest and largest using the function Table:

quantiles=Table[Quantile[ofdata,p],{p,0,1,0.1}]

In[]:=

{{1.6,43},{1.85,51},{2.,55},{2.3,60},{3.6,71},{4.,76},{4.167,78},{4.367,81},{4.533,83},{4.7,86},{5.1,96}}

Out[]=

Let us plot the percentiles, together with the smallest and largest values:

ListPlot[quantiles,AxesLabel{"Duration","WaitingTime"}]

In[]:=

Out[]=

Example 2: Finding Clusters in Bivariate Data (from Chapter 1)

Another look at the Old and Faithful data:

ListPlot[ofdata,AxesLabel{"Duration","WaitingTime"}]

In[]:=

Out[]=

One can see that the data seem to fall into two clusters. One cluster seem to be

Let us visualise the first variable - Eruption time in minutes.

histDuration=Histogram[ofdata[[All,1]],30,"PDF",AxesLabel{"Duration","Probability"}]

In[]:=

Out[]=

We an also visualize the second variable - Waiting time to next eruption in minutes.

histWaitingTime=Histogram[ofdata[[All,2]],30,"PDF",AxesLabel{"WaitingTime","Probability"}]

In[]:=

Out[]=

We can use the function FindClusters to divide the data into clusters.

clusters=FindClusters[ofdata];

In[]:=

Let us visualize the clusters. We use the graphics option PlotStyle to indicate the clusters using different classes.

ListPlot[clusters,PlotStyle{Blue,Red},AxesLabel{"Duration","WaitingTime"}]

In[]:=

Out[]=

Let us plot a histogram of the Eruption time in minutes for the first cluster, with 15 bins.

histDuration1=Histogram[clusters[[1,All,1]],15,"PDF",AxesLabel{"Duration","Probability"}]

In[]:=

Out[]=

Let us plot a histogram of the Eruption time in minutes for the second cluster, with 15 bins.

histDuration2=Histogram[clusters[[2,All,1]],15,"PDF",AxesLabel{"Duration","Probability"}]

In[]:=

Out[]=

We can also plot a histogram of the Waiting time to next eruption in minutes for the first cluster, with 15 bins.

histWaitingTime1=Histogram[clusters[[1,All,2]],15,"PDF",AxesLabel{"Waiting Time","Probability"}]

In[]:=

Out[]=

Then the plot a histogram of the Waiting time to next eruption in minutes for the second cluster, with 15 bins.

histWaitingTime2=Histogram[clusters[[2,All,2]],15,"PDF",AxesLabel{"Waiting Time","Probability"}]

In[]:=

Out[]=

GraphicsGrid[{{histDuration1,histDuration2},{histWaitingTime1,histWaitingTime2}},Frame->All]

In[]:=

Out[]=

We can see the two clusters using 3D histograms:

Histogram3D[clusters,20,"PDF",ChartStyle{Lighter[Blue],Lighter[Red]}]

In[]:=

Out[]=

Let us draw a box and whisker chart of the waiting time to next eruption, for the two clusters.

BoxWhiskerChart[{clusters[[1,All,2]],clusters[[2,All,2]]}]

In[]:=

Out[]=

Clearly, the two clusters are apparent.

Let us plot the histograms of the Waiting time, together - using PairedHistogram:

PairedHistogram[clusters[[1,All,2]],clusters[[2,All,2]]]

In[]:=

Out[]=

PairedSmoothHistogram[clusters[[1,All,2]],clusters[[2,All,2]]]

In[]:=

Out[]=

Cumulative distribution function of the Waiting time:

PairedHistogram[clusters[[1,All,2]],clusters[[2,All,2]],Automatic,"CDF"]

In[]:=

Out[]=

Example 3: Prediction (from Chapter 8)

When the response variable is continuous in nature and is measured on a ratio scale, the problem of assigning values to the dependent variable, based on the observed values of the independent variable can be viewed as a prediction problem. Estimating one's income, based on their level of study, geographical region and other independent variable can be seen as prediction problem. Likewise, calculating the best estimate for tomorrow's rainfall, based on the today's temperature, humidity and so on can also be viewed as a regression problem.

Various statistical and/or machine learning techniques can be used, for classification problems. These include:

Liner Regression

◼

Neural Networks

◼

Gradient Boosted Trees

◼

Random Forest

◼

Decision Tree

◼

Gaussian Process

◼

Prediction problems consist of numerical responses.

Consider the Boston Homes data from the Wolfram Data Repository. This data consists of home values for 506 Boston suburbs with potential influential factors (such as the crime rate, number of rooms, distance to employment centers, etc.).

boston=ResourceData["Sample Data: Boston Homes"];

In[]:=

How big is this data?

Length[boston]

In[]:=

506

Out[]=

Let us visualise a random sample of 10 entries of this data:

SeedRandom[1234];RandomSample[boston,10]

In[]:=

Out[]=

We can also look at the data as a association:

boston[[1;;5]]//Normal

In[]:=

CRIM0.00632,ZN18,INDUS2.31,CHAStract does not bound Charles river,NOX

0.538

ppm

,RM6.575,AGE65.2,DIS4.09,RAD1,TAX296,PTRATIO15.3,BLACK396.9,LSTAT

4.98

,MEDV24,CRIM0.02731,ZN0,INDUS7.07,CHAStract does not bound Charles river,NOX

0.469

ppm

,RM6.421,AGE78.9,DIS4.9671,RAD2,TAX242,PTRATIO17.8,BLACK396.9,LSTAT

9.14

,MEDV21.6,CRIM0.02729,ZN0,INDUS7.07,CHAStract does not bound Charles river,NOX

0.469

ppm

,RM7.185,AGE61.1,DIS4.9671,RAD2,TAX242,PTRATIO17.8,BLACK392.83,LSTAT

4.03

,MEDV34.7,CRIM0.03237,ZN0,INDUS2.18,CHAStract does not bound Charles river,NOX

0.458

ppm

,RM6.998,AGE45.8,DIS6.0622,RAD3,TAX222,PTRATIO18.7,BLACK394.63,LSTAT

2.94

,MEDV33.4,CRIM0.06905,ZN0,INDUS2.18,CHAStract does not bound Charles river,NOX

0.458

ppm

,RM7.147,AGE54.2,DIS6.0622,RAD3,TAX222,PTRATIO18.7,BLACK396.9,LSTAT

5.33

,MEDV36.2

Out[]=

The test and training sets were obtained by creating a random sample of 30% of the data for the test set, using the rest for training.

Let us assign a variable name to the training set:

bostonTrain=ResourceData["Sample Data: Boston Homes","TrainingData"];

In[]:=

How big is this training data?

Length[bostonTrain]

In[]:=

338

Out[]=

Let us also assign a variable name to the test data:

bostonTest=ResourceData["Sample Data: Boston Homes","TestData"];

In[]:=

What is the length of the test data?

Length[bostonTest]

In[]:=

168

Out[]=

The idea is to train a predictor to predict the median value of owner-occupied homes in $1000s, using the predictor variables given.

The function trains a predictor, for numerical outcomes:

bostonPredict=Predict[bostonTrain]

In[]:=

PredictorFunction

Input type:

Mixed (number: 13)

Method: LinearRegression



Out[]=

We have produced a predictor object. The next thing is to measure how good this predictor is for the data it has not seen.

The function PredictorMeasurements does exactly this.

bostonPredictorMeasurements=PredictorMeasurements[bostonPredict,bostonTest]

In[]:=

PredictorMeasurementsObject

Predictor: LinearRegression

Number of test examples: 168



Out[]=

We have produced a predictor measurement object.

What are the properties associated with this object?

propertiesBoston=bostonPredictorMeasurements["Properties"]

In[]:=

{BatchEvaluationTime,BestPredictedExamples,ComparisonPlot,EvaluationTime,Examples,FractionVarianceUnexplained,GeometricMeanProbabilityDensity,LeastCertainExamples,Likelihood,LogLikelihood,MeanCrossEntropy,MeanDeviation,MeanSquare,MostCertainExamples,Perplexity,PredictorFunction,ProbabilityDensities,ProbabilityDensityHistogram,Properties,RejectionRate,Report,ResidualHistogram,ResidualPlot,Residuals,RSquared,StandardDeviation,StandardDeviationBaseline,TotalSquare,WorstPredictedExamples}

Out[]=

Let us get the comparison plot:

bostonPredictorMeasurements["ComparisonPlot"]

In[]:=

Out[]=

We could also tabulate all the properties, as a Dataset:

Dataset[AssociationMap[bostonPredictorMeasurements[#]&,bostonPredictorMeasurements["Properties"]]]

In[]:=

Out[]=

We can specify the method, as the second argument of the function Predict.

For example, we can specify the method to be “NeuralNetwork”.

Predict[bostonTrain,Method->"NeuralNetwork"]

In[]:=

PredictorFunction

Input type:

Mixed (number: 13)

Method: NeuralNetwork



Out[]=

We can obtain predictor measurements associated with a property such as "RSquared" when a predictor is trained using the train set and evaluated on the test data in calculation:

PredictorMeasurements[Predict[bostonTrain,Method->"NeuralNetwork"],bostonTest,"RSquared"]

In[]:=

0.810974

Out[]=

Let us now collect all the available prediction methods in a list and name the list:

predictMethods={"DecisionTree","GradientBoostedTrees","LinearRegression","NearestNeighbors","NeuralNetwork","RandomForest","GaussianProcess"};

In[]:=

Using AssociationMap, we can create an association containing list a certain property for all the available methods:

Dataset[AssociationMap[PredictorMeasurements[Predict[bostonTrain,Method#],bostonTest,"RSquared"]&,predictMethods]]

In[]:=

Out[]=

We can also look at the comparison plots for all the methods:

comparisonPlot=Dataset[AssociationMap[PredictorMeasurements[Predict[bostonTrain,Method#],bostonTest,"ComparisonPlot"]&,predictMethods]];

Normal@comparisonPlot

In[]:=

Out[]=

We can also view the comparison plots individually:

Comparison Plot: Decision Tree

◼

comparisonPlot["DecisionTree"]

In[]:=

Out[]=

Comparison Plot: Gradient Boosted Trees

◼

comparisonPlot["GradientBoostedTrees"]

In[]:=

Out[]=

comparisonPlot["LinearRegression"]

In[]:=

Out[]=

Comparison Plot: Linear Regression

◼

comparisonPlot["LinearRegression"]

In[]:=

Out[]=

Comparison Plot: Nearest Neighbors

◼

comparisonPlot["NearestNeighbors"]

In[]:=

Out[]=

Comparison Plot: Neural Network

◼

comparisonPlot["NeuralNetwork"]

In[]:=

Out[]=

Comparison Plot : Random Forest

◼

comparisonPlot["RandomForest"]

In[]:=

Out[]=