@Hakan Kjellerstrand, thank you for the excellent response!

Cross validation is another thing I am learning. How can one store the 10-fold sets that were used to see if the accuracy is improved using this ensemble technique?

Is there also a way to automatically do this using different classification techniques / algorithms to see if the results can be improved?

Again, thank you for aiding in my learning process!

To answer your first question, using Import is much easier than ReadList. Import is quite intelligent in parsing the data.

For the rest, I would do something like this.

(* Import data and create the dataset *)
dataImport =  Import["https://archive.ics.uci.edu/ml/machine-learning-databases/pima-indians-diabetes/pima-indians-diabetes.data"];

(* create the dataset *)
colNames = {"pregnant", "glucose", "bp", "skin", "insulin", "bmi", "pedigree", "age", "label"};
dataAll = Thread[colNames -> #] & /@ dataImport // Map[Association] // Dataset;

dataSelected = dataAll[All, {"pregnant", "insulin", "bmi", "age", "label"}] // Normal;
data = Thread[Most[#] -> Last[#]] & /@ dataSelected

(* Check the distribution of 0s and 1s, to get a base line:
If one guess 0 for all examples it's 65% correct. *)
Histogram@data[[All, 2]]
Counts@data[[All, 2]]
% / Total@% // N

(* split into  train and test data *)
{dataTrain, dataTest} = TakeDrop[RandomSample[data], Round[Length[data]*0.9]]; 

And then train/test the dataset:

 (* Here we let Classifier run for 60s *)
AbsoluteTiming[cl = Classify[dataTrain, PerformanceGoal -> "Quality", TimeGoal -> 60]]

(* Check the unseen data in the test set. *) 
cm = ClassifierMeasurements[cl, dataTest, PerformanceGoal -> "Quality"]
cm["ConfusionMatrixPlot"]
cm["Accuracy"]
cm["AreaUnderROCCurve"]

Note: There is a lot of statistics in the ClassifierMeasurements, probably all the one you need (and then many more). See the documentation for the complete list.

One thing I miss in Mathematica is the N-fold cross validation (i.e. testing N different folds to get an average accuracy of these fold). So here's a simple (and slow) version that use random folding:

ClearAll[crossValidation, crossValidation1]
crossValidation1[data_, folds_: 10, time_: Automatic, performance_: Automatic] := Module[{len = Length[data], train, test, cl, cm},
  {train, test} = TakeDrop[RandomSample[data], len - Round[len/folds]];
  cl = Classify[train, TrainingProgressReporting -> None, TimeGoal -> time, PerformanceGoal -> performance];
  cm = ClassifierMeasurements[cl, test];
  cm["Accuracy"]
  ]
crossValidation[data_, folds_: 10, time_: Automatic, performance_: Automatic] := Module[{accuracy, cv},
  accuracy = Monitor[Table[cv = crossValidation1[data, folds, time, performance], {i, 1, folds}], {i, cv}];
  Mean[accuracy]
  ]

An example how to use this:

(* Cross validation, 10 random folds, 20 s time limit, and go for quality *)
crossValidation[data, 10, 20, "Quality"]