Machine learning in general extends traditional areas of probability and statistics into the subject of 'data science'. Probabilits would call this 'non-parametric' field of distribution representation. I agree, it may be bit confusing for traditional statisticians alike, but you may think about this as an 'extension' of statistical sciences into adjacent fields of science where we are modelling future given the observation in the past. The entire time series analysis is built on this premise.

The entire theory of data science / machine learning resides on the concept of 'learning from data'. The idea as such is not new, but what is new is its transformation into a scientific subject. Progress on the hardware and software side made this transition feasible. Wolfram Research was one of the first who recognised this aspect and entered into the data science world with the set of tools and routines to make data science as smooth as possible.

What I like in particular on Mathematica is its approach to make the entire ML practice as simple as possible. All functions and algorithms are finely tuned and automated. You do not have to be data science expert to start practicing ML in Mathematica. My article was all about it - once you have your data, you start analysing it and building prediction models quickly and efficiently. Mathematica will select for you the most optimal models to give you decent predictions. Further fine-tuning is possible if you want to drive your predictions 'manually' or if you want to build predictions using particular model,

When you look at the documentation on ML, you will see the information of models, methods, and options available. Yes, you're right - the models are essentially expectations on the outcome given their calibration to the past data. Models examine the data, detect relationships, patterns and features, and then set the parameters for future predictions. Regression in statistics is one of these 'early' examples of how ML work. However, ML is much richer and capable to extract much more dependencies than traditional regression techniques in statistics.

I do encourage to start exploring data science further, it is an exciting area of science, very dynamic and still evolving. People have started re-discovering the power of information contained in the data, so the ML offers bring business prospects. With your background in statistics and number theory, the transition towards the full data scientists profile should be smooth and quick. If you would like to get further guidance in this field, I will be glad to assist further.

Best Igor