On the topic "Loss function with regularization" on the Mathematica stack exchange website there is a reply that shows how to make a layer that implements L2 regularization with NetArray.

Thanks. I am aware of the upgrade to Fit to permit regularization. It works just fine. Two wishes:

(a) regularization would also be added to the more modern LinearModelFit, LogitModelFit, etc so I could take advantage of their greater capabilities (b) someone could give me some hints on how to emulate LASSO or Ridge using the Neural Network architecture. As mentioned earlier, I have some belief that it can be done in 12.2 using NetArray inside a linear layer (http://reference.wolfram.com/language/ref/NetArray.html) and/or NetArrayLayer but, as the documentation for that Experimental function is not fully developed, I have not yet figured out how to do so.

For ridge regression I would suggest using the lmridge package in R (which you already know you can do from Mathematica). There are so many options for choosing the scaling parameter that a home-grown program in Mathematica would likely be fragile at best especially if you need credibility in the results.