Dear Ghobani,

I checked your Mathematica codes and found that your KAN model does not reduce the loss. It seems that only one KANlayer[2,1,15,3] is not complex enough to learn.

kan = NetTrain[NetChain[{KANlayer[2,1,15,3]},"Input"->2],training]

I added one more KANlayer, then the loss is reduced.

dataSDL = ArrayReshape[N@Range[300], {100, 3}];

Two KANlayers:

kanSDLEE = 
 NetTrain[NetChain[{KANlayer[2, 5, 15, 3], KANlayer[5, 1, 15, 3]}], 
  dataSDL[[All, 1 ;; 2]] -> Transpose@{dataSDL[[All, 3]]}].

Three KAN layers:

kanSDLEE03 = 
 NetTrain[
  NetChain[{KANlayer[2, 5, 15, 3], KANlayer[5, 5, 15, 3], 
    KANlayer[5, 1, 15, 3]}], 
  dataSDL[[All, 1 ;; 2]] -> Transpose@{dataSDL[[All, 3]]}].

Hi Dr. Ghorbani,

Note that I quickly created the following data to run MLP and KAN and it is ill-conditioned data, thus there will be infinite number of solutions (the columns are perfect linear combination of other columns), thus any least square estimation or its variations will be unstable. Please use different training dataset that you are familiar with.

dataSDL = ArrayReshape[N@Range[300], {100, 3}];

I remember reading a paper that claims KAN is better than MLP but I also remember a paper claiming that KAN is not. I guess it deepens on the goal, precise prediction with complex model or understandable structure with transparent model.

I think you are in the process of fine-tuning the net structure and it will be a time-consuming process.

It is a good practice in analyses that the input data should be standardized or normalized. I experienced that batch normalization layer improves the fitting well.

kanSDLEE04 = NetTrain[
  NetChain[
   {
    BatchNormalizationLayer[],
    KANlayer[2, 50, 10, 3],
    KANlayer[50, 1, 10, 3]
    }
   ],
  dataSDL[[All, 1 ;; 2]] -> Transpose@{dataSDL[[All, 3]]}, All, 
  MaxTrainingRounds -> 1000
  ]

Dear Andreas: I use modification of your code for my paper: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4419282 (unpublished). What (pseudo-) metrics do you use to fit the polynomials -- MSE, Kullback-Leibler, something different? If you can answer, write me at pblerner18@gmail.com because I rarely use this place.